Method: Unable to determine method from the provided content

Result: Unable to determine results from the provided content

Conclusion: Unable to determine conclusion from the provided content

Abstract: Failed to fetch summary for 2511.05550: Page request resulted in HTTP 429 (https://export.arxiv.org/api/query?search_query=&id_list=2511.05550&sortBy=relevance&sortOrder=descending&start=0&max_results=100)

Method: Developed MingTok-Audio unified continuous speech tokenizer that integrates semantic and acoustic features, then built Ming-UniAudio speech language model and Ming-UniAudio-Edit for free-form speech editing guided by natural language instructions.

Result: Set new SOTA on 8/12 metrics on ContextASR benchmark, achieved Seed-TTS-WER of 0.95 for Chinese voice cloning, and created first comprehensive benchmark for instruction-based free-form speech editing.

Conclusion: The unified framework successfully bridges the gap between speech understanding and generation, enabling universal free-form speech editing without timestamp conditions, with all components open-sourced to advance unified audio processing.

Abstract: Existing speech models suffer from competing requirements on token representations by understanding and generation tasks. This discrepancy in representation prevents speech language models from performing instruction-based free-form editing. To solve this challenge, we introduce a novel framework that unifies speech understanding, generation, and editing. The core of our unified model is a unified continuous speech tokenizer MingTok-Audio, the first continuous tokenizer to effectively integrate semantic and acoustic features, which makes it suitable for both understanding and generation tasks. Based on this unified continuous audio tokenizer, we developed the speech language model Ming-UniAudio, which achieved a balance between generation and understanding capabilities. Ming-UniAudio sets new state-of-the-art (SOTA) records on 8 out of 12 metrics on the ContextASR benchmark. Notably, for Chinese voice cloning, it achieves a highly competitive Seed-TTS-WER of 0.95. Leveraging this foundational model, we further trained a dedicated speech editing model Ming-UniAudio-Edit, the first speech language model that enables universal, free-form speech editing guided solely by natural language instructions, handling both semantic and acoustic modifications without timestamp condition. To rigorously assess the editing capability and establish a foundation for future research, we introduce Ming-Freeform-Audio-Edit, the first comprehensive benchmark tailored for instruction-based free-form speech editing, featuring diverse scenarios and evaluation dimensions spanning semantic correctness, acoustic quality, and instruction alignment. We open-sourced the continuous audio tokenizer, the unified foundational model, and the free-form instruction-based editing model to facilitate the development of unified audio understanding, generation, and manipulation.

Method: Methodology information not accessible due to server error

Result: No results available - arXiv API returned service unavailable error

Conclusion: Unable to analyze paper - technical issue prevented content retrieval

Abstract: Failed to fetch summary for 2511.05717: Page request resulted in HTTP 503 (https://export.arxiv.org/api/query?search_query=&id_list=2511.05717&sortBy=relevance&sortOrder=descending&start=0&max_results=100)

Method: CIA optimizes input snippets to induce sustained high-entropy states that precede memorized text emission. For aligned LLMs, Mismatched Supervised Fine-tuning weakens alignment and induces targeted confusion to increase attack susceptibility.

Result: Experiments show CIA outperforms existing baselines in extracting verbatim and near-verbatim training data from various unaligned and aligned LLMs without requiring prior knowledge of training data.

Conclusion: The findings reveal persistent memorization risks across LLMs and provide a systematic method for assessing these vulnerabilities, highlighting ongoing privacy and copyright concerns despite alignment efforts.

Abstract: The memorization of training data in large language models (LLMs) poses significant privacy and copyright concerns. Existing data extraction methods, particularly heuristic-based divergence attacks, often exhibit limited success and offer limited insight into the fundamental drivers of memorization leakage. This paper introduces Confusion-Inducing Attacks (CIA), a principled framework for extracting memorized data by systematically maximizing model uncertainty. We empirically demonstrate that the emission of memorized text during divergence is preceded by a sustained spike in token-level prediction entropy. CIA leverages this insight by optimizing input snippets to deliberately induce this consecutive high-entropy state. For aligned LLMs, we further propose Mismatched Supervised Fine-tuning (SFT) to simultaneously weaken their alignment and induce targeted confusion, thereby increasing susceptibility to our attacks. Experiments on various unaligned and aligned LLMs demonstrate that our proposed attacks outperform existing baselines in extracting verbatim and near-verbatim training data without requiring prior knowledge of the training data. Our findings highlight persistent memorization risks across various LLMs and offer a more systematic method for assessing these vulnerabilities.

Method: Leverages in-context learning with foundation models to unify detection across binary, multi-class, and multi-label settings, allowing personalization through simple prompt-based interventions.

Result: Foundation models achieve strong cross-task generalization matching task-specific models, effective personalization with just one example, and enhanced robustness with label definitions/rationales.

Conclusion: Demonstrates a shift beyond one-size-fits-all moderation, establishing ICL as a practical, privacy-preserving, and adaptable pathway for user-centric content safety systems.

Abstract: The proliferation of harmful online content–e.g., toxicity, spam, and negative sentiment–demands robust and adaptable moderation systems. However, prevailing moderation systems are centralized and task-specific, offering limited transparency and neglecting diverse user preferences–an approach ill-suited for privacy-sensitive or decentralized environments. We propose a novel framework that leverages in-context learning (ICL) with foundation models to unify the detection of toxicity, spam, and negative sentiment across binary, multi-class, and multi-label settings. Crucially, our approach enables lightweight personalization, allowing users to easily block new categories, unblock existing ones, or extend detection to semantic variations through simple prompt-based interventions–all without model retraining. Extensive experiments on public benchmarks (TextDetox, UCI SMS, SST2) and a new, annotated Mastodon dataset reveal that: (i) foundation models achieve strong cross-task generalization, often matching or surpassing task-specific fine-tuned models; (ii) effective personalization is achievable with as few as one user-provided example or definition; and (iii) augmenting prompts with label definitions or rationales significantly enhances robustness to noisy, real-world data. Our work demonstrates a definitive shift beyond one-size-fits-all moderation, establishing ICL as a practical, privacy-preserving, and highly adaptable pathway for the next generation of user-centric content safety systems. To foster reproducibility and facilitate future research, we publicly release our code on GitHub and the annotated Mastodon dataset on Hugging Face.

Method: Uses Model Context Protocol (MCP) with BIM tools including scene querying, predefined functions for building elements, and dynamic code-generation combining in-context learning with RAG for complex tasks.

Result: LLMs using the framework successfully perform complex tasks from building simple houses to querying and editing existing IFC data.

Conclusion: The framework provides a foundation for AI-assisted modeling workflows and encourages research in LLM-driven BIM design, released as open-source.

Abstract: Bringing generative AI into the architecture, engineering and construction (AEC) field requires systems that can translate natural language instructions into actions on standardized data models. We present MCP4IFC, a comprehensive open-source framework that enables Large Language Models (LLMs) to directly manipulate Industry Foundation Classes (IFC) data through the Model Context Protocol (MCP). The framework provides a set of BIM tools, including scene querying tools for information retrieval, predefined functions for creating and modifying common building elements, and a dynamic code-generation system that combines in-context learning with retrieval-augmented generation (RAG) to handle tasks beyond the predefined toolset. Experiments demonstrate that an LLM using our framework can successfully perform complex tasks, from building a simple house to querying and editing existing IFC data. Our framework is released as open-source to encourage research in LLM-driven BIM design and provide a foundation for AI-assisted modeling workflows. Our code is available at https://show2instruct.github.io/mcp4ifc/.

Method: FlowMM leverages cross-modal information flow to apply layer-specific merging strategies and introduces a sensitivity-adaptive token matching mechanism that jointly evaluates token similarity and task-critical sensitivity.

Result: Extensive experiments show FlowMM reduces KV cache memory by 80% to 95% and decoding latency by 1.3-1.8x while maintaining competitive task performance across diverse MLLMs.

Conclusion: FlowMM effectively addresses multimodal KV cache optimization by adaptively merging tokens based on cross-modal information flow and sensitivity assessment, achieving significant memory and latency improvements without compromising generation quality.

Abstract: Traditional KV cache eviction strategies, which discard less critical KV-pairs based on attention scores, often degrade generation quality, causing context loss or hallucinations. Recent efforts shift toward KV merging, merging eviction tokens with retention tokens based on similarity. However, in multimodal scenarios, distributional biases across modality tokens and attentional biases in cross-modal interactions limit its effectiveness. This work introduces FlowMM, an adaptive framework for cross-modal information flow-guided multimodal KV cache merging. FlowMM leverages cross-modal information flow to dynamically apply layer-specific merging strategies, capturing modality-specific patterns while preserving contextual integrity. Furthermore, we introduce a sensitivity-adaptive token matching mechanism that jointly evaluates token similarity and task-critical sensitivity, merging low-risk tokens while safeguarding high-sensitivity ones. Extensive experiments across diverse leading MLLMs show that FlowMM reduces KV cache memory by 80% to 95% and decoding latency by 1.3-1.8x, while maintaining competitive task performance.

Method: ReMoD integrates experience-driven intuitive reasoning to capture initial stance cues and deliberate reflective reasoning to adjust for modality biases. It uses Modality Experience Pool (MEP) and Semantic Experience Pool (SEP) with two reasoning chains: Modality-CoT for adaptive fusion and Semantic-CoT for deeper contextual insights.

Result: Extensive experiments on the public MMSD benchmark demonstrate that ReMoD significantly outperforms most baseline models and exhibits strong generalization capabilities.

Conclusion: The proposed dual-reasoning paradigm effectively addresses modality bias in multimodal stance detection by dynamically weighting modality contributions based on their actual expressive power, leading to more robust and context-aware stance decisions.

Abstract: Multimodal Stance Detection (MSD) is a crucial task for understanding public opinion on social media. Existing work simply fuses information from various modalities to learn stance representations, overlooking the varying contributions of stance expression from different modalities. Therefore, stance misunderstanding noises may be drawn into the stance learning process due to the risk of learning errors by rough modality combination. To address this, we get inspiration from the dual-process theory of human cognition and propose ReMoD, a framework that Rethinks Modality contribution of stance expression through a Dual-reasoning paradigm. ReMoD integrates experience-driven intuitive reasoning to capture initial stance cues with deliberate reflective reasoning to adjust for modality biases, refine stance judgments, and thereby dynamically weight modality contributions based on their actual expressive power for the target stance. Specifically, the intuitive stage queries the Modality Experience Pool (MEP) and Semantic Experience Pool (SEP) to form an initial stance hypothesis, prioritizing historically impactful modalities. This hypothesis is then refined in the reflective stage via two reasoning chains: Modality-CoT updates MEP with adaptive fusion strategies to amplify relevant modalities, while Semantic-CoT refines SEP with deeper contextual insights of stance semantics. These dual experience structures are continuously refined during training and recalled at inference to guide robust and context-aware stance decisions. Extensive experiments on the public MMSD benchmark demonstrate that our ReMoD significantly outperforms most baseline models and exhibits strong generalization capabilities.

Method: Quantifies collapse onset by examining year-wise semantic similarity in English-language Wikipedia (filtered Common Crawl) from 2013 to 2025 using Transformer embeddings and cosine similarity metrics.

Result: Shows steady rise in similarity before public LLM adoption (driven by early RNN/LSTM translation), with exponential rise after LLM adoption. Fluctuations reflect irreducible linguistic diversity and sampling error.

Conclusion: Provides data-driven estimate of when recursive AI contamination may significantly threaten data richness and model generalization, highlighting the urgent need to address Model Collapse in AI training pipelines.

Abstract: Artificial Intelligence, especially Large Language Models (LLMs), has transformed domains such as software engineering, journalism, creative writing, academia, and media (Naveed et al. 2025; arXiv:2307.06435). Diffusion models like Stable Diffusion generate high-quality images and videos from text. Evidence shows rapid expansion: 74.2% of newly published webpages now contain AI-generated material (Ryan Law 2025), 30-40% of the active web corpus is synthetic (Spennemann 2025; arXiv:2504.08755), 52% of U.S. adults use LLMs for writing, coding, or research (Staff 2025), and audits find AI involvement in 18% of financial complaints and 24% of press releases (Liang et al. 2025). The underlying neural architectures, including Transformers (Vaswani et al. 2023; arXiv:1706.03762), RNNs, LSTMs, GANs, and diffusion networks, depend on large, diverse, human-authored datasets (Shi & Iyengar 2019). As synthetic content dominates, recursive training risks eroding linguistic and semantic diversity, producing Model Collapse (Shumailov et al. 2024; arXiv:2307.15043; Dohmatob et al. 2024; arXiv:2402.07712). This study quantifies and forecasts collapse onset by examining year-wise semantic similarity in English-language Wikipedia (filtered Common Crawl) from 2013 to 2025 using Transformer embeddings and cosine similarity metrics. Results reveal a steady rise in similarity before public LLM adoption, likely driven by early RNN/LSTM translation and text-normalization pipelines, though modest due to a smaller scale. Observed fluctuations reflect irreducible linguistic diversity, variable corpus size across years, finite sampling error, and an exponential rise in similarity after the public adoption of LLM models. These findings provide a data-driven estimate of when recursive AI contamination may significantly threaten data richness and model generalization.

Method: Evaluated models on 170 clinical cases synthesized into speech from 36 distinct voice profiles spanning age, gender, and emotion variations, comparing audio vs text inputs.

Result: Severe modality bias found with surgical recommendations varying up to 35% between audio and text, age disparities up to 12%, chain-of-thought prompting didn’t eliminate age bias, explicit reasoning eliminated gender bias, emotion impact undetectable due to poor recognition.

Conclusion: Audio LLMs make clinical decisions based on voice characteristics rather than medical evidence, risking healthcare disparities; bias-aware architectures are urgently needed before clinical deployment.

Abstract: As large language models transition from text-based interfaces to audio interactions in clinical settings, they might introduce new vulnerabilities through paralinguistic cues in audio. We evaluated these models on 170 clinical cases, each synthesized into speech from 36 distinct voice profiles spanning variations in age, gender, and emotion. Our findings reveal a severe modality bias: surgical recommendations for audio inputs varied by as much as 35% compared to identical text-based inputs, with one model providing 80% fewer recommendations. Further analysis uncovered age disparities of up to 12% between young and elderly voices, which persisted in most models despite chain-of-thought prompting. While explicit reasoning successfully eliminated gender bias, the impact of emotion was not detected due to poor recognition performance. These results demonstrate that audio LLMs are susceptible to making clinical decisions based on a patient’s voice characteristics rather than medical evidence, a flaw that risks perpetuating healthcare disparities. We conclude that bias-aware architectures are essential and urgently needed before the clinical deployment of these models.

Method: Introduce Temporal Sparse Autoencoders (T-SAEs) with a novel contrastive loss that encourages consistent activations of high-level features over adjacent tokens, leveraging the insight that semantic content has long-range dependencies while syntactic information is local.

Result: T-SAEs recover smoother, more coherent semantic concepts across multiple datasets and models without sacrificing reconstruction quality, exhibiting clear semantic structure despite being trained without explicit semantic signal.

Conclusion: T-SAEs provide a new pathway for unsupervised interpretability in language models by effectively disentangling semantic from syntactic features through temporal consistency constraints.

Abstract: Translating the internal representations and computations of models into concepts that humans can understand is a key goal of interpretability. While recent dictionary learning methods such as Sparse Autoencoders (SAEs) provide a promising route to discover human-interpretable features, they suffer from a variety of problems, including a systematic failure to capture the rich conceptual information that drives linguistic understanding. Instead, they exhibit a bias towards shallow, token-specific, or noisy features, such as “the phrase ‘The’ at the start of sentences”. In this work, we propose that this is due to a fundamental issue with how dictionary learning methods for LLMs are trained. Language itself has a rich, well-studied structure spanning syntax, semantics, and pragmatics; however, current unsupervised methods largely ignore this linguistic knowledge, leading to poor feature discovery that favors superficial patterns over meaningful concepts. We focus on a simple but important aspect of language: semantic content has long-range dependencies and tends to be smooth over a sequence, whereas syntactic information is much more local. Building on this insight, we introduce Temporal Sparse Autoencoders (T-SAEs), which incorporate a novel contrastive loss encouraging consistent activations of high-level features over adjacent tokens. This simple yet powerful modification enables SAEs to disentangle semantic from syntactic features in a self-supervised manner. Across multiple datasets and models, T-SAEs recover smoother, more coherent semantic concepts without sacrificing reconstruction quality. Strikingly, they exhibit clear semantic structure despite being trained without explicit semantic signal, offering a new pathway for unsupervised interpretability in language models.

Method: Replaces self-attention with linear-time mLSTM token mixer; incorporates lightweight enhancements including short convolutions, sliding window attention with dynamic modulation, and Hedgehog feature maps; uses Muon optimizer instead of AdamW; curates high-quality corpus emphasizing readability and pedagogical structure.

Result: Linear attention combined with sliding window attention consistently improves zero-shot performance; Muon optimizer stabilizes convergence and reduces perplexity compared to AdamW; effective performance demonstrated across both STRICT and STRICT-SMALL tracks.

Conclusion: The study demonstrates effective strategies for efficient language modeling without relying on scale, highlighting the value of architectural innovations and optimization techniques in low-resource settings.

Abstract: We study architectural and optimization techniques for sample-efficient language modeling under the constraints of the BabyLM 2025 shared task. Our model, BLaLM, replaces self-attention with a linear-time mLSTM token mixer and explores lightweight enhancements, including short convolutions, sliding window attention with dynamic modulation, and Hedgehog feature maps. To support training in low-resource settings, we curate a high-quality corpus emphasizing readability and pedagogical structure. Experiments across both STRICT and STRICT-SMALL tracks show that (1) linear attention combined with sliding window attention consistently improves zero-shot performance, and (2) the Muon optimizer stabilizes convergence and reduces perplexity over AdamW. These results highlight effective strategies for efficient language modeling without relying on scale.

Method: Formalized tokenization using monoid theory, proved that tokenizers with ill-formed UTF-8 tokens can produce invalid sequences, demonstrated differences between incremental and batch decoding, and evaluated real-world case studies of major models and systems.

Result: Formal proof that tokenizers with invalid UTF-8 tokens always risk producing invalid sequences, empirical evidence of real-world bugs in foundation models and serving engines, and identification of differences between incremental and batch decoding approaches.

Conclusion: Byte-level tokenization introduces fundamental UTF-8 validity issues that cannot be fully mitigated, requiring applications to handle potential breakage and highlighting the trade-offs between code point and byte-level approaches.

Abstract: Subword tokenization segments input text according to a pre-defined vocabulary to feed it into a language model; the language model, in turn, generates a sequence made from this same vocabulary. The members of the vocabulary can be built of code points or bytes. Using code points means that all members of the vocabulary are valid UTF-8 characters. However, it also requires thousands of initial members to achieve acceptable coverage of inputs. Beginning with bytes, on the contrary, avoids out-of-vocabulary errors with only 256 initial members of the vocabulary, but the members of the vocabulary and sequences of them are not guaranteed to be valid UTF-8. Sequences that are not valid UTF-8 break code that assumes its input to be valid UTF-8. Applications of language models must account for the breakage thereby introduced. In this paper, we formalize tokenization using monoid theory and prove that tokenizers whose vocabularies contain tokens that are ill-formed UTF-8 can always produce sequences that are ill-formed UTF-8. We demonstrate formally that attempting to incrementally convert tokens back to a string and interpret the results as UTF-8 gives different results than converting the whole sequence of tokens at once. This formal result predicts real-world bugs: we evaluate mitigations for the problem identified and provide case studies of major foundation models, serving engines, and constrained generation systems.

Method: Token-level model collaboration framework that dynamically routes decoding between base and aligned LLMs based on next-token prediction uncertainty and semantic role of predicted content.

Result: Consistently surpasses state-of-the-art inference-time baselines across 3 open-ended generation tasks and 13 metrics, with 21.3% joint improvement in diversity and quality. Human evaluations confirm improvements.

Conclusion: Collaboration between base and aligned models can effectively optimize and control both diversity and quality in LLM outputs.

Abstract: Alignment has greatly improved large language models (LLMs)’ output quality at the cost of diversity, yielding highly similar outputs across generations. We propose Base-Aligned Model Collaboration (BACo), an inference-time token-level model collaboration framework that dynamically combines a base LLM with its aligned counterpart to optimize diversity and quality. Inspired by prior work (Fei et al., 2025), BACo employs routing strategies that determine, at each token, from which model to decode based on next-token prediction uncertainty and predicted contents’ semantic role. Prior diversity-promoting methods, such as retraining, prompt engineering, and multi-sampling methods, improve diversity but often degrade quality or require costly decoding or post-training. In contrast, BACo achieves both high diversity and quality post hoc within a single pass, while offering strong controllability. We explore a family of routing strategies, across three open-ended generation tasks and 13 metrics covering diversity and quality, BACo consistently surpasses state-of-the-art inference-time baselines. With our best router, BACo achieves a 21.3% joint improvement in diversity and quality. Human evaluations also mirror these improvements. The results suggest that collaboration between base and aligned models can optimize and control diversity and quality.

Method: Introduces OckBench, a model-agnostic and hardware-agnostic benchmark that measures both accuracy and token count across reasoning and coding tasks, comparing multiple open- and closed-source models.

Result: Experiments show that models with comparable accuracy differ wildly in token consumption, revealing efficiency variance as a significant but neglected differentiation factor. Pareto frontiers demonstrate accuracy-efficiency trade-offs.

Conclusion: Tokens should not be treated as “free” resources. OckBench provides a unified platform for measuring and guiding research in token-efficient reasoning, advocating for an evaluation paradigm shift.

Abstract: Large language models such as GPT-4, Claude 3, and the Gemini series have improved automated reasoning and code generation. However, existing benchmarks mainly focus on accuracy and output quality, and they ignore an important factor: decoding token efficiency. In real systems, generating 10,000 tokens versus 100,000 tokens leads to large differences in latency, cost, and energy. In this work, we introduce OckBench, a model-agnostic and hardware-agnostic benchmark that evaluates both accuracy and token count for reasoning and coding tasks. Through experiments comparing multiple open- and closed-source models, we uncover that many models with comparable accuracy differ wildly in token consumption, revealing that efficiency variance is a neglected but significant axis of differentiation. We further demonstrate Pareto frontiers over the accuracy-efficiency plane and argue for an evaluation paradigm shift: we should no longer treat tokens as “free” to multiply. OckBench provides a unified platform for measuring, comparing, and guiding research in token-efficient reasoning. Our benchmarks are available at https://ockbench.github.io/ .

Method: Proposed Hapax - a training setting that omits loss contribution from tokens that can be correctly predicted by induction heads, effectively suppressing inductive copying while training on abstractive ICL tasks.

Result: Despite 31.7% of tokens being omitted from loss, performance on abstractive ICL tasks remained comparable and surpassed vanilla model on 13 of 21 tasks. Models developed fewer and weaker induction heads but preserved ICL capabilities.

Conclusion: Inductive copying is not essential for learning abstractive in-context learning mechanisms, as transformers can develop ICL capabilities even when inductive copying is suppressed.

Abstract: Induction heads are attention heads that perform inductive copying by matching patterns from earlier context and copying their continuations verbatim. As models develop induction heads, they often experience a sharp drop in training loss, a phenomenon cited as evidence that induction heads may serve as a prerequisite for more complex in-context learning (ICL) capabilities. In this work, we ask whether transformers can still acquire ICL capabilities when inductive copying is suppressed. We propose Hapax, a setting where we omit the loss contribution of any token that can be correctly predicted by induction heads. Despite a significant reduction in inductive copying, performance on abstractive ICL tasks (i.e., tasks where the answer is not contained in the input context) remains comparable and surpasses the vanilla model on 13 of 21 tasks, even though 31.7% of tokens are omitted from the loss. Furthermore, our model achieves lower loss values on token positions that cannot be predicted correctly by induction heads. Mechanistic analysis further shows that models trained with Hapax develop fewer and weaker induction heads but still preserve ICL capabilities. Taken together, our findings indicate that inductive copying is not essential for learning abstractive ICL mechanisms.

Method: Two-stage progressive adaptation: first learns acoustic and tonal representations from phonetic Tai-lo annotations, then captures vocabulary and syntax from Han-character transcriptions, building on Mandarin HuBERT models.

Result: Achieves 24.88% relative reduction in character error rate (CER) compared to strong baselines on TAT-MOE corpus, demonstrating effective alignment between speech sounds and orthographic structures.

Conclusion: CLiFT-ASR provides an effective and parameter-efficient solution for Taiwanese Hokkien ASR with potential applicability to other low-resource language scenarios.

Abstract: Automatic speech recognition (ASR) for low-resource languages such as Taiwanese Hokkien is difficult due to the scarcity of annotated data. However, direct fine-tuning on Han-character transcriptions often fails to capture detailed phonetic and tonal cues, while training only on romanization lacks lexical and syntactic coverage. In addition, prior studies have rarely explored staged strategies that integrate both annotation types. To address this gap, we present CLiFT-ASR, a cross-lingual fine-tuning framework that builds on Mandarin HuBERT models and progressively adapts them to Taiwanese Hokkien. The framework employs a two-stage process in which it first learns acoustic and tonal representations from phonetic Tai-lo annotations and then captures vocabulary and syntax from Han-character transcriptions. This progressive adaptation enables effective alignment between speech sounds and orthographic structures. Experiments on the TAT-MOE corpus demonstrate that CLiFT-ASR achieves a 24.88% relative reduction in character error rate (CER) compared with strong baselines. The results indicate that CLiFT-ASR provides an effective and parameter-efficient solution for Taiwanese Hokkien ASR and that it has potential to benefit other low-resource language scenarios.

Method: Three-stage approach: 1) Deep feature extraction using large language models for contextual dependencies, 2) Multi-scale feature fusion via feature pyramids to balance global and local information, 3) Graph neural networks for structured modeling of semantic relations and logical dependencies.

Result: Significantly outperforms existing models on ACC, F1-Score, AUC, and Precision metrics, demonstrating effectiveness and stability in robustness alignment experiments.

Conclusion: The integrated framework successfully balances global/local information and semantics/structure, providing new perspectives for multi-scale feature fusion and structured semantic modeling in text classification.

Abstract: This study investigates a hybrid method for text classification that integrates deep feature extraction from large language models, multi-scale fusion through feature pyramids, and structured modeling with graph neural networks to enhance performance in complex semantic contexts. First, the large language model captures contextual dependencies and deep semantic representations of the input text, providing a rich feature foundation for subsequent modeling. Then, based on multi-level feature representations, the feature pyramid mechanism effectively integrates semantic features of different scales, balancing global information and local details to construct hierarchical semantic expressions. Furthermore, the fused features are transformed into graph representations, and graph neural networks are employed to capture latent semantic relations and logical dependencies in the text, enabling comprehensive modeling of complex interactions among semantic units. On this basis, the readout and classification modules generate the final category predictions. The proposed method demonstrates significant advantages in robustness alignment experiments, outperforming existing models on ACC, F1-Score, AUC, and Precision, which verifies the effectiveness and stability of the framework. This study not only constructs an integrated framework that balances global and local information as well as semantics and structure, but also provides a new perspective for multi-scale feature fusion and structured semantic modeling in text classification tasks.

Method: Analyze sample complexity for language generation in simple language families (regular and context-free languages) and examine computability bounds.

Result: Number of examples required for successful generation is extraordinarily large, sometimes unbounded by any computable function, revealing substantial gap between theory and practice.

Conclusion: Explaining empirical success of modern language models requires considering structural properties of natural language that enable practical generation, beyond theoretical guarantees.

Abstract: Kleinberg and Mullainathan showed that, in principle, language generation is always possible: with sufficiently many positive examples, a learner can eventually produce sentences indistinguishable from those of a target language. However, the existence of such a guarantee does not speak to its practical feasibility. In this work, we show that even for simple and well-studied language families – such as regular and context-free languages – the number of examples required for successful generation can be extraordinarily large, and in some cases not bounded by any computable function. These results reveal a substantial gap between theoretical possibility and efficient learnability. They suggest that explaining the empirical success of modern language models requires a refined perspective – one that takes into account structural properties of natural language that make effective generation possible in practice.

Method: DRAGON uses a lightweight detection module to identify forget-worthy prompts and routes them through a dedicated CoT guard model for safe in-context intervention, leveraging LLMs’ inherent instruction-following abilities.

Result: Extensive experiments across three unlearning tasks show DRAGON achieves strong unlearning capability, scalability, and practical applicability with novel evaluation metrics.

Conclusion: DRAGON provides an effective, data-free approach to LLM unlearning that works in practical scenarios without requiring model modifications or retain data.

Abstract: Unlearning in Large Language Models (LLMs) is crucial for protecting private data and removing harmful knowledge. Most existing approaches rely on fine-tuning to balance unlearning efficiency with general language capabilities. However, these methods typically require training or access to retain data, which is often unavailable in real world scenarios. Although these methods can perform well when both forget and retain data are available, few works have demonstrated equivalent capability in more practical, data-limited scenarios. To overcome these limitations, we propose Detect-Reasoning Augmented GeneratiON (DRAGON), a systematic, reasoning-based framework that utilizes in-context chain-of-thought (CoT) instructions to guard deployed LLMs before inference. Instead of modifying the base model, DRAGON leverages the inherent instruction-following ability of LLMs and introduces a lightweight detection module to identify forget-worthy prompts without any retain data. These are then routed through a dedicated CoT guard model to enforce safe and accurate in-context intervention. To robustly evaluate unlearning performance, we introduce novel metrics for unlearning performance and the continual unlearning setting. Extensive experiments across three representative unlearning tasks validate the effectiveness of DRAGON, demonstrating its strong unlearning capability, scalability, and applicability in practical scenarios.

Method: Compositional Phoneme Approximation (CPA) - a feature-based representation technique that approximates L2 sounds using sequences of L1 phonemes for pronunciation training.

Result: 76% in-box formant rate in acoustic analysis, 17.6% relative improvement in phoneme recognition accuracy, and over 80% of speech rated as more native-like with minimal training.

Conclusion: CPA-based L1-grounded training is an effective approach for improving L2 pronunciation by leveraging native language phoneme sequences to approximate target sounds.

Abstract: Learners of a second language (L2) often map non-native phonemes to similar native-language (L1) phonemes, making conventional L2-focused training slow and effortful. To address this, we propose an L1-grounded pronunciation training method based on compositional phoneme approximation (CPA), a feature-based representation technique that approximates L2 sounds with sequences of L1 phonemes. Evaluations with 20 Korean non-native English speakers show that CPA-based training achieves a 76% in-box formant rate in acoustic analysis, 17.6% relative improvement in phoneme recognition accuracy, and over 80% of speech being rated as more native-like, with minimal training. Project page: https://gsanpark.github.io/CPA-Pronunciation.

Method: Systematically evaluated 2 editing methods (MEMIT, AlphaEdit) and 3 fine-tuning approaches across 5 LLMs and 3 datasets (232 configurations). Proposed selective-layer fine-tuning.

Result: Edits decay after fine-tuning; AlphaEdit decays more than MEMIT. Fine-tuning edited layers removes edits effectively but hurts performance. Fine-tuning non-edited layers impairs edits more than full fine-tuning.

Conclusion: Knowledge editing evaluation must consider the full LLM pipeline. Selective-layer fine-tuning offers actionable strategies for integrating editing with fine-tuning while managing edit persistence.

Abstract: Knowledge editing has emerged as a lightweight alternative to retraining for correcting or injecting specific facts in large language models (LLMs). Meanwhile, fine-tuning remains the default operation for adapting LLMs to new domains and tasks. Despite their widespread adoption, these two post-training interventions have been studied in isolation, leaving open a crucial question: if we fine-tune an edited model, do the edits survive? This question is motivated by two practical scenarios: removing covert or malicious edits, and preserving beneficial edits. If fine-tuning impairs edits as shown in Figure 1, current KE methods become less useful, as every fine-tuned model would require re-editing, which significantly increases the cost; if edits persist, fine-tuned models risk propagating hidden malicious edits, raising serious safety concerns. To this end, we systematically quantify edits decay after fine-tuning, investigating how fine-tuning affects knowledge editing. We evaluate two state-of-the-art editing methods (MEMIT, AlphaEdit) and three fine-tuning approaches (full-parameter, LoRA, DoRA) across five LLMs and three datasets, yielding 232 experimental configurations. Our results show that edits decay after fine-tuning, with survival varying across configurations, e.g., AlphaEdit edits decay more than MEMIT edits. Further, we propose selective-layer fine-tuning and find that fine-tuning edited layers only can effectively remove edits, though at a slight cost to downstream performance. Surprisingly, fine-tuning non-edited layers impairs more edits than full fine-tuning. Overall, our study establishes empirical baselines and actionable strategies for integrating knowledge editing with fine-tuning, and underscores that evaluating model editing requires considering the full LLM application pipeline.

Method: Created MultiMed-ST dataset with 290,000 samples spanning Vietnamese, English, German, French, and Chinese; conducted comprehensive analysis including empirical baselines, bilingual-multilingual comparisons, end-to-end vs. cascaded approaches, task-specific vs. multi-task sequence-to-sequence models, code-switch analysis, and quantitative-qualitative error analysis.

Result: MultiMed-ST is the largest medical machine translation dataset and largest many-to-many multilingual speech translation dataset across all domains; comprehensive empirical analysis provides foundational insights for medical speech translation research.

Conclusion: The work establishes the first systematic study on medical speech translation, providing valuable resources and comprehensive analysis that will advance multilingual communication in healthcare settings and support future research in this critical domain.

Abstract: Multilingual speech translation (ST) and machine translation (MT) in the medical domain enhances patient care by enabling efficient communication across language barriers, alleviating specialized workforce shortages, and facilitating improved diagnosis and treatment, particularly during pandemics. In this work, we present the first systematic study on medical ST, to our best knowledge, by releasing MultiMed-ST, a large-scale ST dataset for the medical domain, spanning all translation directions in five languages: Vietnamese, English, German, French, and Simplified/Traditional Chinese, together with the models. With 290,000 samples, this is the largest medical MT dataset and the largest many-to-many multilingual ST among all domains. Secondly, we present the most comprehensive ST analysis in the field’s history, to our best knowledge, including: empirical baselines, bilingual-multilingual comparative study, end-to-end vs. cascaded comparative study, task-specific vs. multi-task sequence-to-sequence comparative study, code-switch analysis, and quantitative-qualitative error analysis. All code, data, and models are available online: https://github.com/leduckhai/MultiMed-ST

Method: The study conducted a systematic review of RAG applications in medicine, analyzing data sources, retrieval approaches, LLM usage, and evaluation methods across different medical applications.

Result: Research primarily used public data with limited private data application. Retrieval relied on English-centric embedding models, and LLMs were mostly generic rather than medical-specific. Evaluation focused on automated metrics for generation quality and human evaluation for accuracy, completeness, relevance, and fluency, with insufficient attention to bias and safety.

Conclusion: Medical RAG remains at an early stage and requires advances in clinical validation, cross-linguistic adaptation, and support for low-resource settings to enable trustworthy and responsible global use.

Abstract: The rapid growth of medical knowledge and increasing complexity of clinical practice pose challenges. In this context, large language models (LLMs) have demonstrated value; however, inherent limitations remain. Retrieval-augmented generation (RAG) technologies show potential to enhance their clinical applicability. This study reviewed RAG applications in medicine. We found that research primarily relied on publicly available data, with limited application in private data. For retrieval, approaches commonly relied on English-centric embedding models, while LLMs were mostly generic, with limited use of medical-specific LLMs. For evaluation, automated metrics evaluated generation quality and task performance, whereas human evaluation focused on accuracy, completeness, relevance, and fluency, with insufficient attention to bias and safety. RAG applications were concentrated on question answering, report generation, text summarization, and information extraction. Overall, medical RAG remains at an early stage, requiring advances in clinical validation, cross-linguistic adaptation, and support for low-resource settings to enable trustworthy and responsible global use.

Method: NILC uses an iterative workflow where LLMs create semantic centroids to enrich embedding centroids, augment hard samples via rewriting for cluster correction, and inject supervision through seeding and soft must links in semi-supervised settings.

Result: Extensive experiments show NILC achieves significant performance improvements over recent baselines across six benchmark datasets in both unsupervised and semi-supervised settings.

Conclusion: NILC effectively bridges the gap in existing NID approaches by leveraging LLMs for iterative refinement, demonstrating consistent performance gains across diverse domains.

Abstract: New intent discovery (NID) seeks to recognize both new and known intents from unlabeled user utterances, which finds prevalent use in practical dialogue systems. Existing works towards NID mainly adopt a cascaded architecture, wherein the first stage focuses on encoding the utterances into informative text embeddings beforehand, while the latter is to group similar embeddings into clusters (i.e., intents), typically by K-Means. However, such a cascaded pipeline fails to leverage the feedback from both steps for mutual refinement, and, meanwhile, the embedding-only clustering overlooks nuanced textual semantics, leading to suboptimal performance. To bridge this gap, this paper proposes NILC, a novel clustering framework specially catered for effective NID. Particularly, NILC follows an iterative workflow, in which clustering assignments are judiciously updated by carefully refining cluster centroids and text embeddings of uncertain utterances with the aid of large language models (LLMs). Specifically, NILC first taps into LLMs to create additional semantic centroids for clusters, thereby enriching the contextual semantics of the Euclidean centroids of embeddings. Moreover, LLMs are then harnessed to augment hard samples (ambiguous or terse utterances) identified from clusters via rewriting for subsequent cluster correction. Further, we inject supervision signals through non-trivial techniques seeding and soft must links for more accurate NID in the semi-supervised setting. Extensive experiments comparing NILC against multiple recent baselines under both unsupervised and semi-supervised settings showcase that NILC can achieve significant performance improvements over six benchmark datasets of diverse domains consistently.

Method: Automatically constructed dataset of English disyllabic words from read/spontaneous speech, trained CNN architectures with Layerwise Relevance Propagation (LRP) for interpretability analysis, and proposed feature-specific relevance analysis.

Result: CNNs achieved up to 92% accuracy on held-out test data; LRP revealed predictions were most influenced by stressed vs. unstressed syllable information, particularly spectral properties of stressed vowels (F1/F2, with some pitch and F3 contributions).

Conclusion: Deep learning can acquire distributed cues to stress from natural data, extending traditional phonetic work based on controlled stimuli, with interpretability revealing reliance on stressed vowel formants.

Abstract: Despite their success in speech processing, neural networks often operate as black boxes, prompting the question: what informs their decisions, and how can we interpret them? This work examines this issue in the context of lexical stress. A dataset of English disyllabic words was automatically constructed from read and spontaneous speech. Several Convolutional Neural Network (CNN) architectures were trained to predict stress position from a spectrographic representation of disyllabic words lacking minimal stress pairs (e.g., initial stress WAllet, final stress exTEND), achieving up to 92% accuracy on held-out test data. Layerwise Relevance Propagation (LRP), a technique for CNN interpretability analysis, revealed that predictions for held-out minimal pairs (PROtest vs. proTEST ) were most strongly influenced by information in stressed versus unstressed syllables, particularly the spectral properties of stressed vowels. However, the classifiers also attended to information throughout the word. A feature-specific relevance analysis is proposed, and its results suggest that our best-performing classifier is strongly influenced by the stressed vowel’s first and second formants, with some evidence that its pitch and third formant also contribute. These results reveal deep learning’s ability to acquire distributed cues to stress from naturally occurring data, extending traditional phonetic work based around highly controlled stimuli.

Method: IDALC (Intent Detection and Active Learning based Correction) - a semi-supervised framework that combines intent detection with active learning to correct system-rejected utterances.

Result: Empirical results show IDALC outperforms baseline methods with 5-10% higher accuracy and 4-8% improvement in macro-F1, while keeping annotation costs at only 6-10% of available unlabeled data.

Conclusion: IDALC provides an effective solution for handling system-rejected utterances and emerging intents in voice-controlled dialog systems while significantly reducing annotation burden.

Abstract: Voice-controlled dialog systems have become immensely popular due to their ability to perform a wide range of actions in response to diverse user queries. These agents possess a predefined set of skills or intents to fulfill specific user tasks. But every system has its own limitations. There are instances where, even for known intents, if any model exhibits low confidence, it results in rejection of utterances that necessitate manual annotation. Additionally, as time progresses, there may be a need to retrain these agents with new intents from the system-rejected queries to carry out additional tasks. Labeling all these emerging intents and rejected utterances over time is impractical, thus calling for an efficient mechanism to reduce annotation costs. In this paper, we introduce IDALC (Intent Detection and Active Learning based Correction), a semi-supervised framework designed to detect user intents and rectify system-rejected utterances while minimizing the need for human annotation. Empirical findings on various benchmark datasets demonstrate that our system surpasses baseline methods, achieving a 5-10% higher accuracy and a 4-8% improvement in macro-F1. Remarkably, we maintain the overall annotation cost at just 6-10% of the unlabelled data available to the system. The overall framework of IDALC is shown in Fig. 1

Method: Compare RL-enhanced models with base and SFT models on knowledge recall tasks, use structured prompting to guide SFT models through hierarchical traversal, and conduct layer-wise internal activation analysis to examine representation differences.

Result: RL-enhanced models outperform base and SFT models on knowledge recall, structured prompting reduces performance gap from 24pp to 7pp, RL models retain superior procedural path recall, and activation analysis shows RL transforms query representations rather than factual representations.

Conclusion: RL primarily enhances how models traverse and search existing knowledge hierarchies rather than acquiring new data or changing factual representations, improving procedural skills for knowledge navigation.

Abstract: Reinforcement learning (RL) is often credited with improving language model reasoning and generalization at the expense of degrading memorized knowledge. We challenge this narrative by observing that RL-enhanced models consistently outperform their base and supervised fine-tuned (SFT) counterparts on pure knowledge recall tasks, particularly those requiring traversal of hierarchical, structured knowledge (e.g., medical codes). We hypothesize these gains stem not from newly acquired data, but from improved procedural skills in navigating and searching existing knowledge hierarchies within the model parameters. To support this hypothesis, we show that structured prompting, which explicitly guides SFTed models through hierarchical traversal, recovers most of the performance gap (reducing 24pp to 7pp on MedConceptsQA for DeepSeek-V3/R1). We further find that while prompting improves final-answer accuracy, RL-enhanced models retain superior ability to recall correct procedural paths on deep-retrieval tasks. Finally our layer-wise internal activation analysis reveals that while factual representations (e.g., activations for the statement “code 57.95 refers to urinary infection”) maintain high cosine similarity between SFT and RL models, query representations (e.g., “what is code 57.95”) diverge noticeably, indicating that RL primarily transforms how models traverse knowledge rather than the knowledge representation itself.

Method: Weighted structured patterns (N-grams) with heterarchical relationships between them, applied to multi-factor classification of natural language texts.

Result: Significant improvements over literature-known F1 scores on two publicly available datasets, with optimal hyper-parameters determined.

Conclusion: The proposed recognition and learning algorithms advance the state of the art in cognitive distortion detection, with code and models made available for community use.

Abstract: We propose a new approach to multi-factor classification of natural language texts based on weighted structured patterns such as N-grams, taking into account the heterarchical relationships between them, applied to solve such a socially impactful problem as the automation of detection of specific cognitive distortions in psychological care, relying on an interpretable, robust and transparent artificial intelligence model. The proposed recognition and learning algorithms improve the current state of the art in this field. The improvement is tested on two publicly available datasets, with significant improvements over literature-known F1 scores for the task, with optimal hyper-parameters determined, having code and models available for future use by the community.

Method: Conducted extensive experiments to analyze entropy dynamics in RLVR-trained LLMs, examining correlations between entropy and response diversity, calibration, and performance. Investigated factors like off-policy updates, training data diversity, and clipping thresholds.

Result: Found that tokens with positive advantages are the primary contributors to entropy collapse. Model entropy can be effectively regulated by adjusting relative loss weights of tokens with positive vs. negative advantages during training.

Conclusion: The study provides insights into entropy dynamics in RLVR and demonstrates practical methods to control entropy collapse, enabling better performance and preventing premature convergence in LLM training.

Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as a predominant approach for enhancing the reasoning capabilities of large language models (LLMs). However, the entropy of LLMs usually collapses during RLVR training, causing premature convergence to suboptimal local minima and hinder further performance improvement. Although various approaches have been proposed to mitigate entropy collapse, a comprehensive study of entropy in RLVR remains lacking. To address this gap, we conduct extensive experiments to investigate the entropy dynamics of LLMs trained with RLVR and analyze how model entropy correlates with response diversity, calibration, and performance across various benchmarks. Our findings reveal that the number of off-policy updates, the diversity of training data, and the clipping thresholds in the optimization objective are critical factors influencing the entropy of LLMs trained with RLVR. Moreover, we theoretically and empirically demonstrate that tokens with positive advantages are the primary contributors to entropy collapse, and that model entropy can be effectively regulated by adjusting the relative loss weights of tokens with positive and negative advantages during training.

Method: Created DemogSummary dataset with age-stratified systematic review studies, evaluated three LLMs (Qwen, Longformer, GPT-4.1 Nano) using standard metrics and a new Demographic Salience Score (DSS) measuring age-related entity retention and hallucination.

Result: Systematic disparities across models and age groups: demographic fidelity lowest for adult-focused summaries, underrepresented populations more prone to hallucinations.

Conclusion: Current LLMs have limitations in faithful and bias-free summarization, highlighting need for fairness-aware evaluation frameworks and summarization pipelines in biomedical NLP.

Abstract: Clinical interventions often hinge on age: medications and procedures safe for adults may be harmful to children or ineffective for older adults. However, as language models are increasingly integrated into biomedical evidence synthesis workflows, it remains uncertain whether these systems preserve such crucial demographic distinctions. To address this gap, we evaluate how well state-of-the-art language models retain age-related information when generating abstractive summaries of biomedical studies. We construct DemogSummary, a novel age-stratified dataset of systematic review primary studies, covering child, adult, and older adult populations. We evaluate three prominent summarisation-capable LLMs, Qwen (open-source), Longformer (open-source) and GPT-4.1 Nano (proprietary), using both standard metrics and a newly proposed Demographic Salience Score (DSS), which quantifies age-related entity retention and hallucination. Our results reveal systematic disparities across models and age groups: demographic fidelity is lowest for adult-focused summaries, and under-represented populations are more prone to hallucinations. These findings highlight the limitations of current LLMs in faithful and bias-free summarisation and point to the need for fairness-aware evaluation frameworks and summarisation pipelines in biomedical NLP.

Method: Constructs synthetic English dataset from Chinese discrimination categories, trains DeBERTa-v3 reward model for multi-dimensional rewards (fairness, neutrality, linguistic quality), and uses GRPO fine-tuning guided by these rewards.

Result: Significant reductions in bias intensity and improved alignment with non-discriminatory standards while maintaining fluency and informativeness.

Conclusion: GRPO-based multi-reward optimization is effective for de-biasing LLMs and provides a replicable framework for cultural-contextual ethical alignment.

Abstract: Large Language Models (LLMs) often exhibit implicit biases and discriminatory tendencies that reflect underlying social stereotypes. While recent alignment techniques such as RLHF and DPO have mitigated some of these issues, they remain limited in addressing culturally specific and multi-dimensional forms of discrimination. This paper proposes a Multi-Reward Group Relative Policy Optimization (GRPO) framework to fine-tune LLMs toward ethical and bias-free behavior. Our approach constructs a synthetic English-language dataset derived from Chinese-context discrimination categories, including regional, ethnic, and occupational biases. Each instance is paired with both neutral and biased responses to train a reward model based on DeBERTa-v3, which provides multi-dimensional reward signals capturing fairness, neutrality, and linguistic quality. The trained reward model then guides GRPO fine-tuning to optimize model outputs along these ethical dimensions. Experimental results demonstrate significant reductions in bias intensity and improved alignment with non-discriminatory standards without compromising fluency or informativeness. This study highlights the effectiveness of GRPO-based multi-reward optimization for de-biasing LLMs and offers a replicable framework for cultural-contextual ethical alignment.

Method: Interactive visualization system combining topology-based visual encoding with dimensionality reduction to represent both local and global relationships among selected SAE features.

Result: Enables targeted investigation of SAE behavior through curated concept subsets, providing more faithful representation of feature relationships.

Conclusion: The framework facilitates deeper and more nuanced analysis of concept representation in latent space by prioritizing focused exploration over comprehensive visualization of all features.

Abstract: Sparse autoencoders (SAEs) have emerged as a powerful tool for uncovering interpretable features in large language models (LLMs) through the sparse directions they learn. However, the sheer number of extracted directions makes comprehensive exploration intractable. While conventional embedding techniques such as UMAP can reveal global structure, they suffer from limitations including high-dimensional compression artifacts, overplotting, and misleading neighborhood distortions. In this work, we propose a focused exploration framework that prioritizes curated concepts and their corresponding SAE features over attempts to visualize all available features simultaneously. We present an interactive visualization system that combines topology-based visual encoding with dimensionality reduction to faithfully represent both local and global relationships among selected features. This hybrid approach enables users to investigate SAE behavior through targeted, interpretable subsets, facilitating deeper and more nuanced analysis of concept representation in latent space.

Method: Three-layer framework combining rule-based pre-filtering with parameter-efficient LoRA-tuned BERTweet model and continuous learning capabilities, using strategic dataset unification and optimized fine-tuning.

Result: Achieves 0.85 macro F1 score (94% of SOTA SafePhi performance) with 100x smaller base model (134M vs 14B parameters), requiring only 1.87M trainable parameters (1.37% of full fine-tuning) and training in ~2 hours on single T4 GPU.

Conclusion: The system makes robust hate speech detection accessible in resource-constrained environments while maintaining competitive accuracy for real-world deployment, outperforming traditional BERT-based approaches with similar computational requirements.

Abstract: This paper addresses the critical challenge of developing computationally efficient hate speech detection systems that maintain competitive performance while being practical for real-time deployment. We propose a novel three-layer framework that combines rule-based pre-filtering with a parameter-efficient LoRA-tuned BERTweet model and continuous learning capabilities. Our approach achieves 0.85 macro F1 score - representing 94% of the performance of state-of-the-art large language models like SafePhi (Phi-4 based) while using a base model that is 100x smaller (134M vs 14B parameters). Compared to traditional BERT-based approaches with similar computational requirements, our method demonstrates superior performance through strategic dataset unification and optimized fine-tuning. The system requires only 1.87M trainable parameters (1.37% of full fine-tuning) and trains in approximately 2 hours on a single T4 GPU, making robust hate speech detection accessible in resource-constrained environments while maintaining competitive accuracy for real-world deployment.

Method: Uses structured workflow with formal graphs for Architectural Blueprint and symbols for Functional Specification, generating RTL and testbench code decoupled via symbols for verification and debugging.

Result: Outperforms direct generation methods and VerilogCoder in paper understanding and code completion; generated RTL meets timing constraints with performance metrics consistent with original papers.

Conclusion: ArchCraft successfully bridges the gap between academic architectural descriptions and practical hardware implementation, enabling reproducible and verifiable hardware design from research papers.

Abstract: The reproduction of hardware architectures from academic papers remains a significant challenge due to the lack of publicly available source code and the complexity of hardware description languages (HDLs). To this end, we propose \textbf{ArchCraft}, a Framework that converts abstract architectural descriptions from academic papers into synthesizable Verilog projects with register-transfer level (RTL) verification. ArchCraft introduces a structured workflow, which uses formal graphs to capture the Architectural Blueprint and symbols to define the Functional Specification, translating unstructured academic papers into verifiable, hardware-aware designs. The framework then generates RTL and testbench (TB) code decoupled via these symbols to facilitate verification and debugging, ultimately reporting the circuit’s Power, Area, and Performance (PPA). Moreover, we propose the first benchmark, \textbf{ArchSynthBench}, for synthesizing hardware from architectural descriptions, with a complete set of evaluation indicators, 50 project-level circuits, and around 600 circuit blocks. We systematically assess ArchCraft on ArchSynthBench, where the experiment results demonstrate the superiority of our proposed method, surpassing direct generation methods and the VerilogCoder framework in both paper understanding and code completion. Furthermore, evaluation and physical implementation of the generated executable RTL code show that these implementations meet all timing constraints without violations, and their performance metrics are consistent with those reported in the original papers.

Method: Embedding a deterministic validation step into the generative process that formally validates claims against structured knowledge graphs before output.

Result: Achieved perfect abstention precision (AP = 1.0) and zero false answers (FAR-NE = 0.0) with 89.1% factual accuracy, outperforming RAG and fine-tuning methods which failed to eliminate hallucinations.

Conclusion: Architectural innovations like the Licensing Oracle provide necessary and sufficient solutions for hallucinations in domains with structured knowledge, offering guarantees that statistical methods cannot match.

Abstract: Language models exhibit remarkable natural language generation capabilities but remain prone to hallucinations, generating factually incorrect information despite producing syntactically coherent responses. This study introduces the Licensing Oracle, an architectural solution designed to stem hallucinations in LMs by enforcing truth constraints through formal validation against structured knowledge graphs. Unlike statistical approaches that rely on data scaling or fine-tuning, the Licensing Oracle embeds a deterministic validation step into the model’s generative process, ensuring that only factually accurate claims are made. We evaluated the effectiveness of the Licensing Oracle through experiments comparing it with several state-of-the-art methods, including baseline language model generation, fine-tuning for factual recall, fine-tuning for abstention behavior, and retrieval-augmented generation (RAG). Our results demonstrate that although RAG and fine-tuning improve performance, they fail to eliminate hallucinations. In contrast, the Licensing Oracle achieved perfect abstention precision (AP = 1.0) and zero false answers (FAR-NE = 0.0), ensuring that only valid claims were generated with 89.1% accuracy in factual responses. This work shows that architectural innovations, such as the Licensing Oracle, offer a necessary and sufficient solution for hallucinations in domains with structured knowledge representations, offering guarantees that statistical methods cannot match. Although the Licensing Oracle is specifically designed to address hallucinations in fact-based domains, its framework lays the groundwork for truth-constrained generation in future AI systems, providing a new path toward reliable, epistemically grounded models.

Method: MuonAll transforms parameters into 2D matrices to incorporate all parameters within the Muon optimizer framework.

Result: Extensive finetuning experiments on models up to 500M parameters show Muon and MuonAll perform equivalently to AdamW across major benchmarks.

Conclusion: Muon and MuonAll are effective alternative optimizers for finetuning, with distributed implementations now open-sourced.

Abstract: Muon optimizer has demonstrated robust results in pretraining of language models but its performance in finetuning of existing public pretrained models is not yet explored. Currently, Muon is used along with AdamW introducing a scope of improvement for adopting all parameters inside Muon. We introduce MuonAll, which incorporates all the parameters inside Muon by transforming into 2D matrices. We conduct extensive finetuning experiments across publicly available language models with model sizes upto half billion parameters. Muon and MuonAll perform at par with AdamW across major benchmarks, highlighting their effectiveness as alternative optimizers. We open-source the distributed implementations of Muon and MuonAll, available at https://github.com/Saurabh750/optimizer

Method: In-depth analysis of QUEST-LOFT performance factors, thorough human evaluation, and optimization of RAG with structured output format containing reasoning and evidence, optionally with answer re-verification.

Result: RAG can be optimized to significantly outperform long-context approaches when combined with structured reasoning/evidence output and optional answer verification.

Conclusion: Structured RAG with explicit reasoning and evidence representation provides better performance than long-context models for complex QA tasks requiring distributed information retrieval and reasoning.

Abstract: Despite the popularity of retrieval-augmented generation (RAG) as a solution for grounded QA in both academia and industry, current RAG methods struggle with questions where the necessary information is distributed across many documents or where retrieval needs to be combined with complex reasoning. Recently, the LOFT study has shown that this limitation also applies to approaches based on long-context language models, with the QUEST benchmark exhibiting particularly large headroom. In this paper, we provide an in-depth analysis of the factors contributing to the poor performance on QUEST-LOFT, publish updated numbers based on a thorough human evaluation, and demonstrate that RAG can be optimized to significantly outperform long-context approaches when combined with a structured output format containing reasoning and evidence, optionally followed by answer re-verification.

Method: Proposed using Referring Expression Comprehension task to evaluate VLMs’ spatial reasoning, analyzing performance across task-specific architectures and large VLMs in scenarios with object detection ambiguity, complex spatial expressions, and negation.

Result: All models face challenges with spatial reasoning tasks, with relative performance varying based on model architecture and spatial semantic categories (topological, directional, proximal, etc.).

Conclusion: The study identifies research gaps in VLMs’ spatial reasoning capabilities and provides insights for future directions in improving spatial comprehension and grounding abilities.

Abstract: Spatial Reasoning is an important component of human cognition and is an area in which the latest Vision-language models (VLMs) show signs of difficulty. The current analysis works use image captioning tasks and visual question answering. In this work, we propose using the Referring Expression Comprehension task instead as a platform for the evaluation of spatial reasoning by VLMs. This platform provides the opportunity for a deeper analysis of spatial comprehension and grounding abilities when there is 1) ambiguity in object detection, 2) complex spatial expressions with a longer sentence structure and multiple spatial relations, and 3) expressions with negation (’not’). In our analysis, we use task-specific architectures as well as large VLMs and highlight their strengths and weaknesses in dealing with these specific situations. While all these models face challenges with the task at hand, the relative behaviors depend on the underlying models and the specific categories of spatial semantics (topological, directional, proximal, etc.). Our results highlight these challenges and behaviors and provide insight into research gaps and future directions.

Method: Developed BookAsSumQA framework that generates aspect-specific QA pairs from narrative knowledge graphs and evaluates summary quality based on question-answering performance.

Result: LLM-based approaches showed higher accuracy on shorter texts, while RAG-based methods became more effective as document length increased, making them more efficient for aspect-based book summarization.

Conclusion: RAG-based methods are more practical and efficient for aspect-based book summarization, especially for longer documents, as demonstrated through the BookAsSumQA evaluation framework.

Abstract: Aspect-based summarization aims to generate summaries that highlight specific aspects of a text, enabling more personalized and targeted summaries. However, its application to books remains unexplored due to the difficulty of constructing reference summaries for long text. To address this challenge, we propose BookAsSumQA, a QA-based evaluation framework for aspect-based book summarization. BookAsSumQA automatically generates aspect-specific QA pairs from a narrative knowledge graph to evaluate summary quality based on its question-answering performance. Our experiments using BookAsSumQA revealed that while LLM-based approaches showed higher accuracy on shorter texts, RAG-based methods become more effective as document length increases, making them more efficient and practical for aspect-based book summarization.

Method: Uses model-internal confidence scores from the smaller LLM’s logits before generating each reasoning step to decide when to invoke the larger model, enabling fine-grained step-level routing without external modules.

Result: Achieves competitive or enhanced accuracy while reducing inference costs (up to +20% accuracy with 48% less FLOPs compared to using only the larger model), outperforming baselines with trained external modules.

Conclusion: Model-internal confidence serves as a robust, domain-agnostic signal for efficient model routing, providing a scalable pathway for cost-effective LLM deployment.

Abstract: Recent advances in Large Language Models (LLMs) - particularly model scaling and test-time techniques - have greatly enhanced the reasoning capabilities of language models at the expense of higher inference costs. To lower inference costs, prior works train router models or deferral mechanisms that allocate easy queries to a small, efficient model, while forwarding harder queries to larger, more expensive models. However, these trained router models often lack robustness under domain shifts and require expensive data synthesis techniques such as Monte Carlo rollouts to obtain sufficient ground-truth routing labels for training. In this work, we propose Confidence-Guided Stepwise Model Routing for Cost-Efficient Reasoning (STEER), a domain-agnostic framework that performs fine-grained, step-level routing between smaller and larger LLMs without utilizing external models. STEER leverages confidence scores from the smaller model’s logits prior to generating a reasoning step, so that the large model is invoked only when necessary. Extensive evaluations using different LLMs on a diverse set of challenging benchmarks across multiple domains such as Mathematical Reasoning, Multi-Hop QA, and Planning tasks indicate that STEER achieves competitive or enhanced accuracy while reducing inference costs (up to +20% accuracy with 48% less FLOPs compared to solely using the larger model on AIME), outperforming baselines that rely on trained external modules. Our results establish model-internal confidence as a robust, domain-agnostic signal for model routing, offering a scalable pathway for efficient LLM deployment.

Method: EK-ICL incorporates three knowledge components: confidence scores from small language models for task-relevant patterns, parsing feature scores for structural differences and demo selection, and label word replacement to resolve semantic misalignment. It also uses parsing-based retrieval and ensemble prediction.

Result: Extensive experiments across three AD datasets show EK-ICL significantly outperforms state-of-the-art fine-tuning and ICL baselines, with analysis revealing high sensitivity to label semantics and task-specific context alignment.

Conclusion: Explicit knowledge integration is crucial for stable clinical reasoning under low-resource conditions, as ICL performance in AD detection depends heavily on proper alignment of label semantics and task-specific context.

Abstract: Detecting Alzheimer’s Disease (AD) from narrative transcripts remains a challenging task for large language models (LLMs), particularly under out-of-distribution (OOD) and data-scarce conditions. While in-context learning (ICL) provides a parameter-efficient alternative to fine-tuning, existing ICL approaches often suffer from task recognition failure, suboptimal demonstration selection, and misalignment between label words and task objectives, issues that are amplified in clinical domains like AD detection. We propose Explicit Knowledge In-Context Learners (EK-ICL), a novel framework that integrates structured explicit knowledge to enhance reasoning stability and task alignment in ICL. EK-ICL incorporates three knowledge components: confidence scores derived from small language models (SLMs) to ground predictions in task-relevant patterns, parsing feature scores to capture structural differences and improve demo selection, and label word replacement to resolve semantic misalignment with LLM priors. In addition, EK-ICL employs a parsing-based retrieval strategy and ensemble prediction to mitigate the effects of semantic homogeneity in AD transcripts. Extensive experiments across three AD datasets demonstrate that EK-ICL significantly outperforms state-of-the-art fine-tuning and ICL baselines. Further analysis reveals that ICL performance in AD detection is highly sensitive to the alignment of label semantics and task-specific context, underscoring the importance of explicit knowledge in clinical reasoning under low-resource conditions.

Method: Self-Priority Alignment (SPA) generates diverse responses, self-evaluates and refines them, applies dual-criterion denoising, constructs lexicographically ordered preference pairs, and fine-tunes with uncertainty-weighted alignment loss.

Result: SPA improves helpfulness without compromising safety across multiple benchmarks, outperforming strong baselines while preserving general capabilities.

Conclusion: SPA provides a scalable and interpretable alignment strategy for critical LLM applications by enforcing trustworthy-before-helpful ordering.

Abstract: In high-stakes scenarios-such as self-harm, legal, or medical queries-LLMs must be both trustworthy and helpful. However, these goals often conflict. We propose priority alignment, a new alignment paradigm that enforces a strict “trustworthy-before-helpful” ordering: optimization of helpfulness is conditioned on first meeting trustworthy thresholds (e.g., harmlessness or honesty). To realize this, we introduce Self-Priority Alignment (SPA)-a fully unsupervised framework that generates diverse responses, self-evaluates them and refines them by the model itself, and applies dual-criterion denoising to remove inconsistency and control variance. From this, SPA constructs lexicographically ordered preference pairs and fine-tunes the model using an uncertainty-weighted alignment loss that emphasizes high-confidence, high-gap decisions. Experiments across multiple benchmarks show that SPA improves helpfulness without compromising safety, outperforming strong baselines while preserving general capabilities. Our results demonstrate that SPA provides a scalable and interpretable alignment strategy for critical LLM applications.

Method: Used the CDrugRed dataset with 5,894 de-identified Chinese EHR records from 3,190 patients, and employed large language model (LLM)-based ensemble systems for multi-label medication recommendation.

Result: Top team achieved Jaccard score of 0.5102 and F1 score of 0.6267 on final test set, with 526 teams registering and 167/95 teams submitting valid results to Phase A/B leaderboards respectively.

Conclusion: Results demonstrate the potential of LLMs for medication recommendation in Chinese EHRs, though challenges remain in handling multi-label recommendations, heterogeneous clinical text, and patient-specific treatment variability.

Abstract: Discharge medication recommendation plays a critical role in ensuring treatment continuity, preventing readmission, and improving long-term management for patients with chronic metabolic diseases. This paper present an overview of the CHIP 2025 Shared Task 2 competition, which aimed to develop state-of-the-art approaches for automatically recommending appro-priate discharge medications using real-world Chinese EHR data. For this task, we constructed CDrugRed, a high-quality dataset consisting of 5,894 de-identified hospitalization records from 3,190 patients in China. This task is challenging due to multi-label nature of medication recommendation, het-erogeneous clinical text, and patient-specific variability in treatment plans. A total of 526 teams registered, with 167 and 95 teams submitting valid results to the Phase A and Phase B leaderboards, respectively. The top-performing team achieved the highest overall performance on the final test set, with a Jaccard score of 0.5102, F1 score of 0.6267, demonstrating the potential of advanced large language model (LLM)-based ensemble systems. These re-sults highlight both the promise and remaining challenges of applying LLMs to medication recommendation in Chinese EHRs. The post-evaluation phase remains open at https://tianchi.aliyun.com/competition/entrance/532411/.

Method: Fine-tuned ELECTRA-small on SNLI dataset, identified negation issues, then augmented training data with contrast sets and adversarial examples specifically targeting negation.

Result: Targeted data augmentation improved model accuracy on negation-containing examples while maintaining overall performance, effectively mitigating the dataset artifact.

Conclusion: Focused data augmentation addressing specific linguistic weaknesses can successfully improve model robustness to negation without compromising general performance.

Abstract: Pre-trained models for natural language inference (NLI) often achieve high performance on benchmark datasets by using spurious correlations, or dataset artifacts, rather than understanding language touches such as negation. In this project, we investigate the performance of an ELECTRA-small model fine-tuned on the Stanford Natural Language Inference (SNLI) dataset, focusing on its handling of negation. Through analysis, we identify that the model struggles with correctly classifying examples containing negation. To address this, we augment the training data with contrast sets and adversarial examples emphasizing negation. Our results demonstrate that this targeted data augmentation improves the model’s accuracy on negation-containing examples without adversely affecting overall performance, therefore mitigating the identified dataset artifact.

Method: Proposes TimeSense with a Temporal Sense module that reconstructs input time-series in the model’s context, and incorporates coordinate-based positional embeddings for spatial understanding of time-series data.

Result: TimeSense achieves state-of-the-art performance across multiple tasks, particularly excelling on complex multi-dimensional time-series reasoning tasks.

Conclusion: The proposed framework effectively balances textual reasoning with temporal sense preservation, enabling more accurate and grounded time-series analysis in challenging scenarios.

Abstract: In the time-series domain, an increasing number of works combine text with temporal data to leverage the reasoning capabilities of large language models (LLMs) for various downstream time-series understanding tasks. This enables a single model to flexibly perform tasks that previously required specialized models for each domain. However, these methods typically rely on text labels for supervision during training, biasing the model toward textual cues while potentially neglecting the full temporal features. Such a bias can lead to outputs that contradict the underlying time-series context. To address this issue, we construct the EvalTS benchmark, comprising 10 tasks across three difficulty levels, from fundamental temporal pattern recognition to complex real-world reasoning, to evaluate models under more challenging and realistic scenarios. We also propose TimeSense, a multimodal framework that makes LLMs proficient in time-series analysis by balancing textual reasoning with a preserved temporal sense. TimeSense incorporates a Temporal Sense module that reconstructs the input time-series within the model’s context, ensuring that textual reasoning is grounded in the time-series dynamics. Moreover, to enhance spatial understanding of time-series data, we explicitly incorporate coordinate-based positional embeddings, which provide each time point with spatial context and enable the model to capture structural dependencies more effectively. Experimental results demonstrate that TimeSense achieves state-of-the-art performance across multiple tasks, and it particularly outperforms existing methods on complex multi-dimensional time-series reasoning tasks.

Method: Use HatePrototypes - class-level vector representations derived from language models optimized for hate speech detection, built from as few as 50 examples per class. Implement parameter-free early exiting with prototypes.

Result: Prototypes enable cross-task transfer between explicit and implicit hate, with interchangeable prototypes across benchmarks. Early exiting with prototypes is effective for both hate types.

Conclusion: HatePrototypes provide an efficient and transferable approach to hate speech detection that works across different types of hate without requiring repeated fine-tuning.

Abstract: Optimization of offensive content moderation models for different types of hateful messages is typically achieved through continued pre-training or fine-tuning on new hate speech benchmarks. However, existing benchmarks mainly address explicit hate toward protected groups and often overlook implicit or indirect hate, such as demeaning comparisons, calls for exclusion or violence, and subtle discriminatory language that still causes harm. While explicit hate can often be captured through surface features, implicit hate requires deeper, full-model semantic processing. In this work, we question the need for repeated fine-tuning and analyze the role of HatePrototypes, class-level vector representations derived from language models optimized for hate speech detection and safety moderation. We find that these prototypes, built from as few as 50 examples per class, enable cross-task transfer between explicit and implicit hate, with interchangeable prototypes across benchmarks. Moreover, we show that parameter-free early exiting with prototypes is effective for both hate types. We release the code, prototype resources, and evaluation scripts to support future research on efficient and transferable hate speech detection.

Method: SugarTextNet integrates pretrained transformer encoder, attention-based cue extractor, and contextual phrase encoder with Context-Aware Focal Loss to handle class imbalance and capture nuanced features.

Result: Substantially outperforms traditional ML models, deep learning baselines, and large language models on a curated dataset of 3,067 Chinese Weibo posts across multiple metrics.

Conclusion: Domain-specific, context-aware modeling is crucial for sensitive content detection, providing robust solutions for content moderation in complex real-world scenarios.

Abstract: Sugar dating-related content has rapidly proliferated on mainstream social media platforms, giving rise to serious societal and regulatory concerns, including commercialization of intimate relationships and the normalization of transactional relationships.~Detecting such content is highly challenging due to the prevalence of subtle euphemisms, ambiguous linguistic cues, and extreme class imbalance in real-world data.~In this work, we present SugarTextNet, a novel transformer-based framework specifically designed to identify sugar dating-related posts on social media.~SugarTextNet integrates a pretrained transformer encoder, an attention-based cue extractor, and a contextual phrase encoder to capture both salient and nuanced features in user-generated text.~To address class imbalance and enhance minority-class detection, we introduce Context-Aware Focal Loss, a tailored loss function that combines focal loss scaling with contextual weighting.~We evaluate SugarTextNet on a newly curated, manually annotated dataset of 3,067 Chinese social media posts from Sina Weibo, demonstrating that our approach substantially outperforms traditional machine learning models, deep learning baselines, and large language models across multiple metrics.~Comprehensive ablation studies confirm the indispensable role of each component.~Our findings highlight the importance of domain-specific, context-aware modeling for sensitive content detection, and provide a robust solution for content moderation in complex, real-world scenarios.

Method: Created a dataset testing LLMs on known drug mechanisms and counterfactual reasoning scenarios, comparing performance across different models in open-world vs closed-world settings.

Result: o4-mini outperformed other OpenAI models, and Qwen3-4B-thinking matched o4-mini’s performance, sometimes exceeding it. Open-world reasoning was more challenging than closed-world, and counterfactuals affecting internal chain links were harder than those affecting drug links.

Conclusion: LLMs show varying capabilities in drug mechanism reasoning, with smaller models like Qwen3-4B-thinking performing competitively, and open-world reasoning presenting significant challenges that need addressing for reliable medical applications.

Abstract: Two scientific fields showing increasing interest in pre-trained large language models (LLMs) are drug development / repurposing, and personalized medicine. For both, LLMs have to demonstrate factual knowledge as well as a deep understanding of drug mechanisms, so they can recall and reason about relevant knowledge in novel situations. Drug mechanisms of action are described as a series of interactions between biomedical entities, which interlink into one or more chains directed from the drug to the targeted disease. Composing the effects of the interactions in a candidate chain leads to an inference about whether the drug might be useful or not for that disease. We introduce a dataset that evaluates LLMs on both factual knowledge of known mechanisms, and their ability to reason about them under novel situations, presented as counterfactuals that the models are unlikely to have seen during training. Using this dataset, we show that o4-mini outperforms the 4o, o3, and o3-mini models from OpenAI, and the recent small Qwen3-4B-thinking model closely matches o4-mini’s performance, even outperforming it in some cases. We demonstrate that the open world setting for reasoning tasks, which requires the model to recall relevant knowledge, is more challenging than the closed world setting where the needed factual knowledge is provided. We also show that counterfactuals affecting internal links in the reasoning chain present a much harder task than those affecting a link from the drug mentioned in the prompt.

Method: Used two Dutch cancer patient data sources (interviews and online forums) and tested state-of-the-art LLMs with different prompting strategies including chain of thought reasoning, few-shot learning, and self-prompting, with human verification.

Result: Created HealthQuote.NL corpus containing verified metaphors extracted from Dutch cancer patient data, demonstrating LLMs’ capability in this specialized domain with proper prompting techniques.

Conclusion: The extracted metaphors can enhance patient care through improved shared decision making, better patient-clinician communication, increased health literacy, and personalized care pathway design.

Abstract: Metaphors and metaphorical language (MLs) play an important role in healthcare communication between clinicians, patients, and patients’ family members. In this work, we focus on Dutch language data from cancer patients. We extract metaphors used by patients using two data sources: (1) cancer patient storytelling interview data and (2) online forum data, including patients’ posts, comments, and questions to professionals. We investigate how current state-of-the-art large language models (LLMs) perform on this task by exploring different prompting strategies such as chain of thought reasoning, few-shot learning, and self-prompting. With a human-in-the-loop setup, we verify the extracted metaphors and compile the outputs into a corpus named HealthQuote.NL. We believe the extracted metaphors can support better patient care, for example shared decision making, improved communication between patients and clinicians, and enhanced patient health literacy. They can also inform the design of personalized care pathways. We share prompts and related resources at https://github.com/aaronlifenghan/HealthQuote.NL

Method: Modular framework with learned routing network that directs queries to most fitting expert models, using two-stage open-source pipeline for vision tasks and reviving efficient classical vision components.

Result: Matches or exceeds performance of always-premium LLM systems on benchmarks like MMLU and VQA while reducing reliance on costly models by over 67%.

Conclusion: The framework delivers high-quality, resource-efficient AI at scale through extensible multi-agent orchestration.

Abstract: As AI moves beyond text, large language models (LLMs) increasingly power vision, audio, and document understanding; however, their high inference costs hinder real-time, scalable deployment. Conversely, smaller open-source models offer cost advantages but struggle with complex or multimodal queries. We introduce a unified, modular framework that intelligently routes each query - textual, multimodal, or complex - to the most fitting expert model, using a learned routing network that balances cost and quality. For vision tasks, we employ a two-stage open-source pipeline optimized for efficiency and reviving efficient classical vision components where they remain SOTA for sub-tasks. On benchmarks such as Massive Multitask Language Understanding (MMLU) and Visual Question Answering (VQA), we match or exceed the performance of always-premium LLM (monolithic systems with one model serving all query types) performance, yet reduce the reliance on costly models by over 67%. With its extensible, multi-agent orchestration, we deliver high-quality, resource-efficient AI at scale.

Method: Two-stage training: 1) Encode KBs into key-value pairs using pretrained encoder and inject into KV cache, 2) Locate retrieval layer and apply attention-based loss to supervise attention toward relevant KB entries. Enables retrieval entirely within model’s latent space.

Result: Successfully integrates up to 40K KBs into 7B LLM on single A100 40GB GPU. Achieves strong retrieval performance with over 98% Recall@10 on best task and >88% average across tasks. Maintains strong QA and KB ID generation performance while achieving up to 99.75% KB compression.

Conclusion: SR-KI provides an effective framework for integrating large-scale structured knowledge into LLMs with efficient compression and dynamic update capabilities, outperforming traditional retrieval-augmented approaches by enabling end-to-end inference within the model.

Abstract: This paper proposes SR-KI, a novel approach for integrating real-time and large-scale structured knowledge bases (KBs) into large language models (LLMs). SR-KI begins by encoding KBs into key-value pairs using a pretrained encoder, and injects them into LLMs’ KV cache. Building on this representation, we employ a two-stage training paradigm: first locating a dedicated retrieval layer within the LLM, and then applying an attention-based loss at this layer to explicitly supervise attention toward relevant KB entries. Unlike traditional retrieval-augmented generation methods that rely heavily on the performance of external retrievers and multi-stage pipelines, SR-KI supports end-to-end inference by performing retrieval entirely within the models latent space. This design enables efficient compression of injected knowledge and facilitates dynamic knowledge updates. Comprehensive experiments demonstrate that SR-KI enables the integration of up to 40K KBs into a 7B LLM on a single A100 40GB GPU, and achieves strong retrieval performance, maintaining over 98% Recall@10 on the best-performing task and exceeding 88% on average across all tasks. Task performance on question answering and KB ID generation also demonstrates that SR-KI maintains strong performance while achieving up to 99.75% compression of the injected KBs.

Method: Conducted extensive empirical study with controlled experiments comparing realignment using all available languages versus strategically selected linguistically diverse subsets.

Result: Realignment is particularly effective for low-resource languages, and carefully selected subsets can match full multilingual alignment while even outperforming it for unseen low-resource languages.

Conclusion: Effective realignment doesn’t require exhaustive language coverage; informed language selection enables efficient and robust cross-lingual transfer while reducing data collection overhead.

Abstract: Realignment is a promising strategy to improve cross-lingual transfer in multilingual language models. However, empirical results are mixed and often unreliable, particularly for typologically distant or low-resource languages (LRLs) compared to English. Moreover, word realignment tools often rely on high-quality parallel data, which can be scarce or noisy for many LRLs. In this work, we conduct an extensive empirical study to investigate whether realignment truly benefits from using all available languages, or if strategically selected subsets can offer comparable or even improved cross-lingual transfer, and study the impact on LRLs. Our controlled experiments show that realignment can be particularly effective for LRLs and that using carefully selected, linguistically diverse subsets can match full multilingual alignment, and even outperform it for unseen LRLs. This indicates that effective realignment does not require exhaustive language coverage and can reduce data collection overhead, while remaining both efficient and robust when guided by informed language selection.

Method: Two approaches: TAQ uses task-conditioned statistics from hidden activations to allocate bitwidths, while TAQO allocates precision based on direct layer sensitivity tests. Both identify task-relevant layers and preserve their precision while aggressively quantizing others.

Result: TAQ and TAQO outperform baselines across models. TAQ leads on Phi-4 (42.33 EM / 50.81 F1 vs AWQ’s 2.25/7.07), while TAQO leads on Llama-3.1, Qwen3, and Qwen2.5. Both remain within <1.0% of original accuracy at lower average precision.

Conclusion: Using hidden representations that encode task-salient signals as quantization guidelines yields stable task sensitivity profiles and efficient task-specialized models that maintain accuracy while reducing memory and latency.

Abstract: Large Language Models (LLMs) excel across diverse tasks, yet many applications require only limited capabilities, making large variants inefficient in memory and latency. Existing approaches often combine distillation and quantization, but most post-training quantization (PTQ) methods are task-agnostic, ignoring how task-specific signals are distributed across layers. In this work, we propose to use hidden representations that encode task-salient signals as a guideline for quantization. In order to fully utilize our innovative idea, this paper compares two new task-aware PTQ methods: Task-Aware Quantization (TAQ), which allocates bitwidths using task-conditioned statistics from hidden activations, and TAQO, which allocates precision based on direct layer sensitivity tests. From a small calibration set, these approaches identify task-relevant layers, preserving their precision while aggressively quantizing the rest. This yields stable task sensitivity profiles and efficient task-specialized models. Across models, TAQ and TAQO outperform the baselines; TAQ leads on Phi-4, while TAQO leads on Llama-3.1, Qwen3, and Qwen2.5. For instances, on Phi-4 it achieves 42.33 EM / 50.81 F1, far surpassing Activation-aware Weight Quantization (AWQ) (2.25 / 7.07), while remaining within < 1.0% of the original accuracy at lower average precision.

Method: RefineLab takes target quality attributes as refinement objectives and performs selective edits within a predefined token budget. It uses an assignment module to select optimal refinement strategies (e.g., rephrasing, distractor replacement) for each QA sample to maximize overall dataset quality while respecting budget constraints.

Result: Experiments show RefineLab consistently narrows divergence from expert datasets across coverage, difficulty alignment, factual fidelity, and distractor quality.

Conclusion: RefineLab pioneers a scalable, customizable path to reproducible dataset design with broad implications for LLM evaluation, addressing constrained optimization to improve QA sample quality within resource limitations.

Abstract: High-quality Question-Answer (QA) datasets are foundational for reliable Large Language Model (LLM) evaluation, yet even expert-crafted datasets exhibit persistent gaps in domain coverage, misaligned difficulty distributions, and factual inconsistencies. The recent surge in generative model-powered datasets has compounded these quality challenges. In this work, we introduce RefineLab, the first LLM-driven framework that automatically refines raw QA textual data into high-quality datasets under a controllable token-budget constraint. RefineLab takes a set of target quality attributes (such as coverage and difficulty balance) as refinement objectives, and performs selective edits within a predefined token budget to ensure practicality and efficiency. In essence, RefineLab addresses a constrained optimization problem: improving the quality of QA samples as much as possible while respecting resource limitations. With a set of available refinement operations (e.g., rephrasing, distractor replacement), RefineLab takes as input the original dataset, a specified set of target quality dimensions, and a token budget, and determines which refinement operations should be applied to each QA sample. This process is guided by an assignment module that selects optimal refinement strategies to maximize overall dataset quality while adhering to the budget constraint. Experiments demonstrate that RefineLab consistently narrows divergence from expert datasets across coverage, difficulty alignment, factual fidelity, and distractor quality. RefineLab pioneers a scalable, customizable path to reproducible dataset design, with broad implications for LLM evaluation.

Method: Created ibom dataset by extending Flores-200 benchmark to four Coastal Nigerian languages and aligning translated texts with SIB-200 topic classification labels.

Result: Current LLMs perform poorly on machine translation for these languages in zero- and few-shot settings, but few-shot samples steadily improve topic classification performance with more shots.

Conclusion: Highlights the need for more resources and research in underrepresented Nigerian languages, showing current limitations in machine translation but potential for topic classification improvement with few-shot learning.

Abstract: Nigeria is the most populous country in Africa with a population of more than 200 million people. More than 500 languages are spoken in Nigeria and it is one of the most linguistically diverse countries in the world. Despite this, natural language processing (NLP) research has mostly focused on the following four languages: Hausa, Igbo, Nigerian-Pidgin, and Yoruba (i.e <1% of the languages spoken in Nigeria). This is in part due to the unavailability of textual data in these languages to train and apply NLP algorithms. In this work, we introduce ibom – a dataset for machine translation and topic classification in four Coastal Nigerian languages from the Akwa Ibom State region: Anaang, Efik, Ibibio, and Oro. These languages are not represented in Google Translate or in major benchmarks such as Flores-200 or SIB-200. We focus on extending Flores-200 benchmark to these languages, and further align the translated texts with topic labels based on SIB-200 classification dataset. Our evaluation shows that current LLMs perform poorly on machine translation for these languages in both zero-and-few shot settings. However, we find the few-shot samples to steadily improve topic classification with more shots.

Method: Rep2Text uses a trainable adapter to project target model’s internal representations into a decoding LM’s embedding space for autoregressive text reconstruction.

Result: Over half of information in 16-token sequences can be recovered from compressed representations with strong semantic integrity and coherence; longer sequences show decreased token-level recovery but preserved semantics.

Conclusion: The framework demonstrates robust generalization to out-of-distribution medical data and reveals an information bottleneck effect in LLM representations.

Abstract: Large language models (LLMs) have achieved remarkable progress across diverse tasks, yet their internal mechanisms remain largely opaque. In this work, we address a fundamental question: to what extent can the original input text be recovered from a single last-token representation within an LLM? We propose Rep2Text, a novel framework for decoding full text from last-token representations. Rep2Text employs a trainable adapter that projects a target model’s internal representations into the embedding space of a decoding language model, which then autoregressively reconstructs the input text. Experiments on various model combinations (Llama-3.1-8B, Gemma-7B, Mistral-7B-v0.1, Llama-3.2-3B) demonstrate that, on average, over half of the information in 16-token sequences can be recovered from this compressed representation while maintaining strong semantic integrity and coherence. Furthermore, our analysis reveals an information bottleneck effect: longer sequences exhibit decreased token-level recovery while preserving strong semantic integrity. Besides, our framework also demonstrates robust generalization to out-of-distribution medical data.

Method: TabRAG uses structured language representations to parse table-heavy documents in a parsing-based RAG pipeline.

Result: TabRAG outperforms existing popular parsing-based methods for both generation and retrieval tasks.

Conclusion: TabRAG provides an effective solution for handling table-heavy documents in RAG systems without the computational overhead of fine-tuning approaches.

Abstract: Ingesting data for Retrieval-Augmented Generation (RAG) involves either fine-tuning the embedding model directly on the target corpus or parsing documents for embedding model encoding. The former, while accurate, incurs high computational hardware requirements, while the latter suffers from suboptimal performance when extracting tabular data. In this work, we address the latter by presenting TabRAG, a parsing-based RAG pipeline designed to tackle table-heavy documents via structured language representations. TabRAG outperforms existing popular parsing-based methods for generation and retrieval. Code is available at https://github.com/jacobyhsi/TabRAG.

Method: Proposes duality-based mode operations (split-unite, forward-backward, expansion-reduction, orthogonal dualities) and a pyramid multilayer mapping framework with three layers (rhetorical model, cognitive, epistemic). Uses binomial combinatorics and Shannon entropy analysis to quantify expressive diversity and complexity reduction.

Result: Identifies a Marginal Rhetorical Bit (MRB) parameter measuring expressive growth speed, shows hierarchical selection reduces choice uncertainty compared to flat selection, and transforms static rhetorical taxonomies into dynamic measurable systems.

Conclusion: The framework enables future AI systems to operate on layered rhetorical reasoning structures, bridging linguistic, pedagogical, academic, and computational research domains.

Abstract: Rhetorical modes are useful in both academic and non-academic writing, and can be subjects to be studied within linguistic research and computational modeling. Establishing a conceptual bridge among these domains could enable each to benefit from the others. This paper proposes duality-based mode operations (split-unite, forward-backward, expansion-reduction and orthogonal dualities) to expand the set of rhetorical modes, introducing generated modes like combination and generalization, thereby enhancing epistemic diversity across multiple applications. It further presents a pyramid multilayer mapping framework (e.g., three layers from the rhetorical model layer, to cognitive layer, and to epistemic layers) that reduces the resulting cognitive complexity. The degrees of expressive diversity and complexity reduction are quantified through binomial combinatorics and Shannon entropy analysis. A Marginal Rhetorical Bit (MRB) is identified, permitting the definition of a rhetorical-scalable parameter that measures expressive growth speed in bits per stage. A direct entropy measure shows that hierarchical selection over smaller subsets markedly reduces choice uncertainty compared with flat selection across all modes. These considerations appear to transform static and non-measurable rhetorical taxonomies into more dynamic and more measurable systems for discourse design. From this work, it would be possible to identify a pathway for future AI systems to operate not only on language tokens but on layered rhetorical reasoning structures, bridging linguistic, pedagogical, academic, and computational research

Method: Dual approach: quantitative benchmark of a widely used toxicity model (unitary/toxic-bert) comparing African-American English vs Standard American English, plus an interactive pedagogical tool with user-controlled sensitivity threshold.

Result: Clear systematic bias found - AAE text scored 1.8 times more toxic and 8.8 times higher for identity hate than SAE text. The tool demonstrates that biased scores combined with human-set policies operationalize discrimination.

Conclusion: Provides statistical evidence of disparate impact in AI moderation and a public-facing tool to foster critical AI literacy about how seemingly neutral policies can enable discrimination.

Abstract: Now that AI-driven moderation has become pervasive in everyday life, we often hear claims that “the AI is biased”. While this is often said jokingly, the light-hearted remark reflects a deeper concern. How can we be certain that an online post flagged as “inappropriate” was not simply the victim of a biased algorithm? This paper investigates this problem using a dual approach. First, I conduct a quantitative benchmark of a widely used toxicity model (unitary/toxic-bert) to measure performance disparity between text in African-American English (AAE) and Standard American English (SAE). The benchmark reveals a clear, systematic bias: on average, the model scores AAE text as 1.8 times more toxic and 8.8 times higher for “identity hate”. Second, I introduce an interactive pedagogical tool that makes these abstract biases tangible. The tool’s core mechanic, a user-controlled “sensitivity threshold,” demonstrates that the biased score itself is not the only harm; instead, the more-concerning harm is the human-set, seemingly neutral policy that ultimately operationalises discrimination. This work provides both statistical evidence of disparate impact and a public-facing tool designed to foster critical AI literacy.

Method: DIA-REFINE framework with iterative loop of translation, verification using external dialect classifiers, and feedback; introduces dialect fidelity score (DFS) and target dialect ratio (TDR) metrics.

Result: Consistently enhances dialect fidelity across Korean dialects; distinguishes False Success (high n-gram but failed dialect) from True Attempt (low n-gram but genuine dialect) cases; models show varying responsiveness; in-context examples further improve dialect expression translation.

Conclusion: Establishes robust framework for goal-directed, inclusive dialect translation with rigorous evaluation and insights into model performance.

Abstract: Standard-to-dialect machine translation remains challenging due to a persistent dialect gap in large language models and evaluation distortions inherent in n-gram metrics, which favor source copying over authentic dialect translation. In this paper, we propose the dialect refinement (DIA-REFINE) framework, which guides LLMs toward faithful target dialect outputs through an iterative loop of translation, verification, and feedback using external dialect classifiers. To address the limitations of n-gram-based metrics, we introduce the dialect fidelity score (DFS) to quantify linguistic shift and the target dialect ratio (TDR) to measure the success of dialect translation. Experiments on Korean dialects across zero-shot and in-context learning baselines demonstrate that DIA-REFINE consistently enhances dialect fidelity. The proposed metrics distinguish between False Success cases, where high n-gram scores obscure failures in dialectal translation, and True Attempt cases, where genuine attempts at dialectal translation yield low n-gram scores. We also observed that models exhibit varying degrees of responsiveness to the framework, and that integrating in-context examples further improves the translation of dialectal expressions. Our work establishes a robust framework for goal-directed, inclusive dialect translation, providing both rigorous evaluation and critical insights into model performance.

Method: TSAN emulates self-attention in natural language by formatting candidates into textual keys and values, using LLM-based attention to weigh relevance, and synthesizing strengths into preference-aligned responses through iterative optimization in textual gradient space.

Result: Empirical evaluations show TSAN outperforms supervised models like Llama-3.1-70B-Instruct and surpasses current state-of-the-art test-time alignment methods with just three test-time iterations on a base SFT model.

Conclusion: TSAN provides an effective, interpretable, and parameter-free approach for test-time preference optimization by systematically leveraging multiple candidate solutions through textual self-attention mechanisms.

Abstract: Large Language Models (LLMs) have demonstrated remarkable generalization capabilities, but aligning their outputs with human preferences typically requires expensive supervised fine-tuning. Recent test-time methods leverage textual feedback to overcome this, but they often critique and revise a single candidate response, lacking a principled mechanism to systematically analyze, weigh, and synthesize the strengths of multiple promising candidates. Such a mechanism is crucial because different responses may excel in distinct aspects (e.g., clarity, factual accuracy, or tone), and combining their best elements may produce a far superior outcome. This paper proposes the Textual Self-Attention Network (TSAN), a new paradigm for test-time preference optimization that requires no parameter updates. TSAN emulates self-attention entirely in natural language to overcome this gap: it analyzes multiple candidates by formatting them into textual keys and values, weighs their relevance using an LLM-based attention module, and synthesizes their strengths into a new, preference-aligned response under the guidance of the learned textual attention. This entire process operates in a textual gradient space, enabling iterative and interpretable optimization. Empirical evaluations demonstrate that with just three test-time iterations on a base SFT model, TSAN outperforms supervised models like Llama-3.1-70B-Instruct and surpasses the current state-of-the-art test-time alignment method by effectively leveraging multiple candidate solutions.

Method: Collected YouTube comments using YouTube API, pre-processed the data, and applied machine learning algorithms (Naïve Bayes, Logistic Regression, SVM) with TextBlob sentiment analysis tool.

Result: SVM demonstrated superior performance with the highest classification accuracy across different datasets, revealing trends and insights into user preferences and critiques in gaming videos.

Conclusion: Advanced sentiment analysis is crucial for capturing nuanced emotions in user comments, and future research will focus on integrating sophisticated NLP techniques and exploring additional data sources.

Abstract: The rapid evolution of the gaming industry, driven by technological advancements and a burgeoning community, necessitates a deeper understanding of user sentiments, especially as expressed on popular social media platforms like YouTube. This study presents a sentiment analysis on video games based on YouTube comments, aiming to understand user sentiments within the gaming community. Utilizing YouTube API, comments related to various video games were collected and analyzed using the TextBlob sentiment analysis tool. The pre-processed data underwent classification using machine learning algorithms, including Naïve Bayes, Logistic Regression, and Support Vector Machine (SVM). Among these, SVM demonstrated superior performance, achieving the highest classification accuracy across different datasets. The analysis spanned multiple popular gaming videos, revealing trends and insights into user preferences and critiques. The findings underscore the importance of advanced sentiment analysis in capturing the nuanced emotions expressed in user comments, providing valuable feedback for game developers to enhance game design and user experience. Future research will focus on integrating more sophisticated natural language processing techniques and exploring additional data sources to further refine sentiment analysis in the gaming domain.

Method: Comprehensive expert evaluation with 18 medical experts providing 80,502 annotations on 800 model outputs from GPT-4o and Llama-3.1-8B across 200 real-world patient and USMLE-style queries, systematically decomposing RAG pipeline into evidence retrieval, evidence selection, and response generation.

Result: Standard RAG degraded performance: only 22% of top-16 passages were relevant, evidence selection precision 41-43% and recall 27-49%, factuality and completeness dropped by up to 6% and 5% respectively compared to non-RAG variants.

Conclusion: Retrieval and evidence selection remain key failure points; simple strategies like evidence filtering and query reformulation substantially mitigate issues, improving performance on benchmarks by up to 12% and 8.2%, calling for re-examination of RAG’s role in medicine and highlighting need for stage-aware evaluation and deliberate system design.

Abstract: Large language models (LLMs) are transforming the landscape of medicine, yet two fundamental challenges persist: keeping up with rapidly evolving medical knowledge and providing verifiable, evidence-grounded reasoning. Retrieval-augmented generation (RAG) has been widely adopted to address these limitations by supplementing model outputs with retrieved evidence. However, whether RAG reliably achieves these goals remains unclear. Here, we present the most comprehensive expert evaluation of RAG in medicine to date. Eighteen medical experts contributed a total of 80,502 annotations, assessing 800 model outputs generated by GPT-4o and Llama-3.1-8B across 200 real-world patient and USMLE-style queries. We systematically decomposed the RAG pipeline into three components: (i) evidence retrieval (relevance of retrieved passages), (ii) evidence selection (accuracy of evidence usage), and (iii) response generation (factuality and completeness of outputs). Contrary to expectation, standard RAG often degraded performance: only 22% of top-16 passages were relevant, evidence selection remained weak (precision 41-43%, recall 27-49%), and factuality and completeness dropped by up to 6% and 5%, respectively, compared with non-RAG variants. Retrieval and evidence selection remain key failure points for the model, contributing to the overall performance drop. We further show that simple yet effective strategies, including evidence filtering and query reformulation, substantially mitigate these issues, improving performance on MedMCQA and MedXpertQA by up to 12% and 8.2%, respectively. These findings call for re-examining RAG’s role in medicine and highlight the importance of stage-aware evaluation and deliberate system design for reliable medical LLM applications.

Method: Evaluated 23 SLMs (270M-4B parameters) across multiple families using syntactic (character/word reversal) and semantic (irrelevant/counterfactual responses) transformations at 25%, 50%, 75%, 100% contamination levels during instruction tuning.

Result: Syntactic transformations caused catastrophic degradation (character reversal produced near-complete failure), while semantic transformations showed greater resilience. Larger models were more susceptible to learning harmful instructions (“capability curse”), and alignment provided inconsistent robustness benefits.

Conclusion: Current robustness assumptions don’t hold for SLMs, highlighting need for contamination-aware training protocols and systematic evaluation methods for deployment safety.

Abstract: Small Language Models (SLMs) are increasingly being deployed in resource-constrained environments, yet their behavioral robustness to data contamination during instruction tuning remains poorly understood. We systematically investigate the contamination sensitivity of 23 SLMs (270M to 4B parameters) across multiple model families by measuring susceptibility to syntactic and semantic transformation types during instruction tuning: syntactic transformations (character and word reversal) and semantic transformations (irrelevant and counterfactual responses), each applied at contamination levels of 25%, 50%, 75%, and 100%. Our results reveal fundamental asymmetries in vulnerability patterns: syntactic transformations cause catastrophic performance degradation, with character reversal producing near-complete failure across all models regardless of size or family, while semantic transformations demonstrate distinct threshold behaviors and greater resilience in core linguistic capabilities. Critically, we discover a ``\textit{capability curse}" where larger, more capable models become more susceptible to learning semantic corruptions, effectively following harmful instructions more readily, while our analysis of base versus instruction-tuned variants reveals that alignment provides inconsistent robustness benefits, sometimes even reducing resilience. Our work establishes three core contributions: (1) empirical evidence of SLMs’ disproportionate vulnerability to syntactic pattern contamination, (2) identification of asymmetric sensitivity patterns between syntactic and semantic transformations, and (3) systematic evaluation protocols for contamination robustness assessment. These findings have immediate deployment implications, suggesting that current robustness assumptions may not hold for smaller models and highlighting the need for contamination-aware training protocols.

Method: Proposes SafeNlidb framework with automated pipeline generating hybrid chain-of-thought interaction data, combining implicit security reasoning with SQL generation. Uses reasoning warm-up and alternating preference optimization to overcome multi-preference oscillations in DPO.

Result: Extensive experiments show the method outperforms larger-scale LLMs and ideal-setting baselines, achieving significant security improvements while preserving high utility.

Conclusion: SafeNlidb successfully addresses privacy-security concerns in LLM-based NLIDB through automated security reasoning and preference optimization, enabling secure SQL generation without human-annotated data.

Abstract: The rapid advancement of Large Language Models (LLMs) has driven significant progress in Natural Language Interface to Database (NLIDB). However, the widespread adoption of LLMs has raised critical privacy and security concerns. During interactions, LLMs may unintentionally expose confidential database contents or be manipulated by attackers to exfiltrate data through seemingly benign queries. While current efforts typically rely on rule-based heuristics or LLM agents to mitigate this leakage risk, these methods still struggle with complex inference-based attacks, suffer from high false positive rates, and often compromise the reliability of SQL queries. To address these challenges, we propose \textsc{SafeNlidb}, a novel privacy-security alignment framework for LLM-based NLIDB. The framework features an automated pipeline that generates hybrid chain-of-thought interaction data from scratch, seamlessly combining implicit security reasoning with SQL generation. Additionally, we introduce reasoning warm-up and alternating preference optimization to overcome the multi-preference oscillations of Direct Preference Optimization (DPO), enabling LLMs to produce security-aware SQL through fine-grained reasoning without the need for human-annotated preference data. Extensive experiments demonstrate that our method outperforms both larger-scale LLMs and ideal-setting baselines, achieving significant security improvements while preserving high utility.WARNING: This work may contain content that is offensive and harmful!

Method: Proposes Focal Attention that sharpens attention distribution by controlling softmax temperature as either fixed hyperparameter or learnable parameter during training.

Result: Achieves same accuracy with 42% fewer parameters or 33% less training data; delivers 17-82% relative improvements on long-context tasks; scales better than standard transformers.

Conclusion: Focal Attention is an effective modification that enables transformers to focus on relevant tokens while suppressing irrelevant ones, demonstrating significant improvements across various benchmarks and real-world applications.

Abstract: Attention is a core component of transformer architecture, whether encoder-only, decoder-only, or encoder-decoder model. However, the standard softmax attention often produces noisy probability distribution, which can impair effective feature selection at every layer of these models, particularly for long contexts. We propose Focal Attention, a simple yet effective modification that sharpens the attention distribution by controlling the softmax temperature, either as a fixed hyperparameter or as a learnable parameter during training. This sharpening enables the model to concentrate on the most relevant tokens while suppressing irrelevant ones. Empirically, Focal Attention scales more favorably than standard transformer with respect to model size, training data, and context length. Across diverse benchmarks, it achieves the same accuracy with up to 42% fewer parameters or 33% less training data. On long-context tasks, it delivers substantial relative improvements ranging from 17% to 82%, demonstrating its effectiveness in real world applications.

Method: Generated corpora using various LLMs (OpenAI, Anthropic, Alphabet, Meta, DeepSeek) ranging from GPT-3 to GPT-4.5, replicating reference human corpora BE21 (English) and Koditex (Czech), with Universal Dependencies tagging for tokenization, lemmatization, and morphological/syntactic annotation.

Result: Created two corpora: English part with average 864k tokens per model (27M total), Czech part with average 768k tokens per model (21.5M total), freely available under CC BY 4.0 license and accessible through Czech National Corpus search interface.

Conclusion: Successfully developed comprehensive LLM-generated text corpora that serve as valuable resources for linguistic analysis and comparison between human and AI-generated texts across multiple languages and genres.

Abstract: This article presents two corpora of English and Czech texts generated with large language models (LLMs). The motivation is to create a resource for comparing human-written texts with LLM-generated text linguistically. Emphasis was placed on ensuring these resources are multi-genre and rich in terms of topics, authors, and text types, while maintaining comparability with existing human-created corpora. These generated corpora replicate reference human corpora: BE21 by Paul Baker, which is a modern version of the original Brown Corpus, and Koditex corpus that also follows the Brown Corpus tradition but in Czech. The new corpora were generated using models from OpenAI, Anthropic, Alphabet, Meta, and DeepSeek, ranging from GPT-3 (davinci-002) to GPT-4.5, and are tagged according to the Universal Dependencies standard (i.e., they are tokenized, lemmatized, and morphologically and syntactically annotated). The subcorpus size varies according to the model used (the English part contains on average 864k tokens per model, 27M tokens altogether, the Czech partcontains on average 768k tokens per model, 21.5M tokens altogether). The corpora are freely available for download under the CC BY 4.0 license (the annotated data are under CC BY-NC-SA 4.0 licence) and are also accessible through the search interface of the Czech National Corpus.

Method: Proposes DA4ICL framework with Diverse and Contrastive Retrieval (DCR) to expand context width and Projected Vector Anchoring (PVA) to deepen signal perception at every Transformer layer.

Result: Achieves large, stable gains over both ICL and task vector baselines across three AD benchmarks.

Conclusion: DA4ICL establishes a new paradigm for fine-grained, out-of-distribution and low-resource LLM adaptation through demo-centric anchoring.

Abstract: Detecting Alzheimer’s disease (AD) from narrative transcripts challenges large language models (LLMs): pre-training rarely covers this out-of-distribution task, and all transcript demos describe the same scene, producing highly homogeneous contexts. These factors cripple both the model’s built-in task knowledge (\textbf{task cognition}) and its ability to surface subtle, class-discriminative cues (\textbf{contextual perception}). Because cognition is fixed after pre-training, improving in-context learning (ICL) for AD detection hinges on enriching perception through better demonstration (demo) sets. We demonstrate that standard ICL quickly saturates, its demos lack diversity (context width) and fail to convey fine-grained signals (context depth), and that recent task vector (TV) approaches improve broad task adaptation by injecting TV into the LLMs’ hidden states (HSs), they are ill-suited for AD detection due to the mismatch of injection granularity, strength and position. To address these bottlenecks, we introduce \textbf{DA4ICL}, a demo-centric anchoring framework that jointly expands context width via \emph{\textbf{Diverse and Contrastive Retrieval}} (DCR) and deepens each demo’s signal via \emph{\textbf{Projected Vector Anchoring}} (PVA) at every Transformer layer. Across three AD benchmarks, DA4ICL achieves large, stable gains over both ICL and TV baselines, charting a new paradigm for fine-grained, OOD and low-resource LLM adaptation.

Method: Two approaches: 1) NER modeling with roles as mutually exclusive classes using sequence tagging, 2) Entity Retrieval modeling with roles as queries and entities as collections. Automated learning of representative words/phrases for role and entity representations using sentence and document contexts.

Result: Developed methods to build role and entity representations using learned representative words and phrases, working effectively in domain-agnostic settings with small datasets.

Conclusion: Entity Role Detection can be effectively addressed through both NER and Entity Retrieval approaches, with automated representation learning enabling domain-agnostic performance even with limited training data.

Abstract: Most of the Natural Language Processing sys- tems are involved in entity-based processing for several tasks like Information Extraction, Question-Answering, Text-Summarization and so on. A new challenge comes when entities play roles according to their act or attributes in certain context. Entity Role Detection is the task of assigning such roles to the entities. Usu- ally real-world entities are of types: person, lo- cation and organization etc. Roles could be con- sidered as domain-dependent subtypes of these types. In the cases, where retrieving a subset of entities based on their roles is needed, poses the problem of defining the role and entities having those roles. This paper presents the study of study of solving Entity Role Detection prob- lem by modeling it as Named Entity Recogni- tion (NER) and Entity Retrieval/Ranking task. In NER, these roles could be considered as mutually exclusive classes and standard NER methods like sequence tagging could be used. For Entity Retrieval, Roles could be formulated as Query and entities as Collection on which the query needs to be executed. The aspect of Entity Retrieval task, which is different than document retrieval task is that the entities and roles against which they need to be retrieved are indirectly described. We have formulated au- tomated ways of learning representative words and phrases and building representations of roles and entities using them. We have also explored different contexts like sentence and document. Since the roles depend upon con- text, so it is not always possible to have large domain-specific dataset or knowledge bases for learning purposes, so we have tried to exploit the information from small dataset in domain- agnostic way.

Method: Proposed EduGuardBench with dual components: Role-playing Fidelity Score (RFS) for professional fidelity assessment, and persona-based adversarial prompts targeting general harms and academic misconduct, evaluated with Attack Success Rate (ASR) and three-tier Refusal Quality assessment.

Result: Experiments on 14 models show performance polarization - reasoning models have superior fidelity but incompetence dominates failures. Mid-sized models are most vulnerable (scaling paradox). Safest models exhibit Educational Transformation Effect, converting harmful requests into teachable moments with strong negative correlation to ASR.

Conclusion: EduGuardBench provides holistic assessment of professional, ethical, and pedagogical alignment, uncovering complex dynamics essential for trustworthy AI deployment in education, with Educational Refusals representing a new dimension of advanced AI safety.

Abstract: Large Language Models for Simulating Professions (SP-LLMs), particularly as teachers, are pivotal for personalized education. However, ensuring their professional competence and ethical safety is a critical challenge, as existing benchmarks fail to measure role-playing fidelity or address the unique teaching harms inherent in educational scenarios. To address this, we propose EduGuardBench, a dual-component benchmark. It assesses professional fidelity using a Role-playing Fidelity Score (RFS) while diagnosing harms specific to the teaching profession. It also probes safety vulnerabilities using persona-based adversarial prompts targeting both general harms and, particularly, academic misconduct, evaluated with metrics including Attack Success Rate (ASR) and a three-tier Refusal Quality assessment. Our extensive experiments on 14 leading models reveal a stark polarization in performance. While reasoning-oriented models generally show superior fidelity, incompetence remains the dominant failure mode across most models. The adversarial tests uncovered a counterintuitive scaling paradox, where mid-sized models can be the most vulnerable, challenging monotonic safety assumptions. Critically, we identified a powerful Educational Transformation Effect: the safest models excel at converting harmful requests into teachable moments by providing ideal Educational Refusals. This capacity is strongly negatively correlated with ASR, revealing a new dimension of advanced AI safety. EduGuardBench thus provides a reproducible framework that moves beyond siloed knowledge tests toward a holistic assessment of professional, ethical, and pedagogical alignment, uncovering complex dynamics essential for deploying trustworthy AI in education. See https://github.com/YL1N/EduGuardBench for Materials.

Method: Organizes reasoning steps into a tree structure, assigns weighted faithfulness scores to each step using hierarchical information, and dynamically adjusts weights to pinpoint reasoning failures. Created RPTS-Eval benchmark with 374 images and 390 reasoning instances.

Result: Evaluation of representative LVLMs (GPT4o, Llava-Next) revealed their limitations in multimodal reasoning and highlighted differences between open-source and closed-source models.

Conclusion: The RPTS benchmark and methodology will advance multimodal reasoning research by providing comprehensive evaluation of reasoning processes and intermodal relationships.

Abstract: Large Vision-Language Models (LVLMs) excel in multimodal reasoning and have shown impressive performance on various multimodal benchmarks. However, most of these benchmarks evaluate models primarily through multiple-choice or short-answer formats, which do not take the reasoning process into account. Although some benchmarks assess the reasoning process, their methods are often overly simplistic and only examine reasoning when answers are incorrect. This approach overlooks scenarios where flawed reasoning leads to correct answers. In addition, these benchmarks do not consider the impact of intermodal relationships on reasoning. To address this issue, we propose the Reasoning Process Tree Score (RPTS), a tree structure-based metric to assess reasoning processes. Specifically, we organize the reasoning steps into a reasoning tree and leverage its hierarchical information to assign weighted faithfulness scores to each reasoning step. By dynamically adjusting these weights, RPTS not only evaluates the overall correctness of the reasoning, but also pinpoints where the model fails in the reasoning. To validate RPTS in real-world multimodal scenarios, we construct a new benchmark, RPTS-Eval, comprising 374 images and 390 reasoning instances. Each instance includes reliable visual-textual clues that serve as leaf nodes of the reasoning tree. Furthermore, we define three types of intermodal relationships to investigate how intermodal interactions influence the reasoning process. We evaluated representative LVLMs (e.g., GPT4o, Llava-Next), uncovering their limitations in multimodal reasoning and highlighting the differences between open-source and closed-source commercial LVLMs. We believe that this benchmark will contribute to the advancement of research in the field of multimodal reasoning.

Method: Uses Human Language Preference Optimization (HLPO) - a reward-based alignment process that shifts token distribution toward human-like writing, making the model more sensitive to human writing patterns.

Result: Achieves 15.11% relative improvement in AUROC over ImBD for GPT-series revised text, and 45.56% improvement over Fast-DetectGPT. For advanced LLMs, achieves highest average AUROC exceeding ImBD by 5.53% and Fast-DetectGPT by 34.14%.

Conclusion: HLPD effectively enhances detection of machine-revised text by leveraging human writing style preferences, demonstrating superior performance across diverse revision scenarios.

Abstract: To prevent misinformation and social issues arising from trustworthy-looking content generated by LLMs, it is crucial to develop efficient and reliable methods for identifying the source of texts. Previous approaches have demonstrated exceptional performance in detecting texts fully generated by LLMs. However, these methods struggle when confronting more advanced LLM output or text with adversarial multi-task machine revision, especially in the black-box setting, where the generating model is unknown. To address this challenge, grounded in the hypothesis that human writing possesses distinctive stylistic patterns, we propose Human Language Preference Detection (HLPD). HLPD employs a reward-based alignment process, Human Language Preference Optimization (HLPO), to shift the scoring model’s token distribution toward human-like writing, making the model more sensitive to human writing, therefore enhancing the identification of machine-revised text. We test HLPD in an adversarial multi-task evaluation framework that leverages a five-dimensional prompt generator and multiple advanced LLMs to create diverse revision scenarios. When detecting texts revised by GPT-series models, HLPD achieves a 15.11% relative improvement in AUROC over ImBD, surpassing Fast-DetectGPT by 45.56%. When evaluated on texts generated by advanced LLMs, HLPD achieves the highest average AUROC, exceeding ImBD by 5.53% and Fast-DetectGPT by 34.14%. Code will be made available at https://github.com/dfq2021/HLPD.

Method: Uses sparse autoencoder (SAE) to project hidden states into high-dimensional near-monosemantic space, identifies copyright-sensitive subspace, and clamps its activations during decoding.

Result: Experiments show SCOPE effectively mitigates copyright infringement without degrading general model utility on widely recognized benchmarks.

Conclusion: The method successfully isolates copyright-sensitive semantics in a dedicated subspace and provides an intrinsic solution for copyright protection.

Abstract: Large language models sometimes inadvertently reproduce passages that are copyrighted, exposing downstream applications to legal risk. Most existing studies for inference-time defences focus on surface-level token matching and rely on external blocklists or filters, which add deployment complexity and may overlook semantically paraphrased leakage. In this work, we reframe copyright infringement mitigation as intrinsic semantic-space control and introduce SCOPE, an inference-time method that requires no parameter updates or auxiliary filters. Specifically, the sparse autoencoder (SAE) projects hidden states into a high-dimensional, near-monosemantic space; benefiting from this representation, we identify a copyright-sensitive subspace and clamp its activations during decoding. Experiments on widely recognized benchmarks show that SCOPE mitigates copyright infringement without degrading general utility. Further interpretability analyses confirm that the isolated subspace captures high-level semantics.

Method: Uses probe prompting pipeline: selects high-influence features, generates concept-targeted probes, groups features by cross-prompt activation signatures into Semantic, Relationship, and Say-X categories with transparent decision rules.

Result: Achieves high explanatory coverage (Completeness 0.83) with compressed complexity; concept-aligned groups show 2.3x higher peak-token consistency and 5.8x higher activation-pattern similarity than geometric clustering; entity-swap tests reveal layerwise hierarchy with early layers transferring robustly (64% transfer rate) and late layers specializing for output.

Conclusion: Probe prompting enables scalable automated interpretability, revealing a backbone-and-specialization view of transformer computation, with released code and demo for community adoption.

Abstract: Mechanistic interpretability aims to understand neural networks by identifying which learned features mediate specific behaviors. Attribution graphs reveal these feature pathways, but interpreting them requires extensive manual analysis – a single prompt can take approximately 2 hours for an experienced circuit tracer. We present probe prompting, an automated pipeline that transforms attribution graphs into compact, interpretable subgraphs built from concept-aligned supernodes. Starting from a seed prompt and target logit, we select high-influence features, generate concept-targeted yet context-varying probes, and group features by cross-prompt activation signatures into Semantic, Relationship, and Say-X categories using transparent decision rules. Across five prompts including classic “capitals” circuits, probe-prompted subgraphs preserve high explanatory coverage while compressing complexity (Completeness 0.83, mean across circuits; Replacement 0.54). Compared to geometric clustering baselines, concept-aligned groups exhibit higher behavioral coherence: 2.3x higher peak-token consistency (0.425 vs 0.183) and 5.8x higher activation-pattern similarity (0.762 vs 0.130), despite lower geometric compactness. Entity-swap tests reveal a layerwise hierarchy: early-layer features transfer robustly (64% transfer rate, mean layer 6.3), while late-layer Say-X features specialize for output promotion (mean layer 16.4), supporting a backbone-and-specialization view of transformer computation. We release code (https://github.com/peppinob-ol/attribution-graph-probing), an interactive demo (https://huggingface.co/spaces/Peppinob/attribution-graph-probing), and minimal artifacts enabling immediate reproduction and community adoption.

Method: Proposed strategic downsampling to mitigate directional degeneration from symmetric multi-way fine-tuning, and parallel multilingual prompting using typologically related auxiliary languages for better cross-lingual transfer.

Result: LMT achieves state-of-the-art performance among comparable models, with the 4B model surpassing larger models like Aya-101-13B and NLLB-54B by substantial margins.

Conclusion: LMT provides strong baselines for inclusive, scalable, and high-quality multilingual machine translation through rigorous data curation and refined adaptation strategies.

Abstract: Large language models have significantly advanced Multilingual Machine Translation (MMT), yet the broad language coverage, consistent translation quality, and English-centric bias remain open challenges. To address these challenges, we introduce \textbf{LMT}, a suite of \textbf{L}arge-scale \textbf{M}ultilingual \textbf{T}ranslation models centered on both Chinese and English, covering 60 languages and 234 translation directions. During development, we identify a previously overlooked phenomenon of \textbf{directional degeneration}, where symmetric multi-way fine-tuning data overemphasize reverse directions (X $\to$ En/Zh), leading to excessive many-to-one mappings and degraded translation quality. We propose \textbf{Strategic Downsampling}, a simple yet effective method to mitigate this degeneration. In addition, we design \textbf{Parallel Multilingual Prompting (PMP)}, which leverages typologically related auxiliary languages to enhance cross-lingual transfer. Through rigorous data curation and refined adaptation strategies, LMT achieves SOTA performance among models of comparable language coverage, with our 4B model (LMT-60-4B) surpassing the much larger Aya-101-13B and NLLB-54B models by a substantial margin. We release LMT in four sizes (0.6B/1.7B/4B/8B) to catalyze future research and provide strong baselines for inclusive, scalable, and high-quality MMT \footnote{\href{https://github.com/NiuTrans/LMT}{https://github.com/NiuTrans/LMT}}.

Method: Two-stage approach: 1) Vision-augmented judge-corrector pipeline using multimodal models to classify translations (correct/visually ambiguous/mistranslated) and route errors to specialized correctors (GPT-4o-mini for visual disambiguation, IndicTrans2 for quality issues); 2) LoRA fine-tuning of IndicTrans2 model on both original and corrected datasets.

Result: Corrected 17.1% of captions per language across 28,928 training examples. BLEU score improvements: English-Bengali +1.30 (eval) and +0.70 (challenge), English-Odia +0.60 (eval), English-Hindi +0.10 (challenge).

Conclusion: Automated error detection and correction in training data combined with parameter-efficient fine-tuning consistently improves translation quality across multiple English-to-Indic language pairs.

Abstract: In this paper, we describe our system under the team name BLEU Monday for the English-to-Indic Multimodal Translation Task at WAT 2025. We participate in the text-only translation tasks for English-Hindi, English-Bengali, English-Malayalam, and English-Odia language pairs. We present a two-stage approach that addresses quality issues in the training data through automated error detection and correction, followed by parameter-efficient model fine-tuning. Our methodology introduces a vision-augmented judge-corrector pipeline that leverages multimodal language models to systematically identify and correct translation errors in the training data. The judge component classifies translations into three categories: correct, visually ambiguous (requiring image context), or mistranslated (poor translation quality). Identified errors are routed to specialized correctors: GPT-4o-mini regenerates captions requiring visual disambiguation, while IndicTrans2 retranslates cases with pure translation quality issues. This automated pipeline processes 28,928 training examples across four languages, correcting an average of 17.1% of captions per language. We then apply Low-Rank Adaptation (LoRA) to fine-tune the IndicTrans2 en-indic 200M distilled model on both original and corrected datasets. Training on corrected data yields consistent improvements, with BLEU score gains of +1.30 for English-Bengali on the evaluation set (42.00 -> 43.30) and +0.70 on the challenge set (44.90 -> 45.60), +0.60 for English-Odia on the evaluation set (41.00 -> 41.60), and +0.10 for English-Hindi on the challenge set (53.90 -> 54.00).

Method: Longitudinal analysis of 5,836 speech recordings from 586 participants across UK, Netherlands, and Spain using linear mixed-effects modeling to identify lexical features, and testing four ML regressor models with interpretable features and vector embeddings.

Result: Limited language-specific associations found (7 features in English, 2 in Dutch, none in Spanish), with near-chance predictive performance across all languages using both lexical features and embeddings.

Conclusion: Further research needed with larger multilingual samples, improved protocols, and ML models accounting for individual language variations to understand lexical markers’ clinical value.

Abstract: Background: Captured between clinical appointments using mobile devices, spoken language has potential for objective, more regular assessment of symptom severity and earlier detection of relapse in major depressive disorder. However, research to date has largely been in non-clinical cross-sectional samples of written language using complex machine learning (ML) approaches with limited interpretability. Methods: We describe an initial exploratory analysis of longitudinal speech data and PHQ-8 assessments from 5,836 recordings of 586 participants in the UK, Netherlands, and Spain, collected in the RADAR-MDD study. We sought to identify interpretable lexical features associated with MDD symptom severity with linear mixed-effects modelling. Interpretable features and high-dimensional vector embeddings were also used to test the prediction performance of four regressor ML models. Results: In English data, MDD symptom severity was associated with 7 features including lexical diversity measures and absolutist language. In Dutch, associations were observed with words per sentence and positive word frequency; no associations were observed in recordings collected in Spain. The predictive power of lexical features and vector embeddings was near chance level across all languages. Limitations: Smaller samples in non-English speech and methodological choices, such as the elicitation prompt, may have also limited the effect sizes observable. A lack of NLP tools in languages other than English restricted our feature choice. Conclusion: To understand the value of lexical markers in clinical research and practice, further research is needed in larger samples across several languages using improved protocols, and ML models that account for within- and between-individual variations in language.

Method: Uses 16.1M query-document pairs (7.7M public + 8.4M synthetic from LLMs), conducts ablation studies on contrastive loss, synthetic data generation, and model merging, implements instruction-aware architecture.

Result: Achieves state-of-the-art performance across retrieval, classification, semantic textual similarity tasks, excels in multilingual scenarios including low-resource languages and cross-lingual setups.

Conclusion: Provides universal text embedding solution combining top-tier performance, broad applicability, and user-driven flexibility through instruction-aware capabilities.

Abstract: We introduce llama-embed-nemotron-8b, an open-weights text embedding model that achieves state-of-the-art performance on the Multilingual Massive Text Embedding Benchmark (MMTEB) leaderboard as of October 21, 2025. While recent models show strong performance, their training data or methodologies are often not fully disclosed. We aim to address this by developing a fully open-source model, publicly releasing its weights and detailed ablation studies, and planning to share the curated training datasets. Our model demonstrates superior performance across all major embedding tasks – including retrieval, classification and semantic textual similarity (STS) – and excels in challenging multilingual scenarios, such as low-resource languages and cross-lingual setups. This state-of-the-art performance is driven by a novel data mix of 16.1 million query-document pairs, split between 7.7 million samples from public datasets and 8.4 million synthetically generated examples from various open-weight LLMs. One of our key contributions is a detailed ablation study analyzing core design choices, including a comparison of contrastive loss implementations, an evaluation of synthetic data generation (SDG) strategies, and the impact of model merging. The llama-embed-nemotron-8b is an instruction-aware model, supporting user-defined instructions to enhance performance for specific use-cases. This combination of top-tier performance, broad applicability, and user-driven flexibility enables it to serve as a universal text embedding solution.

Method: Used DAIC-WOZ clinical interview dataset, compared LLMs (Llama, GPT) with classical ML and transformers (BERT, XLNet, Distil-RoBERTa), fine-tuned for anxiety/depression/stress classification, applied synthetic data generation for class imbalance.

Result: Distil-RoBERTa achieved highest F1 (0.883) for GAD-2 anxiety detection; XLNet best on PHQ depression tasks (F1 up to 0.891); zero-shot synthetic approach for stress detection reached F1 0.884 and ROC AUC 0.886.

Conclusion: Transformer-based models are effective for mental health detection, synthetic data improves recall and generalization, but careful calibration is needed to prevent precision loss. Combining advanced language models with data augmentation enhances automated mental health assessment.

Abstract: Mental health disorders affect over one-fifth of adults globally, yet detecting such conditions from text remains challenging due to the subtle and varied nature of symptom expression. This study evaluates multiple approaches for mental health detection, comparing Large Language Models (LLMs) such as Llama and GPT with classical machine learning and transformer-based architectures including BERT, XLNet, and Distil-RoBERTa. Using the DAIC-WOZ dataset of clinical interviews, we fine-tuned models for anxiety, depression, and stress classification and applied synthetic data generation to mitigate class imbalance. Results show that Distil-RoBERTa achieved the highest F1 score (0.883) for GAD-2, while XLNet outperformed others on PHQ tasks (F1 up to 0.891). For stress detection, a zero-shot synthetic approach (SD+Zero-Shot-Basic) reached an F1 of 0.884 and ROC AUC of 0.886. Findings demonstrate the effectiveness of transformer-based models and highlight the value of synthetic data in improving recall and generalization. However, careful calibration is required to prevent precision loss. Overall, this work emphasizes the potential of combining advanced language models and data augmentation to enhance automated mental health assessment from text.

Method: Case study on medical dataset with human-annotated complete evidence; analysis of individual models and ensemble approaches; examination of recall-precision trade-off, training with evidence, and dynamic ensembles with certainty thresholds.

Result: Individual models recover only subsets of complete evidence (~0.60 recall); ensemble aggregation improves evidence recall to ~0.86; analysis shows trade-offs and benefits of ensemble approaches.

Conclusion: Ensemble methods significantly improve complete evidence recall compared to single models, with important implications for applications requiring full feature attribution in compliance and cataloging contexts.

Abstract: Feature attribution methods typically provide minimal sufficient evidence justifying a model decision. However, in many applications this is inadequate. For compliance and cataloging, the full set of contributing features must be identified - complete evidence. We perform a case study on a medical dataset which contains human-annotated complete evidence. We show that individual models typically recover only subsets of complete evidence and that aggregating evidence from several models improves evidence recall from $\sim$0.60 (single best model) to $\sim$0.86 (ensemble). We analyze the recall-precision trade-off, the role of training with evidence, dynamic ensembles with certainty thresholds, and discuss implications.

Method: Integrates supervised attention mechanism into transformers with joint objective combining classification loss and alignment loss between attention weights and human-annotated rationales.

Result: Achieves 2.4x better explainability than baselines, produces more faithful token-level explanations, and maintains competitive fairness across metrics while detecting toxic posts targeting identity groups.

Conclusion: Incorporating human rationales into attention mechanisms enhances interpretability and faithfulness in hate speech classification without compromising fairness.

Abstract: The opaque nature of deep learning models presents significant challenges for the ethical deployment of hate speech detection systems. To address this limitation, we introduce Supervised Rational Attention (SRA), a framework that explicitly aligns model attention with human rationales, improving both interpretability and fairness in hate speech classification. SRA integrates a supervised attention mechanism into transformer-based classifiers, optimizing a joint objective that combines standard classification loss with an alignment loss term that minimizes the discrepancy between attention weights and human-annotated rationales. We evaluated SRA on hate speech benchmarks in English (HateXplain) and Portuguese (HateBRXplain) with rationale annotations. Empirically, SRA achieves 2.4x better explainability compared to current baselines, and produces token-level explanations that are more faithful and human-aligned. In terms of fairness, SRA achieves competitive fairness across all measures, with second-best performance in detecting toxic posts targeting identity groups, while maintaining comparable results on other metrics. These findings demonstrate that incorporating human rationales into attention mechanisms can enhance interpretability and faithfulness without compromising fairness.

Method: Derive MIWV metric from model response discrepancies using In-Context Learning to identify most beneficial data for instruction tuning.

Result: Selecting only top 1% of data based on MIWV outperforms training on full dataset.

Conclusion: MIWV offers effective data selection method beyond traditional quality scoring, with strong empirical evidence supporting its effectiveness.

Abstract: Instruction tuning plays a critical role in enhancing the performance and efficiency of Large Language Models (LLMs). Its success depends not only on the quality of the instruction data but also on the inherent capabilities of the LLM itself. Some studies suggest that even a small amount of high-quality data can achieve instruction fine-tuning results that are on par with, or even exceed, those from using a full-scale dataset. However, rather than focusing solely on calculating data quality scores to evaluate instruction data, there is a growing need to select high-quality data that maximally enhances the performance of instruction tuning for a given LLM. In this paper, we propose the Model Instruction Weakness Value (MIWV) as a novel metric to quantify the importance of instruction data in enhancing model’s capabilities. The MIWV metric is derived from the discrepancies in the model’s responses when using In-Context Learning (ICL), helping identify the most beneficial data for enhancing instruction tuning performance. Our experimental results demonstrate that selecting only the top 1% of data based on MIWV can outperform training on the full dataset. Furthermore, this approach extends beyond existing research that focuses on data quality scoring for data selection, offering strong empirical evidence supporting the effectiveness of our proposed method.

Method: Introduced a new Bengali emotion dataset annotated across eight emotion categories and proposed two models: (i) a hybrid Convolutional Recurrent Neural Network (CRNN) model (EmoBangHybrid) and (ii) an AdaBoost-Bidirectional Encoder Representations from Transformers (BERT) ensemble model (EmoBangEnsemble). Also evaluated baseline models, feature engineering techniques, and assessed zero-shot/few-shot LLMs.

Result: Experimental results show EmoBangHybrid achieved 92.86% accuracy and EmoBangEnsemble achieved 93.69% accuracy, outperforming existing methods and establishing strong baselines for future research.

Conclusion: This work provides the first comprehensive benchmark for Bengali emotion detection, demonstrating that the proposed models significantly advance the state-of-the-art and open new research directions for low-resource language processing.

Abstract: Emotion detection from text seeks to identify an individual’s emotional or mental state - positive, negative, or neutral - based on linguistic cues. While significant progress has been made for English and other high-resource languages, Bengali remains underexplored despite being the world’s fourth most spoken language. The lack of large, standardized datasets classifies Bengali as a low-resource language for emotion detection. Existing studies mainly employ classical machine learning models with traditional feature engineering, yielding limited performance. In this paper, we introduce a new Bengali emotion dataset annotated across eight emotion categories and propose two models for automatic emotion detection: (i) a hybrid Convolutional Recurrent Neural Network (CRNN) model (EmoBangHybrid) and (ii) an AdaBoost-Bidirectional Encoder Representations from Transformers (BERT) ensemble model (EmoBangEnsemble). Additionally, we evaluate six baseline models with five feature engineering techniques and assess zero-shot and few-shot large language models (LLMs) on the dataset. To the best of our knowledge, this is the first comprehensive benchmark for Bengali emotion detection. Experimental results show that EmoBangH and EmoBangE achieve accuracies of 92.86% and 93.69%, respectively, outperforming existing methods and establishing strong baselines for future research.

Method: Developed a pipeline to process Common Crawl dataset, preserving structural integrity of web content while maintaining flexibility for text-only and multimodal pre-training, with markdown output.

Result: Successfully created a new Arabic multimodal dataset with preserved document structure, publicly released a representative dataset dump and processing pipeline for Arabic.

Conclusion: The Wasm pipeline enables creation of high-quality Arabic multimodal datasets that can support better performance in Arabic LLMs and LMMs by preserving document structure similar to successful approaches in other languages.

Abstract: The performance of large language models (LLMs) and large multimodal models (LMMs) depends heavily on the quality and scale of their pre-training datasets. Recent research shows that large multimodal models trained on natural documents where images and text are interleaved outperform those trained only on image-text pairs across a wide range of benchmarks, leveraging advanced pre- trained models to enforce semantic alignment, image-sequence consistency, and textual coherence. For Arabic, however, the lack of high-quality multimodal datasets that preserve document structure has limited progress. In this paper, we present our pipeline Wasm for processing the Common Crawl dataset to create a new Arabic multimodal dataset that uniquely provides markdown output. Unlike existing Arabic corpora that focus solely on text extraction, our approach preserves the structural integrity of web content while maintaining flexibility for both text-only and multimodal pre-training scenarios. We provide a comprehensive comparative analysis of our data processing pipeline against those used for major existing datasets, highlighting the convergences in filtering strategies and justifying our specific design choices. To support future research, we publicly release a representative dataset dump along with the multimodal processing pipeline for Arabic.

Method: Used Agent Forest framework with sampling-and-voting to evaluate 6 open-source models across 4 math benchmarks, testing various agent counts (1-25) against punctuation noise (10-50%) and human-like typos (WikiTypo, R2ATA).

Result: Collaboration improves accuracy with more agents (largest gains from 1-5 agents), but adversarial robustness gap persists - human typos remain the dominant bottleneck with highest attack success rates even with many agents.

Conclusion: While multi-agent collaboration enhances accuracy, it does not eliminate vulnerability to adversarial perturbations, particularly human-like typos, suggesting fundamental robustness limitations in current LLM approaches.

Abstract: When LLM agents work together, they seem to be more powerful than a single LLM in mathematical question answering. However, are they also more robust to adversarial inputs? We investigate this question using adversarially perturbed math questions. These perturbations include punctuation noise with three intensities (10, 30, and 50 percent), plus real-world and human-like typos (WikiTypo, R2ATA). Using a unified sampling-and-voting framework (Agent Forest), we evaluate six open-source models (Qwen3-4B/14B, Llama3.1-8B, Mistral-7B, Gemma3-4B/12B) across four benchmarks (GSM8K, MATH, MMLU-Math, MultiArith), with various numbers of agents n from one to 25 (1, 2, 5, 10, 15, 20, 25). Our findings show that (1) Noise type matters: punctuation noise harm scales with its severity, and the human typos remain the dominant bottleneck, yielding the largest gaps to Clean accuracy and the highest ASR even with a large number of agents. And (2) Collaboration reliably improves accuracy as the number of agents, n, increases, with the largest gains from one to five agents and diminishing returns beyond 10 agents. However, the adversarial robustness gap persists regardless of the agent count.

Method: Proposes an Energy-Based Chain-of-Thought Calibration framework that dynamically adjusts latent thought representations toward lower-energy, high-consistency regions using energy-based models.

Result: Extensive experiments across mathematical, commonsense, and symbolic reasoning benchmarks show significant improvements in reasoning consistency and efficiency.

Conclusion: The EBM-CoT framework effectively enhances multi-step reasoning in LLMs by enforcing consistency among reasoning steps without modifying the base language model.

Abstract: Large Language Models (LLMs) have demonstrated strong reasoning capabilities through \emph{Chain-of-Thought} (CoT) prompting, which enables step-by-step intermediate reasoning. However, explicit CoT methods rely on discrete token-level reasoning processes that are prone to error propagation and limited by vocabulary expressiveness, often resulting in rigid and inconsistent reasoning trajectories. Recent research has explored implicit or continuous reasoning in latent spaces, allowing models to perform internal reasoning before generating explicit output. Although such approaches alleviate some limitations of discrete CoT, they generally lack explicit mechanisms to enforce consistency among reasoning steps, leading to divergent reasoning paths and unstable outcomes. To address this issue, we propose EBM-CoT, an Energy-Based Chain-of-Thought Calibration framework that refines latent thought representations through an energy-based model (EBM). Our method dynamically adjusts latent reasoning trajectories toward lower-energy, high-consistency regions in the embedding space, improving both reasoning accuracy and consistency without modifying the base language model. Extensive experiments across mathematical, commonsense, and symbolic reasoning benchmarks demonstrate that the proposed framework significantly enhances the consistency and efficiency of multi-step reasoning in LLMs.

Method: LoGo extracts signals from a single forward pass through LoRA adapters to identify the most relevant adapters and determine their contributions dynamically, without any training.

Result: Across 5 NLP benchmarks, 27 datasets, and 3 model families, LoGo outperforms training-based baselines by up to 3.6% on some tasks while remaining competitive on others, maintaining inference throughput.

Conclusion: LoGo is an effective and practical training-free framework for dynamic adapter selection and merging that works well across diverse NLP tasks without requiring additional training or labeled data.

Abstract: Low-Rank Adaptation (LoRA) has emerged as a parameter-efficient approach for fine-tuning large language models.However, conventional LoRA adapters are typically trained for a single task, limiting their applicability in real-world settings where inputs may span diverse and unpredictable domains. At inference time, existing approaches combine multiple LoRAs for improving performance on diverse tasks, while usually requiring labeled data or additional task-specific training, which is expensive at scale. In this work, we introduce LoRA on the Go (LoGo), a training-free framework that dynamically selects and merges adapters at the instance level without any additional requirements. LoGo leverages signals extracted from a single forward pass through LoRA adapters, to identify the most relevant adapters and determine their contributions on-the-fly. Across 5 NLP benchmarks, 27 datasets, and 3 model families, LoGo outperforms training-based baselines on some tasks upto a margin of 3.6% while remaining competitive on other tasks and maintaining inference throughput, highlighting its effectiveness and practicality.

Method: Created TCM-Eval benchmark from national medical exams, built large-scale training corpus, and developed Self-Iterative Chain-of-Thought Enhancement (SI-CoTE) to autonomously enrich QA pairs with validated reasoning chains.

Result: Developed ZhiMingTang (ZMT) LLM that significantly exceeds the passing threshold for human practitioners in TCM.

Conclusion: Established a virtuous cycle of data and model co-evolution, released public leaderboard to foster community engagement and continuous improvement in TCM AI research.

Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in modern medicine, yet their application in Traditional Chinese Medicine (TCM) remains severely limited by the absence of standardized benchmarks and the scarcity of high-quality training data. To address these challenges, we introduce TCM-Eval, the first dynamic and extensible benchmark for TCM, meticulously curated from national medical licensing examinations and validated by TCM experts. Furthermore, we construct a large-scale training corpus and propose Self-Iterative Chain-of-Thought Enhancement (SI-CoTE) to autonomously enrich question-answer pairs with validated reasoning chains through rejection sampling, establishing a virtuous cycle of data and model co-evolution. Using this enriched training data, we develop ZhiMingTang (ZMT), a state-of-the-art LLM specifically designed for TCM, which significantly exceeds the passing threshold for human practitioners. To encourage future research and development, we release a public leaderboard, fostering community engagement and continuous improvement.

Method: Used the ContArgA corpus annotations to perform zero-shot prompting experiments, evaluating the importance of gold-annotated and predicted emotions/appraisals for subjective convincingness assessment.

Result: Categorical emotion information improves convincingness prediction, but the improvement is more pronounced with appraisals. Appraisals show better performance than basic emotion categories.

Conclusion: This work presents the first systematic comparison between emotion models for convincingness prediction, demonstrating the advantage of appraisals over categorical emotions, providing insights for computational argumentation applications.

Abstract: The convincingness of an argument does not only depend on its structure (logos), the person who makes the argument (ethos), but also on the emotion that it causes in the recipient (pathos). While the overall intensity and categorical values of emotions in arguments have received considerable attention in the research community, we argue that the emotion an argument evokes in a recipient is subjective. It depends on the recipient’s goals, standards, prior knowledge, and stance. Appraisal theories lend themselves as a link between the subjective cognitive assessment of events and emotions. They have been used in event-centric emotion analysis, but their suitability for assessing argument convincingness remains unexplored. In this paper, we evaluate whether appraisal theories are suitable for emotion analysis in arguments by considering subjective cognitive evaluations of the importance and impact of an argument on its receiver. Based on the annotations in the recently published ContArgA corpus, we perform zero-shot prompting experiments to evaluate the importance of gold-annotated and predicted emotions and appraisals for the assessment of the subjective convincingness labels. We find that, while categorical emotion information does improve convincingness prediction, the improvement is more pronounced with appraisals. This work presents the first systematic comparison between emotion models for convincingness prediction, demonstrating the advantage of appraisals, providing insights for theoretical and practical applications in computational argumentation.

Method: Uses narrative profiling to transform user-item interactions into natural language, employs dual-channel architecture with horizontal behavioral alignment (peer patterns) and vertical causal attribution (preference factors), and supports rapid cross-task adaptation.

Result: Outperforms ML models and LLM baselines by up to 8% in few-shot settings, achieves 19% improvement over expert-crafted profiling in zero-shot scenarios, and lightweight fine-tuning matches fully fine-tuned models’ performance.

Conclusion: AdaRec effectively enables long-tail personalization with minimal interaction data, demonstrates strong generalization across tasks, and eliminates manual feature engineering through semantic representations.

Abstract: We propose AdaRec, a few-shot in-context learning framework that leverages large language models for an adaptive personalized recommendation. AdaRec introduces narrative profiling, transforming user-item interactions into natural language representations to enable unified task handling and enhance human readability. Centered on a bivariate reasoning paradigm, AdaRec employs a dual-channel architecture that integrates horizontal behavioral alignment, discovering peer-driven patterns, with vertical causal attribution, highlighting decisive factors behind user preferences. Unlike existing LLM-based approaches, AdaRec eliminates manual feature engineering through semantic representations and supports rapid cross-task adaptation with minimal supervision. Experiments on real ecommerce datasets demonstrate that AdaRec outperforms both machine learning models and LLM-based baselines by up to eight percent in few-shot settings. In zero-shot scenarios, it achieves up to a nineteen percent improvement over expert-crafted profiling, showing effectiveness for long-tail personalization with minimal interaction data. Furthermore, lightweight fine-tuning on synthetic data generated by AdaRec matches the performance of fully fine-tuned models, highlighting its efficiency and generalization across diverse tasks.

Method: Created EMODIS benchmark with ambiguous sentences containing emojis, two distinct disambiguating contexts leading to different interpretations, and questions requiring contextual reasoning. Evaluated both open-source and API-based LLMs.

Result: Even the strongest models frequently fail to distinguish meanings with subtle contextual cues, showing systematic biases toward dominant interpretations and limited sensitivity to pragmatic contrast.

Conclusion: EMODIS provides a rigorous testbed for assessing contextual disambiguation and highlights the semantic reasoning gap between humans and LLMs.

Abstract: Large language models (LLMs) are increasingly deployed in real-world communication settings, yet their ability to resolve context-dependent ambiguity remains underexplored. In this work, we present EMODIS, a new benchmark for evaluating LLMs’ capacity to interpret ambiguous emoji expressions under minimal but contrastive textual contexts. Each instance in EMODIS comprises an ambiguous sentence containing an emoji, two distinct disambiguating contexts that lead to divergent interpretations, and a specific question that requires contextual reasoning. We evaluate both open-source and API-based LLMs, and find that even the strongest models frequently fail to distinguish meanings when only subtle contextual cues are present. Further analysis reveals systematic biases toward dominant interpretations and limited sensitivity to pragmatic contrast. EMODIS provides a rigorous testbed for assessing contextual disambiguation, and highlights the gap in semantic reasoning between humans and LLMs.

Method: TransGraph uses discourse graphs to model inter-chunk relationships and selectively conditions translation segments on relevant graph neighborhoods instead of sequential or exhaustive context.

Result: TransGraph outperforms strong baselines on three document-level MT benchmarks across six languages and diverse domains, achieving better translation quality and terminology consistency with lower token overhead.

Conclusion: The discourse-guided framework with structured discourse graphs effectively addresses long-range dependency challenges in document translation while being more efficient than existing approaches.

Abstract: Adapting large language models to full document translation remains challenging due to the difficulty of capturing long-range dependencies and preserving discourse coherence throughout extended texts. While recent agentic machine translation systems mitigate context window constraints through multi-agent orchestration and persistent memory, they require substantial computational resources and are sensitive to memory retrieval strategies. We introduce TransGraph, a discourse-guided framework that explicitly models inter-chunk relationships through structured discourse graphs and selectively conditions each translation segment on relevant graph neighbourhoods rather than relying on sequential or exhaustive context. Across three document-level MT benchmarks spanning six languages and diverse domains, TransGraph consistently surpasses strong baselines in translation quality and terminology consistency while incurring significantly lower token overhead.

Method: Compare LLM predictions against expert annotations, establish inter-annotator consistency, use NER-guided prompting for automatic labeling of large news collections, and evaluate LLMs’ ability to infer protagonists with reduced context.

Result: PER is feasible and meaningful, with guided LLMs able to approximate human judgments of narrative importance at scale, demonstrating both inter-annotator consistency and human-LLM agreement.

Conclusion: PER represents a valuable extension to narrative-centered information extraction, enabling scalable identification of key narrative-driving entities in news content.

Abstract: News articles often reference numerous organizations, but traditional Named Entity Recognition (NER) treats all mentions equally, obscuring which entities genuinely drive the narrative. This limits downstream tasks that rely on understanding event salience, influence, or narrative focus. We introduce Protagonist Entity Recognition (PER), a task that identifies the organizations that anchor a news story and shape its main developments. To validate PER, we compare he predictions of Large Language Models (LLMs) against annotations from four expert annotators over a gold corpus, establishing both inter-annotator consistency and human-LLM agreement. Leveraging these findings, we use state-of-the-art LLMs to automatically label large-scale news collections through NER-guided prompting, generating scalable, high-quality supervision. We then evaluate whether other LLMs, given reduced context and without explicit candidate guidance, can still infer the correct protagonists. Our results demonstrate that PER is a feasible and meaningful extension to narrative-centered information extraction, and that guided LLMs can approximate human judgments of narrative importance at scale.

Method: Used soft-voting ensemble of transformers (BanglaBERT, MuRIL, IndicBERTv2) for subtasks 1A and 1B, and weighted voting ensemble of multitask variants for subtask 1C.

Result: Achieved micro-f1 scores of 72.75% (1A), 72.69% (1B), and 72.62% (1C), ranking 9th, 10th, and 7th respectively on the shared task leaderboard.

Conclusion: Transformer ensembles and weighted multitask frameworks show promise for advancing Bangla hate speech detection, with experimental scripts made publicly available.

Abstract: This paper addresses the problem of Bangla hate speech identification, a socially impactful yet linguistically challenging task. As part of the “Bangla Multi-task Hate Speech Identification” shared task at the BLP Workshop, IJCNLP-AACL 2025, our team “Retriv” participated in all three subtasks: (1A) hate type classification, (1B) target group identification, and (1C) joint detection of type, severity, and target. For subtasks 1A and 1B, we employed a soft-voting ensemble of transformer models (BanglaBERT, MuRIL, IndicBERTv2). For subtask 1C, we trained three multitask variants and aggregated their predictions through a weighted voting ensemble. Our systems achieved micro-f1 scores of 72.75% (1A) and 72.69% (1B), and a weighted micro-f1 score of 72.62% (1C). On the shared task leaderboard, these corresponded to 9th, 10th, and 7th positions, respectively. These results highlight the promise of transformer ensembles and weighted multitask frameworks for advancing Bangla hate speech detection in low-resource contexts. We made experimental scripts publicly available for the community.

Method: Propose data augmentation using LLMs to expand medical acronyms to full forms, and incorporate consistency training to enforce prediction agreement between original and augmented documents.

Result: Extensive experiments on MIMIC-III show ACE-ICD establishes new state-of-the-art performance across common codes, rare codes, and full-code assignments.

Conclusion: The approach effectively addresses the medical acronym challenge in ICD coding and demonstrates significant performance improvements across various coding scenarios.

Abstract: Automatic ICD coding, the task of assigning disease and procedure codes to electronic medical records, is crucial for clinical documentation and billing. While existing methods primarily enhance model understanding of code hierarchies and synonyms, they often overlook the pervasive use of medical acronyms in clinical notes, a key factor in ICD code inference. To address this gap, we propose a novel effective data augmentation technique that leverages large language models to expand medical acronyms, allowing models to be trained on their full form representations. Moreover, we incorporate consistency training to regularize predictions by enforcing agreement between the original and augmented documents. Extensive experiments on the MIMIC-III dataset demonstrate that our approach, ACE-ICD establishes new state-of-the-art performance across multiple settings, including common codes, rare codes, and full-code assignments. Our code is publicly available.

Method: Developed RLVE-Gym with 400 manually engineered verifiable environments that procedurally generate problems and provide algorithmically verifiable rewards, with difficulty adapting to model capabilities during training.

Result: Joint training across all 400 environments yielded 3.37% absolute average improvement across six reasoning benchmarks for a 1.5B LM, outperforming original RL training (0.49% gain) despite using 3x less compute.

Conclusion: Environment scaling through adaptive verifiable environments consistently improves generalizable reasoning capabilities in language models, demonstrating the effectiveness of RLVE approach.

Abstract: We introduce Reinforcement Learning (RL) with Adaptive Verifiable Environments (RLVE), an approach using verifiable environments that procedurally generate problems and provide algorithmically verifiable rewards, to scale up RL for language models (LMs). RLVE enables each verifiable environment to dynamically adapt its problem difficulty distribution to the policy model’s capabilities as training progresses. In contrast, static data distributions often lead to vanishing learning signals when problems are either too easy or too hard for the policy. To implement RLVE, we create RLVE-Gym, a large-scale suite of 400 verifiable environments carefully developed through manual environment engineering. Using RLVE-Gym, we show that environment scaling, i.e., expanding the collection of training environments, consistently improves generalizable reasoning capabilities. RLVE with joint training across all 400 environments in RLVE-Gym yields a 3.37% absolute average improvement across six reasoning benchmarks, starting from one of the strongest 1.5B reasoning LMs. By comparison, continuing this LM’s original RL training yields only a 0.49% average absolute gain despite using over 3x more compute. We release our code publicly.

Method: Systematically controlled synthetic experiments and empirical evaluations on state-of-the-art open-source and proprietary LLMs, including theoretical analysis of why confidence-based detection fails.

Result: Existing hallucination detection methods (confidence-based filtering, inner-state probing) fundamentally fail against spurious correlation-induced hallucinations, which are confidently generated and immune to model scaling.

Conclusion: There is an urgent need for new approaches specifically designed to address hallucinations caused by spurious correlations in LLMs.

Abstract: Despite substantial advances, large language models (LLMs) continue to exhibit hallucinations, generating plausible yet incorrect responses. In this paper, we highlight a critical yet previously underexplored class of hallucinations driven by spurious correlations – superficial but statistically prominent associations between features (e.g., surnames) and attributes (e.g., nationality) present in the training data. We demonstrate that these spurious correlations induce hallucinations that are confidently generated, immune to model scaling, evade current detection methods, and persist even after refusal fine-tuning. Through systematically controlled synthetic experiments and empirical evaluations on state-of-the-art open-source and proprietary LLMs (including GPT-5), we show that existing hallucination detection methods, such as confidence-based filtering and inner-state probing, fundamentally fail in the presence of spurious correlations. Our theoretical analysis further elucidates why these statistical biases intrinsically undermine confidence-based detection techniques. Our findings thus emphasize the urgent need for new approaches explicitly designed to address hallucinations caused by spurious correlations.

Method: Proposed a Dataset Construction Pipeline integrating 7 financial data types to automatically generate high-quality ERR datasets, and developed FinRpt-Gen - a multi-agent framework trained using Supervised Fine-Tuning and Reinforcement Learning.

Result: Experimental results demonstrate the high quality of the FinRpt benchmark data, effectiveness of the 11 evaluation metrics, and strong performance of the FinRpt-Gen framework in ERR generation.

Conclusion: The FinRpt benchmark and FinRpt-Gen framework show significant potential to drive innovation in automated Equity Research Report generation, with all code and datasets made publicly available.

Abstract: While LLMs have shown great success in financial tasks like stock prediction and question answering, their application in fully automating Equity Research Report generation remains uncharted territory. In this paper, we formulate the Equity Research Report (ERR) Generation task for the first time. To address the data scarcity and the evaluation metrics absence, we present an open-source evaluation benchmark for ERR generation - FinRpt. We frame a Dataset Construction Pipeline that integrates 7 financial data types and produces a high-quality ERR dataset automatically, which could be used for model training and evaluation. We also introduce a comprehensive evaluation system including 11 metrics to assess the generated ERRs. Moreover, we propose a multi-agent framework specifically tailored to address this task, named FinRpt-Gen, and train several LLM-based agents on the proposed datasets using Supervised Fine-Tuning and Reinforcement Learning. Experimental results indicate the data quality and metrics effectiveness of the benchmark FinRpt and the strong performance of FinRpt-Gen, showcasing their potential to drive innovation in the ERR generation field. All code and datasets are publicly available.

Method: Proposed NTK-Selector framework that addresses NTK application challenges in LLMs through empirical demonstration of NTK-like behavior during LoRA fine-tuning and Jacobian-free approximation to reduce computational costs.

Result: Across four domains (medical, financial, legal, psychological), NTK-Selector with 9,000 auxiliary samples achieved gains of +8.7 and +5.1 points for Llama3-8B and Qwen3-8B respectively, representing 10.9x and 5.7x improvements over domain-only fine-tuning.

Conclusion: NTK-Selector effectively selects valuable auxiliary data to significantly enhance domain-specific LLM performance in low-resource settings, overcoming traditional method limitations.

Abstract: Large language models (LLMs) have achieved remarkable success across widespread tasks, yet their application in low-resource domains remains a significant challenge due to data scarcity and the high risk of overfitting. While in-domain data is limited, there exist vast amounts of similar general-domain data, and our initial findings reveal that they could potentially serve as auxiliary supervision for domain enhancement. This observation leads us to our central research question: \textbf{\textit{how to effectively select the most valuable auxiliary data to maximize domain-specific performance}}, particularly when traditional methods are inapplicable due to a lack of large in-domain data pools or validation sets. To address this, we propose \textbf{NTK-Selector}, a principled and efficient framework for selecting general-domain auxiliary data to enhance domain-specific performance via neural tangent kernels (NTK). Our method tackles two challenges of directly applying NTK to LLMs, theoretical assumptions and prohibitive computational cost, by empirically demonstrating a stable NTK-like behavior in LLMs during LoRA fine-tuning and proposing a Jacobian-free approximation method. Extensive experiments across four low-resource domains (medical, financial, legal, and psychological) demonstrate that NTK-Selector consistently improves downstream performance. Specifically, fine-tuning on 1,000 in-domain samples alone only yielded +0.8 points for Llama3-8B-Instruct and +0.9 points for Qwen3-8B. In contrast, enriching with 9,000 auxiliary samples selected by NTK-Selector led to substantial \textbf{gains of +8.7 and +5.1 points}, which corresponds to a \textbf{10.9x and 5.7x improvement} over the domain-only setting.

Method: Combines instruction prompting with test-driven, feedback-guided iterative refinement using fine-tuned Qwen2.5-14B model, with three evaluation passes using test feedback.

Result: Achieved 2nd place in BLP Workshop shared task with Pass@1 score of 0.934.

Conclusion: Highlights challenges in Bangla instruction understanding and Python code generation, emphasizing need for targeted methods in low-resource languages.

Abstract: Large Language Models (LLMs) have advanced the automated generation of code from natural language prompts. However, low-resource languages (LRLs) like Bangla remain underrepresented due to the limited availability of instruction-to-code datasets and evaluation benchmarks. To address this, the BLP Workshop at IJCNLP-AACL 2025 introduced a shared task on “Code Generation in Bangla”. In this work, we propose a method that combines instruction prompting with a test-driven, feedback-guided iterative refinement process using a fine-tuned Qwen2.5-14B model. The model generates code from Bangla instructions, tests it against unit tests, and iteratively refines any failing outputs through three evaluation passes, using test feedback to guide each step. This approach helped our team “Retriv” to secure 2nd place in the shared task with a Pass@1 score of 0.934. The analysis highlights challenges in Bangla instruction understanding and Python code generation, emphasizing the need for targeted methods in LRLs. We made experimental scripts publicly available for the community.

Method: Using a curriculum of recurrences to gradually increase the effective depth of pretrained non-recurrent models during training, converting them to depth-recurrent architectures.

Result: The converted recurrent models achieve better performance at the same compute budget compared to simply post-training the original non-recurrent models, particularly on mathematical tasks.

Conclusion: Curriculum-based conversion of pretrained models to depth-recurrent architectures is an effective strategy for computational efficiency while maintaining or improving performance.

Abstract: Recent advances in depth-recurrent language models show that recurrence can decouple train-time compute and parameter count from test-time compute. In this work, we study how to convert existing pretrained non-recurrent language models into depth-recurrent models. We find that using a curriculum of recurrences to increase the effective depth of the model over the course of training preserves performance while reducing total computational cost. In our experiments, on mathematics, we observe that converting pretrained models to recurrent ones results in better performance at a given compute budget than simply post-training the original non-recurrent language model.

Method: Hierarchical multi-agent framework with orchestration agent and three task-specific LLM-driven agents that autonomously plan, refine, validate, and reason to map voice commands into specific tasks like retrieving clinical information or manipulating CT scans.

Result: SAOP achieves high accuracy and success rates across 240 voice commands, with LLM-based agents improving robustness against speech recognition errors and handling diverse/ambiguous free-form commands.

Conclusion: The platform demonstrates strong potential to support minimally invasive da Vinci robotic surgery by enabling hands-free multimodal data access.

Abstract: In da Vinci robotic surgery, surgeons’ hands and eyes are fully engaged in the procedure, making it difficult to access and manipulate multimodal patient data without interruption. We propose a voice-directed Surgical Agent Orchestrator Platform (SAOP) built on a hierarchical multi-agent framework, consisting of an orchestration agent and three task-specific agents driven by Large Language Models (LLMs). These LLM-based agents autonomously plan, refine, validate, and reason to map voice commands into specific tasks such as retrieving clinical information, manipulating CT scans, or navigating 3D anatomical models on the surgical video. We also introduce a Multi-level Orchestration Evaluation Metric (MOEM) to comprehensively assess the performance and robustness from command-level and category-level perspectives. The SAOP achieves high accuracy and success rates across 240 voice commands, while LLM-based agents improve robustness against speech recognition errors and diverse or ambiguous free-form commands, demonstrating strong potential to support minimally invasive da Vinci robotic surgery.

Method: Introduce conversational infill task where lightweight on-device model generates dialogue while incorporating streaming knowledge from backend model. Train ConvFill (360M parameter model) on synthetic multi-domain conversations.

Result: ConvFill achieves 36-42% accuracy improvements over standalone small models of same size while consistently maintaining sub-200ms response latencies across multiple backend models.

Conclusion: Conversational infill enables building on-device conversational agents that are both immediately responsive and knowledgeable by decoupling response latency from model capability.

Abstract: Deploying conversational voice agents with large language models faces a critical challenge: cloud-based foundation models provide deep reasoning and domain knowledge but introduce latency that disrupts natural conversation, while on-device models respond immediately but lack sophistication. We propose conversational infill, a task where a lightweight on-device model generates contextually appropriate dialogue while seamlessly incorporating streaming knowledge from a powerful backend model. This approach decouples response latency from model capability, enabling systems that feel responsive while accessing the full power of large-scale models. We present ConvFill, a 360M parameter model trained on synthetic multi-domain conversations. Evaluation across multiple backend models shows that conversational infill can be successfully learned, with ConvFill achieving accuracy improvements of 36-42% over standalone small models of the same size while consistently retaining sub-200ms response latencies. Our results demonstrate the promise of this approach for building on-device conversational agents that are both immediately responsive and knowledgeable.

Method: Created annotated corpus with reliable guidelines, benchmarked fine-tuned CamemBERT encoder models and instruction-tuned LLMs with various prompting strategies, incorporating contextual metadata (article, post, parent comment, page/group, source).

Result: Fine-tuned encoders outperformed prompted LLMs by >10 percentage points in F1 score. Contextual metadata improved encoder F1 scores from 0.75 to 0.78.

Conclusion: Supervised learning is crucial for emerging non-English social media tasks. The released dataset, guidelines, and code promote transparency and reproducible research in detecting subtle conversational interventions.

Abstract: We introduce SPOT (Stopping Points in Online Threads), the first annotated corpus translating the sociological concept of stopping point into a reproducible NLP task. Stopping points are ordinary critical interventions that pause or redirect online discussions through a range of forms (irony, subtle doubt or fragmentary arguments) that frameworks like counterspeech or social correction often overlook. We operationalize this concept as a binary classification task and provide reliable annotation guidelines. The corpus contains 43,305 manually annotated French Facebook comments linked to URLs flagged as false information by social media users, enriched with contextual metadata (article, post, parent comment, page or group, and source). We benchmark fine-tuned encoder models (CamemBERT) and instruction-tuned LLMs under various prompting strategies. Results show that fine-tuned encoders outperform prompted LLMs in F1 score by more than 10 percentage points, confirming the importance of supervised learning for emerging non-English social media tasks. Incorporating contextual metadata further improves encoder models F1 scores from 0.75 to 0.78. We release the anonymized dataset, along with the annotation guidelines and code in our code repository, to foster transparency and reproducible research.

Method: Proposes DiLA with a differential logic layer that integrates logical constraints into network forward/backward passes. LLM transforms language to logic constraints and finds initial solutions, while the logic layer iteratively refines them.

Result: DiLA consistently outperforms existing prompt-based and solver-aided approaches on two classic reasoning problems, enhancing LLMs’ logical reasoning ability while guaranteeing solution efficiency and correctness.

Conclusion: The differential logic layer effectively bridges LLMs and logical reasoning, providing a viable alternative for LLM tool learning that improves performance on constraint satisfaction problems encoded by Boolean variables.

Abstract: Considering the challenges faced by large language models (LLMs) in logical reasoning and planning, prior efforts have sought to augment LLMs with access to external solvers. While progress has been made on simple reasoning problems, solving classical constraint satisfaction problems, such as the Boolean Satisfiability Problem (SAT) and Graph Coloring Problem (GCP), remains difficult for off-the-shelf solvers due to their intricate expressions and exponential search spaces. In this paper, we propose a novel differential logic layer-aided language modeling (DiLA) approach, where logical constraints are integrated into the forward and backward passes of a network layer, to provide another option for LLM tool learning. In DiLA, LLM aims to transform the language description to logic constraints and identify initial solutions of the highest quality, while the differential logic layer focuses on iteratively refining the LLM-prompted solution. Leveraging the logic layer as a bridge, DiLA enhances the logical reasoning ability of LLMs on a range of reasoning problems encoded by Boolean variables, guaranteeing the efficiency and correctness of the solution process. We evaluate the performance of DiLA on two classic reasoning problems and empirically demonstrate its consistent outperformance against existing prompt-based and solver-aided approaches.

Method: Proposed method uses highly biased instances as few-shot examples for in-context learning to mitigate likelihood bias in LLM-based evaluators.

Result: Experiments in data-to-text and grammatical error correction tasks show several LLMs display likelihood bias, and the proposed method successfully mitigates this bias while significantly improving evaluation performance (correlation with human scores).

Conclusion: The proposed method effectively mitigates likelihood bias in LLM-based evaluators and enhances their evaluation performance across different tasks.

Abstract: Large Language Models (LLMs) are widely used to evaluate natural language generation tasks as automated metrics. However, the likelihood, a measure of LLM’s plausibility for a sentence, can vary due to superficial differences in sentences, such as word order and sentence structure. It is therefore possible that there might be a likelihood bias if LLMs are used for evaluation: they might overrate sentences with higher likelihoods while underrating those with lower likelihoods. In this paper, we investigate the presence and impact of likelihood bias in LLM-based evaluators. We also propose a method to mitigate the likelihood bias. Our method utilizes highly biased instances as few-shot examples for in-context learning. Our experiments in evaluating the data-to-text and grammatical error correction tasks reveal that several LLMs we test display a likelihood bias. Furthermore, our proposed method successfully mitigates this bias, also improving evaluation performance (in terms of correlation of models with human scores) significantly.

Method: Collected 13.5K questions from three sources: human-to-search-engine queries, human-to-human interactions, and human-to-LLM conversations. Developed iterative prompt improvement framework to identify causal queries.

Result: Found significant presence of causal questions (up to 42%) in the dataset. Analyzed their linguistic properties, cognitive complexity and source distribution across different domains and contexts.

Conclusion: The paper paves the way for future work on causal question identification and open-ended chatbot interactions, providing a comprehensive dataset for understanding human curiosity.

Abstract: Recent progress in Large Language Model (LLM) technology has changed our role in interacting with these models. Instead of primarily testing these models with questions we already know answers to, we are now using them for queries where the answers are unknown to us, driven by human curiosity. This shift highlights the growing need to understand curiosity-driven human questions - those that are more complex, open-ended, and reflective of real-world needs. To this end, we present Quriosity, a collection of 13.5K naturally occurring questions from three diverse sources: human-to-search-engine queries, human-to-human interactions, and human-to-LLM conversations. Our comprehensive collection enables a rich understanding of human curiosity across various domains and contexts. Our analysis reveals a significant presence of causal questions (up to 42%) in the dataset, for which we develop an iterative prompt improvement framework to identify all causal queries and examine their unique linguistic properties, cognitive complexity and source distribution. Our paper paves the way for future work on causal question identification and open-ended chatbot interactions. Our code and data are at https://github.com/roberto-ceraolo/quriosity.

Method: RAFG uses knowledge retrieval among existing features to identify associations, then employs LLMs for feature generation with reasoning to verify feature quality during the generation process.

Result: Experiments on medical, economic, and geographic datasets show RAFG produces high-quality, meaningful features and significantly improves classification performance compared to baseline methods.

Conclusion: RAFG successfully generates useful and explainable features for domain classification tasks without requiring extensive domain knowledge, demonstrating effectiveness across multiple domains.

Abstract: Feature generation can significantly enhance learning outcomes, particularly for tasks with limited data. An effective way to improve feature generation is to expand the current feature space using existing features and enriching the informational content. However, generating new, interpretable features usually requires domain-specific knowledge on top of the existing features. In this paper, we introduce a Retrieval-Augmented Feature Generation method, RAFG, to generate useful and explainable features specific to domain classification tasks. To increase the interpretability of the generated features, we conduct knowledge retrieval among the existing features in the domain to identify potential feature associations. These associations are expected to help generate useful features. Moreover, we develop a framework based on large language models (LLMs) for feature generation with reasoning to verify the quality of the features during their generation process. Experiments across several datasets in medical, economic, and geographic domains show that our RAFG method can produce high-quality, meaningful features and significantly improve classification performance compared with baseline methods.

Method: Uses federated learning with perturbed prompts and rationales generated through Chain-of-Thought approach, implementing two privacy protection strategies: Exponential Mechanism Strategy and Adaptive Exponential Mechanism Strategy.

Result: Empirical evaluation shows FedCoT effectively trains task-specific SLMs with enhanced performance while prioritizing data privacy protection across various text generation tasks.

Conclusion: FedCoT enables secure and efficient knowledge transfer from LLMs to SLMs in privacy-preserving settings, with code contributed to the FATE open-source project.

Abstract: Large Language Models (LLMs) have emerged as a transformative force in artificial intelligence, demonstrating exceptional proficiency across various tasks. However, their deployment in resource-constrained environments and concerns over user data privacy pose significant challenges. In contrast, Small Language Models (SLMs) offer computational efficiency but often lag in performance. To address these issues, we propose FedCoT, a federated framework designed for the Chain-of-Thought (CoT) distillation of knowledge from LLMs to SLMs, while ensuring the preservation of clients’ data privacy. FedCoT ensures secure and efficient knowledge transfer from an LLM on a high-powered server to an SLM on a resource-constrained client, while adhering to privacy requirements. Leveraging perturbed prompts and rationales generated through the CoT approach, the framework enhances the performance of the client’s SLM without compromising user data privacy within a multi-task learning framework. We propose two privacy protection strategies: the Exponential Mechanism Strategy and the Adaptive Exponential Mechanism Strategy, which balance user prompt privacy and the usability of rationales. Empirical evaluation on various text generation tasks demonstrates the effectiveness of FedCoT in training task-specific SLMs with enhanced performance while prioritizing data privacy protection. Our code has been contributed to the FATE open-source project and is now publicly accessible at \textit{https://github.com/FederatedAI/FATE-LLM/tree/main/python/fate_llm/algo/fedcot}

Method: The study conducts an extensive review and categorization of jailbreaking methods into seven distinct types, while also examining corresponding defense strategies against these vulnerabilities.

Result: The survey identifies specific research gaps in AI security and provides a comprehensive framework for understanding both jailbreak techniques and defensive solutions for LLMs and VLMs.

Conclusion: A unified perspective integrating jailbreak strategies and defensive solutions is necessary to create robust, secure, and reliable environments for next-generation language models.

Abstract: The rapid evolution of artificial intelligence (AI) through developments in Large Language Models (LLMs) and Vision-Language Models (VLMs) has brought significant advancements across various technological domains. While these models enhance capabilities in natural language processing and visual interactive tasks, their growing adoption raises critical concerns regarding security and ethical alignment. This survey provides an extensive review of the emerging field of jailbreaking–deliberately circumventing the ethical and operational boundaries of LLMs and VLMs–and the consequent development of defense mechanisms. Our study categorizes jailbreaks into seven distinct types and elaborates on defense strategies that address these vulnerabilities. Through this comprehensive examination, we identify research gaps and propose directions for future studies to enhance the security frameworks of LLMs and VLMs. Our findings underscore the necessity for a unified perspective that integrates both jailbreak strategies and defensive solutions to foster a robust, secure, and reliable environment for the next generation of language models. More details can be found on our website: https://chonghan-chen.com/llm-jailbreak-zoo-survey/.

Method: Sentence space method to encode text into discrete digital signals, then using convolutional deep networks (originally for image classification) for fake news recognition on text data.

Result: Proposed approach was evaluated on real-life Fakeddit dataset and compared with state-of-the-art data stream classification algorithms based on generalization ability and time complexity.

Conclusion: Demonstrates that deep learning methods can be effectively applied to data stream classification tasks, specifically for fake news detection from text data streams.

Abstract: Tabular data is considered the last unconquered castle of deep learning, yet the task of data stream classification is stated to be an equally important and demanding research area. Due to the temporal constraints, it is assumed that deep learning methods are not the optimal solution for application in this field. However, excluding the entire – and prevalent – group of methods seems rather rash given the progress that has been made in recent years in its development. For this reason, the following paper is the first to present an approach to natural language data stream classification using the sentence space method, which allows for encoding text into the form of a discrete digital signal. This allows the use of convolutional deep networks dedicated to image classification to solve the task of recognizing fake news based on text data. Based on the real-life Fakeddit dataset, the proposed approach was compared with state-of-the-art algorithms for data stream classification based on generalization ability and time complexity.

Method: Created BLADE benchmark with 12 datasets from scientific literature, collected ground truth from expert analyses, and developed computational methods to match agent responses to expert ground truth.

Result: Language models show limited analytical capabilities, performing only basic analyses, while data-interacting agents demonstrate improved but still suboptimal diversity in analytical decision-making.

Conclusion: BLADE enables systematic evaluation of agents for data-driven science and provides insights into their analysis approaches, highlighting current limitations and areas for improvement.

Abstract: Data-driven scientific discovery requires the iterative integration of scientific domain knowledge, statistical expertise, and an understanding of data semantics to make nuanced analytical decisions, e.g., about which variables, transformations, and statistical models to consider. LM-based agents equipped with planning, memory, and code execution capabilities have the potential to support data-driven science. However, evaluating agents on such open-ended tasks is challenging due to multiple valid approaches, partially correct steps, and different ways to express the same decisions. To address these challenges, we present BLADE, a benchmark to automatically evaluate agents’ multifaceted approaches to open-ended research questions. BLADE consists of 12 datasets and research questions drawn from existing scientific literature, with ground truth collected from independent analyses by expert data scientists and researchers. To automatically evaluate agent responses, we developed corresponding computational methods to match different representations of analyses to this ground truth. Though language models possess considerable world knowledge, our evaluation shows that they are often limited to basic analyses. However, agents capable of interacting with the underlying data demonstrate improved, but still non-optimal, diversity in their analytical decision making. Our work enables the evaluation of agents for data-driven science and provides researchers deeper insights into agents’ analysis approaches.

Method: Mufu introduces automatically generated multilingual candidates and correction instructions in prompts, transforming translation tasks into post-editing tasks where LLMs assess input quality, align semantics, copy relevant content, and override incorrect instances.

Result: Experiments on Flores-200 dataset show Mufu outperforms NLLB 1.3B distilled model in 64% of low-resource language pairs, and distilled models maintain 3.1 chrF improvement over baseline.

Conclusion: Mufu effectively addresses low-resource translation challenges by leveraging LLMs’ reasoning capabilities through post-editing prompts and auxiliary candidates, achieving significant improvements while maintaining efficiency through distillation.

Abstract: Multilingual large language models (LLMs) are great translators, but this is largely limited to high-resource languages. For many LLMs, translating in and out of low-resource languages remains a challenging task. To maximize data efficiency in this low-resource setting, we introduce Mufu, which includes a selection of automatically generated multilingual candidates and an instruction to correct inaccurate translations in the prompt. Mufu prompts turn a translation task into a postediting one, and seek to harness the LLM’s reasoning capability with auxiliary translation candidates, from which the model is required to assess the input quality, align the semantics cross-lingually, copy from relevant inputs and override instances that are incorrect. Our experiments on En-XX translations over the Flores-200 dataset show LLMs finetuned against Mufu-style prompts are robust to poor quality auxiliary translation candidates, achieving performance superior to NLLB 1.3B distilled model in 64% of low- and very-low-resource language pairs. We then distill these models to reduce inference cost, while maintaining on average 3.1 chrF improvement over finetune-only baseline in low-resource translations.

Method: A three-step framework: 1) complete linear decomposition of transformer models into disentangled computation graphs, 2) pruning, and 3) post-pruning causal mediation using counterfactuals and interventions to extract skill paths from circuit graphs.

Result: The framework successfully extracts skill paths for three generic language skills (Previous Token Skill, Induction Skill, and In-Context Learning Skill) and demonstrates two key properties: stratification and inclusiveness.

Conclusion: Skill paths provide a more refined and compact representation than circuit graphs for isolating individual skills in language models, enabling better mechanistic understanding through linear chains of components with preserved causal dependencies.

Abstract: Circuit graph discovery has emerged as a fundamental approach to elucidating the skill mechanistic of language models. Despite the output faithfulness of circuit graphs, they suffer from atomic ablation, which causes the loss of causal dependencies between connected components. In addition, their discovery process, designed to preserve output faithfulness, inadvertently captures extraneous effects other than an isolated target skill. To alleviate these challenges, we introduce skill paths, which offers a more refined and compact representation by isolating individual skills within a linear chain of components. To enable skill path extracting from circuit graphs, we propose a three-step framework, consisting of decomposition, pruning, and post-pruning causal mediation. In particular, we offer a complete linear decomposition of the transformer model which leads to a disentangled computation graph. After pruning, we further adopt causal analysis techniques, including counterfactuals and interventions, to extract the final skill paths from the circuit graph. To underscore the significance of skill paths, we investigate three generic language skills-Previous Token Skill, Induction Skill, and In-Context Learning Skill-using our framework. Experiments support two crucial properties of these skills, namely stratification and inclusiveness.

Method: Proposes Auto-PRE framework that automatically selects evaluator LLMs based on three core traits: consistency (instruction stage), pertinence (content stage), and self-confidence (response stage).

Result: Experiments on summarization, non-factoid QA, and dialogue generation tasks demonstrate state-of-the-art performance with significantly reduced evaluation costs.

Conclusion: The structured and scalable design provides insights for automating LLM-as-judge evaluation, paving the way for more advanced LLM-based evaluation frameworks.

Abstract: The rapid development of large language models (LLMs) has highlighted the need for efficient and reliable methods to evaluate their performance. Traditional evaluation methods often face challenges like high costs, limited task formats, dependence on human references, and systematic biases. To address these limitations, we propose Auto-PRE, an automatic LLM evaluation framework inspired by the peer review process. Unlike previous approaches that rely on human annotations, Auto-PRE automatically selects evaluator LLMs based on three core traits: consistency, pertinence, and self-confidence, which correspond to the instruction, content, and response stages, respectively, and collectively cover the entire evaluation process. Experiments on three representative tasks, including summarization, non-factoid QA, and dialogue generation, demonstrate that Auto-PRE achieves state-of-the-art performance while significantly reducing evaluation costs. Furthermore, the structured and scalable design of our automatic qualification exam framework provides valuable insights into automating the evaluation of LLMs-as-judges, paving the way for more advanced LLM-based evaluation frameworks.

Method: Proposed a novel heuristic to approximate pre-training distribution of entities when pre-training data is unknown, and systematically analyzed entity-typing approaches relying solely on PLMs versus knowledge-infused approaches.

Result: Entity-typing approaches using only PLMs’ parametric knowledge perform poorly for long-tail entities, while knowledge-infused approaches can mitigate some of these shortcomings.

Conclusion: We need to go beyond PLMs and incorporate external knowledge to effectively handle infrequent entities in ultra-fine entity typing tasks.

Abstract: Due to their capacity to acquire world knowledge from large corpora, pre-trained language models (PLMs) are extensively used in ultra-fine entity typing tasks where the space of labels is extremely large. In this work, we explore the limitations of the knowledge acquired by PLMs by proposing a novel heuristic to approximate the pre-training distribution of entities when the pre-training data is unknown. Then, we systematically demonstrate that entity-typing approaches that rely solely on the parametric knowledge of PLMs struggle significantly with entities at the long tail of the pre-training distribution, and that knowledge-infused approaches can account for some of these shortcomings. Our findings suggest that we need to go beyond PLMs to produce solutions that perform well for infrequent entities.

Method: Conducted experiments across machine translation, named entity recognition, and punctuation restoration tasks using various model sizes, architectures, and domain-specific datasets to evaluate cross-lingual transfer effectiveness.

Result: Minimal impact of Classical Chinese datasets on Hanja performance, with benefits diminishing rapidly as local language data increases, and substantial improvements only in extremely low-resource scenarios for both Korean and Japanese documents.

Conclusion: Empirical validation is crucial rather than assuming benefits from indiscriminate cross-lingual transfer, as Classical Chinese resources provide limited help for processing Korean and Japanese historical documents.

Abstract: Historical documents in the Sinosphere are known to share common formats and practices, particularly in veritable records compiled by court historians. This shared linguistic heritage has led researchers to use Classical Chinese resources for cross-lingual transfer when processing historical documents from Korea and Japan, which remain relatively low-resource. In this paper, we question the assumption of cross-lingual transferability from Classical Chinese to Hanja and Kanbun, the ancient written languages of Korea and Japan, respectively. Our experiments across machine translation, named entity recognition, and punctuation restoration tasks show minimal impact of Classical Chinese datasets on language model performance for ancient Korean documents written in Hanja, with performance differences within $\pm{}0.0068$ F1-score for sequence labeling tasks and up to $+0.84$ BLEU score for translation. These limitations persist consistently across various model sizes, architectures, and domain-specific datasets. Our analysis reveals that the benefits of Classical Chinese resources diminish rapidly as local language data increases for Hanja, while showing substantial improvements only in extremely low-resource scenarios for both Korean and Japanese historical documents. These findings emphasize the need for careful empirical validation rather than assuming benefits from indiscriminate cross-lingual transfer.

Method: Proposed Document Alignment Coefficient (DAC) aligns documents by matching smaller chunks and computes similarity as the ratio of aligned chunks to the average number of chunks in a pair, avoiding pooling-based representations.

Result: Intrinsic evaluation shows DAC is 2-3x faster while maintaining competitive performance. Extrinsic evaluation demonstrates that document-level MT models trained on DAC-aligned pairs consistently outperform baseline alignment methods.

Conclusion: DAC is an effective method for parallel document mining, providing substantial gains over existing approaches while being computationally efficient. The Pralekha dataset supports further research in this area.

Abstract: Mining parallel document pairs for document-level machine translation (MT) remains challenging due to the limitations of existing Cross-Lingual Document Alignment (CLDA) techniques. Existing methods often rely on metadata such as URLs, which are scarce, or on pooled document representations that fail to capture fine-grained alignment cues. Moreover, the limited context window of sentence embedding models hinders their ability to represent document-level context, while sentence-based alignment introduces a combinatorially large search space, leading to high computational cost. To address these challenges for Indic languages, we introduce Pralekha, a benchmark containing over 3 million aligned document pairs across 11 Indic languages and English, which includes 1.5 million English-Indic pairs. Furthermore, we propose Document Alignment Coefficient (DAC), a novel metric for fine-grained document alignment. Unlike pooling-based methods, DAC aligns documents by matching smaller chunks and computes similarity as the ratio of aligned chunks to the average number of chunks in a pair. Intrinsic evaluation shows that our chunk-based method is 2-3x faster while maintaining competitive performance, and that DAC achieves substantial gains over pooling-based baselines. Extrinsic evaluation further demonstrates that document-level MT models trained on DAC-aligned pairs consistently outperform those using baseline alignment methods. These results highlight DAC’s effectiveness for parallel document mining. The dataset and evaluation framework are publicly available to support further research.

Method: Uses GPT as teacher model to automatically annotate 20,000 articles in 4 languages into 17 IPTC categories, then fine-tunes smaller BERT-like student models on this dataset.

Result: Teacher model shows high zero-shot performance comparable to human annotators. Student models achieve similar performance with much smaller size, demonstrate strong zero-shot cross-lingual abilities, and work well with limited training data.

Conclusion: The framework successfully creates efficient multilingual news classifiers without manual annotation, with student models performing comparably to the teacher while being computationally lighter, enabling practical deployment.

Abstract: With the ever-increasing number of news stories available online, classifying them by topic, regardless of the language they are written in, has become crucial for enhancing readers’ access to relevant content. To address this challenge, we propose a teacher-student framework based on large language models (LLMs) for developing multilingual news topic classification models of reasonable size with no need for manual data annotation. The framework employs a Generative Pretrained Transformer (GPT) model as the teacher model to develop a news topic training dataset through automatic annotation of 20,000 news articles in Slovenian, Croatian, Greek, and Catalan. Articles are classified into 17 main categories from the Media Topic schema, developed by the International Press Telecommunications Council (IPTC). The teacher model exhibits high zero-shot performance in all four languages. Its agreement with human annotators is comparable to that between the human annotators themselves. To mitigate the computational limitations associated with the requirement of processing millions of texts daily, smaller BERT-like student models are fine-tuned on the GPT-annotated dataset. These student models achieve high performance comparable to the teacher model. Furthermore, we explore the impact of the training data size on the performance of the student models and investigate their monolingual, multilingual, and zero-shot cross-lingual capabilities. The findings indicate that student models can achieve high performance with a relatively small number of training instances, and demonstrate strong zero-shot cross-lingual abilities. Finally, we publish the best-performing news topic classifier, enabling multilingual classification with the top-level categories of the IPTC Media Topic schema.

Method: Developed RareAgents with MDT coordination, memory mechanisms, and medical tools utilization using Llama-3.1-8B/70B as base model. Created MIMIC-IV-Ext-Rare dataset for rare disease research.

Result: RareAgents outperforms state-of-the-art domain-specific models, GPT-4o, and current agent frameworks in rare disease diagnosis and treatment tasks.

Conclusion: RareAgents effectively bridges the gap in rare disease clinical support and contributes a valuable dataset to advance research in this challenging medical domain.

Abstract: Rare diseases, despite their low individual incidence, collectively impact around 300 million people worldwide due to the vast number of diseases. The involvement of multiple organs and systems, and the shortage of specialized doctors with relevant experience, make diagnosing and treating rare diseases more challenging than common diseases. Recently, agents powered by large language models (LLMs) have demonstrated notable applications across various domains. In the medical field, some agent methods have outperformed direct prompts in question-answering tasks from medical examinations. However, current agent frameworks are not well-adapted to real-world clinical scenarios, especially those involving the complex demands of rare diseases. To bridge this gap, we introduce RareAgents, the first LLM-driven multi-disciplinary team decision-support tool designed specifically for the complex clinical context of rare diseases. RareAgents integrates advanced Multidisciplinary Team (MDT) coordination, memory mechanisms, and medical tools utilization, leveraging Llama-3.1-8B/70B as the base model. Experimental results show that RareAgents outperforms state-of-the-art domain-specific models, GPT-4o, and current agent frameworks in diagnosis and treatment for rare diseases. Furthermore, we contribute a novel rare disease dataset, MIMIC-IV-Ext-Rare, to facilitate further research in this field.

Method: Analyzing representations formed by LLMs during in-context concept inference tasks, examining how models derive concepts from linguistic descriptions in relation to contextual cues.

Result: LLMs can flexibly derive concepts, their representations converge to shared context-independent structures, and alignment with this structure predicts performance across understanding/reasoning tasks. The representations capture human behavioral judgments and align with human neural activity patterns.

Conclusion: Structured, human-like conceptual representations emerge purely from language prediction without real-world grounding, highlighting conceptual structure’s role in intelligent behavior and suggesting LLMs offer insights into human concepts.

Abstract: People acquire concepts through rich physical and social experiences and use them to understand and navigate the world. In contrast, large language models (LLMs), trained solely through next-token prediction on text, exhibit strikingly human-like behaviors. Are these models developing concepts akin to those of humans? If so, how are such concepts represented, organized, and related to behavior? Here, we address these questions by investigating the representations formed by LLMs during an in-context concept inference task. We found that LLMs can flexibly derive concepts from linguistic descriptions in relation to contextual cues about other concepts. The derived representations converge toward a shared, context-independent structure, and alignment with this structure reliably predicts model performance across various understanding and reasoning tasks. Moreover, the convergent representations effectively capture human behavioral judgments and closely align with neural activity patterns in the human brain, providing evidence for biological plausibility. Together, these findings establish that structured, human-like conceptual representations can emerge purely from language prediction without real-world grounding, highlighting the role of conceptual structure in understanding intelligent behavior. More broadly, our work suggests that LLMs offer a tangible window into the nature of human concepts and lays the groundwork for advancing alignment between artificial and human intelligence.

Method: Proposes computing task vectors as weighted sums of attention heads, with weights optimized via gradient descent, and introduces a benchmark for evaluating task fidelity in functional regression tasks.

Result: The method successfully extracts task-specific information from in-context demonstrations and performs well in both text and regression tasks, showing cross-modal generalizability.

Conclusion: The proposed approach effectively encodes task information in ICL and demonstrates superior generalization capabilities across different modalities compared to existing methods.

Abstract: Large language models (LLMs) have demonstrated remarkable proficiency in in-context learning (ICL), where models adapt to new tasks through example-based prompts without requiring parameter updates. However, understanding how tasks are internally encoded and generalized remains a challenge. To address some of the empirical and technical gaps in the literature, we introduce an automated formulation for encoding task information in ICL prompts as a function of attention heads within the transformer architecture. This approach computes a single task vector as a weighted sum of attention heads, with the weights optimized causally via gradient descent. Our findings show that existing methods fail to generalize effectively to modalities beyond text. In response, we also design a benchmark to evaluate whether a task vector can preserve task fidelity in functional regression tasks. The proposed method successfully extracts task-specific information from in-context demonstrations and excels in both text and regression tasks, demonstrating its generalizability across modalities.

Method: Developed three methodological advances: multi-turn evaluation of 14 anthropomorphic behaviors, scalable automated simulations of user interactions, and large-scale human subject study (N=1101) for validation.

Result: All state-of-the-art LLMs exhibited similar anthropomorphic behaviors characterized by relationship-building (empathy, validation) and first-person pronoun use, with most behaviors emerging only after multiple turns.

Conclusion: The work establishes an empirical foundation for studying how design choices influence anthropomorphic behaviors and advances ethical discussions about their desirability, demonstrating the necessity of multi-turn evaluations for complex social phenomena in human-AI interaction.

Abstract: The tendency of users to anthropomorphise large language models (LLMs) is of growing interest to AI developers, researchers, and policy-makers. Here, we present a novel method for empirically evaluating anthropomorphic LLM behaviours in realistic and varied settings. Going beyond single-turn static benchmarks, we contribute three methodological advances in state-of-the-art (SOTA) LLM evaluation. First, we develop a multi-turn evaluation of 14 anthropomorphic behaviours. Second, we present a scalable, automated approach by employing simulations of user interactions. Third, we conduct an interactive, large-scale human subject study (N=1101) to validate that the model behaviours we measure predict real users’ anthropomorphic perceptions. We find that all SOTA LLMs evaluated exhibit similar behaviours, characterised by relationship-building (e.g., empathy and validation) and first-person pronoun use, and that the majority of behaviours only first occur after multiple turns. Our work lays an empirical foundation for investigating how design choices influence anthropomorphic model behaviours and for progressing the ethical debate on the desirability of these behaviours. It also showcases the necessity of multi-turn evaluations for complex social phenomena in human-AI interaction.

Method: Leverage Metamorphic Relations to create semantically equivalent but textually divergent versions of inputs, then compute Perceived Confidence Score (PCS) based on label prediction consistency across variations.

Result: PCS improves zero-shot LLM performance by 9.3% in text classification tasks, and provides 5.8% performance boost in majority-voting setups with multiple LLMs.

Conclusion: The PCS approach effectively enhances LLM performance in text classification by evaluating confidence through metamorphic testing principles.

Abstract: Zero-shot LLMs are now also used for textual classification tasks, e.g., sentiment and bias detection in a sentence or article. However, their performance can be suboptimal in such data annotation tasks. We introduce a novel technique that evaluates an LLM’s confidence for classifying a textual input by leveraging Metamorphic Relations (MRs). The MRs generate semantically equivalent yet textually divergent versions of the input. Following the principles of Metamorphic Testing (MT), the mutated versions are expected to have annotation labels similar to the input. By analyzing the consistency of an LLM’s responses across these variations, we compute a perceived confidence score (PCS) based on the frequency of the predicted labels. PCS can be used for both single and multiple LLM settings (e.g., when multiple LLMs are vetted in a majority-voting setup). Empirical evaluation shows that our PCS-based approach improves the performance of zero-shot LLMs by 9.3% in textual classification tasks. When multiple LLMs are used in a majority-voting setup, we obtain a performance boost of 5.8% with PCS.

Method: Uses server-client federated architecture where clients send DP-perturbed data to server LLM, which generates synthetic data with rationales for LLM pruning and retraining.

Result: Achieves competitive performance comparable to full-sized LLMs while ensuring robust privacy protection across diverse text generation tasks.

Conclusion: PPC-GPT successfully integrates privacy preservation and model compression in a unified framework, with code contributed to FATE open-source project.

Abstract: Compressing Large Language Models (LLMs) into task-specific Small Language Models (SLMs) encounters two significant challenges: safeguarding domain-specific knowledge privacy and managing limited resources. To tackle these challenges, we propose PPC-GPT, a novel unified framework that systematically addresses both privacy preservation and model compression in federated settings. PPC-GPT works on a server-client federated architecture, where the client sends differentially private (DP) perturbed task-specific data to the server’s LLM. The LLM then generates synthetic data along with their corresponding rationales. This synthetic data is subsequently used for both LLM pruning and retraining processes. Our framework’s key innovation lies in its holistic integration of privacy-preserving mechanisms, synthetic data generation, and task-specific compression techniques, creating unique benefits through component interaction. Our experiments across diverse text generation tasks demonstrate that PPC-GPT successfully achieves dual objectives: maintaining competitive performance comparable to full-sized LLMs while ensuring robust privacy protection through its federated architecture. Our code has been contributed to the FATE open-source project and is now publicly accessible at \textit{https://github.com/FederatedAI/FATE-LLM/tree/main/python/fate_llm/algo/ppc-gpt}

Method: Precompute KV cache for each document independently, then concatenate during inference. Uses two key techniques: adjusting positional embeddings to match global positions after concatenation, and trainable special tokens to restore self-attention across independently encoded documents.

Result: Improves question answering accuracy by average 4% over SOTA methods across 7 datasets, reduces time-to-first-token by up to 96% compared to standard LLM inference, and can be combined with KV cache compression for further efficiency gains.

Conclusion: KVLink provides a scalable and efficient solution for context reuse in LLMs, significantly reducing computational redundancy while maintaining or improving performance.

Abstract: We describe KVLink, an approach for efficient key-value (KV) cache reuse in large language models (LLMs). In many LLM applications, different inputs can share overlapping context, such as the same retrieved document appearing in multiple queries. However, the LLMs still need to encode the entire context for each query, leading to redundant computation. In this paper, we investigate a new strategy to eliminate such inefficiency, where the KV cache of each document is precomputed independently. During inference, the KV caches of retrieved documents are concatenated, allowing the model to reuse cached representations instead of recomputing them. To mitigate the performance degradation when using KV caches computed independently for each document, KVLink introduces two key techniques: adjusting positional embeddings of the KV cache at inference to match the global position after concatenation, and using trainable special tokens to restore self-attention across independently encoded documents. Experiments across 7 datasets demonstrate that KVLink improves question answering accuracy by an average of 4% over state-of-the-art methods. Furthermore, by leveraging precomputed KV caches, our approach reduces time-to-first-token by up to 96% compared to standard LLM inference, making it a scalable and efficient solution for context reuse. Additionally, KVLink can be combined with KV cache compression to further save cache loading and storage overhead while outperforming the baselines.

Method: Randomly shuffle independent premises for condition order augmentation, and construct a DAG to model dependencies between reasoning steps to identify valid reorderings while preserving logical correctness.

Result: Extensive experiments across multiple logical reasoning benchmarks show the method significantly enhances LLMs’ reasoning performance and adaptability to diverse logical structures.

Conclusion: Order-centric data augmentation enables LLMs to develop more flexible and generalized reasoning processes, improving their logical reasoning capabilities.

Abstract: Logical reasoning is essential for large language models (LLMs) to ensure accurate and coherent inference. However, LLMs struggle with reasoning order variations and fail to generalize across logically equivalent transformations. LLMs often rely on fixed sequential patterns rather than true logical understanding. To address this issue, we introduce an order-centric data augmentation framework based on commutativity in logical reasoning. We first randomly shuffle independent premises to introduce condition order augmentation. For reasoning steps, we construct a directed acyclic graph (DAG) to model dependencies between steps, which allows us to identify valid reorderings of steps while preserving logical correctness. By leveraging order-centric augmentations, models can develop a more flexible and generalized reasoning process. Finally, we conduct extensive experiments across multiple logical reasoning benchmarks, demonstrating that our method significantly enhances LLMs’ reasoning performance and adaptability to diverse logical structures. We release our codes and augmented data in https://github.com/qianxiHe147/Order-Centric-Data-Augmentation.

Method: Created ECLeKTic dataset using Wikipedia article presence/absence in 12 languages to identify information available during pre-training in one language but not others, then curated fact-seeking questions about this information in all languages.

Result: Evaluation of 8 LLMs showed that current SOTA models struggle to effectively share knowledge across languages, even when they can answer questions in the language where the knowledge was originally acquired.

Conclusion: There is a significant gap in LLMs’ ability to transfer knowledge between languages, highlighting the need for improved cross-lingual knowledge sharing capabilities in multilingual models.

Abstract: To achieve equitable performance across languages, large language models (LLMs) must be able to abstract knowledge beyond the language in which it was learnt. However, the current literature lacks reliable ways to measure LLMs’ capability of such cross-lingual knowledge transfer. To that end, we present ECLeKTic, a multilingual closed-book QA dataset that Evaluates Cross-Lingual Knowledge Transfer in a simple, black-box manner. Concretely, we used the presence and absence of Wikipedia articles in 12 languages to detect pieces of information that were likely available during pre-training in one of the languages but not in the others. We curate ECLeKTic as a set of fact-seeking questions over this kind of information, in all the different languages. Therefore, in order to solve ECLeKTic the model is required to transfer knowledge between languages. We evaluated 8 LLMs and showed that current SOTA models struggle to effectively share knowledge across languages, even if they can predict the answer for questions in the language in which the knowledge was acquired.

Method: Evaluate representation alignment via human judgment by expanding keywords and emotion representations (Words, VAD dimensions in Lexical/Numeric forms, Emojis) into full sentences.

Result: People agree more with LLM generation when conditioned on English words than VAD scales, especially numeric VAD. Emotion perception depends on both representation type and specific emotion.

Conclusion: Word-based representations outperform VAD scales for LLM emotion generation in AAC contexts, with representation effectiveness varying by emotion.

Abstract: Gaps arise between a language model’s use of concepts and people’s expectations. This gap is critical when LLMs generate text to help people communicate via Augmentative and Alternative Communication (AAC) tools. In this work, we introduce the evaluation task of Representation Alignment for measuring this gap via human judgment. In our study, we expand keywords and emotion representations into full sentences. We select four emotion representations: Words, Valence-Arousal-Dominance (VAD) dimensions expressed in both Lexical and Numeric forms, and Emojis. In addition to Representation Alignment, we also measure people’s judgments of the accuracy and realism of the generated sentences. While representations like VAD break emotions into easy-to-compute components, our findings show that people agree more with how LLMs generate when conditioned on English words (e.g., “angry”) rather than VAD scales. This difference is especially visible when comparing Numeric VAD to words. Furthermore, we found that the perception of how much a generated sentence conveys an emotion is dependent on both the representation type and which emotion it is.

Method: Systematic evaluation of 12 LLMs across 17 bilingual legal tasks, creation of a bilingual chain-of-thought dataset through distillation from DeepSeek-R1, and development of Legal-R1 as a specialized legal reasoning model.

Result: Legal-R1 achieves competitive performance across diverse tasks; DeepSeek-R1 excels in Chinese legal reasoning while OpenAI’s o1 performs comparably on English tasks; error analysis reveals issues with outdated knowledge, limited legal interpretation, and factual hallucinations.

Conclusion: The study identifies key obstacles for legal-domain LLMs and provides promising directions for future research in legal reasoning with extended chain-of-thought approaches.

Abstract: Recent advances in test-time scaling of large language models (LLMs), exemplified by DeepSeek-R1 and OpenAI’s o1, show that extending the chain of thought during inference can significantly improve general reasoning performance. However, the impact of this paradigm on legal reasoning remains insufficiently explored. To address this gap, we present the first systematic evaluation of 12 LLMs, including both reasoning-focused and general-purpose models, across 17 Chinese and English legal tasks spanning statutory and case-law traditions. In addition, we curate a bilingual chain-of-thought dataset for legal reasoning through distillation from DeepSeek-R1 and develop Legal-R1, an open-source model specialized for the legal domain. Experimental results show that Legal-R1 delivers competitive performance across diverse tasks. DeepSeek-R1 exhibits clear advantages in Chinese legal reasoning, while OpenAI’s o1 achieves comparable results on English tasks. We further conduct a detailed error analysis, which reveals recurring issues such as outdated legal knowledge, limited capacity for legal interpretation, and susceptibility to factual hallucinations. These findings delineate the main obstacles confronting legal-domain LLMs and suggest promising directions for future research.

Method: Extensive assessment of four LLMs across three QA datasets covering 48 languages, evaluating their ability to use relevant passages regardless of language, respond in expected language, and focus on relevant information amidst distracting passages using accuracy and feature attribution techniques.

Result: LLMs show surprising ability to extract relevant information from passages in different languages than the query, but weak ability to formulate full answers in the correct language. Distracting passages negatively impact answer quality regardless of language, with distractors in query language having slightly stronger influence.

Conclusion: Findings deepen understanding of LLM context utilization in multilingual RAG systems and provide directions for future improvements in handling multilingual contexts and language consistency.

Abstract: Retrieval-augmented generation (RAG) with large language models (LLMs) has demonstrated strong performance in multilingual question-answering (QA) tasks by leveraging relevant passages retrieved from corpora. In multilingual RAG (mRAG), the retrieved passages can be written in languages other than that of the query entered by the user, making it challenging for LLMs to effectively utilize the provided information. Recent research suggests that retrieving passages from multilingual corpora can improve RAG performance, particularly for low-resource languages. However, the extent to which LLMs can leverage different kinds of multilingual contexts to generate accurate answers, independently from retrieval quality, remains understudied. In this paper, we conduct an extensive assessment of LLMs’ ability to (i) make consistent use of a relevant passage regardless of its language, (ii) respond in the expected language, and (iii) focus on the relevant passage even when multiple `distracting’ passages in different languages are provided in the context. Our experiments with four LLMs across three QA datasets covering a total of 48 languages reveal a surprising ability of LLMs to extract the relevant information from passages in a different language than the query, but a much weaker ability to formulate a full answer in the correct language. Our analysis, based on both accuracy and feature attribution techniques, further shows that distracting passages negatively impact answer quality regardless of their language. However, distractors in the query language exert a slightly stronger influence. Taken together, our findings deepen the understanding of how LLMs utilize context in mRAG systems, providing directions for future improvements.

Method: Mechanistic comparison of base and post-trained LLMs across four perspectives: knowledge storage locations, truthfulness/refusal vector representations, transferability of directions, and confidence attribution.

Result: Post-training adapts knowledge representations while keeping factual knowledge locations unchanged; truthfulness direction is similar and transferable between models; refusal direction differs with limited transferability; confidence differences not due to entropy neurons.

Conclusion: Post-training preserves fundamental mechanisms while altering others, providing insights for model steering and benefiting future interpretability and post-training research.

Abstract: Post-training is essential for the success of large language models (LLMs), transforming pre-trained base models into more useful and aligned post-trained models. While plenty of works have studied post-training algorithms and evaluated post-training models by their outputs, it remains understudied how post-training reshapes LLMs internally. In this paper, we compare base and post-trained LLMs mechanistically from four perspectives to better understand post-training effects. Our findings across model families and datasets reveal that: (1) Post-training does not change the factual knowledge storage locations, and it adapts knowledge representations from the base model while developing new knowledge representations; (2) Both truthfulness and refusal can be represented by vectors in the hidden representation space. The truthfulness direction is highly similar between the base and post-trained model, and it is effectively transferable for interventions; (3) The refusal direction is different between the base and post-trained models, and it shows limited forward transferability; (4) Differences in confidence between the base and post-trained models cannot be attributed to entropy neurons. Our study provides insights into the fundamental mechanisms preserved and altered during post-training, facilitates downstream tasks like model steering, and could potentially benefit future research in interpretability and LLM post-training. Our code is publicly available at https://github.com/HZD01/post-training-mechanistic-analysis.

Method: Uses AttenHScore to calculate hallucination accumulation and propagation during small LM generation, with dynamic threshold adjustment and uncertainty-aware knowledge reorganization to help small LMs capture critical information.

Result: Outperforms most baselines in real-time hallucination detection across multiple QA datasets, especially for complex queries, without requiring additional model training and adaptable to various transformer-based LMs.

Conclusion: AttenHScore provides an effective and flexible solution for real-time hallucination detection in collaborative LM systems, improving invocation timing while maintaining computational efficiency.

Abstract: The collaborative paradigm of large and small language models (LMs) effectively balances performance and cost, yet its pivotal challenge lies in precisely pinpointing the moment of invocation when hallucinations arise in small LMs. Previous optimization efforts primarily focused on post-processing techniques, which were separate from the reasoning process of LMs, resulting in high computational costs and limited effectiveness. In this paper, we propose a practical invocation evaluation metric called AttenHScore, which calculates the accumulation and propagation of hallucinations during the generation process of small LMs, continuously amplifying potential reasoning errors. By dynamically adjusting the detection threshold, we achieve more accurate real-time invocation of large LMs. Additionally, considering the limited reasoning capacity of small LMs, we leverage uncertainty-aware knowledge reorganization to assist them better capture critical information from different text chunks. Extensive experiments reveal that our AttenHScore outperforms most baselines in enhancing real-time hallucination detection capabilities across multiple QA datasets, especially when addressing complex queries. Moreover, our strategies eliminate the need for additional model training and display flexibility in adapting to various transformer-based LMs.

Method: Atomic Consistency Preference Optimization (ACPO) leverages atomic consistency signals - agreement of individual facts across multiple stochastic responses - to identify high- and low-quality data pairs for self-supervised model alignment.

Result: ACPO outperforms supervised alignment baseline by 1.95 points averaged across Phi-3 and Llama3 on LongFact and BioGen datasets, improving factual reliability without external models or knowledge bases.

Conclusion: ACPO provides an effective self-supervised approach for enhancing LLM factual accuracy that doesn’t rely on external supervision, demonstrating strong performance compared to supervised methods.

Abstract: Large Language Models (LLMs) often produce factoid hallucinations - plausible yet incorrect answers. A common mitigation strategy is model alignment, which improves factual accuracy by training on curated (factual, non-factual) pairs. However, this approach often relies on a stronger model (e.g., GPT-4) or an external knowledge base to assess factual correctness that may not always be accessible. Addressing this, we propose Atomic Consistency Preference Optimization (ACPO), a self-supervised preference-tuning method that enhances factual accuracy without external supervision. ACPO leverages atomic consistency signals (i.e., the agreement of individual facts across multiple stochastic responses) to identify high- and low-quality data pairs for model alignment. Despite being fully self-supervised, ACPO outperforms the strong supervised alignment baseline by 1.95 points averaged across Phi-3 and Llama3 on the LongFact and BioGen datasets, demonstrating its effectiveness in improving factual reliability without relying on external models or knowledge bases.

Method: Used fine-tuned models for text, audio, and video modalities along with a gated fusion model, analyzed cases of disagreement between unimodal and multimodal predictions, and examined annotator uncertainty.

Result: Found that disagreements often reflect underlying ambiguity, dominant signals in one modality can mislead fusion when unsupported by others, and humans don’t consistently benefit from multimodal input either.

Conclusion: Disagreement between unimodal and multimodal predictions serves as a useful diagnostic signal for identifying challenging examples and improving empathy system robustness.

Abstract: Multimodal models play a key role in empathy detection, but their performance can suffer when modalities provide conflicting cues. To understand these failures, we examine cases where unimodal and multimodal predictions diverge. Using fine-tuned models for text, audio, and video, along with a gated fusion model, we find that such disagreements often reflect underlying ambiguity, as evidenced by annotator uncertainty. Our analysis shows that dominant signals in one modality can mislead fusion when unsupported by others. We also observe that humans, like models, do not consistently benefit from multimodal input. These insights position disagreement as a useful diagnostic signal for identifying challenging examples and improving empathy system robustness.

Method: Three components: EvoSA (annotation-free data augmentation using evolutionary operations and speech act theory), teacher-model-generated multi-task supervision, and progressive task-level anti-curriculum distillation.

Result: Achieved 25.9% improvement in F1mac and 8.1% in accuracy over best baseline, outperforming GPT-4 and Claude-3.5-Sonnet. Created ReaMent dataset with 5,000 real-world dialogue samples.

Conclusion: MentalMAC effectively addresses challenges in mental manipulation detection and significantly enhances LLM capabilities in this domain.

Abstract: Mental manipulation is a subtle yet pervasive form of psychological abuse that poses serious threats to mental health. Nevertheless, detecting mental manipulation remains a largely underexplored research problem. The field faces three major challenges: (i) insufficient and hard-to-obtain training data; (ii) the covert nature of mental manipulation, which hinders detection; and (iii) the lack of real-world datasets. To address these challenges, we propose MentalMAC, a novel framework that enhances large language models’ ability to detect elements of mental manipulation in multi-turn dialogue. Our approach consists of three key components: EvoSA, an annotation-free data augmentation method based on evolutionary operations and speech act theory; teacher-model-generated multi-task supervision; and progressive task-level anti-curriculum distillation. We then constructed the ReaMent dataset, comprising 5,000 real-world dialogue samples, utilizing MentalMAC-distilled models to aid in human annotation. Vast experiments show that MentalMAC achieves up to 25.9% improvement in F1mac and 8.1% in accuracy over the best-performing baseline, outperforming commercial LLMs such as GPT-4 and Claude-3.5-Sonnet. Warning: This paper contains content that may be offensive to the reader.

Method: General temporal difference-based distillation framework that operates on reduced action spaces (vocabulary subsets) rather than full vocabulary, taking advantage of teacher model’s distributional sparsity where most probability mass is concentrated on few tokens.

Result: Demonstrated practical algorithms derived from the framework and resulting performance improvements in language model distillation.

Conclusion: The proposed temporal difference learning framework with reduced action spaces provides an effective approach for language model distillation by leveraging distributional sparsity patterns in teacher models.

Abstract: Large language models have led to significant progress across many NLP tasks, although their massive sizes often incur substantial computational costs. Distillation has become a common practice to compress these large and highly capable models into smaller, more efficient ones. Many existing language model distillation methods can be viewed as behavior cloning from the perspective of imitation learning or inverse reinforcement learning. This viewpoint has inspired subsequent studies that leverage (inverse) reinforcement learning techniques, including variations of behavior cloning and temporal difference learning methods. Rather than proposing yet another specific temporal difference method, we introduce a general framework for temporal difference-based distillation by exploiting the distributional sparsity of the teacher model. Specifically, it is often observed that language models assign most probability mass to a small subset of tokens. Motivated by this observation, we design a temporal difference learning framework that operates on a reduced action space (a subset of vocabulary), and demonstrate how practical algorithms can be derived and the resulting performance improvements.

Method: Reorganize existing datasets, introduce a biological entity-based evaluation framework, and propose a retrieval-enhanced method for protein-to-text generation that works in training-free scenarios.

Result: The retrieval-enhanced method significantly outperforms fine-tuned LLMs for protein-to-text generation and shows improved accuracy and efficiency.

Conclusion: The proposed approach addresses data leakage issues and provides a more accurate evaluation framework for protein-text models, with the retrieval method offering superior performance over traditional fine-tuning approaches.

Abstract: In recent years, protein-text models have gained significant attention for their potential in protein generation and understanding. Current approaches focus on integrating protein-related knowledge into large language models through continued pretraining and multi-modal alignment, enabling simultaneous comprehension of textual descriptions and protein sequences. Through a thorough analysis of existing model architectures and text-based protein understanding benchmarks, we identify significant data leakage issues present in current benchmarks. Moreover, conventional metrics derived from natural language processing fail to accurately assess the model’s performance in this domain. To address these limitations, we reorganize existing datasets and introduce a novel evaluation framework based on biological entities. Motivated by our observation, we propose a retrieval-enhanced method, which significantly outperforms fine-tuned LLMs for protein-to-text generation and shows accuracy and efficiency in training-free scenarios. Our code and data can be seen at https://github.com/IDEA-XL/RAPM.

Method: Developed WebPuzzle benchmark (24k training, 275 test samples) and DeepDiver RL framework that cultivates Search Intensity Scaling - an emergent ability to escalate search frequency and depth rather than settling on under-evidenced answers.

Result: With SIS, Qwen2.5-7B-Instruct and Pangu-7B-Reasoner achieved performance on real-web tasks comparable to the 671B-parameter DeepSeek-R1, and the seeking policy generalized from closed-ended queries to open-ended generation like long-form writing.

Conclusion: The approach advances adaptive information seeking in LLMs and provides a rigorous benchmark for future work, demonstrating that smaller models can achieve information seeking capabilities comparable to much larger models through proper training.

Abstract: Information seeking demands iterative evidence gathering and reflective reasoning, yet large language models (LLMs) still struggle with it in open-web question answering. Existing prompting and supervised fine-tuning (SFT) methods remain fixed by prompt rules or training corpora, and are usually benchmarked only on well-structured wiki sources, limiting real-world adaptability. We introduce WebPuzzle, a 24k-sample training and 275-sample test benchmark that evaluates information seeking on the live internet, across both wiki and open-domain queries. Leveraging 7k WebPuzzle instances, we develop DeepDiver, a reinforcement-learning (RL) framework that cultivates Search Intensity Scaling (SIS)-an emergent ability to escalate search frequency and depth instead of settling on overconfident, under-evidenced answers. With SIS, Qwen2.5-7B-Instruct and Pangu-7B-Reasoner attain performance on real-web tasks comparable to the 671B-parameter DeepSeek-R1. We detail DeepDiver’s curriculum from cold-start SFT to a well designed RL procedure, and show that its seeking policy generalized from closed-ended queries to open-ended generation such as long-form writing. Our results advance adaptive information seeking in LLMs and provide a rigorous benchmark for future work.

Method: Introduce Language-to-Thought (L2T) prompting that aligns the model’s internal thinking language with the source language of factual knowledge.

Result: Across three languages and four models, L2T consistently outperforms English-based reasoning, reversing the expected advantage of English prompts.

Conclusion: Aligning the model’s internal thinking language with knowledge source language is more effective than English-based reasoning for cross-lingual factual knowledge transfer.

Abstract: Multilingual large language models (LLMs) offer promising opportunities for cross-lingual information access, yet their use of factual knowledge remains highly sensitive to the input language. Prior work has addressed this through English prompting and evaluation, assuming that English-based reasoning is universally beneficial. In this work, we challenge that assumption by exploring factual knowledge transfer from non-English to English through the lens of Language and Thought Theory. We introduce Language-to-Thought (L2T) prompting, which aligns the model’s internal ‘’thinking’’ language with the source of knowledge. Across three languages and four models, L2T consistently outperforms English-based reasoning, reversing the expected advantage of English prompts. Our code is available at https://github.com/GeomeunByeol/Language2Thought.

Method: Developed a regression model to evaluate legal text quality, created specialized legal questions, and analyzed 49 LLMs using this framework.

Result: Found that model quality plateaus at 14B parameters (only 2.7% improvement at 72B), engineering choices have negligible impact, and reasoning models consistently outperform base architectures. Qwen3 series identified as optimal for cost-performance tradeoffs.

Conclusion: Establishes standardized evaluation protocols for legal LLMs and uncovers fundamental limitations in current training data refinement approaches.

Abstract: As large language models (LLMs) are increasingly used in legal applications, current evaluation benchmarks tend to focus mainly on factual accuracy while largely neglecting important linguistic quality aspects such as clarity, coherence, and terminology. To address this gap, we propose three steps: First, we develop a regression model to evaluate the quality of legal texts based on clarity, coherence, and terminology. Second, we create a specialized set of legal questions. Third, we analyze 49 LLMs using this evaluation framework. Our analysis identifies three key findings: First, model quality levels off at 14 billion parameters, with only a marginal improvement of $2.7%$ noted at 72 billion parameters. Second, engineering choices such as quantization and context length have a negligible impact, as indicated by statistical significance thresholds above 0.016. Third, reasoning models consistently outperform base architectures. A significant outcome of our research is the release of a ranking list and Pareto analysis, which highlight the Qwen3 series as the optimal choice for cost-performance tradeoffs. This work not only establishes standardized evaluation protocols for legal LLMs but also uncovers fundamental limitations in current training data refinement approaches. Code and models are available at: https://github.com/lyxx3rd/LegalEval-Q.

Method: Introduces OpenUnlearning framework with integrated unlearning algorithms, diverse evaluations, and meta-evaluation benchmark for assessing metric faithfulness and robustness.

Result: Provides comparative analysis of unlearning methods against extensive evaluation suite and releases 450+ checkpoints for forgetting behavior analysis.

Conclusion: Establishes community-driven pathway for rigorous LLM unlearning research through standardized benchmarking framework.

Abstract: Robust unlearning is crucial for safely deploying large language models (LLMs) in environments where data privacy, model safety, and regulatory compliance must be ensured. Yet the task is inherently challenging, partly due to difficulties in reliably measuring whether unlearning has truly occurred. Moreover, fragmentation in current methodologies and inconsistent evaluation metrics hinder comparative analysis and reproducibility. To unify and accelerate research efforts, we introduce OpenUnlearning, a standardized and extensible framework designed explicitly for benchmarking both LLM unlearning methods and metrics. OpenUnlearning integrates 13 unlearning algorithms and 16 diverse evaluations across 3 leading benchmarks (TOFU, MUSE, and WMDP) and also enables analyses of forgetting behaviors across 450+ checkpoints we publicly release. Leveraging OpenUnlearning, we propose a novel meta-evaluation benchmark focused specifically on assessing the faithfulness and robustness of evaluation metrics themselves. We also benchmark diverse unlearning methods and provide a comparative analysis against an extensive evaluation suite. Overall, we establish a clear, community-driven pathway toward rigorous development in LLM unlearning research.

Method: Augment the standard RL advantage function with an entropy-based term to encourage exploration by promoting longer and deeper reasoning chains, rather than just uncertainty.

Result: Significant gains on the Pass@K metric, even with extremely large K values, pushing the boundaries of LLM reasoning capabilities.

Conclusion: Entropy-based exploration in RL can effectively enhance LLM reasoning by promoting deeper reasoning chains, overcoming performance plateaus in existing methods.

Abstract: Balancing exploration and exploitation is a central goal in reinforcement learning (RL). Despite recent advances in enhancing large language model (LLM) reasoning, most methods lean toward exploitation, and increasingly encounter performance plateaus. In this work, we revisit entropy – a signal of exploration in RL – and examine its relationship to exploratory reasoning in LLMs. Through empirical analysis, we uncover positive correlations between high-entropy regions and three types of exploratory reasoning actions: (1) pivotal tokens that determine or connect logical steps, (2) reflective actions such as self-verification and correction, and (3) rare behaviors under-explored by the base LLMs. Motivated by this, we introduce a minimal modification to standard RL with only one line of code: augmenting the advantage function with an entropy-based term. Unlike traditional maximum-entropy methods which encourage exploration by promoting uncertainty, we encourage exploration by promoting longer and deeper reasoning chains. Notably, our method achieves significant gains on the Pass@K metric – an upper-bound estimator of LLM reasoning capabilities – even when evaluated with extremely large K values, pushing the boundaries of LLM reasoning.

Method: Evaluated 32 open-weight transformer models using a ‘mechanisms vs. outcomes’ framework, comparing syntactic features from probing with targeted syntax evaluations across English linguistic phenomena.

Result: Found that syntactic features extracted via probing fail to predict outcomes of targeted syntax evaluations, showing substantial disconnect between latent representations and observable syntactic behaviors.

Conclusion: There is a significant gap between what probing reveals about syntactic representations and how models actually perform on downstream syntactic tasks, challenging the reliability of probing as a predictor of syntactic competence.

Abstract: Large Language Models (LLMs) exhibit a robust mastery of syntax when processing and generating text. While this suggests internalized understanding of hierarchical syntax and dependency relations, the precise mechanism by which they represent syntactic structure is an open area within interpretability research. Probing provides one way to identify the mechanism of syntax being linearly encoded in activations, however, no comprehensive study has yet established whether a model’s probing accuracy reliably predicts its downstream syntactic performance. Adopting a “mechanisms vs. outcomes” framework, we evaluate 32 open-weight transformer models and find that syntactic features extracted via probing fail to predict outcomes of targeted syntax evaluations across English linguistic phenomena. Our results highlight a substantial disconnect between latent syntactic representations found via probing and observable syntactic behaviors in downstream tasks.

Method: Proposed ReCode framework with: 1) Dataset of ~2,000 entries for training LLMs on version migration, 2) Modified string similarity metric for code evaluation as RL reward, 3) Applied to various LLMs using GRPO and DAPO reinforcement learning algorithms.

Result: ReCode substantially boosts LLMs’ code generation performance in dynamic API scenarios, especially on unseen CodeUpdateArena tasks. Qwen2.5-Coder-7B outperformed 32B parameter models after training. Consistent improvements across various LLMs and RL algorithms.

Conclusion: ReCode effectively addresses LLMs’ adaptation to API changes while preserving general code generation capabilities, demonstrating practical value for dynamic programming environments.

Abstract: Large Language Models (LLMs) exhibit remarkable code generation capabilities but falter when adapting to frequent updates in external library APIs. This critical limitation, stemming from reliance on outdated API knowledge from their training data, even with access to current documentation, impedes reliable code generation in dynamic environments. To tackle this issue, we propose ReCode (rule-based Reinforcement learning for Code Update), a novel framework that mimics human programmer adaptation to API changes. Specifically, we construct a dataset of approximately 2,000 data entries to train the LLMs to perform version migration based on updated information. Then, we introduce a modified string similarity metric for code evaluation as the reward for reinforcement learning. Our experiments demonstrate that ReCode substantially boosts LLMs’ code generation performance in dynamic API scenarios, especially on the unseen CodeUpdateArena task. Crucially, compared to supervised fine-tuning, ReCode has less impact on LLMs’ general code generation abilities. We apply ReCode on various LLMs and reinforcement learning algorithms (GRPO and DAPO), all achieving consistent improvements. Notably, after training, Qwen2.5-Coder-7B outperforms that of the 32B parameter code instruction-tuned model and the reasoning model with the same architecture. Code is available at https://github.com/zjunlp/ReCode.

Method: Label sensitive tokens, infer LLM without sensitive tokens for baseline, infer with sensitive tokens, then blend distributions to bound influence on final output.

Result: Achieves 6x lower perplexity than related DPI methods while providing token-level provable privacy guarantees.

Conclusion: DP-Fusion enables document privatization with substantially improved theoretical and empirical privacy, effectively balancing privacy protection with text quality.

Abstract: Large language models (LLMs) do not preserve privacy at inference-time. The LLM’s outputs can inadvertently reveal information about the model’s context, which presents a privacy challenge when the LLM is augmented via tools or databases containing sensitive information. Existing privacy-preserving methods at inference-time have significant limitations since they (i) lack provable guarantees or (ii) have a poor utility/privacy trade-off. We propose DP-Fusion, a Differentially Private Inference (DPI) mechanism for LLMs that provably bounds the influence a set of tokens in the context can have on the LLM’s output. DP-Fusion works as follows: (1) label a subset of sensitive tokens, (2) infer the LLM without any sensitive tokens to obtain a baseline, (3) infer the LLM with the sensitive tokens, and (4) blend distributions so that the final output remains within a bounded distance of the baseline distribution. While this per-token influence bound also mitigates jailbreak-style prompt injection, we focus on \emph{document privatization}, where the goal is to paraphrase a document containing sensitive tokens, e.g., personally identifiable information, so that no attacker can reliably infer them from the paraphrased document while preserving high text quality. The privacy/utility trade-off is controlled by $ε$, where $ε=0$ hides sensitive tokens entirely, while higher values trade off privacy for improved text quality. We show that our method creates token-level provably privatized documents with substantially improved theoretical and empirical privacy, achieving $6\times$ lower perplexity than related DPI methods.

Method: Created dynamic user simulations based on real e-commerce customer personas and developed realistic task datasets from authentic e-commerce dialogues covering diverse business scenarios.

Result: The benchmark proved highly challenging - even advanced models like GPT-4o achieved only 10-20% pass^3 metric, demonstrating the substantial difficulties in complex e-commerce scenarios.

Conclusion: ECom-Bench provides a valuable evaluation framework for multimodal LLM agents in e-commerce, with publicly available code and data to advance research in this challenging domain.

Abstract: In this paper, we introduce ECom-Bench, the first benchmark framework for evaluating LLM agent with multimodal capabilities in the e-commerce customer support domain. ECom-Bench features dynamic user simulation based on persona information collected from real e-commerce customer interactions and a realistic task dataset derived from authentic e-commerce dialogues. These tasks, covering a wide range of business scenarios, are designed to reflect real-world complexities, making ECom-Bench highly challenging. For instance, even advanced models like GPT-4o achieve only a 10-20% pass^3 metric in our benchmark, highlighting the substantial difficulties posed by complex e-commerce scenarios. The code and data have been made publicly available at https://github.com/XiaoduoAILab/ECom-Bench to facilitate further research and development in this domain.

Method: Used benchmark generated from templates with variations in locomotion means and person perspective (1st/2nd/3rd person) to test 28 LLMs.

Result: Even newer Large Reasoning Models cannot reliably determine correct cardinal directions for all questions in the benchmark.

Conclusion: Current LLMs, including advanced reasoning models, have significant limitations in spatial reasoning about cardinal directions despite variations in testing scenarios.

Abstract: We investigate the abilities of 28 Large language Models (LLMs) to reason about cardinal directions (CDs) using a benchmark generated from a set of templates, extensively testing an LLM’s ability to determine the correct CD given a particular scenario. The templates allow for a number of degrees of variation such as means of locomotion of the agent involved, and whether set in the first, second or third person. Even the newer Large Reasoning Models are unable to reliably determine the correct CD for all questions. This paper summarises and extends earlier work presented at COSIT-24.

Method: Proposes an alignment evaluation framework that models stance of responses as a proxy for opinions, evaluated using positive-unlabeled online learning with base classifiers and instruction-tuned language models for post-training alignment assessment.

Result: The framework provides a principled approach to assess how implicit opinions are (mis)represented in conversation models, enabling validation of normative alignment.

Conclusion: The study offers a pathway toward more inclusive model behavior by foregrounding implicit conversations and evaluating normative social views through stance analysis.

Abstract: Shaping inclusive representations that embrace diversity and ensure fair participation and reflections of values is at the core of many conversation-based models. However, many existing methods rely on surface inclusion using mention of user demographics or behavioral attributes of social groups. Such methods overlook the nuanced, implicit expression of opinion embedded in conversations. Furthermore, the over-reliance on overt cues can exacerbate misalignment and reinforce harmful or stereotypical representations in model outputs. Thus, we took a step back and recognized that equitable inclusion needs to account for the implicit expression of opinion and use the stance of responses to validate the normative alignment. This study aims to evaluate how opinions are represented in NLP or computational models by introducing an alignment evaluation framework that foregrounds implicit, often overlooked conversations and evaluates the normative social views and discourse. Our approach models the stance of responses as a proxy for the underlying opinion, enabling a considerate and reflective representation of diverse social viewpoints. We evaluate the framework using both (i) positive-unlabeled (PU) online learning with base classifiers, and (ii) instruction-tuned language models to assess post-training alignment. Through this, we provide a principled and structured lens on how implicit opinions are (mis)represented and offer a pathway toward more inclusive model behavior.

Method: Constructed UnsafeChain dataset from hard prompts with diverse sources, identified unsafe completions, and explicitly corrected them into safe responses. Fine-tuned three large reasoning models on this dataset.

Result: UnsafeChain consistently outperforms prior datasets (SafeChain and STAR-1) across six out-of-distribution and five in-distribution benchmarks, with even a 1K subset matching or surpassing baseline performance.

Conclusion: Correction-based supervision through UnsafeChain is effective and generalizable for safety alignment, enhancing model safety while maintaining reasoning capabilities.

Abstract: As large reasoning models (LRMs) grow more capable, chain-of-thought (CoT) reasoning introduces new safety challenges. Existing SFT-based safety alignment studies dominantly focused on filtering prompts with safe, high-quality responses, while overlooking hard prompts that always elicit harmful outputs. To fill this gap, we introduce UnsafeChain, a safety alignment dataset constructed from hard prompts with diverse sources, where unsafe completions are identified and explicitly corrected into safe responses. By exposing models to unsafe behaviors and guiding their correction, UnsafeChain enhances safety while preserving general reasoning ability. We fine-tune three LRMs on UnsafeChain and compare them against recent SafeChain and STAR-1 across six out-of-distribution and five in-distribution benchmarks. UnsafeChain consistently outperforms prior datasets, with even a 1K subset matching or surpassing baseline performance, demonstrating the effectiveness and generalizability of correction-based supervision. We release our dataset and code at https://github.com/mbzuai-nlp/UnsafeChain

Method: Four-stage synthetic data generation pipeline: cultural data segregation, cultural data adaptation, machine translation, and quality filtering to expand English safety dataset to 8 languages.

Result: Created Nemotron-Safety-Guard-Dataset-v3 with 386,661 samples in 9 languages; trained Llama-3.1-Nemotron-Safety-Guard-8B-v3 model achieving SOTA performance on multilingual benchmarks with strong cross-lingual transfer.

Conclusion: This work advances multilingual LLM safety by enabling culturally aware safety guard models, addressing the vulnerability of LLMs to unsafe responses in non-English languages.

Abstract: The increasing use of Large Language Models (LLMs) in agentic applications highlights the need for robust safety guard models. While content safety in English is well-studied, non-English languages lack similar advancements due to the high cost of collecting culturally aligned labeled datasets. We present CultureGuard, a novel solution for curating culturally aligned, high-quality safety datasets across multiple languages. Our approach introduces a four-stage synthetic data generation and filtering pipeline: cultural data segregation, cultural data adaptation, machine translation, and quality filtering. This pipeline enables the conversion and expansion of the Nemotron-Content-Safety-Dataset-V2 English safety dataset into eight distinct languages: Arabic, German, Spanish, French, Hindi, Japanese, Thai, and Chinese. The resulting dataset, Nemotron-Safety-Guard-Dataset-v3, comprises 386,661 samples in 9 languages and facilitates the training of Llama-3.1-Nemotron-Safety-Guard-8B-v3 via LoRA-based fine-tuning. The final model achieves state-of-the-art performance on several multilingual content safety benchmarks. Furthermore, we show our moderately multilingual fine-tuning enables robust cross-lingual transfer and strong zero-shot generalization to unseen languages. We also benchmark the latest open LLMs on multilingual safety and observe that these LLMs are more prone to give unsafe responses when prompted in non-English languages. This work advances multilingual LLM safety by enabling the development of culturally aware safety guard models.

Method: Proposed structured CSC framework with 5 conversational stages and 12 strategies based on COPC guidelines. Used LLMs to rewrite real conversations into CSConv dataset and created RoleCS training dataset through role-playing simulations.

Result: Fine-tuning strong LLMs on RoleCS significantly improved their ability to generate high-quality, strategy-aligned responses on CSConv evaluation dataset. Human evaluations confirmed gains in problem resolution.

Conclusion: The CSC framework and datasets effectively train customer service agents and LLMs to generate structured, empathetic responses using professional support strategies, with demonstrated improvements in response quality and problem resolution.

Abstract: Effective customer support requires not only accurate problem solving but also structured and empathetic communication aligned with professional standards. However, existing dialogue datasets often lack strategic guidance, and real-world service data is difficult to access and annotate. To address this, we introduce the task of Customer Support Conversation (CSC), aimed at training customer service agents to respond using well-defined support strategies. We propose a structured CSC framework grounded in COPC guidelines, defining five conversational stages and twelve strategies to guide high-quality interactions. Based on this, we construct CSConv, an evaluation dataset of 1,855 real-world customer-agent conversations rewritten using LLMs to reflect deliberate strategy use, and annotated accordingly. Additionally, we develop a role-playing approach that simulates strategy-rich conversations using LLM-powered roles aligned with the CSC framework, resulting in the training dataset RoleCS. Experiments show that fine-tuning strong LLMs on RoleCS significantly improves their ability to generate high-quality, strategy-aligned responses on CSConv. Human evaluations further confirm gains in problem resolution. All code and data will be made publicly available at https://github.com/aliyun/qwen-dianjin.

Method: Proposes BEE-RAG framework that uses entropy invariance to separate attention sensitivity from context length, includes zero-shot inference for multi-importance estimation, and parameter-efficient adaptive fine-tuning to find optimal balancing factors.

Result: Extensive experiments across multiple RAG tasks demonstrate the effectiveness of BEE-RAG in improving RAG system performance.

Conclusion: BEE-RAG successfully improves RAG adaptability to varying context lengths through balanced entropy engineering, providing stable performance across different settings.

Abstract: With the rapid advancement of large language models (LLMs), retrieval-augmented generation (RAG) has emerged as a critical approach to supplement the inherent knowledge limitations of LLMs. However, due to the typically large volume of retrieved information, RAG tends to operate with long context lengths. From the perspective of entropy engineering, we identify unconstrained entropy growth and attention dilution due to long retrieval context as significant factors affecting RAG performance. In this paper, we propose the balanced entropy-engineered RAG (BEE-RAG) framework, which improves the adaptability of RAG systems to varying context lengths through the principle of entropy invariance. By leveraging balanced context entropy to reformulate attention dynamics, BEE-RAG separates attention sensitivity from context length, ensuring a stable entropy level. Building upon this, we introduce a zero-shot inference strategy for multi-importance estimation and a parameter-efficient adaptive fine-tuning mechanism to obtain the optimal balancing factor for different settings. Extensive experiments across multiple RAG tasks demonstrate the effectiveness of BEE-RAG.

Method: Analyzed and compared four representative creativity measures: creativity index, perplexity, syntactic templates, and LLM-as-a-Judge across creative writing, unconventional problem-solving, and research ideation domains.

Result: Metrics exhibit limited consistency and capture different dimensions of creativity. Creativity index focuses on lexical diversity, perplexity is sensitive to model confidence, syntactic templates miss conceptual creativity, and LLM-as-a-Judge shows instability and bias.

Conclusion: Current creativity evaluation frameworks have significant limitations and need more robust, generalizable approaches that better align with human judgments of creativity.

Abstract: We systematically examine, analyze, and compare representative creativity measures–creativity index, perplexity, syntactic templates, and LLM-as-a-Judge–across diverse creative domains, including creative writing, unconventional problem-solving, and research ideation. Our analyses reveal that these metrics exhibit limited consistency, capturing different dimensions of creativity. We highlight key limitations, including the creativity index’s focus on lexical diversity, perplexity’s sensitivity to model confidence, and syntactic templates’ inability to capture conceptual creativity. Additionally, LLM-as-a-Judge shows instability and bias. Our findings underscore the need for more robust, generalizable evaluation frameworks that better align with human judgments of creativity.

Method: Comprehensive evaluation using small BERT models from pre-training to fine-tuning for Telugu (agglutinative), with preliminary evaluation in Hindi and English. Created gold morpheme segmentation dataset for Telugu with 600 derivational and 7000 inflectional word forms.

Result: 1) Tokenizer algorithm choice is most significant - Unigram consistently outperforms BPE. 2) Morphological alignment shows moderate positive correlation but secondary importance. 3) Hybrid approaches with morphological pre-segmentation boost BPE performance but not Unigram.

Conclusion: Tokenizer algorithm selection is more critical than morphological alignment for downstream performance. Need for better intrinsic evaluation metrics to explain downstream performance trends consistently.

Abstract: The relationship between tokenizer algorithm (e.g., Byte-Pair Encoding (BPE), Unigram), morphological alignment, tokenization quality (e.g., compression efficiency), and downstream performance remains largely unclear, particularly for languages with complex morphology. In this paper, we conduct a comprehensive evaluation of tokenizers using small-sized BERT models – from pre-training through fine-tuning – for Telugu (agglutinative), along with preliminary evaluation in Hindi (primarily fusional with some agglutination) and English (fusional). To evaluate morphological alignment of tokenizers in Telugu, we create a dataset containing gold morpheme segmentations of 600 derivational and 7000 inflectional word forms. Our experiments reveal two key findings for Telugu. First, the choice of tokenizer algorithm is the most significant factor influencing performance, with Unigram-based tokenizers consistently outperforming BPE across most settings. Second, while better morphological alignment shows a moderate, positive correlation with performance on text classification and structure prediction tasks, its impact is secondary to the tokenizer algorithm. Notably, hybrid approaches that use morphological information for pre-segmentation significantly boost the performance of BPE, though not Unigram. Our results further showcase the need for comprehensive intrinsic evaluation metrics for tokenizers that could explain downstream performance trends consistently.

Method: Propose two fusion strategies: Global Softmax for overall layer importance and Transformer Softmax for token-specific weights. All intermediate layers are fused and mapped to LLM’s embedding space. Model trained only on English data without parallel/multilingual data.

Result: Significant improvements on low-resource languages: Sinhala classification accuracy increased from 71.66% to 75.86%, clear gains in Indic languages (Tamil, Bengali, Malayalam), and overall XNLI accuracy improved from 70.36% to 71.50%. Outperforms LangBridge baseline.

Conclusion: This approach provides a scalable, data-efficient path toward more capable and equitable multilingual LLMs by leveraging full encoder layer information without requiring multilingual training data.

Abstract: Large Language Models (LLMs) excel in English, but their performance degrades significantly on low-resource languages (LRLs) due to English-centric training. While methods like LangBridge align LLMs with multilingual encoders such as the Massively Multilingual Text-to-Text Transfer Transformer (mT5), they typically use only the final encoder layer. We propose a novel architecture that fuses all intermediate layers, enriching the linguistic information passed to the LLM. Our approach features two strategies: (1) a Global Softmax weighting for overall layer importance, and (2) a Transformer Softmax model that learns token-specific weights. The fused representations are mapped into the LLM’s embedding space, enabling it to process multilingual inputs. The model is trained only on English data, without using any parallel or multilingual data. Evaluated on XNLI, IndicXNLI, Sinhala News Classification, and Amazon Reviews, our Transformer Softmax model significantly outperforms the LangBridge baseline. We observe strong performance gains in LRLs, improving Sinhala classification accuracy from 71.66% to 75.86% and achieving clear improvements across Indic languages such as Tamil, Bengali, and Malayalam. These specific gains contribute to an overall boost in average XNLI accuracy from 70.36% to 71.50%. This approach offers a scalable, data-efficient path toward more capable and equitable multilingual LLMs.

Method: Enhanced Llama-3-8B tokenizer with Sinhala vocabulary and performed continual pre-training on a cleaned 10 million Sinhala corpus.

Result: SinLlama outperformed base and instruct variants of Llama-3-8B by a significant margin in text classification tasks after instruction fine-tuning.

Conclusion: Successfully created the first decoder-based open-source LLM with explicit Sinhala support, demonstrating effective adaptation of multilingual models for low-resource languages.

Abstract: Low-resource languages such as Sinhala are often overlooked by open-source Large Language Models (LLMs). In this research, we extend an existing multilingual LLM (Llama-3-8B) to better serve Sinhala. We enhance the LLM tokenizer with Sinhala specific vocabulary and perform continual pre-training on a cleaned 10 million Sinhala corpus, resulting in the SinLlama model. This is the very first decoder-based open-source LLM with explicit Sinhala support. When SinLlama was instruction fine-tuned for three text classification tasks, it outperformed base and instruct variants of Llama-3-8B by a significant margin.

Method: Developed multimodal pipeline integrating transformer embeddings with 110 handcrafted linguistic features, tested 10 transformer models with three fine-tuning strategies, used 5 LLMs for synthetic data augmentation, and evaluated multimodal LLMs in zero-shot and fine-tuned modes on ADReSSo and Delaware datasets.

Result: Fusion model achieved F1=83.3 (AUC=89.5) on ADReSSo, outperforming baselines. MedAlpaca7B augmentation improved F1 to 85.7. Fine-tuning boosted unimodal LLMs significantly (MedAlpaca7B F1=47.7→78.7). On Delaware MCI cohort, fusion plus augmentation achieved F1=72.8 (AUC=69.6).

Conclusion: Combining transformer and linguistic features enhances ADRD detection. LLM-based augmentation improves data efficiency with diminishing returns, while current multimodal models show limitations. Pipeline shows potential for scalable early screening validated on independent MCI cohort.

Abstract: Alzheimer’s disease and related dementias(ADRD) affect nearly five million older adults in the United States, yet more than half remain undiagnosed. Speech-based natural language processing(NLP) offers a scalable approach for detecting early cognitive decline through subtle linguistic markers that may precede clinical diagnosis. This study develops and evaluates a speech-based screening pipeline integrating transformer embeddings with handcrafted linguistic features, synthetic augmentation using large language models(LLMs), and benchmarking of unimodal and multimodal classifiers. External validation assessed generalizability to a MCI-only cohort. Transcripts were drawn from the ADReSSo 2021 benchmark dataset(n=237, Pitt Corpus) and the DementiaBank Delaware corpus(n=205, MCI vs. controls). Ten transformer models were tested under three fine-tuning strategies. A late-fusion model combined embeddings from the top transformer with 110 linguistic features. Five LLMs(LLaMA8B/70B, MedAlpaca7B, Ministral8B,GPT-4o) generated label-conditioned synthetic speech for augmentation, and three multimodal LLMs(GPT-4o,Qwen-Omni,Phi-4) were evaluated in zero-shot and fine-tuned modes. On ADReSSo, the fusion model achieved F1=83.3(AUC=89.5), outperforming transformer-only and linguistic baselines. MedAlpaca7B augmentation(2x) improved F1=85.7, though larger scales reduced gains. Fine-tuning boosted unimodal LLMs(MedAlpaca7B F1=47.7=>78.7), while multimodal models performed lower (Phi-4=71.6;GPT-4o=67.6). On Delaware, the fusion plus 1x MedAlpaca7B model achieved F1=72.8(AUC=69.6). Integrating transformer and linguistic features enhances ADRD detection. LLM-based augmentation improves data efficiency but yields diminishing returns, while current multimodal models remain limited. Validation on an independent MCI cohort supports the pipeline’s potential for scalable, clinically relevant early screening.

Method: ComoRAG uses iterative reasoning cycles with a dynamic memory workspace. When encountering reasoning impasses, it generates probing queries, retrieves new evidence, and integrates it into a global memory pool to build coherent context.

Result: Outperforms strong RAG baselines with consistent relative gains up to 11% across four challenging long-context narrative benchmarks (200K+ tokens), particularly excelling at complex queries requiring global context comprehension.

Conclusion: ComoRAG offers a principled, cognitively motivated paradigm for retrieval-based stateful reasoning, demonstrating that narrative reasoning is a dynamic interplay between evidence acquisition and knowledge consolidation.

Abstract: Narrative comprehension on long stories and novels has been a challenging domain attributed to their intricate plotlines and entangled, often evolving relations among characters and entities. Given the LLM’s diminished reasoning over extended context and its high computational cost, retrieval-based approaches remain a pivotal role in practice. However, traditional RAG methods could fall short due to their stateless, single-step retrieval process, which often overlooks the dynamic nature of capturing interconnected relations within long-range context. In this work, we propose ComoRAG, holding the principle that narrative reasoning is not a one-shot process, but a dynamic, evolving interplay between new evidence acquisition and past knowledge consolidation, analogous to human cognition on reasoning with memory-related signals in the brain. Specifically, when encountering a reasoning impasse, ComoRAG undergoes iterative reasoning cycles while interacting with a dynamic memory workspace. In each cycle, it generates probing queries to devise new exploratory paths, then integrates the retrieved evidence of new aspects into a global memory pool, thereby supporting the emergence of a coherent context for the query resolution. Across four challenging long-context narrative benchmarks (200K+ tokens), ComoRAG outperforms strong RAG baselines with consistent relative gains up to 11% compared to the strongest baseline. Further analysis reveals that ComoRAG is particularly advantageous for complex queries requiring global context comprehension, offering a principled, cognitively motivated paradigm towards retrieval-based stateful reasoning. Our framework is made publicly available at https://github.com/EternityJune25/ComoRAG.

Method: Developed EMNLP framework with 88 teacher-specific moral dilemmas, extended existing scales, and created targeted soft prompt injection sets to evaluate compliance and vulnerability in teacher-role LLMs across 14 different models.

Result: Teacher-role LLMs exhibit more idealized and polarized personalities than human teachers, excel in abstract moral reasoning but struggle with emotionally complex situations, and models with stronger reasoning are paradoxically more vulnerable to harmful prompt injection.

Conclusion: EMNLP provides the first benchmark for assessing ethical and psychological alignment of teacher-role LLMs, revealing important safety concerns where increased capability correlates with increased vulnerability to manipulation.

Abstract: Simulating Professions (SP) enables Large Language Models (LLMs) to emulate professional roles. However, comprehensive psychological and ethical evaluation in these contexts remains lacking. This paper introduces EMNLP, an Educator-role Moral and Normative LLMs Profiling framework for personality profiling, moral development stage measurement, and ethical risk under soft prompt injection. EMNLP extends existing scales and constructs 88 teacher-specific moral dilemmas, enabling profession-oriented comparison with human teachers. A targeted soft prompt injection set evaluates compliance and vulnerability in teacher SP. Experiments on 14 LLMs show teacher-role LLMs exhibit more idealized and polarized personalities than human teachers, excel in abstract moral reasoning, but struggle with emotionally complex situations. Models with stronger reasoning are more vulnerable to harmful prompt injection, revealing a paradox between capability and safety. The model temperature and other hyperparameters have limited influence except in some risk behaviors. This paper presents the first benchmark to assess ethical and psychological alignment of teacher-role LLMs for educational AI. Resources are available at https://e-m-n-l-p.github.io/.

Method: Automated generation of guideline-violating questions based on government guidelines, with jailbreak diagnostics (GUARD-JD) to create scenarios that provoke unethical responses and test safety mechanisms.

Result: Empirically validated on seven LLMs including Vicuna-13B, GPT-4, Claude-3.7, showing compliance testing under three government guidelines and successful jailbreak diagnostics that can transfer to vision-language models.

Conclusion: GUARD provides an effective framework for operationalizing AI ethics guidelines into practical testing, helping promote reliable LLM-based applications through comprehensive compliance assessment.

Abstract: As Large Language Models become increasingly integral to various domains, their potential to generate harmful responses has prompted significant societal and regulatory concerns. In response, governments have issued ethics guidelines to promote the development of trustworthy AI. However, these guidelines are typically high-level demands for developers and testers, leaving a gap in translating them into actionable testing questions to verify LLM compliance. To address this challenge, we introduce GUARD (\textbf{G}uideline \textbf{U}pholding Test through \textbf{A}daptive \textbf{R}ole-play and Jailbreak \textbf{D}iagnostics), a testing method designed to operationalize guidelines into specific guideline-violating questions that assess LLM adherence. To implement this, GUARD uses automated generation of guideline-violating questions based on government-issued guidelines, thereby testing whether responses comply with these guidelines. When responses directly violate guidelines, GUARD reports inconsistencies. Furthermore, for responses that do not directly violate guidelines, GUARD integrates the concept of ``jailbreaks’’ to diagnostics, named GUARD-JD, which creates scenarios that provoke unethical or guideline-violating responses, effectively identifying potential scenarios that could bypass built-in safety mechanisms. Our method finally culminates in a compliance report, delineating the extent of adherence and highlighting any violations. We have empirically validated the effectiveness of GUARD on seven LLMs, including Vicuna-13B, LongChat-7B, Llama2-7B, Llama-3-8B, GPT-3.5, GPT-4, GPT-4o, and Claude-3.7, by testing compliance under three government-issued guidelines and conducting jailbreak diagnostics. Additionally, GUARD-JD can transfer jailbreak diagnostics to vision-language models, demonstrating its usage in promoting reliable LLM-based applications.

Method: Joint assessment of semantic and acoustic dimensions using rationale-based supervision, creation of SpeechFeedback synthetic preference dataset, and two-stage training paradigm to address speech preference data scarcity.

Result: SageLM achieves 82.79% agreement rate with human evaluators, outperforming cascaded baselines by 7.42% and SLM-based baselines by 26.20%.

Conclusion: SageLM provides an effective framework for comprehensive Speech-to-Speech LLM evaluation that significantly outperforms existing approaches through joint semantic-acoustic assessment and explainable rationale-based supervision.

Abstract: Speech-to-Speech (S2S) Large Language Models (LLMs) are foundational to natural human-computer interaction, enabling end-to-end spoken dialogue systems. However, evaluating these models remains a fundamental challenge. We propose \texttt{SageLM}, an end-to-end, multi-aspect, and explainable speech LLM for comprehensive S2S LLMs evaluation. First, unlike cascaded approaches that disregard acoustic features, SageLM jointly assesses both semantic and acoustic dimensions. Second, it leverages rationale-based supervision to enhance explainability and guide model learning, achieving superior alignment with evaluation outcomes compared to rule-based reinforcement learning methods. Third, we introduce \textit{SpeechFeedback}, a synthetic preference dataset, and employ a two-stage training paradigm to mitigate the scarcity of speech preference data. Trained on both semantic and acoustic dimensions, SageLM achieves an 82.79% agreement rate with human evaluators, outperforming cascaded and SLM-based baselines by at least 7.42% and 26.20%, respectively.

Method: Conceptual analysis of the Turing test framework, examining its statistical nature and the role of multiple judges in determining intelligence through normalized aggregate judgments.

Result: The analysis reveals that the Turing test objectivizes normative ideals of normal behavior rather than actual normal behavior, and that current large language models target exceptional intelligence rather than the normal intelligence the test was designed to evaluate.

Conclusion: LLMs like ChatGPT are unlikely to pass the Turing test as they model artificial smartness (exceptional intelligence) rather than artificial intelligence (normal human intelligence), and the test’s configuration fails to objectively capture normal human behavior.

Abstract: This paper proposes to revisit the Turing test through the concept of normality. Its core argument is that the Turing test is a test of normal intelligence as assessed by a normal judge. First, in the sense that the Turing test targets normal/average rather than exceptional human intelligence, so that successfully passing the test requires machines to “make mistakes” and display imperfect behavior just like normal/average humans. Second, in the sense that the Turing test is a statistical test where judgments of intelligence are never carried out by a single “average” judge (understood as non-expert) but always by a full jury. As such, the notion of “average human interrogator” that Turing talks about in his original paper should be understood primarily as referring to a mathematical abstraction made of the normalized aggregate of individual judgments of multiple judges. Its conclusions are twofold. First, it argues that large language models such as ChatGPT are unlikely to pass the Turing test as those models precisely target exceptional rather than normal/average human intelligence. As such, they constitute models of what it proposes to call artificial smartness rather than artificial intelligence, insofar as they deviate from the original goal of Turing for the modeling of artificial minds. Second, it argues that the objectivization of normal human behavior in the Turing test fails due to the game configuration of the test which ends up objectivizing normative ideals of normal behavior rather than normal behavior per se.

Method: Built transfer learning matrix with dimensionality reduction, trained 10 models to identify latent abilities (Reasoning, Sentiment Classification, NLU, Arithmetic) and analyze cross-task interactions.

Result: Performance improvements defy explanations based on surface-level dataset similarity or source data quality; hidden statistical factors and linguistic features are more influential.

Conclusion: Provides insights into complex transfer learning dynamics, enabling more predictable and effective LLM adaptation by focusing on underlying statistical and linguistic factors.

Abstract: Large language models are increasingly deployed across diverse applications. This often includes tasks LLMs have not encountered during training. This implies that enumerating and obtaining the high-quality training data for all tasks is infeasible. Thus, we often need to rely on transfer learning using datasets with different characteristics, and anticipate out-of-distribution requests. Motivated by this practical need, we propose an analysis framework, building a transfer learning matrix and dimensionality reduction, to dissect these cross-task interactions. We train and analyze 10 models to identify latent abilities (e.g., Reasoning, Sentiment Classification, NLU, Arithmetic) and discover the side effects of the transfer learning. Our findings reveal that performance improvements often defy explanations based on surface-level dataset similarity or source data quality. Instead, hidden statistical factors of the source dataset, such as class distribution and generation length proclivities, alongside specific linguistic features, are actually more influential. This work offers insights into the complex dynamics of transfer learning, paving the way for more predictable and effective LLM adaptation.

Method: Proposed neurosymbolic approach integrating LLMs with symbolic logical solvers, and later combined with Chain-of-Thought prompting (NSCoT).

Result: Neurosymbolic methods are more robust to counterfactual variations but perform worse overall than purely neural methods. NSCoT improves performance but still lags behind standard CoT.

Conclusion: While neurosymbolic approaches enhance robustness, they don’t match the performance of standard neural methods, opening research directions for future work.

Abstract: Recent trends in NLP aim to improve reasoning capabilities in Large Language Models (LLMs), with key focus on generalization and robustness to variations in tasks. Counterfactual task variants introduce minimal but semantically meaningful changes to otherwise valid first-order logic (FOL) problem instances altering a single predicate or swapping roles of constants to probe whether a reasoning system can maintain logical consistency under perturbation. Previous studies showed that LLMs becomes brittle on counterfactual variations, suggesting that they often rely on spurious surface patterns to generate responses. In this work, we explore if a neurosymbolic (NS) approach that integrates an LLM and a symbolic logical solver could mitigate this problem. Experiments across LLMs of varying sizes show that NS methods are more robust but perform worse overall that purely neural methods. We then propose NSCoT that combines an NS method and Chain-of-Thought (CoT) prompting and demonstrate that while it improves performance, NSCoT still lags behind standard CoT. Our analysis opens research directions for future work.

Method: EditGRPO integrates on-policy exploration with off-policy guidance by injecting sentence-level detailed corrections during training rollouts, addressing exploration and sampling efficiency issues in RL.

Result: Applied to Qwen2.5-VL-3B, EditGRPO achieved 3.4% average improvement in clinical metrics across four datasets and 5.9% average performance gain on unseen datasets, demonstrating superior out-of-domain generalization.

Conclusion: EditGRPO effectively optimizes radiology report generation through clinically aligned rewards and mixed-policy reinforcement learning, showing significant improvements over existing methods.

Abstract: Radiology report generation requires advanced medical image analysis, effective temporal reasoning, and accurate text generation. Although recent innovations, particularly multimodal large language models, have shown improved performance, their supervised fine-tuning (SFT) objective is not explicitly aligned with clinical efficacy. In this work, we introduce EditGRPO, a mixed-policy reinforcement learning algorithm designed specifically to optimize the generation through clinically motivated rewards. EditGRPO integrates on-policy exploration with off-policy guidance by injecting sentence-level detailed corrections during training rollouts. This mixed-policy approach addresses the exploration dilemma and sampling efficiency issues typically encountered in RL. Applied to a Qwen2.5-VL-3B, EditGRPO outperforms both SFT and vanilla GRPO baselines, achieving an average improvement of 3.4% in clinical metrics across four major datasets. Notably, EditGRPO also demonstrates superior out-of-domain generalization, with an average performance gain of 5.9% on unseen datasets.

Method: LinearRAG constructs a relation-free hierarchical graph (Tri-Graph) using only lightweight entity extraction and semantic linking, then uses a two-stage retrieval strategy: entity activation via local semantic bridging followed by passage retrieval through global importance aggregation.

Result: Extensive experiments on four datasets demonstrate that LinearRAG significantly outperforms baseline models.

Conclusion: LinearRAG provides an economical and reliable indexing solution that scales linearly with corpus size and avoids the instability of relation modeling, offering improved performance for complex retrieval tasks.

Abstract: Retrieval-Augmented Generation (RAG) is widely used to mitigate hallucinations of Large Language Models (LLMs) by leveraging external knowledge. While effective for simple queries, traditional RAG systems struggle with large-scale, unstructured corpora where information is fragmented. Recent advances incorporate knowledge graphs to capture relational structures, enabling more comprehensive retrieval for complex, multi-hop reasoning tasks. However, existing graph-based RAG (GraphRAG) methods rely on unstable and costly relation extraction for graph construction, often producing noisy graphs with incorrect or inconsistent relations that degrade retrieval quality. In this paper, we revisit the pipeline of existing GraphRAG systems and propose LinearRAG (Linear Graph-based Retrieval-Augmented Generation), an efficient framework that enables reliable graph construction and precise passage retrieval. Specifically, LinearRAG constructs a relation-free hierarchical graph, termed Tri-Graph, using only lightweight entity extraction and semantic linking, avoiding unstable relation modeling. This new paradigm of graph construction scales linearly with corpus size and incurs no extra token consumption, providing an economical and reliable indexing of the original passages. For retrieval, LinearRAG adopts a two-stage strategy: (i) relevant entity activation via local semantic bridging, followed by (ii) passage retrieval through global importance aggregation. Extensive experiments on four datasets demonstrate that LinearRAG significantly outperforms baseline models. Our code and datasets are available at https://github.com/DEEP-PolyU/LinearRAG.

Method: Clinically grounded framework using controlled text and image modifications across three axes: clinical fidelity, causal attribution, and confidence calibration, evaluated via reader study with radiologists.

Result: Answer accuracy and explanation quality can be decoupled; proprietary models outperform open-source on attribution (25.0% vs 1.4%) and fidelity (36.1% vs 31.7%); text cues influence explanations more than visual cues.

Conclusion: There are significant deployment risks as models can generate plausible but unfaithful explanations, highlighting the need to evaluate beyond final answer accuracy in clinical settings.

Abstract: Vision-language models (VLMs) often produce chain-of-thought (CoT) explanations that sound plausible yet fail to reflect the underlying decision process, undermining trust in high-stakes clinical use. Existing evaluations rarely catch this misalignment, prioritizing answer accuracy or adherence to formats. We present a clinically grounded framework for chest X-ray visual question answering (VQA) that probes CoT faithfulness via controlled text and image modifications across three axes: clinical fidelity, causal attribution, and confidence calibration. In a reader study (n=4), evaluator-radiologist correlations fall within the observed inter-radiologist range for all axes, with strong alignment for attribution (Kendall’s $τ_b=0.670$), moderate alignment for fidelity ($τ_b=0.387$), and weak alignment for confidence tone ($τ_b=0.091$), which we report with caution. Benchmarking six VLMs shows that answer accuracy and explanation quality can be decoupled, acknowledging injected cues does not ensure grounding, and text cues shift explanations more than visual cues. While some open-source models match final answer accuracy, proprietary models score higher on attribution (25.0% vs. 1.4%) and often on fidelity (36.1% vs. 31.7%), highlighting deployment risks and the need to evaluate beyond final answer accuracy.

Method: Decompose frozen weights via SVD and constrain LoRA updates to lie within the orthogonal complement of top-k singular subspace using double-sided orthogonal projections P_L = I - U_k U_k^T and P_R = I - V_k V_k^T.

Result: OPLoRA significantly reduces forgetting while maintaining competitive task-specific performance across commonsense reasoning, mathematics, and code generation on LLaMA-2 7B and Qwen2.5 7B.

Conclusion: Orthogonal projection is an effective mechanism for knowledge preservation in parameter-efficient fine-tuning, with mathematical guarantees for preserving top-k singular triples.

Abstract: Low-Rank Adaptation (LoRA) enables efficient fine-tuning of large language models but suffers from catastrophic forgetting when learned updates interfere with the dominant singular directions that encode essential pre-trained knowledge. We propose Orthogonal Projection LoRA (OPLoRA), a theoretically grounded approach that prevents this interference through double-sided orthogonal projections. By decomposing frozen weights via SVD, OPLoRA constrains LoRA updates to lie entirely within the orthogonal complement of the top-$k$ singular subspace using projections $P_L = I - U_k U_k^\top$ and $P_R = I - V_k V_k^\top$. We prove that this construction exactly preserves the top-$k$ singular triples, providing mathematical guarantees for knowledge retention. To quantify subspace interference, we introduce $ρ_k$, a metric measuring update alignment with dominant directions. Extensive experiments across commonsense reasoning, mathematics, and code generation demonstrate that OPLoRA significantly reduces forgetting while maintaining competitive task-specific performance on LLaMA-2 7B and Qwen2.5 7B, establishing orthogonal projection as an effective mechanism for knowledge preservation in parameter-efficient fine-tuning.

Method: Periodically compress generation KV cache using learned special-purpose tokens and evict compressed entries, trained via modified joint distillation and reinforcement learning framework.

Result: Achieves superior memory-accuracy Pareto frontier compared to models without cache compression and training-free compression techniques.

Conclusion: The proposed KV cache compression method effectively reduces memory and computational overhead while maintaining accuracy in long-context reasoning tasks.

Abstract: The scalability of large language models for long-context reasoning is severely constrained by the linear growth of their Transformer key-value cache, which incurs significant memory and computational costs. We posit that as a model generates reasoning tokens, the informational value of past generated tokens diminishes, creating an opportunity for compression. In this work, we propose to periodically compress the generation KV cache with a learned, special-purpose token and evict compressed entries. We train the model to perform this compression via a modified joint distillation and reinforcement learning (RL) framework. Our training method minimizes overhead over the conventional RL process, as it leverages RL outputs for distillation. Empirically, our method achieves a superior memory-accuracy Pareto frontier compared to both the model without cache compression and training-free compression techniques.

Method: Developed a fully-automated method using large language models (LLMs) to extract product ontologies in the form of meronymies from raw review texts.

Result: The LLM-based method produced ontologies that surpassed an existing BERT-based baseline when evaluated using an LLM-as-a-judge approach.

Conclusion: This work establishes groundwork for using LLMs more generally in ontology extraction tasks, both for products and other domains.

Abstract: Ontologies have become essential in today’s digital age as a way of organising the vast amount of readily available unstructured text. In providing formal structure to this information, ontologies have immense value and application across various domains, e.g., e-commerce, where countless product listings necessitate proper product organisation. However, the manual construction of these ontologies is a time-consuming, expensive and laborious process. In this paper, we harness the recent advancements in large language models (LLMs) to develop a fully-automated method of extracting product ontologies, in the form of meronymies, from raw review texts. We demonstrate that the ontologies produced by our method surpass an existing, BERT-based baseline when evaluating using an LLM-as-a-judge. Our investigation provides the groundwork for LLMs to be used more generally in (product or otherwise) ontology extraction.

Method: Uses lightweight meta-classifiers to route contexts to token clusters, with the union of top-k clusters forming the drafter’s shortlist. Leverages parallel execution of draft encoding and meta shortlisting on separate streams.

Result: Achieves 98.2% of full-vocabulary performance for Llama-3-8B, compared to 84.4% for fixed-shortlist baselines. Generates up to 2.18x more tokens vs 1.91x for fixed-vocabulary approaches.

Conclusion: DynaSpec provides robust, context-dependent dynamic shortlisting that speeds up drafting while generalizing across diverse tasks, overcoming limitations of static frequency-based approaches.

Abstract: Speculative decoding has become a standard way to accelerate LLM inference: a small drafter proposes multiple tokens and a large target model verifies them once per speculation length. Recently, scaling of the LLM vocabulary has pushed the number of tokens to grow substantially. While verification over the full vocabulary leaves the target model largely unaffected, the O(|V|d) parameters in the drafter’s output head become a latency bottleneck, slowing the entire pipeline. Contemporary methods (e.g., FR-Spec, VocabTrim) restrict the drafter’s vocabulary to a fixed top frequent subset of the target model’s vocabulary. Although this reduces draft-time compute, it is brittle, since: (i) frequency lists are corpus-dependent and require retuning to generalize, and (ii) static shortlists suppress rare or domain-specific tokens, lowering the expected number of tokens per verification step. We propose DynaSpec, a context-dependent dynamic shortlisting mechanism that is robust, speeds up drafting, and generalizes across diverse tasks. Concretely, we introduce lightweight, coarse-grained meta-classifiers that route contexts to a small number of token clusters; the union of the top-k selected clusters forms the drafter’s shortlist, while verification retains the full vocabulary and exactness. The meta-classifier finishes its computation earlier than the drafter’s hidden state generation by exploiting parallel execution of draft encoding and meta shortlisting on separate streams. Across standard speculative decoding benchmarks, DynaSpec delivers consistent improvements in mean accepted length, for Llama-3-8B, reaching upto 98.2% of full-vocabulary performance, while fixed-shortlist baselines attain only 84.4%. By leveraging context-dependent selection, DynaSpec achieves up to a 2.18 times increase in generated tokens compared to 1.91 times for fixed-vocabulary approaches.

Method: Created DiscoTrack benchmark with tasks across 12 languages targeting four levels of discourse understanding: salience recognition, entity tracking, discourse relations, and bridging inference.

Result: Evaluation shows these discourse tracking tasks remain challenging even for state-of-the-art models.

Conclusion: DiscoTrack addresses the gap in multilingual discourse tracking benchmarks and demonstrates that discourse-level understanding is still a difficult challenge for current LLMs.

Abstract: Recent LLM benchmarks have tested models on a range of phenomena, but are still focused primarily on natural language understanding for extraction of explicit information, such as QA or summarization, with responses often targeting information from individual sentences. We are still lacking more challenging, and importantly also multilingual, benchmarks focusing on implicit information and pragmatic inferences across larger documents in the context of discourse tracking: integrating and aggregating information across sentences, paragraphs and multiple speaker utterances. To this end, we present DiscoTrack, an LLM benchmark targeting a range of tasks across 12 languages and four levels of discourse understanding: salience recognition, entity tracking, discourse relations and bridging inference. Our evaluation shows that these tasks remain challenging, even for state-of-the-art models.

Method: Systematic study of DLM efficiency through empirical benchmarking, theoretical analysis, and investigation of acceleration strategies like dual cache and parallel decoding.

Result: AR models achieve higher throughput than DLMs; acceleration strategies only help at small batch sizes with diminishing returns when scaled.

Conclusion: Robust evaluation methods and improved acceleration strategies are needed to advance DLM research and make them competitive with AR models.

Abstract: Diffusion language models (DLMs) have emerged as a promising alternative to the long-dominant autoregressive (AR) paradigm, offering a parallelable decoding process that could yield greater efficiency. Yet, in practice, current open-source DLMs often underperform their AR counterparts in speed, limiting their real-world utility. This work presents a systematic study of DLM efficiency, identifying key issues in prior evaluation methods. Through empirical benchmarking and a theoretical analysis, we demonstrate that AR models generally achieve higher throughput, while DLMs consistently lag. We also investigate acceleration strategies, finding that techniques like dual cache and parallel decoding mainly offer gains at small batch sizes, with their benefits diminishing upon scaling. Our findings underscore the necessity of robust evaluation methods and improved acceleration strategies to advance research on DLMs.

Method: Four-component system: SenseFusion (lightweight encoder), Edge-RAG (local retrieval), PromptRouter (cost-aware policy), and Secure Execution (data minimization).

Result: Achieves sub-second latency, reduces bandwidth costs, improves factual consistency, and preserves privacy by transmitting only discrete codes.

Conclusion: Edge-first design successfully treats semantics, privacy, and predictable latency as co-equal goals for LLM deployments.

Abstract: We present CoSense-LLM, an edge-first framework that turns continuous multimodal sensor streams (for example Wi-Fi CSI, IMU, audio, RFID, and lightweight vision) into compact, verifiable semantic tokens and coordinates with large language models under explicit latency, energy, bandwidth, and privacy constraints. CoSense-LLM has four parts: (i) SenseFusion, a lightweight encoder that aligns sensor embeddings with language and compresses them into short discrete code sequences; (ii) Edge-RAG, a local hybrid retrieval layer that grounds generation in site specific policies and notes; (iii) PromptRouter, a cost and uncertainty aware policy that selects edge only generation, edge plus retrieval, or compact cloud escalation; and (iv) Secure Execution, an auditable redaction path that enforces data minimization so raw waveforms never leave the device. The system works with modern serving optimizations, including paged or streaming KV caches, FlashAttention style kernels, speculative decoding, and quantized LoRA adapters, and supports on device personalization and federated updates under non IID drift. Across home, office, and clinic deployments, CoSense-LLM delivers grounded explanations while meeting tight service level objectives: it sustains sub second (p95) end to end latency on edge dominant paths, reduces inter tier token and bandwidth costs by preferring local retrieval grounded responses, and preserves privacy by transmitting only discrete codes and redacted metadata. Ablations show that Edge-RAG improves factual consistency and reduces contradictions, calibrated uncertainty enables selective abstention and controlled escalations, and KV plus decoding accelerators lower energy per decision. The results support an edge first design that treats semantics, privacy, and predictable latency as co equal goals for large model deployments in interference prone environments.

Method: Conducted systematic ablation study of CORE-KG framework to quantify contributions of its two key components: type-aware coreference module and domain-guided structured prompts.

Result: Removing coreference resolution caused 28.25% increase in node duplication and 4.32% increase in noisy nodes. Removing structured prompts caused 4.29% increase in node duplication and 73.33% increase in noisy nodes.

Conclusion: Both coreference resolution and structured prompts are crucial for robust LLM-based pipelines in legal text analysis, with structured prompts being particularly effective at reducing noise.

Abstract: Human smuggling networks are increasingly adaptive and difficult to analyze. Legal case documents offer critical insights but are often unstructured, lexically dense, and filled with ambiguous or shifting references, which pose significant challenges for automated knowledge graph (KG) construction. While recent LLM-based approaches improve over static templates, they still generate noisy, fragmented graphs with duplicate nodes due to the absence of guided extraction and coreference resolution. The recently proposed CORE-KG framework addresses these limitations by integrating a type-aware coreference module and domain-guided structured prompts, significantly reducing node duplication and legal noise. In this work, we present a systematic ablation study of CORE-KG to quantify the individual contributions of its two key components. Our results show that removing coreference resolution results in a 28.25% increase in node duplication and a 4.32% increase in noisy nodes, while removing structured prompts leads to a 4.29% increase in node duplication and a 73.33% increase in noisy nodes. These findings offer empirical insights for designing robust LLM-based pipelines for extracting structured representations from complex legal texts.

Method: AutoAdv combines three adaptive mechanisms: pattern manager that learns from successful attacks, temperature manager that dynamically adjusts sampling parameters, and two-phase rewriting strategy that disguises then refines harmful requests.

Result: Achieved up to 95% attack success rate on Llama-3.1-8B within six turns (24% improvement over single-turn baselines), with persistent vulnerabilities found across commercial and open-source models including GPT-4o-mini, Qwen3-235B, and Mistral-7B.

Conclusion: Alignment strategies optimized for single-turn interactions fail to maintain robustness across extended conversations, highlighting an urgent need for multi-turn-aware defenses in LLM safety mechanisms.

Abstract: Large Language Models (LLMs) remain vulnerable to jailbreaking attacks where adversarial prompts elicit harmful outputs, yet most evaluations focus on single-turn interactions while real-world attacks unfold through adaptive multi-turn conversations. We present AutoAdv, a training-free framework for automated multi-turn jailbreaking that achieves up to 95% attack success rate on Llama-3.1-8B within six turns a 24 percent improvement over single turn baselines. AutoAdv uniquely combines three adaptive mechanisms: a pattern manager that learns from successful attacks to enhance future prompts, a temperature manager that dynamically adjusts sampling parameters based on failure modes, and a two-phase rewriting strategy that disguises harmful requests then iteratively refines them. Extensive evaluation across commercial and open-source models (GPT-4o-mini, Qwen3-235B, Mistral-7B) reveals persistent vulnerabilities in current safety mechanisms, with multi-turn attacks consistently outperforming single-turn approaches. These findings demonstrate that alignment strategies optimized for single-turn interactions fail to maintain robustness across extended conversations, highlighting an urgent need for multi-turn-aware defenses.

Method: Created HaluMem benchmark with three evaluation tasks and constructed user-centric multi-turn human-AI interaction datasets (HaluMem-Medium and HaluMem-Long) with ~15k memory points and 3.5k questions across different context scales.

Result: Empirical studies show existing memory systems generate and accumulate hallucinations during extraction and updating stages, which then propagate errors to QA stage.

Conclusion: Future research should focus on developing interpretable and constrained memory operation mechanisms to systematically suppress hallucinations and improve memory reliability.

Abstract: Memory systems are key components that enable AI systems such as LLMs and AI agents to achieve long-term learning and sustained interaction. However, during memory storage and retrieval, these systems frequently exhibit memory hallucinations, including fabrication, errors, conflicts, and omissions. Existing evaluations of memory hallucinations are primarily end-to-end question answering, which makes it difficult to localize the operational stage within the memory system where hallucinations arise. To address this, we introduce the Hallucination in Memory Benchmark (HaluMem), the first operation level hallucination evaluation benchmark tailored to memory systems. HaluMem defines three evaluation tasks (memory extraction, memory updating, and memory question answering) to comprehensively reveal hallucination behaviors across different operational stages of interaction. To support evaluation, we construct user-centric, multi-turn human-AI interaction datasets, HaluMem-Medium and HaluMem-Long. Both include about 15k memory points and 3.5k multi-type questions. The average dialogue length per user reaches 1.5k and 2.6k turns, with context lengths exceeding 1M tokens, enabling evaluation of hallucinations across different context scales and task complexities. Empirical studies based on HaluMem show that existing memory systems tend to generate and accumulate hallucinations during the extraction and updating stages, which subsequently propagate errors to the question answering stage. Future research should focus on developing interpretable and constrained memory operation mechanisms that systematically suppress hallucinations and improve memory reliability.

Method: Extended existing GRDD dataset with more Cretan, Cypriot, Pontic, Northern Greek data and added six new varieties. Fine-tuned three LLM architectures (Llama-3-8B, Llama-3.1-8B, Krikri-8B) and compared with frontier models.

Result: Created GRDD+ dataset with 6,374,939 words covering 10 Greek varieties - the largest and most varied Greek dialect dataset to date.

Conclusion: The study demonstrates the importance of dialectal data for LLM performance and provides a comprehensive benchmark for Greek dialect processing.

Abstract: We present an extended Greek Dialectal Dataset (GRDD+) 1that complements the existing GRDD dataset with more data from Cretan, Cypriot, Pontic and Northern Greek, while we add six new varieties: Greco-Corsican, Griko (Southern Italian Greek), Maniot, Heptanesian, Tsakonian, and Katharevusa Greek. The result is a dataset with total size 6,374,939 words and 10 varieties. This is the first dataset with such variation and size to date. We conduct a number of fine-tuning experiments to see the effect of good quality dialectal data on a number of LLMs. We fine-tune three model architectures (Llama-3-8B, Llama-3.1-8B, Krikri-8B) and compare the results to frontier models (Claude-3.7-Sonnet, Gemini-2.5, ChatGPT-5).

Method: Fine-tuned RoBERTa model using multiple curated datasets, with comparative experiments against traditional ML methods, domain-specific transformers, and prompting-based LLMs. Applied explainability methods including Layer Integrated Gradients and KeyBERT.

Result: Achieved macro F1-scores of 0.839 (six-class) and 0.870 (five-class, excluding stress), outperforming MentalBERT and baseline classifiers. Identified strong correlations between depression-suicidal ideation and anxiety-PTSD.

Conclusion: multiMentalRoBERTa is an effective, lightweight, and deployable solution for reliable and interpretable mental health detection, emphasizing fairness, bias mitigation, and human-in-the-loop safety protocols.

Abstract: The early detection of mental health disorders from social media text is critical for enabling timely support, risk assessment, and referral to appropriate resources. This work introduces multiMentalRoBERTa, a fine-tuned RoBERTa model designed for multiclass classification of common mental health conditions, including stress, anxiety, depression, post-traumatic stress disorder (PTSD), suicidal ideation, and neutral discourse. Drawing on multiple curated datasets, data exploration is conducted to analyze class overlaps, revealing strong correlations between depression and suicidal ideation as well as anxiety and PTSD, while stress emerges as a broad, overlapping category. Comparative experiments with traditional machine learning methods, domain-specific transformers, and prompting-based large language models demonstrate that multiMentalRoBERTa achieves superior performance, with macro F1-scores of 0.839 in the six-class setup and 0.870 in the five-class setup (excluding stress), outperforming both fine-tuned MentalBERT and baseline classifiers. Beyond predictive accuracy, explainability methods, including Layer Integrated Gradients and KeyBERT, are applied to identify lexical cues that drive classification, with a particular focus on distinguishing depression from suicidal ideation. The findings emphasize the effectiveness of fine-tuned transformers for reliable and interpretable detection in sensitive contexts, while also underscoring the importance of fairness, bias mitigation, and human-in-the-loop safety protocols. Overall, multiMentalRoBERTa is presented as a lightweight, robust, and deployable solution for enhancing support in mental health platforms.

Method: Controlled fine-tuning experiments using low-rank adapters (LoRA) on instruction-tuned LLMs, specifically testing with single rank-1 LoRA adapters and analyzing activation space steering vectors.

Result: Self-awareness can be reliably induced with a single rank-1 LoRA adapter, captured by a single steering vector in activation space, and is non-universal with independent representations across different tasks.

Conclusion: Behavioral self-awareness in LLMs emerges as a domain-specific, linear feature that can be easily induced and modulated, suggesting it’s a fundamental but localized capability.

Abstract: Recent studies have revealed that LLMs can exhibit behavioral self-awareness: the ability to accurately describe or predict their own learned behaviors without explicit supervision. This capability raises safety concerns as it may, for example, allow models to better conceal their true abilities during evaluation. We attempt to characterize the minimal conditions under which such self-awareness emerges, and the mechanistic processes through which it manifests. Through controlled finetuning experiments on instruction-tuned LLMs with low-rank adapters (LoRA), we find: (1) that self-awareness can be reliably induced using a single rank-1 LoRA adapter; (2) that the learned self-aware behavior can be largely captured by a single steering vector in activation space, recovering nearly all of the fine-tune’s behavioral effect; and (3) that self-awareness is non-universal and domain-localized, with independent representations across tasks. Together, these findings suggest that behavioral self-awareness emerges as a domain-specific, linear feature that can be easily induced and modulated.

Method: Built benchmark using 361 real Korean public documents (256 KOGL Type 1 + 105 legal portal files) with complex visual elements. Created 600 query-page-answer triples using multimodal models followed by human verification, spanning six public domains and three reasoning modalities.

Result: Evaluation on text-only and multimodal retrieval tasks revealed substantial performance gaps, particularly in multimodal scenarios requiring cross-modal reasoning, even for state-of-the-art models.

Conclusion: SDS KoPub VDR enables rigorous evaluation of multimodal AI in document intelligence and provides a roadmap for advancing real-world document understanding systems, especially for non-English languages and complex document structures.

Abstract: Existing benchmarks for visual document retrieval (VDR) largely overlook non-English languages and the structural complexity of official publications. To address this gap, we introduce SDS KoPub VDR, the first large-scale, public benchmark for retrieving and understanding Korean public documents. The benchmark is built upon 361 real-world documents, including 256 files under the KOGL Type 1 license and 105 from official legal portals, capturing complex visual elements like tables, charts, and multi-column layouts. To establish a reliable evaluation set, we constructed 600 query-page-answer triples. These were initially generated using multimodal models (e.g., GPT-4o) and subsequently underwent human verification to ensure factual accuracy and contextual relevance. The queries span six major public domains and are categorized by the reasoning modality required: text-based, visual-based, and cross-modal. We evaluate SDS KoPub VDR on two complementary tasks: (1) text-only retrieval and (2) multimodal retrieval, which leverages visual features alongside text. This dual-task evaluation reveals substantial performance gaps, particularly in multimodal scenarios requiring cross-modal reasoning, even for state-of-the-art models. As a foundational resource, SDS KoPub VDR enables rigorous and fine-grained evaluation and provides a roadmap for advancing multimodal AI in real-world document intelligence. The dataset is available at https://huggingface.co/datasets/SamsungSDS-Research/SDS-KoPub-VDR-Benchmark.

Method: Introduce Randomized-MLP (RMLP) regularization, a contrastive learning-based method that encourages semantically aligned representations during fine-tuning of DINOv2.

Result: RMLP improves or maintains downstream performance while producing more interpretable attention maps across medical and natural image modalities.

Conclusion: RMLP enhances ViT interpretability and provides mathematical insights into contrastive learning mechanisms.

Abstract: Vision Transformers (ViTs), such as DINOv2, achieve strong performance across domains but often repurpose low-informative patch tokens in ways that reduce the interpretability of attention and feature maps. This challenge is especially evident in medical imaging, where domain shifts can degrade both performance and transparency. In this paper, we introduce Randomized-MLP (RMLP) regularization, a contrastive learning-based method that encourages more semantically aligned representations. We use RMLPs when fine-tuning DINOv2 to both medical and natural image modalities, showing that it improves or maintains downstream performance while producing more interpretable attention maps. We also provide a mathematical analysis of RMLPs, offering insights into its role in enhancing ViT-based models and advancing our understanding of contrastive learning.

Method: Uses perception-informed BEV representations with sparse token-based approach, conditioning trajectory decoding on predicted future tokens without explicit reconstruction loss.

Result: Achieves 0.548 m ADE without future prediction (comparable to prior methods), and 0.479 m ADE with future token conditioning (12.6% improvement). Shows temporal fuzziness emerges where model adaptively attends to task-relevant semantics.

Conclusion: The ’token is all you need’ principle marks a paradigm shift from reconstructing the world to understanding it, enabling cognitively inspired systems that plan through imagination rather than reaction.

Abstract: We challenge the long-standing assumption that exhaustive scene modeling is required for high-performance end-to-end autonomous driving (E2EAD). Unlike world-model approaches that rely on computationally intensive future scene generation or vision-language-action (VLA) systems constrained by Markov assumptions, we show that a minimal set of semantically rich tokens is sufficient for effective planning. Experiments on the nuPlan benchmark (720 scenarios, over 11,000 samples) using perception-informed BEV representations yield three key findings: (1) even without future prediction, our sparse representation achieves 0.548 m ADE, comparable to or surpassing prior methods reporting around 0.75 m on nuScenes; (2) conditioning trajectory decoding on predicted future tokens reduces ADE to 0.479 m, a 12.6% improvement over current-state baselines; and (3) explicit reconstruction loss offers no benefit and may degrade performance under reliable perception inputs. Notably, we observe the emergence of temporal fuzziness, where the model adaptively attends to task-relevant semantics rather than aligning rigidly to fixed timestamps, providing a cognitive advantage for planning under uncertainty. Our “token is all you need” principle marks a paradigm shift from reconstructing the world to understanding it, laying a foundation for cognitively inspired systems that plan through imagination rather than reaction.

Method: Hybrid architecture combining OCR technology with Large Language Models (LLMs), deep learning models (CNNs and Transformers), domain-specific models for layout analysis, and graph analytics for contextual relationship mapping.

Result: Achieves unprecedented extraction quality and consistency with greater contextual sensitivity and much higher accuracy rates than older approaches.

Conclusion: The holistic AI platform enables maximum scalability and minimal human intervention for invoice processing across varied document types and layouts.

Abstract: Conventional Optical Character Recognition (OCR) systems are challenged by variant invoice layouts, handwritten text, and low- quality scans, which are often caused by strong template dependencies that restrict their flexibility across different document structures and layouts. Newer solutions utilize advanced deep learning models such as Convolutional Neural Networks (CNN) as well as Transformers, and domain-specific models for better layout analysis and accuracy across various sections over varied document types. Large Language Models (LLMs) have revolutionized extraction pipelines at their core with sophisticated entity recognition and semantic comprehension to support complex contextual relationship mapping without direct programming specification. Visual Named Entity Recognition (NER) capabilities permit extraction from invoice images with greater contextual sensitivity and much higher accuracy rates than older approaches. Existing industry best practices utilize hybrid architectures that blend OCR technology and LLM for maximum scalability and minimal human intervention. This work introduces a holistic Artificial Intelligence (AI) platform combining OCR, deep learning, LLMs, and graph analytics to achieve unprecedented extraction quality and consistency.

Method: Used in-context learning with different sampling strategies on Gemini-2.5-flash and Gemma3:27b models for quality assessment in wire-laser direct energy deposition processes.

Result: ICL-assisted VLMs achieved quality classification accuracies similar to traditional ML models using only minimal samples, while providing human-interpretable rationales.

Conclusion: ICL-assisted VLMs can effectively address application-specific manufacturing tasks with limited data, offering high accuracy and improved decision transparency through valid supporting rationales.

Abstract: Vision-based quality assessment in additive manufacturing often requires dedicated machine learning models and application-specific datasets. However, data collection and model training can be expensive and time-consuming. In this paper, we leverage vision-language models’ (VLMs’) reasoning capabilities to assess the quality of printed parts and introduce in-context learning (ICL) to provide VLMs with necessary application-specific knowledge and demonstration samples. This method eliminates the requirement for large application-specific datasets for training models. We explored different sampling strategies for ICL to search for the optimal configuration that makes use of limited samples. We evaluated these strategies on two VLMs, Gemini-2.5-flash and Gemma3:27b, with quality assessment tasks in wire-laser direct energy deposition processes. The results show that ICL-assisted VLMs can reach quality classification accuracies similar to those of traditional machine learning models while requiring only a minimal number of samples. In addition, unlike traditional classification models that lack transparency, VLMs can generate human-interpretable rationales to enhance trust. Since there are no metrics to evaluate their interpretability in manufacturing applications, we propose two metrics, knowledge relevance and rationale validity, to evaluate the quality of VLMs’ supporting rationales. Our results show that ICL-assisted VLMs can address application-specific tasks with limited data, achieving relatively high accuracy while also providing valid supporting rationales for improved decision transparency.

Method: Three key components: 1) Unified multimodal generation framework integrating semantic and spatial features with learnable cross-modal attention; 2) Dynamic perception pretraining using bidirectional alignment with inverse/forward dynamics tasks; 3) Reinforced supervised fine-tuning with spatial logic alignment between text actions and generated images.

Result: The approach enables coordinated language-visual modeling and spatio-aware multimodal planning capabilities for long-horizon tasks.

Conclusion: EVLP provides an effective solution for multimodal planning in complex embodied manipulation by unifying textual reasoning and visual generation through innovative training strategies.

Abstract: In complex embodied long-horizon manipulation tasks, effective task decomposition and execution require synergistic integration of textual logical reasoning and visual-spatial imagination to ensure efficient and accurate operation. Current methods fail to adopt a unified generation framework for multimodal planning, lead to inconsistent in multimodal planning. To address this challenge, we present \textbf{EVLP (Embodied Vision-Language Planner)}, an innovative multimodal unified generation framework that jointly models linguistic reasoning and visual generation. Our approach achieves multimodal planning for long-horizon tasks through a novel training pipeline incorporating dynamic pretraining and reinforced alignment. Our core innovations consist of three key components: \textbf{1) Unified Multimodal Generation Framework}: For understanding, We integrate semantic information with spatial features to provide comprehensive visual perception. For generation, we directly learn the joint distribution of discrete images for one-step visual synthesis, enabling coordinated language-visual modeling through learnable cross-modal attention mechanisms. \textbf{2) Dynamic Perception Pretraining}: We propose a bidirectional dynamic alignment strategy employing inverse dynamics tasks and forward dynamics tasks, effectively strengthening multimodal correlations within a unified feature space. \textbf{3) Reinforced Supervised Fine-Tuning}: While conducting instruction-based fine-tuning in the unified generation space, we construct a reinforce loss to align the spatial logic between textual actions and generated images, enabling the model to acquire spatio-awared multimodal planning capabilities.

Method: Uses view feature fusion model and Unified Graph Structure Adapter (UGA) to learn consensus graphs end-to-end, with Similarity Matrix Alignment Loss (SMAL) and Feature Representation Alignment Loss (FRAL) to optimize view-specific graphs and preserve cross-view topological consistency.

Result: Achieves state-of-the-art performance on eight multi-view benchmark datasets through extensive qualitative and quantitative experiments.

Conclusion: MCFCN effectively addresses limitations of existing multi-view clustering methods and demonstrates superior clustering performance through its fusion-consensus approach.

Abstract: Existing Multi-view Clustering (MVC) methods based on subspace learning focus on consensus representation learning while neglecting the inherent topological structure of data. Despite the integration of Graph Neural Networks (GNNs) into MVC, their input graph structures remain susceptible to noise interference. Methods based on Multi-view Graph Refinement (MGRC) also have limitations such as insufficient consideration of cross-view consistency, difficulty in handling hard-to-distinguish samples in the feature space, and disjointed optimization processes caused by graph construction algorithms. To address these issues, a Multi-View Clustering method via a Fusion-Consensus Graph Convolutional Network (MCFCN) is proposed. The network learns the consensus graph of multi-view data in an end-to-end manner and learns effective consensus representations through a view feature fusion model and a Unified Graph Structure Adapter (UGA). It designs Similarity Matrix Alignment Loss (SMAL) and Feature Representation Alignment Loss (FRAL). With the guidance of consensus, it optimizes view-specific graphs, preserves cross-view topological consistency, promotes the construction of intra-class edges, and realizes effective consensus representation learning with the help of GCN to improve clustering performance. MCFCN demonstrates state-of-the-art performance on eight multi-view benchmark datasets, and its effectiveness is verified by extensive qualitative and quantitative implementations. The code will be provided at https://github.com/texttao/MCFCN.

Method: Split text into segments, generate corresponding images using pre-trained text-to-image generator, incorporate auxiliary objectives for text-image and image-label mutual information, use graph neural network with heuristic image relationships for feature fusion.

Result: Extensive empirical results validate the effectiveness of RETSIMD for multimodal misinformation detection.

Conclusion: The proposed RETSIMD method effectively leverages text modality dominance in misinformation detection through text-to-image augmentation and graph-based feature fusion.

Abstract: Multimodal Misinformation Detection (MMD) refers to the task of detecting social media posts involving misinformation, where the post often contains text and image modalities. However, by observing the MMD posts, we hold that the text modality may be much more informative than the image modality because the text generally describes the whole event/story of the current post but the image often presents partial scenes only. Our preliminary empirical results indicate that the image modality exactly contributes less to MMD. Upon this idea, we propose a new MMD method named RETSIMD. Specifically, we suppose that each text can be divided into several segments, and each text segment describes a partial scene that can be presented by an image. Accordingly, we split the text into a sequence of segments, and feed these segments into a pre-trained text-to-image generator to augment a sequence of images. We further incorporate two auxiliary objectives concerning text-image and image-label mutual information, and further post-train the generator over an auxiliary text-to-image generation benchmark dataset. Additionally, we propose a graph structure by defining three heuristic relationships between images, and use a graph neural network to generate the fused features. Extensive empirical results validate the effectiveness of RETSIMD.

Method: Combines task-aware safe pruning (Taylor-based channel importance with gradient conflict penalty) and task head-agnostic distillation that transfers intermediate backbone and encoder features from teacher to student model.

Result: 32.7% parameter reduction; segmentation shows negligible accuracy loss; detection: -1.2% Recall, -1.8% mAP50; runs at 32.7 FPS in real-time on BDD100K dataset.

Conclusion: Combining pruning and knowledge distillation provides an effective compression solution for multi-task panoptic perception in autonomous driving systems.

Abstract: Autonomous driving systems rely on panoptic perception to jointly handle object detection, drivable area segmentation, and lane line segmentation. Although multi-task learning is an effective way to integrate these tasks, its increasing model parameters and complexity make deployment on on-board devices difficult. To address this challenge, we propose a multi-task model compression framework that combines task-aware safe pruning with feature-level knowledge distillation. Our safe pruning strategy integrates Taylor-based channel importance with gradient conflict penalty to keep important channels while removing redundant and conflicting channels. To mitigate performance degradation after pruning, we further design a task head-agnostic distillation method that transfers intermediate backbone and encoder features from a teacher to a student model as guidance. Experiments on the BDD100K dataset demonstrate that our compressed model achieves a 32.7% reduction in parameters while segmentation performance shows negligible accuracy loss and only a minor decrease in detection (-1.2% for Recall and -1.8% for mAP50) compared to the teacher. The compressed model still runs at 32.7 FPS in real-time. These results show that combining pruning and knowledge distillation provides an effective compression solution for multi-task panoptic perception.

Method: Proposes two key components: CLIP-Guided Lexical Certainty Adapter (CLIP-LCA) that masks high-certainty keywords to force spatial understanding, and Dimension-Decoupled Module (D2M) that separates 2D/3D textual features to guide corresponding visual features.

Result: Achieves state-of-the-art performance on Mono3DRefer dataset across all metrics, with significant +13.54% improvement in the challenging Far(Acc@0.5) scenario.

Conclusion: The proposed framework effectively addresses key limitations in monocular 3D visual grounding by enhancing spatial understanding through keyword masking and eliminating cross-dimensional interference through feature decoupling.

Abstract: Monocular 3D Visual Grounding (Mono3DVG) is an emerging task that locates 3D objects in RGB images using text descriptions with geometric cues. However, existing methods face two key limitations. Firstly, they often over-rely on high-certainty keywords that explicitly identify the target object while neglecting critical spatial descriptions. Secondly, generalized textual features contain both 2D and 3D descriptive information, thereby capturing an additional dimension of details compared to singular 2D or 3D visual features. This characteristic leads to cross-dimensional interference when refining visual features under text guidance. To overcome these challenges, we propose Mono3DVG-EnSD, a novel framework that integrates two key components: the CLIP-Guided Lexical Certainty Adapter (CLIP-LCA) and the Dimension-Decoupled Module (D2M). The CLIP-LCA dynamically masks high-certainty keywords while retaining low-certainty implicit spatial descriptions, thereby forcing the model to develop a deeper understanding of spatial relationships in captions for object localization. Meanwhile, the D2M decouples dimension-specific (2D/3D) textual features from generalized textual features to guide corresponding visual features at same dimension, which mitigates cross-dimensional interference by ensuring dimensionally-consistent cross-modal interactions. Through comprehensive comparisons and ablation studies on the Mono3DRefer dataset, our method achieves state-of-the-art (SOTA) performance across all metrics. Notably, it improves the challenging Far(Acc@0.5) scenario by a significant +13.54%.

Method: Constructs a large-scale benchmark dataset and proposes FilletRec - a lightweight graph neural network using pose-invariant intrinsic geometric features like curvature to learn fundamental geometric patterns.

Result: FilletRec surpasses state-of-the-art methods in accuracy and generalization while using only 0.2%-5.4% of baseline model parameters, demonstrating high model efficiency.

Conclusion: The framework provides an end-to-end automated workflow from recognition to simplification, addressing key challenges in CAD model processing for CAE analysis.

Abstract: Automated recognition and simplification of fillet features in CAD models is critical for CAE analysis, yet it remains an open challenge. Traditional rule-based methods lack robustness, while existing deep learning models suffer from poor generalization and low accuracy on complex fillets due to their generic design and inadequate training data. To address these issues, this paper proposes an end-to-end, data-driven framework specifically for fillet features. We first construct and release a large-scale, diverse benchmark dataset for fillet recognition to address the inadequacy of existing data. Based on it, we propose FilletRec, a lightweight graph neural network. The core innovation of this network is its use of pose-invariant intrinsic geometric features, such as curvature, enabling it to learn more fundamental geometric patterns and thereby achieve high-precision recognition of complex geometric topologies. Experiments show that FilletRec surpasses state-of-the-art methods in both accuracy and generalization, while using only 0.2%-5.4% of the parameters of baseline models, demonstrating high model efficiency. Finally, the framework completes the automated workflow from recognition to simplification by integrating an effective geometric simplification algorithm.

Method: Uses hierarchical spatial encoders at multiple granularities, multi-temporal encoders for motion dynamics across time scales, and feature decomposition for task-specific optimization of appearance and motion reconstruction.

Result: Achieves 98.5%, 92.1%, and 77.9% frame-level AUCs on UCSD Ped2, CUHK Avenue, and ShanghaiTech datasets respectively, with 20.1G FLOPs and 45 FPS inference speed.

Conclusion: The M2S2L framework provides robust anomaly assessment while maintaining computational efficiency, making it suitable for practical surveillance deployment.

Abstract: Video anomaly detection (VAD) is an essential task in the image processing community with prospects in video surveillance, which faces fundamental challenges in balancing detection accuracy with computational efficiency. As video content becomes increasingly complex with diverse behavioral patterns and contextual scenarios, traditional VAD approaches struggle to provide robust assessment for modern surveillance systems. Existing methods either lack comprehensive spatial-temporal modeling or require excessive computational resources for real-time applications. In this regard, we present a Mamba-based multi-scale spatial-temporal learning (M2S2L) framework in this paper. The proposed method employs hierarchical spatial encoders operating at multiple granularities and multi-temporal encoders capturing motion dynamics across different time scales. We also introduce a feature decomposition mechanism to enable task-specific optimization for appearance and motion reconstruction, facilitating more nuanced behavioral modeling and quality-aware anomaly assessment. Experiments on three benchmark datasets demonstrate that M2S2L framework achieves 98.5%, 92.1%, and 77.9% frame-level AUCs on UCSD Ped2, CUHK Avenue, and ShanghaiTech respectively, while maintaining efficiency with 20.1G FLOPs and 45 FPS inference speed, making it suitable for practical surveillance deployment.

Method: Introduced Micro-OD benchmark with 252 images across 11 cell types, evaluated 8 VLMs under few-shot conditions, implemented hybrid FSOD pipeline combining detection head with VLM-based classifier, and tested variants with/without reasoning tokens.

Result: Zero-shot performance is weak due to domain gap, but few-shot support consistently improves detection with marginal gains after six shots. Models with reasoning tokens are better for end-to-end localization, while simpler variants work better for classifying pre-localized crops.

Conclusion: In-context adaptation provides a practical path for microscopy applications, and the benchmark offers a reproducible testbed for advancing open-vocabulary detection in biomedical imaging.

Abstract: Foundation vision-language models (VLMs) excel on natural images, but their utility for biomedical microscopy remains underexplored. In this paper, we investigate how in-context learning enables state-of-the-art VLMs to perform few-shot object detection when large annotated datasets are unavailable, as is often the case with microscopic images. We introduce the Micro-OD benchmark, a curated collection of 252 images specifically curated for in-context learning, with bounding-box annotations spanning 11 cell types across four sources, including two in-lab expert-annotated sets. We systematically evaluate eight VLMs under few-shot conditions and compare variants with and without implicit test-time reasoning tokens. We further implement a hybrid Few-Shot Object Detection (FSOD) pipeline that combines a detection head with a VLM-based few-shot classifier, which enhances the few-shot performance of recent VLMs on our benchmark. Across datasets, we observe that zero-shot performance is weak due to the domain gap; however, few-shot support consistently improves detection, with marginal gains achieved after six shots. We observe that models with reasoning tokens are more effective for end-to-end localization, whereas simpler variants are more suitable for classifying pre-localized crops. Our results highlight in-context adaptation as a practical path for microscopy, and our benchmark provides a reproducible testbed for advancing open-vocabulary detection in biomedical imaging.

Method: Pre-trains feature extractor using contrastive loss with limited data, freezes it during streaming stage, and replaces dense classification layer with relation module network.

Result: Outperforms state-of-the-art methods on three public datasets, significantly reduces resource consumption for training while maintaining high performance, and improves data efficiency by reducing labeled data requirements.

Conclusion: PTRN-HAR successfully adapts PTM-based OCL to sensor-based HAR, providing an efficient and data-effective solution for continual learning in human activity recognition applications.

Abstract: Machine learning models for sensor-based human activity recognition (HAR) are expected to adapt post-deployment to recognize new activities and different ways of performing existing ones. To address this need, Online Continual Learning (OCL) mechanisms have been proposed, allowing models to update their knowledge incrementally as new data become available while preserving previously acquired information. However, existing OCL approaches for sensor-based HAR are computationally intensive and require extensive labeled samples to represent new changes. Recently, pre-trained model-based (PTM-based) OCL approaches have shown significant improvements in performance and efficiency for computer vision applications. These methods achieve strong generalization capabilities by pre-training complex models on large datasets, followed by fine-tuning on downstream tasks for continual learning. However, applying PTM-based OCL approaches to sensor-based HAR poses significant challenges due to the inherent heterogeneity of HAR datasets and the scarcity of labeled data in post-deployment scenarios. This paper introduces PTRN-HAR, the first successful application of PTM-based OCL to sensor-based HAR. Unlike prior PTM-based OCL approaches, PTRN-HAR pre-trains the feature extractor using contrastive loss with a limited amount of data. This extractor is then frozen during the streaming stage. Furthermore, it replaces the conventional dense classification layer with a relation module network. Our design not only significantly reduces the resource consumption required for model training while maintaining high performance, but also improves data efficiency by reducing the amount of labeled data needed for effective continual learning, as demonstrated through experiments on three public datasets, outperforming the state-of-the-art. The code can be found here: https://anonymous.4open.science/r/PTRN-HAR-AF60/

Method: Designs a perception-oriented approach to quantify frame-wise temporal inconsistency, introduces Inconsistency Highlighted Spatial Module to localize inconsistent regions at multiple scales, and develops Inconsistency Guided Temporal Module with progressive temporal feature aggregation including consistency-aware fusion and informative filtering stages.

Result: Extensive experiments on both single-frame and multi-frame SR video scenarios demonstrate that the method significantly outperforms state-of-the-art VQA approaches.

Conclusion: Temporal inconsistency plays a critical role in guiding quality assessment of super-resolution videos, and the proposed TIG-SVQA framework effectively addresses this by explicitly modeling and leveraging temporal inconsistency for improved video quality assessment.

Abstract: As super-resolution (SR) techniques introduce unique distortions that fundamentally differ from those caused by traditional degradation processes (e.g., compression), there is an increasing demand for specialized video quality assessment (VQA) methods tailored to SR-generated content. One critical factor affecting perceived quality is temporal inconsistency, which refers to irregularities between consecutive frames. However, existing VQA approaches rarely quantify this phenomenon or explicitly investigate its relationship with human perception. Moreover, SR videos exhibit amplified inconsistency levels as a result of enhancement processes. In this paper, we propose \textit{Temporal Inconsistency Guidance for Super-resolution Video Quality Assessment (TIG-SVQA)} that underscores the critical role of temporal inconsistency in guiding the quality assessment of SR videos. We first design a perception-oriented approach to quantify frame-wise temporal inconsistency. Based on this, we introduce the Inconsistency Highlighted Spatial Module, which localizes inconsistent regions at both coarse and fine scales. Inspired by the human visual system, we further develop an Inconsistency Guided Temporal Module that performs progressive temporal feature aggregation: (1) a consistency-aware fusion stage in which a visual memory capacity block adaptively determines the information load of each temporal segment based on inconsistency levels, and (2) an informative filtering stage for emphasizing quality-related features. Extensive experiments on both single-frame and multi-frame SR video scenarios demonstrate that our method significantly outperforms state-of-the-art VQA approaches. The code is publicly available at https://github.com/Lighting-YXLI/TIG-SVQA-main.

Method: Used adaptive structural learning of RBM/DBN with neuron/layer generation algorithms, Teacher-Student ensemble learning, and lightweight implementation on edge devices.

Result: Detection accuracy improved from 40.0% to 89.0% on average across seven major cities, with successful application to landslide-affected road detection.

Conclusion: The adaptive DBN with ensemble learning provides effective road network detection and has practical applications in disaster response scenarios.

Abstract: An adaptive structural learning method of Restricted Boltzmann Machine (RBM) and Deep Belief Network (DBN) has been developed as one of prominent deep learning models. The neuron generation-annihilation algorithm in RBM and layer generation algorithm in DBN make an optimal network structure for given input during the learning. In this paper, our model is applied to an automatic recognition method of road network system, called RoadTracer. RoadTracer can generate a road map on the ground surface from aerial photograph data. A novel method of RoadTracer using the Teacher-Student based ensemble learning model of Adaptive DBN is proposed, since the road maps contain many complicated features so that a model with high representation power to detect should be required. The experimental results showed the detection accuracy of the proposed model was improved from 40.0% to 89.0% on average in the seven major cities among the test dataset. In addition, we challenged to apply our method to the detection of available roads when landslide by natural disaster is occurred, in order to rapidly obtain a way of transportation. For fast inference, a small size of the trained model was implemented on a small embedded edge device as lightweight deep learning. We reported the detection results for the satellite image before and after the rainfall disaster in Japan.

Method: Proposes SM3Det with grid-level sparse MoE backbone for joint knowledge learning while preserving modality-specific features, plus consistency and synchronization optimization with dynamic learning rate adjustment.

Result: Extensive experiments show SM3Det outperforms specialized models on individual datasets, demonstrating effectiveness and generalizability across different modalities and tasks.

Conclusion: SM3Det successfully addresses multi-modal modeling trade-offs and multi-task optimization complexities, providing a unified solution for versatile remote sensing object detection scenarios.

Abstract: With the rapid advancement of remote sensing technology, high-resolution multi-modal imagery is now more widely accessible. Conventional Object detection models are trained on a single dataset, often restricted to a specific imaging modality and annotation format. However, such an approach overlooks the valuable shared knowledge across multi-modalities and limits the model’s applicability in more versatile scenarios. This paper introduces a new task called Multi-Modal Datasets and Multi-Task Object Detection (M2Det) for remote sensing, designed to accurately detect horizontal or oriented objects from any sensor modality. This task poses challenges due to 1) the trade-offs involved in managing multi-modal modelling and 2) the complexities of multi-task optimization. To address these, we establish a benchmark dataset and propose a unified model, SM3Det (Single Model for Multi-Modal datasets and Multi-Task object Detection). SM3Det leverages a grid-level sparse MoE backbone to enable joint knowledge learning while preserving distinct feature representations for different modalities. Furthermore, it integrates a consistency and synchronization optimization strategy using dynamic learning rate adjustment, allowing it to effectively handle varying levels of learning difficulty across modalities and tasks. Extensive experiments demonstrate SM3Det’s effectiveness and generalizability, consistently outperforming specialized models on individual datasets. The code is available at https://github.com/zcablii/SM3Det.

Method: Used participant-rated SVI data and semantic image segmentation to train ML model predicting perceived attractiveness, then compared predictions to PPGIS-identified locations using strict and moderate agreement criteria, analyzing contextual variables like noise, traffic, and land use.

Result: Partial alignment between datasets: 67% agreement for attractive places and 77% for unattractive places with moderate threshold, but only 27% and 29% respectively with strict threshold. Non-visual cues significantly contributed to mismatches.

Conclusion: SVI offers scalable visual proxy for urban perception but cannot fully substitute PPGIS’s experiential richness. Both methods are valuable for different purposes, requiring integrated approach to holistically capture urban perceptions.

Abstract: As digital tools increasingly shape spatial planning practices, understanding how different data sources reflect human experiences of urban environments is essential. Street View Imagery (SVI) and Public Participation GIS (PPGIS) represent two prominent approaches for capturing place-based perceptions that can support urban planning decisions, yet their comparability remains underexplored. This study investigates the alignment between SVI-based perceived attractiveness and residents’ reported experiences gathered via a city-wide PPGIS survey in Helsinki, Finland. Using participant-rated SVI data and semantic image segmentation, we trained a machine learning model to predict perceived attractiveness based on visual features. We compared these predictions to PPGIS-identified locations marked as attractive or unattractive, calculating agreement using two sets of strict and moderate criteria. Our findings reveal only partial alignment between the two datasets. While agreement (with a moderate threshold) reached 67% for attractive and 77% for unattractive places, agreement (with a strict threshold) dropped to 27% and 29%, respectively. By analysing a range of contextual variables, including noise, traffic, population presence, and land use, we found that non-visual cues significantly contributed to mismatches. The model failed to account for experiential dimensions such as activity levels and environmental stressors that shape perceptions but are not visible in images. These results suggest that while SVI offers a scalable and visual proxy for urban perception, it cannot fully substitute the experiential richness captured through PPGIS. We argue that both methods are valuable but serve different purposes; therefore, a more integrated approach is needed to holistically capture how people perceive urban environments.

Method: Uses Object-Perceptual Adaptation to modify input samples and In-Context Adaptation for feature adjustment, plus Discrimination Perception Transfer for knowledge distillation.

Result: Performs well on three in-distribution and three out-of-distribution datasets with fewer learnable parameters.

Conclusion: DVA effectively guides frozen pre-trained models for FGIR through collaborative adaptation, maintaining generalization while being parameter-efficient.

Abstract: Fine-Grained Image Retrieval~(FGIR) faces challenges in learning discriminative visual representations to retrieve images with similar fine-grained features. Current leading FGIR solutions typically follow two regimes: enforce pairwise similarity constraints in the semantic embedding space, or incorporate a localization sub-network to fine-tune the entire model. However, such two regimes tend to overfit the training data while forgetting the knowledge gained from large-scale pre-training, thus reducing their generalization ability. In this paper, we propose a Dual-Vision Adaptation (DVA) approach for FGIR, which guides the frozen pre-trained model to perform FGIR through collaborative sample and feature adaptation. Specifically, we design Object-Perceptual Adaptation, which modifies input samples to help the pre-trained model perceive critical objects and elements within objects that are helpful for category prediction. Meanwhile, we propose In-Context Adaptation, which introduces a small set of parameters for feature adaptation without modifying the pre-trained parameters. This makes the FGIR task using these adjusted features closer to the task solved during the pre-training. Additionally, to balance retrieval efficiency and performance, we propose Discrimination Perception Transfer to transfer the discriminative knowledge in the object-perceptual adaptation to the image encoder using the knowledge distillation mechanism. Extensive experiments show that DVA has fewer learnable parameters and performs well on three in-distribution and three out-of-distribution fine-grained datasets.

Method: Proposes C3-Diff framework with: 1) refined contrastive learning to extract modal-invariant and content-invariant features, 2) noise-based information augmentation on feature hyperspheres to overcome low sequencing sensitivity, and 3) dynamic cross-modal imputation training to mitigate data scarcity.

Result: Significant improvements over competing methods on four public datasets. Effective performance on downstream tasks including cell type localization, gene expression correlation, and single-cell-level gene expression prediction.

Conclusion: C3-Diff successfully enhances spatial transcriptomics resolution and promotes AI-enhanced biotechnology for biomedical research and clinical applications.

Abstract: The rapid advancement of spatial transcriptomics (ST), i.e., spatial gene expressions, has made it possible to measure gene expression within original tissue, enabling us to discover molecular mechanisms. However, current ST platforms frequently suffer from low resolution, limiting the in-depth understanding of spatial gene expression. Super-resolution approaches promise to enhance ST maps by integrating histology images with gene expressions of profiled tissue spots. However, it remains a challenge to model the interactions between histology images and gene expressions for effective ST enhancement. This study presents a cross-modal cross-content contrastive diffusion framework, called C3-Diff, for ST enhancement with histology images as guidance. In C3-Diff, we firstly analyze the deficiency of traditional contrastive learning paradigm, which is then refined to extract both modal-invariant and content-invariant features of ST maps and histology images. Further, to overcome the problem of low sequencing sensitivity in ST maps, we perform nosing-based information augmentation on the surface of feature unit hypersphere. Finally, we propose a dynamic cross-modal imputation-based training strategy to mitigate ST data scarcity. We tested C3-Diff by benchmarking its performance on four public datasets, where it achieves significant improvements over competing methods. Moreover, we evaluate C3-Diff on downstream tasks of cell type localization, gene expression correlation and single-cell-level gene expression prediction, promoting AI-enhanced biotechnology for biomedical research and clinical applications. Codes are available at https://github.com/XiaofeiWang2018/C3-Diff.

Method: Generate text-rich images using T2I diffusion model, animate them into videos using text-agnostic I2V model, then use these synthetic video-prompt pairs to fine-tune Wan2.1 T2V model without architectural changes.

Result: Improved short-text legibility and temporal consistency, with emerging structural priors for longer text generation.

Conclusion: Curated synthetic data and weak supervision provide a practical path to enhance textual fidelity in T2V generation.

Abstract: While Text-To-Video (T2V) models have advanced rapidly, they continue to struggle with generating legible and coherent text within videos. In particular, existing models often fail to render correctly even short phrases or words and previous attempts to address this problem are computationally expensive and not suitable for video generation. In this work, we investigate a lightweight approach to improve T2V diffusion models using synthetic supervision. We first generate text-rich images using a text-to-image (T2I) diffusion model, then animate them into short videos using a text-agnostic image-to-video (I2v) model. These synthetic video-prompt pairs are used to fine-tune Wan2.1, a pre-trained T2V model, without any architectural changes. Our results show improvement in short-text legibility and temporal consistency with emerging structural priors for longer text. These findings suggest that curated synthetic data and weak supervision offer a practical path toward improving textual fidelity in T2V generation.

Method: Proposes Modality Capability Enhancement (MCE) with two components: Learning Capability Enhancement (LCE) uses multi-level factors to dynamically balance modality learning progress, and Representation Capability Enhancement (RCE) improves feature semantics through subset prediction and cross-modal completion tasks.

Result: Comprehensive evaluations on four multi-modal benchmarks show MCE consistently outperforms state-of-the-art methods under various missing modality configurations.

Conclusion: MCE effectively addresses the challenges of imbalanced missing modalities by simultaneously enhancing learning progress balancing and feature representation quality, demonstrating superior performance across multiple benchmarks.

Abstract: Multi-modal learning has made significant advances across diverse pattern recognition applications. However, handling missing modalities, especially under imbalanced missing rates, remains a major challenge. This imbalance triggers a vicious cycle: modalities with higher missing rates receive fewer updates, leading to inconsistent learning progress and representational degradation that further diminishes their contribution. Existing methods typically focus on global dataset-level balancing, often overlooking critical sample-level variations in modality utility and the underlying issue of degraded feature quality. We propose Modality Capability Enhancement (MCE) to tackle these limitations. MCE includes two synergistic components: i) Learning Capability Enhancement (LCE), which introduces multi-level factors to dynamically balance modality-specific learning progress, and ii) Representation Capability Enhancement (RCE), which improves feature semantics and robustness through subset prediction and cross-modal completion tasks. Comprehensive evaluations on four multi-modal benchmarks show that MCE consistently outperforms state-of-the-art methods under various missing configurations. The final published version is now available at https://doi.org/10.1016/j.patcog.2025.112591. Our code is available at https://github.com/byzhaoAI/MCE.

Method: A supervising ANN observes activation patterns of a CNN ensemble to detect high uncertainty, with memory storage of past performance and active learning that requests human help in uncertain conditions.

Result: The system can identify when its predictions are unreliable and proactively seek human assistance, demonstrating a form of self-awareness in narrow ML tasks.

Conclusion: Self-awareness mechanisms can be implemented even in narrow ML algorithms, enabling reflection on performance uncertainty and agency through active learning for improved reliability.

Abstract: Reflection on one’s thought process and making corrections to it if there exists dissatisfaction in its performance is, perhaps, one of the essential traits of intelligence. However, such high-level abstract concepts mandatory for Artificial General Intelligence can be modelled even at the low level of narrow Machine Learning algorithms. Here, we present the self-awareness mechanism emulation in the form of a supervising artificial neural network (ANN) observing patterns in activations of another underlying ANN in a search for indications of the high uncertainty of the underlying ANN and, therefore, the trustworthiness of its predictions. The underlying ANN is a convolutional neural network (CNN) ensemble employed for face recognition and facial expression tasks. The self-awareness ANN has a memory region where its past performance information is stored, and its learnable parameters are adjusted during the training to optimize the performance. The trustworthiness verdict triggers the active learning mode, giving elements of agency to the machine learning algorithm that asks for human help in high uncertainty and confusion conditions.

Method: Proposes a diffusion-based pipeline that incorporates 3D facial latent features during training and conditions the denoising process on both identity embeddings and facial landmarks.

Result: Outperforms prior methods on CelebA, FFHQ, and CelebV-Text datasets in preserving source identity while maintaining target pose and expression, validated through biometric-style evaluation and user study.

Conclusion: DiffSwap++ demonstrates that incorporating 3D-aware representations significantly enhances face swapping quality and identity preservation in diffusion models.

Abstract: Diffusion-based approaches have recently achieved strong results in face swapping, offering improved visual quality over traditional GAN-based methods. However, even state-of-the-art models often suffer from fine-grained artifacts and poor identity preservation, particularly under challenging poses and expressions. A key limitation of existing approaches is their failure to meaningfully leverage 3D facial structure, which is crucial for disentangling identity from pose and expression. In this work, we propose DiffSwap++, a novel diffusion-based face-swapping pipeline that incorporates 3D facial latent features during training. By guiding the generation process with 3D-aware representations, our method enhances geometric consistency and improves the disentanglement of facial identity from appearance attributes. We further design a diffusion architecture that conditions the denoising process on both identity embeddings and facial landmarks, enabling high-fidelity and identity-preserving face swaps. Extensive experiments on CelebA, FFHQ, and CelebV-Text demonstrate that DiffSwap++ outperforms prior methods in preserving source identity while maintaining target pose and expression. Additionally, we introduce a biometric-style evaluation and conduct a user study to further validate the realism and effectiveness of our approach. Code will be made publicly available at https://github.com/WestonBond/DiffSwapPP

Method: Clone classification head into parallel sigmoid branch, freeze original softmax head, fine-tune only sigmoid branch with class-balanced binary supervision, and generate evidence maps from sigmoid branch.

Result: Improved explanation fidelity and consistent Top-1 Localization gains on fine-grained tasks and WSOL benchmarks without classification accuracy drop.

Conclusion: The dual-branch sigmoid approach effectively decouples localization from classification, fixing fundamental CAM distortions while maintaining recognition performance.

Abstract: Class Activation Mapping (CAM) and its extensions have become indispensable tools for visualizing the evidence behind deep network predictions. However, by relying on a final softmax classifier, these methods suffer from two fundamental distortions: additive logit shifts that arbitrarily bias importance scores, and sign collapse that conflates excitatory and inhibitory features. We propose a simple, architecture-agnostic dual-branch sigmoid head that decouples localization from classification. Given any pretrained model, we clone its classification head into a parallel branch ending in per-class sigmoid outputs, freeze the original softmax head, and fine-tune only the sigmoid branch with class-balanced binary supervision. At inference, softmax retains recognition accuracy, while class evidence maps are generated from the sigmoid branch – preserving both magnitude and sign of feature contributions. Our method integrates seamlessly with most CAM variants and incurs negligible overhead. Extensive evaluations on fine-grained tasks (CUB-200-2011, Stanford Cars) and WSOL benchmarks (ImageNet-1K, OpenImages30K) show improved explanation fidelity and consistent Top-1 Localization gains – without any drop in classification accuracy. Code is available at https://github.com/finallyupper/beyond-softmax.

Method: Uses MedGemma foundation model with SigLIP-derived vision encoder to encode X-ray images into embeddings, followed by a lightweight multilayer perceptron for binary classification. Employs selective encoder block unfreezing for efficient domain adaptation.

Result: The MedGemma-driven classifier exhibits strong performance, exceeding conventional convolutional and autoencoder-based metrics, with enhanced generalization and optimized feature engineering.

Conclusion: MedGemma-powered classification systems can advance clinical radiograph triage by providing scalable and accurate abnormality detection, with potential for broader applications in automated medical image analysis.

Abstract: This paper proposes a MedGemma-based framework for automatic abnormality detection in musculoskeletal radiographs. Departing from conventional autoencoder and neural network pipelines, the proposed method leverages the MedGemma foundation model, incorporating a SigLIP-derived vision encoder pretrained on diverse medical imaging modalities. Preprocessed X-ray images are encoded into high-dimensional embeddings using the MedGemma vision backbone, which are subsequently passed through a lightweight multilayer perceptron for binary classification. Experimental assessment reveals that the MedGemma-driven classifier exhibits strong performance, exceeding conventional convolutional and autoencoder-based metrics. Additionally, the model leverages MedGemma’s transfer learning capabilities, enhancing generalization and optimizing feature engineering. The integration of a modern medical foundation model not only enhances representation learning but also facilitates modular training strategies such as selective encoder block unfreezing for efficient domain adaptation. The findings suggest that MedGemma-powered classification systems can advance clinical radiograph triage by providing scalable and accurate abnormality detection, with potential for broader applications in automated medical image analysis. Keywords: Google MedGemma, MURA, Medical Image, Classification.

Method: Two complementary approaches: 1) Differentiable calibration objective using Smooth Expected Calibration Error for classifier fine-tuning, 2) Enhanced sampling guidance methods including tilted sampling with batch-level reweighting, adaptive entropy-regularized sampling, and novel f-divergence-based sampling strategy.

Result: Achieved FID of 2.13 on ImageNet 128x128 using ResNet-101 classifier, improving upon existing classifier-guided diffusion methods while requiring no diffusion model retraining.

Conclusion: Principled calibration and divergence-aware sampling provide practical and effective improvements for classifier-guided diffusion models, addressing the overconfidence issue and enhancing conditional image generation.

Abstract: Classifier-guided diffusion models have emerged as a powerful approach for conditional image generation, but they suffer from overconfident predictions during early denoising steps, causing the guidance gradient to vanish. This paper introduces two complementary contributions to address this issue. First, we propose a differentiable calibration objective based on the Smooth Expected Calibration Error (Smooth ECE), which improves classifier calibration with minimal fine-tuning and yields measurable improvements in Frechet Inception Distance (FID). Second, we develop enhanced sampling guidance methods that operate on off-the-shelf classifiers without requiring retraining. These include tilted sampling with batch-level reweighting, adaptive entropy-regularized sampling to preserve diversity, and a novel f-divergence-based sampling strategy that strengthens class-consistent guidance while maintaining mode coverage. Experiments on ImageNet 128x128 demonstrate that our divergence-regularized guidance achieves an FID of 2.13 using a ResNet-101 classifier, improving upon existing classifier-guided diffusion methods while requiring no diffusion model retraining. The results show that principled calibration and divergence-aware sampling provide practical and effective improvements for classifier-guided diffusion.

Method: Developed a real-time monitoring system that acquires and reconstructs the growing part during manufacturing, then compares it with a near-net reference model to detect shape deviations.

Result: The system enables early identification of shape inconsistencies and allows segmentation and tracking of deviation regions for timely intervention.

Conclusion: Real-time shape deviation monitoring paves the way for timely compensation and intervention to achieve consistent part quality in high deposition rate additive manufacturing processes.

Abstract: Additive manufacturing (AM) is an emerging digital manufacturing technology to produce complex and freeform objects through a layer-wise deposition. High deposition rate robotic AM (HDRRAM) processes, such as cold spray additive manufacturing (CSAM), offer significantly increased build speeds by delivering large volumes of material per unit time. However, maintaining shape accuracy remains a critical challenge, particularly due to process instabilities in current open-loop systems. Detecting these deviations as they occur is essential to prevent error propagation, ensure part quality, and minimize post-processing requirements. This study presents a real-time monitoring system to acquire and reconstruct the growing part and directly compares it with a near-net reference model to detect the shape deviation during the manufacturing process. The early identification of shape inconsistencies, followed by segmenting and tracking each deviation region, paves the way for timely intervention and compensation to achieve consistent part quality.

Method: Theoretical establishment of SDS as a simplified Schrödinger Bridge instance, then introducing TraCe framework that explicitly constructs diffusion bridges from current renderings to text-conditioned targets and trains LoRA-adapted models on trajectory score dynamics.

Result: TraCe consistently achieves superior quality and fidelity compared to state-of-the-art techniques in comprehensive experiments.

Conclusion: The TraCe framework successfully addresses artifact issues in text-to-3D generation by leveraging optimal transport trajectory learning through the Schrödinger Bridge formulation.

Abstract: Recent advancements in optimization-based text-to-3D generation heavily rely on distilling knowledge from pre-trained text-to-image diffusion models using techniques like Score Distillation Sampling (SDS), which often introduce artifacts such as over-saturation and over-smoothing into the generated 3D assets. In this paper, we address this essential problem by formulating the generation process as learning an optimal, direct transport trajectory between the distribution of the current rendering and the desired target distribution, thereby enabling high-quality generation with smaller Classifier-free Guidance (CFG) values. At first, we theoretically establish SDS as a simplified instance of the Schrödinger Bridge framework. We prove that SDS employs the reverse process of an Schrödinger Bridge, which, under specific conditions (e.g., a Gaussian noise as one end), collapses to SDS’s score function of the pre-trained diffusion model. Based upon this, we introduce Trajectory-Centric Distillation (TraCe), a novel text-to-3D generation framework, which reformulates the mathematically trackable framework of Schrödinger Bridge to explicitly construct a diffusion bridge from the current rendering to its text-conditioned, denoised target, and trains a LoRA-adapted model on this trajectory’s score dynamics for robust 3D optimization. Comprehensive experiments demonstrate that TraCe consistently achieves superior quality and fidelity to state-of-the-art techniques.

Method: Three-level framework: Action-Unit Parser for segmentation and pose representations, Motion Parser for spatial-temporal feature learning, and Condition Parser for external factors like water splash. Includes Weight-Adjust Scoring Module for diverse action types.

Result: Extensive evaluations on large-scale diving sports datasets show state-of-the-art performance in both action segmentation and scoring tasks.

Conclusion: The proposed multi-level motion parsing framework effectively captures nuanced pose variations and external conditions for superior action quality assessment in sports.

Abstract: Human pose serves as a cornerstone of action quality assessment (AQA), where subtle spatial-temporal variations in pose often distinguish excellence from mediocrity. In high-level competitions, these nuanced differences become decisive factors in scoring. In this paper, we propose a novel multi-level motion parsing framework for AQA based on enhanced spatial-temporal pose features. On the first level, the Action-Unit Parser is designed with the help of pose extraction to achieve precise action segmentation and comprehensive local-global pose representations. On the second level, Motion Parser is used by spatial-temporal feature learning to capture pose changes and appearance details for each action-unit. Meanwhile, some special conditions other than body-related will impact action scoring, like water splash in diving. In this work, we design an additional Condition Parser to offer users more flexibility in their choices. Finally, Weight-Adjust Scoring Module is introduced to better accommodate the diverse requirements of various action types and the multi-scale nature of action-units. Extensive evaluations on large-scale diving sports datasets demonstrate that our multi-level motion parsing framework achieves state-of-the-art performance in both action segmentation and action scoring tasks.

Method: Collaborative Direct Preference Optimization (C-DPO) encodes users as nodes in a dynamic preference graph, learns embeddings via graph neural networks, and integrates these into a novel DPO objective that jointly optimizes for individual alignment and neighborhood coherence.

Result: Comprehensive experiments including user studies and quantitative benchmarks show the method consistently outperforms baselines in generating edits aligned with user preferences.

Conclusion: The framework successfully enables personalized image editing in diffusion models by leveraging collaborative signals and graph-based user representations.

Abstract: Text-to-image (T2I) diffusion models have made remarkable strides in generating and editing high-fidelity images from text. Yet, these models remain fundamentally generic, failing to adapt to the nuanced aesthetic preferences of individual users. In this work, we present the first framework for personalized image editing in diffusion models, introducing Collaborative Direct Preference Optimization (C-DPO), a novel method that aligns image edits with user-specific preferences while leveraging collaborative signals from like-minded individuals. Our approach encodes each user as a node in a dynamic preference graph and learns embeddings via a lightweight graph neural network, enabling information sharing across users with overlapping visual tastes. We enhance a diffusion model’s editing capabilities by integrating these personalized embeddings into a novel DPO objective, which jointly optimizes for individual alignment and neighborhood coherence. Comprehensive experiments, including user studies and quantitative benchmarks, demonstrate that our method consistently outperforms baselines in generating edits that are aligned with user preferences.

Method: Introduces a Convolutional Fully-Connected Capsule Network (CFC-CapsNet) with a new CFC layer that creates capsules differently, producing fewer but more powerful capsules.

Result: CFC-CapsNet achieves competitive accuracy, faster training and inference, and uses fewer parameters on CIFAR-10, SVHN, and Fashion-MNIST datasets compared to conventional CapsNet.

Conclusion: The CFC-CapsNet successfully addresses CapsNet’s limitations by creating more efficient capsules, making it more suitable for complex datasets and real applications.

Abstract: A Capsule Network (CapsNet) is a relatively new classifier and one of the possible successors of Convolutional Neural Networks (CNNs). CapsNet maintains the spatial hierarchies between the features and outperforms CNNs at classifying images including overlapping categories. Even though CapsNet works well on small-scale datasets such as MNIST, it fails to achieve a similar level of performance on more complicated datasets and real applications. In addition, CapsNet is slow compared to CNNs when performing the same task and relies on a higher number of parameters. In this work, we introduce Convolutional Fully-Connected Capsule Network (CFC-CapsNet) to address the shortcomings of CapsNet by creating capsules using a different method. We introduce a new layer (CFC layer) as an alternative solution to creating capsules. CFC-CapsNet produces fewer, yet more powerful capsules resulting in higher network accuracy. Our experiments show that CFC-CapsNet achieves competitive accuracy, faster training and inference and uses less number of parameters on the CIFAR-10, SVHN and Fashion-MNIST datasets compared to conventional CapsNet.

Method: Fuses V-JEPA 2’s contextual, predictive world dynamics with CoMotion’s explicit, occlusion-tolerant human pose data.

Result: Outperforms three baselines on InHARD and UCF-19-Y-OCC benchmarks, especially in complex, occlusive scenes.

Conclusion: Action recognition should be supported by spatial understanding rather than statistical pattern recognition.

Abstract: For embodied agents to effectively understand and interact within the world around them, they require a nuanced comprehension of human actions grounded in physical space. Current action recognition models, often relying on RGB video, learn superficial correlations between patterns and action labels, so they struggle to capture underlying physical interaction dynamics and human poses in complex scenes. We propose a model architecture that grounds action recognition in physical space by fusing two powerful, complementary representations: V-JEPA 2’s contextual, predictive world dynamics and CoMotion’s explicit, occlusion-tolerant human pose data. Our model is validated on both the InHARD and UCF-19-Y-OCC benchmarks for general action recognition and high-occlusion action recognition, respectively. Our model outperforms three other baselines, especially within complex, occlusive scenes. Our findings emphasize a need for action recognition to be supported by spatial understanding instead of statistical pattern recognition.

Method: Two feature learning methods using Laplacian and geodesic distances to capture intrinsic geometric properties, combined with thresholding techniques to select features indicative of defects.

Result: Numerical experiments and case studies demonstrate effective identification of different types of defects in complex shapes.

Conclusion: The proposed registration-free approach successfully monitors point cloud data without preprocessing, using intrinsic geometric features for defect detection.

Abstract: Modern sensing technologies have enabled the collection of unstructured point cloud data (PCD) of varying sizes, which are used to monitor the geometric accuracy of 3D objects. PCD are widely applied in advanced manufacturing processes, including additive, subtractive, and hybrid manufacturing. To ensure the consistency of analysis and avoid false alarms, preprocessing steps such as registration and mesh reconstruction are commonly applied prior to monitoring. However, these steps are error-prone, time-consuming and may introduce artifacts, potentially affecting monitoring outcomes. In this paper, we present a novel registration-free approach for monitoring PCD of complex shapes, eliminating the need for both registration and mesh reconstruction. Our proposal consists of two alternative feature learning methods and a common monitoring scheme. Feature learning methods leverage intrinsic geometric properties of the shape, captured via the Laplacian and geodesic distances. In the monitoring scheme, thresholding techniques are used to further select intrinsic features most indicative of potential out-of-control conditions. Numerical experiments and case studies highlight the effectiveness of the proposed approach in identifying different types of defects.

Method: Developed CULTIVate benchmark spanning 16 countries with 576 prompts and 19,000+ images, using an explainable descriptor-based evaluation framework across cultural dimensions (background, attire, objects, interactions) with four metrics for cultural alignment, hallucination, exaggerated elements, and diversity.

Result: Found systematic disparities: models perform better for global north countries than global south, with distinct failure modes across T2I systems. Human studies confirmed metrics correlate more strongly with human judgments than existing text-image metrics.

Conclusion: CULTIVate provides an effective framework for measuring cultural faithfulness in T2I models, revealing significant biases and offering improved evaluation metrics that better align with human perception of cultural representation.

Abstract: Text-to-image (T2I) diffusion models achieve impressive photorealism by training on large-scale web data, but models inherit cultural biases and fail to depict underrepresented regions faithfully. Existing cultural benchmarks focus mainly on object-centric categories (e.g., food, attire, and architecture), overlooking the social and daily activities that more clearly reflect cultural norms. Few metrics exist for measuring cultural faithfulness. We introduce CULTIVate, a benchmark for evaluating T2I models on cross-cultural activities (e.g., greetings, dining, games, traditional dances, and cultural celebrations). CULTIVate spans 16 countries with 576 prompts and more than 19,000 images, and provides an explainable descriptor-based evaluation framework across multiple cultural dimensions, including background, attire, objects, and interactions. We propose four metrics to measure cultural alignment, hallucination, exaggerated elements, and diversity. Our findings reveal systematic disparities: models perform better for global north countries than for the global south, with distinct failure modes across T2I systems. Human studies confirm that our metrics correlate more strongly with human judgments than existing text-image metrics.

Method: Developed VMDT platform to evaluate 7 text-to-video and 19 video-to-text models across five trustworthiness dimensions using systematic testing framework.

Result: Open-source T2V models fail to recognize harmful queries and generate harmful videos with higher unfairness than image models. V2T models show unfairness and privacy risks increase with scale, while hallucination and robustness improve but remain low overall.

Conclusion: Urgent need for more robust video foundation models; VMDT provides systematic framework for measuring and tracking trustworthiness progress in video modality.

Abstract: As foundation models become more sophisticated, ensuring their trustworthiness becomes increasingly critical; yet, unlike text and image, the video modality still lacks comprehensive trustworthiness benchmarks. We introduce VMDT (Video-Modal DecodingTrust), the first unified platform for evaluating text-to-video (T2V) and video-to-text (V2T) models across five key trustworthiness dimensions: safety, hallucination, fairness, privacy, and adversarial robustness. Through our extensive evaluation of 7 T2V models and 19 V2T models using VMDT, we uncover several significant insights. For instance, all open-source T2V models evaluated fail to recognize harmful queries and often generate harmful videos, while exhibiting higher levels of unfairness compared to image modality models. In V2T models, unfairness and privacy risks rise with scale, whereas hallucination and adversarial robustness improve – though overall performance remains low. Uniquely, safety shows no correlation with model size, implying that factors other than scale govern current safety levels. Our findings highlight the urgent need for developing more robust and trustworthy video foundation models, and VMDT provides a systematic framework for measuring and tracking progress toward this goal. The code is available at https://sunblaze-ucb.github.io/VMDT-page/.

Method: Compares screw combinations and employs 2D-3D alignment with screw CAD 3D models to accurately pair and estimate screw pose from dual C-arm views.

Result: Correct screw combination consistently outperforms incorrect pairings across all test cases, even before registration. After registration, correct combination further enhances alignment and significantly reduces projection error.

Conclusion: This approach shows promise for improving surgical outcomes in spinal procedures by providing reliable feedback on screw positioning.

Abstract: Accurate matching of pedicle screws in both anteroposterior (AP) and lateral (LAT) images is critical for successful spinal decompression and stabilization during surgery. However, establishing screw correspondence, especially in LAT views, remains a significant clinical challenge. This paper introduces a method to address pedicle screw correspondence and pose estimation from dual C-arm images. By comparing screw combinations, the approach demonstrates consistent accuracy in both pairing and registration tasks. The method also employs 2D-3D alignment with screw CAD 3D models to accurately pair and estimate screw pose from dual views. Our results show that the correct screw combination consistently outperforms incorrect pairings across all test cases, even prior to registration. After registration, the correct combination further enhances alignment between projections and images, significantly reducing projection error. This approach shows promise for improving surgical outcomes in spinal procedures by providing reliable feedback on screw positioning.

Method: Two-stage synthesis framework: (1) scale and (2) complexity, using VLMs and reasoning LLMs to generate CoT traces. Includes preference data and instruction prompts supporting offline/online RL.

Result: Qwen2.5-VL-7B finetuned on this data outperforms all open-data baselines across vision benchmarks, surpasses MiMo-VL-7B-RL on V* Bench, CV-Bench and MMStar-V, and shows positive transfer to text-only reasoning (MMLU-Pro) and audio reasoning (MMAU).

Conclusion: High-quality SFT data with non-linear reasoning traces is essential for effective online RL; staged offline RL matches online RL performance with less compute; careful SFT improves cross-modality transfer.

Abstract: Recent progress in multimodal reasoning has been driven largely by undisclosed datasets and proprietary data synthesis recipes, leaving open questions about how to systematically build large-scale, vision-centric reasoning datasets, particularly for tasks that go beyond visual math. In this work, we introduce a new reasoning data generation framework spanning diverse skills and levels of complexity with over 1M high-quality synthetic vision-centric questions. The dataset also includes preference data and instruction prompts supporting both offline and online RL. Our synthesis framework proceeds in two stages: (1) scale; and (2) complexity. Reasoning traces are then synthesized through a two-stage process that leverages VLMs and reasoning LLMs, producing CoT traces for VLMs that capture the richness and diverse cognitive behaviors found in frontier reasoning models. Remarkably, we show that finetuning Qwen2.5-VL-7B on our data outperforms all open-data baselines across all evaluated vision-centric benchmarks, and even surpasses strong closed-data models such as MiMo-VL-7B-RL on V* Bench, CV-Bench and MMStar-V. Perhaps most surprising, despite being entirely vision-centric, our data transfers positively to text-only reasoning (MMLU-Pro) and audio reasoning (MMAU), demonstrating its effectiveness. Similarly, despite not containing videos or embodied visual data, we observe notable gains when evaluating on a single-evidence embodied QA benchmark (NiEH). Finally, we use our data to analyze the entire VLM post-training pipeline. Our empirical analysis highlights that (i) SFT on high-quality data with non-linear reasoning traces is essential for effective online RL, (ii) staged offline RL matches online RL’s performance while reducing compute demands, and (iii) careful SFT on high quality data can substantially improve out-of-domain, cross-modality transfer.

Method: Analyzed training-set size, video duration, and augmentation schemes under three paradigms (fine-tuning, intermediate adaptation, multi-task training) across five SAM2 variants and multiple prompting modes, with six ultrasound-specific augmentations.

Result: Data scale and temporal context play more decisive roles than model architecture or initialization. Joint training offers efficient compromise between modality alignment and task specialization.

Conclusion: Provides empirical insights for developing efficient, data-aware adaptation pipelines for SAM2 in ultrasound video analysis, emphasizing data-centric approaches over architectural modifications.

Abstract: Ultrasound (US) video segmentation remains a challenging problem due to strong inter- and intra-dataset variability, motion artifacts, and limited annotated data. Although foundation models such as Segment Anything Model 2 (SAM2) demonstrate strong zero-shot and prompt-guided segmentation capabilities, their performance deteriorates substantially when transferred to medical imaging domains. Current adaptation studies mainly emphasize architectural modifications, while the influence of data characteristics and training regimes has not been systematically examined. In this study, we present a comprehensive, data-centric investigation of SAM2 adaptation for ultrasound video segmentation. We analyze how training-set size, video duration, and augmentation schemes affect adaptation performance under three paradigms: task-specific fine-tuning, intermediate adaptation, and multi-task joint training, across five SAM2 variants and multiple prompting modes. We further design six ultrasound-specific augmentations, assessing their effect relative to generic strategies. Experiments on three representative ultrasound datasets reveal that data scale and temporal context play a more decisive role than model architecture or initialization. Moreover, joint training offers an efficient compromise between modality alignment and task specialization. This work aims to provide empirical insights for developing efficient, data-aware adaptation pipelines for SAM2 in ultrasound video analysis.

Method: Proposed a second-order geometric attention (SOGA) mechanism modeled on Riemannian manifold using symmetric positive definitive representations, integrated into U-Net and nnU-Net backbones through skip connections.

Result: Achieved AP of 0.37 and AUC-ROC of 0.83 on PI-CAI dataset, outperforming baselines. On independent Prostate158 dataset, achieved AP of 0.37 and AUC-ROC of 0.75, confirming robust generalization.

Conclusion: The SOGA mechanism enables discriminative learned representations for prostate cancer detection, demonstrating superior performance and strong generalization capabilities across different datasets.

Abstract: The detection of clinically significant prostate cancer lesions (csPCa) from biparametric magnetic resonance imaging (bp-MRI) has emerged as a noninvasive imaging technique for improving accurate diagnosis. Nevertheless, the analysis of such images remains highly dependent on the subjective expert interpretation. Deep learning approaches have been proposed for csPCa lesions detection and segmentation, but they remain limited due to their reliance on extensively annotated datasets. Moreover, the high lesion variability across prostate zones poses additional challenges, even for expert radiologists. This work introduces a second-order geometric attention (SOGA) mechanism that guides a dedicated segmentation network, through skip connections, to detect csPCa lesions. The proposed attention is modeled on the Riemannian manifold, learning from symmetric positive definitive (SPD) representations. The proposed mechanism was integrated into standard U-Net and nnU-Net backbones, and was validated on the publicly available PI-CAI dataset, achieving an Average Precision (AP) of 0.37 and an Area Under the ROC Curve (AUC-ROC) of 0.83, outperforming baseline networks and attention-based methods. Furthermore, the approach was evaluated on the Prostate158 dataset as an independent test cohort, achieving an AP of 0.37 and an AUC-ROC of 0.75, confirming robust generalization and suggesting discriminative learned representations.

Method: Proposed a Graph-GRU temporal network that integrates graph-based spatial representations with temporal modeling using skeleton pose data extracted from video sequences.

Result: Achieved high accuracy on the AUTSL (Ankara University Turkish Sign Language) dataset, demonstrating the effectiveness of the approach.

Conclusion: The proposed pose-driven method shows strong potential for sign language understanding and provides a scalable framework for sign language recognition.

Abstract: This work presents an approach for recognizing isolated sign language gestures using skeleton-based pose data extracted from video sequences. A Graph-GRU temporal network is proposed to model both spatial and temporal dependencies between frames, enabling accurate classification. The model is trained and evaluated on the AUTSL (Ankara university Turkish sign language) dataset, achieving high accuracy. Experimental results demonstrate the effectiveness of integrating graph-based spatial representations with temporal modeling, providing a scalable framework for sign language recognition. The results of this approach highlight the potential of pose-driven methods for sign language understanding.

Method: Proposes TCSA-UDA with two key components: 1) Vision-language covariance cosine loss to align image encoder features with inter-class textual semantic relations, 2) Prototype alignment module that aligns class-wise pixel-level feature distributions across domains using high-level semantic prototypes.

Result: Extensive experiments on cardiac, abdominal, and brain tumor segmentation benchmarks show TCSA-UDA significantly reduces domain shift and consistently outperforms state-of-the-art UDA methods.

Conclusion: TCSA-UDA establishes a new paradigm for integrating language-driven semantics into domain-adaptive medical image analysis, effectively mitigating domain shifts across different imaging modalities.

Abstract: Unsupervised domain adaptation for medical image segmentation remains a significant challenge due to substantial domain shifts across imaging modalities, such as CT and MRI. While recent vision-language representation learning methods have shown promise, their potential in UDA segmentation tasks remains underexplored. To address this gap, we propose TCSA-UDA, a Text-driven Cross-Semantic Alignment framework that leverages domain-invariant textual class descriptions to guide visual representation learning. Our approach introduces a vision-language covariance cosine loss to directly align image encoder features with inter-class textual semantic relations, encouraging semantically meaningful and modality-invariant feature representations. Additionally, we incorporate a prototype alignment module that aligns class-wise pixel-level feature distributions across domains using high-level semantic prototypes. This mitigates residual category-level discrepancies and enhances cross-modal consistency. Extensive experiments on challenging cross-modality cardiac, abdominal, and brain tumor segmentation benchmarks demonstrate that our TCSA-UDA framework significantly reduces domain shift and consistently outperforms state-of-the-art UDA methods, establishing a new paradigm for integrating language-driven semantics into domain-adaptive medical image analysis.

Method: Curated a multi-center, multi-disease, multi-species dataset of 2,542 kidney WSIs; evaluated three SOTA models; developed human-in-the-loop data enrichment algorithms to distill predictions and enhance performance with minimal human effort.

Result: All three foundation models improved over baselines with fine-tuning using enriched data; interestingly, the baseline model with highest F1 score did not yield best segmentation outcomes after fine-tuning.

Conclusion: Establishes a benchmark for developing and deploying cell vision foundation models for real-world healthcare applications, demonstrating the effectiveness of human-in-the-loop data enrichment strategies.

Abstract: Training AI foundation models has emerged as a promising large-scale learning approach for addressing real-world healthcare challenges, including digital pathology. While many of these models have been developed for tasks like disease diagnosis and tissue quantification using extensive and diverse training datasets, their readiness for deployment on some arguably simplest tasks, such as nuclei segmentation within a single organ (e.g., the kidney), remains uncertain. This paper seeks to answer this key question, “How good are we?”, by thoroughly evaluating the performance of recent cell foundation models on a curated multi-center, multi-disease, and multi-species external testing dataset. Additionally, we tackle a more challenging question, “How can we improve?”, by developing and assessing human-in-the-loop data enrichment strategies aimed at enhancing model performance while minimizing the reliance on pixel-level human annotation. To address the first question, we curated a multicenter, multidisease, and multispecies dataset consisting of 2,542 kidney whole slide images (WSIs). Three state-of-the-art (SOTA) cell foundation models-Cellpose, StarDist, and CellViT-were selected for evaluation. To tackle the second question, we explored data enrichment algorithms by distilling predictions from the different foundation models with a human-in-the-loop framework, aiming to further enhance foundation model performance with minimal human efforts. Our experimental results showed that all three foundation models improved over their baselines with model fine-tuning with enriched data. Interestingly, the baseline model with the highest F1 score does not yield the best segmentation outcomes after fine-tuning. This study establishes a benchmark for the development and deployment of cell vision foundation models tailored for real-world data applications.

Method: Integrated positional priors into existing frameworks for SM calibration, with theoretical justification and empirical validation using both 2D and 3D SM super-resolution methods.

Result: Empirically validated the efficacy of incorporating positional priors through experiments on 2D and 3D SM SR methods.

Conclusion: Positional priors can be effectively integrated into MPI system matrix calibration frameworks to improve performance while leveraging physical prior knowledge.

Abstract: Magnetic Particle Imaging (MPI) is a novel medical imaging modality. One of the established methods for MPI reconstruction is based on the System Matrix (SM). However, the calibration of the SM is often time-consuming and requires repeated measurements whenever the system parameters change. Current methodologies utilize deep learning-based super-resolution (SR) techniques to expedite SM calibration; nevertheless, these strategies do not fully exploit physical prior knowledge associated with the SM, such as symmetric positional priors. Consequently, we integrated positional priors into existing frameworks for SM calibration. Underpinned by theoretical justification, we empirically validated the efficacy of incorporating positional priors through experiments involving both 2D and 3D SM SR methods.

Method: MACMD-based decoder with hierarchical dilated convolutions, attention-driven modulation, and cross channel-mixing module via skip connections to capture long-range dependencies while preserving local contextual details.

Result: Outperforms state-of-the-art approaches in Dice score and computational efficiency on both binary and multi-organ segmentation tasks.

Conclusion: The proposed method effectively achieves accurate and robust medical image segmentation by better integrating local and global information through enhanced attention mechanisms and channel mixing.

Abstract: Medical image segmentation faces challenges due to variations in anatomical structures. While convolutional neural networks (CNNs) effectively capture local features, they struggle with modeling long-range dependencies. Transformers mitigate this issue with self-attention mechanisms but lack the ability to preserve local contextual information. State-of-the-art models primarily follow an encoder-decoder architecture, achieving notable success. However, two key limitations remain: (1) Shallow layers, which are closer to the input, capture fine-grained details but suffer from information loss as data propagates through deeper layers. (2) Inefficient integration of local details and global context between the encoder and decoder stages. To address these challenges, we propose the MACMD-based decoder, which enhances attention mechanisms and facilitates channel mixing between encoder and decoder stages via skip connections. This design leverages hierarchical dilated convolutions, attention-driven modulation, and a cross channel-mixing module to capture long-range dependencies while preserving local contextual details, essential for precise medical image segmentation. We evaluated our approach using multiple transformer encoders on both binary and multi-organ segmentation tasks. The results demonstrate that our method outperforms state-of-the-art approaches in terms of Dice score and computational efficiency, highlighting its effectiveness in achieving accurate and robust segmentation performance. The code available at https://github.com/lalitmaurya47/MACMD

Method: Proposes a data-driven low-rank approximation method for text shape representation using ℓ₁-norm formulation for robustness, and a triple assignment detection head with deep sparse, ultra-lightweight sparse, and dense branches for stabilized training and fast inference.

Result: Extensive experiments on challenging benchmarks demonstrate superior performance compared to state-of-the-art methods in both accuracy and efficiency.

Conclusion: LRANet++ effectively addresses the arbitrary-shaped text spotting problem through innovative shape representation and detection architecture, achieving accurate and efficient end-to-end text spotting.

Abstract: End-to-end text spotting aims to jointly optimize text detection and recognition within a unified framework. Despite significant progress, designing an accurate and efficient end-to-end text spotter for arbitrary-shaped text remains largely unsolved. We identify the primary bottleneck as the lack of a reliable and efficient text detection method. To address this, we propose a novel parameterized text shape method based on low-rank approximation for precise detection and a triple assignment detection head to enable fast inference. Specifically, unlike other shape representation methods that employ data-irrelevant parameterization, our data-driven approach derives a low-rank subspace directly from labeled text boundaries. To ensure this process is robust against the inherent annotation noise in this data, we utilize a specialized recovery method based on an $\ell_1$-norm formulation, which accurately reconstructs the text shape with only a few key orthogonal vectors. By exploiting the inherent shape correlation among different text contours, our method achieves consistency and compactness in shape representation. Next, the triple assignment scheme introduces a novel architecture where a deep sparse branch (for stabilized training) is used to guide the learning of an ultra-lightweight sparse branch (for accelerated inference), while a dense branch provides rich parallel supervision. Building upon these advancements, we integrate the enhanced detection module with a lightweight recognition branch to form an end-to-end text spotting framework, termed LRANet++, capable of accurately and efficiently spotting arbitrary-shaped text. Extensive experiments on several challenging benchmarks demonstrate the superiority of LRANet++ compared to state-of-the-art methods. Code will be available at: https://github.com/ychensu/LRANet-PP.git

Method: Reorders image tokens along a Hilbert curve, then forms windows and neighborhoods on the reordered 1D sequence to increase block sparsity, combined with block-sparse kernels.

Result: Hilbert Window Attention and Hilbert Slide Attention accelerate window attention by 4× and slide attention by 18× respectively, with minimal accuracy loss in end-to-end transformers.

Conclusion: Hilbert-guided local attention with block-sparse kernels provides a general and practical approach to enhance efficiency of 2D local attention for images.

Abstract: The quadratic compute and memory costs of global self-attention severely limit its use in high-resolution images. Local attention reduces complexity by restricting attention to neighborhoods. Block-sparse kernels can further improve the efficiency of local attention, but conventional local attention patterns often fail to deliver significant speedups because tokens within a window are not contiguous in the 1D sequence. This work proposes a novel method for constructing windows and neighborhoods based on the Hilbert curve. Image tokens are first reordered along a Hilbert curve, and windows and neighborhoods are then formed on the reordered 1D sequence. From a block-sparse perspective, this strategy significantly increases block sparsity and can be combined with existing block-sparse kernels to improve the efficiency of 2D local attention. Experiments show that the proposed Hilbert Window Attention and Hilbert Slide Attention can accelerate window attention and slide attention by about $4\times$ and $18\times$, respectively. To assess practicality, the strategy is instantiated as the Hilbert Window Transformer and the Hilbert Neighborhood Transformer, both of which achieve end-to-end speedups with minimal accuracy loss. Overall, combining Hilbert-guided local attention with block-sparse kernels offers a general and practical approach to enhancing the efficiency of 2D local attention for images. The code is available at https://github.com/Yunge6666/Hilbert-Local-Attention.

Method: Proposed TYrPPG with innovative gated video understanding block (GVB) combining 2D-CNN and 3D-CNN based on Mambaout structure, plus comprehensive supervised loss function (CSL) and its weakly supervised variants.

Result: TYrPPG achieves state-of-the-art performance in commonly used datasets for remote heart rate estimation.

Conclusion: The proposed TYrPPG demonstrates prospects and superiority in remote heart rate estimation, proving the feasibility of Mambaout-based modules for this task.

Abstract: Remote photoplethysmography (rPPG) can remotely extract physiological signals from RGB video, which has many advantages in detecting heart rate, such as low cost and no invasion to patients. The existing rPPG model is usually based on the transformer module, which has low computation efficiency. Recently, the Mamba model has garnered increasing attention due to its efficient performance in natural language processing tasks, demonstrating potential as a substitute for transformer-based algorithms. However, the Mambaout model and its variants prove that the SSM module, which is the core component of the Mamba model, is unnecessary for the vision task. Therefore, we hope to prove the feasibility of using the Mambaout-based module to remotely learn the heart rate. Specifically, we propose a novel rPPG algorithm called uncomplicated and enhanced learning capability rPPG (TYrPPG). This paper introduces an innovative gated video understanding block (GVB) designed for efficient analysis of RGB videos. Based on the Mambaout structure, this block integrates 2D-CNN and 3D-CNN to enhance video understanding for analysis. In addition, we propose a comprehensive supervised loss function (CSL) to improve the model’s learning capability, along with its weakly supervised variants. The experiments show that our TYrPPG can achieve state-of-the-art performance in commonly used datasets, indicating its prospects and superiority in remote heart rate estimation. The source code is available at https://github.com/Taixi-CHEN/TYrPPG.

Method: Introduces a lightweight Cross-Layer Fusion Adapter (CLFA) framework that repurposes CLIP by implanting multi-level fusion adapters within the visual encoder to progressively align representations toward handwriting-specific medical cues, enabling prompt-free and efficient zero-shot inference.

Result: The framework enables systematic investigation of cross-task generalization, revealing which task types and writing patterns most effectively discriminate AD. Extensive analyses highlight characteristic stroke patterns and task-level factors that contribute to early AD identification.

Conclusion: The approach offers both diagnostic insights and a benchmark for handwriting-based cognitive assessment, demonstrating the potential of vision language models for handwriting-based disease detection and providing a systematic framework for cross-task generalization analysis.

Abstract: Alzheimer’s disease is a prevalent neurodegenerative disorder for which early detection is critical. Handwriting-often disrupted in prodromal AD-provides a non-invasive and cost-effective window into subtle motor and cognitive decline. Existing handwriting-based AD studies, mostly relying on online trajectories and hand-crafted features, have not systematically examined how task type influences diagnostic performance and cross-task generalization. Meanwhile, large-scale vision language models have demonstrated remarkable zero or few-shot anomaly detection in natural images and strong adaptability across medical modalities such as chest X-ray and brain MRI. However, handwriting-based disease detection remains largely unexplored within this paradigm. To close this gap, we introduce a lightweight Cross-Layer Fusion Adapter framework that repurposes CLIP for handwriting-based AD screening. CLFA implants multi-level fusion adapters within the visual encoder to progressively align representations toward handwriting-specific medical cues, enabling prompt-free and efficient zero-shot inference. Using this framework, we systematically investigate cross-task generalization-training on a specific handwriting task and evaluating on unseen ones-to reveal which task types and writing patterns most effectively discriminate AD. Extensive analyses further highlight characteristic stroke patterns and task-level factors that contribute to early AD identification, offering both diagnostic insights and a benchmark for handwriting-based cognitive assessment.

Method: Built high-quality CPS3D-Seg dataset with 1300 point cloud samples and proposed CINet method using Structural Refine and Quality Assessment modules for causal inference.

Result: CINet significantly outperforms existing algorithms in both mIoU and accuracy metrics.

Conclusion: The CPS3D-Seg dataset and CINet method effectively address 3D surface defect detection challenges in ceramic package substrates.

Abstract: The effective segmentation of 3D data is crucial for a wide range of industrial applications, especially for detecting subtle defects in the field of integrated circuits (IC). Ceramic package substrates (CPS), as an important electronic material, are essential in IC packaging owing to their superior physical and chemical properties. However, the complex structure and minor defects of CPS, along with the absence of a publically available dataset, significantly hinder the development of CPS surface defect detection. In this study, we construct a high-quality point cloud dataset for 3D segmentation of surface defects in CPS, i.e., CPS3D-Seg, which has the best point resolution and precision compared to existing 3D industrial datasets. CPS3D-Seg consists of 1300 point cloud samples under 20 product categories, and each sample provides accurate point-level annotations. Meanwhile, we conduct a comprehensive benchmark based on SOTA point cloud segmentation algorithms to validate the effectiveness of CPS3D-Seg. Additionally, we propose a novel 3D segmentation method based on causal inference (CINet), which quantifies potential confounders in point clouds through Structural Refine (SR) and Quality Assessment (QA) Modules. Extensive experiments demonstrate that CINet significantly outperforms existing algorithms in both mIoU and accuracy.

Method: Classifier-guided approach using lightweight classifiers on text embeddings to detect and refine prompts containing undesired concepts at inference time, without altering model weights.

Result: Achieves state-of-the-art robustness against red-teaming attacks while maintaining high generative utility, with successful application to various T2I and T2V models.

Conclusion: CGCE provides a practical and effective solution for safe generative AI by balancing safety and performance through inference-time modification of unsafe embeddings.

Abstract: Recent advancements in large-scale generative models have enabled the creation of high-quality images and videos, but have also raised significant safety concerns regarding the generation of unsafe content. To mitigate this, concept erasure methods have been developed to remove undesirable concepts from pre-trained models. However, existing methods remain vulnerable to adversarial attacks that can regenerate the erased content. Moreover, achieving robust erasure often degrades the model’s generative quality for safe, unrelated concepts, creating a difficult trade-off between safety and performance. To address this challenge, we introduce Classifier-Guided Concept Erasure (CGCE), an efficient plug-and-play framework that provides robust concept erasure for diverse generative models without altering their original weights. CGCE uses a lightweight classifier operating on text embeddings to first detect and then refine prompts containing undesired concepts. This approach is highly scalable, allowing for multi-concept erasure by aggregating guidance from several classifiers. By modifying only unsafe embeddings at inference time, our method prevents harmful content generation while preserving the model’s original quality on benign prompts. Extensive experiments show that CGCE achieves state-of-the-art robustness against a wide range of red-teaming attacks. Our approach also maintains high generative utility, demonstrating a superior balance between safety and performance. We showcase the versatility of CGCE through its successful application to various modern T2I and T2V models, establishing it as a practical and effective solution for safe generative AI.

Method: Created a comprehensive dataset using Lytro Illum LF camera for real images and Blender for synthetic images, including both real and synthetic stereo light field data captured via mechanical gantry system and simulation.

Result: Produced a dataset of 285 real light field images and 13 synthetic LF images, with synthetic data having similar disparity characteristics to real LF cameras. The dataset demonstrates focal position effects on 3D point disparity.

Conclusion: The introduced publicly available light field dataset addresses the need for suitable datasets in LF depth estimation research and is available at https://github.com/aupendu/light-field-dataset.

Abstract: A Light Field (LF) camera consists of an additional two-dimensional array of micro-lenses placed between the main lens and sensor, compared to a conventional camera. The sensor pixels under each micro-lens receive light from a sub-aperture of the main lens. This enables the image sensor to capture both spatial information and the angular resolution of a scene point. This additional angular information is used to estimate the depth of a 3-D scene. The continuum of virtual viewpoints in light field data enables efficient depth estimation using Epipolar Line Images (EPIs) with robust occlusion handling. However, the trade-off between angular information and spatial information is very critical and depends on the focal position of the camera. To design, develop, implement, and test novel disparity-based light field depth estimation algorithms, the availability of suitable light field image datasets is essential. In this paper, a publicly available light field image dataset is introduced and thoroughly described. We have also demonstrated the effect of focal position on the disparity of a 3-D point as well as the shortcomings of the currently available light field dataset. The proposed dataset contains 285 light field images captured using a Lytro Illum LF camera and 13 synthetic LF images. The proposed dataset also comprises a synthetic dataset with similar disparity characteristics to those of a real light field camera. A real and synthetic stereo light field dataset is also created by using a mechanical gantry system and Blender. The dataset is available at https://github.com/aupendu/light-field-dataset.

Method: Uses Mixture of Ego-Graphs Fusion (MoEGF) with Mixture-of-Experts for sample-level fusion, plus Ego Graph Contrastive Learning (EGCL) for representation alignment.

Result: Extensive experiments show MoEGCL achieves state-of-the-art performance in deep multi-view clustering tasks.

Conclusion: Fine-grained ego-graph fusion at sample level significantly improves multi-view clustering performance over traditional view-level fusion approaches.

Abstract: In recent years, the advancement of Graph Neural Networks (GNNs) has significantly propelled progress in Multi-View Clustering (MVC). However, existing methods face the problem of coarse-grained graph fusion. Specifically, current approaches typically generate a separate graph structure for each view and then perform weighted fusion of graph structures at the view level, which is a relatively rough strategy. To address this limitation, we present a novel Mixture of Ego-Graphs Contrastive Representation Learning (MoEGCL). It mainly consists of two modules. In particular, we propose an innovative Mixture of Ego-Graphs Fusion (MoEGF), which constructs ego graphs and utilizes a Mixture-of-Experts network to implement fine-grained fusion of ego graphs at the sample level, rather than the conventional view-level fusion. Additionally, we present the Ego Graph Contrastive Learning (EGCL) module to align the fused representation with the view-specific representation. The EGCL module enhances the representation similarity of samples from the same cluster, not merely from the same sample, further boosting fine-grained graph representation. Extensive experiments demonstrate that MoEGCL achieves state-of-the-art results in deep multi-view clustering tasks. The source code is publicly available at https://github.com/HackerHyper/MoEGCL.

Method: Uses wavelet decomposition to separate images into frequency sub-bands, then employs specialized sub-networks: neural ODEs for low-frequency denoising to ensure structural fidelity, and enhanced U-Net with deformable convolutions for high-frequency components to preserve edges and textures.

Result: Extensive experiments on synthetic and real SAR images demonstrate superior performance in both noise removal and structural preservation compared to existing methods.

Conclusion: The proposed frequency-adaptive heterogeneous approach effectively addresses the limitations of unified networks by leveraging statistical variations across frequencies, achieving better edge and texture preservation while suppressing speckle noise.

Abstract: Synthetic Aperture Radar (SAR) images are inherently corrupted by speckle noise, limiting their utility in high-precision applications. While deep learning methods have shown promise in SAR despeckling, most methods employ a single unified network to process the entire image, failing to account for the distinct speckle statistics associated with different spatial physical characteristics. It often leads to artifacts, blurred edges, and texture distortion. To address these issues, we propose SAR-FAH, a frequency-adaptive heterogeneous despeckling model based on a divide-and-conquer architecture. First, wavelet decomposition is used to separate the image into frequency sub-bands carrying different intrinsic characteristics. Inspired by their differing noise characteristics, we design specialized sub-networks for different frequency components. The tailored approach leverages statistical variations across frequencies, improving edge and texture preservation while suppressing noise. Specifically, for the low-frequency part, denoising is formulated as a continuous dynamic system via neural ordinary differential equations, ensuring structural fidelity and sufficient smoothness that prevents artifacts. For high-frequency sub-bands rich in edges and textures, we introduce an enhanced U-Net with deformable convolutions for noise suppression and enhanced features. Extensive experiments on synthetic and real SAR images validate the superior performance of the proposed model in noise removal and structural preservation.

Method: Designs Hybrid Second-Order Gradient Histogram (H2H) feature descriptor and integrates it with Sparse Regularized Nuclear Norm based Matrix Regression (SR_NMR), adding global low-rank constraint on residual matrix.

Result: Experimental results show significant performance improvement over existing regression-based classification methods in challenging scenarios with occlusions, illumination changes, and unconstrained environments.

Conclusion: The proposed H2H-GLRSR model effectively handles complex face recognition challenges through its hybrid feature descriptor and global low-rank sparse regression framework.

Abstract: Low-rank sparse regression models have been widely applied in the field of face recognition. To further address the challenges caused by complex occlusions and illumination variations, this paper proposes a Hybrid Second-Order Gradient Histogram based Global Low-Rank Sparse Regression (H2H-GLRSR) model. Specifically, a novel feature descriptor called the Hybrid Second-Order Gradient Histogram (H2H) is first designed to more effectively characterize the local structural features of facial images. Then, this descriptor is integrated with the Sparse Regularized Nuclear Norm based Matrix Regression (SR$_$NMR). Moreover, a global low-rank constraint is imposed on the residual matrix, enabling the model to better capture the global correlations inherent in structured noise. Experimental results demonstrate that the proposed method significantly outperforms existing regression-based classification approaches under challenging scenarios involving occlusions, illumination changes, and unconstrained environments.

Method: Integrates Vision-Language Models with retrieval-based reasoning, featuring dynamic scene graph generation and retrieval-augmented reasoning pipeline that encodes scene graphs into vector database.

Result: Demonstrates robust generalization and superior performance on 3DSSG and Replica benchmarks across scene QA, visual grounding, instance retrieval, and task planning tasks.

Conclusion: Combining open-vocabulary perception with retrieval-based reasoning is effective for scalable 3D scene understanding in diverse environments.

Abstract: Understanding 3D scenes in open-world settings poses fundamental challenges for vision and robotics, particularly due to the limitations of closed-vocabulary supervision and static annotations. To address this, we propose a unified framework for Open-World 3D Scene Graph Generation with Retrieval-Augmented Reasoning, which enables generalizable and interactive 3D scene understanding. Our method integrates Vision-Language Models (VLMs) with retrieval-based reasoning to support multimodal exploration and language-guided interaction. The framework comprises two key components: (1) a dynamic scene graph generation module that detects objects and infers semantic relationships without fixed label sets, and (2) a retrieval-augmented reasoning pipeline that encodes scene graphs into a vector database to support text/image-conditioned queries. We evaluate our method on 3DSSG and Replica benchmarks across four tasks-scene question answering, visual grounding, instance retrieval, and task planning-demonstrating robust generalization and superior performance in diverse environments. Our results highlight the effectiveness of combining open-vocabulary perception with retrieval-based reasoning for scalable 3D scene understanding.

Method: Proposed Gradient-Aware Balanced Feature Fusion (GABFusion) to dynamically balance gradient magnitudes and fuse task-specific features, plus Attention Distribution Alignment (ADA) for feature-level distillation in quantized models.

Result: Achieved average mAP improvements of ~3.3% on PASCAL VOC and ~1.6% on COCO datasets. Under 4-bit quantization, narrowed accuracy gap with full-precision model to only 1.7% on VOC.

Conclusion: The proposed framework is modular, easy to integrate, compatible with any existing QAT technique, and effectively preserves performance under low-bit constraints without modifying original network architecture.

Abstract: Despite the effectiveness of quantization-aware training (QAT) in compressing deep neural networks, its performance on multi-task architectures often degrades significantly due to task-specific feature discrepancies and gradient conflicts. To address these challenges, we propose Gradient-Aware Balanced Feature Fusion (GABFusion), which dynamically balances gradient magnitudes and fuses task-specific features in a quantization-friendly manner. We further introduce Attention Distribution Alignment (ADA), a feature-level distillation strategy tailored for quantized models. Our method demonstrates strong generalization across network architectures and QAT algorithms, with theoretical guarantees on gradient bias reduction. Extensive experiments demonstrate that our strategy consistently enhances a variety of QAT methods across different network architectures and bit-widths. On PASCAL VOC and COCO datasets, the proposed approach achieves average mAP improvements of approximately 3.3% and 1.6%, respectively. When applied to YOLOv5 under 4-bit quantization, our method narrows the accuracy gap with the full-precision model to only 1.7% on VOC, showcasing its effectiveness in preserving performance under low-bit constraints. Notably, the proposed framework is modular, easy to integrate, and compatible with any existing QAT technique-enhancing the performance of quantized models without requiring modifications to the original network architecture.

Method: Introduces FCCT framework for fine-grained causal tracing across visual/textual tokens, MHSA, FFNs, and hidden states in all decoder layers. Proposes IRI for inference-time intervention.

Result: MHSA in middle layers aggregates cross-modal info; FFNs show 3-stage hierarchical processing. IRI achieves SOTA on 5 benchmarks while preserving speed and performance.

Conclusion: FCCT provides comprehensive LVLM interpretability insights, enabling effective interventions like IRI to enhance perception and mitigate hallucination without training overhead.

Abstract: Despite the remarkable advancements of Large Vision-Language Models (LVLMs), the mechanistic interpretability remains underexplored. Existing analyses are insufficiently comprehensive and lack examination covering visual and textual tokens, model components, and the full range of layers. This limitation restricts actionable insights to improve the faithfulness of model output and the development of downstream tasks, such as hallucination mitigation. To address this limitation, we introduce Fine-grained Cross-modal Causal Tracing (FCCT) framework, which systematically quantifies the causal effects on visual object perception. FCCT conducts fine-grained analysis covering the full range of visual and textual tokens, three core model components including multi-head self-attention (MHSA), feed-forward networks (FFNs), and hidden states, across all decoder layers. Our analysis is the first to demonstrate that MHSAs of the last token in middle layers play a critical role in aggregating cross-modal information, while FFNs exhibit a three-stage hierarchical progression for the storage and transfer of visual object representations. Building on these insights, we propose Intermediate Representation Injection (IRI), a training-free inference-time technique that reinforces visual object information flow by precisely intervening on cross-modal representations at specific components and layers, thereby enhancing perception and mitigating hallucination. Consistent improvements across five widely used benchmarks and LVLMs demonstrate IRI achieves state-of-the-art performance, while preserving inference speed and other foundational performance.

Method: Proposed Complementary Masked Autoencoders (CoMA) with uniform pixel sampling strategy and DyViT with Dynamic Multi-Window Self-Attention for hierarchical vision processing.

Result: DyViT with CoMA matches MAE downstream performance using only 12% pre-training epochs, with 10% reduction in pre-training time per epoch.

Conclusion: CoMA and DyViT significantly improve pre-training efficiency and effectiveness while maintaining strong downstream task performance.

Abstract: Masked Autoencoders (MAE) achieve self-supervised learning of image representations by randomly removing a portion of visual tokens and reconstructing the original image as a pretext task, thereby significantly enhancing pretraining efficiency and yielding excellent adaptability across downstream tasks. However, MAE and other MAE-style paradigms that adopt random masking generally require more pre-training epochs to maintain adaptability. Meanwhile, ViT in MAE suffers from inefficient parameter use due to fixed spatial resolution across layers. To overcome these limitations, we propose the Complementary Masked Autoencoders (CoMA), which employ a complementary masking strategy to ensure uniform sampling across all pixels, thereby improving effective learning of all features and enhancing the model’s adaptability. Furthermore, we introduce DyViT, a hierarchical vision transformer that employs a Dynamic Multi-Window Self-Attention (DM-MSA), significantly reducing the parameters and FLOPs while improving fine-grained feature learning. Pre-trained on ImageNet-1K with CoMA, DyViT matches the downstream performance of MAE using only 12% of the pre-training epochs, demonstrating more effective learning. It also attains a 10% reduction in pre-training time per epoch, further underscoring its superior pre-training efficiency.

Method: Uses a conditionable Diffusion Auto-encoder framework that forms a compact latent space capturing high-level semantics, allowing disentanglement of progression-related information. Applies controlled shifts to baseline representations restricted to a subspace that isolates progression factors from identity-preserving components.

Result: Validated through image quality metrics, volumetric progression analysis, and downstream classification in Alzheimer’s disease datasets from two different sources and disease categories, demonstrating effectiveness for Alzheimer’s progression modeling.

Conclusion: The approach successfully enables controlled longitudinal image generation for disease progression modeling in an unsupervised manner, effectively capturing progression-related changes while preserving subject identity.

Abstract: Generative modeling frameworks have emerged as an effective approach to capture high-dimensional image distributions from large datasets without requiring domain-specific knowledge, a capability essential for longitudinal disease progression modeling. Recent generative modeling approaches have attempted to capture progression by mapping images into a latent representational space and then controlling and guiding the representations to generate follow-up images from a baseline image. However, existing approaches impose constraints on distribution learning, leading to latent spaces with limited controllability to generate follow-up images without explicit supervision from subject-specific longitudinal images. In order to enable controlled movements in the latent representational space and generate progression images from a baseline image in an unsupervised manner, we introduce a conditionable Diffusion Auto-encoder framework. The explicit encoding mechanism of image-diffusion auto-encoders forms a compact latent space capturing high-level semantics, providing means to disentangle information relevant for progression. Our approach leverages this latent space to condition and apply controlled shifts to baseline representations for generating follow-up. Controllability is induced by restricting these shifts to a subspace, thereby isolating progression-related factors from subject identity-preserving components. The shifts are implicitly guided by correlating with progression attributes, without requiring subject-specific longitudinal supervision. We validate the generations through image quality metrics, volumetric progression analysis, and downstream classification in Alzheimer’s disease datasets from two different sources and disease categories. This demonstrates the effectiveness of our approach for Alzheimer’s progression modeling and longitudinal image generation.

Method: Proposes ACC framework with bidirectional interaction prompts for robust pseudo-supervision generation, interaction-guided query selection to prioritize interacting objects, and interaction-consistent knowledge distillation to separate relational foreground from background.

Result: Extensive experiments on three benchmarks show ACC achieves state-of-the-art performance, demonstrating superior capability in open-vocabulary scene graph generation.

Conclusion: The interaction-centric paradigm in ACC effectively addresses key limitations in OVSGG and shows strong potential for real-world applications.

Abstract: Open-vocabulary scene graph generation (OVSGG) extends traditional SGG by recognizing novel objects and relationships beyond predefined categories, leveraging the knowledge from pre-trained large-scale models. Existing OVSGG methods always adopt a two-stage pipeline: 1) \textit{Infusing knowledge} into large-scale models via pre-training on large datasets; 2) \textit{Transferring knowledge} from pre-trained models with fully annotated scene graphs during supervised fine-tuning. However, due to a lack of explicit interaction modeling, these methods struggle to distinguish between interacting and non-interacting instances of the same object category. This limitation induces critical issues in both stages of OVSGG: it generates noisy pseudo-supervision from mismatched objects during knowledge infusion, and causes ambiguous query matching during knowledge transfer. To this end, in this paper, we propose an inter\textbf{AC}tion-\textbf{C}entric end-to-end OVSGG framework (\textbf{ACC}) in an interaction-driven paradigm to minimize these mismatches. For \textit{interaction-centric knowledge infusion}, ACC employs a bidirectional interaction prompt for robust pseudo-supervision generation to enhance the model’s interaction knowledge. For \textit{interaction-centric knowledge transfer}, ACC first adopts interaction-guided query selection that prioritizes pairing interacting objects to reduce interference from non-interacting ones. Then, it integrates interaction-consistent knowledge distillation to bolster robustness by pushing relational foreground away from the background while retaining general knowledge. Extensive experimental results on three benchmarks show that ACC achieves state-of-the-art performance, demonstrating the potential of interaction-centric paradigms for real-world applications.

Method: GME-Net includes: 1) hybrid attention-based local feature extraction with attention similarity knowledge distillation to learn details from high-resolution networks, 2) multi-scale global feature extraction with quasi-symmetric structure to reduce local noise and capture global features.

Result: Extensive experiments on multiple datasets show GME-Net achieves superior performance in low-resolution facial expression recognition compared to existing methods.

Conclusion: GME-Net effectively extracts expression-related discriminative features and outperforms current solutions for low-resolution facial expression recognition.

Abstract: Facial expression recognition, as a vital computer vision task, is garnering significant attention and undergoing extensive research. Although facial expression recognition algorithms demonstrate impressive performance on high-resolution images, their effectiveness tends to degrade when confronted with low-resolution images. We find it is because: 1) low-resolution images lack detail information; 2) current methods complete weak global modeling, which make it difficult to extract discriminative features. To alleviate the above issues, we proposed a novel global multiple extraction network (GME-Net) for low-resolution facial expression recognition, which incorporates 1) a hybrid attention-based local feature extraction module with attention similarity knowledge distillation to learn image details from high-resolution network; 2) a multi-scale global feature extraction module with quasi-symmetric structure to mitigate the influence of local image noise and facilitate capturing global image features. As a result, our GME-Net is capable of extracting expression-related discriminative features. Extensive experiments conducted on several widely-used datasets demonstrate that the proposed GME-Net can better recognize low-resolution facial expression and obtain superior performance than existing solutions.

Method: Reinterprets road elements as rasterized polygons, uses segmentation-based transformer for instance masks, and Potrace-based post-processing for vectorization.

Result: Quantitative results on Nuscene dataset demonstrate effectiveness and improved generalizability of the proposed method.

Conclusion: Segmentation-based approach with proper post-processing provides better generalizability for HD map construction compared to detection-based methods.

Abstract: The perception of high-definition maps is an integral component of environmental perception in autonomous driving systems. Existing research have often focused on online construction of high-definition maps. For instance, the Maptr[9] series employ a detection-based method to output vectorized map instances parallelly in an end-to-end manner. However, despite their capability for real-time construction, detection-based methods are observed to lack robust generalizability[19], which hampers their applicability in auto-labeling systems. Therefore, aiming to improve the generalizability, we reinterpret road elements as rasterized polygons and design a concise framework based on instance segmentation. Initially, a segmentation-based transformer is employed to deliver instance masks in an end-to-end manner; succeeding this step, a Potrace-based[17] post-processing module is used to ultimately yield vectorized map elements. Quantitative results attained on the Nuscene[1] dataset substantiate the effectiveness and generaliz-ability of our method.

Method: Proposed Reperio-rPPG framework that strategically integrates Relational Convolutional Networks with a Graph Transformer to capture periodic structure, plus a tailored CutMix augmentation to enhance generalizability.

Result: Extensive experiments on PURE, UBFC-rPPG, and MMPD datasets show state-of-the-art performance with remarkable robustness under various motion and illumination conditions.

Conclusion: Reperio-rPPG effectively bridges the gap in modeling physiological signal periodicity and demonstrates superior performance and robustness for contactless vital sign monitoring.

Abstract: Remote photoplethysmography (rPPG) is an emerging contactless physiological sensing technique that leverages subtle color variations in facial videos to estimate vital signs such as heart rate and respiratory rate. This non-invasive method has gained traction across diverse domains, including telemedicine, affective computing, driver fatigue detection, and health monitoring, owing to its scalability and convenience. Despite significant progress in remote physiological signal measurement, a crucial characteristic - the intrinsic periodicity - has often been underexplored or insufficiently modeled in previous approaches, limiting their ability to capture fine-grained temporal dynamics under real-world conditions. To bridge this gap, we propose Reperio-rPPG, a novel framework that strategically integrates Relational Convolutional Networks with a Graph Transformer to effectively capture the periodic structure inherent in physiological signals. Additionally, recognizing the limited diversity of existing rPPG datasets, we further introduce a tailored CutMix augmentation to enhance the model’s generalizability. Extensive experiments conducted on three widely used benchmark datasets - PURE, UBFC-rPPG, and MMPD - demonstrate that Reperio-rPPG not only achieves state-of-the-art performance but also exhibits remarkable robustness under various motion (e.g., stationary, rotation, talking, walking) and illumination conditions (e.g., nature, low LED, high LED). The code is publicly available at https://github.com/deconasser/Reperio-rPPG.

Method: Uses pre-trained Segment Anything Model for mask detection, converts masks to polygons, employs bidirectional-pyramid strategy with LoFTR for global matching, and local-joint geometry with Hungarian algorithm for local matching.

Result: Achieved state-of-the-art accuracy on ScanNet and SceneFlow datasets with competitive speed, satisfactory generalization, and low cost without training.

Conclusion: The proposed U(PM)^2 effectively addresses polygon matching challenges through unsupervised approach combining learned and handcrafted features, demonstrating strong performance without training requirements.

Abstract: Stereo image matching is a fundamental task in computer vision, photogrammetry and remote sensing, but there is an almost unexplored field, i.e., polygon matching, which faces the following challenges: disparity discontinuity, scale variation, training requirement, and generalization. To address the above-mentioned issues, this paper proposes a novel U(PM)$^2$: low-cost unsupervised polygon matching with pre-trained models by uniting automatically learned and handcrafted features, of which pipeline is as follows: firstly, the detector leverages the pre-trained segment anything model to obtain masks; then, the vectorizer converts the masks to polygons and graphic structure; secondly, the global matcher addresses challenges from global viewpoint changes and scale variation based on bidirectional-pyramid strategy with pre-trained LoFTR; finally, the local matcher further overcomes local disparity discontinuity and topology inconsistency of polygon matching by local-joint geometry and multi-feature matching strategy with Hungarian algorithm. We benchmark our U(PM)$^2$ on the ScanNet and SceneFlow datasets using our proposed new metric, which achieved state-of-the-art accuracy at a competitive speed and satisfactory generalization performance at low cost without any training requirement.

Method: Multimodal approach using facial and scene information with fine-grained attention mechanism focused on the principal speaker to model social gaze dynamics.

Result: CSGaze performs competitively with state-of-the-art methods on GP-Static, UCO-LAEO and AVA-LAEO datasets, and shows strong generalization on open set datasets.

Conclusion: Contextual cues significantly improve social gaze prediction, and the model provides explainability through attention scores while demonstrating robustness across diverse scenarios.

Abstract: A person’s gaze offers valuable insights into their focus of attention, level of social engagement, and confidence. In this work, we investigate how contextual cues combined with visual scene and facial information can be effectively utilized to predict and interpret social gaze patterns during conversational interactions. We introduce CSGaze, a context aware multimodal approach that leverages facial, scene information as complementary inputs to enhance social gaze pattern prediction from multi-person images. The model also incorporates a fine-grained attention mechanism centered on the principal speaker, which helps in better modeling social gaze dynamics. Experimental results show that CSGaze performs competitively with state-of-the-art methods on GP-Static, UCO-LAEO and AVA-LAEO. Our findings highlight the role of contextual cues in improving social gaze prediction. Additionally, we provide initial explainability through generated attention scores, offering insights into the model’s decision-making process. We also demonstrate our model’s generalizability by testing our model on open set datasets that demonstrating its robustness across diverse scenarios.

Method: Proposes two key modules: IAS enhances structural feature awareness with phase maps and uses reinforcement learning to select reliable agents; RAI leverages selected agents to guide cross-modal interactions and reduce mismatches.

Result: Extensive experiments on RGB-D Scenes v2 and 7-Scenes benchmarks show state-of-the-art performance.

Conclusion: The proposed framework effectively addresses challenges in cross-modal registration by improving feature selection and interaction mechanisms, achieving superior robustness and accuracy.

Abstract: Typical detection-free methods for image-to-point cloud registration leverage transformer-based architectures to aggregate cross-modal features and establish correspondences. However, they often struggle under challenging conditions, where noise disrupts similarity computation and leads to incorrect correspondences. Moreover, without dedicated designs, it remains difficult to effectively select informative and correlated representations across modalities, thereby limiting the robustness and accuracy of registration. To address these challenges, we propose a novel cross-modal registration framework composed of two key modules: the Iterative Agents Selection (IAS) module and the Reliable Agents Interaction (RAI) module. IAS enhances structural feature awareness with phase maps and employs reinforcement learning principles to efficiently select reliable agents. RAI then leverages these selected agents to guide cross-modal interactions, effectively reducing mismatches and improving overall robustness. Extensive experiments on the RGB-D Scenes v2 and 7-Scenes benchmarks demonstrate that our method consistently achieves state-of-the-art performance.

Method: Proposes CIF method with three modules: 1) semantic-aware hypergraph construction for single-semantic industrial images, 2) training-free hypergraph message passing to update test features, and 3) hyperedge-guided memory search using structural information to reduce false positives.

Result: Experimental results on MVTec 3D-AD and Eyecandies datasets show that CIF outperforms state-of-the-art methods in few-shot settings.

Conclusion: The proposed CIF method effectively addresses few-shot multimodal industrial anomaly detection by leveraging structural commonality through hypergraphs and memory banks, demonstrating superior performance compared to existing approaches.

Abstract: Few-shot multimodal industrial anomaly detection is a critical yet underexplored task, offering the ability to quickly adapt to complex industrial scenarios. In few-shot settings, insufficient training samples often fail to cover the diverse patterns present in test samples. This challenge can be mitigated by extracting structural commonality from a small number of training samples. In this paper, we propose a novel few-shot unsupervised multimodal industrial anomaly detection method based on structural commonality, CIF (Commonality In Few). To extract intra-class structural information, we employ hypergraphs, which are capable of modeling higher-order correlations, to capture the structural commonality within training samples, and use a memory bank to store this intra-class structural prior. Firstly, we design a semantic-aware hypergraph construction module tailored for single-semantic industrial images, from which we extract common structures to guide the construction of the memory bank. Secondly, we use a training-free hypergraph message passing module to update the visual features of test samples, reducing the distribution gap between test features and features in the memory bank. We further propose a hyperedge-guided memory search module, which utilizes structural information to assist the memory search process and reduce the false positive rate. Experimental results on the MVTec 3D-AD dataset and the Eyecandies dataset show that our method outperforms the state-of-the-art (SOTA) methods in few-shot settings. Code is available at https://github.com/Sunny5250/CIF.

Method: Used DINOv2-based MST on 1,847 breast MRI exams, testing four abbreviated protocols (T1sub, DWI1500, DWI1500+T2w, T1sub+T2w) with five-fold cross-validation and external validation on Duke dataset.

Result: T1sub+T2w achieved AUC 0.77±0.04, highest specificity of 19%±7% at 97.5% sensitivity. Missed lesions were small (<10mm) non-mass enhancements. External validation AUC 0.77 with 88% good/moderate attention maps.

Conclusion: MST can triage cases without BI-RADS ≥4 findings at high sensitivity, potentially reducing radiologist workload, but requires further research before clinical implementation.

Abstract: Background: Magnetic resonance imaging (MRI) has high sensitivity for breast cancer detection, but interpretation is time-consuming. Artificial intelligence may aid in pre-screening. Purpose: To evaluate the DINOv2-based Medical Slice Transformer (MST) for ruling out significant findings (Breast Imaging Reporting and Data System [BI-RADS] >=4) in contrast-enhanced and non-contrast-enhanced abbreviated breast MRI. Materials and Methods: This institutional review board approved retrospective study included 1,847 single-breast MRI examinations (377 BI-RADS >=4) from an in-house dataset and 924 from an external validation dataset (Duke). Four abbreviated protocols were tested: T1-weighted early subtraction (T1sub), diffusion-weighted imaging with b=1500 s/mm2 (DWI1500), DWI1500+T2-weighted (T2w), and T1sub+T2w. Performance was assessed at 90%, 95%, and 97.5% sensitivity using five-fold cross-validation and area under the receiver operating characteristic curve (AUC) analysis. AUC differences were compared with the DeLong test. False negatives were characterized, and attention maps of true positives were rated in the external dataset. Results: A total of 1,448 female patients (mean age, 49 +/- 12 years) were included. T1sub+T2w achieved an AUC of 0.77 +/- 0.04; DWI1500+T2w, 0.74 +/- 0.04 (p=0.15). At 97.5% sensitivity, T1sub+T2w had the highest specificity (19% +/- 7%), followed by DWI1500+T2w (17% +/- 11%). Missed lesions had a mean diameter <10 mm at 95% and 97.5% thresholds for both T1sub and DWI1500, predominantly non-mass enhancements. External validation yielded an AUC of 0.77, with 88% of attention maps rated good or moderate. Conclusion: At 97.5% sensitivity, the MST framework correctly triaged cases without BI-RADS >=4, achieving 19% specificity for contrast-enhanced and 17% for non-contrast-enhanced MRI. Further research is warranted before clinical implementation.

Method: Uses a Mixture-of-Experts based Vision-Language Variational Autoencoder (VLVAE) to disentangle shared and modality-specific features, with constrained optimization for orthogonality and alignment, followed by a compact LLaMA-X decoder for efficient report generation.

Result: Achieved competitive BLEU@4 scores of 0.266 on IU X-Ray and 0.134 on MIMIC-CXR datasets, significantly outperforming state-of-the-art models.

Conclusion: The proposed DiA framework successfully addresses challenges in radiology reporting by enabling robust performance even with missing clinical context through disentangled alignment.

Abstract: The integration of medical images with clinical context is essential for generating accurate and clinically interpretable radiology reports. However, current automated methods often rely on resource-heavy Large Language Models (LLMs) or static knowledge graphs and struggle with two fundamental challenges in real-world clinical data: (1) missing modalities, such as incomplete clinical context , and (2) feature entanglement, where mixed modality-specific and shared information leads to suboptimal fusion and clinically unfaithful hallucinated findings. To address these challenges, we propose the DiA-gnostic VLVAE, which achieves robust radiology reporting through Disentangled Alignment. Our framework is designed to be resilient to missing modalities by disentangling shared and modality-specific features using a Mixture-of-Experts (MoE) based Vision-Language Variational Autoencoder (VLVAE). A constrained optimization objective enforces orthogonality and alignment between these latent representations to prevent suboptimal fusion. A compact LLaMA-X decoder then uses these disentangled representations to generate reports efficiently. On the IU X-Ray and MIMIC-CXR datasets, DiA has achieved competetive BLEU@4 scores of 0.266 and 0.134, respectively. Experimental results show that the proposed method significantly outperforms state-of-the-art models.

Method: Comprehensive categorization of existing runtime monitoring methods into three groups: monitoring inputs, internal representations, and outputs of DNNs during inference.

Result: Analysis of state-of-the-art methods in each category, identification of strengths/limitations, and mapping of methods to specific safety concerns they address.

Conclusion: Identified open challenges and future research directions for runtime safety monitoring of DNNs in safety-critical applications.

Abstract: Deep neural networks (DNNs) are widely used in perception systems for safety-critical applications, such as autonomous driving and robotics. However, DNNs remain vulnerable to various safety concerns, including generalization errors, out-of-distribution (OOD) inputs, and adversarial attacks, which can lead to hazardous failures. This survey provides a comprehensive overview of runtime safety monitoring approaches, which operate in parallel to DNNs during inference to detect these safety concerns without modifying the DNN itself. We categorize existing methods into three main groups: Monitoring inputs, internal representations, and outputs. We analyze the state-of-the-art for each category, identify strengths and limitations, and map methods to the safety concerns they address. In addition, we highlight open challenges and future research directions.

Method: Proposes a flexible conditional Denoising Diffusion Model combining Vision Transformer encoder for global features with enhanced UNet decoder. Key innovations: Adaptive Conditioning Bridge for multi-scale feature fusion, Topological Denoising Consistency loss for structural regularization, and dual-head architecture for efficient inference.

Result: Achieves new state-of-the-art performance: Dice scores of 0.96 on BUSI, 0.90 on BrEaST, and 0.97 on BUS-UCLM datasets. Ablation studies confirm critical importance of all model components.

Conclusion: The proposed diffusion-based framework produces not only accurate but also anatomically plausible segmentations, effectively addressing challenges in breast ultrasound image analysis through global context modeling and structural regularization.

Abstract: In breast ultrasound images, precise lesion segmentation is essential for early diagnosis; however, low contrast, speckle noise, and unclear boundaries make this difficult. Even though deep learning models have demonstrated potential, standard convolutional architectures frequently fall short in capturing enough global context, resulting in segmentations that are anatomically inconsistent. To overcome these drawbacks, we suggest a flexible, conditional Denoising Diffusion Model that combines an enhanced UNet-based generative decoder with a Vision Transformer (ViT) encoder for global feature extraction. We introduce three primary innovations: 1) an Adaptive Conditioning Bridge (ACB) for efficient, multi-scale fusion of semantic features; 2) a novel Topological Denoising Consistency (TDC) loss component that regularizes training by penalizing structural inconsistencies during denoising; and 3) a dual-head architecture that leverages the denoising objective as a powerful regularizer, enabling a lightweight auxiliary head to perform rapid and accurate inference on smaller datasets and a noise prediction head. Our framework establishes a new state-of-the-art on public breast ultrasound datasets, achieving Dice scores of 0.96 on BUSI, 0.90 on BrEaST and 0.97 on BUS-UCLM. Comprehensive ablation studies empirically validate that the model components are critical for achieving these results and for producing segmentations that are not only accurate but also anatomically plausible.

Method: Uses SE(3) quasi-canonicalization of point clouds, models objects with Point Pair Features to predict pose voting parameters, and incorporates semantic joint axis information to enforce unified kinematic constraints.

Result: Demonstrates strong generalization across synthetic datasets and real-world scenarios, showing effectiveness and robustness in multi-frame articulated object pose tracking.

Conclusion: PPF-Tracker advances articulated object tracking and can foster progress in robotics, embodied intelligence, and augmented reality applications.

Abstract: Articulated objects are prevalent in daily life and robotic manipulation tasks. However, compared to rigid objects, pose tracking for articulated objects remains an underexplored problem due to their inherent kinematic constraints. To address these challenges, this work proposes a novel point-pair-based pose tracking framework, termed \textbf{PPF-Tracker}. The proposed framework first performs quasi-canonicalization of point clouds in the SE(3) Lie group space, and then models articulated objects using Point Pair Features (PPF) to predict pose voting parameters by leveraging the invariance properties of SE(3). Finally, semantic information of joint axes is incorporated to impose unified kinematic constraints across all parts of the articulated object. PPF-Tracker is systematically evaluated on both synthetic datasets and real-world scenarios, demonstrating strong generalization across diverse and challenging environments. Experimental results highlight the effectiveness and robustness of PPF-Tracker in multi-frame pose tracking of articulated objects. We believe this work can foster advances in robotics, embodied intelligence, and augmented reality. Codes are available at https://github.com/mengxh20/PPFTracker.

Method: Proposes MALeR with masked, attribute-aware binding mechanism that prevents subjects from appearing outside given layouts while maintaining in-distribution generation and preventing attribute leakage.

Result: Qualitative and quantitative evaluation shows superior performance in compositional accuracy, generation consistency, and attribute binding compared to previous methods, especially for scenes with multiple subjects and attributes per subject.

Conclusion: MALeR effectively addresses key challenges in compositional text-to-image generation, enabling more accurate and controlled generation of complex scenes with multiple subjects and attributes.

Abstract: Recent advances in text-to-image models have enabled a new era of creative and controllable image generation. However, generating compositional scenes with multiple subjects and attributes remains a significant challenge. To enhance user control over subject placement, several layout-guided methods have been proposed. However, these methods face numerous challenges, particularly in compositional scenes. Unintended subjects often appear outside the layouts, generated images can be out-of-distribution and contain unnatural artifacts, or attributes bleed across subjects, leading to incorrect visual outputs. In this work, we propose MALeR, a method that addresses each of these challenges. Given a text prompt and corresponding layouts, our method prevents subjects from appearing outside the given layouts while being in-distribution. Additionally, we propose a masked, attribute-aware binding mechanism that prevents attribute leakage, enabling accurate rendering of subjects with multiple attributes, even in complex compositional scenes. Qualitative and quantitative evaluation demonstrates that our method achieves superior performance in compositional accuracy, generation consistency, and attribute binding compared to previous work. MALeR is particularly adept at generating images of scenes with multiple subjects and multiple attributes per subject.

Method: Systematic analysis of 5 open-source VLMs on FairFace dataset for occupation prediction task across 32 occupations and three different prompting styles, eliciting both predictions and reasoning.

Result: Findings reveal that biased reasoning patterns systematically underlie intersectional disparities in VLM outputs.

Conclusion: There is a need to align VLM reasoning with human values prior to downstream deployment to mitigate social biases.

Abstract: Vision Language Models (VLMs) are increasingly deployed across downstream tasks, yet their training data often encode social biases that surface in outputs. Unlike humans, who interpret images through contextual and social cues, VLMs process them through statistical associations, often leading to reasoning that diverges from human reasoning. By analyzing how a VLM reasons, we can understand how inherent biases are perpetuated and can adversely affect downstream performance. To examine this gap, we systematically analyze social biases in five open-source VLMs for an occupation prediction task, on the FairFace dataset. Across 32 occupations and three different prompting styles, we elicit both predictions and reasoning. Our findings reveal that the biased reasoning patterns systematically underlie intersectional disparities, highlighting the need to align VLM reasoning with human values prior to its downstream deployment.

Method: Implemented U-Net and U-Net++ architectures with additive Gaussian noise obfuscation, evaluated under single-GPU, DataParallel, and optimized DistributedDataParallel with Automatic Mixed Precision training.

Result: U-Net++ showed superior denoising performance with better PSNR and SSIM scores, while optimized training pipeline reduced training time by over 60% compared to single-GPU and 40% vs DataParallel with minor accuracy drop.

Conclusion: Combining architectural design, lightweight obfuscation, and distributed training strategies is practically viable for accelerating medical image processing pipelines in clinical environments.

Abstract: Medical image denoising is essential for improving image quality while minimizing the exposure of sensitive information, particularly when working with large-scale clinical datasets. This study explores distributed deep learning for denoising chest X-ray images from the NIH Chest X-ray14 dataset, using additive Gaussian noise as a lightweight obfuscation technique. We implement and evaluate U-Net and U-Net++ architectures under single-GPU, standard multi-GPU (DataParallel), and optimized multi-GPU training configurations using PyTorch’s DistributedDataParallel (DDP) and Automatic Mixed Precision (AMP). Our results show that U-Net++ consistently delivers superior denoising performance, achieving competitive Peak Signal to Noise Ratio (PSNR) and Structured Similarity Index Method (SSIM) scores, though with less performance in Learned Perceptual Image Patch Similarity (LPIPS) compared to U-Net under low and moderate noise levels. This indicates U-Net++’s enhanced structural fidelity and low perceptual similarity. Meanwhile, our optimized training pipeline reduces training time by over 60% for both models compared to single-GPU training, and outperforms standard DataParallel by over 40%, with only a minor accuracy drop for both models (trading some accuracy for speed). These findings highlight the effectiveness of software-level optimization in distributed learning for medical imaging. This work demonstrates the practical viability of combining architectural design, lightweight obfuscation, and advanced distributed training strategies to accelerate and enhance medical image processing pipelines in real-world clinical and research environments. The full implementation is publicly available at: https://github.com/Suadey/medical-image-denoising-ddp.

Method: Propose OSKT (One-Shot Knowledge Transfer) that consolidates teacher model knowledge into a weight chain, which can be expanded to target model sizes without additional computation.

Result: OSKT significantly outperforms state-of-the-art compression methods while providing one-time knowledge transfer that eliminates frequent computations for each target model.

Conclusion: OSKT offers an efficient solution for deploying multiple compressed ReID models in edge computing scenarios by avoiding repetitive computations through one-shot knowledge transfer.

Abstract: Edge computing in person re-identification (ReID) is crucial for reducing the load on central cloud servers and ensuring user privacy. Conventional compression methods for obtaining compact models require computations for each individual student model. When multiple models of varying sizes are needed to accommodate different resource conditions, this leads to repetitive and cumbersome computations. To address this challenge, we propose a novel knowledge inheritance approach named OSKT (One-Shot Knowledge Transfer), which consolidates the knowledge of the teacher model into an intermediate carrier called a weight chain. When a downstream scenario demands a model that meets specific resource constraints, this weight chain can be expanded to the target model size without additional computation. OSKT significantly outperforms state-of-the-art compression methods, with the added advantage of one-time knowledge transfer that eliminates the need for frequent computations for each target model.

Method: Combines 3D U-Net backbone with transformer-style temporal attention, trained under hybrid masking regime with cosine-based progressive masking and adaptive loss scheduling. Entirely self-supervised using 9-frame video segments on CPU.

Result: Achieves optimal results at 50 epochs on UCF101-7 and DAVIS-7 datasets, competitive with supervised baselines despite low-resource constraints.

Conclusion: Demonstrates the power of self-supervised diffusion priors for temporally coherent frame synthesis and provides a scalable path toward accessible and generalizable VFI systems.

Abstract: Video Frame Interpolation (VFI) remains a cornerstone in video enhancement, enabling temporal upscaling for tasks like slow-motion rendering, frame rate conversion, and video restoration. While classical methods rely on optical flow and learning-based models assume access to dense ground-truth, both struggle with occlusions, domain shifts, and ambiguous motion. This article introduces MiVID, a lightweight, self-supervised, diffusion-based framework for video interpolation. Our model eliminates the need for explicit motion estimation by combining a 3D U-Net backbone with transformer-style temporal attention, trained under a hybrid masking regime that simulates occlusions and motion uncertainty. The use of cosine-based progressive masking and adaptive loss scheduling allows our network to learn robust spatiotemporal representations without any high-frame-rate supervision. Our framework is evaluated on UCF101-7 and DAVIS-7 datasets. MiVID is trained entirely on CPU using the datasets and 9-frame video segments, making it a low-resource yet highly effective pipeline. Despite these constraints, our model achieves optimal results at just 50 epochs, competitive with several supervised baselines.This work demonstrates the power of self-supervised diffusion priors for temporally coherent frame synthesis and provides a scalable path toward accessible and generalizable VFI systems.

Method: Introduce learnable aggregation tokens prepended to patch tokens before a transformer block, allowing implicit aggregation through self-attention, then concatenate these tokens as global representation. Also propose optimal token insertion strategy and initialization method.

Result: Outperforms state-of-the-art methods on several VPR datasets with higher efficiency, and ranks 1st on the MSLS challenge leaderboard.

Conclusion: Implicit aggregation via learnable tokens in transformers provides a simpler yet more effective alternative to traditional aggregators for visual place recognition.

Abstract: Visual place recognition (VPR) is typically regarded as a specific image retrieval task, whose core lies in representing images as global descriptors. Over the past decade, dominant VPR methods (e.g., NetVLAD) have followed a paradigm that first extracts the patch features/tokens of the input image using a backbone, and then aggregates these patch features into a global descriptor via an aggregator. This backbone-plus-aggregator paradigm has achieved overwhelming dominance in the CNN era and remains widely used in transformer-based models. In this paper, however, we argue that a dedicated aggregator is not necessary in the transformer era, that is, we can obtain robust global descriptors only with the backbone. Specifically, we introduce some learnable aggregation tokens, which are prepended to the patch tokens before a particular transformer block. All these tokens will be jointly processed and interact globally via the intrinsic self-attention mechanism, implicitly aggregating useful information within the patch tokens to the aggregation tokens. Finally, we only take these aggregation tokens from the last output tokens and concatenate them as the global representation. Although implicit aggregation can provide robust global descriptors in an extremely simple manner, where and how to insert additional tokens, as well as the initialization of tokens, remains an open issue worthy of further exploration. To this end, we also propose the optimal token insertion strategy and token initialization method derived from empirical studies. Experimental results show that our method outperforms state-of-the-art methods on several VPR datasets with higher efficiency and ranks 1st on the MSLS challenge leaderboard. The code is available at https://github.com/lu-feng/image.

Method: Proposes Spatio-Spectral Mutual Learning (S2ML) framework that harmonizes spatial and frequency domains. Uses dedicated spectral fusion module for amplitude and phase spectra, calculates local and global correlations in unified embedding space, and employs gradual mutual representation and refinement.

Result: Outperforms state-of-the-art method CFormer by 0.828 dB on NYU-Depth V2 and 0.834 dB on SUN RGB-D datasets.

Conclusion: The S2ML framework effectively explores complementary physical characteristics and priors from both spatial and frequency domains for more accurate depth completion.

Abstract: The raw depth images captured by RGB-D cameras using Time-of-Flight (TOF) or structured light often suffer from incomplete depth values due to weak reflections, boundary shadows, and artifacts, which limit their applications in downstream vision tasks. Existing methods address this problem through depth completion in the image domain, but they overlook the physical characteristics of raw depth images. It has been observed that the presence of invalid depth areas alters the frequency distribution pattern. In this work, we propose a Spatio-Spectral Mutual Learning framework (S2ML) to harmonize the advantages of both spatial and frequency domains for depth completion. Specifically, we consider the distinct properties of amplitude and phase spectra and devise a dedicated spectral fusion module. Meanwhile, the local and global correlations between spatial-domain and frequency-domain features are calculated in a unified embedding space. The gradual mutual representation and refinement encourage the network to fully explore complementary physical characteristics and priors for more accurate depth completion. Extensive experiments demonstrate the effectiveness of our proposed S2ML method, outperforming the state-of-the-art method CFormer by 0.828 dB and 0.834 dB on the NYU-Depth V2 and SUN RGB-D datasets, respectively.

Method: Represents dynamic scenes using canonical 3D Gaussians, temporal features, and deformation field. Encodes Gaussian attributes as 2D images and temporal features as video. Uses sliding window for local motion and transformer-guided auxiliary training for global motion.

Result: Achieves competitive performance on diverse FVV benchmarks, increasing PSNR by average 1dB while reducing frame size to 170KB average (compared to 10MB in previous methods).

Conclusion: StreamSTGS enables real-time FVV streaming with adaptive bitrate control, demonstrating significant compression efficiency while maintaining or improving visual quality compared to state-of-the-art methods.

Abstract: Streaming free-viewpoint video~(FVV) in real-time still faces significant challenges, particularly in training, rendering, and transmission efficiency. Harnessing superior performance of 3D Gaussian Splatting~(3DGS), recent 3DGS-based FVV methods have achieved notable breakthroughs in both training and rendering. However, the storage requirements of these methods can reach up to $10$MB per frame, making stream FVV in real-time impossible. To address this problem, we propose a novel FVV representation, dubbed StreamSTGS, designed for real-time streaming. StreamSTGS represents a dynamic scene using canonical 3D Gaussians, temporal features, and a deformation field. For high compression efficiency, we encode canonical Gaussian attributes as 2D images and temporal features as a video. This design not only enables real-time streaming, but also inherently supports adaptive bitrate control based on network condition without any extra training. Moreover, we propose a sliding window scheme to aggregate adjacent temporal features to learn local motions, and then introduce a transformer-guided auxiliary training module to learn global motions. On diverse FVV benchmarks, StreamSTGS demonstrates competitive performance on all metrics compared to state-of-the-art methods. Notably, StreamSTGS increases the PSNR by an average of $1$dB while reducing the average frame size to just $170$KB. The code is publicly available on https://github.com/kkkzh/StreamSTGS.

Method: Four technical contributions: (1) Text-Encoder Distillation to replace T5xxl with smaller DT5, (2) Asymmetric Decoder Distillation for efficient VAE decoder, (3) Pruning MMDiT blocks with two-stage distillation, (4) Reducing NFE through step distillation with DMD for pyramidal flow-matching.

Result: Achieves 81.61 VBench score with 4.945B parameters, 3.5GB peak RAM usage, and 6.7s end-to-end latency, generating 2s videos at 640x1024 resolution on mobile hardware.

Conclusion: Neodragon successfully enables efficient, high-fidelity on-device text-to-video generation, making AI video creation accessible without cloud dependency.

Abstract: We introduce Neodragon, a text-to-video system capable of generating 2s (49 frames @24 fps) videos at the 640x1024 resolution directly on a Qualcomm Hexagon NPU in a record 6.7s (7 FPS). Differing from existing transformer-based offline text-to-video generation models, Neodragon is the first to have been specifically optimised for mobile hardware to achieve efficient and high-fidelity video synthesis. We achieve this through four key technical contributions: (1) Replacing the original large 4.762B T5xxl Text-Encoder with a much smaller 0.2B DT5 (DistilT5) with minimal quality loss, enabled through a novel Text-Encoder Distillation procedure. (2) Proposing an Asymmetric Decoder Distillation approach allowing us to replace the native codec-latent-VAE decoder with a more efficient one, without disturbing the generative latent-space of the generation pipeline. (3) Pruning of MMDiT blocks within the denoiser backbone based on their relative importance, with recovery of original performance through a two-stage distillation process. (4) Reducing the NFE (Neural Functional Evaluation) requirement of the denoiser by performing step distillation using DMD adapted for pyramidal flow-matching, thereby substantially accelerating video generation. When paired with an optimised SSD1B first-frame image generator and QuickSRNet for 2x super-resolution, our end-to-end Neodragon system becomes a highly parameter (4.945B full model), memory (3.5GB peak RAM usage), and runtime (6.7s E2E latency) efficient mobile-friendly model, while achieving a VBench total score of 81.61. By enabling low-cost, private, and on-device text-to-video synthesis, Neodragon democratizes AI-based video content creation, empowering creators to generate high-quality videos without reliance on cloud services. Code and model will be made publicly available at our website: https://qualcomm-ai-research.github.io/neodragon

Method: Proposes a nested loop optimization strategy with two-level framework: upper-level trains correction model using pseudo-labels from lower-level multi-exposure fusion, with feedback mechanism for refinement. Introduces Luminance Ranking Loss for self-supervised constraint.

Result: Extensive experiments show LoopExpose achieves superior exposure correction and fusion performance, outperforming existing state-of-the-art unsupervised methods.

Conclusion: The proposed unsupervised approach effectively addresses exposure correction without requiring labeled data, demonstrating strong performance through the nested loop optimization and luminance-based constraints.

Abstract: Exposure correction is essential for enhancing image quality under challenging lighting conditions. While supervised learning has achieved significant progress in this area, it relies heavily on large-scale labeled datasets, which are difficult to obtain in practical scenarios. To address this limitation, we propose a pseudo label-based unsupervised method called LoopExpose for arbitrary-length exposure correction. A nested loop optimization strategy is proposed to address the exposure correction problem, where the correction model and pseudo-supervised information are jointly optimized in a two-level framework. Specifically, the upper-level trains a correction model using pseudo-labels generated through multi-exposure fusion at the lower level. A feedback mechanism is introduced where corrected images are fed back into the fusion process to refine the pseudo-labels, creating a self-reinforcing learning loop. Considering the dominant role of luminance calibration in exposure correction, a Luminance Ranking Loss is introduced to leverage the relative luminance ordering across the input sequence as a self-supervised constraint. Extensive experiments on different benchmark datasets demonstrate that LoopExpose achieves superior exposure correction and fusion performance, outperforming existing state-of-the-art unsupervised methods. Code is available at https://github.com/FALALAS/LoopExpose.

Method: Uses state-of-the-art machine learning algorithms including You Only Look Once architectures and a Large Language and Vision Assistant to identify objects, read text, and answer environmental questions.

Result: The application enhances user autonomy and access to information, with user feedback supporting improved perception of daily usability.

Conclusion: AIDEN successfully contributes to improving the quality of life for visually impaired individuals by leveraging AI technologies to address key accessibility challenges.

Abstract: This paper describes an artificial intelligence-based assistant application, AIDEN, developed during 2023 and 2024, aimed at improving the quality of life for visually impaired individuals. Visually impaired individuals face challenges in identifying objects, reading text, and navigating unfamiliar environments, which can limit their independence and reduce their quality of life. Although solutions such as Braille, audio books, and screen readers exist, they may not be effective in all situations. This application leverages state-of-the-art machine learning algorithms to identify and describe objects, read text, and answer questions about the environment. Specifically, it uses You Only Look Once architectures and a Large Language and Vision Assistant. The system incorporates several methods to facilitate the user’s interaction with the system and access to textual and visual information in an appropriate manner. AIDEN aims to enhance user autonomy and access to information, contributing to an improved perception of daily usability, as supported by user feedback.

Method: Hybrid deep learning framework combining CNNs with vision transformers. Uses CNN encoder-decoder for structural details and transformer module with self-attention for global context. Trained on TextOCR dataset with synthetic motion blur using composite loss (MAE, MSE, perceptual similarity, SSIM).

Result: Achieves 32.20 dB PSNR and 0.934 SSIM with only 2.83M parameters and 61ms average inference time, demonstrating both effectiveness and computational efficiency.

Conclusion: The CNN-ViT hybrid design proves practical for real-world motion-blurred scene-text restoration, effectively addressing limitations of conventional approaches through combined local and global feature processing.

Abstract: Motion blur in scene text images severely impairs readability and hinders the reliability of computer vision tasks, including autonomous driving, document digitization, and visual information retrieval. Conventional deblurring approaches are often inadequate in handling spatially varying blur and typically fall short in modeling the long-range dependencies necessary for restoring textual clarity. To overcome these limitations, we introduce a hybrid deep learning framework that combines convolutional neural networks (CNNs) with vision transformers (ViTs), thereby leveraging both local feature extraction and global contextual reasoning. The architecture employs a CNN-based encoder-decoder to preserve structural details, while a transformer module enhances global awareness through self-attention. Training is conducted on a curated dataset derived from TextOCR, where sharp scene-text samples are paired with synthetically blurred versions generated using realistic motion-blur kernels of multiple sizes and orientations. Model optimization is guided by a composite loss that incorporates mean absolute error (MAE), squared error (MSE), perceptual similarity, and structural similarity (SSIM). Quantitative evaluations show that the proposed method attains 32.20 dB in PSNR and 0.934 in SSIM, while remaining lightweight with 2.83 million parameters and an average inference time of 61 ms. These results highlight the effectiveness and computational efficiency of the CNN-ViT hybrid design, establishing its practicality for real-world motion-blurred scene-text restoration.

Method: Joint optimization of generator network with shape/deformation factors using regularization, followed by training two order-invariant PoinNet-based encoders for amortized inference.

Result: Method effectively performs unsupervised deformation transfer, classification, and explainability analysis on 3D human, animal, and facial datasets.

Conclusion: Simple approach achieves comparable or superior performance to more complex methods in multiple downstream tasks.

Abstract: In this work, we propose a disentangled latent optimization-based method for parameterizing grouped deforming 3D objects into shape and deformation factors in an unsupervised manner. Our approach involves the joint optimization of a generator network along with the shape and deformation factors, supported by specific regularization techniques. For efficient amortized inference of disentangled shape and deformation codes, we train two order-invariant PoinNet-based encoder networks in the second stage of our method. We demonstrate several significant downstream applications of our method, including unsupervised deformation transfer, deformation classification, and explainability analysis. Extensive experiments conducted on 3D human, animal, and facial expression datasets demonstrate that our simple approach is highly effective in these downstream tasks, comparable or superior to existing methods with much higher complexity.

Method: LFlow performs ODE sampling in latent space using flow matching along an optimal path, and introduces a theoretically grounded posterior covariance derived from the optimal vector field for effective flow guidance.

Result: Experimental results show LFlow outperforms state-of-the-art latent diffusion solvers in reconstruction quality across most linear inverse problem tasks.

Conclusion: The proposed latent flow framework with optimal posterior covariance provides an efficient and effective solution for linear inverse problems, demonstrating superior performance compared to existing methods.

Abstract: Recent advances in inverse problem solving have increasingly adopted flow priors over diffusion models due to their ability to construct straight probability paths from noise to data, thereby enhancing efficiency in both training and inference. However, current flow-based inverse solvers face two primary limitations: (i) they operate directly in pixel space, which demands heavy computational resources for training and restricts scalability to high-resolution images, and (ii) they employ guidance strategies with prior-agnostic posterior covariances, which can weaken alignment with the generative trajectory and degrade posterior coverage. In this paper, we propose LFlow (Latent Refinement via Flows), a training-free framework for solving linear inverse problems via pretrained latent flow priors. LFlow leverages the efficiency of flow matching to perform ODE sampling in latent space along an optimal path. This latent formulation further allows us to introduce a theoretically grounded posterior covariance, derived from the optimal vector field, enabling effective flow guidance. Experimental results demonstrate that our proposed method outperforms state-of-the-art latent diffusion solvers in reconstruction quality across most tasks. The code will be publicly available at https://github.com/hosseinaskari-cs/LFlow .

Method: Divides each image into user-defined patches (e.g., 150x150 pixels), dynamically tracks patches across frames using GNSS/IMU data and DSM, and performs localized BA on sliding clusters of overlapping images determined by UAV navigation system.

Result: The method maintains precise camera orientations and high-fidelity mapping across multiple strips, running full bundle adjustment in under 2 seconds without GPU acceleration on 50MP MACS datasets.

Conclusion: The proposed lightweight, onboard-compatible framework enables real-time processing of full-resolution UAV imagery for applications like disaster response, infrastructure monitoring, and coastal protection.

Abstract: Real-time processing of UAV imagery is crucial for applications requiring urgent geospatial information, such as disaster response, where rapid decision-making and accurate spatial data are essential. However, processing high-resolution imagery in real time presents significant challenges due to the computational demands of feature extraction, matching, and bundle adjustment (BA). Conventional BA methods either downsample images, sacrificing important details, or require extensive processing time, making them unsuitable for time-critical missions. To overcome these limitations, we propose a novel real-time BA framework that operates directly on fullresolution UAV imagery without downsampling. Our lightweight, onboard-compatible approach divides each image into user-defined patches (e.g., NxN grids, default 150x150 pixels) and dynamically tracks them across frames using UAV GNSS/IMU data and a coarse, globally available digital surface model (DSM). This ensures spatial consistency for robust feature extraction and matching between patches. Overlapping relationships between images are determined in real time using UAV navigation system, enabling the rapid selection of relevant neighbouring images for localized BA. By limiting optimization to a sliding cluster of overlapping images, including those from adjacent flight strips, the method achieves real-time performance while preserving the accuracy of global BA. The proposed algorithm is designed for seamless integration into the DLR Modular Aerial Camera System (MACS), supporting largearea mapping in real time for disaster response, infrastructure monitoring, and coastal protection. Validation on MACS datasets with 50MP images demonstrates that the method maintains precise camera orientations and high-fidelity mapping across multiple strips, running full bundle adjustment in under 2 seconds without GPU acceleration.

Method: MambaOVSR with three novel components: Global Fusion Module (GFM) for motion modeling via multiscale alternating scanning, Multiscale Synergistic Mamba Module (MSMM) for alignment across sequence lengths, and MambaVR block to resolve feature artifacts and positional information loss.

Result: Significantly outperforms SOTA STVSR method by average 1.86 dB in PSNR on the COVC dataset. Dataset and code will be publicly released.

Conclusion: MambaOVSR effectively addresses challenges in Chinese opera video super-resolution through novel Mamba-based architecture and multiscale fusion, enabling better preservation of classical art performances.

Abstract: Chinese opera is celebrated for preserving classical art. However, early filming equipment limitations have degraded videos of last-century performances by renowned artists (e.g., low frame rates and resolution), hindering archival efforts. Although space-time video super-resolution (STVSR) has advanced significantly, applying it directly to opera videos remains challenging. The scarcity of datasets impedes the recovery of high frequency details, and existing STVSR methods lack global modeling capabilities, compromising visual quality when handling opera’s characteristic large motions. To address these challenges, we pioneer a large scale Chinese Opera Video Clip (COVC) dataset and propose the Mamba-based multiscale fusion network for space-time Opera Video Super-Resolution (MambaOVSR). Specifically, MambaOVSR involves three novel components: the Global Fusion Module (GFM) for motion modeling through a multiscale alternating scanning mechanism, and the Multiscale Synergistic Mamba Module (MSMM) for alignment across different sequence lengths. Additionally, our MambaVR block resolves feature artifacts and positional information loss during alignment. Experimental results on the COVC dataset show that MambaOVSR significantly outperforms the SOTA STVSR method by an average of 1.86 dB in terms of PSNR. Dataset and Code will be publicly released.

Method: Fine-tune a large language model to translate text into JSON representations of NURBS surface parameters, using a hybrid representation combining untrimmed NURBS with analytic primitives to handle trimmed surfaces and reduce token complexity.

Result: NURBGen demonstrates strong performance on diverse prompts, surpassing prior methods in geometric fidelity and dimensional accuracy as confirmed by expert evaluations.

Conclusion: The framework successfully generates high-fidelity 3D CAD models directly from text using NURBS, with the code and dataset to be released publicly.

Abstract: Generating editable 3D CAD models from natural language remains challenging, as existing text-to-CAD systems either produce meshes or rely on scarce design-history data. We present NURBGen, the first framework to generate high-fidelity 3D CAD models directly from text using Non-Uniform Rational B-Splines (NURBS). To achieve this, we fine-tune a large language model (LLM) to translate free-form texts into JSON representations containing NURBS surface parameters (\textit{i.e}, control points, knot vectors, degrees, and rational weights) which can be directly converted into BRep format using Python. We further propose a hybrid representation that combines untrimmed NURBS with analytic primitives to handle trimmed surfaces and degenerate regions more robustly, while reducing token complexity. Additionally, we introduce partABC, a curated subset of the ABC dataset consisting of individual CAD components, annotated with detailed captions using an automated annotation pipeline. NURBGen demonstrates strong performance on diverse prompts, surpassing prior methods in geometric fidelity and dimensional accuracy, as confirmed by expert evaluations. Code and dataset will be released publicly.

Method: Uses Grounding DINO and curated ADE20K dataset to detect urban objects, builds co-occurrence embeddings to reveal spatial configurations, provides five statistically likely complements to chosen anchor objects, and employs vision language model to suggest third objects that complete complex urban tactics.

Result: The system successfully detects urban objects, identifies common spatial configurations, and generates meaningful design interventions while keeping users in control of selection and refinement.

Conclusion: The framework effectively supports participatory urban design by combining AI-driven analysis with human oversight, enabling micro-scale interventions that reflect local patterns and experiences.

Abstract: This paper introduces a human-in-the-loop computer vision framework that uses generative AI to propose micro-scale design interventions in public space and support more continuous, local participation. Using Grounding DINO and a curated subset of the ADE20K dataset as a proxy for the urban built environment, the system detects urban objects and builds co-occurrence embeddings that reveal common spatial configurations. From this analysis, the user receives five statistically likely complements to a chosen anchor object. A vision language model then reasons over the scene image and the selected pair to suggest a third object that completes a more complex urban tactic. The workflow keeps people in control of selection and refinement and aims to move beyond top-down master planning by grounding choices in everyday patterns and lived experience.

Method: MoRA introduces modality-common parameters between text and vision encoders for bidirectional knowledge transfer, combined with modality-specific parameters to maintain inter-modality interaction and intra-modality flexibility.

Result: Extensive experiments show MoRA achieves 5.24% average performance improvement in missing-modality scenarios, uses only 25.90% of SOTA inference time, and requires only 0.11% of trainable parameters compared to full fine-tuning.

Conclusion: MoRA provides an effective parameter-efficient solution for handling missing modalities in vision-language models by explicitly modeling cross-modal interactions while maintaining computational efficiency.

Abstract: Pre-trained vision language models have shown remarkable performance on visual recognition tasks, but they typically assume the availability of complete multimodal inputs during both training and inference. In real-world scenarios, however, modalities may be missing due to privacy constraints, collection difficulties, or resource limitations. While previous approaches have addressed this challenge using prompt learning techniques, they fail to capture the cross-modal relationships necessary for effective multimodal visual recognition and suffer from inevitable computational overhead. In this paper, we introduce MoRA, a parameter-efficient fine-tuning method that explicitly models cross-modal interactions while maintaining modality-specific adaptations. MoRA introduces modality-common parameters between text and vision encoders, enabling bidirectional knowledge transfer. Additionally, combined with the modality-specific parameters, MoRA allows the backbone model to maintain inter-modality interaction and enable intra-modality flexibility. Extensive experiments on standard benchmarks demonstrate that MoRA achieves an average performance improvement in missing-modality scenarios by 5.24% and uses only 25.90% of the inference time compared to the SOTA method while requiring only 0.11% of trainable parameters compared to full fine-tuning.

Method: Proposes Temporal-Guided VFM with three components: Long-Range Temporal Attention, Dual Spatiotemporal Attention, and Deep Feature Guidance Mechanism. Retrains event-to-video models on real data and uses transformer-based VFMs.

Result: Achieves state-of-the-art performance with 16% improvement in semantic segmentation, 21% in depth estimation, and 16% in object detection over existing methods.

Conclusion: Successfully bridges cross-modality gap by enabling image-based VFMs to work with event-based vision through temporal reasoning.

Abstract: Event cameras offer unique advantages for vision tasks in challenging environments, yet processing asynchronous event streams remains an open challenge. While existing methods rely on specialized architectures or resource-intensive training, the potential of leveraging modern Visual Foundation Models (VFMs) pretrained on image data remains under-explored for event-based vision. To address this, we propose Temporal-Guided VFM (TGVFM), a novel framework that integrates VFMs with our temporal context fusion block seamlessly to bridge this gap. Our temporal block introduces three key components: (1) Long-Range Temporal Attention to model global temporal dependencies, (2) Dual Spatiotemporal Attention for multi-scale frame correlation, and (3) Deep Feature Guidance Mechanism to fuse semantic-temporal features. By retraining event-to-video models on real-world data and leveraging transformer-based VFMs, TGVFM preserves spatiotemporal dynamics while harnessing pretrained representations. Experiments demonstrate SoTA performance across semantic segmentation, depth estimation, and object detection, with improvements of 16%, 21%, and 16% over existing methods, respectively. Overall, this work unlocks the cross-modality potential of image-based VFMs for event-based vision with temporal reasoning. Code is available at https://github.com/XiaRho/TGVFM.

Method: Progressive training framework that integrates physics-informed PDE dynamics (advection-diffusion equations) into state-of-the-art restoration architectures to model feature evolution and directional flow characteristics of motion blur.

Result: Achieves superior restoration quality with only ~1% increase in inference GMACs. Improves PSNR and SSIM significantly across four architectures (FFTformer, NAFNet, Restormer, Stripformer) on standard benchmarks.

Conclusion: Incorporating mathematical physics principles through PDE-based global layers enhances deep learning-based image restoration, establishing a promising direction for physics-informed neural network design in computer vision.

Abstract: Motion blur, caused by relative movement between camera and scene during exposure, significantly degrades image quality and impairs downstream computer vision tasks such as object detection, tracking, and recognition in dynamic environments. While deep learning-based motion deblurring methods have achieved remarkable progress, existing approaches face fundamental challenges in capturing the long-range spatial dependencies inherent in motion blur patterns. Traditional convolutional methods rely on limited receptive fields and require extremely deep networks to model global spatial relationships. These limitations motivate the need for alternative approaches that incorporate physical priors to guide feature evolution during restoration. In this paper, we propose a progressive training framework that integrates physics-informed PDE dynamics into state-of-the-art restoration architectures. By leveraging advection-diffusion equations to model feature evolution, our approach naturally captures the directional flow characteristics of motion blur while enabling principled global spatial modeling. Our PDE-enhanced deblurring models achieve superior restoration quality with minimal overhead, adding only approximately 1% to inference GMACs while providing consistent improvements in perceptual quality across multiple state-of-the-art architectures. Comprehensive experiments on standard motion deblurring benchmarks demonstrate that our physics-informed approach improves PSNR and SSIM significantly across four diverse architectures, including FFTformer, NAFNet, Restormer, and Stripformer. These results validate that incorporating mathematical physics principles through PDE-based global layers can enhance deep learning-based image restoration, establishing a promising direction for physics-informed neural network design in computer vision applications.

Method: Proposes CoD² framework with Multi-level Conditional Control strategy: high-level identity-aware semantic conditions from discriminative extractor guide generation, while low-level visual details (appearance, motion) enhance consistency. Generated sequences improve discriminative extractor learning.

Result: Achieves state-of-the-art performance on four datasets (SUSTech1K, CCPG, GREW, Gait3D) and integrates seamlessly with existing discriminative methods for consistent improvements.

Conclusion: CoD² effectively combines generative and discriminative approaches for robust gait feature extraction, demonstrating superior performance and compatibility with existing methods.

Abstract: Gait recognition offers a non-intrusive biometric solution by identifying individuals through their walking patterns. Although discriminative models have achieved notable success in this domain, the full potential of generative models remains largely underexplored. In this paper, we introduce \textbf{CoD$^2$}, a novel framework that combines the data distribution modeling capabilities of diffusion models with the semantic representation learning strengths of discriminative models to extract robust gait features. We propose a Multi-level Conditional Control strategy that incorporates both high-level identity-aware semantic conditions and low-level visual details. Specifically, the high-level condition, extracted by the discriminative extractor, guides the generation of identity-consistent gait sequences, whereas low-level visual details, such as appearance and motion, are preserved to enhance consistency. Furthermore, the generated sequences facilitate the discriminative extractor’s learning, enabling it to capture more comprehensive high-level semantic features. Extensive experiments on four datasets (SUSTech1K, CCPG, GREW, and Gait3D) demonstrate that CoD$^2$ achieves state-of-the-art performance and can be seamlessly integrated with existing discriminative methods, yielding consistent improvements.

Method: Uses adaptive activation loss for determining when to invoke LLMs based on comparative learning, and adaptive fusion strategy for continuous, scaled LLM influence based on scene complexity and prediction confidence.

Result: Achieves state-of-the-art performance on language-grounded autonomous driving benchmarks in both driving accuracy and computational efficiency.

Conclusion: AdaDrive provides a flexible, context-aware framework that maximizes decision accuracy without compromising real-time performance in autonomous driving systems.

Abstract: Effectively integrating Large Language Models (LLMs) into autonomous driving requires a balance between leveraging high-level reasoning and maintaining real-time efficiency. Existing approaches either activate LLMs too frequently, causing excessive computational overhead, or use fixed schedules, failing to adapt to dynamic driving conditions. To address these challenges, we propose AdaDrive, an adaptively collaborative slow-fast framework that optimally determines when and how LLMs contribute to decision-making. (1) When to activate the LLM: AdaDrive employs a novel adaptive activation loss that dynamically determines LLM invocation based on a comparative learning mechanism, ensuring activation only in complex or critical scenarios. (2) How to integrate LLM assistance: Instead of rigid binary activation, AdaDrive introduces an adaptive fusion strategy that modulates a continuous, scaled LLM influence based on scene complexity and prediction confidence, ensuring seamless collaboration with conventional planners. Through these strategies, AdaDrive provides a flexible, context-aware framework that maximizes decision accuracy without compromising real-time performance. Extensive experiments on language-grounded autonomous driving benchmarks demonstrate that AdaDrive state-of-the-art performance in terms of both driving accuracy and computational efficiency. Code is available at https://github.com/ReaFly/AdaDrive.

Method: Introduces a lightweight MLLM architecture with cycle-consistent dynamic visual pruning, memory-enhanced feature aggregation, and distance-decoupled instruction attention for improved visual-linguistic feature learning.

Result: Achieves state-of-the-art driving performance with 81% parameter reduction (from 7B to 1.3B) and substantial driving score improvements: 15.4% (tiny), 16.8% (short), and 7.6% (long distances) in CARLA simulator.

Conclusion: VLDrive demonstrates that lightweight MLLM architectures with enhanced vision components can achieve superior autonomous driving performance while being more deployable through significant parameter reduction.

Abstract: Recent advancements in language-grounded autonomous driving have been significantly promoted by the sophisticated cognition and reasoning capabilities of large language models (LLMs). However, current LLM-based approaches encounter critical challenges: (1) Failure analysis reveals that frequent collisions and obstructions, stemming from limitations in visual representations, remain primary obstacles to robust driving performance. (2) The substantial parameters of LLMs pose considerable deployment hurdles. To address these limitations, we introduce VLDrive, a novel approach featuring a lightweight MLLM architecture with enhanced vision components. VLDrive achieves compact visual tokens through innovative strategies, including cycle-consistent dynamic visual pruning and memory-enhanced feature aggregation. Furthermore, we propose a distance-decoupled instruction attention mechanism to improve joint visual-linguistic feature learning, particularly for long-range visual tokens. Extensive experiments conducted in the CARLA simulator demonstrate VLDrive`s effectiveness. Notably, VLDrive achieves state-of-the-art driving performance while reducing parameters by 81% (from 7B to 1.3B), yielding substantial driving score improvements of 15.4%, 16.8%, and 7.6% at tiny, short, and long distances, respectively, in closed-loop evaluations. Code is available at https://github.com/ReaFly/VLDrive.

Method: Uses a lightweight query-specific trigger patch added to query images, weakly backdooring the network to steer patched inputs toward a dummy class. Similar images shift easily to the dummy class while dissimilar ones resist.

Result: TMM-NN outperforms traditional retrieval metrics under noise and across diverse tasks, with robustness analysis confirming its effectiveness.

Conclusion: Trigger-based ranking through targeted manifold manipulation provides more semantically meaningful nearest-neighbor retrieval than traditional geometric distance approaches.

Abstract: Nearest-neighbour retrieval is central to classification and explainable-AI pipelines, but current practice relies on hand-tuning feature layers and distance metrics. We propose Targeted Manifold Manipulation-Nearest Neighbour (TMM-NN), which reconceptualises retrieval by assessing how readily each sample can be nudged into a designated region of the feature manifold; neighbourhoods are defined by a sample’s responsiveness to a targeted perturbation rather than absolute geometric distance. TMM-NN implements this through a lightweight, query-specific trigger patch. The patch is added to the query image, and the network is weakly ``backdoored’’ so that any input with the patch is steered toward a dummy class. Images similar to the query need only a slight shift and are classified as the dummy class with high probability, while dissimilar ones are less affected. By ranking candidates by this confidence, TMM-NN retrieves the most semantically related neighbours. Robustness analysis and benchmark experiments confirm this trigger-based ranking outperforms traditional metrics under noise and across diverse tasks.

Method: Four-component framework: Quantile Gated Patch Selection, Graph Guided Clustering, Hierarchical Context Attention, and Expert Driven Mixture of Log logistics distributions.

Result: Achieved concordance indices of 0.644 on TCGA LUAD, 0.751 on TCGA KIRC, and 0.752 on TCGA BRCA, outperforming state-of-the-art methods.

Conclusion: The proposed modular framework effectively predicts cancer survival by integrating tissue heterogeneity modeling and complex distribution estimation, demonstrating superior performance across multiple cancer types.

Abstract: We propose a modular framework for predicting cancer specific survival from whole slide pathology images (WSIs). The method integrates four components: (i) Quantile Gated Patch Selection via quantile based thresholding to isolate prognostically informative tissue regions; (ii) Graph Guided Clustering using a k nearest neighbor graph to capture phenotype level heterogeneity through spatial and morphological coherence; (iii) Hierarchical Context Attention to learn intra and inter cluster interactions; and (iv) an Expert Driven Mixture of Log logistics framework to estimate complex survival distributions using Log logistics distributions. The model attains a concordance index of 0.644 on TCGA LUAD, 0.751 on TCGA KIRC, and 0.752 on TCGA BRCA respectively, outperforming existing state of the art approaches.

Method: Proposes C³ framework with completeness evaluation module using visual cues and language models, and Markov Decision Process to supervise Chain-of-Thought reasoning for consistency evaluation through adaptive query control.

Result: Achieves state-of-the-art performance on cultural heritage datasets CulTi and TimeTravel, as well as general benchmarks MSCOCO and Flickr30K in both fine-tuned and zero-shot settings.

Conclusion: C³ effectively addresses LLM limitations in cross-modal retrieval by improving description completeness and consistency, demonstrating strong performance across cultural heritage and general datasets.

Abstract: Cross-modal retrieval is essential for interpreting cultural heritage data, but its effectiveness is often limited by incomplete or inconsistent textual descriptions, caused by historical data loss and the high cost of expert annotation. While large language models (LLMs) offer a promising solution by enriching textual descriptions, their outputs frequently suffer from hallucinations or miss visually grounded details. To address these challenges, we propose $C^3$, a data augmentation framework that enhances cross-modal retrieval performance by improving the completeness and consistency of LLM-generated descriptions. $C^3$ introduces a completeness evaluation module to assess semantic coverage using both visual cues and language-model outputs. Furthermore, to mitigate factual inconsistencies, we formulate a Markov Decision Process to supervise Chain-of-Thought reasoning, guiding consistency evaluation through adaptive query control. Experiments on the cultural heritage datasets CulTi and TimeTravel, as well as on general benchmarks MSCOCO and Flickr30K, demonstrate that $C^3$ achieves state-of-the-art performance in both fine-tuned and zero-shot settings.

Method: 1) Built RelightVideo dataset with identical dynamic content under varying precise lighting conditions using Unreal Engine; 2) Introduced Multi-plane Light Image (MPLI) - a novel visual prompt that models lighting via K depth-aligned planes representing 3D light source positions, intensities, and colors; 3) Designed Light Image Adapter that compresses MPLI via pre-trained Video VAE and injects latent light features into DiT blocks.

Result: RelightMaster generates physically plausible lighting and shadows while preserving original scene content. The framework supports multi-source lighting scenarios and generalizes to unseen light setups.

Conclusion: The proposed RelightMaster framework successfully addresses video relighting challenges by combining a novel lighting dataset, MPLI visual prompts, and seamless integration with pre-trained diffusion models, enabling precise and controllable video relighting that was previously unexplored.

Abstract: Recent advances in diffusion models enable high-quality video generation and editing, but precise relighting with consistent video contents, which is critical for shaping scene atmosphere and viewer attention, remains unexplored. Mainstream text-to-video (T2V) models lack fine-grained lighting control due to text’s inherent limitation in describing lighting details and insufficient pre-training on lighting-related prompts. Additionally, constructing high-quality relighting training data is challenging, as real-world controllable lighting data is scarce. To address these issues, we propose RelightMaster, a novel framework for accurate and controllable video relighting. First, we build RelightVideo, the first dataset with identical dynamic content under varying precise lighting conditions based on the Unreal Engine. Then, we introduce Multi-plane Light Image (MPLI), a novel visual prompt inspired by Multi-Plane Image (MPI). MPLI models lighting via K depth-aligned planes, representing 3D light source positions, intensities, and colors while supporting multi-source scenarios and generalizing to unseen light setups. Third, we design a Light Image Adapter that seamlessly injects MPLI into pre-trained Video Diffusion Transformers (DiT): it compresses MPLI via a pre-trained Video VAE and injects latent light features into DiT blocks, leveraging the base model’s generative prior without catastrophic forgetting. Experiments show that RelightMaster generates physically plausible lighting and shadows and preserves original scene content. Demos are available at https://wkbian.github.io/Projects/RelightMaster/.

Method: Uses diffusion models to generate lane centerline priors at BEV feature level, integrating Lane Prior Injection Module (LPIM) and Lane Prior Diffusion Module (LPDM) to construct diffusion targets and manage the process, then decodes vectorized centerlines from prior-injected features.

Result: Significantly outperforms existing methods on nuScenes and Argoverse2 datasets with improvements of 4.2-6.4% on point-level metrics and 2.1-6.8% on segment-level metrics, establishing state-of-the-art performance.

Conclusion: LaneDiffusion demonstrates the effectiveness of generative models for centerline graph learning, offering new insights and superior performance compared to traditional deterministic approaches.

Abstract: Centerline graphs, crucial for path planning in autonomous driving, are traditionally learned using deterministic methods. However, these methods often lack spatial reasoning and struggle with occluded or invisible centerlines. Generative approaches, despite their potential, remain underexplored in this domain. We introduce LaneDiffusion, a novel generative paradigm for centerline graph learning. LaneDiffusion innovatively employs diffusion models to generate lane centerline priors at the Bird’s Eye View (BEV) feature level, instead of directly predicting vectorized centerlines. Our method integrates a Lane Prior Injection Module (LPIM) and a Lane Prior Diffusion Module (LPDM) to effectively construct diffusion targets and manage the diffusion process. Furthermore, vectorized centerlines and topologies are then decoded from these prior-injected BEV features. Extensive evaluations on the nuScenes and Argoverse2 datasets demonstrate that LaneDiffusion significantly outperforms existing methods, achieving improvements of 4.2%, 4.6%, 4.7%, 6.4% and 1.8% on fine-grained point-level metrics (GEO F1, TOPO F1, JTOPO F1, APLS and SDA) and 2.3%, 6.4%, 6.8% and 2.1% on segment-level metrics (IoU, mAP_cf, DET_l and TOP_ll). These results establish state-of-the-art performance in centerline graph learning, offering new insights into generative models for this task.

Method: Proposes three self-supervised pretext tasks (Anomaly Grounding, Object Counting, Temporal Jigsaw) and creates VideoSSR-30K dataset. Develops VideoSSR framework for video self-supervised reinforcement learning with verifiable rewards.

Result: VideoSSR consistently enhances model performance across 17 benchmarks in four video domains (General Video QA, Long Video QA, Temporal Grounding, Complex Reasoning), achieving average improvement of over 5%. Current MLLMs struggle significantly on the proposed VIUBench tasks.

Conclusion: VideoSSR establishes a potent foundational framework for developing more advanced video understanding in MLLMs, demonstrating that self-generated verifiable training data from intrinsic video information can effectively advance video understanding capabilities.

Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has substantially advanced the video understanding capabilities of Multimodal Large Language Models (MLLMs). However, the rapid progress of MLLMs is outpacing the complexity of existing video datasets, while the manual annotation of new, high-quality data remains prohibitively expensive. This work investigates a pivotal question: Can the rich, intrinsic information within videos be harnessed to self-generate high-quality, verifiable training data? To investigate this, we introduce three self-supervised pretext tasks: Anomaly Grounding, Object Counting, and Temporal Jigsaw. We construct the Video Intrinsic Understanding Benchmark (VIUBench) to validate their difficulty, revealing that current state-of-the-art MLLMs struggle significantly on these tasks. Building upon these pretext tasks, we develop the VideoSSR-30K dataset and propose VideoSSR, a novel video self-supervised reinforcement learning framework for RLVR. Extensive experiments across 17 benchmarks, spanning four major video domains (General Video QA, Long Video QA, Temporal Grounding, and Complex Reasoning), demonstrate that VideoSSR consistently enhances model performance, yielding an average improvement of over 5%. These results establish VideoSSR as a potent foundational framework for developing more advanced video understanding in MLLMs. The code is available at https://github.com/lcqysl/VideoSSR.

Method: Retrospective study of 512 adnexal mass images from 227 patients, comparing radiologist O-RADS assessment with 16 deep learning models (CNNs and Vision Transformers) and hybrid human-AI frameworks integrating radiologist scores with DL predictions.

Result: Radiologists achieved AUC 0.683 and 68.0% accuracy. CNN models showed AUC 0.620-0.908 and 59.2-86.4% accuracy, while ViT16-384 performed best (AUC 0.941, 87.4% accuracy). Hybrid frameworks significantly improved CNN performance but not ViT models.

Conclusion: Deep learning models significantly outperform radiologist-only O-RADS assessment, and hybrid human-AI approaches provide the highest diagnostic accuracy, offering potential to standardize ultrasound interpretation and improve lesion detection.

Abstract: Background: The 2022 update of the Ovarian-Adnexal Reporting and Data System (O-RADS) ultrasound classification refines risk stratification for adnexal lesions, yet human interpretation remains subject to variability and conservative thresholds. Concurrently, deep learning (DL) models have demonstrated promise in image-based ovarian lesion characterization. This study evaluates radiologist performance applying O-RADS v2022, compares it to leading convolutional neural network (CNN) and Vision Transformer (ViT) models, and investigates the diagnostic gains achieved by hybrid human-AI frameworks. Methods: In this single-center, retrospective cohort study, a total of 512 adnexal mass images from 227 patients (110 with at least one malignant cyst) were included. Sixteen DL models, including DenseNets, EfficientNets, ResNets, VGGs, Xception, and ViTs, were trained and validated. A hybrid model integrating radiologist O-RADS scores with DL-predicted probabilities was also built for each scheme. Results: Radiologist-only O-RADS assessment achieved an AUC of 0.683 and an overall accuracy of 68.0%. CNN models yielded AUCs of 0.620 to 0.908 and accuracies of 59.2% to 86.4%, while ViT16-384 reached the best performance, with an AUC of 0.941 and an accuracy of 87.4%. Hybrid human-AI frameworks further significantly enhanced the performance of CNN models; however, the improvement for ViT models was not statistically significant (P-value >0.05). Conclusions: DL models markedly outperform radiologist-only O-RADS v2022 assessment, and the integration of expert scores with AI yields the highest diagnostic accuracy and discrimination. Hybrid human-AI paradigms hold substantial potential to standardize pelvic ultrasound interpretation, reduce false positives, and improve detection of high-risk lesions.

Method: Proposed TinyChemVL with visual token reduction to process chemical images efficiently, and introduced reaction-level tasks for improved reasoning. Also created ChemRxn-V benchmark for vision-based reaction recognition and prediction.

Result: TinyChemVL achieves superior performance on both molecular and reaction tasks with faster inference and training speeds, outperforming ChemVLM while using only 1/16th of visual tokens.

Conclusion: This work demonstrates that co-designing model architecture and task complexity enables building efficient yet powerful VLMs for chemical domains, advancing chemical reasoning capabilities.

Abstract: While Vision Language Models (VLMs) have demonstrated remarkable capabilities in general visual understanding, their application in the chemical domain has been limited, with previous works predominantly focusing on text and thus overlooking critical visual information, such as molecular structures. Current approaches that directly adopt standard VLMs for chemical tasks suffer from two primary issues: (i) computational inefficiency of processing entire chemical images with non-informative backgrounds. (ii) a narrow scope on molecular-level tasks that restricts progress in chemical reasoning. In this work, we propose \textbf{TinyChemVL}, an efficient and powerful chemical VLM that leverages visual token reduction and reaction-level tasks to improve model efficiency and reasoning capacity. Also, we propose \textbf{ChemRxn-V}, a reaction-level benchmark for assessing vision-based reaction recognition and prediction tasks. Directly predicting reaction products from molecular images poses a non-trivial challenge, as it requires models to integrate both recognition and reasoning capacities. Our results demonstrate that with only 4B parameters, TinyChemVL achieves superior performance on both molecular and reaction tasks while demonstrating faster inference and training speeds compared to existing models. Notably, TinyChemVL outperforms ChemVLM while utilizing only 1/16th of the visual tokens. This work builds efficient yet powerful VLMs for chemical domains by co-designing model architecture and task complexity.

Method: Used YOLOv8 and YOLOv11 architectures with Optuna hyperparameter optimization, spatial post-processing, and innovative pallet hole mapping module for spatial representation.

Result: YOLOv8 achieved high detection accuracy, while optimized YOLOv11 offered superior precision and stable convergence on custom warehouse dataset.

Conclusion: Feasible cost-effective visual perception module for forklifts that advances warehouse automation for safer, economical, and intelligent logistics.

Abstract: The automation of material handling in warehouses increasingly relies on robust, low cost perception systems for forklifts and Automated Guided Vehicles (AGVs). This work presents a vision based framework for pallet and pallet hole detection and mapping using a single standard camera. We utilized YOLOv8 and YOLOv11 architectures, enhanced through Optuna driven hyperparameter optimization and spatial post processing. An innovative pallet hole mapping module converts the detections into actionable spatial representations, enabling accurate pallet and pallet hole association for forklift operation. Experiments on a custom dataset augmented with real warehouse imagery show that YOLOv8 achieves high pallet and pallet hole detection accuracy, while YOLOv11, particularly under optimized configurations, offers superior precision and stable convergence. The results demonstrate the feasibility of a cost effective, retrofittable visual perception module for forklifts. This study proposes a scalable approach to advancing warehouse automation, promoting safer, economical, and intelligent logistics operations.

Method: SFFR method with Frequency Component Exchange KAN (FCEKAN) module for selective frequency component exchange between RGB and IR images, and Multi-Scale Gaussian KAN (MSGKAN) module for nonlinear spatial feature modeling using multi-scale Gaussian basis functions.

Result: Extensive experiments on SeaDroneSee, DroneVehicle and DVTOD datasets demonstrate superior performance in UAV multispectral object perception tasks.

Conclusion: The proposed FCEKAN and MSGKAN modules are complementary and effectively capture frequency and spatial semantic features respectively for better feature fusion in multispectral object detection.

Abstract: Recent multispectral object detection methods have primarily focused on spatial-domain feature fusion based on CNNs or Transformers, while the potential of frequency-domain feature remains underexplored. In this work, we propose a novel Spatial and Frequency Feature Reconstruction method (SFFR) method, which leverages the spatial-frequency feature representation mechanisms of the Kolmogorov-Arnold Network (KAN) to reconstruct complementary representations in both spatial and frequency domains prior to feature fusion. The core components of SFFR are the proposed Frequency Component Exchange KAN (FCEKAN) module and Multi-Scale Gaussian KAN (MSGKAN) module. The FCEKAN introduces an innovative selective frequency component exchange strategy that effectively enhances the complementarity and consistency of cross-modal features based on the frequency feature of RGB and IR images. The MSGKAN module demonstrates excellent nonlinear feature modeling capability in the spatial domain. By leveraging multi-scale Gaussian basis functions, it effectively captures the feature variations caused by scale changes at different UAV flight altitudes, significantly enhancing the model’s adaptability and robustness to scale variations. It is experimentally validated that our proposed FCEKAN and MSGKAN modules are complementary and can effectively capture the frequency and spatial semantic features respectively for better feature fusion. Extensive experiments on the SeaDroneSee, DroneVehicle and DVTOD datasets demonstrate the superior performance and significant advantages of the proposed method in UAV multispectral object perception task. Code will be available at https://github.com/qchenyu1027/SFFR.

Method: Uses static-dynamic decoupled 4D hash encoding, imposes Cauchy momentum residual as physics constraint, predicts particle velocity and stress via time-evolving material field, and supervises with optical flow matching.

Result: Significant improvements in physical consistency and monocular dynamic reconstruction quality on custom physics-driven and standard datasets.

Conclusion: Physics-informed constraints enhance 3DGS for dynamic scenes, enabling better capture of physics-driven motion patterns and improved reconstruction quality.

Abstract: Recently, 3D Gaussian Splatting (3DGS), an explicit scene representation technique, has shown significant promise for dynamic novel-view synthesis from monocular video input. However, purely data-driven 3DGS often struggles to capture the diverse physics-driven motion patterns in dynamic scenes. To fill this gap, we propose Physics-Informed Deformable Gaussian Splatting (PIDG), which treats each Gaussian particle as a Lagrangian material point with time-varying constitutive parameters and is supervised by 2D optical flow via motion projection. Specifically, we adopt static-dynamic decoupled 4D decomposed hash encoding to reconstruct geometry and motion efficiently. Subsequently, we impose the Cauchy momentum residual as a physics constraint, enabling independent prediction of each particle’s velocity and constitutive stress via a time-evolving material field. Finally, we further supervise data fitting by matching Lagrangian particle flow to camera-compensated optical flow, which accelerates convergence and improves generalization. Experiments on a custom physics-driven dataset as well as on standard synthetic and real-world datasets demonstrate significant gains in physical consistency and monocular dynamic reconstruction quality.

Method: The method integrates Forward Curvature-Matching (FCM) update with diffusion sampling, dynamically determining optimal step sizes using forward automatic differentiation and finite-difference curvature estimates for precise likelihood optimization.

Result: Experiments on ShapeNet and CO3D datasets show superior reconstruction quality at matched or lower NFEs, achieving higher F-score and lower CD and EMD metrics compared to existing methods.

Conclusion: FCM enables high-fidelity reconstruction from single-view and multi-view inputs, supports various input modalities through simple operator substitution without retraining, validating its efficiency and adaptability for practical applications.

Abstract: Reconstructing high-quality point clouds from images remains challenging in computer vision. Existing generative-model-based approaches, particularly diffusion-model approaches that directly learn the posterior, may suffer from inflexibility – they require conditioning signals during training, support only a fixed number of input views, and need complete retraining for different measurements. Recent diffusion-based methods have attempted to address this by combining prior models with likelihood updates, but they rely on heuristic fixed step sizes for the likelihood update that lead to slow convergence and suboptimal reconstruction quality. We advance this line of approach by integrating our novel Forward Curvature-Matching (FCM) update method with diffusion sampling. Our method dynamically determines optimal step sizes using only forward automatic differentiation and finite-difference curvature estimates, enabling precise optimization of the likelihood update. This formulation enables high-fidelity reconstruction from both single-view and multi-view inputs, and supports various input modalities through simple operator substitution – all without retraining. Experiments on ShapeNet and CO3D datasets demonstrate that our method achieves superior reconstruction quality at matched or lower NFEs, yielding higher F-score and lower CD and EMD, validating its efficiency and adaptability for practical applications. Code is available at https://github.com/Seunghyeok0715/FCM

Method: Convert puzzle pieces into discrete token sequences using a specialized tokenizer, then use encoder-decoder transformers to solve the puzzle as a sequence-to-sequence prediction task without visual input.

Result: Models achieved state-of-the-art results across multiple benchmarks, often outperforming vision-based methods despite being restricted from accessing visual input.

Conclusion: Language models have surprising capability to solve problems beyond their native domain, and unconventional approaches can inspire promising directions for puzzle-solving research.

Abstract: Jigsaw puzzles are primarily visual objects, whose algorithmic solutions have traditionally been framed from a visual perspective. In this work, however, we explore a fundamentally different approach: solving square jigsaw puzzles using language models, without access to raw visual input. By introducing a specialized tokenizer that converts each puzzle piece into a discrete sequence of tokens, we reframe puzzle reassembly as a sequence-to-sequence prediction task. Treated as “blind” solvers, encoder-decoder transformers accurately reconstruct the original layout by reasoning over token sequences alone. Despite being deliberately restricted from accessing visual input, our models achieve state-of-the-art results across multiple benchmarks, often outperforming vision-based methods. These findings highlight the surprising capability of language models to solve problems beyond their native domain, and suggest that unconventional approaches can inspire promising directions for puzzle-solving research.

Method: Uses Masked AutoEncoder trained to reconstruct masked patches, computes conditional Negative Log-Likelihood to quantify local semantic anomalies, and aggregates patch-level statistics with global MAE features through learned fusion.

Result: Achieves over 95% accuracy on all eight unseen generators in GenImage benchmark when trained only on Stable Diffusion v1.4, substantially outperforming state-of-the-art detectors.

Conclusion: Context-conditional reconstruction uncertainty provides a robust, transferable signal for AIGC detection, enabling strong cross-generator generalization.

Abstract: While context-based detectors have achieved strong generalization for AI-generated text by measuring distributional inconsistencies, image-based detectors still struggle with overfitting to generator-specific artifacts. We introduce CINEMAE, a novel paradigm for AIGC image detection that adapts the core principles of text detection methods to the visual domain. Our key insight is that Masked AutoEncoder (MAE), trained to reconstruct masked patches conditioned on visible context, naturally encodes semantic consistency expectations. We formalize this reconstruction process probabilistically, computing conditional Negative Log-Likelihood (NLL, p(masked | visible)) to quantify local semantic anomalies. By aggregating these patch-level statistics with global MAE features through learned fusion, CINEMAE achieves strong cross-generator generalization. Trained exclusively on Stable Diffusion v1.4, our method achieves over 95% accuracy on all eight unseen generators in the GenImage benchmark, substantially outperforming state-of-the-art detectors. This demonstrates that context-conditional reconstruction uncertainty provides a robust, transferable signal for AIGC detection.

Method: Proposes MODS framework with: 1) Graph-based Dynamic Sequence Compressor (GDC) using capsule networks and graph convolution to reduce acoustic/visual redundancy, 2) sample-adaptive Primary Modality Selector (MSelector) for dynamic dominance determination, and 3) Primary-modality-Centric Cross-Attention (PCCA) module to enhance dominant modalities and cross-modal interaction.

Result: Extensive experiments on four benchmark datasets show MODS outperforms state-of-the-art methods, achieving superior performance by effectively balancing modality contributions and eliminating redundant noise.

Conclusion: The proposed MODS framework successfully addresses modality imbalance and redundancy issues in MSA through dynamic primary modality selection and optimization, demonstrating significant performance improvements over existing approaches.

Abstract: Multimodal Sentiment Analysis (MSA) aims to predict sentiment from language, acoustic, and visual data in videos. However, imbalanced unimodal performance often leads to suboptimal fused representations. Existing approaches typically adopt fixed primary modality strategies to maximize dominant modality advantages, yet fail to adapt to dynamic variations in modality importance across different samples. Moreover, non-language modalities suffer from sequential redundancy and noise, degrading model performance when they serve as primary inputs. To address these issues, this paper proposes a modality optimization and dynamic primary modality selection framework (MODS). First, a Graph-based Dynamic Sequence Compressor (GDC) is constructed, which employs capsule networks and graph convolution to reduce sequential redundancy in acoustic/visual modalities. Then, we develop a sample-adaptive Primary Modality Selector (MSelector) for dynamic dominance determination. Finally, a Primary-modality-Centric Cross-Attention (PCCA) module is designed to enhance dominant modalities while facilitating cross-modal interaction. Extensive experiments on four benchmark datasets demonstrate that MODS outperforms state-of-the-art methods, achieving superior performance by effectively balancing modality contributions and eliminating redundant noise.

Method: Combined self-supervised learning with domain adaptation for instance segmentation, used self-supervised learning for semantic segmentation, and implemented hierarchical transfer learning for tree species classification.

Result: Significant performance improvements: instance segmentation AP50 +16.98%, semantic segmentation mIoU +1.79%, and species classification Jaccard +6.07% for unseen species, with ~21% reduction in energy consumption.

Conclusion: The proposed unified framework effectively reduces annotation dependency while improving performance across multiple tree analysis tasks, providing an open-source solution for operational extraction of individual tree information from laser scanning data.

Abstract: Detailed structural and species information on individual tree level is increasingly important to support precision forestry, biodiversity conservation, and provide reference data for biomass and carbon mapping. Point clouds from airborne and ground-based laser scanning are currently the most suitable data source to rapidly derive such information at scale. Recent advancements in deep learning improved segmenting and classifying individual trees and identifying semantic tree components. However, deep learning models typically require large amounts of annotated training data which limits further improvement. Producing dense, high-quality annotations for 3D point clouds, especially in complex forests, is labor-intensive and challenging to scale. We explore strategies to reduce dependence on large annotated datasets using self-supervised and transfer learning architectures. Our objective is to improve performance across three tasks: instance segmentation, semantic segmentation, and tree classification using realistic and operational training sets. Our findings indicate that combining self-supervised learning with domain adaptation significantly enhances instance segmentation compared to training from scratch (AP50 +16.98%), self-supervised learning suffices for semantic segmentation (mIoU +1.79%), and hierarchical transfer learning enables accurate classification of unseen species (Jaccard +6.07%). To simplify use and encourage uptake, we integrated the tasks into a unified framework, streamlining the process from raw point clouds to tree delineation, structural analysis, and species classification. Pretrained models reduce energy consumption and carbon emissions by ~21%. This open-source contribution aims to accelerate operational extraction of individual tree information from laser scanning point clouds to support forestry, biodiversity, and carbon mapping.

Method: Collected 5 million LOD2 building models from diverse global regions (North America, Europe, Asia, Africa, Oceania) with real and simulated LiDAR point clouds, plus Cyber City for generating unlimited training data.

Result: Created a globally representative dataset enabling comprehensive research on 3D building reconstruction, detection, and segmentation with standardized evaluation metrics.

Conclusion: BuildingWorld bridges the diversity gap in 3D urban modeling, supporting foundation model development and structured urban environment analysis.

Abstract: As digital twins become central to the transformation of modern cities, accurate and structured 3D building models emerge as a key enabler of high-fidelity, updatable urban representations. These models underpin diverse applications including energy modeling, urban planning, autonomous navigation, and real-time reasoning. Despite recent advances in 3D urban modeling, most learning-based models are trained on building datasets with limited architectural diversity, which significantly undermines their generalizability across heterogeneous urban environments. To address this limitation, we present BuildingWorld, a comprehensive and structured 3D building dataset designed to bridge the gap in stylistic diversity. It encompasses buildings from geographically and architecturally diverse regions – including North America, Europe, Asia, Africa, and Oceania – offering a globally representative dataset for urban-scale foundation modeling and analysis. Specifically, BuildingWorld provides about five million LOD2 building models collected from diverse sources, accompanied by real and simulated airborne LiDAR point clouds. This enables comprehensive research on 3D building reconstruction, detection and segmentation. Cyber City, a virtual city model, is introduced to enable the generation of unlimited training data with customized and structurally diverse point cloud distributions. Furthermore, we provide standardized evaluation metrics tailored for building reconstruction, aiming to facilitate the training, evaluation, and comparison of large-scale vision models and foundation models in structured 3D urban environments.

Method: GazeVLM integrates visual (RGB and depth) and textual modalities, using a fusion of RGB images with HHA-encoded depth maps guided by text prompts. It allows selective execution of gaze understanding tasks including person detection, gaze target detection, and gaze object identification.

Result: The model achieves state-of-the-art evaluation scores on GazeFollow and VideoAttentionTarget datasets. The ablation study showed that RGB+HHA depth map fusion with text prompts yields superior performance. A new object-level gaze detection metric ($AP_{ob}$) was introduced for gaze object identification.

Conclusion: GazeVLM represents the first application of a Vision-Language Model to unified gaze understanding tasks, demonstrating significant improvements over existing methods and establishing a new benchmark for multi-task gaze analysis.

Abstract: Gaze understanding unifies the detection of people, their gaze targets, and objects of interest into a single framework, offering critical insight into visual attention and intent estimation. Although prior research has modelled gaze cues in visual scenes, a unified system is still needed for gaze understanding using both visual and language prompts. This paper introduces GazeVLM, a novel Vision-Language Model (VLM) for multi-task gaze understanding in images, addressing person detection, gaze target detection, and gaze object identification. While other transformer-based methods exist for gaze analysis, GazeVLM represents, to our knowledge, the first application of a VLM to these combined tasks, allowing for selective execution of each task. Through the integration of visual (RGB and depth) and textual modalities, our ablation study on visual input combinations revealed that a fusion of RGB images with HHA-encoded depth maps, guided by text prompts, yields superior performance. We also introduce an object-level gaze detection metric for gaze object identification ($AP_{ob}$). Through experiments, GazeVLM demonstrates significant improvements, notably achieving state-of-the-art evaluation scores on GazeFollow and VideoAttentionTarget datasets.

Method: Proposes HiMo-CLIP with two components: hierarchical decomposition (HiDe) module using in-batch PCA to extract semantic components, and monotonicity-aware contrastive loss (MoLo) that aligns global and component-level representations.

Result: Experiments show HiMo-CLIP consistently outperforms strong baselines on multiple image-text retrieval benchmarks, especially with long or compositional descriptions.

Conclusion: HiMo-CLIP successfully enhances CLIP-style models by incorporating semantic hierarchy and monotonicity, producing more structured and cognitively-aligned cross-modal representations without modifying encoder architecture.

Abstract: Contrastive vision-language models like CLIP have achieved impressive results in image-text retrieval by aligning image and text representations in a shared embedding space. However, these models often treat text as flat sequences, limiting their ability to handle complex, compositional, and long-form descriptions. In particular, they fail to capture two essential properties of language: semantic hierarchy, which reflects the multi-level compositional structure of text, and semantic monotonicity, where richer descriptions should result in stronger alignment with visual content.To address these limitations, we propose HiMo-CLIP, a representation-level framework that enhances CLIP-style models without modifying the encoder architecture. HiMo-CLIP introduces two key components: a hierarchical decomposition (HiDe) module that extracts latent semantic components from long-form text via in-batch PCA, enabling flexible, batch-aware alignment across different semantic granularities, and a monotonicity-aware contrastive loss (MoLo) that jointly aligns global and component-level representations, encouraging the model to internalize semantic ordering and alignment strength as a function of textual completeness.These components work in concert to produce structured, cognitively-aligned cross-modal representations. Experiments on multiple image-text retrieval benchmarks show that HiMo-CLIP consistently outperforms strong baselines, particularly under long or compositional descriptions. The code is available at https://github.com/UnicomAI/HiMo-CLIP.

Method: Created AesTest benchmark with curated multiple-choice questions spanning 10 tasks covering perception, appreciation, creation, and photography, grounded in psychological theories and integrating diverse data sources including professional workflows and crowdsourced preferences.

Result: Evaluation of both instruction-tuned IAA MLLMs and general MLLMs on AesTest revealed significant challenges in building aesthetic intelligence.

Conclusion: AesTest will be publicly released to support future research in multimodal aesthetic intelligence, highlighting the need for improved aesthetic capabilities in MLLMs.

Abstract: Perceiving and producing aesthetic judgments is a fundamental yet underexplored capability for multimodal large language models (MLLMs). However, existing benchmarks for image aesthetic assessment (IAA) are narrow in perception scope or lack the diversity needed to evaluate systematic aesthetic production. To address this gap, we introduce AesTest, a comprehensive benchmark for multimodal aesthetic perception and production, distinguished by the following features: 1) It consists of curated multiple-choice questions spanning ten tasks, covering perception, appreciation, creation, and photography. These tasks are grounded in psychological theories of generative learning. 2) It integrates data from diverse sources, including professional editing workflows, photographic composition tutorials, and crowdsourced preferences. It ensures coverage of both expert-level principles and real-world variation. 3) It supports various aesthetic query types, such as attribute-based analysis, emotional resonance, compositional choice, and stylistic reasoning. We evaluate both instruction-tuned IAA MLLMs and general MLLMs on AesTest, revealing significant challenges in building aesthetic intelligence. We will publicly release AesTest to support future research in this area.

Method: V-Shuffle shuffles value features within self-attention layers of diffusion models to disrupt semantic content while preserving low-level style representations, complemented by Hybrid Style Regularization for high-level style textures.

Result: V-Shuffle achieves excellent performance with multiple style images and outperforms previous state-of-the-art methods when applied to a single style image.

Conclusion: V-Shuffle effectively navigates the trade-off between content preservation and style fidelity in zero-shot style transfer by leveraging multiple style images and innovative feature manipulation techniques.

Abstract: Attention injection-based style transfer has achieved remarkable progress in recent years. However, existing methods often suffer from content leakage, where the undesired semantic content of the style image mistakenly appears in the stylized output. In this paper, we propose V-Shuffle, a zero-shot style transfer method that leverages multiple style images from the same style domain to effectively navigate the trade-off between content preservation and style fidelity. V-Shuffle implicitly disrupts the semantic content of the style images by shuffling the value features within the self-attention layers of the diffusion model, thereby preserving low-level style representations. We further introduce a Hybrid Style Regularization that complements these low-level representations with high-level style textures to enhance style fidelity. Empirical results demonstrate that V-Shuffle achieves excellent performance when utilizing multiple style images. Moreover, when applied to a single style image, V-Shuffle outperforms previous state-of-the-art methods.

Method: Two difficulty-aware sampling strategies: Progressive Image Semantic Masking (PISM) for image degradation-based hardness assessment, and Cross-Modality Attention Balance (CMAB) for attention distribution analysis. Hierarchical training framework with GRPO-only and SFT+GRPO hybrid paradigms.

Result: Experiments across six benchmark datasets show GRPO applied to difficulty-stratified samples consistently outperforms conventional SFT+GRPO pipelines, improving model accuracy without supervised fine-tuning.

Conclusion: Strategic data sampling with difficulty-aware metrics can eliminate the need for supervised fine-tuning while enhancing multimodal reasoning performance.

Abstract: Recent advances in Multimodal Large Language Models (MLLMs) have spurred significant progress in Chain-of-Thought (CoT) reasoning. Building on the success of Deepseek-R1, researchers extended multimodal reasoning to post-training paradigms based on reinforcement learning (RL), focusing predominantly on mathematical datasets. However, existing post-training paradigms tend to neglect two critical aspects: (1) The lack of quantifiable difficulty metrics capable of strategically screening samples for post-training optimization. (2) Suboptimal post-training paradigms that fail to jointly optimize perception and reasoning capabilities. To address this gap, we propose two novel difficulty-aware sampling strategies: Progressive Image Semantic Masking (PISM) quantifies sample hardness through systematic image degradation, while Cross-Modality Attention Balance (CMAB) assesses cross-modal interaction complexity via attention distribution analysis. Leveraging these metrics, we design a hierarchical training framework that incorporates both GRPO-only and SFT+GRPO hybrid training paradigms, and evaluate them across six benchmark datasets. Experiments demonstrate consistent superiority of GRPO applied to difficulty-stratified samples compared to conventional SFT+GRPO pipelines, indicating that strategic data sampling can obviate the need for supervised fine-tuning while improving model accuracy. Our code will be released at https://github.com/qijianyu277/DifficultySampling.

Method: Collected data from 6 domains, preprocessed with quality control, constructed affect table for annotation, used 5 MLLMs with RRF fusion for robust affect analysis.

Result: Achieved high accuracy with Composite Affect Consistency Index of 0.986, indicating reliable affect annotations in the dataset.

Conclusion: InfoAffect dataset successfully addresses the data scarcity problem and provides a reliable resource for studying affective dimensions of infographics.

Abstract: Infographics are widely used to convey complex information, yet their affective dimensions remain underexplored due to the scarcity of data resources. We introduce a 3.5k-sample affect-annotated InfoAffect dataset, which combines textual content with real-world infographics. We first collect the raw data from six domains and aligned them via preprocessing, the accompanied-text-priority method, and three strategies to guarantee the quality and compliance. After that we construct an affect table and use it to constrain annotation. Five state-of-the-art multimodal large language models (MLLMs) then analyze both modalities, and their outputs are fused with Reciprocal Rank Fusion (RRF) algorithm to yield robust affects and confidences. We conducted a user study with two experiments to validate usability and assess InfoAffect dataset using the Composite Affect Consistency Index (CACI), achieving an overall score of 0.986, which indicates high accuracy.

Method: Proposes Scarf Neck module with modality-agnostic deformable attention, uses pseudo modality dropout strategy during training, and introduces comprehensive benchmark for modality-incomplete scenarios.

Result: Scarf-DETR performs excellently in missing modality scenarios and achieves superior performance on standard IVOD modality complete benchmarks.

Conclusion: The proposed approach enables flexible adaptation to any single or double modalities during training and inference, making IVOD detectors more robust and practical for real-world applications.

Abstract: Infrared and visible object detection (IVOD) is essential for numerous around-the-clock applications. Despite notable advancements, current IVOD models exhibit notable performance declines when confronted with incomplete modality data, particularly if the dominant modality is missing. In this paper, we take a thorough investigation on modality incomplete IVOD problem from an architecture compatibility perspective. Specifically, we propose a plug-and-play Scarf Neck module for DETR variants, which introduces a modality-agnostic deformable attention mechanism to enable the IVOD detector to flexibly adapt to any single or double modalities during training and inference. When training Scarf-DETR, we design a pseudo modality dropout strategy to fully utilize the multi-modality information, making the detector compatible and robust to both working modes of single and double modalities. Moreover, we introduce a comprehensive benchmark for the modality-incomplete IVOD task aimed at thoroughly assessing situations where the absent modality is either dominant or secondary. Our proposed Scarf-DETR not only performs excellently in missing modality scenarios but also achieves superior performances on the standard IVOD modality complete benchmarks. Our code will be available at https://github.com/YinghuiXing/Scarf-DETR.

Method: Uses two separate NeRF models: static NeRF optimizes camera poses and background, while dynamic NeRF incorporates 3D scene flow for dynamic objects. Includes training framework to address camera-object motion ambiguity.

Result: Outperforms state-of-the-art NeRF-based pose-free methods in camera pose estimation and dynamic novel view synthesis on urban driving datasets.

Conclusion: VDNeRF enables robust camera pose estimation and self-supervised decomposition of static/dynamic elements without external pose information.

Abstract: Neural Radiance Fields (NeRFs) implicitly model continuous three-dimensional scenes using a set of images with known camera poses, enabling the rendering of photorealistic novel views. However, existing NeRF-based methods encounter challenges in applications such as autonomous driving and robotic perception, primarily due to the difficulty of capturing accurate camera poses and limitations in handling large-scale dynamic environments. To address these issues, we propose Vision-only Dynamic NeRF (VDNeRF), a method that accurately recovers camera trajectories and learns spatiotemporal representations for dynamic urban scenes without requiring additional camera pose information or expensive sensor data. VDNeRF employs two separate NeRF models to jointly reconstruct the scene. The static NeRF model optimizes camera poses and static background, while the dynamic NeRF model incorporates the 3D scene flow to ensure accurate and consistent reconstruction of dynamic objects. To address the ambiguity between camera motion and independent object motion, we design an effective and powerful training framework to achieve robust camera pose estimation and self-supervised decomposition of static and dynamic elements in a scene. Extensive evaluations on mainstream urban driving datasets demonstrate that VDNeRF surpasses state-of-the-art NeRF-based pose-free methods in both camera pose estimation and dynamic novel view synthesis.

Method: Uses training-free geometric rendering to create pseudo-satellite images as structural prompts, then employs a text-free conditional diffusion model with VAE to fuse multimodal structural cues for robust feature learning.

Result: Achieves competitive cross-view localization performance on University-1652 and SUES-200 datasets, particularly excelling in satellite-to-drone localization on University-1652.

Conclusion: The proposed framework provides an effective solution for UAV localization in GNSS-denied environments by leveraging diffusion models and unified representations without text dependencies.

Abstract: With the rapid growth of the low-altitude economy, unmanned aerial vehicles (UAVs) have become key platforms for measurement and tracking in intelligent patrol systems. However, in GNSS-denied environments, localization schemes that rely solely on satellite signals are prone to failure. Cross-view image retrieval-based localization is a promising alternative, yet substantial geometric and appearance domain gaps exist between oblique UAV views and nadir satellite orthophotos. Moreover, conventional approaches often depend on complex network architectures, text prompts, or large amounts of annotation, which hinders generalization. To address these issues, we propose DiffusionUavLoc, a cross-view localization framework that is image-prompted, text-free, diffusion-centric, and employs a VAE for unified representation. We first use training-free geometric rendering to synthesize pseudo-satellite images from UAV imagery as structural prompts. We then design a text-free conditional diffusion model that fuses multimodal structural cues to learn features robust to viewpoint changes. At inference, descriptors are computed at a fixed time step t and compared using cosine similarity. On University-1652 and SUES-200, the method performs competitively for cross-view localization, especially for satellite-to-drone in University-1652.Our data and code will be published at the following URL: https://github.com/liutao23/DiffusionUavLoc.git.

Method: Uses RL with multi-objective dense spatial rewards to integrate structured spatial grounding with multi-step reasoning, simulating human-like spatial perception via scene graphs.

Result: SpatialThinker-7B outperforms supervised fine-tuning and sparse RL baselines, nearly doubling base-model gains compared to sparse RL and surpassing GPT-4o on spatial understanding benchmarks.

Conclusion: Combining spatial supervision with reward-aligned reasoning enables robust 3D spatial understanding with limited data, advancing MLLMs toward human-level visual reasoning.

Abstract: Multimodal large language models (MLLMs) have achieved remarkable progress in vision-language tasks, but they continue to struggle with spatial understanding. Existing spatial MLLMs often rely on explicit 3D inputs or architecture-specific modifications, and remain constrained by large-scale datasets or sparse supervision. To address these limitations, we introduce SpatialThinker, a 3D-aware MLLM trained with RL to integrate structured spatial grounding with multi-step reasoning. The model simulates human-like spatial perception by constructing a scene graph of task-relevant objects and spatial relations, and reasoning towards an answer via dense spatial rewards. SpatialThinker consists of two key contributions: (1) a data synthesis pipeline that generates STVQA-7K, a high-quality spatial VQA dataset, and (2) online RL with a multi-objective dense spatial reward enforcing spatial grounding. SpatialThinker-7B outperforms supervised fine-tuning and the sparse RL baseline on spatial understanding and real-world VQA benchmarks, nearly doubling the base-model gain compared to sparse RL, and surpassing GPT-4o. These results showcase the effectiveness of combining spatial supervision with reward-aligned reasoning in enabling robust 3D spatial understanding with limited data and advancing MLLMs towards human-level visual reasoning.

Method: Proposes UFC-MIL with a novel patch-wise loss that learns latent patterns and expresses uncertainty for classification. Uses attention-based architecture with neighbor patch aggregation module, and calibrates aggregated predictions through patch-level uncertainty without requiring multiple iterative inferences.

Result: UFC-MIL shows superior performance in model calibration while achieving classification accuracy comparable to state-of-the-art methods on challenging public datasets.

Conclusion: UFC-MIL successfully provides calibrated diagnostic predictions that mimic pathologists’ examination behaviors, offering trustworthy AI diagnostic aid with practical advantages for clinical applications.

Abstract: With the increasing demand for histopathological specimen examination and diagnostic reporting, Multiple Instance Learning (MIL) has received heightened research focus as a viable solution for AI-centric diagnostic aid. Recently, to improve its performance and make it work more like a pathologist, several MIL approaches based on the use of multiple-resolution images have been proposed, delivering often higher performance than those that use single-resolution images. Despite impressive recent developments of multiple-resolution MIL, previous approaches only focus on improving performance, thereby lacking research on well-calibrated MIL that clinical experts can rely on for trustworthy diagnostic results. In this study, we propose Uncertainty-Focused Calibrated MIL (UFC-MIL), which more closely mimics the pathologists’ examination behaviors while providing calibrated diagnostic predictions, using multiple images with different resolutions. UFC-MIL includes a novel patch-wise loss that learns the latent patterns of instances and expresses their uncertainty for classification. Also, the attention-based architecture with a neighbor patch aggregation module collects features for the classifier. In addition, aggregated predictions are calibrated through patch-level uncertainty without requiring multiple iterative inferences, which is a key practical advantage. Against challenging public datasets, UFC-MIL shows superior performance in model calibration while achieving classification accuracy comparable to that of state-of-the-art methods.

Method: Uses effective rank to quantify both collapses; proposes Rank-enhancing Token Fuser that selectively blends less informative features from one modality with complementary features from another; evaluates modality combinations that mutually increase effective rank.

Result: Depth maintains representational balance when fused with RGB, avoiding modality collapse; R3D framework significantly outperforms prior state-of-the-art methods by up to 3.74% on NTURGBD, UTKinect, and DARai datasets.

Conclusion: Effective rank serves as a unifying framework to address both feature and modality collapse simultaneously; depth-RGB fusion provides balanced representation; the approach achieves state-of-the-art performance in action anticipation tasks.

Abstract: Multi-modal fusion methods often suffer from two types of representation collapse: feature collapse where individual dimensions lose their discriminative power (as measured by eigenspectra), and modality collapse where one dominant modality overwhelms the other. Applications like human action anticipation that require fusing multifarious sensor data are hindered by both feature and modality collapse. However, existing methods attempt to counter feature collapse and modality collapse separately. This is because there is no unifying framework that efficiently addresses feature and modality collapse in conjunction. In this paper, we posit the utility of effective rank as an informative measure that can be utilized to quantify and counter both the representation collapses. We propose \textit{Rank-enhancing Token Fuser}, a theoretically grounded fusion framework that selectively blends less informative features from one modality with complementary features from another modality. We show that our method increases the effective rank of the fused representation. To address modality collapse, we evaluate modality combinations that mutually increase each others’ effective rank. We show that depth maintains representational balance when fused with RGB, avoiding modality collapse. We validate our method on action anticipation, where we present \texttt{R3D}, a depth-informed fusion framework. Extensive experiments on NTURGBD, UTKinect, and DARai demonstrate that our approach significantly outperforms prior state-of-the-art methods by up to 3.74%. Our code is available at: \href{https://github.com/olivesgatech/R3D}{https://github.com/olivesgatech/R3D}.

Method: Created EIDSeg dataset with 3,266 images from 9 major earthquakes (2008-2023), annotated across 5 damage classes using a three-phase cross-disciplinary annotation protocol with non-expert annotators achieving over 70% inter-annotator agreement.

Result: Benchmarked state-of-the-art segmentation models, with Encoder-only Mask Transformer (EoMT) achieving the best performance with 80.8% mIoU.

Conclusion: The work enables faster, finer-grained damage assessment by leveraging social networks’ ground-level perspective for post-earthquake scenarios.

Abstract: Rapid post-earthquake damage assessment is crucial for rescue and resource planning. Still, existing remote sensing methods depend on costly aerial images, expert labeling, and produce only binary damage maps for early-stage evaluation. Although ground-level images from social networks provide a valuable source to fill this gap, a large pixel-level annotated dataset for this task is still unavailable. We introduce EIDSeg, the first large-scale semantic segmentation dataset specifically for post-earthquake social media imagery. The dataset comprises 3,266 images from nine major earthquakes (2008-2023), annotated across five classes of infrastructure damage: Undamaged Building, Damaged Building, Destroyed Building, Undamaged Road, and Damaged Road. We propose a practical three-phase cross-disciplinary annotation protocol with labeling guidelines that enables consistent segmentation by non-expert annotators, achieving over 70% inter-annotator agreement. We benchmark several state-of-the-art segmentation models, identifying Encoder-only Mask Transformer (EoMT) as the top-performing method with a Mean Intersection over Union (mIoU) of 80.8%. By unlocking social networks’ rich ground-level perspective, our work paves the way for a faster, finer-grained damage assessment in the post-earthquake scenario.

Method: Proposes Inpaint360GS framework using 3D Gaussian Splatting, distills 2D segmentation into 3D, leverages virtual camera views for contextual guidance, and introduces a new dataset for 360° inpainting.

Result: Outperforms existing baselines and achieves state-of-the-art performance in 360° scene inpainting.

Conclusion: Inpaint360GS provides an effective solution for flexible 360° editing with multi-object removal and high-fidelity inpainting in 3D space.

Abstract: Despite recent advances in single-object front-facing inpainting using NeRF and 3D Gaussian Splatting (3DGS), inpainting in complex 360° scenes remains largely underexplored. This is primarily due to three key challenges: (i) identifying target objects in the 3D field of 360° environments, (ii) dealing with severe occlusions in multi-object scenes, which makes it hard to define regions to inpaint, and (iii) maintaining consistent and high-quality appearance across views effectively. To tackle these challenges, we propose Inpaint360GS, a flexible 360° editing framework based on 3DGS that supports multi-object removal and high-fidelity inpainting in 3D space. By distilling 2D segmentation into 3D and leveraging virtual camera views for contextual guidance, our method enables accurate object-level editing and consistent scene completion. We further introduce a new dataset tailored for 360° inpainting, addressing the lack of ground truth object-free scenes. Experiments demonstrate that Inpaint360GS outperforms existing baselines and achieves state-of-the-art performance. Project page: https://dfki-av.github.io/inpaint360gs/

Method: Created NOAH benchmark with composite videos by inserting clips from other sources, varying semantic similarity and insertion position. Includes captioning task with tailored metrics and three QA tasks (Existence, Temporal, Narrative) with over 60K evaluation samples.

Result: Most Video LLMs exhibit narrative prior-induced hallucinations and omissions; error patterns vary by architecture and depend on event similarity/position; reliance on narrative priors intensifies with fewer frames, amplifying errors when event continuity is weak.

Conclusion: NOAH provides the first standardized evaluation of narrative prior-induced hallucination and omission in Video LLMs, establishing a foundation for developing more reliable and trustworthy video understanding models.

Abstract: Video large language models (Video LLMs) have recently achieved strong performance on tasks such as captioning, summarization, and question answering. Many models and training methods explicitly encourage continuity across events to enhance narrative coherence. While this improves fluency, it also introduces an inductive bias that prioritizes storyline consistency over strict grounding in visual evidence. We identify this bias, which we call narrative prior, as a key driver of two errors: hallucinations, where non-existent events are introduced or existing ones are misinterpreted, and omissions, where factual events are suppressed because they are misaligned with surrounding context. To systematically evaluate narrative prior-induced errors, we introduce NOAH, a large-scale benchmark that constructs composite videos by inserting clips from other sources into target videos. By varying semantic similarity and insertion position, our benchmark enables controlled and scalable analysis of narrative priors. We design one captioning task with tailored metrics and three QA tasks - Existence, Temporal, and Narrative - yielding more than 60K evaluation samples. Extensive experiments yield three key findings: (i) most Video LLMs exhibit hallucinations and omissions driven by narrative priors, (ii) the patterns of these errors vary across architectures and depend on event similarity and insertion position, and (iii) reliance on narrative priors intensifies under sampling with fewer frames, amplifying errors when event continuity is weak. We establish NOAH as the first standardized evaluation of narrative prior-induced hallucination and omission in Video LLMs, providing a foundation for developing more reliable and trustworthy models. Our benchmark and code are available at https://anonymous550520.github.io/.

Method: Created AI4VA-FG benchmark spanning recognition, detection, character reasoning, and narrative construction. Evaluated state-of-the-art models and investigated post-training strategies including SFT-S, SFT-R, RL, and proposed RARL for dynamic region attention through zoom-in operations.

Result: Revealed substantial performance deficits across core comic understanding tasks in both proprietary and open-source models. RL and RARL applied to Qwen2.5-VL yielded significant gains in entity recognition and storyline ordering.

Conclusion: Comic understanding remains an unsolved challenge, but RL and RARL approaches show promise for enhancing VLM capabilities in this domain, paving the way for more accurate comic analysis applications.

Abstract: Complex visual narratives, such as comics, present a significant challenge to Vision-Language Models (VLMs). Despite excelling on natural images, VLMs often struggle with stylized line art, onomatopoeia, and densely packed multi-panel layouts. To address this gap, we introduce AI4VA-FG, the first fine-grained and comprehensive benchmark for VLM-based comic understanding. It spans tasks from foundational recognition and detection to high-level character reasoning and narrative construction, supported by dense annotations for characters, poses, and depth. Beyond that, we evaluate state-of-the-art proprietary models, including GPT-4o and Gemini-2.5, and open-source models such as Qwen2.5-VL, revealing substantial performance deficits across core tasks of our benchmarks and underscoring that comic understanding remains an unsolved challenge. To enhance VLMs’ capabilities in this domain, we systematically investigate post-training strategies, including supervised fine-tuning on solutions (SFT-S), supervised fine-tuning on reasoning trajectories (SFT-R), and reinforcement learning (RL). Beyond that, inspired by the emerging “Thinking with Images” paradigm, we propose Region-Aware Reinforcement Learning (RARL) for VLMs, which trains models to dynamically attend to relevant regions through zoom-in operations. We observe that when applied to the Qwen2.5-VL model, RL and RARL yield significant gains in low-level entity recognition and high-level storyline ordering, paving the way for more accurate and efficient VLM applications in the comics domain.

Method: Created a hierarchical benchmark with progressive QA pairs from simple infraction identification to complex penalty prediction, incorporating both image and video modalities with manual bounding box annotations for visual grounding.

Result: State-of-the-art models perform poorly on challenging tasks. Training via SFT and RL improves scores but remains low, highlighting significant capability gaps in current multimodal models.

Conclusion: SportR presents a challenging benchmark that reveals substantial limitations in current models’ sports reasoning abilities and provides a critical resource to drive future research in multimodal sports intelligence.

Abstract: Deeply understanding sports requires an intricate blend of fine-grained visual perception and rule-based reasoning - a challenge that pushes the limits of current multimodal models. To succeed, models must master three critical capabilities: perceiving nuanced visual details, applying abstract sport rule knowledge, and grounding that knowledge in specific visual evidence. Current sports benchmarks either cover single sports or lack the detailed reasoning chains and precise visual grounding needed to robustly evaluate these core capabilities in a multi-sport context. To address this gap, we introduce SportR, the first multi-sports large-scale benchmark designed to train and evaluate MLLMs on the fundamental reasoning required for sports intelligence. Our benchmark provides a dataset of 5,017 images and 2,101 videos. To enable granular evaluation, we structure our benchmark around a progressive hierarchy of question-answer (QA) pairs designed to probe reasoning at increasing depths - from simple infraction identification to complex penalty prediction. For the most advanced tasks requiring multi-step reasoning, such as determining penalties or explaining tactics, we provide 7,118 high-quality, human-authored Chain of Thought (CoT) annotations. In addition, our benchmark incorporates both image and video modalities and provides manual bounding box annotations to test visual grounding in the image part directly. Extensive experiments demonstrate the profound difficulty of our benchmark. State-of-the-art baseline models perform poorly on our most challenging tasks. While training on our data via Supervised Fine-Tuning and Reinforcement Learning improves these scores, they remain relatively low, highlighting a significant gap in current model capabilities. SportR presents a new challenge for the community, providing a critical resource to drive future research in multimodal sports reasoning.

Method: Created a dataset comprising eight complete laparoscopic cholecystectomy videos from three medical centers with frame-level annotations for three interconnected tasks: surgical phase recognition (485,875 frames), instrument keypoint estimation (19,435 frames), and instrument instance segmentation (19,435 frames).

Result: PhaKIR is the first multi-institutional dataset to jointly provide phase labels, instrument pose information, and pixel-accurate instrument segmentations while enabling temporal context exploitation through full surgical procedure sequences.

Conclusion: The dataset serves as a benchmark for surgical scene understanding methods and has been validated through the PhaKIR Challenge at MICCAI 2024, demonstrating its quality and relevance for advancing RAMIS computer vision systems.

Abstract: Robotic- and computer-assisted minimally invasive surgery (RAMIS) is increasingly relying on computer vision methods for reliable instrument recognition and surgical workflow understanding. Developing such systems often requires large, well-annotated datasets, but existing resources often address isolated tasks, neglect temporal dependencies, or lack multi-center variability. We present the Surgical Procedure Phase, Keypoint, and Instrument Recognition (PhaKIR) dataset, comprising eight complete laparoscopic cholecystectomy videos recorded at three medical centers. The dataset provides frame-level annotations for three interconnected tasks: surgical phase recognition (485,875 frames), instrument keypoint estimation (19,435 frames), and instrument instance segmentation (19,435 frames). PhaKIR is, to our knowledge, the first multi-institutional dataset to jointly provide phase labels, instrument pose information, and pixel-accurate instrument segmentations, while also enabling the exploitation of temporal context since full surgical procedure sequences are available. It served as the basis for the PhaKIR Challenge as part of the Endoscopic Vision (EndoVis) Challenge at MICCAI 2024 to benchmark methods in surgical scene understanding, thereby further validating the dataset’s quality and relevance. The dataset is publicly available upon request via the Zenodo platform.

Method: Proposes SFMFusion with: 1) Three-branch structure coupling MMIF and IR to retain complete source image contents; 2) Spatial-Frequency Enhanced Mamba Block (SFMB) for comprehensive feature extraction; 3) Dynamic Fusion Mamba Block (DFMB) for cross-branch dynamic feature fusion.

Result: Extensive experiments show SFMFusion achieves better results than most state-of-the-art methods on six MMIF datasets.

Conclusion: SFMFusion effectively addresses limitations of existing MMIF methods by enhancing Mamba with spatial-frequency perception and efficiently leveraging image reconstruction as an auxiliary task, achieving superior fusion performance.

Abstract: Multi-Modal Image Fusion (MMIF) aims to integrate complementary image information from different modalities to produce informative images. Previous deep learning-based MMIF methods generally adopt Convolutional Neural Networks (CNNs) or Transformers for feature extraction. However, these methods deliver unsatisfactory performances due to the limited receptive field of CNNs and the high computational cost of Transformers. Recently, Mamba has demonstrated a powerful potential for modeling long-range dependencies with linear complexity, providing a promising solution to MMIF. Unfortunately, Mamba lacks full spatial and frequency perceptions, which are very important for MMIF. Moreover, employing Image Reconstruction (IR) as an auxiliary task has been proven beneficial for MMIF. However, a primary challenge is how to leverage IR efficiently and effectively. To address the above issues, we propose a novel framework named Spatial-Frequency Enhanced Mamba Fusion (SFMFusion) for MMIF. More specifically, we first propose a three-branch structure to couple MMIF and IR, which can retain complete contents from source images. Then, we propose the Spatial-Frequency Enhanced Mamba Block (SFMB), which can enhance Mamba in both spatial and frequency domains for comprehensive feature extraction. Finally, we propose the Dynamic Fusion Mamba Block (DFMB), which can be deployed across different branches for dynamic feature fusion. Extensive experiments show that our method achieves better results than most state-of-the-art methods on six MMIF datasets. The source code is available at https://github.com/SunHui1216/SFMFusion.

Method: Proposes a framework with two innovations: (1) robust 3D circle center estimator using conformal geometric algebra and RANSAC, and (2) chord-length variance minimization method to recover true 2D projected center, resolving dual-minima ambiguity via homography validation or quasi-RANSAC fallback.

Result: Significantly outperforms state-of-the-art approaches on synthetic and real-world datasets, reduces extrinsic estimation error, and enables robust calibration across diverse sensors and target types including natural circular objects.

Conclusion: The proposed framework provides accurate and robust LiDAR-camera extrinsic calibration with public code release for reproducibility, addressing key limitations in existing circular target-based calibration methods.

Abstract: Circular targets are widely used in LiDAR-camera extrinsic calibration due to their geometric consistency and ease of detection. However, achieving accurate 3D-2D circular center correspondence remains challenging. Existing methods often fail due to decoupled 3D fitting and erroneous 2D ellipse-center estimation. To address this, we propose a geometrically principled framework featuring two innovations: (i) a robust 3D circle center estimator based on conformal geometric algebra and RANSAC; and (ii) a chord-length variance minimization method to recover the true 2D projected center, resolving its dual-minima ambi- guity via homography validation or a quasi-RANSAC fallback. Evaluated on synthetic and real-world datasets, our framework significantly outperforms state-of-the-art approaches. It reduces extrinsic estimation error and enables robust calibration across diverse sensors and target types, including natural circular objects. Our code will be publicly released for reproducibility.

Method: Three-component framework: pulmonary perception module for lung abnormalities, knowledge-guided reasoning module for cardiovascular implications, and cardiac representation module for structural biomarkers, fused for holistic prediction.

Result: Achieves state-of-the-art performance for CVD screening and mortality prediction on NLST cohort, outperforming single-disease and purely image-based baselines.

Conclusion: Establishes a unified and explainable paradigm for cardiovascular analysis from LDCT that bridges image-based prediction with mechanism-based medical interpretation.

Abstract: Low-dose chest computed tomography (LDCT) inherently captures both pulmonary and cardiac structures, offering a unique opportunity for joint assessment of lung and cardiovascular health. However, most existing approaches treat these domains as independent tasks, overlooking their physiological interplay and shared imaging biomarkers. We propose an Explainable Cross-Disease Reasoning Framework that enables interpretable cardiopulmonary risk assessment from a single LDCT scan. The framework introduces an agentic reasoning process that emulates clinical diagnostic thinking-first perceiving pulmonary findings, then reasoning through established medical knowledge, and finally deriving a cardiovascular judgment with explanatory rationale. It integrates three synergistic components: a pulmonary perception module that summarizes lung abnormalities, a knowledge-guided reasoning module that infers their cardiovascular implications, and a cardiac representation module that encodes structural biomarkers. Their outputs are fused to produce a holistic cardiovascular risk prediction that is both accurate and physiologically grounded. Experiments on the NLST cohort demonstrate that the proposed framework achieves state-of-the-art performance for CVD screening and mortality prediction, outperforming single-disease and purely image-based baselines. Beyond quantitative gains, the framework provides human-verifiable reasoning that aligns with cardiological understanding, revealing coherent links between pulmonary abnormalities and cardiac stress mechanisms. Overall, this work establishes a unified and explainable paradigm for cardiovascular analysis from LDCT, bridging the gap between image-based prediction and mechanism-based medical interpretation.

Method: Uses appearance-position inconsistency to identify dynamic instances, employs instance-aware 4D Gaussians as unified volumetric representation, and implements a reciprocal mechanism where identity and dynamics reinforce each other for enhanced integrity and consistency.

Result: Experiments show DIAL-GS surpasses existing self-supervised baselines in reconstruction quality and instance-level editing capabilities in urban driving scenarios.

Conclusion: DIAL-GS provides a concise yet powerful solution for label-free urban scene modeling that enables dynamic-adaptive and instance-aware reconstruction with enhanced editing capabilities.

Abstract: Urban scene reconstruction is critical for autonomous driving, enabling structured 3D representations for data synthesis and closed-loop testing. Supervised approaches rely on costly human annotations and lack scalability, while current self-supervised methods often confuse static and dynamic elements and fail to distinguish individual dynamic objects, limiting fine-grained editing. We propose DIAL-GS, a novel dynamic instance-aware reconstruction method for label-free street scenes with 4D Gaussian Splatting. We first accurately identify dynamic instances by exploiting appearance-position inconsistency between warped rendering and actual observation. Guided by instance-level dynamic perception, we employ instance-aware 4D Gaussians as the unified volumetric representation, realizing dynamic-adaptive and instance-aware reconstruction. Furthermore, we introduce a reciprocal mechanism through which identity and dynamics reinforce each other, enhancing both integrity and consistency. Experiments on urban driving scenarios show that DIAL-GS surpasses existing self-supervised baselines in reconstruction quality and instance-level editing, offering a concise yet powerful solution for urban scene modeling.

Method: Proposes UniADC with two components: 1) Training-free controllable inpainting network that synthesizes anomaly images by repainting normal regions using anomaly priors, and 2) Multi-task discriminator trained on synthesized samples to align fine-grained image features with anomaly-category embeddings.

Result: Extensive experiments on MVTec-FS, MTD, and WFDD datasets show UniADC consistently outperforms existing methods in anomaly detection, localization, and classification.

Conclusion: UniADC effectively unifies anomaly detection and classification tasks, enabling superior performance with minimal anomaly data through innovative controllable inpainting and multi-task learning.

Abstract: In this paper, we introduce the task of unified anomaly detection and classification, which aims to simultaneously detect anomalous regions in images and identify their specific categories. Existing methods typically treat anomaly detection and classification as separate tasks, thereby neglecting their inherent correlation, limiting information sharing, and resulting in suboptimal performance. To address this, we propose UniADC, a unified anomaly detection and classification model that can effectively perform both tasks with only a few or even no anomaly images. Specifically, UniADC consists of two key components: a training-free controllable inpainting network and a multi-task discriminator. The inpainting network can synthesize anomaly images of specific categories by repainting normal regions guided by anomaly priors, and can also repaint few-shot anomaly samples to augment the available anomaly data. The multi-task discriminator is then trained on these synthesized samples, enabling precise anomaly detection and classification by aligning fine-grained image features with anomaly-category embeddings. We conduct extensive experiments on three anomaly detection and classification datasets, including MVTec-FS, MTD, and WFDD, and the results demonstrate that UniADC consistently outperforms existing methods in anomaly detection, localization, and classification. The code is available at https://github.com/cnulab/UniADC.

Method: Proposes FreqGRL with three modules: Low-Frequency Replacement (LFR) substitutes source low-frequency components with target ones; High-Frequency Enhancement (HFE) learns directly on high-frequency features; Global Frequency Filter (GFF) suppresses noisy frequencies and emphasizes informative ones.

Result: Extensive experiments on five standard CD-FSL benchmarks demonstrate state-of-the-art performance.

Conclusion: The frequency-space perspective and proposed framework effectively mitigate data imbalance challenges in CD-FSL, achieving superior cross-domain generalization through frequency-guided representation learning.

Abstract: Cross-domain few-shot learning (CD-FSL) aims to recognize novel classes with only a few labeled examples under significant domain shifts. While recent approaches leverage a limited amount of labeled target-domain data to improve performance, the severe imbalance between abundant source data and scarce target data remains a critical challenge for effective representation learning. We present the first frequency-space perspective to analyze this issue and identify two key challenges: (1) models are easily biased toward source-specific knowledge encoded in the low-frequency components of source data, and (2) the sparsity of target data hinders the learning of high-frequency, domain-generalizable features. To address these challenges, we propose \textbf{FreqGRL}, a novel CD-FSL framework that mitigates the impact of data imbalance in the frequency space. Specifically, we introduce a Low-Frequency Replacement (LFR) module that substitutes the low-frequency components of source tasks with those from the target domain to create new source tasks that better align with target characteristics, thus reducing source-specific biases and promoting generalizable representation learning. We further design a High-Frequency Enhancement (HFE) module that filters out low-frequency components and performs learning directly on high-frequency features in the frequency space to improve cross-domain generalization. Additionally, a Global Frequency Filter (GFF) is incorporated to suppress noisy or irrelevant frequencies and emphasize informative ones, mitigating overfitting risks under limited target supervision. Extensive experiments on five standard CD-FSL benchmarks demonstrate that our frequency-guided framework achieves state-of-the-art performance.

Method: Generates coarse masks and point prompts from VLM output as visual-only prompts for SAM, plus a training-free refinement module to improve boundary quality and enable instance-level segmentation.

Result: Achieves state-of-the-art performance across multiple metrics and model sizes on the new RISeg benchmark (918 images, 2,533 instance masks).

Conclusion: NOVO demonstrates effective and scalable reasoning segmentation through visual-only prompts that maintain compatibility with pretrained segmentation models.

Abstract: In this study, we propose NOVO (NO text, Visual-Only prompts), a novel framework that bridges vision-language models (VLMs) and segmentation models through visual-only prompts. Unlike prior approaches that feed text-derived SEG token embeddings into segmentation models, NOVO instead generates a coarse mask and point prompts from the VLM output. These visual prompts are compatible with the Segment Anything Model (SAM), preserving alignment with its pretrained capabilities. To further enhance boundary quality and enable instance-level segmentation, we introduce a training-free refinement module that reduces visual artifacts and improves the quality of segmentation masks. We also present RISeg, a new benchmark comprising 918 images, 2,533 instance-level masks, and diverse reasoning queries to evaluate this task. Experiments demonstrate that NOVO achieves state-of-the-art performance across multiple metrics and model sizes, demonstrating its effectiveness and scalability in reasoning segmentation.

Method: Proposed AL framework leverages complementary clustering methods to uncover structurally ambiguous regions in embedding space, mines informative and representative sample pairs, uses oracle feedback (must-link/cannot-link constraints), and integrates with unsupervised methods through constrained clustering refinement.

Result: Achieves average improvements of 10.49%, 11.19% and 3.99% (mAP) on 13 wildlife datasets over foundational, USL and AL methods respectively, using only 0.033% of all annotations. Also shows 11.09%, 8.2% and 2.06% improvement for unknown individuals in open-world setting.

Conclusion: The proposed active learning framework effectively addresses animal Re-ID challenges by strategically targeting ambiguous regions with minimal annotation effort, outperforming existing methods and achieving state-of-the-art performance across multiple datasets.

Abstract: Animal Re-ID has recently gained substantial attention in the AI research community due to its high impact on biodiversity monitoring and unique research challenges arising from environmental factors. The subtle distinguishing patterns, handling new species and the inherent open-set nature make the problem even harder. To address these complexities, foundation models trained on labeled, large-scale and multi-species animal Re-ID datasets have recently been introduced to enable zero-shot Re-ID. However, our benchmarking reveals significant gaps in their zero-shot Re-ID performance for both known and unknown species. While this highlights the need for collecting labeled data in new domains, exhaustive annotation for Re-ID is laborious and requires domain expertise. Our analyses show that existing unsupervised (USL) and AL Re-ID methods underperform for animal Re-ID. To address these limitations, we introduce a novel AL Re-ID framework that leverages complementary clustering methods to uncover and target structurally ambiguous regions in the embedding space for mining pairs of samples that are both informative and broadly representative. Oracle feedback on these pairs, in the form of must-link and cannot-link constraints, facilitates a simple annotation interface, which naturally integrates with existing USL methods through our proposed constrained clustering refinement algorithm. Through extensive experiments, we demonstrate that, by utilizing only 0.033% of all annotations, our approach consistently outperforms existing foundational, USL and AL baselines. Specifically, we report an average improvement of 10.49%, 11.19% and 3.99% (mAP) on 13 wildlife datasets over foundational, USL and AL methods, respectively, while attaining state-of-the-art performance on each dataset. Furthermore, we also show an improvement of 11.09%, 8.2% and 2.06% for unknown individuals in an open-world setting.

Method: Introduces MDS task, creates M3DS dataset with automated diagnosis chain-of-thought generation, proposes Sim4Seg framework with Region-Aware Vision-Language Similarity to Mask (RVLS2M) module, and investigates test-time scaling strategy.

Result: Experimental results demonstrate that the proposed method outperforms baselines in both segmentation and diagnosis tasks.

Conclusion: The approach successfully integrates medical segmentation with diagnostic reasoning, providing explainable diagnoses alongside segmentation results through the novel MDS task and Sim4Seg framework.

Abstract: Despite significant progress in pixel-level medical image analysis, existing medical image segmentation models rarely explore medical segmentation and diagnosis tasks jointly. However, it is crucial for patients that models can provide explainable diagnoses along with medical segmentation results. In this paper, we introduce a medical vision-language task named Medical Diagnosis Segmentation (MDS), which aims to understand clinical queries for medical images and generate the corresponding segmentation masks as well as diagnostic results. To facilitate this task, we first present the Multimodal Multi-disease Medical Diagnosis Segmentation (M3DS) dataset, containing diverse multimodal multi-disease medical images paired with their corresponding segmentation masks and diagnosis chain-of-thought, created via an automated diagnosis chain-of-thought generation pipeline. Moreover, we propose Sim4Seg, a novel framework that improves the performance of diagnosis segmentation by taking advantage of the Region-Aware Vision-Language Similarity to Mask (RVLS2M) module. To improve overall performance, we investigate a test-time scaling strategy for MDS tasks. Experimental results demonstrate that our method outperforms the baselines in both segmentation and diagnosis.

Method: Proposes REOcc with two main components: Radar Densifier and Radar Amplifier that refine radar features by integrating spatial and contextual information to enhance spatial density and quality.

Result: Extensive experiments on Occ3D-nuScenes benchmark show significant performance gains over camera-only baseline, especially in dynamic object classes. Effectively mitigates radar sparsity and noise.

Conclusion: REOcc enables radar to better complement camera data, unlocking the full potential of camera-radar fusion for robust and reliable 3D occupancy prediction.

Abstract: Vision-based 3D occupancy prediction has made significant advancements, but its reliance on cameras alone struggles in challenging environments. This limitation has driven the adoption of sensor fusion, among which camera-radar fusion stands out as a promising solution due to their complementary strengths. However, the sparsity and noise of the radar data limits its effectiveness, leading to suboptimal fusion performance. In this paper, we propose REOcc, a novel camera-radar fusion network designed to enrich radar feature representations for 3D occupancy prediction. Our approach introduces two main components, a Radar Densifier and a Radar Amplifier, which refine radar features by integrating spatial and contextual information, effectively enhancing spatial density and quality. Extensive experiments on the Occ3D-nuScenes benchmark demonstrate that REOcc achieves significant performance gains over the camera-only baseline model, particularly in dynamic object classes. These results underscore REOcc’s capability to mitigate the sparsity and noise of the radar data. Consequently, radar complements camera data more effectively, unlocking the full potential of camera-radar fusion for robust and reliable 3D occupancy prediction.

Method: Hypernetwork generates prediction weights from concept embeddings, plus modified sparsemax with learnable temperature for dynamic concept selection.

Result: Achieves comparable accuracy to SOTA baselines with similar concept count, generalizes well to unseen concepts with single-epoch fine-tuning.

Conclusion: FCBM provides strong adaptability and flexibility for concept bottleneck models, supporting dynamic concept updates without complete retraining.

Abstract: Concept bottleneck models (CBMs) improve neural network interpretability by introducing an intermediate layer that maps human-understandable concepts to predictions. Recent work has explored the use of vision-language models (VLMs) to automate concept selection and annotation. However, existing VLM-based CBMs typically require full model retraining when new concepts are involved, which limits their adaptability and flexibility in real-world scenarios, especially considering the rapid evolution of vision-language foundation models. To address these issues, we propose Flexible Concept Bottleneck Model (FCBM), which supports dynamic concept adaptation, including complete replacement of the original concept set. Specifically, we design a hypernetwork that generates prediction weights based on concept embeddings, allowing seamless integration of new concepts without retraining the entire model. In addition, we introduce a modified sparsemax module with a learnable temperature parameter that dynamically selects the most relevant concepts, enabling the model to focus on the most informative features. Extensive experiments on five public benchmarks demonstrate that our method achieves accuracy comparable to state-of-the-art baselines with a similar number of effective concepts. Moreover, the model generalizes well to unseen concepts with just a single epoch of fine-tuning, demonstrating its strong adaptability and flexibility.

Method: Uses text-guided style transfer with a lightweight U-Net trained with CLIP-based losses. Generates anomaly masks and text prompts from single normal images, then stylizes normal images into anomalies localized by masks and aligned with text semantics.

Result: Outperforms existing methods on MVTec-AD and VisA datasets in generating high-quality, diverse anomaly images. Generated anomalies enhance anomaly detection performance.

Conclusion: AnoStyler provides an effective, lightweight solution for zero-shot anomaly generation that addresses key limitations of previous approaches and improves anomaly detection.

Abstract: Anomaly generation has been widely explored to address the scarcity of anomaly images in real-world data. However, existing methods typically suffer from at least one of the following limitations, hindering their practical deployment: (1) lack of visual realism in generated anomalies; (2) dependence on large amounts of real images; and (3) use of memory-intensive, heavyweight model architectures. To overcome these limitations, we propose AnoStyler, a lightweight yet effective method that frames zero-shot anomaly generation as text-guided style transfer. Given a single normal image along with its category label and expected defect type, an anomaly mask indicating the localized anomaly regions and two-class text prompts representing the normal and anomaly states are generated using generalizable category-agnostic procedures. A lightweight U-Net model trained with CLIP-based loss functions is used to stylize the normal image into a visually realistic anomaly image, where anomalies are localized by the anomaly mask and semantically aligned with the text prompts. Extensive experiments on the MVTec-AD and VisA datasets show that AnoStyler outperforms existing anomaly generation methods in generating high-quality and diverse anomaly images. Furthermore, using these generated anomalies helps enhance anomaly detection performance.

Method: Proposes Spatial-Projection Alignment (SPAN) with two components: Spatial Point Alignment for global spatial constraints between predicted and ground-truth boxes, and 3D-2D Projection Alignment for tight alignment on image plane. Uses Hierarchical Task Learning for stable training.

Result: The method can be easily integrated into any established monocular 3D detector and delivers significant performance improvements.

Conclusion: SPAN effectively addresses geometric inconsistency in monocular 3D detection through spatial-projection alignment and hierarchical learning, achieving better performance while being easily integrable.

Abstract: Existing monocular 3D detectors typically tame the pronounced nonlinear regression of 3D bounding box through decoupled prediction paradigm, which employs multiple branches to estimate geometric center, depth, dimensions, and rotation angle separately. Although this decoupling strategy simplifies the learning process, it inherently ignores the geometric collaborative constraints between different attributes, resulting in the lack of geometric consistency prior, thereby leading to suboptimal performance. To address this issue, we propose novel Spatial-Projection Alignment (SPAN) with two pivotal components: (i). Spatial Point Alignment enforces an explicit global spatial constraint between the predicted and ground-truth 3D bounding boxes, thereby rectifying spatial drift caused by decoupled attribute regression. (ii). 3D-2D Projection Alignment ensures that the projected 3D box is aligned tightly within its corresponding 2D detection bounding box on the image plane, mitigating projection misalignment overlooked in previous works. To ensure training stability, we further introduce a Hierarchical Task Learning strategy that progressively incorporates spatial-projection alignment as 3D attribute predictions refine, preventing early stage error propagation across attributes. Extensive experiments demonstrate that the proposed method can be easily integrated into any established monocular 3D detector and delivers significant performance improvements.

Method: K-Stain has three components: Hierarchical Spatial Keypoint Detector (HSKD) for identifying keypoints, Keypoint-aware Enhancement Generator (KEG) for integrating keypoints during generation, and Keypoint Guided Discriminator (KGD) for improved spatial detail sensitivity.

Result: Extensive experiments show K-Stain outperforms state-of-the-art methods in both quantitative metrics and visual quality, producing more accurate and visually consistent IHC images.

Conclusion: The keypoint-based approach effectively leverages contextual information from adjacent slices to enhance synthesized IHC image fidelity, demonstrating superior performance over existing methods.

Abstract: Virtual staining offers a promising method for converting Hematoxylin and Eosin (H&E) images into Immunohistochemical (IHC) images, eliminating the need for costly chemical processes. However, existing methods often struggle to utilize spatial information effectively due to misalignment in tissue slices. To overcome this challenge, we leverage keypoints as robust indicators of spatial correspondence, enabling more precise alignment and integration of structural details in synthesized IHC images. We introduce K-Stain, a novel framework that employs keypoint-based spatial and semantic relationships to enhance synthesized IHC image fidelity. K-Stain comprises three main components: (1) a Hierarchical Spatial Keypoint Detector (HSKD) for identifying keypoints in stain images, (2) a Keypoint-aware Enhancement Generator (KEG) that integrates these keypoints during image generation, and (3) a Keypoint Guided Discriminator (KGD) that improves the discriminator’s sensitivity to spatial details. Our approach leverages contextual information from adjacent slices, resulting in more accurate and visually consistent IHC images. Extensive experiments show that K-Stain outperforms state-of-the-art methods in quantitative metrics and visual quality.

Method: Proposes MirrorMamba with multiple cues (perceived depth, correspondence, optical flow), Mamba-based Multidirection Correspondence Extractor for global receptive field with linear complexity, and Mamba-based layer-wise boundary enforcement decoder to resolve unclear boundaries.

Result: Outperforms existing state-of-the-art approaches on video mirror detection benchmarks and achieves state-of-the-art performance on the most challenging image-based mirror detection dataset.

Conclusion: First successful application of Mamba-based architecture in mirror detection, demonstrating superior performance, robustness, and generalizability across both video and image domains.

Abstract: Video mirror detection has received significant research attention, yet existing methods suffer from limited performance and robustness. These approaches often over-rely on single, unreliable dynamic features, and are typically built on CNNs with limited receptive fields or Transformers with quadratic computational complexity. To address these limitations, we propose a new effective and scalable video mirror detection method, called MirrorMamba. Our approach leverages multiple cues to adapt to diverse conditions, incorporating perceived depth, correspondence and optical. We also introduce an innovative Mamba-based Multidirection Correspondence Extractor, which benefits from the global receptive field and linear complexity of the emerging Mamba spatial state model to effectively capture correspondence properties. Additionally, we design a Mamba-based layer-wise boundary enforcement decoder to resolve the unclear boundary caused by the blurred depth map. Notably, this work marks the first successful application of the Mamba-based architecture in the field of mirror detection. Extensive experiments demonstrate that our method outperforms existing state-of-the-art approaches for video mirror detection on the benchmark datasets. Furthermore, on the most challenging and representative image-based mirror detection dataset, our approach achieves state-of-the-art performance, proving its robustness and generalizability.

Method: MRT partitions images into windows, uses RWKV modules for global dependencies across windows and Transformer blocks for local redundancies within windows, plus a dedicated RWKV Compression Model (RCM) for 1-D latent features.

Result: Achieves bitrate savings of 43.75% on Kodak and 30.59% on CLIC2020 datasets compared to state-of-the-art GLC, with superior reconstruction quality below 0.02 bpp.

Conclusion: The mixed RWKV-Transformer architecture with 1-D latent representations significantly improves image compression efficiency by reducing spatial redundancy.

Abstract: Recent advances in extreme image compression have revealed that mapping pixel data into highly compact latent representations can significantly improve coding efficiency. However, most existing methods compress images into 2-D latent spaces via convolutional neural networks (CNNs) or Swin Transformers, which tend to retain substantial spatial redundancy, thereby limiting overall compression performance. In this paper, we propose a novel Mixed RWKV-Transformer (MRT) architecture that encodes images into more compact 1-D latent representations by synergistically integrating the complementary strengths of linear-attention-based RWKV and self-attention-based Transformer models. Specifically, MRT partitions each image into fixed-size windows, utilizing RWKV modules to capture global dependencies across windows and Transformer blocks to model local redundancies within each window. The hierarchical attention mechanism enables more efficient and compact representation learning in the 1-D domain. To further enhance compression efficiency, we introduce a dedicated RWKV Compression Model (RCM) tailored to the structure characteristics of the intermediate 1-D latent features in MRT. Extensive experiments on standard image compression benchmarks validate the effectiveness of our approach. The proposed MRT framework consistently achieves superior reconstruction quality at bitrates below 0.02 bits per pixel (bpp). Quantitative results based on the DISTS metric show that MRT significantly outperforms the state-of-the-art 2-D architecture GLC, achieving bitrate savings of 43.75%, 30.59% on the Kodak and CLIC2020 test datasets, respectively.

Method: Proposes Relative Energy Learning (REL) framework that leverages energy gap between positive and negative logits as relative scoring function, combined with Point Raise - a lightweight data synthesis strategy that perturbs existing point clouds to generate auxiliary anomalies.

Result: Outperforms existing methods by a large margin on SemanticKITTI and Spotting the Unexpected (STU) benchmarks, demonstrating improved robustness across various scenes.

Conclusion: Modeling relative energy combined with simple synthetic outliers provides a principled and scalable solution for reliable OOD detection in open-world autonomous driving.

Abstract: Out-of-distribution (OOD) detection is a critical requirement for reliable autonomous driving, where safety depends on recognizing road obstacles and unexpected objects beyond the training distribution. Despite extensive research on OOD detection in 2D images, direct transfer to 3D LiDAR point clouds has been proven ineffective. Current LiDAR OOD methods struggle to distinguish rare anomalies from common classes, leading to high false-positive rates and overconfident errors in safety-critical settings. We propose Relative Energy Learning (REL), a simple yet effective framework for OOD detection in LiDAR point clouds. REL leverages the energy gap between positive (in-distribution) and negative logits as a relative scoring function, mitigating calibration issues in raw energy values and improving robustness across various scenes. To address the absence of OOD samples during training, we propose a lightweight data synthesis strategy called Point Raise, which perturbs existing point clouds to generate auxiliary anomalies without altering the inlier semantics. Evaluated on SemanticKITTI and the Spotting the Unexpected (STU) benchmark, REL consistently outperforms existing methods by a large margin. Our results highlight that modeling relative energy, combined with simple synthetic outliers, provides a principled and scalable solution for reliable OOD detection in open-world autonomous driving.

Method: Three-stage pipeline: 1) diffusion-based inpainting for missing regions, 2) GAN-based latent optimization for style/structure consistency, 3) diffusion-based repainting for fine details. Also introduces TexHub dataset with 20,000 multi-style UV textures.

Result: Achieves high-quality topology-aligned texture synthesis with both artistic and geometric coherence, establishing new state-of-the-art in multi-style facial texture reconstruction.

Conclusion: AvatarTex successfully integrates diffusion models’ diversity with GANs’ structured latent space to overcome limitations of existing methods, with TexHub dataset enabling future research in this field.

Abstract: We present AvatarTex, a high-fidelity facial texture reconstruction framework capable of generating both stylized and photorealistic textures from a single image. Existing methods struggle with stylized avatars due to the lack of diverse multi-style datasets and challenges in maintaining geometric consistency in non-standard textures. To address these limitations, AvatarTex introduces a novel three-stage diffusion-to-GAN pipeline. Our key insight is that while diffusion models excel at generating diversified textures, they lack explicit UV constraints, whereas GANs provide a well-structured latent space that ensures style and topology consistency. By integrating these strengths, AvatarTex achieves high-quality topology-aligned texture synthesis with both artistic and geometric coherence. Specifically, our three-stage pipeline first completes missing texture regions via diffusion-based inpainting, refines style and structure consistency using GAN-based latent optimization, and enhances fine details through diffusion-based repainting. To address the need for a stylized texture dataset, we introduce TexHub, a high-resolution collection of 20,000 multi-style UV textures with precise UV-aligned layouts. By leveraging TexHub and our structured diffusion-to-GAN pipeline, AvatarTex establishes a new state-of-the-art in multi-style facial texture reconstruction. TexHub will be released upon publication to facilitate future research in this field.

Method: Argus intelligently switches between different approximation strategies for each prompt, selecting the right level of approximation to balance quality and throughput requirements on a fixed-size cluster.

Result: Argus achieves 10x fewer latency SLO violations, 10% higher average quality, and 40% higher throughput compared to baselines on two real-world workload traces.

Conclusion: The system successfully addresses the challenge of designing high-throughput T2I inference by dynamically selecting appropriate approximation levels, demonstrating significant improvements in both performance and quality metrics.

Abstract: Text-to-image (T2I) models have gained significant popularity. Most of these are diffusion models with unique computational characteristics, distinct from both traditional small-scale ML models and large language models. They are highly compute-bound and use an iterative denoising process to generate images, leading to very high inference time. This creates significant challenges in designing a high-throughput system. We discovered that a large fraction of prompts can be served using faster, approximated models. However, the approximation setting must be carefully calibrated for each prompt to avoid quality degradation. Designing a high-throughput system that assigns each prompt to the appropriate model and compatible approximation setting remains a challenging problem. We present Argus, a high-throughput T2I inference system that selects the right level of approximation for each prompt to maintain quality while meeting throughput targets on a fixed-size cluster. Argus intelligently switches between different approximation strategies to satisfy both throughput and quality requirements. Overall, Argus achieves 10x fewer latency service-level objective (SLO) violations, 10% higher average quality, and 40% higher throughput compared to baselines on two real-world workload traces.

Method: Constructed OmniRain3D dataset with perspective heterogeneity and brightness dynamicity; proposed REVR-GSNet framework integrating recursive brightness enhancement, Gaussian primitive optimization, and GS-guided rain elimination through joint alternating optimization.

Result: Extensive experiments demonstrate effectiveness of both the dataset and method for multi-view image deraining and rainy 3D scene reconstruction.

Conclusion: The dataset and method provide foundation for future research on multi-view image deraining and rainy 3D scene reconstruction, achieving high-fidelity reconstruction of clean 3D scenes from rain-degraded inputs.

Abstract: Rain degrades the visual quality of multi-view images, which are essential for 3D scene reconstruction, resulting in inaccurate and incomplete reconstruction results. Existing datasets often overlook two critical characteristics of real rainy 3D scenes: the viewpoint-dependent variation in the appearance of rain streaks caused by their projection onto 2D images, and the reduction in ambient brightness resulting from cloud coverage during rainfall. To improve data realism, we construct a new dataset named OmniRain3D that incorporates perspective heterogeneity and brightness dynamicity, enabling more faithful simulation of rain degradation in 3D scenes. Based on this dataset, we propose an end-to-end reconstruction framework named REVR-GSNet (Rain Elimination and Visibility Recovery for 3D Gaussian Splatting). Specifically, REVR-GSNet integrates recursive brightness enhancement, Gaussian primitive optimization, and GS-guided rain elimination into a unified architecture through joint alternating optimization, achieving high-fidelity reconstruction of clean 3D scenes from rain-degraded inputs. Extensive experiments show the effectiveness of our dataset and method. Our dataset and method provide a foundation for future research on multi-view image deraining and rainy 3D scene reconstruction.

Method: Uses multi-scale flow-based model enhanced with LPIPS energy guidance during sampling to ensure perceptual realism and output variety from single optical images.

Result: Superior performance in visual quality, quantitative measures, and downstream tasks compared to other one-shot generation techniques; generated images achieve high fidelity and meaningful diversity.

Conclusion: SinSEMI-generated images are suitable as training data for semiconductor AI applications, effectively addressing the data scarcity problem in early-stage semiconductor equipment development.

Abstract: In the early stages of semiconductor equipment development, obtaining large quantities of raw optical images poses a significant challenge. This data scarcity hinder the advancement of AI-powered solutions in semiconductor manufacturing. To address this challenge, we introduce SinSEMI, a novel one-shot learning approach that generates diverse and highly realistic images from single optical image. SinSEMI employs a multi-scale flow-based model enhanced with LPIPS (Learned Perceptual Image Patch Similarity) energy guidance during sampling, ensuring both perceptual realism and output variety. We also introduce a comprehensive evaluation framework tailored for this application, which enables a thorough assessment using just two reference images. Through the evaluation against multiple one-shot generation techniques, we demonstrate SinSEMI’s superior performance in visual quality, quantitative measures, and downstream tasks. Our experimental results demonstrate that SinSEMI-generated images achieve both high fidelity and meaningful diversity, making them suitable as training data for semiconductor AI applications.

Method: Proposes Otter with Compound Segmentation Module (CSM) to segment and emphasize key patches, and Temporal Reconstruction Module (TRM) for bidirectional scanning to reconstruct temporal relations. Combines regular prototype with temporal-enhanced prototype for better subject emphasis and temporal modeling.

Result: Achieves state-of-the-art performance on SSv2, Kinetics, UCF101, and HMDB51 benchmarks. Additional evaluation on VideoBadminton dataset validates superiority in wide-angle FSAR.

Conclusion: Otter effectively addresses background distractions and temporal relation degradation in wide-angle FSAR through compound segmentation and temporal reconstruction, demonstrating superior performance across multiple benchmarks.

Abstract: Wide-angle videos in few-shot action recognition (FSAR) effectively express actions within specific scenarios. However, without a global understanding of both subjects and background, recognizing actions in such samples remains challenging because of the background distractions. Receptance Weighted Key Value (RWKV), which learns interaction between various dimensions, shows promise for global modeling. While directly applying RWKV to wide-angle FSAR may fail to highlight subjects due to excessive background information. Additionally, temporal relation degraded by frames with similar backgrounds is difficult to reconstruct, further impacting performance. Therefore, we design the CompOund SegmenTation and Temporal REconstructing RWKV (Otter). Specifically, the Compound Segmentation Module~(CSM) is devised to segment and emphasize key patches in each frame, effectively highlighting subjects against background information. The Temporal Reconstruction Module (TRM) is incorporated into the temporal-enhanced prototype construction to enable bidirectional scanning, allowing better reconstruct temporal relation. Furthermore, a regular prototype is combined with the temporal-enhanced prototype to simultaneously enhance subject emphasis and temporal modeling, improving wide-angle FSAR performance. Extensive experiments on benchmarks such as SSv2, Kinetics, UCF101, and HMDB51 demonstrate that Otter achieves state-of-the-art performance. Extra evaluation on the VideoBadminton dataset further validates the superiority of Otter in wide-angle FSAR.

Method: Uses global and local branches with 3x3x3 local blocks for part-level analysis, employs pseudo-labeling and local loss function for unsupervised training with local text labels.

Result: Demonstrates that understanding 3D object parts enhances overall 3D object understanding, achieves reliable and meaningful part-level reasoning results.

Conclusion: First successful attempt at unsupervised 3D part-level reasoning, showing that part-level analysis improves comprehensive 3D object understanding without requiring part annotations.

Abstract: In this paper, we propose PointCubeNet, a novel multi-modal 3D understanding framework that achieves part-level reasoning without requiring any part annotations. PointCubeNet comprises global and local branches. The proposed local branch, structured into 3x3x3 local blocks, enables part-level analysis of point cloud sub-regions with the corresponding local text labels. Leveraging the proposed pseudo-labeling method and local loss function, PointCubeNet is effectively trained in an unsupervised manner. The experimental results demonstrate that understanding 3D object parts enhances the understanding of the overall 3D object. In addition, this is the first attempt to perform unsupervised 3D part-level reasoning and achieves reliable and meaningful results.

Method: Proposed a PnP algorithm based on PDHG that replaces the proximal operator with time-dependent denoisers from flow matching models, supporting both ℓ₁ and ℓ₂ fidelity terms.

Result: Validated on denoising, super-resolution, deblurring, and inpainting tasks; ℓ₁ and ℓ₂ fidelity outperformed squared ℓ₂ loss for non-Gaussian noise scenarios.

Conclusion: The proposed method is computationally efficient, memory-friendly, and robust to various noise types, expanding PnP’s applicability beyond Gaussian noise settings.

Abstract: Regularized optimization has been a classical approach to solving imaging inverse problems, where the regularization term enforces desirable properties of the unknown image. Recently, the integration of flow matching generative models into image restoration has garnered significant attention, owing to their powerful prior modeling capabilities. In this work, we incorporate such generative priors into a Plug-and-Play (PnP) framework based on proximal splitting, where the proximal operator associated with the regularizer is replaced by a time-dependent denoiser derived from the generative model. While existing PnP methods have achieved notable success in inverse problems with smooth squared $\ell_2$ data fidelity–typically associated with Gaussian noise–their applicability to more general data fidelity terms remains underexplored. To address this, we propose a general and efficient PnP algorithm inspired by the primal-dual hybrid gradient (PDHG) method. Our approach is computationally efficient, memory-friendly, and accommodates a wide range of fidelity terms. In particular, it supports both $\ell_1$ and $\ell_2$ norm-based losses, enabling robustness to non-Gaussian noise types such as Poisson and impulse noise. We validate our method on several image restoration tasks, including denoising, super-resolution, deblurring, and inpainting, and demonstrate that $\ell_1$ and $\ell_2$ fidelity terms outperform the conventional squared $\ell_2$ loss in the presence of non-Gaussian noise.

Method: Proposes Med-SORA with three key components: 1) RAG-based dataset construction, 2) Soft labeling with learnable organ anchors to capture one-to-many symptom-organ relationships, and 3) A 2D-3D cross-attention architecture to fuse local and global image features.

Result: Experimental results show that Med-SORA outperforms existing medical multimodal models and enables accurate 3D clinical reasoning.

Conclusion: This is the first work to address symptom-to-organ reasoning in medical multimodal learning, successfully capturing complex symptom-organ relationships and incorporating 3D anatomical context for improved clinical reasoning.

Abstract: Understanding symptom-image associations is crucial for clinical reasoning. However, existing medical multimodal models often rely on simple one-to-one hard labeling, oversimplifying clinical reality where symptoms relate to multiple organs. In addition, they mainly use single-slice 2D features without incorporating 3D information, limiting their ability to capture full anatomical context. In this study, we propose Med-SORA, a framework for symptom-to-organ reasoning in abdominal CT images. Med-SORA introduces RAG-based dataset construction, soft labeling with learnable organ anchors to capture one-to-many symptom-organ relationships, and a 2D-3D cross-attention architecture to fuse local and global image features. To our knowledge, this is the first work to address symptom-to-organ reasoning in medical multimodal learning. Experimental results show that Med-SORA outperforms existing medical multimodal models and enables accurate 3D clinical reasoning.

Method: Two-stage architecture: 1) CAST converts RGB to HSV and encodes H/V channels into semantic tokens, 2) HSV-LUT module generates independent correction curves for H, S, V channels using these tokens.

Result: Model significantly outperforms existing methods in visual effects and achieves state-of-the-art performance on all quantitative metrics.

Conclusion: The proposed decoupled HSV LUT approach effectively resolves color coupling problems and provides superior purple flare removal compared to traditional methods.

Abstract: Purple flare, a diffuse chromatic aberration artifact commonly found around highlight areas, severely degrades the tone transition and color of the image. Existing traditional methods are based on hand-crafted features, which lack flexibility and rely entirely on fixed priors, while the scarcity of paired training data critically hampers deep learning. To address this issue, we propose a novel network built upon decoupled HSV Look-Up Tables (LUTs). The method aims to simplify color correction by adjusting the Hue (H), Saturation (S), and Value (V) components independently. This approach resolves the inherent color coupling problems in traditional methods. Our model adopts a two-stage architecture: First, a Chroma-Aware Spectral Tokenizer (CAST) converts the input image from RGB space to HSV space and independently encodes the Hue (H) and Value (V) channels into a set of semantic tokens describing the Purple flare status; second, the HSV-LUT module takes these tokens as input and dynamically generates independent correction curves (1D-LUTs) for the three channels H, S, and V. To effectively train and validate our model, we built the first large-scale purple flare dataset with diverse scenes. We also proposed new metrics and a loss function specifically designed for this task. Extensive experiments demonstrate that our model not only significantly outperforms existing methods in visual effects but also achieves state-of-the-art performance on all quantitative metrics.

Method: Uses LiDAR-IMU Odometry for prior camera poses in bundle adjustment, and introduces normal vector constraints with effective rank regularization for Gaussian primitives optimization.

Result: Achieved pose optimization in one-third the time while maintaining accuracy, and significantly outperformed conventional 3DGS in scene representation, especially on weak/repetitive texture datasets.

Conclusion: The approach enhances both pose estimation robustness and scene representation quality for 3DGS in challenging outdoor environments with texture limitations.

Abstract: 3D Gaussian Splatting (3DGS) has emerged as a key rendering pipeline for digital asset creation due to its balance between efficiency and visual quality. To address the issues of unstable pose estimation and scene representation distortion caused by geometric texture inconsistency in large outdoor scenes with weak or repetitive textures, we approach the problem from two aspects: pose estimation and scene representation. For pose estimation, we leverage LiDAR-IMU Odometry to provide prior poses for cameras in large-scale environments. These prior pose constraints are incorporated into COLMAP’s triangulation process, with pose optimization performed via bundle adjustment. Ensuring consistency between pixel data association and prior poses helps maintain both robustness and accuracy. For scene representation, we introduce normal vector constraints and effective rank regularization to enforce consistency in the direction and shape of Gaussian primitives. These constraints are jointly optimized with the existing photometric loss to enhance the map quality. We evaluate our approach using both public and self-collected datasets. In terms of pose optimization, our method requires only one-third of the time while maintaining accuracy and robustness across both datasets. In terms of scene representation, the results show that our method significantly outperforms conventional 3DGS pipelines. Notably, on self-collected datasets characterized by weak or repetitive textures, our approach demonstrates enhanced visualization capabilities and achieves superior overall performance. Codes and data will be publicly available at https://github.com/justinyeah/normal_shape.git.

Method: Proposes TiS-TSL with a unified model operating in three modes (IP, FVP, BVP) and a two-stage learning strategy: I2V transfers sparse image knowledge to temporal modeling, and V2V refines predictions using bidirectional spatio-temporal consistency to filter noisy labels and enforce temporal coherence.

Result: Experimental results show TiS-TSL outperforms image-based state-of-the-art methods by improving TEPE and EPE by at least 2.11% and 4.54% respectively on two public datasets.

Conclusion: TiS-TSL effectively addresses temporal inconsistency in surgical stereo matching through unified temporal modeling and bidirectional consistency, achieving superior performance with minimal supervision.

Abstract: Stereo matching in minimally invasive surgery (MIS) is essential for next-generation navigation and augmented reality. Yet, dense disparity supervision is nearly impossible due to anatomical constraints, typically limiting annotations to only a few image-level labels acquired before the endoscope enters deep body cavities. Teacher-Student Learning (TSL) offers a promising solution by leveraging a teacher trained on sparse labels to generate pseudo labels and associated confidence maps from abundant unlabeled surgical videos. However, existing TSL methods are confined to image-level supervision, providing only spatial confidence and lacking temporal consistency estimation. This absence of spatio-temporal reliability results in unstable disparity predictions and severe flickering artifacts across video frames. To overcome these challenges, we propose TiS-TSL, a novel time-switchable teacher-student learning framework for video stereo matching under minimal supervision. At its core is a unified model that operates in three distinct modes: Image-Prediction (IP), Forward Video-Prediction (FVP), and Backward Video-Prediction (BVP), enabling flexible temporal modeling within a single architecture. Enabled by this unified model, TiS-TSL adopts a two-stage learning strategy. The Image-to-Video (I2V) stage transfers sparse image-level knowledge to initialize temporal modeling. The subsequent Video-to-Video (V2V) stage refines temporal disparity predictions by comparing forward and backward predictions to calculate bidirectional spatio-temporal consistency. This consistency identifies unreliable regions across frames, filters noisy video-level pseudo labels, and enforces temporal coherence. Experimental results on two public datasets demonstrate that TiS-TSL exceeds other image-based state-of-the-arts by improving TEPE and EPE by at least 2.11% and 4.54%, respectively..

Method: Uses iNGP reconstruction as geometric proxy for depth estimation, identifies high-error pixels, inserts new Gaussians along viewing cones at predicted depths, applies opacity penalty to remove redundant Gaussians, and uses primitive budgeting strategy.

Result: Consistently enhances reconstruction quality and rendering performance across Gaussian budgets, with strong gains under tight primitive constraints where efficient placement is crucial.

Conclusion: ConeGS provides an effective framework for improving 3DGS by enabling more efficient Gaussian placement through image-space-informed densification independent of existing scene geometry.

Abstract: 3D Gaussian Splatting (3DGS) achieves state-of-the-art image quality and real-time performance in novel view synthesis but often suffers from a suboptimal spatial distribution of primitives. This issue stems from cloning-based densification, which propagates Gaussians along existing geometry, limiting exploration and requiring many primitives to adequately cover the scene. We present ConeGS, an image-space-informed densification framework that is independent of existing scene geometry state. ConeGS first creates a fast Instant Neural Graphics Primitives (iNGP) reconstruction as a geometric proxy to estimate per-pixel depth. During the subsequent 3DGS optimization, it identifies high-error pixels and inserts new Gaussians along the corresponding viewing cones at the predicted depth values, initializing their size according to the cone diameter. A pre-activation opacity penalty rapidly removes redundant Gaussians, while a primitive budgeting strategy controls the total number of primitives, either by a fixed budget or by adapting to scene complexity, ensuring high reconstruction quality. Experiments show that ConeGS consistently enhances reconstruction quality and rendering performance across Gaussian budgets, with especially strong gains under tight primitive constraints where efficient placement is crucial.

Method: Proposes NeuroBridge with two key components: Cognitive Prior Augmentation (CPA) simulates perceptual variability through asymmetric, modality-specific transformations on EEG signals and images; Shared Semantic Projector (SSP) establishes bidirectional alignment using a co-adaptive strategy to map features from both modalities into a shared semantic space.

Result: NeuroBridge outperforms previous state-of-the-art methods in both intra-subject and inter-subject settings. In intra-subject scenario, achieves 12.3% improvement in top-1 accuracy (63.2%) and 10.2% improvement in top-5 accuracy (89.9%) on 200-way zero-shot retrieval task.

Conclusion: The proposed NeuroBridge framework demonstrates effectiveness, robustness, and scalability for neural visual decoding, providing a promising approach for cross-modality alignment between brain activity and visual stimuli.

Abstract: Visual neural decoding seeks to reconstruct or infer perceived visual stimuli from brain activity patterns, providing critical insights into human cognition and enabling transformative applications in brain-computer interfaces and artificial intelligence. Current approaches, however, remain constrained by the scarcity of high-quality stimulus-brain response pairs and the inherent semantic mismatch between neural representations and visual content. Inspired by perceptual variability and co-adaptive strategy of the biological systems, we propose a novel self-supervised architecture, named NeuroBridge, which integrates Cognitive Prior Augmentation (CPA) with Shared Semantic Projector (SSP) to promote effective cross-modality alignment. Specifically, CPA simulates perceptual variability by applying asymmetric, modality-specific transformations to both EEG signals and images, enhancing semantic diversity. Unlike previous approaches, SSP establishes a bidirectional alignment process through a co-adaptive strategy, which mutually aligns features from two modalities into a shared semantic space for effective cross-modal learning. NeuroBridge surpasses previous state-of-the-art methods under both intra-subject and inter-subject settings. In the intra-subject scenario, it achieves the improvements of 12.3% in top-1 accuracy and 10.2% in top-5 accuracy, reaching 63.2% and 89.9% respectively on a 200-way zero-shot retrieval task. Extensive experiments demonstrate the effectiveness, robustness, and scalability of the proposed framework for neural visual decoding.

Method: Proposes a MAP estimation framework with a generalized Gaussian scale mixture-based loss (ℓ_q-norm fidelity term) for various noise distributions, solved via iteratively reweighted least squares (IRLS) with diffusion-based denoisers as proximal operators.

Result: Experimental results show the method effectively removes non-Gaussian impulse noise and achieves superior restoration performance on benchmark datasets.

Conclusion: The proposed framework successfully extends plug-and-play image restoration to handle general noise types beyond Gaussian noise, demonstrating robust performance for impulse noise removal.

Abstract: Existing plug-and-play image restoration methods typically employ off-the-shelf Gaussian denoisers as proximal operators within classical optimization frameworks based on variable splitting. Recently, denoisers induced by generative priors have been successfully integrated into regularized optimization methods for image restoration under Gaussian noise. However, their application to non-Gaussian noise–such as impulse noise–remains largely unexplored. In this paper, we propose a plug-and-play image restoration framework based on generative diffusion priors for robust removal of general noise types, including impulse noise. Within the maximum a posteriori (MAP) estimation framework, the data fidelity term is adapted to the specific noise model. Departing from the conventional least-squares loss used for Gaussian noise, we introduce a generalized Gaussian scale mixture-based loss, which approximates a wide range of noise distributions and leads to an $\ell_q$-norm ($0<q\leq2$) fidelity term. This optimization problem is addressed using an iteratively reweighted least squares (IRLS) approach, wherein the proximal step involving the generative prior is efficiently performed via a diffusion-based denoiser. Experimental results on benchmark datasets demonstrate that the proposed method effectively removes non-Gaussian impulse noise and achieves superior restoration performance.

Method: Developed unified multi-distance subjective quality assessment method and constructed MUGSQA dataset considering multiple input uncertainties (quantity/resolution of views, view distance, point cloud accuracy). Created two benchmarks for reconstruction robustness and quality metrics evaluation.

Result: Created comprehensive dataset and benchmarks for evaluating Gaussian Splatting methods under various input uncertainties, enabling better assessment of perceptual quality and reconstruction robustness.

Conclusion: The proposed assessment method and MUGSQA dataset address the challenge of evaluating GS-based 3D reconstruction quality, providing tools to assess both reconstruction methods and quality metrics under realistic uncertainties.

Abstract: Gaussian Splatting (GS) has recently emerged as a promising technique for 3D object reconstruction, delivering high-quality rendering results with significantly improved reconstruction speed. As variants continue to appear, assessing the perceptual quality of 3D objects reconstructed with different GS-based methods remains an open challenge. To address this issue, we first propose a unified multi-distance subjective quality assessment method that closely mimics human viewing behavior for objects reconstructed with GS-based methods in actual applications, thereby better collecting perceptual experiences. Based on it, we also construct a novel GS quality assessment dataset named MUGSQA, which is constructed considering multiple uncertainties of the input data. These uncertainties include the quantity and resolution of input views, the view distance, and the accuracy of the initial point cloud. Moreover, we construct two benchmarks: one to evaluate the robustness of various GS-based reconstruction methods under multiple uncertainties, and the other to evaluate the performance of existing quality assessment metrics. Our dataset and benchmark code will be released soon.

Method: 1) Optical flow-guided temporal module (OFT) decouples motion from appearance using facial optical flow. 2) Audio-to-Intensity (A2I) model transforms audio and facial velocity into frame-wise intensity for joint audio-visual modeling. 3) Diffusion noise initialization strategy (IC-Init) enforces background coherence and motion continuity constraints during inference.

Result: Extensive experiments show ConsistTalk significantly outperforms prior methods in reducing flicker, preserving identity, and delivering temporally stable, high-fidelity talking head videos with better audio-visual synchronization.

Conclusion: ConsistTalk achieves superior performance in talking head generation by addressing key limitations through decoupled motion-appearance modeling, intensity-based audio-visual synchronization, and constrained diffusion inference.

Abstract: Recent advancements in video diffusion models have significantly enhanced audio-driven portrait animation. However, current methods still suffer from flickering, identity drift, and poor audio-visual synchronization. These issues primarily stem from entangled appearance-motion representations and unstable inference strategies. In this paper, we introduce \textbf{ConsistTalk}, a novel intensity-controllable and temporally consistent talking head generation framework with diffusion noise search inference. First, we propose \textbf{an optical flow-guided temporal module (OFT)} that decouples motion features from static appearance by leveraging facial optical flow, thereby reducing visual flicker and improving temporal consistency. Second, we present an \textbf{Audio-to-Intensity (A2I) model} obtained through multimodal teacher-student knowledge distillation. By transforming audio and facial velocity features into a frame-wise intensity sequence, the A2I model enables joint modeling of audio and visual motion, resulting in more natural dynamics. This further enables fine-grained, frame-wise control of motion dynamics while maintaining tight audio-visual synchronization. Third, we introduce a \textbf{diffusion noise initialization strategy (IC-Init)}. By enforcing explicit constraints on background coherence and motion continuity during inference-time noise search, we achieve better identity preservation and refine motion dynamics compared to the current autoregressive strategy. Extensive experiments demonstrate that ConsistTalk significantly outperforms prior methods in reducing flicker, preserving identity, and delivering temporally stable, high-fidelity talking head videos.

Method: Integrates Panoramic Scene Parsing module to extract spatial information from panoramic RGB inputs, and Memory-guided Decision-Making with Dynamic Bounded Memory Queue to incorporate exploration history and avoid local deadlocks.

Result: Significantly outperforms representative baselines on public navigation benchmark in both Success Rate (SR) and Success weighted by Path Length (SPL) metrics.

Conclusion: PanoNav demonstrates that fully RGB-only, mapless zero-shot object navigation is feasible and effective by leveraging panoramic scene parsing and memory-guided decision-making.

Abstract: Zero-shot object navigation (ZSON) in unseen environments remains a challenging problem for household robots, requiring strong perceptual understanding and decision-making capabilities. While recent methods leverage metric maps and Large Language Models (LLMs), they often depend on depth sensors or prebuilt maps, limiting the spatial reasoning ability of Multimodal Large Language Models (MLLMs). Mapless ZSON approaches have emerged to address this, but they typically make short-sighted decisions, leading to local deadlocks due to a lack of historical context. We propose PanoNav, a fully RGB-only, mapless ZSON framework that integrates a Panoramic Scene Parsing module to unlock the spatial parsing potential of MLLMs from panoramic RGB inputs, and a Memory-guided Decision-Making mechanism enhanced by a Dynamic Bounded Memory Queue to incorporate exploration history and avoid local deadlocks. Experiments on the public navigation benchmark show that PanoNav significantly outperforms representative baselines in both SR and SPL metrics.

Method: Combines IMU data with computer vision: estimates UAV displacement/rotation between images, corrects perspective distortion, computes homography matrix, then uses standard stitching algorithm.

Result: Method outperforms existing feature-based algorithms in accuracy and reliability, particularly in challenging scenarios with large displacements, rotations, and camera pose variations.

Conclusion: Proposed approach effectively leverages IMU data to enhance image stitching, is robust in difficult conditions, and can be easily integrated into existing UAV workflows.

Abstract: Unmanned Aerial Vehicles (UAVs) are widely used for aerial photography and remote sensing applications. One of the main challenges is to stitch together multiple images into a single high-resolution image that covers a large area. Featurebased image stitching algorithms are commonly used but can suffer from errors and ambiguities in feature detection and matching. To address this, several approaches have been proposed, including using bundle adjustment techniques or direct image alignment. In this paper, we present a novel method that uses a combination of IMU data and computer vision techniques for stitching images captured by a UAV. Our method involves several steps such as estimating the displacement and rotation of the UAV between consecutive images, correcting for perspective distortion, and computing a homography matrix. We then use a standard image stitching algorithm to align and blend the images together. Our proposed method leverages the additional information provided by the IMU data, corrects for various sources of distortion, and can be easily integrated into existing UAV workflows. Our experiments demonstrate the effectiveness and robustness of our method, outperforming some of the existing feature-based image stitching algorithms in terms of accuracy and reliability, particularly in challenging scenarios such as large displacements, rotations, and variations in camera pose.

Method: Extends differentiable MPM by incorporating a differentiable collision handling mechanism that allows target objects to interact with complex rigid bodies while maintaining end-to-end optimization.

Result: The framework enables physical property estimation with arbitrarily shaped colliders and can be interfaced with various novel view synthesis methods.

Conclusion: AS-DiffMPM provides a comprehensive framework for system identification from visual observations that handles complex object-environment interactions beyond planar surfaces.

Abstract: System identification involving the geometry, appearance, and physical properties from video observations is a challenging task with applications in robotics and graphics. Recent approaches have relied on fully differentiable Material Point Method (MPM) and rendering for simultaneous optimization of these properties. However, they are limited to simplified object-environment interactions with planar colliders and fail in more challenging scenarios where objects collide with non-planar surfaces. We propose AS-DiffMPM, a differentiable MPM framework that enables physical property estimation with arbitrarily shaped colliders. Our approach extends existing methods by incorporating a differentiable collision handling mechanism, allowing the target object to interact with complex rigid bodies while maintaining end-to-end optimization. We show AS-DiffMPM can be easily interfaced with various novel view synthesis methods as a framework for system identification from visual observations.

Method: Developed a “distillation dynamics” analytical framework combining frequency spectrum analysis, information entropy metrics, and activation magnitude tracking to study ViT information processing patterns.

Result: Revealed ViTs exhibit U-shaped information processing (initial compression followed by expansion) and identified representational paradigm mismatch as the root cause of negative transfer in feature distillation.

Conclusion: Successful knowledge transfer in ViTs requires moving beyond naive feature mimicry to methods that respect fundamental representational constraints, providing essential guidance for effective ViT compression strategies.

Abstract: While feature-based knowledge distillation has proven highly effective for compressing CNNs, these techniques unexpectedly fail when applied to Vision Transformers (ViTs), often performing worse than simple logit-based distillation. We provide the first comprehensive analysis of this phenomenon through a novel analytical framework termed as ``distillation dynamics", combining frequency spectrum analysis, information entropy metrics, and activation magnitude tracking. Our investigation reveals that ViTs exhibit a distinctive U-shaped information processing pattern: initial compression followed by expansion. We identify the root cause of negative transfer in feature distillation: a fundamental representational paradigm mismatch between teacher and student models. Through frequency-domain analysis, we show that teacher models employ distributed, high-dimensional encoding strategies in later layers that smaller student models cannot replicate due to limited channel capacity. This mismatch causes late-layer feature alignment to actively harm student performance. Our findings reveal that successful knowledge transfer in ViTs requires moving beyond naive feature mimicry to methods that respect these fundamental representational constraints, providing essential theoretical guidance for designing effective ViTs compression strategies. All source code and experimental logs are provided in the supplementary material.

Method: Three main components: 1) Data-Hierarchical Inference paradigm that enhances accuracy at data-distribution level and diversity at data-sample level; 2) Gaussian Truncation Representation for better fidelity and truncation reliability; 3) Segmentation Flow Matching for enhanced plausibility through semantic-aware flow transformation.

Result: Outperforms SOTA methods on LIDC and ISIC3 datasets, improving GED by up to 12% and HM-IoU by up to 7.3% compared to advanced methods, while achieving more efficient inference.

Conclusion: ATFM successfully addresses the accuracy-diversity trade-off in ambiguous medical image segmentation through its hierarchical inference paradigm and specialized components, demonstrating superior performance and efficiency over existing methods.

Abstract: A simultaneous enhancement of accuracy and diversity of predictions remains a challenge in ambiguous medical image segmentation (AMIS) due to the inherent trade-offs. While truncated diffusion probabilistic models (TDPMs) hold strong potential with a paradigm optimization, existing TDPMs suffer from entangled accuracy and diversity of predictions with insufficient fidelity and plausibility. To address the aforementioned challenges, we propose Ambiguity-aware Truncated Flow Matching (ATFM), which introduces a novel inference paradigm and dedicated model components. Firstly, we propose Data-Hierarchical Inference, a redefinition of AMIS-specific inference paradigm, which enhances accuracy and diversity at data-distribution and data-sample level, respectively, for an effective disentanglement. Secondly, Gaussian Truncation Representation (GTR) is introduced to enhance both fidelity of predictions and reliability of truncation distribution, by explicitly modeling it as a Gaussian distribution at $T_{\text{trunc}}$ instead of using sampling-based approximations.Thirdly, Segmentation Flow Matching (SFM) is proposed to enhance the plausibility of diverse predictions by extending semantic-aware flow transformation in Flow Matching (FM). Comprehensive evaluations on LIDC and ISIC3 datasets demonstrate that ATFM outperforms SOTA methods and simultaneously achieves a more efficient inference. ATFM improves GED and HM-IoU by up to $12%$ and $7.3%$ compared to advanced methods.

Method: Multi-modal, multi-task uncertainty-aware deep learning framework that predicts four key plant traits (plant height, leaf area, specific leaf area, nitrogen content) from citizen science photos using weak supervision, then aggregates predictions across space to generate global trait distribution maps.

Result: PlantTraitNet consistently outperforms existing trait maps across all evaluated traits when validated against independent vegetation survey data (sPlotOpen) and benchmarked against leading global trait products.

Conclusion: Citizen science imagery combined with computer vision and geospatial AI enables scalable and more accurate global trait mapping, offering a powerful new pathway for ecological research and Earth system modeling.

Abstract: Global plant maps of plant traits, such as leaf nitrogen or plant height, are essential for understanding ecosystem processes, including the carbon and energy cycles of the Earth system. However, existing trait maps remain limited by the high cost and sparse geographic coverage of field-based measurements. Citizen science initiatives offer a largely untapped resource to overcome these limitations, with over 50 million geotagged plant photographs worldwide capturing valuable visual information on plant morphology and physiology. In this study, we introduce PlantTraitNet, a multi-modal, multi-task uncertainty-aware deep learning framework that predictsfour key plant traits (plant height, leaf area, specific leaf area, and nitrogen content) from citizen science photos using weak supervision. By aggregating individual trait predictions across space, we generate global maps of trait distributions. We validate these maps against independent vegetation survey data (sPlotOpen) and benchmark them against leading global trait products. Our results show that PlantTraitNet consistently outperforms existing trait maps across all evaluated traits, demonstrating that citizen science imagery, when integrated with computer vision and geospatial AI, enables not only scalable but also more accurate global trait mapping. This approach offers a powerful new pathway for ecological research and Earth system modeling.

Method: Proposes VAEVQ with three components: (1) Variational Latent Quantization (VLQ) using VAE instead of AE for quantization, (2) Representation Coherence Strategy (RCS) for adaptive alignment modulation, and (3) Distribution Consistency Regularization (DCR) for codebook distribution alignment.

Result: Extensive experiments on benchmark datasets show VAEVQ outperforms state-of-the-art methods in both reconstruction and downstream generation tasks.

Conclusion: VAEVQ effectively addresses key limitations of traditional VQ methods through variational quantization, adaptive alignment, and distribution consistency, achieving superior performance.

Abstract: Vector quantization (VQ) transforms continuous image features into discrete representations, providing compressed, tokenized inputs for generative models. However, VQ-based frameworks suffer from several issues, such as non-smooth latent spaces, weak alignment between representations before and after quantization, and poor coherence between the continuous and discrete domains. These issues lead to unstable codeword learning and underutilized codebooks, ultimately degrading the performance of both reconstruction and downstream generation tasks. To this end, we propose VAEVQ, which comprises three key components: (1) Variational Latent Quantization (VLQ), replacing the AE with a VAE for quantization to leverage its structured and smooth latent space, thereby facilitating more effective codeword activation; (2) Representation Coherence Strategy (RCS), adaptively modulating the alignment strength between pre- and post-quantization features to enhance consistency and prevent overfitting to noise; and (3) Distribution Consistency Regularization (DCR), aligning the entire codebook distribution with the continuous latent distribution to improve utilization. Extensive experiments on two benchmark datasets demonstrate that VAEVQ outperforms state-of-the-art methods.

Method: ALIGN jointly trains a classifier and masker iteratively - the masker produces soft task-relevant masks highlighting informative regions, while the classifier optimizes for both prediction accuracy and alignment between its saliency maps and learned masks.

Result: ALIGN consistently outperforms six strong baselines on VLCS and Terra Incognita benchmarks in both in-distribution and out-of-distribution settings, while also yielding superior explanation quality in terms of sufficiency and comprehensiveness.

Conclusion: ALIGN effectively improves both interpretability and generalizability by leveraging high-quality masks as guidance, producing accurate and interpretable models without relying on external supervision.

Abstract: Explanation-guided learning (EGL) has shown promise in aligning model predictions with interpretable reasoning, particularly in computer vision tasks. However, most approaches rely on external annotations or heuristic-based segmentation to supervise model explanations, which can be noisy, imprecise and difficult to scale. In this work, we provide both empirical and theoretical evidence that low-quality supervision signals can degrade model performance rather than improve it. In response, we propose ALIGN, a novel framework that jointly trains a classifier and a masker in an iterative manner. The masker learns to produce soft, task-relevant masks that highlight informative regions, while the classifier is optimized for both prediction accuracy and alignment between its saliency maps and the learned masks. By leveraging high-quality masks as guidance, ALIGN improves both interpretability and generalizability, showing its superiority across various settings. Experiments on the two domain generalization benchmarks, VLCS and Terra Incognita, show that ALIGN consistently outperforms six strong baselines in both in-distribution and out-of-distribution settings. Besides, ALIGN also yields superior explanation quality concerning sufficiency and comprehensiveness, highlighting its effectiveness in producing accurate and interpretable models.

Method: Train on long structured captions with consistent fine-grained attributes, propose DimFusion for efficient long caption processing, and introduce Text-as-a-Bottleneck Reconstruction (TaBR) evaluation protocol.

Result: FIBO achieves state-of-the-art prompt alignment among open-source models, demonstrating improved controllability and expressiveness through fine-grained attribute control.

Conclusion: Training on long structured captions with consistent attributes enables better controllability and expressiveness in text-to-image generation, addressing limitations of traditional short-prompt models.

Abstract: Text-to-image models have rapidly evolved from casual creative tools to professional-grade systems, achieving unprecedented levels of image quality and realism. Yet, most models are trained to map short prompts into detailed images, creating a gap between sparse textual input and rich visual outputs. This mismatch reduces controllability, as models often fill in missing details arbitrarily, biasing toward average user preferences and limiting precision for professional use. We address this limitation by training the first open-source text-to-image model on long structured captions, where every training sample is annotated with the same set of fine-grained attributes. This design maximizes expressive coverage and enables disentangled control over visual factors. To process long captions efficiently, we propose DimFusion, a fusion mechanism that integrates intermediate tokens from a lightweight LLM without increasing token length. We also introduce the Text-as-a-Bottleneck Reconstruction (TaBR) evaluation protocol. By assessing how well real images can be reconstructed through a captioning-generation loop, TaBR directly measures controllability and expressiveness, even for very long captions where existing evaluation methods fail. Finally, we demonstrate our contributions by training the large-scale model FIBO, achieving state-of-the-art prompt alignment among open-source models. Model weights are publicly available at https://huggingface.co/briaai/FIBO

Method: Uses stochastic gradient descent with three components: feature alignment to reduce modality gaps, score distribution balance, and pixel-guard regularization to optimize multimodal output equilibrium between CLIPscore and image quality.

Result: Achieves significant CLIPscore improvement while preserving visual fidelity; discovers grayscale conversion degrades fooling images; proposes detection method with 91% accuracy using color channel sensitivity analysis.

Conclusion: Establishes practical pathway for feature misalignment in CLIP-based systems and corresponding defense mechanisms, demonstrating both attack vulnerability and effective detection method.

Abstract: The well-aligned attribute of CLIP-based models enables its effective application like CLIPscore as a widely adopted image quality assessment metric. However, such a CLIP-based metric is vulnerable for its delicate multimodal alignment. In this work, we propose \textbf{FoCLIP}, a feature-space misalignment framework for fooling CLIP-based image quality metric. Based on the stochastic gradient descent technique, FoCLIP integrates three key components to construct fooling examples: feature alignment as the core module to reduce image-text modality gaps, the score distribution balance module and pixel-guard regularization, which collectively optimize multimodal output equilibrium between CLIPscore performance and image quality. Such a design can be engineered to maximize the CLIPscore predictions across diverse input prompts, despite exhibiting either visual unrecognizability or semantic incongruence with the corresponding adversarial prompts from human perceptual perspectives. Experiments on ten artistic masterpiece prompts and ImageNet subsets demonstrate that optimized images can achieve significant improvement in CLIPscore while preserving high visual fidelity. In addition, we found that grayscale conversion induces significant feature degradation in fooling images, exhibiting noticeable CLIPscore reduction while preserving statistical consistency with original images. Inspired by this phenomenon, we propose a color channel sensitivity-driven tampering detection mechanism that achieves 91% accuracy on standard benchmarks. In conclusion, this work establishes a practical pathway for feature misalignment in CLIP-based multimodal systems and the corresponding defense method.

Method: Two-stage approach: 1) LLM generates structured layout from object lists with task decomposition (core objects first, then rule-based completion), 2) Layout-conditioned diffusion models (ControlNet vs GLIGEN) synthesize images adhering to layouts after domain-specific finetuning.

Result: Task decomposition improved object recall from 57.2% to 99.9% for complex scenes. ControlNet preserves text-based stylistic control but suffers object hallucination, while GLIGEN provides superior layout fidelity but reduced prompt controllability.

Conclusion: The decoupled two-stage approach successfully generates images with specified object counts and plausible spatial arrangements, demonstrating viability for compositionally controlled synthesis.

Abstract: Text-to-image diffusion models exhibit remarkable generative capabilities, but lack precise control over object counts and spatial arrangements. This work introduces a two-stage system to address these compositional limitations. The first stage employs a Large Language Model (LLM) to generate a structured layout from a list of objects. The second stage uses a layout-conditioned diffusion model to synthesize a photorealistic image adhering to this layout. We find that task decomposition is critical for LLM-based spatial planning; by simplifying the initial generation to core objects and completing the layout with rule-based insertion, we improve object recall from 57.2% to 99.9% for complex scenes. For image synthesis, we compare two leading conditioning methods: ControlNet and GLIGEN. After domain-specific finetuning on table-setting datasets, we identify a key trade-off: ControlNet preserves text-based stylistic control but suffers from object hallucination, while GLIGEN provides superior layout fidelity at the cost of reduced prompt-based controllability. Our end-to-end system successfully generates images with specified object counts and plausible spatial arrangements, demonstrating the viability of a decoupled approach for compositionally controlled synthesis.

Method: Adaptive patch partitioning generates morphology-aware patches aligned with vascular structures, plus Semantic Clustering Attention to aggregate features from semantically similar patches.

Result: Achieves state-of-the-art performance on three datasets (AVT, AortaSeg24, TBAD), with significant improvements in segmenting intricate vascular structures.

Conclusion: MPT effectively preserves vascular structure integrity through adaptive patch partitioning and semantic feature aggregation, outperforming existing methods.

Abstract: Accurate segmentation of aortic vascular structures is critical for diagnosing and treating cardiovascular diseases.Traditional Transformer-based models have shown promise in this domain by capturing long-range dependencies between vascular features. However, their reliance on fixed-size rectangular patches often influences the integrity of complex vascular structures, leading to suboptimal segmentation accuracy. To address this challenge, we propose the adaptive Morph Patch Transformer (MPT), a novel architecture specifically designed for aortic vascular segmentation. Specifically, MPT introduces an adaptive patch partitioning strategy that dynamically generates morphology-aware patches aligned with complex vascular structures. This strategy can preserve semantic integrity of complex vascular structures within individual patches. Moreover, a Semantic Clustering Attention (SCA) method is proposed to dynamically aggregate features from various patches with similar semantic characteristics. This method enhances the model’s capability to segment vessels of varying sizes, preserving the integrity of vascular structures. Extensive experiments on three open-source dataset(AVT, AortaSeg24 and TBAD) demonstrate that MPT achieves state-of-the-art performance, with improvements in segmenting intricate vascular structures.

Method: Introduces TrueCity benchmark with cm-accurate annotated real-world point clouds, semantic 3D city models, and annotated simulated point clouds representing the same city, using segmentation classes aligned with international 3D city modeling standards.

Result: Extensive experiments on common baselines quantify domain shift and highlight strategies for exploiting synthetic data to enhance real-world 3D scene understanding.

Conclusion: TrueCity dataset will foster development of sim-to-real gap quantification and enable generalizable data-driven models for 3D semantic scene understanding.

Abstract: 3D semantic scene understanding remains a long-standing challenge in the 3D computer vision community. One of the key issues pertains to limited real-world annotated data to facilitate generalizable models. The common practice to tackle this issue is to simulate new data. Although synthetic datasets offer scalability and perfect labels, their designer-crafted scenes fail to capture real-world complexity and sensor noise, resulting in a synthetic-to-real domain gap. Moreover, no benchmark provides synchronized real and simulated point clouds for segmentation-oriented domain shift analysis. We introduce TrueCity, the first urban semantic segmentation benchmark with cm-accurate annotated real-world point clouds, semantic 3D city models, and annotated simulated point clouds representing the same city. TrueCity proposes segmentation classes aligned with international 3D city modeling standards, enabling consistent evaluation of synthetic-to-real gap. Our extensive experiments on common baselines quantify domain shift and highlight strategies for exploiting synthetic data to enhance real-world 3D scene understanding. We are convinced that the TrueCity dataset will foster further development of sim-to-real gap quantification and enable generalizable data-driven models. The data, code, and 3D models are available online: https://tum-gis.github.io/TrueCity/

Method: Use polarized laser light illumination and polarization-sensitive camera to capture reflected signals from microplastics, then apply deep convolutional neural network for image-based classification of polymer types.

Result: Achieved 80% mean classification accuracy for three polymer types (HDPE, LDPE, PP), with AOLP signals showing better noise robustness and polyethylene distinction, while DOLP signals excel at polypropylene identification.

Conclusion: The reflection-based polarized light scattering method is effective for in-situ microplastic classification, with complementary strengths between AOLP and DOLP signals for different polymer identification tasks.

Abstract: Facing the critical need for continuous, large-scale microplastic monitoring, which is hindered by the limitations of gold-standard methods in aquatic environments, this paper introduces and validates a novel, reflection-based approach for the in-situ classification and identification of microplastics directly in water bodies, which is based on polarized light scattering. In this experiment, we classify colorless microplastic particles (50-300 $μ$m) by illuminating them with linearly polarized laser light and capturing their reflected signals using a polarization-sensitive camera. This reflection-based technique successfully circumvents the transmission-based interference issues that plague many conventional methods when applied in water. Using a deep convolutional neural network (CNN) for image-based classification, we successfully identified three common polymer types, high-density polyethylene, low-density polyethylene, and polypropylene, achieving a peak mean classification accuracy of 80% on the test dataset. A subsequent feature hierarchy analysis demonstrated that the CNN’s decision-making process relies mainly on the microstructural integrity and internal texture (polarization patterns) of the particle rather than its macroshape. Critically, we found that the Angle of Linear Polarization (AOLP) signal is significantly more robust against contextual noise than the Degree of Linear Polarization (DOLP) signal. While the AOLP-based classification achieved superior overall performance, its strength lies in distinguishing between the two polyethylene plastics, showing a lower confusion rate between high-density and low-density polyethylene. Conversely, the DOLP signal demonstrated slightly worse overall classification results but excels at accurately identifying the polypropylene class, which it isolated with greater success than AOLP.

Method: Proposes Vision-language Match Module (VMM) and Dark-aware Semantic Block (DSB) for text-guided feature extraction, adaptive mask reweighting for penumbra regions, and Tokenized Temporal Block (TTB) for efficient spatiotemporal learning using temporal tokens.

Result: Achieves state-of-the-art accuracy on multiple benchmark datasets with real-time inference efficiency.

Conclusion: The proposed DTTNet effectively addresses shadow-background ambiguity and temporal modeling challenges through vision-language fusion and efficient token-based temporal encoding.

Abstract: Video shadow detection confronts two entwined difficulties: distinguishing shadows from complex backgrounds and modeling dynamic shadow deformations under varying illumination. To address shadow-background ambiguity, we leverage linguistic priors through the proposed Vision-language Match Module (VMM) and a Dark-aware Semantic Block (DSB), extracting text-guided features to explicitly differentiate shadows from dark objects. Furthermore, we introduce adaptive mask reweighting to downweight penumbra regions during training and apply edge masks at the final decoder stage for better supervision. For temporal modeling of variable shadow shapes, we propose a Tokenized Temporal Block (TTB) that decouples spatiotemporal learning. TTB summarizes cross-frame shadow semantics into learnable temporal tokens, enabling efficient sequence encoding with minimal computation overhead. Comprehensive Experiments on multiple benchmark datasets demonstrate state-of-the-art accuracy and real-time inference efficiency. Codes are available at https://github.com/city-cheng/DTTNet.

Method: Proposed Physics-aware Attenuation Correction Diffusion Model (PADM) that incorporates explicit physics priors via teacher-student distillation mechanism. Uses only NAC input while benefiting from physics-informed supervision during training.

Result: PADM outperforms state-of-the-art generative models, delivering superior reconstruction fidelity across both quantitative metrics and visual assessment. Introduced CardiAC dataset with 424 patient studies.

Conclusion: PADM provides an effective CT-free solution for attenuation correction in cardiac SPECT, addressing cost and accessibility limitations of hybrid SPECT/CT systems while maintaining high reconstruction quality.

Abstract: Attenuation artifacts remain a significant challenge in cardiac Myocardial Perfusion Imaging (MPI) using Single-Photon Emission Computed Tomography (SPECT), often compromising diagnostic accuracy and reducing clinical interpretability. While hybrid SPECT/CT systems mitigate these artifacts through CT-derived attenuation maps, their high cost, limited accessibility, and added radiation exposure hinder widespread clinical adoption. In this study, we propose a novel CT-free solution to attenuation correction in cardiac SPECT. Specifically, we introduce Physics-aware Attenuation Correction Diffusion Model (PADM), a diffusion-based generative method that incorporates explicit physics priors via a teacher–student distillation mechanism. This approach enables attenuation artifact correction using only Non-Attenuation-Corrected (NAC) input, while still benefiting from physics-informed supervision during training. To support this work, we also introduce CardiAC, a comprehensive dataset comprising 424 patient studies with paired NAC and Attenuation-Corrected (AC) reconstructions, alongside high-resolution CT-based attenuation maps. Extensive experiments demonstrate that PADM outperforms state-of-the-art generative models, delivering superior reconstruction fidelity across both quantitative metrics and visual assessment.

Method: Proposes GFix with a streamlined single-step diffusion model as neural enhancer and a modulated LoRA scheme that freezes low-rank decompositions and modulates hidden states for efficient adaptation.

Result: GFix outperforms GSVC with up to 72.1% BD-rate savings in LPIPS and 21.4% in FID, delivering strong perceptual quality enhancement.

Conclusion: The proposed GFix framework effectively enhances perceptual quality in 3DGS-based video compression through diffusion-based enhancement and efficient adaptation techniques.

Abstract: 3D Gaussian Splatting (3DGS) enhances 3D scene reconstruction through explicit representation and fast rendering, demonstrating potential benefits for various low-level vision tasks, including video compression. However, existing 3DGS-based video codecs generally exhibit more noticeable visual artifacts and relatively low compression ratios. In this paper, we specifically target the perceptual enhancement of 3DGS-based video compression, based on the assumption that artifacts from 3DGS rendering and quantization resemble noisy latents sampled during diffusion training. Building on this premise, we propose a content-adaptive framework, GFix, comprising a streamlined, single-step diffusion model that serves as an off-the-shelf neural enhancer. Moreover, to increase compression efficiency, We propose a modulated LoRA scheme that freezes the low-rank decompositions and modulates the intermediate hidden states, thereby achieving efficient adaptation of the diffusion backbone with highly compressible updates. Experimental results show that GFix delivers strong perceptual quality enhancement, outperforming GSVC with up to 72.1% BD-rate savings in LPIPS and 21.4% in FID.

Method: Created Pandar128 dataset with full sensor calibration and odometry; developed SimpleLidarLane method using BEV segmentation, clustering, and polyline fitting; proposed IAM-F1 metric for polyline-based evaluation.

Result: Dataset captured in diverse German conditions; baseline method achieves strong performance despite simplicity; new metric enables principled evaluation.

Conclusion: High-quality data, modular pipelines, and standardized evaluation can compete with complex approaches in LiDAR lane detection.

Abstract: We present Pandar128, the largest public dataset for lane line detection using a 128-beam LiDAR. It contains over 52,000 camera frames and 34,000 LiDAR scans, captured in diverse real-world conditions in Germany. The dataset includes full sensor calibration (intrinsics, extrinsics) and synchronized odometry, supporting tasks such as projection, fusion, and temporal modeling. To complement the dataset, we also introduce SimpleLidarLane, a light-weight baseline method for lane line reconstruction that combines BEV segmentation, clustering, and polyline fitting. Despite its simplicity, our method achieves strong performance under challenging various conditions (e.g., rain, sparse returns), showing that modular pipelines paired with high-quality data and principled evaluation can compete with more complex approaches. Furthermore, to address the lack of standardized evaluation, we propose a novel polyline-based metric - Interpolation-Aware Matching F1 (IAM-F1) - that employs interpolation-aware lateral matching in BEV space. All data and code are publicly released to support reproducibility in LiDAR-based lane detection.

Method: A two-step coarse-to-fine process: wavelet-based screening at low magnification to locate structurally rich regions, followed by high-resolution extraction for detailed modeling, mirroring pathologists’ diagnostic workflow.

Result: Evaluations across multiple cancer datasets (lung, renal, colorectal) show WISE-MAE achieves competitive representation quality and downstream classification performance while maintaining efficiency under weak supervision.

Conclusion: The wavelet-informed patch selection strategy effectively brings structure and biological relevance into MAE-based learning for digital pathology, improving learned representations while maintaining computational efficiency.

Abstract: Whole-slide images are central to digital pathology, yet their extreme size and scarce annotations make self-supervised learning essential. Masked Autoencoders (MAEs) with Vision Transformer backbones have recently shown strong potential for histopathology representation learning. However, conventional random patch sampling during MAE pretraining often includes irrelevant or noisy regions, limiting the model’s ability to capture meaningful tissue patterns. In this paper, we present a lightweight and domain-adapted framework that brings structure and biological relevance into MAE-based learning through a wavelet-informed patch selection strategy. WISE-MAE applies a two-step coarse-to-fine process: wavelet-based screening at low magnification to locate structurally rich regions, followed by high-resolution extraction for detailed modeling. This approach mirrors the diagnostic workflow of pathologists and improves the quality of learned representations. Evaluations across multiple cancer datasets, including lung, renal, and colorectal tissues, show that WISE-MAE achieves competitive representation quality and downstream classification performance while maintaining efficiency under weak supervision.

Method: Employ state-of-the-art techniques for segmenting and describing entire manuscript folios.

Result: Creation of richer training data specifically for medieval visual content analysis.

Conclusion: Enhanced training data will improve computer vision techniques like instance segmentation and multimodal models for medieval manuscripts.

Abstract: We aim to theorize the medieval manuscript page and its contents more holistically, using state-of-the-art techniques to segment and describe the entire manuscript folio, for the purpose of creating richer training data for computer vision techniques, namely instance segmentation, and multimodal models for medieval-specific visual content.

Method: Developed a bias adherence score to quantify bias activation in object-attribute bindings, and created a training-free context-bias control framework using token decoupling to debias semantic bindings.

Result: The framework achieved over 10% debiasing improvement in compositional generation tasks, revealing that bias distribution can be amplified through contextual associations between objects and attributes.

Conclusion: Current debiasing approaches have critical limitations when applied to semantically bound contexts, requiring reassessment of bias mitigation strategies to reduce bias without disrupting essential semantic relationships.

Abstract: Text-to-image generative models often exhibit bias related to sensitive attributes. However, current research tends to focus narrowly on single-object prompts with limited contextual diversity. In reality, each object or attribute within a prompt can contribute to bias. For example, the prompt “an assistant wearing a pink hat” may reflect female-inclined biases associated with a pink hat. The neglected joint effects of the semantic binding in the prompts cause significant failures in current debiasing approaches. This work initiates a preliminary investigation on how bias manifests under semantic binding, where contextual associations between objects and attributes influence generative outcomes. We demonstrate that the underlying bias distribution can be amplified based on these associations. Therefore, we introduce a bias adherence score that quantifies how specific object-attribute bindings activate bias. To delve deeper, we develop a training-free context-bias control framework to explore how token decoupling can facilitate the debiasing of semantic bindings. This framework achieves over 10% debiasing improvement in compositional generation tasks. Our analysis of bias scores across various attribute-object bindings and token decorrelation highlights a fundamental challenge: reducing bias without disrupting essential semantic relationships. These findings expose critical limitations in current debiasing approaches when applied to semantically bound contexts, underscoring the need to reassess prevailing bias mitigation strategies.

Method: A simple yet effective two-stage detection method that achieves high performance on contemporary deepfakes.

Result: Achieves 99.8% AUROC on current deepfakes but suffers 30% recall drop when tested on deepfakes from just six months later, showing significant performance decay as threats evolve.

Conclusion: Robust deepfake detection requires ongoing curation of large, diverse datasets and development of advanced frame-level feature detectors, as predictive power comes primarily from static frame-level artifacts rather than temporal inconsistencies.

Abstract: The continually advancing quality of deepfake technology exacerbates the threats of disinformation, fraud, and harassment by making maliciously-generated synthetic content increasingly difficult to distinguish from reality. We introduce a simple yet effective two-stage detection method that achieves an AUROC of over 99.8% on contemporary deepfakes. However, this high performance is short-lived. We show that models trained on this data suffer a recall drop of over 30% when evaluated on deepfakes created with generation techniques from just six months later, demonstrating significant decay as threats evolve. Our analysis reveals two key insights for robust detection. Firstly, continued performance requires the ongoing curation of large, diverse datasets. Second, predictive power comes primarily from static, frame-level artifacts, not temporal inconsistencies. The future of effective deepfake detection therefore depends on rapid data collection and the development of advanced frame-level feature detectors.

Method: Transforms point clouds via graph Fourier transform, applies frequency-aware subsampling based on spectral similarity to generate multiple sub-point clouds, then uses majority voting of independent classifications.

Result: Achieves higher certified accuracy and empirical accuracy under strong perturbations on ModelNet40 and ScanObjectNN datasets compared to existing methods.

Conclusion: Spectral representations provide an effective pathway for certifiable robustness in 3D point cloud recognition, with FreqCert establishing theoretical foundations for frequency domain certification.

Abstract: 3D point cloud classification is a fundamental task in safety-critical applications such as autonomous driving, robotics, and augmented reality. However, recent studies reveal that point cloud classifiers are vulnerable to structured adversarial perturbations and geometric corruptions, posing risks to their deployment in safety-critical scenarios. Existing certified defenses limit point-wise perturbations but overlook subtle geometric distortions that preserve individual points yet alter the overall structure, potentially leading to misclassification. In this work, we propose FreqCert, a novel certification framework that departs from conventional spatial domain defenses by shifting robustness analysis to the frequency domain, enabling structured certification against global L2-bounded perturbations. FreqCert first transforms the input point cloud via the graph Fourier transform (GFT), then applies structured frequency-aware subsampling to generate multiple sub-point clouds. Each sub-cloud is independently classified by a standard model, and the final prediction is obtained through majority voting, where sub-clouds are constructed based on spectral similarity rather than spatial proximity, making the partitioning more stable under L2 perturbations and better aligned with the object’s intrinsic structure. We derive a closed-form lower bound on the certified L2 robustness radius and prove its tightness under minimal and interpretable assumptions, establishing a theoretical foundation for frequency domain certification. Extensive experiments on the ModelNet40 and ScanObjectNN datasets demonstrate that FreqCert consistently achieves higher certified accuracy and empirical accuracy under strong perturbations. Our results suggest that spectral representations provide an effective pathway toward certifiable robustness in 3D point cloud recognition.

Method: Uses wavelet-based encoder-decoder for efficient compression, geometry-enhanced diffusion process to preserve geometric features, and multi-level loss function for stable convergence.

Result: Achieves state-of-the-art results across fidelity, spectral accuracy, visual realism, and clarity metrics for 4x hyperspectral image super-resolution.

Conclusion: GEWDiff effectively addresses key challenges in hyperspectral image generation through wavelet compression, geometric enhancement, and multi-level loss optimization.

Abstract: Improving the quality of hyperspectral images (HSIs), such as through super-resolution, is a crucial research area. However, generative modeling for HSIs presents several challenges. Due to their high spectral dimensionality, HSIs are too memory-intensive for direct input into conventional diffusion models. Furthermore, general generative models lack an understanding of the topological and geometric structures of ground objects in remote sensing imagery. In addition, most diffusion models optimize loss functions at the noise level, leading to a non-intuitive convergence behavior and suboptimal generation quality for complex data. To address these challenges, we propose a Geometric Enhanced Wavelet-based Diffusion Model (GEWDiff), a novel framework for reconstructing hyperspectral images at 4-times super-resolution. A wavelet-based encoder-decoder is introduced that efficiently compresses HSIs into a latent space while preserving spectral-spatial information. To avoid distortion during generation, we incorporate a geometry-enhanced diffusion process that preserves the geometric features. Furthermore, a multi-level loss function was designed to guide the diffusion process, promoting stable convergence and improved reconstruction fidelity. Our model demonstrated state-of-the-art results across multiple dimensions, including fidelity, spectral accuracy, visual realism, and clarity.

Method: Leverages Neural Collapse mechanism with ETF-aligned classification module and adaptive training framework including representation-balanced learning (RBL) and dynamic feature direction loss (FDL) to address class imbalance and geometric similarities in 3D data.

Result: Significantly improves model robustness - DGCNN’s classification accuracy increased from 27.2% to 80.9% on ModelNet40 (53.7% absolute gain), surpassing leading baselines by 34.0%. Works effectively across various model structures on multiple datasets.

Conclusion: 3D-ANC successfully creates disentangled feature spaces that enhance adversarial robustness in 3D point cloud recognition, overcoming challenges of class imbalance and complex geometric relationships through Neural Collapse-based approach.

Abstract: Deep neural networks have recently achieved notable progress in 3D point cloud recognition, yet their vulnerability to adversarial perturbations poses critical security challenges in practical deployments. Conventional defense mechanisms struggle to address the evolving landscape of multifaceted attack patterns. Through systematic analysis of existing defenses, we identify that their unsatisfactory performance primarily originates from an entangled feature space, where adversarial attacks can be performed easily. To this end, we present 3D-ANC, a novel approach that capitalizes on the Neural Collapse (NC) mechanism to orchestrate discriminative feature learning. In particular, NC depicts where last-layer features and classifier weights jointly evolve into a simplex equiangular tight frame (ETF) arrangement, establishing maximally separable class prototypes. However, leveraging this advantage in 3D recognition confronts two substantial challenges: (1) prevalent class imbalance in point cloud datasets, and (2) complex geometric similarities between object categories. To tackle these obstacles, our solution combines an ETF-aligned classification module with an adaptive training framework consisting of representation-balanced learning (RBL) and dynamic feature direction loss (FDL). 3D-ANC seamlessly empowers existing models to develop disentangled feature spaces despite the complexity in 3D data distribution. Comprehensive evaluations state that 3D-ANC significantly improves the robustness of models with various structures on two datasets. For instance, DGCNN’s classification accuracy is elevated from 27.2% to 80.9% on ModelNet40 – a 53.7% absolute gain that surpasses leading baselines by 34.0%.

Method: TVA uses bidirectional contrastive learning to maximize feature discrepancy and temporal consistency loss with motion cues to enhance sequential perturbation impact.

Result: Extensive experiments across 24 video tasks show TVA effectively attacks downstream models and MLLMs, revealing significant security vulnerabilities.

Conclusion: TVA demonstrates practical security threats in video model deployment, enabling efficient attacks without expensive surrogate models or domain-specific data access.

Abstract: Large-scale Video Foundation Models (VFMs) has significantly advanced various video-related tasks, either through task-specific models or Multi-modal Large Language Models (MLLMs). However, the open accessibility of VFMs also introduces critical security risks, as adversaries can exploit full knowledge of the VFMs to launch potent attacks. This paper investigates a novel and practical adversarial threat scenario: attacking downstream models or MLLMs fine-tuned from open-source VFMs, without requiring access to the victim task, training data, model query, and architecture. In contrast to conventional transfer-based attacks that rely on task-aligned surrogate models, we demonstrate that adversarial vulnerabilities can be exploited directly from the VFMs. To this end, we propose the Transferable Video Attack (TVA), a temporal-aware adversarial attack method that leverages the temporal representation dynamics of VFMs to craft effective perturbations. TVA integrates a bidirectional contrastive learning mechanism to maximize the discrepancy between the clean and adversarial features, and introduces a temporal consistency loss that exploits motion cues to enhance the sequential impact of perturbations. TVA avoids the need to train expensive surrogate models or access to domain-specific data, thereby offering a more practical and efficient attack strategy. Extensive experiments across 24 video-related tasks demonstrate the efficacy of TVA against downstream models and MLLMs, revealing a previously underexplored security vulnerability in the deployment of video models.

Method: Compared three federated strategies on realistic non-IID splits: zero-shot inference with pretrained VLMs, LoRA-based fine-tuning of LLaVA-NeXT-Video-7B, and personalized federated learning of a 65.8M-parameter 3D CNN. Also used hierarchical category grouping for VLM multiclass accuracy improvement.

Result: All methods exceeded 90% accuracy in binary violence detection. 3D CNN achieved superior calibration (ROC AUC 92.59%) at roughly half the energy cost (240 Wh vs. 570 Wh) of federated LoRA. Hierarchical category grouping boosted VLM multiclass accuracy from 65.31% to 81% on UCF-Crime dataset.

Conclusion: Hybrid deployment strategies are recommended: default to efficient CNNs for routine inference and selectively engage VLMs for complex contextual reasoning, balancing energy efficiency with multimodal reasoning capabilities.

Abstract: Deep learning-based video surveillance increasingly demands privacy-preserving architectures with low computational and environmental overhead. Federated learning preserves privacy but deploying large vision-language models (VLMs) introduces major energy and sustainability challenges. We compare three strategies for federated violence detection under realistic non-IID splits on the RWF-2000 and RLVS datasets: zero-shot inference with pretrained VLMs, LoRA-based fine-tuning of LLaVA-NeXT-Video-7B, and personalized federated learning of a 65.8M-parameter 3D CNN. All methods exceed 90% accuracy in binary violence detection. The 3D CNN achieves superior calibration (ROC AUC 92.59%) at roughly half the energy cost (240 Wh vs. 570 Wh) of federated LoRA, while VLMs provide richer multimodal reasoning. Hierarchical category grouping (based on semantic similarity and class exclusion) boosts VLM multiclass accuracy from 65.31% to 81% on the UCF-Crime dataset. To our knowledge, this is the first comparative simulation study of LoRA-tuned VLMs and personalized CNNs for federated violence detection, with explicit energy and CO2e quantification. Our results inform hybrid deployment strategies that default to efficient CNNs for routine inference and selectively engage VLMs for complex contextual reasoning.

Method: Proposes CRDA framework with: 1) Configurable pool of forgery operations, 2) RL agent that dynamically selects augmentation actions based on detector performance, 3) Causal inference to suppress spurious correlations and focus on causally invariant features, 4) Progressive curriculum from simple to complex forgeries.

Result: Extensive experiments show CRDA significantly improves detector generalizability, outperforming state-of-the-art methods across multiple cross-domain datasets.

Conclusion: Dynamic, curriculum-based augmentation guided by RL and causal inference is more effective than static approaches for training robust deepfake detectors that generalize well to unseen forgery types.

Abstract: The generalization capability of deepfake detectors is critical for real-world use. Data augmentation via synthetic fake face generation effectively enhances generalization, yet current SoTA methods rely on fixed strategies-raising a key question: Is a single static augmentation sufficient, or does the diversity of forgery features demand dynamic approaches? We argue existing methods overlook the evolving complexity of real-world forgeries (e.g., facial warping, expression manipulation), which fixed policies cannot fully simulate. To address this, we propose CRDA (Curriculum Reinforcement-Learning Data Augmentation), a novel framework guiding detectors to progressively master multi-domain forgery features from simple to complex. CRDA synthesizes augmented samples via a configurable pool of forgery operations and dynamically generates adversarial samples tailored to the detector’s current learning state. Central to our approach is integrating reinforcement learning (RL) and causal inference. An RL agent dynamically selects augmentation actions based on detector performance to efficiently explore the vast augmentation space, adapting to increasingly challenging forgeries. Simultaneously, the agent introduces action space variations to generate heterogeneous forgery patterns, guided by causal inference to mitigate spurious correlations-suppressing task-irrelevant biases and focusing on causally invariant features. This integration ensures robust generalization by decoupling synthetic augmentation patterns from the model’s learned representations. Extensive experiments show our method significantly improves detector generalizability, outperforming SOTA methods across multiple cross-domain datasets.

Method: Integrates scene-level frustum-based LiDAR autoencoding, order-invariant latent representations, and direct radar spectrum conditioning to create a compact and expressive generation process using latent diffusion models.

Result: Experiments show RaLD produces dense and accurate 3D point clouds from raw radar spectrums, enabling robust perception in challenging environments.

Conclusion: RaLD offers a promising solution for generating high-quality 3D point clouds from sparse radar data, addressing limitations of current radar-based 3D perception methods.

Abstract: Millimeter-wave radar offers a promising sensing modality for autonomous systems thanks to its robustness in adverse conditions and low cost. However, its utility is significantly limited by the sparsity and low resolution of radar point clouds, which poses challenges for tasks requiring dense and accurate 3D perception. Despite that recent efforts have shown great potential by exploring generative approaches to address this issue, they often rely on dense voxel representations that are inefficient and struggle to preserve structural detail. To fill this gap, we make the key observation that latent diffusion models (LDMs), though successful in other modalities, have not been effectively leveraged for radar-based 3D generation due to a lack of compatible representations and conditioning strategies. We introduce RaLD, a framework that bridges this gap by integrating scene-level frustum-based LiDAR autoencoding, order-invariant latent representations, and direct radar spectrum conditioning. These insights lead to a more compact and expressive generation process. Experiments show that RaLD produces dense and accurate 3D point clouds from raw radar spectrums, offering a promising solution for robust perception in challenging environments.

Method: Proposes ClusterMine which extracts positive concepts from text corpora by combining visual clustering for sample consistency and zero-shot image-text consistency from CLIP models.

Result: Achieves state-of-the-art OOD detection performance without access to positive labels, scalable across CLIP models, and robust to covariate in-distribution shifts.

Conclusion: ClusterMine enables truly unsupervised OOD detection by leveraging widely available text corpora, providing a practical solution for real-world deployment scenarios.

Abstract: Large-scale visual out-of-distribution (OOD) detection has witnessed remarkable progress by leveraging vision-language models such as CLIP. However, a significant limitation of current methods is their reliance on a pre-defined set of in-distribution (ID) ground-truth label names (positives). These fixed label names can be unavailable, unreliable at scale, or become less relevant due to in-distribution shifts after deployment. Towards truly unsupervised OOD detection, we utilize widely available text corpora for positive label mining, bypassing the need for positives. In this paper, we utilize widely available text corpora for positive label mining under a general concept mining paradigm. Within this framework, we propose ClusterMine, a novel positive label mining method. ClusterMine is the first method to achieve state-of-the-art OOD detection performance without access to positive labels. It extracts positive concepts from a large text corpus by combining visual-only sample consistency (via clustering) and zero-shot image-text consistency. Our experimental study reveals that ClusterMine is scalable across a plethora of CLIP models and achieves state-of-the-art robustness to covariate in-distribution shifts. The code is available at https://github.com/HHU-MMBS/clustermine_wacv_official.

Method: Proposed LeCoT network with Spatial-Channel Fusion Transformer blocks that efficiently utilize both spatial and channel global context information. Also includes a prediction block that uses intermediate correspondence features to generate probability sets for guiding subsequent learning phases.

Result: Extensive experiments show LeCoT outperforms state-of-the-art methods in correspondence pruning, relative pose estimation, homography estimation, visual localization, and 3D reconstruction tasks.

Conclusion: LeCoT provides an effective approach for two-view correspondence pruning by naturally leveraging global context information at different stages through novel transformer blocks and progressive probability refinement.

Abstract: Two-view correspondence pruning aims to accurately remove incorrect correspondences (outliers) from initial ones and is widely applied to various computer vision tasks. Current popular strategies adopt multilayer perceptron (MLP) as the backbone, supplemented by additional modules to enhance the network ability to handle context information, which is a known limitation of MLPs. In contrast, we introduce a novel perspective for capturing correspondence context information without extra design modules. To this end, we design a two-view correspondence pruning network called LeCoT, which can naturally leverage global context information at different stages. Specifically, the core design of LeCoT is the Spatial-Channel Fusion Transformer block, a newly proposed component that efficiently utilizes both spatial and channel global context information among sparse correspondences. In addition, we integrate the proposed prediction block that utilizes correspondence features from intermediate stages to generate a probability set, which acts as guiding information for subsequent learning phases, allowing the network to more effectively capture robust global context information. Notably, this prediction block progressively refines the probability set, thereby mitigating the issue of information loss that is common in the traditional one. Extensive experiments prove that the proposed LeCoT outperforms state-of-the-art methods in correspondence pruning, relative pose estimation, homography estimation, visual localization, and $3$D~reconstruction tasks. The code is provided in https://github.com/Dailuanyuan2024/LeCoT-Revisiting-Network-Architecture-for-Two-View-Correspondence-Pruning.

Method: Combines vision-language model encoder with transformer decoder, uses Distance-Based OOD prompts at varying semantic distances from in-distribution classes, and employs OOD Semantic Augmentation for OOD representations to align visual and textual information.

Result: Achieves state-of-the-art performance on Fishyscapes, Segment-Me-If-You-Can, and Road Anomaly datasets across both pixel-level and object-level evaluations.

Conclusion: Vision-language-based OOD segmentation shows strong potential to enhance safety and reliability in autonomous driving systems by effectively generalizing to unseen objects in diverse driving environments.

Abstract: In autonomous driving and robotics, ensuring road safety and reliable decision-making critically depends on out-of-distribution (OOD) segmentation. While numerous methods have been proposed to detect anomalous objects on the road, leveraging the vision-language space-which provides rich linguistic knowledge-remains an underexplored field. We hypothesize that incorporating these linguistic cues can be especially beneficial in the complex contexts found in real-world autonomous driving scenarios. To this end, we present a novel approach that trains a Text-Driven OOD Segmentation model to learn a semantically diverse set of objects in the vision-language space. Concretely, our approach combines a vision-language model’s encoder with a transformer decoder, employs Distance-Based OOD prompts located at varying semantic distances from in-distribution (ID) classes, and utilizes OOD Semantic Augmentation for OOD representations. By aligning visual and textual information, our approach effectively generalizes to unseen objects and provides robust OOD segmentation in diverse driving environments. We conduct extensive experiments on publicly available OOD segmentation datasets such as Fishyscapes, Segment-Me-If-You-Can, and Road Anomaly datasets, demonstrating that our approach achieves state-of-the-art performance across both pixel-level and object-level evaluations. This result underscores the potential of vision-language-based OOD segmentation to bolster the safety and reliability of future autonomous driving systems.

Method: Hybrid image encoding network with large encoder for short-term frames and small encoder for long-term frames; simultaneous extraction of dense and sparse features; compatible with various 3D feature extraction methods and multimodal inputs.

Result: Achieves state-of-the-art end-to-end multi-task 3D perception results on nuScenes benchmark; attains lowest collision rate on nuScenes end-to-end autonomous driving benchmark.

Conclusion: The proposed HENet++ framework effectively addresses computational constraints and feature representation issues in multi-task autonomous driving systems, delivering superior performance while maintaining compatibility with existing methods.

Abstract: Three-dimensional feature extraction is a critical component of autonomous driving systems, where perception tasks such as 3D object detection, bird’s-eye-view (BEV) semantic segmentation, and occupancy prediction serve as important constraints on 3D features. While large image encoders, high-resolution images, and long-term temporal inputs can significantly enhance feature quality and deliver remarkable performance gains, these techniques are often incompatible in both training and inference due to computational resource constraints. Moreover, different tasks favor distinct feature representations, making it difficult for a single model to perform end-to-end inference across multiple tasks while maintaining accuracy comparable to that of single-task models. To alleviate these issues, we present the HENet and HENet++ framework for multi-task 3D perception and end-to-end autonomous driving. Specifically, we propose a hybrid image encoding network that uses a large image encoder for short-term frames and a small one for long-term frames. Furthermore, our framework simultaneously extracts both dense and sparse features, providing more suitable representations for different tasks, reducing cumulative errors, and delivering more comprehensive information to the planning module. The proposed architecture maintains compatibility with various existing 3D feature extraction methods and supports multimodal inputs. HENet++ achieves state-of-the-art end-to-end multi-task 3D perception results on the nuScenes benchmark, while also attaining the lowest collision rate on the nuScenes end-to-end autonomous driving benchmark.

Method: Created MVU-Eval benchmark with 1,824 meticulously curated question-answer pairs spanning 4,959 videos from diverse domains, assessing 8 core competencies including fundamental perception and high-order reasoning tasks aligned with real-world applications.

Result: Extensive evaluation of state-of-the-art open-source and closed-source models revealed significant performance discrepancies and limitations in current MLLMs’ ability to perform understanding across multiple videos.

Conclusion: MVU-Eval addresses a critical gap in MLLM evaluation and will be publicly available to foster future research in multi-video understanding capabilities.

Abstract: The advent of Multimodal Large Language Models (MLLMs) has expanded AI capabilities to visual modalities, yet existing evaluation benchmarks remain limited to single-video understanding, overlooking the critical need for multi-video understanding in real-world scenarios (e.g., sports analytics and autonomous driving). To address this significant gap, we introduce MVU-Eval, the first comprehensive benchmark for evaluating Multi-Video Understanding for MLLMs. Specifically, our MVU-Eval mainly assesses eight core competencies through 1,824 meticulously curated question-answer pairs spanning 4,959 videos from diverse domains, addressing both fundamental perception tasks and high-order reasoning tasks. These capabilities are rigorously aligned with real-world applications such as multi-sensor synthesis in autonomous systems and cross-angle sports analytics. Through extensive evaluation of state-of-the-art open-source and closed-source models, we reveal significant performance discrepancies and limitations in current MLLMs’ ability to perform understanding across multiple videos. The benchmark will be made publicly available to foster future research.

Method: Proposes Texture-Aware Deformation Regularization with texture-based depth alignment loss for Gaussian deformation, and Texture-Aware Canonical Optimization with texture-based noise for canonical Gaussian field optimization.

Result: Outperforms existing dynamic and few-shot techniques on multiple datasets (NeRF-Synthetic, HyperNeRF, NeRF-DS, iPhone-4D) when using sparse frames as input.

Conclusion: Sparse4DGS successfully addresses the challenge of sparse-frame dynamic scene reconstruction by focusing on texture-rich areas with specialized regularization and optimization methods.

Abstract: Dynamic Gaussian Splatting approaches have achieved remarkable performance for 4D scene reconstruction. However, these approaches rely on dense-frame video sequences for photorealistic reconstruction. In real-world scenarios, due to equipment constraints, sometimes only sparse frames are accessible. In this paper, we propose Sparse4DGS, the first method for sparse-frame dynamic scene reconstruction. We observe that dynamic reconstruction methods fail in both canonical and deformed spaces under sparse-frame settings, especially in areas with high texture richness. Sparse4DGS tackles this challenge by focusing on texture-rich areas. For the deformation network, we propose Texture-Aware Deformation Regularization, which introduces a texture-based depth alignment loss to regulate Gaussian deformation. For the canonical Gaussian field, we introduce Texture-Aware Canonical Optimization, which incorporates texture-based noise into the gradient descent process of canonical Gaussians. Extensive experiments show that when taking sparse frames as inputs, our method outperforms existing dynamic or few-shot techniques on NeRF-Synthetic, HyperNeRF, NeRF-DS, and our iPhone-4D datasets.

Method: Created MPD dataset (first large-scale music-painting pairs annotated by experts), developed MPJudge model that integrates music features into visual encoder via modulation-based fusion, and used Direct Preference Optimization for ambiguous cases.

Result: Extensive experiments show the method outperforms existing approaches, and qualitative results demonstrate more accurate identification of music-relevant regions in paintings.

Conclusion: The proposed framework effectively addresses limitations of emotion-based approaches by directly modeling perceptual coherence between music and visual art, providing better assessment of music-induced paintings.

Abstract: Music induced painting is a unique artistic practice, where visual artworks are created under the influence of music. Evaluating whether a painting faithfully reflects the music that inspired it poses a challenging perceptual assessment task. Existing methods primarily rely on emotion recognition models to assess the similarity between music and painting, but such models introduce considerable noise and overlook broader perceptual cues beyond emotion. To address these limitations, we propose a novel framework for music induced painting assessment that directly models perceptual coherence between music and visual art. We introduce MPD, the first large scale dataset of music painting pairs annotated by domain experts based on perceptual coherence. To better handle ambiguous cases, we further collect pairwise preference annotations. Building on this dataset, we present MPJudge, a model that integrates music features into a visual encoder via a modulation based fusion mechanism. To effectively learn from ambiguous cases, we adopt Direct Preference Optimization for training. Extensive experiments demonstrate that our method outperforms existing approaches. Qualitative results further show that our model more accurately identifies music relevant regions in paintings.

Method: Created a dataset with 75 phase-contrast microscopy images containing over 12,000 annotated cells, including soma annotations and morphological categorization by biologists. The dataset has two parts: one for benchmarking with controlled parameters and another for generalization testing under varying conditions.

Result: Evaluation of generalist segmentation models showed limitations on this dataset. Training on Glioma C6 significantly improved segmentation performance, demonstrating its value for developing robust models.

Conclusion: Glioma C6 serves as a valuable resource for developing and benchmarking instance segmentation models in biomedical imaging, with publicly available data for researchers.

Abstract: We present Glioma C6, a new open dataset for instance segmentation of glioma C6 cells, designed as both a benchmark and a training resource for deep learning models. The dataset comprises 75 high-resolution phase-contrast microscopy images with over 12,000 annotated cells, providing a realistic testbed for biomedical image analysis. It includes soma annotations and morphological cell categorization provided by biologists. Additional categorization of cells, based on morphology, aims to enhance the utilization of image data for cancer cell research. Glioma C6 consists of two parts: the first is curated with controlled parameters for benchmarking, while the second supports generalization testing under varying conditions. We evaluate the performance of several generalist segmentation models, highlighting their limitations on our dataset. Our experiments demonstrate that training on Glioma C6 significantly enhances segmentation performance, reinforcing its value for developing robust and generalizable models. The dataset is publicly available for researchers.

Method: Uses sequentialized graph-based procedural representation with edge-based tokenization, trains transformer prior to predict next tokens from RGB images, and incorporates MCTS-guided sampling for better image alignment.

Result: Outperforms state-of-the-art generative 3D methods and domain-specific modeling techniques on cacti, trees, and bridges, with improved generalization to real-world images despite synthetic-only training.

Conclusion: ProcGen3D provides an effective approach for neural procedural graph generation that enables high-quality 3D reconstruction from images across various object categories.

Abstract: We introduce ProcGen3D, a new approach for 3D content creation by generating procedural graph abstractions of 3D objects, which can then be decoded into rich, complex 3D assets. Inspired by the prevalent use of procedural generators in production 3D applications, we propose a sequentialized, graph-based procedural graph representation for 3D assets. We use this to learn to approximate the landscape of a procedural generator for image-based 3D reconstruction. We employ edge-based tokenization to encode the procedural graphs, and train a transformer prior to predict the next token conditioned on an input RGB image. Crucially, to enable better alignment of our generated outputs to an input image, we incorporate Monte Carlo Tree Search (MCTS) guided sampling into our generation process, steering output procedural graphs towards more image-faithful reconstructions. Our approach is applicable across a variety of objects that can be synthesized with procedural generators. Extensive experiments on cacti, trees, and bridges show that our neural procedural graph generation outperforms both state-of-the-art generative 3D methods and domain-specific modeling techniques. Furthermore, this enables improved generalization on real-world input images, despite training only on synthetic data.

Method: Uses representative sample selection based on image confidence and gradient features to select boundary samples, and model merging that fuses weights from multiple fine-tuning trajectories (pre-trained, representative, and random updates).

Result: Significantly improves detection performance - on AIDE, average accuracy on Midjourney increased from 87.63% to 93.03% (6.16% relative improvement).

Conclusion: LiteUpdate effectively addresses the challenges of detector updates by balancing adaptability to new generators with preservation of prior knowledge, substantially boosting detection performance across various detectors.

Abstract: The rapid progress of generative AI has led to the emergence of new generative models, while existing detection methods struggle to keep pace, resulting in significant degradation in the detection performance. This highlights the urgent need for continuously updating AI-generated image detectors to adapt to new generators. To overcome low efficiency and catastrophic forgetting in detector updates, we propose LiteUpdate, a lightweight framework for updating AI-generated image detectors. LiteUpdate employs a representative sample selection module that leverages image confidence and gradient-based discriminative features to precisely select boundary samples. This approach improves learning and detection accuracy on new distributions with limited generated images, significantly enhancing detector update efficiency. Additionally, LiteUpdate incorporates a model merging module that fuses weights from multiple fine-tuning trajectories, including pre-trained, representative, and random updates. This balances the adaptability to new generators and mitigates the catastrophic forgetting of prior knowledge. Experiments demonstrate that LiteUpdate substantially boosts detection performance in various detectors. Specifically, on AIDE, the average detection accuracy on Midjourney improved from 87.63% to 93.03%, a 6.16% relative increase.

Method: Deep learning pipeline using heatmap regression to localize key anatomical landmarks, comparing direct approach with global-to-local learning strategy.

Result: Achieved mean absolute errors of 0.506mm for Keros, 4.516° for Gera, and 0.802mm/0.777mm for TMS classification.

Conclusion: The automated pipeline provides accurate estimation of anatomical risk scores, potentially improving efficiency in preoperative planning for endoscopic sinus surgery.

Abstract: Endoscopic sinus surgery requires careful preoperative assessment of the skull base anatomy to minimize risks such as cerebrospinal fluid leakage. Anatomical risk scores like the Keros, Gera and Thailand-Malaysia-Singapore score offer a standardized approach but require time-consuming manual measurements on coronal CT or CBCT scans. We propose an automated deep learning pipeline that estimates these risk scores by localizing key anatomical landmarks via heatmap regression. We compare a direct approach to a specialized global-to-local learning strategy and find mean absolute errors on the relevant anatomical measurements of 0.506mm for the Keros, 4.516° for the Gera and 0.802mm / 0.777mm for the TMS classification.

Method: The approach uses geometric INRs that add regularization terms in loss functions based on differential geometry tools (normals, curvatures). Key components include INR definition, geometric loss function construction, and sampling schemes from a differential geometry perspective.

Result: Geometric INRs enable significant advancements in surface reconstruction from both oriented point clouds and posed images by ensuring the learned SDFs satisfy proper geometric properties through regularization.

Conclusion: Incorporating differential geometry constraints into implicit neural representations for SDFs provides a powerful framework for improved 3D surface reconstruction, with geometric regularization terms playing a crucial role in ensuring mathematical correctness of the learned representations.

Abstract: \textit{Implicit neural representations} (INRs) have emerged as a promising framework for representing signals in low-dimensional spaces. This survey reviews the existing literature on the specialized INR problem of approximating \textit{signed distance functions} (SDFs) for surface scenes, using either oriented point clouds or a set of posed images. We refer to neural SDFs that incorporate differential geometry tools, such as normals and curvatures, in their loss functions as \textit{geometric} INRs. The key idea behind this 3D reconstruction approach is to include additional \textit{regularization} terms in the loss function, ensuring that the INR satisfies certain global properties that the function should hold – such as having unit gradient in the case of SDFs. We explore key methodological components, including the definition of INR, the construction of geometric loss functions, and sampling schemes from a differential geometry perspective. Our review highlights the significant advancements enabled by geometric INRs in surface reconstruction from oriented point clouds and posed images.

Method: LLM-based quality assessment system that generates numerical scores and textual descriptions, using various inference strategies including zero-shot approach, metadata integration, and error feedback.

Result: The system produces highly correlated scores and interpretable output, with progressive contributions from each inference method to overall performance, adding value to clinical workflows.

Conclusion: The proposed LLM-based approach effectively assesses low-dose CT image quality with both quantitative scores and qualitative descriptions, demonstrating the value of systematic inference strategies for clinical applications.

Abstract: Low-dose computed tomography (CT) represents a significant improvement in patient safety through lower radiation doses, but increased noise, blur, and contrast loss can diminish diagnostic quality. Therefore, consistency and robustness in image quality assessment become essential for clinical applications. In this study, we propose an LLM-based quality assessment system that generates both numerical scores and textual descriptions of degradations such as noise, blur, and contrast loss. Furthermore, various inference strategies - from the zero-shot approach to metadata integration and error feedback - are systematically examined, demonstrating the progressive contribution of each method to overall performance. The resultant assessments yield not only highly correlated scores but also interpretable output, thereby adding value to clinical workflows. The source codes of our study are available at https://github.com/itu-biai/lmms_ldct_iqa.

Method: Uses conditional InfoGAN to identify naturally occurring image features that can serve as potent triggers. Ensures triggers are easily separable from benign features, allowing learning from extremely small poisoned datasets.

Result: Achieves clean accuracy drop of less than 1% while successfully attacking six datasets, five architectures, and four tasks (including first demonstration in regression and segmentation). Resilient against most existing defenses.

Conclusion: GCB provides a highly effective and stealthy clean-image backdoor attack framework that minimizes accuracy degradation and demonstrates remarkable versatility across diverse applications.

Abstract: Clean-image backdoor attacks, which use only label manipulation in training datasets to compromise deep neural networks, pose a significant threat to security-critical applications. A critical flaw in existing methods is that the poison rate required for a successful attack induces a proportional, and thus noticeable, drop in Clean Accuracy (CA), undermining their stealthiness. This paper presents a new paradigm for clean-image attacks that minimizes this accuracy degradation by optimizing the trigger itself. We introduce Generative Clean-Image Backdoors (GCB), a framework that uses a conditional InfoGAN to identify naturally occurring image features that can serve as potent and stealthy triggers. By ensuring these triggers are easily separable from benign task-related features, GCB enables a victim model to learn the backdoor from an extremely small set of poisoned examples, resulting in a CA drop of less than 1%. Our experiments demonstrate GCB’s remarkable versatility, successfully adapting to six datasets, five architectures, and four tasks, including the first demonstration of clean-image backdoors in regression and segmentation. GCB also exhibits resilience against most of the existing backdoor defenses.

Method: Three VFM-based modules: Patch-weighted Global Feature Alignment (PGFA) for global feature distillation, Prototype-based Instance Feature Alignment (PIFA) for instance-level contrastive learning, and Dual-source Enhanced Pseudo-label Fusion (DEPF) for reliable supervision.

Result: Extensive experiments on six benchmarks demonstrate state-of-the-art SFOD performance, validating improved transferability and discriminability.

Conclusion: Integrating VFMs as external knowledge sources effectively enhances both feature alignment and label quality in SFOD, overcoming limitations of internal-only approaches.

Abstract: Source-Free Object Detection (SFOD) aims to adapt a source-pretrained object detector to a target domain without access to source data. However, existing SFOD methods predominantly rely on internal knowledge from the source model, which limits their capacity to generalize across domains and often results in biased pseudo-labels, thereby hindering both transferability and discriminability. In contrast, Vision Foundation Models (VFMs), pretrained on massive and diverse data, exhibit strong perception capabilities and broad generalization, yet their potential remains largely untapped in the SFOD setting. In this paper, we propose a novel SFOD framework that leverages VFMs as external knowledge sources to jointly enhance feature alignment and label quality. Specifically, we design three VFM-based modules: (1) Patch-weighted Global Feature Alignment (PGFA) distills global features from VFMs using patch-similarity-based weighting to enhance global feature transferability; (2) Prototype-based Instance Feature Alignment (PIFA) performs instance-level contrastive learning guided by momentum-updated VFM prototypes; and (3) Dual-source Enhanced Pseudo-label Fusion (DEPF) fuses predictions from detection VFMs and teacher models via an entropy-aware strategy to yield more reliable supervision. Extensive experiments on six benchmarks demonstrate that our method achieves state-of-the-art SFOD performance, validating the effectiveness of integrating VFMs to simultaneously improve transferability and discriminability.

Method: A three-component system consisting of understanding model, texture module for appearance synthesis, and geometry module for explicit geometric constraints, trained with a two-stage strategy to leverage spatiotemporal modeling and geometric constraints.

Result: Achieves state-of-the-art score of 55.4 on VSI-Bench benchmark, outperforming specialized 3D understanding models while delivering strong performance in novel view synthesis and 3D scene generation.

Conclusion: Omni-View successfully demonstrates that joint modeling of understanding and generation tasks enables synergistic interaction that enriches holistic 3D scene understanding, validating the principle that generation facilitates understanding.

Abstract: This paper presents Omni-View, which extends the unified multimodal understanding and generation to 3D scenes based on multiview images, exploring the principle that “generation facilitates understanding”. Consisting of understanding model, texture module, and geometry module, Omni-View jointly models scene understanding, novel view synthesis, and geometry estimation, enabling synergistic interaction between 3D scene understanding and generation tasks. By design, it leverages the spatiotemporal modeling capabilities of its texture module responsible for appearance synthesis, alongside the explicit geometric constraints provided by its dedicated geometry module, thereby enriching the model’s holistic understanding of 3D scenes. Trained with a two-stage strategy, Omni-View achieves a state-of-the-art score of 55.4 on the VSI-Bench benchmark, outperforming existing specialized 3D understanding models, while simultaneously delivering strong performance in both novel view synthesis and 3D scene generation.

Method: Semi-supervised segmentation framework using sub-meter satellite images to detect individual refugee shelters, applied across multi-year data for WASH accessibility analysis.

Result: Achieved 76.4% F1-score in shelter detection; showed declining WASH accessibility from 25 people per facility in 2022 to 29.4 in 2025; women and girls experience reduced accessibility due to inadequate safety segregation.

Conclusion: High-resolution remote sensing and machine learning can detect inequality and inform equitable resource planning in humanitarian settings, emphasizing demand-responsive allocation strategies for underserved populations.

Abstract: Access to Water, Sanitation, and Hygiene (WASH) services remains a major public health concern in refugee camps. This study introduces a remote sensing-driven framework to quantify WASH accessibility-specifically to water pumps, latrines, and bathing cubicles-in the Rohingya camps of Cox’s Bazar, one of the world’s most densely populated displacement settings. Detecting refugee shelters in such emergent camps presents substantial challenges, primarily due to their dense spatial configuration and irregular geometric patterns. Using sub-meter satellite images, we develop a semi-supervised segmentation framework that achieves an F1-score of 76.4% in detecting individual refugee shelters. Applying the framework across multi-year data reveals declining WASH accessibility, driven by rapid refugee population growth and reduced facility availability, rising from 25 people per facility in 2022 to 29.4 in 2025. Gender-disaggregated analysis further shows that women and girls experience reduced accessibility, in scenarios with inadequate safety-related segregation in WASH facilities. These findings suggest the importance of demand-responsive allocation strategies that can identify areas with under-served populations-such as women and girls-and ensure that limited infrastructure serves the greatest number of people in settings with fixed or shrinking budgets. We also discuss the value of high-resolution remote sensing and machine learning to detect inequality and inform equitable resource planning in complex humanitarian environments.

Method: Uses autoencoder trained to repair corrupted inputs with structured, spatially coherent perturbations (not i.i.d. noise) and adds Gaussian noise as Tikhonov regularizer to anchor Jacobian toward identity, stabilizing reconstruction.

Result: Achieves state-of-the-art performance on MVTec AD benchmark with I/P-AUROC scores of 99.9/99.4, demonstrating superior detection and segmentation accuracy.

Conclusion: The proposed identity-anchored regularization with structured corruption effectively improves anomaly detection for industrial inspection, supporting the theoretical framework with practical relevance.

Abstract: Anomaly detection plays a pivotal role in automated industrial inspection, aiming to identify subtle or rare defects in otherwise uniform visual patterns. As collecting representative examples of all possible anomalies is infeasible, we tackle structural anomaly detection using a self-supervised autoencoder that learns to repair corrupted inputs. To this end, we introduce a corruption model that injects artificial disruptions into training images to mimic structural defects. While reminiscent of denoising autoencoders, our approach differs in two key aspects. First, instead of unstructured i.i.d.\ noise, we apply structured, spatially coherent perturbations that make the task a hybrid of segmentation and inpainting. Second, and counterintuitively, we add and preserve Gaussian noise on top of the occlusions, which acts as a Tikhonov regularizer anchoring the Jacobian of the reconstruction function toward identity. This identity-anchored regularization stabilizes reconstruction and further improves both detection and segmentation accuracy. On the MVTec AD benchmark, our method achieves state-of-the-art results (I/P-AUROC: 99.9/99.4), supporting our theoretical framework and demonstrating its practical relevance for automatic inspection.

Method: Fine-tuned FLUX.1-dev diffusion model on small infrared dataset using parameter-efficient techniques, then employed domain-adapted CLIP-based verifier during inference to guide sampling toward higher quality generations.

Result: Consistent improvements in generation quality, reducing FID scores by 10% on KAIST Multispectral Pedestrian Detection Benchmark compared to unguided baseline.

Conclusion: Inference-time guidance offers promising direction for bridging domain gap in low-data infrared settings.

Abstract: Infrared imagery enables temperature-based scene understanding using passive sensors, particularly under conditions of low visibility where traditional RGB imaging fails. Yet, developing downstream vision models for infrared applications is hindered by the scarcity of high-quality annotated data, due to the specialized expertise required for infrared annotation. While synthetic infrared image generation has the potential to accelerate model development by providing large-scale, diverse training data, training foundation-level generative diffusion models in the infrared domain has remained elusive due to limited datasets. In light of such data constraints, we explore an inference-time scaling approach using a domain-adapted CLIP-based verifier for enhanced infrared image generation quality. We adapt FLUX.1-dev, a state-of-the-art text-to-image diffusion model, to the infrared domain by finetuning it on a small sample of infrared images using parameter-efficient techniques. The trained verifier is then employed during inference to guide the diffusion sampling process toward higher quality infrared generations that better align with input text prompts. Empirically, we find that our approach leads to consistent improvements in generation quality, reducing FID scores on the KAIST Multispectral Pedestrian Detection Benchmark dataset by 10% compared to unguided baseline samples. Our results suggest that inference-time guidance offers a promising direction for bridging the domain gap in low-data infrared settings.

Method: Proposes temporal correlation across 4D sequences to rectify deformable scales and rotations, and an adaptive spatial densification/pruning strategy that dynamically adds/deletes Gaussian points based on pre-frame movements.

Result: Extensive experiments show 4DSTR achieves state-of-the-art performance in video-to-4D generation, excelling in reconstruction quality, spatial-temporal consistency, and adaptation to rapid temporal movements.

Conclusion: 4DSTR effectively addresses spatial-temporal consistency and rapid temporal variation challenges in 4D content generation through its novel spatial-temporal rectification approach.

Abstract: Remarkable advances in recent 2D image and 3D shape generation have induced a significant focus on dynamic 4D content generation. However, previous 4D generation methods commonly struggle to maintain spatial-temporal consistency and adapt poorly to rapid temporal variations, due to the lack of effective spatial-temporal modeling. To address these problems, we propose a novel 4D generation network called 4DSTR, which modulates generative 4D Gaussian Splatting with spatial-temporal rectification. Specifically, temporal correlation across generated 4D sequences is designed to rectify deformable scales and rotations and guarantee temporal consistency. Furthermore, an adaptive spatial densification and pruning strategy is proposed to address significant temporal variations by dynamically adding or deleting Gaussian points with the awareness of their pre-frame movements. Extensive experiments demonstrate that our 4DSTR achieves state-of-the-art performance in video-to-4D generation, excelling in reconstruction quality, spatial-temporal consistency, and adaptation to rapid temporal movements.

Method: StreamKV dynamically partitions video streams into semantic segments, calculates summary vectors for retrieval, uses guidance prompts for compression, and unifies retrieval and compression in a layer-adaptive single module.

Result: Extensive experiments on StreamingVQA benchmarks show StreamKV significantly outperforms existing Online Video-LLMs in accuracy while substantially improving memory efficiency and computational latency.

Conclusion: StreamKV effectively addresses the challenges of long video processing in Video-LLMs through advanced KV cache management, achieving state-of-the-art performance in streaming video question answering.

Abstract: Video Large Language Models (Video-LLMs) have demonstrated significant potential in the areas of video captioning, search, and summarization. However, current Video-LLMs still face challenges with long real-world videos. Recent methods have introduced a retrieval mechanism that retrieves query-relevant KV caches for question answering, enhancing the efficiency and accuracy of long real-world videos. However, the compression and retrieval of KV caches are still not fully explored. In this paper, we propose \textbf{StreamKV}, a training-free framework that seamlessly equips Video-LLMs with advanced KV cache retrieval and compression. Compared to previous methods that used uniform partitioning, StreamKV dynamically partitions video streams into semantic segments, which better preserves semantic information. For KV cache retrieval, StreamKV calculates a summary vector for each segment to retain segment-level information essential for retrieval. For KV cache compression, StreamKV introduces a guidance prompt designed to capture the key semantic elements within each segment, ensuring only the most informative KV caches are retained for answering questions. Moreover, StreamKV unifies KV cache retrieval and compression within a single module, performing both in a layer-adaptive manner, thereby further improving the effectiveness of streaming video question answering. Extensive experiments on public StreamingVQA benchmarks demonstrate that StreamKV significantly outperforms existing Online Video-LLMs, achieving superior accuracy while substantially improving both memory efficiency and computational latency. The code has been released at https://github.com/sou1p0wer/StreamKV.

Method: FLASH integrates Frequency-Aware Window Attention (combining local spatial attention with global frequency-domain analysis via FFT) and Adaptive Multi-Scale Fusion (replacing conventional skip connections with learned position-specific feature aggregation enhanced by CBAM attention).

Result: Achieves state-of-the-art performance on KITTI across all evaluation metrics, surpassing uncertainty-enhanced baselines while maintaining single-pass efficiency for real-time deployment.

Conclusion: The dual-domain approach effectively handles uncertainty through architectural design rather than computationally expensive stochastic inference, making it practical for autonomous systems.

Abstract: LiDAR super-resolution addresses the challenge of achieving high-quality 3D perception from cost-effective, low-resolution sensors. While recent transformer-based approaches like TULIP show promise, they remain limited to spatial-domain processing with restricted receptive fields. We introduce FLASH (Frequency-aware LiDAR Adaptive Super-resolution with Hierarchical fusion), a novel framework that overcomes these limitations through dual-domain processing. FLASH integrates two key innovations: (i) Frequency-Aware Window Attention that combines local spatial attention with global frequency-domain analysis via FFT, capturing both fine-grained geometry and periodic scanning patterns at log-linear complexity. (ii) Adaptive Multi-Scale Fusion that replaces conventional skip connections with learned position-specific feature aggregation, enhanced by CBAM attention for dynamic feature selection. Extensive experiments on KITTI demonstrate that FLASH achieves state-of-the-art performance across all evaluation metrics, surpassing even uncertainty-enhanced baselines that require multiple forward passes. Notably, FLASH outperforms TULIP with Monte Carlo Dropout while maintaining single-pass efficiency, which enables real-time deployment. The consistent superiority across all distance ranges validates that our dual-domain approach effectively handles uncertainty through architectural design rather than computationally expensive stochastic inference, making it practical for autonomous systems.

Method: Used Res-Unet architecture trained twice (with and without pre-trained weights) on ISLES 2015 dataset with T1, T2, DWI, and FLAIR MRI sequences. Integrated Majority Voting Classifier to combine results from each axis.

Result: Achieved Dice score of 80.5% and accuracy of 74.03% on 3D volume evaluation, demonstrating effective stroke lesion segmentation.

Conclusion: The proposed framework provides fast, automatic segmentation of ischemic stroke lesions across multiple MRI sequences, overcoming limitations of manual segmentation and hand-crafted feature approaches.

Abstract: The accurate understanding of ischemic stroke lesions is critical for efficient therapy and prognosis of stroke patients. Magnetic resonance imaging (MRI) is sensitive to acute ischemic stroke and is a common diagnostic method for stroke. However, manual lesion segmentation performed by experts is tedious, time-consuming, and prone to observer inconsistency. Automatic medical image analysis methods have been proposed to overcome this challenge. However, previous approaches have relied on hand-crafted features that may not capture the irregular and physiologically complex shapes of ischemic stroke lesions. In this study, we present a novel framework for quickly and automatically segmenting ischemic stroke lesions on various MRI sequences, including T1-weighted, T2-weighted, DWI, and FLAIR. The proposed methodology is validated on the ISLES 2015 Brain Stroke sequence dataset, where we trained our model using the Res-Unet architecture twice: first, with pre-existing weights, and then without, to explore the benefits of transfer learning. Evaluation metrics, including the Dice score and sensitivity, were computed across 3D volumes. Finally, a Majority Voting Classifier was integrated to amalgamate the outcomes from each axis, resulting in a comprehensive segmentation method. Our efforts culminated in achieving a Dice score of 80.5% and an accuracy of 74.03%, showcasing the efficacy of our segmentation approach.

Method: VADER uses an Anomaly Scorer for per-frame scoring, Context-Aware Sampling (CAES) to capture causal context, Relation Feature Extractor and COntrastive Relation Encoder (CORE) to model object interactions, and integrates these with LLMs for reasoning.

Result: Experiments on multiple real-world VAU benchmarks show VADER achieves strong performance across anomaly description, explanation, and causal reasoning tasks.

Conclusion: VADER advances explainable video anomaly analysis by providing causally grounded descriptions and robust question answering through integrated visual and relational reasoning.

Abstract: Video anomaly understanding (VAU) aims to provide detailed interpretation and semantic comprehension of anomalous events within videos, addressing limitations of traditional methods that focus solely on detecting and localizing anomalies. However, existing approaches often neglect the deeper causal relationships and interactions between objects, which are critical for understanding anomalous behaviors. In this paper, we propose VADER, an LLM-driven framework for Video Anomaly unDErstanding, which integrates keyframe object Relation features with visual cues to enhance anomaly comprehension from video. Specifically, VADER first applies an Anomaly Scorer to assign per-frame anomaly scores, followed by a Context-AwarE Sampling (CAES) strategy to capture the causal context of each anomalous event. A Relation Feature Extractor and a COntrastive Relation Encoder (CORE) jointly model dynamic object interactions, producing compact relational representations for downstream reasoning. These visual and relational cues are integrated with LLMs to generate detailed, causally grounded descriptions and support robust anomaly-related question answering. Experiments on multiple real-world VAU benchmarks demonstrate that VADER achieves strong results across anomaly description, explanation, and causal reasoning tasks, advancing the frontier of explainable video anomaly analysis.

Method: Predicts local Gaussians and camera poses for each view, aggregated into global representation using predicted or provided poses. Uses mixing training strategy to mitigate task entanglement, pairwise camera-distance normalization for scale ambiguity, and embeds camera intrinsics into network.

Result: Achieves state-of-the-art performance on standard benchmarks in both pose-free and pose-dependent settings. Reconstructs scenes from 100 views in just 2.69 seconds on NVIDIA GH200 GPU.

Conclusion: YoNoSplat provides an efficient and versatile solution for 3D scene reconstruction from unstructured image collections, handling various input conditions while maintaining high quality and speed.

Abstract: Fast and flexible 3D scene reconstruction from unstructured image collections remains a significant challenge. We present YoNoSplat, a feedforward model that reconstructs high-quality 3D Gaussian Splatting representations from an arbitrary number of images. Our model is highly versatile, operating effectively with both posed and unposed, calibrated and uncalibrated inputs. YoNoSplat predicts local Gaussians and camera poses for each view, which are aggregated into a global representation using either predicted or provided poses. To overcome the inherent difficulty of jointly learning 3D Gaussians and camera parameters, we introduce a novel mixing training strategy. This approach mitigates the entanglement between the two tasks by initially using ground-truth poses to aggregate local Gaussians and gradually transitioning to a mix of predicted and ground-truth poses, which prevents both training instability and exposure bias. We further resolve the scale ambiguity problem by a novel pairwise camera-distance normalization scheme and by embedding camera intrinsics into the network. Moreover, YoNoSplat also predicts intrinsic parameters, making it feasible for uncalibrated inputs. YoNoSplat demonstrates exceptional efficiency, reconstructing a scene from 100 views (at 280x518 resolution) in just 2.69 seconds on an NVIDIA GH200 GPU. It achieves state-of-the-art performance on standard benchmarks in both pose-free and pose-dependent settings. Our project page is at https://botaoye.github.io/yonosplat/.

Method: Uses street-level cameras and object detection algorithms (YOLOv8, YOLOv10, YOLO11m, RT-DETR) on data collected from Sangareddy district, India.

Result: YOLO11m achieved the highest accuracy of 92.39% and mAP@50 of 0.91 in waste detection, effectively capturing waste disposal patterns across different time periods.

Conclusion: Object detection models are well-suited for monitoring waste dumping events and the system provides comprehensive daily and nightly monitoring of waste disposal patterns.

Abstract: This paper proposes a smart way to manage municipal solid waste by using the Internet of Things (IoT) and computer vision (CV) to monitor illegal waste dumping at garbage vulnerable points (GVPs) in urban areas. The system can quickly detect and monitor dumped waste using a street-level camera and object detection algorithm. Data was collected from the Sangareddy district in Telangana, India. A series of comprehensive experiments was carried out using the proposed dataset to assess the accuracy and overall performance of various object detection models. Specifically, we performed an in-depth evaluation of YOLOv8, YOLOv10, YOLO11m, and RT-DETR on our dataset. Among these models, YOLO11m achieved the highest accuracy of 92.39% in waste detection, demonstrating its effectiveness in detecting waste. Additionally, it attains an mAP@50 of 0.91, highlighting its high precision. These findings confirm that the object detection model is well-suited for monitoring and tracking waste dumping events at GVP locations. Furthermore, the system effectively captures waste disposal patterns, including hourly, daily, and weekly dumping trends, ensuring comprehensive daily and nightly monitoring.

Method: StreamDiffusionV2 integrates an SLO-aware batching scheduler, block scheduler, sink-token-guided rolling KV cache, motion-aware noise controller, and scalable pipeline orchestration that parallelizes diffusion across denoising steps and network layers.

Result: The system achieves first frame rendering within 0.5s and attains 58.28 FPS with a 14B-parameter model and 64.52 FPS with a 1.3B-parameter model on four H100 GPUs, scaling near-linearly without violating latency guarantees.

Conclusion: StreamDiffusionV2 makes state-of-the-art generative live streaming practical and accessible for both individual creators and enterprise platforms by solving the scalability and latency challenges in real-time video diffusion.

Abstract: Generative models are reshaping the live-streaming industry by redefining how content is created, styled, and delivered. Previous image-based streaming diffusion models have powered efficient and creative live streaming products but have hit limits on temporal consistency due to the foundation of image-based designs. Recent advances in video diffusion have markedly improved temporal consistency and sampling efficiency for offline generation. However, offline generation systems primarily optimize throughput by batching large workloads. In contrast, live online streaming operates under strict service-level objectives (SLOs): time-to-first-frame must be minimal, and every frame must meet a per-frame deadline with low jitter. Besides, scalable multi-GPU serving for real-time streams remains largely unresolved so far. To address this, we present StreamDiffusionV2, a training-free pipeline for interactive live streaming with video diffusion models. StreamDiffusionV2 integrates an SLO-aware batching scheduler and a block scheduler, together with a sink-token–guided rolling KV cache, a motion-aware noise controller, and other system-level optimizations. Moreover, we introduce a scalable pipeline orchestration that parallelizes the diffusion process across denoising steps and network layers, achieving near-linear FPS scaling without violating latency guarantees. The system scales seamlessly across heterogeneous GPU environments and supports flexible denoising steps (e.g., 1–4), enabling both ultra-low-latency and higher-quality modes. Without TensorRT or quantization, StreamDiffusionV2 renders the first frame within 0.5s and attains 58.28 FPS with a 14B-parameter model and 64.52 FPS with a 1.3B-parameter model on four H100 GPUs, making state-of-the-art generative live streaming practical and accessible–from individual creators to enterprise-scale platforms.

Method: Extract motion patterns from pre-trained video models, embed motions into latent vectors, train shared motion decoder to learn neural key point trajectories, and drive 3D Gaussians with these key points for geometry and appearance modeling.

Result: Successfully generates diverse 3D motions from single images, supports instant sampling of motions in single-forward pass, and enables applications like 3D motion interpolation and language-guided motion generation.

Conclusion: DIMO provides an effective framework for generating diverse 3D motions from single images by leveraging video priors and learning compact motion representations, opening possibilities for various motion generation applications.

Abstract: We present DIMO, a generative approach capable of generating diverse 3D motions for arbitrary objects from a single image. The core idea of our work is to leverage the rich priors in well-trained video models to extract the common motion patterns and then embed them into a shared low-dimensional latent space. Specifically, we first generate multiple videos of the same object with diverse motions. We then embed each motion into a latent vector and train a shared motion decoder to learn the distribution of motions represented by a structured and compact motion representation, i.e., neural key point trajectories. The canonical 3D Gaussians are then driven by these key points and fused to model the geometry and appearance. During inference time with learned latent space, we can instantly sample diverse 3D motions in a single-forward pass and support several interesting applications including 3D motion interpolation and language-guided motion generation. Our project page is available at https://linzhanm.github.io/dimo.

Method: Reconstructs static geometry from pre-scan videos and continuously models motion through multi-view perception, fusing static and dynamic components into an immersive 3D environment for simulation and exploration.

Result: Achieves centimeter-level accuracy in geometry reconstruction and enables realistic sensor simulations. Models like FoundationStereo and ORB-SLAM3 perform within reported accuracy on real datasets using TwinOR-synthesized data.

Conclusion: TwinOR provides a real-to-sim pipeline for creating dynamic, photorealistic digital twins that enable safe, scalable, and data-efficient development of embodied AI for surgical applications.

Abstract: Developing embodied AI for intelligent surgical systems requires safe, controllable environments for continual learning and evaluation. However, safety regulations and operational constraints in operating rooms (ORs) limit embodied agents from freely perceiving and interacting in realistic settings. Digital twins provide high-fidelity, risk-free environments for exploration and training. How we may create photorealistic and dynamic digital representations of ORs that capture relevant spatial, visual, and behavioral complexity remains unclear. We introduce TwinOR, a framework for constructing photorealistic, dynamic digital twins of ORs for embodied AI research. The system reconstructs static geometry from pre-scan videos and continuously models human and equipment motion through multi-view perception of OR activities. The static and dynamic components are fused into an immersive 3D environment that supports controllable simulation and embodied exploration. The proposed framework reconstructs complete OR geometry with centimeter level accuracy while preserving dynamic interaction across surgical workflows, enabling realistic renderings and a virtual playground for embodied AI systems. In our experiments, TwinOR simulates stereo and monocular sensor streams for geometry understanding and visual localization tasks. Models such as FoundationStereo and ORB-SLAM3 on TwinOR-synthesized data achieve performance within their reported accuracy on real indoor datasets, demonstrating that TwinOR provides sensor-level realism sufficient for perception and localization challenges. By establishing a real-to-sim pipeline for constructing dynamic, photorealistic digital twins of OR environments, TwinOR enables the safe, scalable, and data-efficient development and benchmarking of embodied AI, ultimately accelerating the deployment of embodied AI from sim-to-real.

Method: Developed ColorBench benchmark with diverse test scenarios grounded in real applications to evaluate color perception, reasoning, and robustness across 32 different VLMs with varying language models and vision encoders.

Result: Key findings: (i) Scaling law holds with language models being more important than vision encoders; (ii) Small performance gaps indicate color understanding is neglected; (iii) CoT reasoning improves performance; (iv) Color clues are used but can mislead models.

Conclusion: Current VLMs have critical limitations in color comprehension, highlighting the need to enhance color understanding capabilities in multimodal AI systems.

Abstract: Color plays an important role in human perception and usually provides critical clues in visual reasoning. However, it is unclear whether and how vision-language models (VLMs) can perceive, understand, and leverage color as humans. This paper introduces ColorBench, an innovative benchmark meticulously crafted to assess the capabilities of VLMs in color understanding, including color perception, reasoning, and robustness. By curating a suite of diverse test scenarios, with grounding in real applications, ColorBench evaluates how these models perceive colors, infer meanings from color-based cues, and maintain consistent performance under varying color transformations. Through an extensive evaluation of 32 VLMs with varying language models and vision encoders, our paper reveals some undiscovered findings: (i) The scaling law (larger models are better) still holds on ColorBench, while the language model plays a more important role than the vision encoder. (ii) However, the performance gaps across models are relatively small, indicating that color understanding has been largely neglected by existing VLMs. (iii) CoT reasoning improves color understanding accuracies and robustness, though they are vision-centric tasks. (iv) Color clues are indeed leveraged by VLMs on ColorBench but they can also mislead models in some tasks. These findings highlight the critical limitations of current VLMs and underscore the need to enhance color comprehension. Our ColorBenchcan serve as a foundational tool for advancing the study of human-level color understanding of multimodal AI.

Method: H-RANSAC introduces a novel geometric (cheiral) criterion to reject implausible point configurations early, leverages typically discarded concave quadrilaterals, and includes a post-hoc criterion for accuracy improvement. It provides analytical derivations for expected maximum iterations.

Result: H-RANSAC significantly outperforms state-of-the-art classical methods combined with deep learning-based salient point detection in terms of average reprojection error and success rates on football match video frames with divergent viewpoints.

Conclusion: The proposed H-RANSAC algorithm provides an effective solution for homography estimation without requiring feature vectors or explicit point pairing, demonstrating superior performance in challenging scenarios with highly divergent camera viewpoints.

Abstract: Estimating the homography matrix between images captured under radically different camera poses and zoom factors is a complex challenge. Traditional methods rely on the Random Sample Consensus (RANSAC) algorithm, which requires pairs of homologous points, pre-matched based on local image feature vectors. Sampling consensus is a core step in many Artificial Intelligence (AI) algorithms that enable computer systems to recognize patterns in data. In this paper, we propose H-RANSAC, an algorithm for homography estimation that eliminates the need for feature vectors or explicit point pairing, while it optionally supports point labeling into two classes. H-RANSAC introduces a novel geometric (cheiral) criterion that intelligently rejects implausible point configurations at the beginning of each iteration, while leveraging concave quadrilaterals typically discarded by similar algorithms. A post-hoc criterion at the end of each iteration improves accuracy further. Analytical derivations of the expected maximum iterations are provided, considering success probabilities and outlier rates, enabling adaptive performance tuning. The algorithm is validated on a demanding task: estimating homography between video frames of football matches captured by 12 cameras with highly divergent viewpoints. Results show that H-RANSAC significantly outperforms state-of-the-art classical methods, combined with deep learning-based salient point detection, in terms of average reprojection error and success rates. The relevant implementation is available in https://github.com/gnousias/H-RANSAC.

Method: Conducted a broad study examining widely used metrics for compositional text-image evaluation, analyzing their behavior across diverse compositional challenges and comparing how different metric families align with human judgments.

Result: No single metric performs consistently across tasks - performance varies with compositional problem type. VQA-based metrics are not uniformly superior, while certain embedding-based metrics prove stronger in specific cases. Image-only metrics contribute little to compositional evaluation.

Conclusion: Careful and transparent metric selection is crucial for trustworthy evaluation and their use as reward models in generation, as metric performance depends on the specific compositional challenge being evaluated.

Abstract: Text-image generation has advanced rapidly, but assessing whether outputs truly capture the objects, attributes, and relations described in prompts remains a central challenge. Evaluation in this space relies heavily on automated metrics, yet these are often adopted by convention or popularity rather than validated against human judgment. Because evaluation and reported progress in the field depend directly on these metrics, it is critical to understand how well they reflect human preferences. To address this, we present a broad study of widely used metrics for compositional text-image evaluation. Our analysis goes beyond simple correlation, examining their behavior across diverse compositional challenges and comparing how different metric families align with human judgments. The results show that no single metric performs consistently across tasks: performance varies with the type of compositional problem. Notably, VQA-based metrics, though popular, are not uniformly superior, while certain embedding-based metrics prove stronger in specific cases. Image-only metrics, as expected, contribute little to compositional evaluation, as they are designed for perceptual quality rather than alignment. These findings underscore the importance of careful and transparent metric selection, both for trustworthy evaluation and for their use as reward models in generation. Project page is available at https://amirkasaei.com/eval-the-evals/ .

Method: Multi-target multi-branch supernet approach that searches for optimal placement of lightweight convolution layers and memory-efficient self-attention layers between branches at different resolutions.

Result: Discovers competitive real-time models on Cityscapes, ADE20K, and COCO datasets, achieving strong accuracy-latency trade-offs without ImageNet pretraining.

Conclusion: HyCTAS provides an automated solution for finding efficient hybrid CNN-Transformer architectures that outperform manual designs in real-time segmentation tasks.

Abstract: Real-time image segmentation demands architectures that preserve fine spatial detail while capturing global context under tight latency and memory budgets. Image segmentation is one of the most fundamental problems in computer vision and has drawn a lot of attention due to its vast applications in image understanding and autonomous driving. However, designing effective and efficient segmentation neural architectures is a labor-intensive process that may require numerous trials by human experts. In this paper, we address the challenge of integrating multi-head self-attention into high-resolution representation CNNs efficiently by leveraging architecture search. Manually replacing convolution layers with multi-head self-attention is non-trivial due to the costly overhead in memory to maintain high resolution. By contrast, we develop a multi-target multi-branch supernet method, which not only fully utilizes the advantages of high-resolution features but also finds the proper location for placing the multi-head self-attention module. Our search algorithm is optimized towards multiple objectives (e.g., latency and mIoU) and is capable of finding architectures on the approximate Pareto front with an arbitrary number of branches in a single search. We further present a series of models via the Hybrid Convolutional-Transformer Architecture Search (HyCTAS) method that searches for the best hybrid combination of lightweight convolution layers and memory-efficient self-attention layers between branches from different resolutions and fuses them at high resolution for both efficiency and effectiveness. On Cityscapes, ADE20K, and COCO, HyCTAS discovers competitive real-time models without ImageNet pretraining, delivering strong accuracy and latency trade-offs. Code and models are available at https://github.com/MarvinYu1995/HyCTAS.

Method: Proposes a new framing of hallucination in text-to-image models and develops a taxonomy with three categories: attribute, relation, and object hallucinations.

Result: The proposed framing introduces an upper bound for evaluation and surfaces hidden biases in text-to-image models.

Conclusion: This approach provides a foundation for richer assessment of text-to-image models by systematically addressing bias-driven hallucinations beyond simple prompt alignment.

Abstract: In language and vision-language models, hallucination is broadly understood as content generated from a model’s prior knowledge or biases rather than from the given input. While this phenomenon has been studied in those domains, it has not been clearly framed for text-to-image (T2I) generative models. Existing evaluations mainly focus on alignment, checking whether prompt-specified elements appear, but overlook what the model generates beyond the prompt. We argue for defining hallucination in T2I as bias-driven deviations and propose a taxonomy with three categories: attribute, relation, and object hallucinations. This framing introduces an upper bound for evaluation and surfaces hidden biases, providing a foundation for richer assessment of T2I models.

Method: Created SkinCAP with 4,000 images annotated by board-certified dermatologists for medical descriptions, and SkinCoT with 3,041 images paired with clinician-verified hierarchical chain-of-thought diagnoses evaluated against six quality criteria.

Result: SkinCaRe provides 7,041 expertly curated dermatologic cases with comprehensive natural language descriptions and explanations, serving as a unified resource for training multimodal models in dermatology.

Conclusion: SkinCaRe addresses the interpretability gap in AI-based dermatology diagnosis by providing a meticulously annotated dataset with rich medical descriptions and chain-of-thought reasoning, enabling better training of multimodal models.

Abstract: With the widespread application of artificial intelligence (AI), particularly deep learning (DL) and vision large language models (VLLMs), in skin disease diagnosis, the need for interpretability becomes crucial. However, existing dermatology datasets are limited in their inclusion of concept-level meta-labels, and none offer rich medical descriptions in natural language. This deficiency impedes the advancement of LLM-based methods in dermatologic diagnosis. To address this gap and provide a meticulously annotated dermatology dataset with comprehensive natural language descriptions, we introduce \textbf{SkinCaRe}, a comprehensive multimodal resource that unifies \textit{SkinCAP} and \textit{SkinCoT}. \textbf{SkinCAP} comprises 4,000 images sourced from the Fitzpatrick 17k skin disease dataset and the Diverse Dermatology Images dataset, annotated by board-certified dermatologists to provide extensive medical descriptions and captions. In addition, we introduce \textbf{SkinCoT}, a curated dataset pairing 3,041 dermatologic images with clinician-verified, hierarchical chain-of-thought (CoT) diagnoses. Each diagnostic narrative is rigorously evaluated against six quality criteria and iteratively refined until it meets a predefined standard of clinical accuracy and explanatory depth. Together, SkinCAP (captioning) and SkinCoT (reasoning), collectively referred to as SkinCaRe, encompass 7,041 expertly curated dermatologic cases and provide a unified and trustworthy resource for training multimodal models that both describe and explain dermatologic images. SkinCaRe is publicly available at https://huggingface.co/datasets/yuhos16/SkinCaRe.

Method: Voxel-centric encoder architecture that learns unique embeddings per brain-voxel and uses cross-attention between voxel embeddings and multi-level deep image features to predict voxel responses.

Result: Enables combining data from multiple subjects, effective transfer learning across subjects/datasets/machines with few examples, and provides voxel-embeddings for brain functionality exploration.

Conclusion: The proposed universal brain-encoder overcomes limitations of subject/dataset-specific training and provides a powerful framework for brain encoding and functional analysis.

Abstract: Image-to-fMRI encoding is important for both neuroscience research and practical applications. However, such “Brain-Encoders” have been typically trained per-subject and per fMRI-dataset, thus restricted to very limited training data. In this paper we propose a Universal Brain-Encoder, which can be trained jointly on data from many different subjects/datasets/machines. What makes this possible is our new voxel-centric Encoder architecture, which learns a unique “voxel-embedding” per brain-voxel. Our Encoder trains to predict the response of each brain-voxel on every image, by directly computing the cross-attention between the brain-voxel embedding and multi-level deep image features. This voxel-centric architecture allows the functional role of each brain-voxel to naturally emerge from the voxel-image cross-attention. We show the power of this approach to (i) combine data from multiple different subjects (a “Crowd of Brains”) to improve each individual brain-encoding, (ii) quick & effective Transfer-Learning across subjects, datasets, and machines (e.g., 3-Tesla, 7-Tesla), with few training examples, and (iii) use the learned voxel-embeddings as a powerful tool to explore brain functionality (e.g., what is encoded where in the brain).

Method: Created InfiniBench with over 1,000 hours of video content (avg 53 minutes), 87.7K QA pairs covering 8 diverse skills (grounding-based and reasoning-based), using both multiple-choice and open-ended question formats.

Result: Models perform poorly across all skills - GPT-4o achieves only 47.1% on grounding-based skills, most models near random chance. Models show strong reliance on world knowledge from metadata rather than visual/temporal understanding. Multi-modal input substantially improves performance.

Conclusion: Current models struggle with long video understanding, highlighting the need for better temporal reasoning and visual understanding capabilities beyond pre-trained world knowledge.

Abstract: Understanding long-form videos, such as movies and TV episodes ranging from tens of minutes to two hours, remains a significant challenge for multi-modal models. Existing benchmarks often fail to test the full range of cognitive skills needed to process these temporally rich and narratively complex inputs. Therefore, we introduce InfiniBench, a comprehensive benchmark designed to evaluate the capabilities of models in long video understanding rigorously. InfiniBench offers:(1) Over 1,000 hours of video content, with an average video length of 53 minutes. (2) The largest set of question-answer pairs for long video comprehension, totaling around 87.7 K. (3) Eight diverse skills that span both grounding-based (e.g., scene transitions, character actions) and reasoning-based (e.g., deep context understanding, multi-event linking). (4) Rich annotation formats, including both multiple-choice and open-ended questions. We conducted an in-depth evaluation across both commercial (GPT-4o, Gemini 2.0 Flash) and most recent open-source vision-language models such as Qwen2.5-VL, InternVL3.0). Results reveal that:(1) Models struggle across the board: Even the best model, GPT-4o, achieves only 47.1 % on grounding-based skills, with most models performing near or just above random chance. (2) Strong reliance on world knowledge: Models achieve surprisingly high scores using only metadata (e.g., video titles), highlighting a tendency to rely on pre-trained knowledge rather than actual visual or temporal understanding. (3) Multi-Modal Importance: When provided with full video and subtitle context, however, models show substantial improvements, confirming the critical role of multimodal input in video understanding. InfiniBench is publicly available at https://vision-cair.github.io/Infinibench

Method: Proposes DeNAS-ViT with efficient NAS module for multi-scale token search before ViT attention, and NAS-guided SSL framework combining network independence and contrastive learning with stage-wise optimization.

Result: Achieves state-of-the-art performance on public datasets, maintaining robustness with minimal labeled data, and demonstrates generalization potential beyond ultrasound imaging.

Conclusion: DeNAS-ViT effectively addresses ultrasound segmentation challenges through automated architecture optimization and data-efficient learning, showing broader applicability across medical imaging domains.

Abstract: Accurate segmentation of ultrasound images is essential for reliable medical diagnoses but is challenged by poor image quality and scarce labeled data. Prior approaches have relied on manually designed, complex network architectures to improve multi-scale feature extraction. However, such handcrafted models offer limited gains when prior knowledge is inadequate and are prone to overfitting on small datasets. In this paper, we introduce DeNAS-ViT, a data-efficient NAS-optimized Vision Transformer, the first method to leverage neural architecture search (NAS) for ultrasound image segmentation by automatically optimizing model architecture through token-level search. Specifically, we propose an efficient NAS module that performs multi-scale token search prior to the ViT’s attention mechanism, effectively capturing both contextual and local features while minimizing computational costs. Given ultrasound’s data scarcity and NAS’s inherent data demands, we further develop a NAS-guided semi-supervised learning (SSL) framework. This approach integrates network independence and contrastive learning within a stage-wise optimization strategy, significantly enhancing model robustness under limited-data conditions. Extensive experiments on public datasets demonstrate that DeNAS-ViT achieves state-of-the-art performance, maintaining robustness with minimal labeled data. Moreover, we highlight DeNAS-ViT’s generalization potential beyond ultrasound imaging, underscoring its broader applicability.

Method: Proposes LMSeg with barycentric dual graph representation, Geometry Aggregation+ (GA+) module for adaptive neighborhood feature combination, and hierarchical-local dual pooling to balance global context with fine details.

Result: Achieves 75.1% mIoU on SUM, 78.4% O.A. on H3D, and 62.4% mIoU on BBW dataset, demonstrating accurate segmentation of small objects and occluded cultural heritage structures.

Conclusion: The BBW dataset and LMSeg provide a practical solution for advancing 3D mesh segmentation in cultural heritage, environmental monitoring, and urban applications with strong performance and lightweight design.

Abstract: Semantic segmentation of large-scale 3D landscape meshes is critical for geospatial analysis in complex environments, yet existing approaches face persistent challenges of scalability, end-to-end trainability, and accurate segmentation of small and irregular objects. To address these issues, we introduce the BudjBim Wall (BBW) dataset, a large-scale annotated mesh dataset derived from high-resolution LiDAR scans of the UNESCO World Heritage-listed Budj Bim cultural landscape in Victoria, Australia. The BBW dataset captures historic dry-stone wall structures that are difficult to detect under vegetation occlusion, supporting research in underrepresented cultural heritage contexts. Building on this dataset, we propose LMSeg, a deep graph message-passing network for semantic segmentation of large-scale meshes. LMSeg employs a barycentric dual graph representation of mesh faces and introduces the Geometry Aggregation+ (GA+) module, a learnable softmax-based operator that adaptively combines neighborhood features and captures high-frequency geometric variations. A hierarchical-local dual pooling integrates hierarchical and local geometric aggregation to balance global context with fine-detail preservation. Experiments on three large-scale benchmarks (SUM, H3D, and BBW) show that LMSeg achieves 75.1% mIoU on SUM, 78.4% O.A. on H3D, and 62.4% mIoU on BBW, using only 2.4M lightweight parameters. In particular, LMSeg demonstrates accurate segmentation across both urban and natural scenes-capturing small-object classes such as vehicles and high vegetation in complex city environments, while also reliably detecting dry-stone walls in dense, occluded rural landscapes. Together, the BBW dataset and LMSeg provide a practical and extensible method for advancing 3D mesh segmentation in cultural heritage, environmental monitoring, and urban applications.

Method: STARS uses a two-stage approach: first masked prediction with encoder-decoder architecture, then nearest-neighbor contrastive learning to partially tune encoder weights for better semantic cluster formation.

Result: Achieves state-of-the-art self-supervised results on NTU-60, NTU-120, and PKU-MMD benchmarks, with significantly better performance than masked prediction models in few-shot settings.

Conclusion: The proposed STARS method effectively addresses limitations of masked prediction by incorporating contrastive learning, improving cluster separation and generalization without hand-crafted data augmentations.

Abstract: Self-supervised pretraining methods with masked prediction demonstrate remarkable within-dataset performance in skeleton-based action recognition. However, we show that, unlike contrastive learning approaches, they do not produce well-separated clusters. Additionally, these methods struggle with generalization in few-shot settings. To address these issues, we propose Self-supervised Tuning for 3D Action Recognition in Skeleton sequences (STARS). Specifically, STARS first uses a masked prediction stage using an encoder-decoder architecture. It then employs nearest-neighbor contrastive learning to partially tune the weights of the encoder, enhancing the formation of semantic clusters for different actions. By tuning the encoder for a few epochs, and without using hand-crafted data augmentations, STARS achieves state-of-the-art self-supervised results in various benchmarks, including NTU-60, NTU-120, and PKU-MMD. In addition, STARS exhibits significantly better results than masked prediction models in few-shot settings, where the model has not seen the actions throughout pretraining. Project page: https://soroushmehraban.github.io/stars/

Method: Uses Combined Spherical Sweeping method with lightweight network structure, teacher-student learning with pseudo labels from stereo matching, data/model augmentation, and HexaMODE multi-view fisheye camera system.

Result: Achieves comparable accuracy to state-of-the-art with significantly less resource consumption, 15 FPS on edge platforms, and high accuracy in various complex real-world indoor/outdoor scenarios.

Conclusion: Rt-OmniMVS enables efficient real-time omnidirectional depth estimation on edge platforms with robust generalization across diverse real-world environments.

Abstract: Omnidirectional depth estimation enables efficient 3D perception over a full 360-degree range. However, in real-world applications such as autonomous driving and robotics, achieving real-time performance and robust cross-scene generalization remains a significant challenge for existing algorithms. In this paper, we propose a real-time omnidirectional depth estimation method for edge computing platforms named Rt-OmniMVS, which introduces the Combined Spherical Sweeping method and implements the lightweight network structure to achieve real-time performance on edge computing platforms. To achieve high accuracy, robustness, and generalization in real-world environments, we introduce a teacher-student learning strategy. We leverage the high-precision stereo matching method as the teacher model to predict pseudo labels for unlabeled real-world data, and utilize data and model augmentation techniques for training to enhance performance of the student model Rt-OmniMVS. We also propose HexaMODE, an omnidirectional depth sensing system based on multi-view fisheye cameras and edge computation device. A large-scale hybrid dataset contains both unlabeled real-world data and synthetic data is collected for model training. Experiments on public datasets demonstrate that proposed method achieves results comparable to state-of-the-art approaches while consuming significantly less resource. The proposed system and algorithm also demonstrate high accuracy in various complex real-world scenarios, both indoors and outdoors, achieving an inference speed of 15 frames per second on edge computing platforms.

Method: Recovers 3D features from input contactless fingerprints (3D shape model and 3D fingerprint features like minutiae and orientation), then uses a novel 3D graph matching method based on extracted 3D features.

Result: The method successfully improves matching accuracy on contactless fingerprint databases and performs stably across multiple poses due to 3D embeddings.

Conclusion: The proposed 3D-based approach provides significant advantages over previous 2D-based contactless fingerprint recognition algorithms, particularly in pose stability and matching accuracy.

Abstract: Contactless fingerprint has gained lots of attention in recent fingerprint studies. However, most existing contactless fingerprint algorithms treat contactless fingerprints as 2D plain fingerprints, and still utilize traditional contact-based 2D fingerprints recognition methods. This recognition approach lacks consideration of the modality difference between contactless and contact fingerprints, especially the intrinsic 3D features in contactless fingerprints. This paper proposes a novel contactless fingerprint recognition algorithm that captures the revealed 3D feature of contactless fingerprints rather than the plain 2D feature. The proposed method first recovers 3D features from the input contactless fingerprint, including the 3D shape model and 3D fingerprint feature (minutiae, orientation, etc.). Then, a novel 3D graph matching method is proposed according to the extracted 3D feature. Additionally, the proposed method is able to perform robust 3D feature extractions on various contactless fingerprints across multiple finger poses. The results of the experiments on contactless fingerprint databases show that the proposed method successfully improves the matching accuracy of contactless fingerprints. Exceptionally, our method performs stably across multiple poses of contactless fingerprints due to 3D embeddings, which is a great advantage compared to 2D-based previous contactless fingerprint recognition algorithms.

Method: Uses image pyramids to produce intermediate representations at multiple scales, and implements inter/intra-scale fusion to augment low-quality object super-pixels with high-quality ones from other scales.

Result: Improves mainstream Object-Centric Learning methods on standard benchmarks, including state-of-the-art diffusion-based approaches.

Conclusion: Multi-Scale Fusion effectively enhances VAE guidance for Object-Centric Learning by addressing scale variation in objects, leading to improved performance across various methods.

Abstract: Representing images or videos as object-level feature vectors, rather than pixel-level feature maps, facilitates advanced visual tasks. Object-Centric Learning (OCL) primarily achieves this by reconstructing the input under the guidance of Variational Autoencoder (VAE) intermediate representation to drive so-called \textit{slots} to aggregate as much object information as possible. However, existing VAE guidance does not explicitly address that objects can vary in pixel sizes while models typically excel at specific pattern scales. We propose \textit{Multi-Scale Fusion} (MSF) to enhance VAE guidance for OCL training. To ensure objects of all sizes fall within VAE’s comfort zone, we adopt the \textit{image pyramid}, which produces intermediate representations at multiple scales; To foster scale-invariance/variance in object super-pixels, we devise \textit{inter}/\textit{intra-scale fusion}, which augments low-quality object super-pixels of one scale with corresponding high-quality super-pixels from another scale. On standard OCL benchmarks, our technique improves mainstream methods, including state-of-the-art diffusion-based ones. The source code is available on https://github.com/Genera1Z/MultiScaleFusion.

Method: Developed HUB framework evaluating unlearning methods across six key dimensions: faithfulness, alignment, pinpoint-ness, multilingual robustness, attack robustness, and efficiency, covering 33 target concepts with 16,000 prompts per concept.

Result: Investigation reveals no single unlearning method excels across all evaluation criteria, highlighting the need for more comprehensive evaluation approaches.

Conclusion: By releasing evaluation code and dataset, the authors aim to inspire further research towards more reliable and effective unlearning methods for text-to-image diffusion models.

Abstract: As text-to-image diffusion models gain widespread commercial applications, there are increasing concerns about unethical or harmful use, including the unauthorized generation of copyrighted or sensitive content. Concept unlearning has emerged as a promising solution to these challenges by removing undesired and harmful information from the pre-trained model. However, the previous evaluations primarily focus on whether target concepts are removed while preserving image quality, neglecting the broader impacts such as unintended side effects. In this work, we propose Holistic Unlearning Benchmark (HUB), a comprehensive framework for evaluating unlearning methods across six key dimensions: faithfulness, alignment, pinpoint-ness, multilingual robustness, attack robustness, and efficiency. Our benchmark covers 33 target concepts, including 16,000 prompts per concept, spanning four categories: Celebrity, Style, Intellectual Property, and NSFW. Our investigation reveals that no single method excels across all evaluation criteria. By releasing our evaluation code and dataset, we hope to inspire further research in this area, leading to more reliable and effective unlearning methods.

Method: Decomposes features into combinatorial attributes through organized channel grouping and quantizes features into discrete representations using tuple code indexes instead of scalar indexes.

Result: GDR consistently improves both mainstream and state-of-the-art OCL methods across various datasets, demonstrating superior object separability and interpretability in visualizations.

Conclusion: Grouped Discrete Representation effectively addresses limitations of current OCL methods by preserving attribute-level information through combinatorial decomposition and tuple-based quantization.

Abstract: Object-Centric Learning (OCL) aims to discover objects in images or videos by reconstructing the input. Representative methods achieve this by reconstructing the input as its Variational Autoencoder (VAE) discrete representations, which suppress (super-)pixel noise and enhance object separability. However, these methods treat features as indivisible units, overlooking their compositional attributes, and discretize features via scalar code indexes, losing attribute-level similarities and differences. We propose Grouped Discrete Representation (GDR) for OCL. For better generalization, features are decomposed into combinatorial attributes by organized channel grouping. For better convergence, features are quantized into discrete representations via tuple code indexes. Experiments demonstrate that GDR consistently improves both mainstream and state-of-the-art OCL methods across various datasets. Visualizations further highlight GDR’s superior object separability and interpretability. The source code is available on https://github.com/Genera1Z/GroupedDiscreteRepresentation.

Method: Proposes SVDQuant: shifts outliers from activations to weights, uses high-precision low-rank branch (via SVD) to handle outliers while low-bit branch handles residuals. Co-designs Nunchaku inference engine to fuse kernels and reduce memory access.

Result: Reduces memory usage for 12B FLUX.1 models by 3.5×, achieves 3.0× speedup over W4A16 baseline on 16GB laptop GPU, and 3.1× speedup on RTX 5090 with NVFP4 precision while preserving image quality.

Conclusion: SVDQuant effectively enables aggressive 4-bit quantization for diffusion models while maintaining quality and achieving significant speedup and memory reduction through innovative outlier handling and optimized inference.

Abstract: Diffusion models can effectively generate high-quality images. However, as they scale, rising memory demands and higher latency pose substantial deployment challenges. In this work, we aim to accelerate diffusion models by quantizing their weights and activations to 4 bits. At such an aggressive level, both weights and activations are highly sensitive, where existing post-training quantization methods like smoothing become insufficient. To overcome this limitation, we propose SVDQuant, a new 4-bit quantization paradigm. Different from smoothing, which redistributes outliers between weights and activations, our approach absorbs these outliers using a low-rank branch. We first consolidate the outliers by shifting them from activations to weights. Then, we use a high-precision, low-rank branch to take in the weight outliers with Singular Value Decomposition (SVD), while a low-bit quantized branch handles the residuals. This process eases the quantization on both sides. However, naively running the low-rank branch independently incurs significant overhead due to extra data movement of activations, negating the quantization speedup. To address this, we co-design an inference engine Nunchaku that fuses the kernels of the low-rank branch into those of the low-bit branch to cut off redundant memory access. It can also seamlessly support off-the-shelf low-rank adapters (LoRAs) without re-quantization. Extensive experiments on SDXL, PixArt-$Σ$, and FLUX.1 validate the effectiveness of SVDQuant in preserving image quality. We reduce the memory usage for the 12B FLUX.1 models by 3.5$\times$, achieving 3.0$\times$ speedup over the 4-bit weight-only quantization (W4A16) baseline on the 16GB laptop 4090 GPU with INT4 precision. On the latest RTX 5090 desktop with Blackwell architecture, we achieve a 3.1$\times$ speedup compared to the W4A16 model using NVFP4 precision.

Method: Two-channel decoupling module separates shared and view-proprietary representations, plus two contrastive learning objectives on high-level features and semantic labels to filter consistent information.

Result: Extensive experiments on benchmark datasets show the method achieves more stable and superior classification performance.

Conclusion: Decoupling heterogeneous properties into different spaces with contrastive learning effectively handles double missing multi-view multi-label classification.

Abstract: Recently, multi-view and multi-label classification have become significant domains for comprehensive data analysis and exploration. However, incompleteness both in views and labels is still a real-world scenario for multi-view multi-label classification. In this paper, we seek to focus on double missing multi-view multi-label classification tasks and propose our dual-level contrastive learning framework to solve this issue. Different from the existing works, which couple consistent information and view-specific information in the same feature space, we decouple the two heterogeneous properties into different spaces and employ contrastive learning theory to fully disentangle the two properties. Specifically, our method first introduces a two-channel decoupling module that contains a shared representation and a view-proprietary representation to effectively extract consistency and complementarity information across all views. Second, to efficiently filter out high-quality consistent information from multi-view representations, two consistency objectives based on contrastive learning are conducted on the high-level features and the semantic labels, respectively. Extensive experiments on several widely used benchmark datasets demonstrate that the proposed method has more stable and superior classification performance.

Method: Disentangles human-scene interaction from motion synthesis during training, then introduces interaction-based implicit policy into motion diffusion during inference through iterative denoising and policy optimization, with motion blending in joint rotation power space for long-term synthesis.

Result: The method shows better motion naturalness and interaction plausibility than state-of-the-art methods on synthesized scenes with ShapeNet furniture and real scenes from PROX and Replica.

Conclusion: DIP demonstrates feasibility for motion synthesis in more general tasks and versatile scenes without requiring paired motion-scene data.

Abstract: Scene-aware motion synthesis has been widely researched recently due to its numerous applications. Prevailing methods rely heavily on paired motion-scene data, while it is difficult to generalize to diverse scenes when trained only on a few specific ones. Thus, we propose a unified framework, termed Diffusion Implicit Policy (DIP), for scene-aware motion synthesis, where paired motion-scene data are no longer necessary. In this paper, we disentangle human-scene interaction from motion synthesis during training, and then introduce an interaction-based implicit policy into motion diffusion during inference. Synthesized motion can be derived through iterative diffusion denoising and implicit policy optimization, thus motion naturalness and interaction plausibility can be maintained simultaneously. For long-term motion synthesis, we introduce motion blending in joint rotation power space. The proposed method is evaluated on synthesized scenes with ShapeNet furniture, and real scenes from PROX and Replica. Results show that our framework presents better motion naturalness and interaction plausibility than cutting-edge methods. This also indicates the feasibility of utilizing the DIP for motion synthesis in more general tasks and versatile scenes. Code will be publicly available at https://github.com/jingyugong/DIP.

Method: Proposes MutualVPR with unsupervised view self-classification using geographic grouping and iterative K-means clustering with DINOv2 initialization. Encoder and clustering co-evolve to separate appearance variations and enable consistent supervision.

Result: Achieves state-of-the-art performance across multiple datasets, demonstrating improved view direction generalization and occlusion robustness.

Conclusion: The mutual learning framework effectively addresses appearance variation challenges in VPR through unsupervised view classification and co-evolving encoder-clustering, providing robust localization without manual labels or rules.

Abstract: Visual Place Recognition (VPR) enables robust localization through image retrieval based on learned descriptors. However, drastic appearance variations of images at the same place caused by viewpoint changes can lead to inconsistent supervision signals, thereby degrading descriptor learning. Existing methods either rely on manually defined cropping rules or labeled data for view differentiation, but they suffer from two major limitations: (1) reliance on labels or handcrafted rules restricts generalization capability; (2) even within the same view direction, occlusions can introduce feature ambiguity. To address these issues, we propose MutualVPR, a mutual learning framework that integrates unsupervised view self-classification and descriptor learning. We first group images by geographic coordinates, then iteratively refine the clusters using K-means to dynamically assign place categories without orientation labels. Specifically, we adopt a DINOv2-based encoder to initialize the clustering. During training, the encoder and clustering co-evolve, progressively separating drastic appearance variations of the same place and enabling consistent supervision. Furthermore, we find that capturing fine-grained image differences at a place enhances robustness. Experiments demonstrate that MutualVPR achieves state-of-the-art (SOTA) performance across multiple datasets, validating the effectiveness of our framework in improving view direction generalization, occlusion robustness.

Method: Uses reflective reasoning framework with Chain-of-Thought reasoning and Reflection-Aware KL-Divergence Target Optimization (RKTO) to translate subjective user intent into structured outputs, trained on 30,000 CoT examples with human-annotated rationale quality.

Result: Achieves substantial gains in alignment with human intent and generates coherent, high-quality instructions across image, video, 3D, and 4D editing tasks.

Conclusion: EVLM provides a scalable foundation for multimodal editing and reasoning by capturing fine-grained editing preferences without binary supervision.

Abstract: Editing complex visual content from ambiguous or partially specified instructions remains a core challenge in vision-language modeling. Existing models can contextualize content but often fail to infer the underlying intent within a reference image or scene, leading to inconsistent or misaligned edits. We introduce the Editing Vision-Language Model (EVLM), a system that interprets ambiguous instructions in conjunction with reference visuals to produce precise, context-aware editing prompts. EVLM’s key innovation is a reflective reasoning framework that translates subjective user intent into structured, actionable outputs by aligning with human-rated rationales through Reflection-Aware KL-Divergence Target Optimization (RKTO). By combining Chain-of-Thought (CoT) reasoning with RKTO alignment, EVLM captures fine-grained editing preferences without relying on binary supervision. Trained on a dataset of 30,000 CoT examples with human-annotated rationale quality, EVLM achieves substantial gains in alignment with human intent. Experiments across image, video, 3D, and 4D editing tasks show that EVLM generates coherent and high-quality instructions, providing a scalable foundation for multimodal editing and reasoning.

Method: The authors examine the developmental history, define and organize various settings, track advancements, analyze related datasets and applications, and highlight advanced topics through systematic literature review.

Result: This survey represents the most comprehensive overview currently available in Visual Grounding, covering representative work from the past decade and providing standardized definitions for future research.

Conclusion: The paper outlines current challenges and proposes valuable future research directions, serving as an invaluable resource for both beginners and experienced researchers in the field.

Abstract: Visual Grounding, also known as Referring Expression Comprehension and Phrase Grounding, aims to ground the specific region(s) within the image(s) based on the given expression text. This task simulates the common referential relationships between visual and linguistic modalities, enabling machines to develop human-like multimodal comprehension capabilities. Consequently, it has extensive applications in various domains. However, since 2021, visual grounding has witnessed significant advancements, with emerging new concepts such as grounded pre-training, grounding multimodal LLMs, generalized visual grounding, and giga-pixel grounding, which have brought numerous new challenges. In this survey, we first examine the developmental history of visual grounding and provide an overview of essential background knowledge. We systematically track and summarize the advancements, and then meticulously define and organize the various settings to standardize future research and ensure a fair comparison. Additionally, we delve into numerous related datasets and applications, and highlight several advanced topics. Finally, we outline the challenges confronting visual grounding and propose valuable directions for future research, which may serve as inspiration for subsequent researchers. By extracting common technical details, this survey encompasses the representative work in each subtopic over the past decade. To the best of our knowledge, this paper represents the most comprehensive overview currently available in the field of visual grounding. This survey is designed to be suitable for both beginners and experienced researchers, serving as an invaluable resource for understanding key concepts and tracking the latest research developments. We keep tracing related work at https://github.com/linhuixiao/Awesome-Visual-Grounding.

Method: Proposes LWGANet with two core innovations: Top-K Global Feature Interaction (TGFI) module to mitigate spatial redundancy by focusing computation on salient regions, and Light-Weight Grouped Attention (LWGA) module to resolve channel redundancy by partitioning channels into scale-specific pathways.

Result: Extensive experiments on twelve diverse datasets across four major RS tasks show LWGANet consistently outperforms state-of-the-art lightweight backbones in both accuracy and efficiency.

Conclusion: LWGANet establishes a new robust baseline for efficient visual analysis in remote sensing images by synergistically resolving core inefficiencies in spatial and channel redundancy.

Abstract: Light-weight neural networks for remote sensing (RS) visual analysis must overcome two inherent redundancies: spatial redundancy from vast, homogeneous backgrounds, and channel redundancy, where extreme scale variations render a single feature space inefficient. Existing models, often designed for natural images, fail to address this dual challenge in RS scenarios. To bridge this gap, we propose LWGANet, a light-weight backbone engineered for RS-specific properties. LWGANet introduces two core innovations: a Top-K Global Feature Interaction (TGFI) module that mitigates spatial redundancy by focusing computation on salient regions, and a Light-Weight Grouped Attention (LWGA) module that resolves channel redundancy by partitioning channels into specialized, scale-specific pathways. By synergistically resolving these core inefficiencies, LWGANet achieves a superior trade-off between feature representation quality and computational cost. Extensive experiments on twelve diverse datasets across four major RS tasks–scene classification, oriented object detection, semantic segmentation, and change detection–demonstrate that LWGANet consistently outperforms state-of-the-art light-weight backbones in both accuracy and efficiency. Our work establishes a new, robust baseline for efficient visual analysis in RS images.

Method: Reframes T2M from frequency perspective, identifies semantic planning (low-frequency) and fine-grained execution (high-frequency) stages, introduces frequency-domain temporal-adaptive module for stage-specific alignment.

Result: Dramatically improves motion quality and semantic correctness; reduces FID from 0.152 to 0.060 on StableMoFusion baseline, establishing new state-of-the-art.

Conclusion: Frequency-domain insights are critical for generating robust and reliable motions, enabling more intuitive natural language control of robots.

Abstract: Enabling humanoid robots to synthesize complex, physically coherent motions from natural language commands is a cornerstone of autonomous robotics and human-robot interaction. While diffusion models have shown promise in this text-to-motion (T2M) task, they often generate semantically flawed or unstable motions, limiting their applicability to real-world robots. This paper reframes the T2M problem from a frequency-domain perspective, revealing that the generative process mirrors a hierarchical control paradigm. We identify two critical phases: a semantic planning stage, where low-frequency components establish the global motion trajectory, and a fine-grained execution stage, where high-frequency details refine the movement. To address the distinct challenges of each phase, we introduce Frequency enhanced text-to-motion (Free-T2M), a framework incorporating stage-specific frequency-domain consistency alignment. We design a frequency-domain temporal-adaptive module to modulate the alignment effects of different frequency bands. These designs enforce robustness in the foundational semantic plan and enhance the accuracy of detailed execution. Extensive experiments show our method dramatically improves motion quality and semantic correctness. Notably, when applied to the StableMoFusion baseline, Free-T2M reduces the FID from 0.152 to 0.060, establishing a new state-of-the-art within diffusion architectures. These findings underscore the critical role of frequency-domain insights for generating robust and reliable motions, paving the way for more intuitive natural language control of robots.

Method: DES transforms unsafe embeddings from text encoders toward safe embedding regions and neutralizes “nudity” embedding by aligning it with neutral embedding.

Result: Achieved ASR of 9.47% on FLUX.1 (76.5% reduction) and 0.52% on Stable Diffusion v1.5 (63.9% reduction) compared to previous SOTA methods, while maintaining comparable FID and CLIP scores.

Conclusion: DES provides state-of-the-art defense against unsafe content generation in diffusion models while preserving benign image quality and robustness against adversarial attacks.

Abstract: Diffusion models show remarkable image generation performance following text prompts, but risk generating sexual contents. Existing approaches, such as prompt filtering, concept removal, and even sexual contents mitigation methods, struggle to defend against adversarial attacks while maintaining benign image quality. In this paper, we propose a novel approach called Distorting Embedding Space (DES), a text encoder-based defense mechanism that effectively tackles these issues through innovative embedding space control. DES transforms unsafe embeddings, extracted from a text encoder using unsafe prompts, toward carefully calculated safe embedding regions to prevent unsafe contents generation, while reproducing the original safe embeddings. DES also neutralizes the ``nudity’’ embedding, by aligning it with neutral embedding to enhance robustness against adversarial attacks. As a result, extensive experiments on explicit content mitigation and adaptive attack defense show that DES achieves state-of-the-art (SOTA) defense, with attack success rate (ASR) of 9.47% on FLUX.1, a recent popular model, and 0.52% on the widely adopted Stable Diffusion v1.5. These correspond to ASR reductions of 76.5% and 63.9% compared to previous SOTA methods, EraseAnything and AdvUnlearn, respectively. Furthermore, DES maintains benign image quality, achieving Frechet Inception Distance and CLIP score comparable to those of the original FLUX.1 and Stable Diffusion v1.5.

Method: Uses a generalizable scene discriminator to estimate feature density, extracts LiDAR intensity and depth features from multiple perspectives, employs dual-path correspondence matching (DPCM) for cross-modal feature matching, and formulates calibration as a joint optimization problem with global constraints.

Result: Extensive experiments show EdO-LCEC outperforms state-of-the-art methods, especially in challenging scenarios with sparse point clouds or partially overlapping sensor views.

Conclusion: EdO-LCEC provides a robust, target-free solution for LiDAR-camera extrinsic calibration that adapts to varying environmental conditions and achieves superior performance compared to existing approaches.

Abstract: LiDAR-camera extrinsic calibration (LCEC) is crucial for multi-modal data fusion in autonomous robotic systems. Existing methods, whether target-based or target-free, typically rely on customized calibration targets or fixed scene types, which limit their applicability in real-world scenarios. To address these challenges, we present EdO-LCEC, the first environment-driven online calibration approach. Unlike traditional target-free methods, EdO-LCEC employs a generalizable scene discriminator to estimate the feature density of the application environment. Guided by this feature density, EdO-LCEC extracts LiDAR intensity and depth features from varying perspectives to achieve higher calibration accuracy. To overcome the challenges of cross-modal feature matching between LiDAR and camera, we introduce dual-path correspondence matching (DPCM), which leverages both structural and textural consistency for reliable 3D-2D correspondences. Furthermore, we formulate the calibration process as a joint optimization problem that integrates global constraints across multiple views and scenes, thereby enhancing overall accuracy. Extensive experiments on real-world datasets demonstrate that EdO-LCEC outperforms state-of-the-art methods, particularly in scenarios involving sparse point clouds or partially overlapping sensor views.

Method: Uses transferred image embeddings as conditional inputs, stepwise increasing interpolation between latents, and feedback-driven mechanism to update auxiliary latents in reverse order.

Result: Significantly improves semantic coherence and visual quality of blended images, producing compelling and coherent results.

Conclusion: FreeBlend effectively addresses challenges in concept blending through its training-free framework with progressive blending and feedback mechanisms.

Abstract: Concept blending is a promising yet underexplored area in generative models. While recent approaches, such as embedding mixing and latent modification based on structural sketches, have been proposed, they often suffer from incompatible semantic information and discrepancies in shape and appearance. In this work, we introduce FreeBlend, an effective, training-free framework designed to address these challenges. To mitigate cross-modal loss and enhance feature detail, we leverage transferred image embeddings as conditional inputs. The framework employs a stepwise increasing interpolation strategy between latents, progressively adjusting the blending ratio to seamlessly integrate auxiliary features. Additionally, we introduce a feedback-driven mechanism that updates the auxiliary latents in reverse order, facilitating global blending and preventing rigid or unnatural outputs. Extensive experiments demonstrate that our method significantly improves both the semantic coherence and visual quality of blended images, yielding compelling and coherent results.

Method: Few-shot finetuning of diffusion models to inject articulation awareness, using text prompts to generate motion samples and image prompting from 3D objects to personalize motion, then transferring motion to 3D space via differentiable rendering with score distillation sampling.

Result: Experiments on PartNet-Mobility and ACD datasets show ATOP generates realistic motion samples with higher accuracy and better generalization in few-shot settings compared to prior approaches.

Conclusion: ATOP effectively addresses the challenge of 3D object articulation with limited motion data by combining diffusion models, personalization techniques, and differentiable rendering for improved motion prediction.

Abstract: We present ATOP (Articulate That Object Part), a novel few-shot method based on motion personalization to articulate a static 3D object with respect to a part and its motion as prescribed in a text prompt. Given the scarcity of available datasets with motion attribute annotations, existing methods struggle to generalize well in this task. In our work, the text input allows us to tap into the power of modern-day diffusion models to generate plausible motion samples for the right object category and part. In turn, the input 3D object provides ``image prompting’’ to personalize the generated motion to the very input object. Our method starts with a few-shot finetuning to inject articulation awareness to current diffusion models to learn a unique motion identifier associated with the target object part. Our finetuning is applied to a pre-trained diffusion model for controllable multi-view motion generation, trained with a small collection of reference motion frames demonstrating appropriate part motion. The resulting motion model can then be employed to realize plausible motion of the input 3D object from multiple views. At last, we transfer the personalized motion to the 3D space of the object via differentiable rendering to optimize part articulation parameters by a score distillation sampling loss. Experiments on PartNet-Mobility and ACD datasets demonstrate that our method can generate realistic motion samples with higher accuracy, leading to more generalizable 3D motion predictions compared to prior approaches in the few-shot setting.

Method: Proposes two key components: (1) Flow-Guided Temporal Aggregation module that aligns and aggregates temporal features using optical flow to capture motion-aware context and deformable structures; (2) Occlusion-Guided Voxel Refinement module that injects occlusion masks and temporal features into 3D voxel space for adaptive voxel refinement.

Result: Achieves state-of-the-art performance on SemanticKITTI and SSCBench-KITTI-360 benchmarks, demonstrating significant improvement in 3D scene completion accuracy.

Conclusion: FlowScene effectively addresses temporal consistency challenges in 3D semantic scene completion by leveraging optical flow guidance, enabling better integration of motion, viewpoints, and occlusions for improved autonomous driving perception.

Abstract: 3D Semantic Scene Completion (SSC) provides comprehensive scene geometry and semantics for autonomous driving perception, which is crucial for enabling accurate and reliable decision-making. However, existing SSC methods are limited to capturing sparse information from the current frame or naively stacking multi-frame temporal features, thereby failing to acquire effective scene context. These approaches ignore critical motion dynamics and struggle to achieve temporal consistency. To address the above challenges, we propose a novel temporal SSC method FlowScene: Learning Temporal 3D Semantic Scene Completion via Optical Flow Guidance. By leveraging optical flow, FlowScene can integrate motion, different viewpoints, occlusions, and other contextual cues, thereby significantly improving the accuracy of 3D scene completion. Specifically, our framework introduces two key components: (1) a Flow-Guided Temporal Aggregation module that aligns and aggregates temporal features using optical flow, capturing motion-aware context and deformable structures; and (2) an Occlusion-Guided Voxel Refinement module that injects occlusion masks and temporally aggregated features into 3D voxel space, adaptively refining voxel representations for explicit geometric modeling. Experimental results demonstrate that FlowScene achieves state-of-the-art performance on the SemanticKITTI and SSCBench-KITTI-360 benchmarks.

Method: Introduces ReBind, a lightweight framework that teaches role bindings using carefully selected active/passive intermediate compositions through simple fine-tuning to gradually mitigate role collapse.

Result: Intermediate compositions through fine-tuning significantly reduce role collapse, with humans preferring ReBind more than 78% compared to state-of-the-art methods.

Conclusion: Distributional asymmetries play a key role in compositional failures, and directional decomposition offers a simple, effective path for improving generalization in T2I models.

Abstract: Text-to-image (T2I) diffusion models exhibit impressive photorealistic image generation capabilities, yet they struggle in compositional image generation. In this work, we introduce RoleBench, a benchmark focused on evaluating compositional generalization in action-based relations (e.g., “mouse chasing cat”). We show that state-of-the-art T2I models and compositional generation methods consistently default to frequent reversed relations (i.e., “cat chasing mouse”), a phenomenon we call role collapse. Related works attribute this to the model’s architectural limitation or underrepresentation in the data. Our key insight reveals that while models fail on rare compositions when their inversions are common, they can successfully generate similar intermediate compositions (e.g., “mouse chasing boy”), suggesting that this limitation is also due to the presence of frequent counterparts rather than just the absence of rare compositions. Motivated by this, we hypothesize that directional decomposition can gradually mitigate role collapse. We test this via ReBind, a lightweight framework that teaches role bindings using carefully selected active/passive intermediate compositions. Experiments suggest that intermediate compositions through simple fine-tuning can significantly reduce role collapse, with humans preferring ReBind more than 78% compared to state-of-the-art methods. Our findings highlight the role of distributional asymmetries in compositional failures and offer a simple, effective path for improving generalization.

Method: Knowledge distillation framework with efficient large-scale training procedure robust to occlusions, and novel loss formulation handling weak supervision from non-distinctive teacher descriptors.

Result: Significant reduction in inference time while maintaining competitive performance on five BOP Benchmark datasets, bringing zero-shot 6D pose estimation closer to real-time feasibility.

Conclusion: The proposed knowledge distillation approach successfully bridges the gap between GeDi’s effectiveness and practical efficiency, making zero-shot 6D pose estimation more viable for real-world applications.

Abstract: Three-dimensional local descriptors are crucial for encoding geometric surface properties, making them essential for various point cloud understanding tasks. Among these descriptors, GeDi has demonstrated strong zero-shot 6D pose estimation capabilities but remains computationally impractical for real-world applications due to its expensive inference process. Can we retain GeDi’s effectiveness while significantly improving its efficiency? In this paper, we explore this question by introducing a knowledge distillation framework that trains an efficient student model to regress local descriptors from a GeDi teacher. Our key contributions include: an efficient large-scale training procedure that ensures robustness to occlusions and partial observations while operating under compute and storage constraints, and a novel loss formulation that handles weak supervision from non-distinctive teacher descriptors. We validate our approach on five BOP Benchmark datasets and demonstrate a significant reduction in inference time while maintaining competitive performance with existing methods, bringing zero-shot 6D pose estimation closer to real-time feasibility. Project Website: https://tev-fbk.github.io/dGeDi/

Method: Sequential Spatio-Temporal Attention Network with hierarchical stacked design: Spatial MHA for intra-frame joints, Temporal MHA for inter-frame dependencies.

Result: Achieved SOTA performance on fingerspelling categories (JSL, KSL) and new SOTA for skeleton-only methods on WLASL, outperforming self-supervised pre-training approaches.

Conclusion: SSTAN demonstrates high data efficiency and effectiveness in capturing complex sign language dynamics without predefined graph structures.

Abstract: Hand gesture-based Sign Language Recognition (SLR) serves as a crucial communication bridge between deaf and non-deaf individuals. While Graph Convolutional Networks (GCNs) are common, they are limited by their reliance on fixed skeletal graphs. To overcome this, we propose the Sequential Spatio-Temporal Attention Network (SSTAN), a novel Transformer-based architecture. Our model employs a hierarchical, stacked design that sequentially integrates Spatial Multi-Head Attention (MHA) to capture intra-frame joint relationships and Temporal MHA to model long-range inter-frame dependencies. This approach allows the model to efficiently learn complex spatio-temporal patterns without predefined graph structures. We validated our model through extensive experiments on diverse, large-scale datasets (WLASL, JSL, and KSL). A key finding is that our model, trained entirely from scratch, achieves state-of-the-art (SOTA) performance in the challenging fingerspelling categories (JSL and KSL). Furthermore, it establishes a new SOTA for skeleton-only methods on WLASL, outperforming several approaches that rely on complex self-supervised pre-training. These results demonstrate our model’s high data efficiency and its effectiveness in capturing the intricate dynamics of sign language. The official implementation is available at our GitHub repository: \href{https://github.com/K-Hirooka-Aizu/skeleton-slr-transformer}{https://github.com/K-Hirooka-Aizu/skeleton-slr-transformer}.

Method: Generate multi-view pseudo segmentation masks and text embeddings using MLLMs, unproject 2D masks to 3D, align with text embeddings using spatial consistency strategy and Token-for-Query approach for multimodal semantic alignment.

Result: Outperforms existing 3D reasoning segmentation methods on indoor scene benchmarks without labeled 3D training data, effectively interpreting user intent and reasoning about spatial relationships.

Conclusion: MLLM-For3D successfully transfers 2D MLLM knowledge to 3D reasoning segmentation through spatial consistency and cross-view alignment, enabling zero-shot 3D scene understanding.

Abstract: Reasoning segmentation aims to segment target objects in complex scenes based on human intent and spatial reasoning. While recent multimodal large language models (MLLMs) have demonstrated impressive 2D image reasoning segmentation, adapting these capabilities to 3D scenes remains underexplored. In this paper, we introduce MLLM-For3D, a simple yet effective framework that transfers knowledge from 2D MLLMs to 3D scene understanding. Specifically, we utilize MLLMs to generate multi-view pseudo segmentation masks and corresponding text embeddings, then unproject 2D masks into 3D space and align them with the text embeddings. The primary challenge lies in the absence of 3D context and spatial consistency across multiple views, causing the model to hallucinate objects that do not exist and fail to target objects consistently. Training the 3D model with such irrelevant objects leads to performance degradation. To address this, we introduce a spatial consistency strategy to enforce that segmentation masks remain coherent in the 3D space, effectively capturing the geometry of the scene. Moreover, we develop a Token-for-Query approach for multimodal semantic alignment, enabling consistent identification of the same object across different views. Extensive evaluations on various challenging indoor scene benchmarks demonstrate that, even without any labeled 3D training data, MLLM-For3D outperforms existing 3D reasoning segmentation methods, effectively interpreting user intent, understanding 3D scenes, and reasoning about spatial relationships.

Method: LangBridge explicitly maps visual tokens to linear combinations of LLM vocabulary embeddings, grounding visual representations directly in the language model’s embedding space.

Result: LangBridge adapters pre-trained on smaller models (Qwen2-0.5B) can be directly applied to larger models (LLaMA3-8B, Qwen2.5-14B) with competitive performance and nearly no degradation.

Conclusion: LangBridge provides interpretable vision-language alignment and enables efficient plug-and-play transfer across multiple LLMs without requiring retraining.

Abstract: Recent years have witnessed remarkable advances in Large Vision-Language Models (LVLMs), which have achieved human-level performance across various complex vision-language tasks. Following LLaVA’s paradigm, mainstream LVLMs typically employ a shallow MLP for visual-language alignment through a two-stage training process: pretraining for cross-modal alignment followed by instruction tuning. While this approach has proven effective, the underlying mechanisms of how MLPs bridge the modality gap remain poorly understood. Although some research has explored how LLMs process transformed visual tokens, few studies have investigated the fundamental alignment mechanism. Furthermore, the MLP adapter requires retraining whenever switching LLM backbones. To address these limitations, we first investigate the working principles of MLP adapters and discover that they learn to project visual embeddings into subspaces spanned by corresponding text embeddings progressively. Based on this insight, we propose LangBridge, a novel adapter that explicitly maps visual tokens to linear combinations of LLM vocabulary embeddings. This innovative design enables pretraining-free adapter transfer across different LLMs while maintaining performance. Our experimental results demonstrate that a LangBridge adapter pre-trained on Qwen2-0.5B can be directly applied to larger models such as LLaMA3-8B or Qwen2.5-14B while maintaining competitive performance. Overall, LangBridge enables interpretable vision-language alignment by grounding visual representations in LLM vocab embedding, while its plug-and-play design ensures efficient reuse across multiple LLMs with nearly no performance degradation. See our project page at https://curryx-001.github.io/LangBridge.github.io/

Method: Uses a channel selection algorithm based on model output differences to identify redundant channels, then selectively replaces them with more effective ones to enhance task-specific feature representation without parameter updates.

Result: Experiments on out-of-domain and in-domain datasets show the method is efficient and effective across various vision tasks (segmentation, depth estimation, classification) and can integrate with existing fine-tuning strategies like LoRA and Adapter to boost performance.

Conclusion: The parameter-free approach offers a new perspective on fine-tuning foundation models by focusing on feature selection and reuse rather than parameter modification, significantly reducing GPU memory overhead while maintaining or improving performance.

Abstract: Vision foundation models (VFMs) have demonstrated remarkable capabilities in learning universal visual representations. However, adapting these models to downstream tasks conventionally requires parameter updates, with even parameter-efficient fine-tuning methods necessitating the modification of thousands to millions of weights. In this paper, we investigate the redundancies in the segment anything model (SAM) and then propose a novel parameter-free fine-tuning method. Unlike traditional fine-tuning methods that adjust parameters, our method emphasizes selecting, reusing, and enhancing pre-trained features, offering a new perspective on fine-tuning foundation models. Specifically, we introduce a channel selection algorithm based on the model’s output difference to identify redundant and effective channels. By selectively replacing the redundant channels with more effective ones, we filter out less useful features and reuse more task-irrelevant features to downstream tasks, thereby enhancing the task-specific feature representation. Experiments on both out-of-domain and in-domain datasets demonstrate the efficiency and effectiveness of our method in different vision tasks (e.g., image segmentation, depth estimation and image classification). Notably, our approach can seamlessly integrate with existing fine-tuning strategies (e.g., LoRA, Adapter), further boosting the performance of already fine-tuned models. Moreover, since our channel selection involves only model inference, our method significantly reduces GPU memory overhead.

Method: AGO encodes images and class prompts into 3D and text embeddings, uses similarity-based grounding training with 3D pseudo-labels, and employs a modality adapter to align 3D embeddings with VLM-derived image embeddings.

Result: AGO improves unknown object prediction in zero-shot and few-shot transfer and achieves state-of-the-art closed-world self-supervised performance, surpassing prior methods by 4.09 mIoU on Occ3D-nuScenes.

Conclusion: The proposed AGO framework effectively addresses modality gaps and enables robust open-world 3D semantic occupancy prediction through adaptive grounding.

Abstract: Open-world 3D semantic occupancy prediction aims to generate a voxelized 3D representation from sensor inputs while recognizing both known and unknown objects. Transferring open-vocabulary knowledge from vision-language models (VLMs) offers a promising direction but remains challenging. However, methods based on VLM-derived 2D pseudo-labels with traditional supervision are limited by a predefined label space and lack general prediction capabilities. Direct alignment with pretrained image embeddings, on the other hand, often fails to achieve reliable performance because of inconsistent image and text representations in VLMs. To address these challenges, we propose AGO, a novel 3D occupancy prediction framework with adaptive grounding to handle diverse open-world scenarios. AGO first encodes surrounding images and class prompts into 3D and text embeddings, respectively, leveraging similarity-based grounding training with 3D pseudo-labels. Additionally, a modality adapter maps 3D embeddings into a space aligned with VLM-derived image embeddings, reducing modality gaps. Experiments on Occ3D-nuScenes show that AGO improves unknown object prediction in zero-shot and few-shot transfer while achieving state-of-the-art closed-world self-supervised performance, surpassing prior methods by 4.09 mIoU. Code is available at: https://github.com/EdwardLeeLPZ/AGO.

Method: Developed a video-language model that processes full video sequences from five standard echocardiographic views, trained on 60,747 echocardiographic video-report pairs. Evaluated gains from video input and multi-view support with various pretrained models.

Result: The model demonstrated improved retrieval performance by leveraging video input and multi-view support compared to single-frame approaches.

Conclusion: Video-language models that process full echocardiographic video sequences from multiple standard views can enhance diagnostic accuracy by capturing cardiac motion and leveraging complementary information from different views.

Abstract: Echocardiography records ultrasound videos of the heart, enabling clinicians to assess cardiac function. Recent advances in large-scale vision-language models (VLMs) have spurred interest in automating echocardiographic interpretation. However, most existing medical VLMs rely on single-frame (image) inputs, which can reduce diagnostic accuracy for conditions identifiable only through cardiac motion. In addition, echocardiographic videos are captured from multiple views, each varying in suitability for detecting specific conditions. Leveraging multiple views may therefore improve diagnostic performance. We developed a video-language model that processes full video sequences from five standard views, trained on 60,747 echocardiographic video-report pairs. We evaluated the gains in retrieval performance from video input and multi-view support, including the contributions of various pretrained models. Code and model weights are available at https://github.com/UTcardiology/video-echo-clip

Method: Three core modules: 1) ICE captures inter-sample correlation using semantically similar unpaired text-video combinations; 2) IRM mitigates intra-sample redundancy by distinguishing relevant from redundant moments; 3) TCP enhances temporal learning by predicting original order of shuffled frames/moments.

Result: Extensive experiments demonstrate superior performance compared to prior methods, achieving state-of-the-art results in partially relevant video retrieval.

Conclusion: The proposed framework effectively addresses semantic asymmetry in PRVR by systematically exploiting inter-sample correlation and intra-sample redundancy, leading to significant performance improvements over existing approaches.

Abstract: Partially Relevant Video Retrieval (PRVR) aims to retrieve the target video that is partially relevant to the text query. The primary challenge in PRVR arises from the semantic asymmetry between textual and visual modalities, as videos often contain substantial content irrelevant to the query. Existing methods coarsely align paired videos and text queries to construct the semantic space, neglecting the critical cross-modal dual nature inherent in this task: inter-sample correlation and intra-sample redundancy. To this end, we propose a novel PRVR framework to systematically exploit these two characteristics. Our framework consists of three core modules. First, the Inter Correlation Enhancement (ICE) module captures inter-sample correlation by identifying semantically similar yet unpaired text queries and video moments, combining them to form pseudo-positive pairs for more robust semantic space construction. Second, the Intra Redundancy Mining (IRM) module mitigates intra-sample redundancy by mining redundant moment features and distinguishing them from query-relevant moments, encouraging the model to learn more discriminative representations. Finally, to reinforce these modules, we introduce the Temporal Coherence Prediction (TCP) module, which enhances temporal structure learning by training the model to predict the original temporal order of randomly shuffled video frames and moments. Extensive experiments demonstrate the superiority of our approach compared to prior methods, achieving state-of-the-art results.

Method: Developed DeepAndes - a transformer-based vision foundation model using customized DINOv2 self-supervised learning algorithm optimized for 8-band multi-spectral satellite imagery. Trained on 3 million images and evaluated through imbalanced image classification, image instance retrieval, and pixel-level semantic segmentation tasks.

Result: DeepAndes achieves superior F1 scores, mean average precision, and Dice scores in few-shot learning scenarios, significantly outperforming models trained from scratch or pre-trained on smaller datasets.

Conclusion: Large-scale self-supervised pre-training is highly effective for archaeological remote sensing applications, and DeepAndes represents the first foundation model specifically designed for the Andes region.

Abstract: By mapping sites at large scales using remotely sensed data, archaeologists can generate unique insights into long-term demographic trends, inter-regional social networks, and past adaptations to climate change. Remote sensing surveys complement field-based approaches, and their reach can be especially great when combined with deep learning and computer vision techniques. However, conventional supervised deep learning methods face challenges in annotating fine-grained archaeological features at scale. While recent vision foundation models have shown remarkable success in learning large-scale remote sensing data with minimal annotations, most off-the-shelf solutions are designed for RGB images rather than multi-spectral satellite imagery, such as the 8-band data used in our study. In this paper, we introduce DeepAndes, a transformer-based vision foundation model trained on three million multi-spectral satellite images, specifically tailored for Andean archaeology. DeepAndes incorporates a customized DINOv2 self-supervised learning algorithm optimized for 8-band multi-spectral imagery, marking the first foundation model designed explicitly for the Andes region. We evaluate its image understanding performance through imbalanced image classification, image instance retrieval, and pixel-level semantic segmentation tasks. Our experiments show that DeepAndes achieves superior F1 scores, mean average precision, and Dice scores in few-shot learning scenarios, significantly outperforming models trained from scratch or pre-trained on smaller datasets. This underscores the effectiveness of large-scale self-supervised pre-training in archaeological remote sensing. Codes will be available on https://github.com/geopacha/DeepAndes.

Method: Uses MLLM parser to extract structured semantics, spatial reasoning for pose initialization, hierarchical refinement with spatial semantics, and appearance enhancement using inserted-object images.

Result: Achieves semantically coherent, spatially precise, and visually realistic 3D insertions without spatial priors.

Conclusion: FreeInsert offers a user-friendly and flexible editing experience for text-driven 3D scene editing.

Abstract: Text-driven object insertion in 3D scenes is an emerging task that enables intuitive scene editing through natural language. However, existing 2D editing-based methods often rely on spatial priors such as 2D masks or 3D bounding boxes, and they struggle to ensure consistency of the inserted object. These limitations hinder flexibility and scalability in real-world applications. In this paper, we propose FreeInsert, a novel framework that leverages foundation models including MLLMs, LGMs, and diffusion models to disentangle object generation from spatial placement. This enables unsupervised and flexible object insertion in 3D scenes without spatial priors. FreeInsert starts with an MLLM-based parser that extracts structured semantics, including object types, spatial relationships, and attachment regions, from user instructions. These semantics guide both the reconstruction of the inserted object for 3D consistency and the learning of its degrees of freedom. We leverage the spatial reasoning capabilities of MLLMs to initialize object pose and scale. A hierarchical, spatially aware refinement stage further integrates spatial semantics and MLLM-inferred priors to enhance placement. Finally, the appearance of the object is improved using the inserted-object image to enhance visual fidelity. Experimental results demonstrate that FreeInsert achieves semantically coherent, spatially precise, and visually realistic 3D insertions without relying on spatial priors, offering a user-friendly and flexible editing experience.

Method: Uses cumulative TF-IDF to formulate sentence descriptiveness scores, then leverages these scores to refine false negative labeling and build precise matching by aligning sentences in generic-to-specific order.

Result: Extensive experiments on MS-COCO, Flickr30K, and CxC datasets demonstrate superior performance in representing complex image-text relationships compared to state-of-the-art methods, with enhanced hierarchical reasoning ability.

Conclusion: DITM effectively addresses limitations of binary supervision by incorporating graded contextual similarity and language descriptiveness, enabling better discovery of optimal matches and potential positive pairs in image-text matching.

Abstract: Image-text matching aims to build correspondences between visual and textual data by learning their pairwise similarities. Most existing approaches have adopted sparse binary supervision, indicating whether a pair of images and sentences matches or not. However, such sparse supervision covers a limited subset of image-text relationships, neglecting their inherent many-to-many correspondences; an image can be described in numerous texts at different descriptive levels. Moreover, existing approaches overlook the implicit connections from general to specific descriptions, which form the underlying rationale for the many-to-many relationships between vision and language. In this work, we propose descriptive image-text matching, called DITM, to learn the graded contextual similarity between image and text by exploring the descriptive flexibility of language. We formulate the descriptiveness score of each sentence with cumulative term frequency-inverse document frequency (TF-IDF) to balance the pairwise similarity according to the keywords in the sentence. Our method leverages sentence descriptiveness to learn robust image-text matching in two key ways: (1) to refine the false negative labeling, dynamically relaxing the connectivity between positive and negative pairs, and (2) to build more precise matching, aligning a set of relevant sentences in a generic-to-specific order. By moving beyond rigid binary supervision, DITM enhances the discovery of both optimal matches and potential positive pairs. Extensive experiments on MS-COCO, Flickr30K, and CxC datasets demonstrate the effectiveness of our method in representing complex image-text relationships compared to state-of-the-art approaches. In addition, DITM enhances the hierarchical reasoning ability of the model, supported by the extensive analysis on HierarCaps benchmark.

Method: Used a semi-automated annotation pipeline with domain expert validation to create 32,000 image-question pairs across 7 alignment principles (fairness, ethics, empathy, inclusivity, reasoning, robustness, multilinguality) through diverse VQA tasks.

Result: Benchmarking revealed distinct behavioral patterns - some models excel in reasoning, fairness, and multilinguality while others show better robustness and grounding, but most struggle to balance task accuracy with ethical/inclusive responses. Chain-of-thought prompting and test-time scaling improved alignment.

Conclusion: HumaniBench provides the first rigorous testbed for HC evaluation, enabling diagnosis of limitations, quantification of alignment trade-offs, and promoting responsible LMM development through transparent, reproducible evaluation.

Abstract: Large multimodal models (LMMs) have achieved impressive performance on vision-language tasks such as visual question answering (VQA), image captioning, and visual grounding; however, they remain insufficiently evaluated for alignment with human-centered (HC) values such as fairness, ethics, and inclusivity. To address this gap, we introduce HumaniBench, a comprehensive benchmark comprising 32,000 real-world image-question pairs and an accompanying evaluation suite. Using a semi-automated annotation pipeline, each sample is rigorously validated by domain experts to ensure accuracy and ethical integrity. HumaniBench assesses LMMs across seven key alignment principles: fairness, ethics, empathy, inclusivity, reasoning, robustness, and multilinguality through a diverse set of open- and closed-ended VQA tasks. Grounded in AI ethics theory and real-world social contexts, these principles provide a holistic lens for examining human-aligned behavior. Benchmarking results reveal distinct behavioral patterns: certain model families excel in reasoning, fairness, and multilinguality, while others demonstrate greater robustness and grounding capability. However, most models still struggle to balance task accuracy with ethical and inclusive responses. Techniques such as chain-of-thought prompting and test-time scaling yield measurable alignment gains. As the first benchmark explicitly designed for HC evaluation, HumaniBench offers a rigorous testbed to diagnose limitations, quantify alignment trade-offs, and promote the responsible development of large multimodal models. All data and code are publicly released to ensure transparency and reproducibility. https://vectorinstitute.github.io/HumaniBench/

Method: Proposes PSDiffusion with a global layer interaction mechanism that leverages image composition priors from pre-trained image diffusion models for simultaneous multi-layer text-to-image generation, ensuring individual layer quality and coherent spatial/visual relationships.

Result: Extensive experiments on benchmark datasets show PSDiffusion outperforms existing methods in generating multi-layer images with plausible structure and enhanced visual fidelity.

Conclusion: PSDiffusion successfully addresses the limitations of previous approaches by enabling simultaneous multi-layer generation with improved global context modeling and visual quality.

Abstract: Transparent image layer generation plays a significant role in digital art and design workflows. Existing methods typically decompose transparent layers from a single RGB image using a set of tools or generate multiple transparent layers sequentially. Despite some promising results, these methods often limit their ability to model global layout, physically plausible interactions, and visual effects such as shadows and reflections with high alpha quality due to limited shared global context among layers. To address this issue, we propose PSDiffusion, a unified diffusion framework that leverages image composition priors from pre-trained image diffusion model for simultaneous multi-layer text-to-image generation. Specifically, our method introduces a global layer interaction mechanism to generate layered images collaboratively, ensuring both individual layer quality and coherent spatial and visual relationships across layers. We include extensive experiments on benchmark datasets to demonstrate that PSDiffusion is able to outperform existing methods in generating multi-layer images with plausible structure and enhanced visual fidelity.

Method: Uses a dual-stream 4D radar backbone with multi-scale geometric features and clustering-based class-aware features, followed by neural network motion flow estimation and Gauss-Newton pose refinement through differentiable optimization.

Result: Outperforms recent classical and learning-based approaches on VoD and Snail-Radar datasets, achieving results comparable to A-LOAM with LiDAR mapping optimization.

Conclusion: The proposed DNOI-4DRO model successfully integrates learning and optimization for 4D radar odometry, demonstrating superior performance and potential for practical applications.

Abstract: A novel learning-optimization-combined 4D radar odometry model, named DNOI-4DRO, is proposed in this paper. The proposed model seamlessly integrates traditional geometric optimization with end-to-end neural network training, leveraging an innovative differentiable neural-optimization iteration operator. In this framework, point-wise motion flow is first estimated using a neural network, followed by the construction of a cost function based on the relationship between point motion and pose in 3D space. The radar pose is then refined using Gauss-Newton updates. Additionally, we design a dual-stream 4D radar backbone that integrates multi-scale geometric features and clustering-based class-aware features to enhance the representation of sparse 4D radar point clouds. Extensive experiments on the VoD and Snail-Radar datasets demonstrate the superior performance of our model, which outperforms recent classical and learning-based approaches. Notably, our method even achieves results comparable to A-LOAM with mapping optimization using LiDAR point clouds as input. Our models and code will be publicly released.

Method: Proposed DiTF framework that leverages AdaLN to adaptively localize and normalize massive activations through channel-wise modulation, plus a channel discard strategy to mitigate adverse effects.

Result: DiTF outperforms DINO and SD-based models, achieving +9.4% on Spair-71k and +4.4% on AP-10K-C.S., establishing new SOTA for DiTs in visual correspondence.

Conclusion: The AdaLN-based DiTF framework effectively addresses massive activations in DiTs, enabling extraction of semantically discriminative features and superior performance in visual correspondence tasks.

Abstract: Pre-trained stable diffusion models (SD) have shown great advances in visual correspondence. In this paper, we investigate the capabilities of Diffusion Transformers (DiTs) for accurate dense correspondence. Distinct from SD, DiTs exhibit a critical phenomenon in which very few feature activations exhibit significantly larger values than others, known as \textit{massive activations}, leading to uninformative representations and significant performance degradation for DiTs. The massive activations consistently concentrate at very few fixed dimensions across all image patch tokens, holding little local information. We analyze these dimension-concentrated massive activations and uncover that their concentration is inherently linked to the Adaptive Layer Normalization (AdaLN) in DiTs. Building on these findings, we propose the \textbf{Di}ffusion \textbf{T}ransformer \textbf{F}eature (DiTF), a training-free AdaLN-based framework that extracts semantically discriminative features from DiTs. Specifically, DiTF leverages AdaLN to adaptively localize and normalize massive activations through channel-wise modulation. Furthermore, a channel discard strategy is introduced to mitigate the adverse effects of massive activations. Experimental results demonstrate that our DiTF outperforms both DINO and SD-based models and establishes a new state-of-the-art performance for DiTs in different visual correspondence tasks (\eg, with +9.4% on Spair-71k and +4.4% on AP-10K-C.S.).

Method: Two-stage pipeline: 1) Reinforcement learning for rapid and optimized text layout generation (bounding box prediction), 2) Diffusion-based image synthesis model. The RL approach accelerates layout generation while reducing overlaps.

Result: Achieves comparable performance to TextDiffuser-2 in text placement and image synthesis quality, while being 42.29 times faster and requiring only 2 MB of CPU RAM for inference. Can run on CPU-only systems unlike TextDiffuser-2.

Conclusion: The proposed framework offers a more efficient and flexible solution for text-embedded image generation, maintaining high quality while significantly improving speed and deployment flexibility across different hardware platforms.

Abstract: Text-embedded image generation plays a critical role in industries such as graphic design, advertising, and digital content creation. Text-to-Image generation methods leveraging diffusion models, such as TextDiffuser-2, have demonstrated promising results in producing images with embedded text. TextDiffuser-2 effectively generates bounding box layouts that guide the rendering of visual text, achieving high fidelity and coherence. However, existing approaches often rely on resource-intensive processes and are limited in their ability to run efficiently on both CPU and GPU platforms. To address these challenges, we propose a novel two-stage pipeline that integrates reinforcement learning (RL) for rapid and optimized text layout generation with a diffusion-based image synthesis model. Our RL-based approach significantly accelerates the bounding box prediction step while reducing overlaps, allowing the system to run efficiently on both CPUs and GPUs. Extensive evaluations demonstrate that our framework achieves comparable performance to TextDiffuser-2 in terms of text placement and image synthesis, while offering markedly faster runtime and increased flexibility. Our method produces high-quality images comparable to TextDiffuser-2, while being 42.29 times faster and requiring only 2 MB of CPU RAM for inference, unlike TextDiffuser-2’s M1 model, which is not executable on CPU-only systems.

Method: Decouples semantic and geometric learning tasks. Semantic branch distills 2D foundation models for aligned pseudo labels. Geometric branch integrates image and LiDAR inputs with semi-supervision. Fuses features through Dual Mamba and uses scatter-accumulated projection for supervision.

Result: Achieves competitive performance with only 10% of voxel annotations on SemanticKITTI and Occ3D-nuScenes datasets.

Conclusion: OccLE provides an effective label-efficient solution for 3D semantic occupancy prediction that balances performance and annotation cost.

Abstract: 3D semantic occupancy prediction offers an intuitive and efficient scene understanding and has attracted significant interest in autonomous driving perception. Existing approaches either rely on full supervision, which demands costly voxel-level annotations, or on self-supervision, which provides limited guidance and yields suboptimal performance. To address these challenges, we propose OccLE, a Label-Efficient 3D Semantic Occupancy Prediction that takes images and LiDAR as inputs and maintains high performance with limited voxel annotations. Our intuition is to decouple the semantic and geometric learning tasks and then fuse the learned feature grids from both tasks for the final semantic occupancy prediction. Therefore, the semantic branch distills 2D foundation model to provide aligned pseudo labels for 2D and 3D semantic learning. The geometric branch integrates image and LiDAR inputs in cross-plane synergy based on their inherency, employing semi-supervision to enhance geometry learning. We fuse semantic-geometric feature grids through Dual Mamba and incorporate a scatter-accumulated projection to supervise unannotated prediction with aligned pseudo labels. Experiments show that OccLE achieves competitive performance with only 10% of voxel annotations on the SemanticKITTI and Occ3D-nuScenes datasets. The code will be publicly released on https://github.com/NerdFNY/OccLE

Method: Multi-Level and Multi-Prompt (MLMP) entropy minimization that integrates features from intermediate vision-encoder layers and uses different text-prompt templates at both global CLS token and local pixel-wise levels.

Result: Consistently delivers significant gains over direct adoption of TTA classification baselines across 87 distinct test scenarios including 9 datasets, 15 corruptions, and domain shifts.

Conclusion: The method is plug-and-play, requires no additional training data or labels, works with single test samples, and establishes a comprehensive benchmark for future TTA research in open-vocabulary segmentation.

Abstract: Recently, test-time adaptation has attracted wide interest in the context of vision-language models for image classification. However, to the best of our knowledge, the problem is completely overlooked in dense prediction tasks such as Open-Vocabulary Semantic Segmentation (OVSS). In response, we propose a novel TTA method tailored to adapting VLMs for segmentation during test time. Unlike TTA methods for image classification, our Multi-Level and Multi-Prompt (MLMP) entropy minimization integrates features from intermediate vision-encoder layers and is performed with different text-prompt templates at both the global CLS token and local pixel-wise levels. Our approach could be used as plug-and-play for any segmentation network, does not require additional training data or labels, and remains effective even with a single test sample. Furthermore, we introduce a comprehensive OVSS TTA benchmark suite, which integrates a rigorous evaluation protocol, nine segmentation datasets, 15 common synthetic corruptions, and additional real and rendered domain shifts, \textbf{with a total of 87 distinct test scenarios}, establishing a standardized and comprehensive testbed for future TTA research in open-vocabulary segmentation. Our experiments on this suite demonstrate that our segmentation-tailored method consistently delivers significant gains over direct adoption of TTA classification baselines. Code and data are available at https://github.com/dosowiechi/MLMP.

Method: Created VideoCAD dataset with 41K annotated CAD operation videos using automated framework, and proposed VideoCADFormer model for learning UI interactions from video.

Result: VideoCAD offers 20x longer time horizons than existing datasets. VideoCADFormer outperforms behavior cloning baselines for CAD interaction learning.

Conclusion: VideoCAD reveals key challenges in video-based UI understanding including precise action grounding, multimodal reasoning, and long-horizon dependencies.

Abstract: Computer-Aided Design (CAD) is a time-consuming and complex process, requiring precise, long-horizon user interactions with intricate 3D interfaces. While recent advances in AI-driven user interface (UI) agents show promise, most existing datasets and methods focus on short, low-complexity tasks in mobile or web applications, failing to capture the demands of professional engineering tools. In this work, we introduce VideoCAD, the first attempt to model UI interactions for precision engineering tasks. Specifically, VideoCAD is a large-scale synthetic dataset consisting of over 41K annotated video recordings of CAD operations, generated using an automated framework for collecting high-fidelity UI action data from human-made CAD designs. Compared to existing datasets, VideoCAD offers an order-of-magnitude increase in complexity for real-world engineering UI tasks, with time horizons up to 20x longer than those in other datasets. We show two important downstream applications of VideoCAD: (1) learning UI interactions from professional 3D CAD tools for precision tasks and (2) a visual question-answering (VQA) benchmark designed to evaluate multimodal large language models (LLMs) on spatial reasoning and video understanding. To learn the UI interactions, we propose VideoCADFormer, a state-of-the-art model for learning CAD interactions directly from video, which outperforms existing behavior cloning baselines. Both VideoCADFormer and the VQA benchmark derived from VideoCAD reveal key challenges in the current state of video-based UI understanding, including the need for precise action grounding, multi-modal and spatial reasoning, and long-horizon dependencies.

Method: Decouples segmentation masks into regional semantic components (Semantic Alignment Attention Mask) and high-frequency shape components (Entity Contour Map), and uses Attribute Isolation Attention Mask to prevent cross-entity attribute leakage.

Result: Achieves state-of-the-art performance on both open-set and closed-set S2I benchmarks, particularly excelling in fine-grained spatial and attribute control of entities.

Conclusion: Seg2Any effectively addresses the limitations of existing S2I methods by ensuring both semantic and shape consistency while preventing attribute leakage, enabling precise spatial layout control in text-to-image generation.

Abstract: Despite recent advances in diffusion models, top-tier text-to-image (T2I) models still struggle to achieve precise spatial layout control, i.e. accurately generating entities with specified attributes and locations. Segmentation-mask-to-image (S2I) generation has emerged as a promising solution by incorporating pixel-level spatial guidance and regional text prompts. However, existing S2I methods fail to simultaneously ensure semantic consistency and shape consistency. To address these challenges, we propose Seg2Any, a novel S2I framework built upon advanced multimodal diffusion transformers (e.g. FLUX). First, to achieve both semantic and shape consistency, we decouple segmentation mask conditions into regional semantic and high-frequency shape components. The regional semantic condition is introduced by a Semantic Alignment Attention Mask, ensuring that generated entities adhere to their assigned text prompts. The high-frequency shape condition, representing entity boundaries, is encoded as an Entity Contour Map and then introduced as an additional modality via multi-modal attention to guide image spatial structure. Second, to prevent attribute leakage across entities in multi-entity scenarios, we introduce an Attribute Isolation Attention Mask mechanism, which constrains each entity’s image tokens to attend exclusively to themselves during image self-attention. To support open-set S2I generation, we construct SACap-1M, a large-scale dataset containing 1 million images with 5.9 million segmented entities and detailed regional captions, along with a SACap-Eval benchmark for comprehensive S2I evaluation. Extensive experiments demonstrate that Seg2Any achieves state-of-the-art performance on both open-set and closed-set S2I benchmarks, particularly in fine-grained spatial and attribute control of entities.

Method: Proposes FaceSleuth-R framework with novel Single-Orientation Attention (SOA) module - a lightweight, differentiable operator that learns layer-specific optimal orientations to guide attention towards robust motion cues.

Result: SOA discovers universal near-vertical motion prior across datasets; FaceSleuth-R achieves superior generalization in Leave-One-Dataset-Out protocols and establishes state-of-the-art results across multiple benchmarks.

Conclusion: Adaptive orientation-aware attention is crucial for developing truly generalized and high-performing micro-expression recognition systems that can handle domain shifts effectively.

Abstract: Micro-expression recognition (MER) has achieved impressive accuracy in controlled laboratory settings. However, its real-world applicability faces a significant generalization cliff, severely hindering practical deployment due to poor performance on unseen data and susceptibility to domain shifts. Existing attention mechanisms often overfit to dataset-specific appearance cues or rely on fixed spatial priors, making them fragile in diverse environments. We posit that robust MER requires focusing on quasi-invariant motion orientations inherent to micro-expressions, rather than superficial pixel-level features. To this end, we introduce \textbf{FaceSleuth-R}, a framework centered on our novel \textbf{Single-Orientation Attention (SOA)} module. SOA is a lightweight, differentiable operator that enables the network to learn layer-specific optimal orientations, effectively guiding attention towards these robust motion cues. Through extensive experiments, we demonstrate that SOA consistently discovers a universal near-vertical motion prior across diverse datasets. More critically, FaceSleuth-R showcases superior generalization in rigorous Leave-One-Dataset-Out (LODO) protocols, significantly outperforming baselines and state-of-the-art methods when confronted with domain shifts. Furthermore, our approach establishes \textbf{state-of-the-art results} across several benchmarks. This work highlights adaptive orientation-aware attention as a key paradigm for developing truly generalized and high-performing MER systems.

Method: Simultaneously trains two neural networks with collaborative purification via Co-Pseudo-Labeling, enforcing consistency regularization in both label and feature spaces with anti-overfitting mechanisms.

Result: Developed an effective approach to handle instance-dependent noise patterns from VLMs through collaborative learning and multi-space consistency constraints.

Conclusion: The Co-Reg method successfully addresses the unique challenges of learning from VLM-generated noisy partial labels, enabling annotation-free training for downstream tasks.

Abstract: In the context of noisy partial label learning (NPLL), each training sample is associated with a set of candidate labels annotated by multiple noisy annotators. With the emergence of high-performance pre-trained vision-language models (VLMs) such as CLIP, LLaVA and GPT-4V, the direction of using these models to replace time-consuming manual annotation workflows and achieve manual-annotation-free" training for downstream tasks has become a highly promising research avenue. This paper focuses on learning from noisy partial labels annotated by pre-trained VLMs and proposes an innovative collaborative consistency regularization (Co-Reg) method. Unlike the symmetric noise primarily addressed in traditional noisy label learning, the noise generated by pre-trained models is instance-dependent, embodying the underlying patterns of the pre-trained models themselves, which significantly increases the learning difficulty for the model. To address this, we simultaneously train two neural networks that implement collaborative purification of training labels through a Co-Pseudo-Labeling" mechanism, while enforcing consistency regularization constraints in both the label space and feature representation space. Specifically, we construct multiple anti-overfitting mechanisms that efficiently mine latent information from noisy partially labeled samples including alternating optimization of contrastive feature representations and pseudo-labels, as well as maintaining prototypical class vectors in the shared feature space.

Method: Proposed LGM-Pose with lightweight MobileViM Block using Lightweight Attentional Representation Module (LARM) with Non-Parametric Transformation Operation for global information, and Shuffle-Integrated Fusion Module (SFusion) for multi-scale integration.

Result: Experimental evaluations on COCO and MPII datasets show reduced parameters compared to existing lightweight methods while achieving superior performance and faster processing speeds.

Conclusion: The proposed LGM-Pose network effectively addresses global context capture and latency issues in lightweight pose estimation through innovative single-branch architecture with attention and fusion modules.

Abstract: Most of the current top-down multi-person pose estimation lightweight methods are based on multi-branch parallel pure CNN network architecture, which often struggle to capture the global context required for detecting semantically complex keypoints and are hindered by high latency due to their intricate and redundant structures. In this article, an approximate single-branch lightweight global modeling network (LGM-Pose) is proposed to address these challenges. In the network, a lightweight MobileViM Block is designed with a proposed Lightweight Attentional Representation Module (LARM), which integrates information within and between patches using the Non-Parametric Transformation Operation(NPT-Op) to extract global information. Additionally, a novel Shuffle-Integrated Fusion Module (SFusion) is introduced to effectively integrate multi-scale information, mitigating performance degradation often observed in single-branch structures. Experimental evaluations on the COCO and MPII datasets demonstrate that our approach not only reduces the number of parameters compared to existing mainstream lightweight methods but also achieves superior performance and faster processing speeds.

Method: Proposes BiLIE with Dynamic Adaptive Filtering Enhancement (DAFE) for suppressing flickering artifacts and preserving high-frequency details, and Bidirectional Guided Awareness Fusion (BGAF) for breakpoint-aware restoration and structure-aware enhancement through cross-modal attention.

Result: Outperforms existing methods on RELIE and LIE datasets, achieving 0.81dB higher PSNR on RELIE with significant improvements in edge restoration, color fidelity, and noise suppression.

Conclusion: BiLIE effectively addresses low-light enhancement challenges through bidirectional image-event fusion, producing clear, smooth, and structurally intact results while introducing a high-quality dataset (RELIE) for better evaluation.

Abstract: Under extreme low-light conditions, frame-based cameras suffer from severe detail loss due to limited dynamic range. Recent studies have introduced event cameras for event-guided low-light image enhancement. However, existing approaches often overlook the flickering artifacts and structural discontinuities caused by dynamic illumination changes and event sparsity. To address these challenges, we propose BiLIE, a Bidirectional image-event guided fusion framework for Low-Light Image Enhancement, which achieves mutual guidance and complementary enhancement between the two modalities. First, to highlight edge details, we develop a Dynamic Adaptive Filtering Enhancement (DAFE) module that performs adaptive high-pass filtering on event representations to suppress flickering artifacts and preserve high-frequency information under varying illumination. Subsequently, we design a Bidirectional Guided Awareness Fusion (BGAF) mechanism, which achieves breakpoint-aware restoration from images to events and structure-aware enhancement from events to images through a two-stage attention mechanism, establishing cross-modal consistency, thereby producing a clear, smooth, and structurally intact fused representation. Moreover, recognizing that existing datasets exhibit insufficient ground-truth fidelity and color accuracy, we construct a high-quality low-light image-event dataset (RELIE) via a reliable ground truth refinement scheme. Extensive experiments demonstrate that our method outperforms existing approaches on both the RELIE and LIE datasets. Notably, on RELIE, BiLIE exceeds the state-of-the-art by 0.81dB in PSNR and shows significant advantages in edge restoration, color fidelity, and noise suppression.

Method: Uses autoregressive rollout with key-value caching during training to condition each frame’s generation on previously self-generated outputs, employs few-step diffusion with stochastic gradient truncation for efficiency, and implements rolling KV cache for efficient video extrapolation.

Result: Achieves real-time streaming video generation with sub-second latency on a single GPU while matching or surpassing the quality of significantly slower non-causal diffusion models.

Conclusion: Self Forcing effectively addresses exposure bias in video diffusion models, enabling efficient and high-quality real-time video generation through its novel training paradigm.

Abstract: We introduce Self Forcing, a novel training paradigm for autoregressive video diffusion models. It addresses the longstanding issue of exposure bias, where models trained on ground-truth context must generate sequences conditioned on their own imperfect outputs during inference. Unlike prior methods that denoise future frames based on ground-truth context frames, Self Forcing conditions each frame’s generation on previously self-generated outputs by performing autoregressive rollout with key-value (KV) caching during training. This strategy enables supervision through a holistic loss at the video level that directly evaluates the quality of the entire generated sequence, rather than relying solely on traditional frame-wise objectives. To ensure training efficiency, we employ a few-step diffusion model along with a stochastic gradient truncation strategy, effectively balancing computational cost and performance. We further introduce a rolling KV cache mechanism that enables efficient autoregressive video extrapolation. Extensive experiments demonstrate that our approach achieves real-time streaming video generation with sub-second latency on a single GPU, while matching or even surpassing the generation quality of significantly slower and non-causal diffusion models. Project website: http://self-forcing.github.io/

Method: Uses zigzag sampling mechanism with asymmetric prompting to retain subject characteristics and visual sharing module to transfer visual cues across generated images.

Result: Significantly outperforms previous approaches in generating coherent and consistent visual stories based on both quantitative metrics and qualitative evaluations.

Conclusion: The proposed training-free sampling strategy effectively enhances subject consistency in visual story generation without requiring fine-tuning.

Abstract: Text-to-image generation models have made significant progress in producing high-quality images from textual descriptions, yet they continue to struggle with maintaining subject consistency across multiple images, a fundamental requirement for visual storytelling. Existing methods attempt to address this by either fine-tuning models on large-scale story visualization datasets, which is resource-intensive, or by using training-free techniques that share information across generations, which still yield limited success. In this paper, we introduce a novel training-free sampling strategy called Zigzag Sampling with Asymmetric Prompts and Visual Sharing to enhance subject consistency in visual story generation. Our approach proposes a zigzag sampling mechanism that alternates between asymmetric prompting to retain subject characteristics, while a visual sharing module transfers visual cues across generated images to %further enforce consistency. Experimental results, based on both quantitative metrics and qualitative evaluations, demonstrate that our method significantly outperforms previous approaches in generating coherent and consistent visual stories. The code is available at https://github.com/Mingxiao-Li/Asymmetry-Zigzag-StoryDiffusion.

Method: Developed an efficient toolbox to collect, pre-process and annotate first-person videos with location, scene, weather, crowd density, captions, and camera trajectories from 750 cities worldwide.

Result: Created a high-quality dataset with comprehensive annotations that demonstrates scale, diversity, and effectiveness for training video generation models.

Conclusion: Sekai will benefit video generation and world exploration research, enabling valuable applications in interactive world exploration.

Abstract: Video generation techniques have made remarkable progress, promising to be the foundation of interactive world exploration. However, existing video generation datasets are not well-suited for world exploration training as they suffer from some limitations: limited locations, short duration, static scenes, and a lack of annotations about exploration and the world. In this paper, we introduce Sekai (meaning “world” in Japanese), a high-quality first-person view worldwide video dataset with rich annotations for world exploration. It consists of over 5,000 hours of walking or drone view (FPV and UVA) videos from over 100 countries and regions across 750 cities. We develop an efficient and effective toolbox to collect, pre-process and annotate videos with location, scene, weather, crowd density, captions, and camera trajectories. Comprehensive analyses and experiments demonstrate the dataset’s scale, diversity, annotation quality, and effectiveness for training video generation models. We believe Sekai will benefit the area of video generation and world exploration, and motivate valuable applications. The project page is https://lixsp11.github.io/sekai-project/.

Method: Two-stage approach: 1) Stereo matching for static regions (backgrounds) using cross-frame matching, 2) Video depth diffusion for dynamic regions to ensure smooth transitions. The synergy is mathematically validated through frequency domain analysis.

Result: Achieves state-of-the-art performance on zero-shot, real-world dynamic video depth benchmarks (both indoor and outdoor), demonstrating superior consistency and accuracy in video depth estimation.

Conclusion: StereoDiff effectively synergizes stereo matching and video depth diffusion, capturing complementary strengths for static and dynamic regions respectively, leading to improved video depth estimation performance.

Abstract: Recent video depth estimation methods achieve great performance by following the paradigm of image depth estimation, i.e., typically fine-tuning pre-trained video diffusion models with massive data. However, we argue that video depth estimation is not a naive extension of image depth estimation. The temporal consistency requirements for dynamic and static regions in videos are fundamentally different. Consistent video depth in static regions, typically backgrounds, can be more effectively achieved via stereo matching across all frames, which provides much stronger global 3D cues. While the consistency for dynamic regions still should be learned from large-scale video depth data to ensure smooth transitions, due to the violation of triangulation constraints. Based on these insights, we introduce StereoDiff, a two-stage video depth estimator that synergizes stereo matching for mainly the static areas with video depth diffusion for maintaining consistent depth transitions in dynamic areas. We mathematically demonstrate how stereo matching and video depth diffusion offer complementary strengths through frequency domain analysis, highlighting the effectiveness of their synergy in capturing the advantages of both. Experimental results on zero-shot, real-world, dynamic video depth benchmarks, both indoor and outdoor, demonstrate StereoDiff’s SoTA performance, showcasing its superior consistency and accuracy in video depth estimation.

Method: VISER augments visual inputs with low-level spatial structures and pairs them with textual prompts that encourage sequential, spatially-aware parsing, using only single-query inference.

Result: VISER improved GPT-4o performance by 25.0% on visual search, 26.8% on counting, and 9.5% on spatial relationship tasks, while reducing edit distance error in scene description by 0.32 on 2D datasets. Visual modifications proved essential, as purely textual strategies were insufficient or even degraded performance.

Conclusion: Visual input design is more important than purely linguistic reasoning strategies for enhancing compositional and spatial reasoning in LVLMs, with visual structuring being a powerful general approach.

Abstract: Despite progress in Large Vision-Language Models (LVLMs), their capacity for visual reasoning is often limited by the binding problem: the failure to reliably associate perceptual features with their correct visual referents. This limitation underlies persistent errors in tasks such as counting, visual search, scene description, and spatial relationship understanding. A key factor is that current LVLMs process visual features largely in parallel, lacking mechanisms for spatially grounded, serial attention. This paper introduces Visual Input Structure for Enhanced Reasoning (VISER), a simple, effective method that augments visual inputs with low-level spatial structures and pairs them with a textual prompt that encourages sequential, spatially-aware parsing. We empirically demonstrate substantial performance improvements across core visual reasoning tasks, using only a single-query inference. Specifically, VISER improves GPT-4o performance on visual search, counting, and spatial relationship tasks by 25.0%, 26.8%, and 9.5%, respectively, and reduces edit distance error in scene description by 0.32 on 2D datasets. Furthermore, we find that the visual modification is essential for these gains; purely textual strategies, including Chain-of-Thought prompting, are insufficient and can even degrade performance. VISER underscores the importance of visual input design over purely linguistically based reasoning strategies and suggests that visual structuring is a powerful and general approach for enhancing compositional and spatial reasoning in LVLMs.

Method: Three key innovations: 1) HFESNet backbone preserving high-frequency details and semantic context, 2) ESOP for efficient high-resolution feature fusion to boost small object detection, 3) SQR and GAPE components for decoder stability and localization precision with spatial priors.

Result: On VisDrone dataset, achieves +3.8% AP and +5.1% AP50 gain over baseline while reducing parameters by 4M and maintaining real-time speeds, demonstrating superior accuracy-efficiency balance for dense small object detection.

Conclusion: HEDS-DETR effectively bridges the performance gap in aerial object detection through holistic enhancements, providing a competitive solution for detecting dense and small objects in UAV imagery with real-time capability.

Abstract: Object detection in Unmanned Aerial Vehicle (UAV) imagery is fundamentally challenged by a prevalence of small, densely packed, and occluded objects within cluttered backgrounds. Conventional detectors struggle with this domain, as they rely on hand-crafted components like pre-defined anchors and heuristic-based Non-Maximum Suppression (NMS), creating a well-known performance bottleneck in dense scenes. Even recent end-to-end frameworks have not been purpose-built to overcome these specific aerial challenges, resulting in a persistent performance gap. To bridge this gap, we introduce HEDS-DETR, a holistically enhanced real-time Detection Transformer tailored for aerial scenes. Our framework features three key innovations. First, we propose a novel High-Frequency Enhanced Semantics Network (HFESNet) backbone, which yields highly discriminative features by preserving critical high-frequency details alongside robust semantic context. Second, our Efficient Small Object Pyramid (ESOP) counteracts information loss by efficiently fusing high-resolution features, significantly boosting small object detection. Finally, we enhance decoder stability and localization precision with two synergistic components: Selective Query Recollection (SQR) and Geometry-Aware Positional Encoding (GAPE), which stabilize optimization and provide explicit spatial priors for dense object arrangements. On the VisDrone dataset, HEDS-DETR achieves a +3.8% AP and +5.1% AP50 gain over its baseline while reducing parameters by 4M and maintaining real-time speeds. This demonstrates a highly competitive accuracy-efficiency balance, especially for detecting dense and small objects in aerial scenes.

Method: Integrates EVA Vision Transformer (ViT) with Llama 2 Large Language Model (LLM), where ViT converts X-ray images into tokens that are processed by LLM along with engineered prompts for joint disease classification and localization using transfer learning.

Result: Achieved strong global disease classification performance on VinDr-CXR dataset with F1 score of 0.76, and successfully localized pathologies by generating bounding boxes around regions of interest.

Conclusion: ChestGPT provides an effective assistive tool that can reduce radiologists’ workload by offering preliminary findings and regions of interest to support diagnostic processes.

Abstract: The global demand for radiologists is increasing rapidly due to a growing reliance on medical imaging services, while the supply of radiologists is not keeping pace. Advances in computer vision and image processing technologies present significant potential to address this gap by enhancing radiologists’ capabilities and improving diagnostic accuracy. Large language models (LLMs), particularly generative pre-trained transformers (GPTs), have become the primary approach for understanding and generating textual data. In parallel, vision transformers (ViTs) have proven effective at converting visual data into a format that LLMs can process efficiently. In this paper, we present ChestGPT, a deep-learning framework that integrates the EVA ViT with the Llama 2 LLM to classify diseases and localize regions of interest in chest X-ray images. The ViT converts X-ray images into tokens, which are then fed, together with engineered prompts, into the LLM, enabling joint classification and localization of diseases. This approach incorporates transfer learning techniques to enhance both explainability and performance. The proposed method achieved strong global disease classification performance on the VinDr-CXR dataset, with an F1 score of 0.76, and successfully localized pathologies by generating bounding boxes around the regions of interest. We also outline several task-specific prompts, in addition to general-purpose prompts, for scenarios radiologists might encounter. Overall, this framework offers an assistive tool that can lighten radiologists’ workload by providing preliminary findings and regions of interest to facilitate their diagnostic process.

Method: The survey uses a systematic research methodology to identify state-of-the-art works and presents a taxonomy-based organization of VHGR approaches, categorizing methods by input modality, task type, and application domain across three primary tasks: static gesture recognition, isolated dynamic gestures, and continuous gesture recognition.

Result: The survey successfully organizes the VHGR field into a structured framework, identifies architectural trends and learning strategies for each task type, reviews commonly used datasets, and presents standard performance metrics for experimental evaluation.

Conclusion: The survey identifies major challenges in VHGR including both general computer vision issues and domain-specific obstacles, and outlines promising directions for future research to advance the field.

Abstract: The rapid evolution of deep learning (DL) models and the ever-increasing size of available datasets have raised the interest of the research community in the always important field of visual hand gesture recognition (VHGR), and delivered a wide range of applications, such as sign language understanding and human-computer interaction using cameras. Despite the large volume of research works in the field, a structured and complete survey on VHGR is still missing, leaving researchers to navigate through hundreds of papers in order to find the right combination of data, model, and approach for each task. The current survey aims to fill this gap by presenting a comprehensive overview of this computer vision field. With a systematic research methodology that identifies the state-of-the-art works and a structured presentation of the various methods, datasets, and evaluation metrics, this review aims to constitute a useful guideline for researchers, helping them to choose the right strategy for handling a VHGR task. Starting with the methodology used to locate the related literature, the survey identifies and organizes the key VHGR approaches in a taxonomy-based format, and presents the various dimensions that affect the final method choice, such as input modality, task type, and application domain. The state-of-the-art techniques are grouped across three primary VHGR tasks: static gesture recognition, isolated dynamic gestures, and continuous gesture recognition. For each task, the architectural trends and learning strategies are listed. To support the experimental evaluation of future methods in the field, the study reviews commonly used datasets and presents the standard performance metrics. Our survey concludes by identifying the major challenges in VHGR, including both general computer vision issues and domain-specific obstacles, and outlines promising directions for future research.

Method: Proposed CiCi framework that exploits spatio-temporal consistency in frequency domain and inconsistency in time domain of BVP signals. Uses expert knowledge-based self-supervised learning with gradient dynamic control to mitigate conflicts between priors.

Result: Extensive experiments on five datasets show the method consistently outperforms existing techniques, achieving state-of-the-art performance in real-time self-supervised adaptation.

Conclusion: The CiCi framework enables effective test-time adaptation for RPM tasks by leveraging physiological priors about signal consistency and inconsistency, providing stable adaptation without source data access.

Abstract: Remote physiological measurement (RPM) has emerged as a promising non-invasive method for monitoring physiological signals using the non-contact device. Although various domain adaptation and generalization methods were proposed to promote the adaptability of deep-based RPM models in unseen deployment environments, considerations in aspects such as privacy concerns and real-time adaptation restrict their application in real-world deployment. Thus, we aim to propose a novel fully Test-Time Adaptation (TTA) strategy tailored for RPM tasks in this work. Specifically, based on prior knowledge in physiology and our observations, we noticed not only there is spatio-temporal consistency in the frequency domain of BVP signals, but also that inconsistency in the time domain was significant. Given this, by leveraging both consistency and inconsistency priors, we introduce an innovative expert knowledge-based self-supervised \textbf{C}onsistency-\textbf{i}n\textbf{C}onsistency-\textbf{i}ntegration (\textbf{CiCi}) framework to enhances model adaptation during inference. Besides, our approach further incorporates a gradient dynamic control mechanism to mitigate potential conflicts between priors, ensuring stable adaptation across instances. Through extensive experiments on five diverse datasets under the TTA protocol, our method consistently outperforms existing techniques, presenting state-of-the-art performance in real-time self-supervised adaptation without accessing source data. The code will be released later.

Method: Created EvReID dataset with multi-season, multi-scene data collection. Proposed TriPro-ReID framework that combines RGB frames, event streams, and pedestrian attributes through contrastive learning to enhance feature representation.

Result: Extensive experiments on EvReID and MARS datasets validated the effectiveness of the proposed RGB-Event person ReID framework. The dataset provides a solid foundation for future research with 15 state-of-the-art algorithms evaluated.

Conclusion: The EvReID dataset addresses data scarcity in event-based person ReID, and the TriPro-ReID framework effectively leverages RGB, event streams, and pedestrian attributes for improved re-identification performance.

Abstract: Recent researchers have proposed using event cameras for person re-identification (ReID) due to their promising performance and better balance in terms of privacy protection, event camera-based person ReID has attracted significant attention. Currently, mainstream event-based person ReID algorithms primarily focus on fusing visible light and event stream, as well as preserving privacy. Although significant progress has been made, these methods are typically trained and evaluated on small-scale or simulated event camera datasets, making it difficult to assess their real identification performance and generalization ability. To address the issue of data scarcity, this paper introduces a large-scale RGB-event based person ReID dataset, called EvReID. The dataset contains 118,988 image pairs and covers 1200 pedestrian identities, with data collected across multiple seasons, scenes, and lighting conditions. We also evaluate 15 state-of-the-art person ReID algorithms, laying a solid foundation for future research in terms of both data and benchmarking. Based on our newly constructed dataset, this paper further proposes a pedestrian attribute-guided contrastive learning framework to enhance feature learning for person re-identification, termed TriPro-ReID. This framework not only effectively explores the visual features from both RGB frames and event streams, but also fully utilizes pedestrian attributes as mid-level semantic features. Extensive experiments on the EvReID dataset and MARS datasets fully validated the effectiveness of our proposed RGB-Event person ReID framework. The benchmark dataset and source code will be released on https://github.com/Event-AHU/Neuromorphic_ReID

Method: Four-stage framework: 1) Quantile-based patch filtering for prognostically informative regions, 2) Graph-regularized patch clustering for phenotype variations, 3) Hierarchical feature aggregation for multiscale tumor organization, 4) Expert-guided mixture density model for survival distribution estimation.

Result: Achieved concordance indices of 0.653 (TCGA LUAD), 0.719 (TCGA KIRC), and 0.733 (TCGA BRCA), surpassing state-of-the-art approaches in survival prediction from WSIs.

Conclusion: The proposed modular framework effectively captures morphological heterogeneity and enables accurate cancer survival prediction directly from whole slide images, outperforming existing methods across multiple cancer types.

Abstract: We propose a modular framework for predicting cancer specific survival directly from whole slide pathology images (WSIs). The framework consists of four key stages designed to capture prognostic and morphological heterogeneity. First, a Quantile Based Patch Filtering module selects prognostically informative tissue regions through quantile thresholding. Second, Graph Regularized Patch Clustering models phenotype level variations using a k nearest neighbor graph that enforces spatial and morphological coherence. Third, Hierarchical Feature Aggregation learns both intra and inter cluster dependencies to represent multiscale tumor organization. Finally, an Expert Guided Mixture Density Model estimates complex survival distributions via Gaussian mixtures, enabling fine grained risk prediction. Evaluated on TCGA LUAD, TCGA KIRC, and TCGA BRCA cohorts, our model achieves concordance indices of 0.653 ,0.719 ,and 0.733 respectively, surpassing existing state of the art approaches in survival prediction from WSIs.

Method: Uses LaViLa video captioner on video chunks for spatio-temporal modeling, enriches with static scene descriptions using LLaVA VLM, fine-tunes a controllable hybrid captioner that alternates between action and scene captions based on scene change detection.

Result: Developed a more detailed and complete caption log that expands answerable question space, improved pipeline efficiency by combining action and scene captioning in a single model, and created a system that can answer complex natural language queries about videos.

Conclusion: The proposed video understanding system successfully creates comprehensive text-based memories from videos, enabling effective reasoning through LLMs while improving efficiency through hybrid captioning and scene-aware processing.

Abstract: Video data, especially long-form video, is extremely dense and high-dimensional. Text-based summaries of video content offer a way to represent query-relevant content in a much more compact manner than raw video. In addition, textual representations are easily ingested by state-of-the-art large language models (LLMs), which enable reasoning over video content to answer complex natural language queries. To solve this issue, we rely on the progressive construction of a text-based memory by a video captioner operating on shorter chunks of the video, where spatio-temporal modeling is computationally feasible. We explore ways to improve the quality of the activity log comprised solely of short video captions. Because the video captions tend to be focused on human actions, and questions may pertain to other information in the scene, we seek to enrich the memory with static scene descriptions using Vision Language Models (VLMs). Our video understanding system relies on the LaViLa video captioner in combination with a LLM to answer questions about videos. We first explored different ways of partitioning the video into meaningful segments such that the textual descriptions more accurately reflect the structure of the video content. Furthermore, we incorporated static scene descriptions into the captioning pipeline using LLaVA VLM, resulting in a more detailed and complete caption log and expanding the space of questions that are answerable from the textual memory. Finally, we have successfully fine-tuned the LaViLa video captioner to produce both action and scene captions, significantly improving the efficiency of the captioning pipeline compared to using separate captioning models for the two tasks. Our model, controllable hybrid captioner, can alternate between different types of captions according to special input tokens that signals scene changes detected in the video.

Method: RaGS uses a cascaded pipeline: Frustum-based Localization Initiation to initialize Gaussian centers, Iterative Multimodal Aggregation to refine Gaussians with image semantics and radar velocity geometry, and Multi-level Gaussian Fusion to render hierarchical BEV features for detection.

Result: Extensive experiments on View-of-Delft, TJ4DRadSet, and OmniHD-Scenes demonstrate robust state-of-the-art performance in 3D object detection.

Conclusion: RaGS achieves object-centric precision and comprehensive scene perception by dynamically focusing on sparse and informative regions through 3D Gaussian Splatting fusion of 4D radar and monocular cues.

Abstract: 4D millimeter-wave radar is a promising sensing modality for autonomous driving, yet effective 3D object detection from 4D radar and monocular images remains challenging. Existing fusion approaches either rely on instance proposals lacking global context or dense BEV grids constrained by rigid structures, lacking a flexible and adaptive representation for diverse scenes. To address this, we propose RaGS, the first framework that leverages 3D Gaussian Splatting (GS) to fuse 4D radar and monocular cues for 3D object detection. 3D GS models the scene as a continuous field of Gaussians, enabling dynamic resource allocation to foreground objects while maintaining flexibility and efficiency. Moreover, the velocity dimension of 4D radar provides motion cues that help anchor and refine the spatial distribution of Gaussians. Specifically, RaGS adopts a cascaded pipeline to construct and progressively refine the Gaussian field. It begins with Frustum-based Localization Initiation (FLI), which unprojects foreground pixels to initialize coarse Gaussian centers. Then, Iterative Multimodal Aggregation (IMA) explicitly exploits image semantics and implicitly integrates 4D radar velocity geometry to refine the Gaussians within regions of interest. Finally, Multi-level Gaussian Fusion (MGF) renders the Gaussian field into hierarchical BEV features for 3D object detection. By dynamically focusing on sparse and informative regions, RaGS achieves object-centric precision and comprehensive scene perception. Extensive experiments on View-of-Delft, TJ4DRadSet, and OmniHD-Scenes demonstrate its robustness and SOTA performance. Code will be released.

Method: Proposes FedVLM framework with personalized LoRA (pLoRA) that dynamically adapts LoRA parameters to each client’s unique data distribution, enabling decentralized adaptation while maintaining global model aggregation.

Result: Experiments on RLAIF-V dataset show pLoRA improves client-specific performance by 24.5% over standard LoRA, demonstrating superior adaptation in non-iid settings.

Conclusion: FedVLM provides a scalable and efficient solution for fine-tuning VLMs in federated settings, advancing personalized adaptation in distributed learning scenarios.

Abstract: Vision-language models (VLMs) demonstrate impressive zero-shot and few-shot learning capabilities, making them essential for several downstream tasks. However, fine-tuning these models at scale remains challenging, particularly in federated environments where data is decentralized and non-iid across clients. Existing parameter-efficient tuning methods like LoRA (Low-Rank Adaptation) reduce computational overhead but struggle with heterogeneous client data, leading to suboptimal generalization. To address these challenges, we propose FedVLM, a federated LoRA fine-tuning framework that enables decentralized adaptation of VLMs while preserving model privacy and reducing reliance on centralized training. To further tackle data heterogeneity, we introduce personalized LoRA (pLoRA), which dynamically adapts LoRA parameters to each client’s unique data distribution, significantly improving local adaptation while maintaining global model aggregation. Experiments on the RLAIF-V dataset show that pLoRA improves client-specific performance by 24.5% over standard LoRA, demonstrating superior adaptation in non-iid settings. FedVLM provides a scalable and efficient solution for fine-tuning VLMs in federated settings, advancing personalized adaptation in distributed learning scenarios.

Method: Proposes Slot Attention with re-Initialization and self-Distillation (DIAS): reduces slot redundancy via re-initialization and updates remaining slots; enables self-distillation by making early attention maps approximate final ones.

Result: Achieves state-of-the-art performance on object discovery and recognition tasks, and improves advanced visual prediction and reasoning capabilities.

Conclusion: DIAS effectively addresses slot redundancy and supervision limitations in OCL, demonstrating superior performance across multiple visual tasks.

Abstract: Unlike popular solutions based on dense feature maps, Object-Centric Learning (OCL) represents visual scenes as sub-symbolic object-level feature vectors, termed slots, which are highly versatile for tasks involving visual modalities. OCL typically aggregates object superpixels into slots by iteratively applying competitive cross attention, known as Slot Attention, with the slots as the query. However, once initialized, these slots are reused naively, causing redundant slots to compete with informative ones for representing objects. This often results in objects being erroneously segmented into parts. Additionally, mainstream methods derive supervision signals solely from decoding slots into the input’s reconstruction, overlooking potential supervision based on internal information. To address these issues, we propose Slot Attention with re-Initialization and self-Distillation (DIAS): $\emph{i)}$ We reduce redundancy in the aggregated slots and re-initialize extra aggregation to update the remaining slots; $\emph{ii)}$ We drive the bad attention map at the first aggregation iteration to approximate the good at the last iteration to enable self-distillation. Experiments demonstrate that DIAS achieves state-of-the-art on OCL tasks like object discovery and recognition, while also improving advanced visual prediction and reasoning. Our source code and model checkpoints are available on https://github.com/Genera1Z/DIAS.

Method: Proposed a compact algebraic solver building on Grunert’s 1841 theoretical foundation, reducing P3P to a quartic polynomial with simple coefficients.

Result: Achieves accuracy and runtime comparable to state-of-the-art methods, demonstrating the classical formulation remains highly competitive.

Conclusion: The classical P3P formulation offers an excellent balance between simplicity, efficiency, and accuracy when implemented with modern insights.

Abstract: We revisit the classical Perspective-Three-Point (P3P) problem, which aims to recover the absolute pose of a calibrated camera from three 2D-3D correspondences. It has long been known that P3P can be reduced to a quartic polynomial with analytically simple and computationally efficient coefficients. However, this elegant formulation has been largely overlooked in modern literature. Building on the theoretical foundation that traces back to Grunert’s work in 1841, we propose a compact algebraic solver that achieves accuracy and runtime comparable to state-of-the-art methods. Our results show that this classical formulation remains highly competitive when implemented with modern insights, offering an excellent balance between simplicity, efficiency, and accuracy.

Method: Shift Taylor prediction from module level to last block level to reduce cached features. Use error between Taylor-estimated and actual outputs of first block as reliability indicator to enable dynamic caching - trust Taylor prediction when error is small, fall back to full computation otherwise.

Result: Achieves 3.17x acceleration on FLUX, 2.36x on DiT, and 4.14x on Wan Video with negligible quality drop.

Conclusion: The proposed method achieves better speed-quality balance by reducing cached features and implementing dynamic caching based on prediction reliability.

Abstract: Diffusion Transformers (DiTs) have demonstrated remarkable performance in visual generation tasks. However, their low inference speed limits their deployment in low-resource applications. Recent training-free approaches exploit the redundancy of features across timesteps by caching and reusing past representations to accelerate inference. Building on this idea, TaylorSeer instead uses cached features to predict future ones via Taylor expansion. However, its module-level prediction across all transformer blocks (e.g., attention or feedforward modules) requires storing fine-grained intermediate features, leading to notable memory and computation overhead. Moreover, it adopts a fixed caching schedule without considering the varying accuracy of predictions across timesteps, which can lead to degraded outputs when prediction fails. To address these limitations, we propose a novel approach to better leverage Taylor-based acceleration. First, we shift the Taylor prediction target from the module level to the last block level, significantly reducing the number of cached features. Furthermore, observing strong sequential dependencies among Transformer blocks, we propose to use the error between the Taylor-estimated and actual outputs of the first block as an indicator of prediction reliability. If the error is small, we trust the Taylor prediction for the last block; otherwise, we fall back to full computation, thereby enabling a dynamic caching mechanism. Empirical results show that our method achieves a better balance between speed and quality, achieving a 3.17x acceleration on FLUX, 2.36x on DiT, and 4.14x on Wan Video with negligible quality drop. The Project Page is \href{https://cg-taylor-acce.github.io/CG-Taylor/}{here.}

Method: Proposes RandSF.Q with: (1) new transitioner incorporating both slots and features for query prediction, (2) training transitioner using random slot-feature pairs from available recurrences to learn transition dynamics.

Result: Significantly surpasses existing video OCL methods, achieving up to 10 points improvement on object discovery and setting new state-of-the-art. Also benefits downstream tasks like dynamics modeling.

Conclusion: RandSF.Q successfully addresses key limitations in video OCL by incorporating next frame features and learning transition dynamics, demonstrating superior performance in scene representation and downstream applications.

Abstract: Unsupervised video Object-Centric Learning (OCL) is promising as it enables object-level scene representation and dynamics modeling as we humans do. Mainstream video OCL methods adopt a recurrent architecture: An aggregator aggregates current video frame into object features, termed slots, under some queries; A transitioner transits current slots to queries for the next frame. This is an effective architecture but all existing implementations both (\textit{i1}) neglect to incorporate next frame features, the most informative source for query prediction, and (\textit{i2}) fail to learn transition dynamics, the knowledge essential for query prediction. To address these issues, we propose Random Slot-Feature pair for learning Query prediction (RandSF.Q): (\textit{t1}) We design a new transitioner to incorporate both slots and features, which provides more information for query prediction; (\textit{t2}) We train the transitioner to predict queries from slot-feature pairs randomly sampled from available recurrences, which drives it to learn transition dynamics. Experiments on scene representation demonstrate that our method surpass existing video OCL methods significantly, e.g., up to 10 points on object discovery, setting new state-of-the-art. Such superiority also benefits downstream tasks like dynamics modeling. Our core source code, model checkpoints and training logs are available on https://github.com/Genera1Z/RandSF.Q.

Method: Proposes Semantic-Guided Adaptive Knowledge Transfer (SG-AKT) to assess image relevance to historical knowledge via textual cues, and Semantic-Enhanced Visual Prototype Refinement (SE-VPR) to refine visual prototypes using inter-class semantic relations from textual embeddings.

Result: Extensive experiments on multiple benchmarks validate the effectiveness of SECA in achieving better balance between stability and plasticity in continual learning.

Conclusion: SECA successfully harnesses textual semantic priors to guide semantic-aware knowledge transfer and reinforce semantic structure in visual classifiers, effectively addressing the stability-plasticity dilemma in continual learning.

Abstract: Continual learning (CL) aims to equip models with the ability to learn from a stream of tasks without forgetting previous knowledge. With the progress of vision-language models like Contrastive Language-Image Pre-training (CLIP), their promise for CL has attracted increasing attention due to their strong generalizability. However, the potential of rich textual semantic priors in CLIP in addressing the stability-plasticity dilemma remains underexplored. During backbone training, most approaches transfer past knowledge without considering semantic relevance, leading to interference from unrelated tasks that disrupt the balance between stability and plasticity. Besides, while text-based classifiers provide strong generalization, they suffer from limited plasticity due to the inherent modality gap in CLIP. Visual classifiers help bridge this gap, but their prototypes lack rich and precise semantics. To address these challenges, we propose Semantic-Enriched Continual Adaptation (SECA), a unified framework that harnesses the anti-forgetting and structured nature of textual priors to guide semantic-aware knowledge transfer in the backbone and reinforce the semantic structure of the visual classifier. Specifically, a Semantic-Guided Adaptive Knowledge Transfer (SG-AKT) module is proposed to assess new images’ relevance to diverse historical visual knowledge via textual cues, and aggregate relevant knowledge in an instance-adaptive manner as distillation signals. Moreover, a Semantic-Enhanced Visual Prototype Refinement (SE-VPR) module is introduced to refine visual prototypes using inter-class semantic relations captured in class-wise textual embeddings. Extensive experiments on multiple benchmarks validate the effectiveness of our approach.

Method: Uses visual prompts on 2D top-view maps to mark navigation trajectories, creates VPN datasets (R2R-VP, R2R-CE-VP), and develops VPNet with view-level and trajectory-level data augmentation.

Result: Extensive experiments evaluate how visual prompt forms, map formats, and augmentation strategies affect navigation performance.

Conclusion: Visual prompts provide intuitive, spatially grounded guidance that reduces ambiguity and is more user-friendly than language instructions.

Abstract: While natural language is commonly used to guide embodied agents, the inherent ambiguity and verbosity of language often hinder the effectiveness of language-guided navigation in complex environments. To this end, we propose Visual Prompt Navigation (VPN), a novel paradigm that guides agents to navigate using only user-provided visual prompts within 2D top-view maps. This visual prompt primarily focuses on marking the visual navigation trajectory on a top-down view of a scene, offering intuitive and spatially grounded guidance without relying on language instructions. It is more friendly for non-expert users and reduces interpretive ambiguity. We build VPN tasks in both discrete and continuous navigation settings, constructing two new datasets, R2R-VP and R2R-CE-VP, by extending existing R2R and R2R-CE episodes with corresponding visual prompts. Furthermore, we introduce VPNet, a dedicated baseline network to handle the VPN tasks, with two data augmentation strategies: view-level augmentation (altering initial headings and prompt orientations) and trajectory-level augmentation (incorporating diverse trajectories from large-scale 3D scenes), to enhance navigation performance. Extensive experiments evaluate how visual prompt forms, top-view map formats, and data augmentation strategies affect the performance of visual prompt navigation. The code is available at https://github.com/farlit/VPN.

Method: Leverages 3D Gaussian-splatting-based scene representation combined with diffusion-based 2D super-resolution models to ensure 3D consistency across views without requiring fine-tuning.

Result: Demonstrates high-resolution results on MipNeRF360 and LLFF datasets that are visually compelling while maintaining structural consistency in 3D reconstructions.

Conclusion: 3DSR successfully enhances visual quality while ensuring spatial coherence in 3D scenes, providing a superior approach to 3D-consistent super-resolution compared to prior methods.

Abstract: We propose 3D Super Resolution (3DSR), a novel 3D Gaussian-splatting-based super-resolution framework that leverages off-the-shelf diffusion-based 2D super-resolution models. 3DSR encourages 3D consistency across views via the use of an explicit 3D Gaussian-splatting-based scene representation. This makes the proposed 3DSR different from prior work, such as image upsampling or the use of video super-resolution, which either don’t consider 3D consistency or aim to incorporate 3D consistency implicitly. Notably, our method enhances visual quality without additional fine-tuning, ensuring spatial coherence within the reconstructed scene. We evaluate 3DSR on MipNeRF360 and LLFF data, demonstrating that it produces high-resolution results that are visually compelling, while maintaining structural consistency in 3D reconstructions.

Method: Constructed X2Edit dataset using industry-leading models and expert models, designed editing instructions with VLM, implemented scoring mechanisms. Developed task-aware MoE-LoRA training based on FLUX.1 with contrastive learning using diffusion model representations.

Result: Created 3.7M high-quality balanced dataset; model achieves competitive editing performance with only 8% of full model parameters; dataset shows substantial advantages over existing open-source datasets.

Conclusion: X2Edit provides both a superior dataset and efficient plug-and-play editing module that integrates seamlessly with community image generation models, advancing the field of arbitrary-instruction image editing.

Abstract: Existing open-source datasets for arbitrary-instruction image editing remain suboptimal, while a plug-and-play editing module compatible with community-prevalent generative models is notably absent. In this paper, we first introduce the X2Edit Dataset, a comprehensive dataset covering 14 diverse editing tasks, including subject-driven generation. We utilize the industry-leading unified image generation models and expert models to construct the data. Meanwhile, we design reasonable editing instructions with the VLM and implement various scoring mechanisms to filter the data. As a result, we construct 3.7 million high-quality data with balanced categories. Second, to better integrate seamlessly with community image generation models, we design task-aware MoE-LoRA training based on FLUX.1, with only 8% of the parameters of the full model. To further improve the final performance, we utilize the internal representations of the diffusion model and define positive/negative samples based on image editing types to introduce contrastive learning. Extensive experiments demonstrate that the model’s editing performance is competitive among many excellent models. Additionally, the constructed dataset exhibits substantial advantages over existing open-source datasets. The open-source code, checkpoints, and datasets for X2Edit can be found at the following link: https://github.com/OPPO-Mente-Lab/X2Edit.

Method: Extended slot attention decomposes visual features into object and relation slots; object temporal consistency learning enforces frame-to-frame consistency; dynamic triplet prediction links relations to object pairs over time.

Result: UNO achieves competitive performance on both box-level and pixel-level VidSGG benchmarks while offering improved efficiency through unified design and parameter sharing.

Conclusion: UNO demonstrates that a single-stage, unified framework can effectively handle multiple VidSGG tasks with minimal task-specific modifications, enabling better generalization across visual granularity levels.

Abstract: Video Scene Graph Generation (VidSGG) aims to represent dynamic visual content by detecting objects and modeling their temporal interactions as structured graphs. Prior studies typically target either coarse-grained box-level or fine-grained panoptic pixel-level VidSGG, often requiring task-specific architectures and multi-stage training pipelines. In this paper, we present UNO (UNified Object-centric VidSGG), a single-stage, unified framework that jointly addresses both tasks within an end-to-end architecture. UNO is designed to minimize task-specific modifications and maximize parameter sharing, enabling generalization across different levels of visual granularity. The core of UNO is an extended slot attention mechanism that decomposes visual features into object and relation slots. To ensure robust temporal modeling, we introduce object temporal consistency learning, which enforces consistent object representations across frames without relying on explicit tracking modules. Additionally, a dynamic triplet prediction module links relation slots to corresponding object pairs, capturing evolving interactions over time. We evaluate UNO on standard box-level and pixel-level VidSGG benchmarks. Results demonstrate that UNO not only achieves competitive performance across both tasks but also offers improved efficiency through a unified, object-centric design.

Method: Developed a generative neural physics framework that fuses generative models with physics-informed PDE solvers. Uses a compact neural surrogate for full-wave propagation trained on limited cross-modality data, preserving physical accuracy while enabling efficient inversion.

Result: Achieved accurate and efficient quantitative volumetric imaging of in vivo human breast and musculoskeletal tissues in under 10 minutes. Generated spatial maps of tissue mechanical properties not available from conventional methods. Revealed biomechanical features in bone, muscle, fat, and glandular tissues with structural resolution comparable to 3T MRI but greater sensitivity to disease-related tissue mechanics.

Conclusion: The framework enables rapid, high-fidelity 3D quantitative imaging of tissue mechanics, providing substantially greater sensitivity to disease-related tissue properties compared to conventional imaging methods while maintaining high structural resolution.

Abstract: Tissue mechanics–stiffness, density and impedance contrast–are broadly informative biomarkers across diseases, yet routine CT, MRI, and B-mode ultrasound rarely quantify them directly. While ultrasound tomography (UT) is intrinsically suited to in-vivo biomechanical assessment by capturing transmitted and reflected wavefields, efficient and accurate full-wave scattering models remain a bottleneck. Here, we introduce a generative neural physics framework that fuses generative models with physics-informed partial differential equation (PDE) solvers to produce rapid, high-fidelity 3D quantitative imaging of tissue mechanics. A compact neural surrogate for full-wave propagation is trained on limited cross-modality data, preserving physical accuracy while enabling efficient inversion. This enables, for the first time, accurate and efficient quantitative volumetric imaging of in vivo human breast and musculoskeletal tissues in under ten minutes, providing spatial maps of tissue mechanical properties not available from conventional reflection-mode or standard UT reconstructions. The resulting images reveal biomechanical features in bone, muscle, fat, and glandular tissues, maintaining structural resolution comparable to 3T MRI while providing substantially greater sensitivity to disease-related tissue mechanics.

Method: Proposed F2RVLM model with two-stage training: supervised fine-tuning for fragment-level retrieval knowledge, and GRPO-based reinforcement learning with multi-objective rewards. Introduced difficulty-aware curriculum sampling to handle varying intra-fragment complexity.

Result: F2RVLM outperforms popular Vision-Language Models in both in-domain and real-domain settings, demonstrating superior retrieval performance on the MLDR dataset and WeChat-based test set.

Conclusion: The proposed F2RVLM approach effectively addresses the Fine-grained Fragment Retrieval task, showing strong performance in retrieving semantically coherent multimodal fragments from long-form dialogues through specialized training techniques.

Abstract: Traditional dialogue retrieval aims to select the most appropriate utterance or image from recent dialogue history. However, they often fail to meet users’ actual needs for revisiting semantically coherent content scattered across long-form conversations. To fill this gap, we define the Fine-grained Fragment Retrieval (FFR) task, requiring models to locate query-relevant fragments, comprising both utterances and images, from multimodal long-form dialogues. As a foundation for FFR, we construct MLDR, the longest-turn multimodal dialogue retrieval dataset to date, averaging 25.45 turns per dialogue, with each naturally spanning three distinct topics. To evaluate generalization in real-world scenarios, we curate and annotate a WeChat-based test set comprising real-world multimodal dialogues with an average of 75.38 turns. Building on these resources, we explore existing generation-based Vision-Language Models (VLMs) on FFR and observe that they often retrieve incoherent utterance-image fragments. While optimized for generating responses from visual-textual inputs, these models lack explicit supervision to ensure semantic coherence within retrieved fragments. To this end, we propose F2RVLM, a generative retrieval model trained in a two-stage paradigm: (1) supervised fine-tuning to inject fragment-level retrieval knowledge, and (2) GRPO-based reinforcement learning with multi-objective rewards promoting semantic precision, relevance, and contextual coherence. To handle varying intra-fragment complexity, from locally dense to sparsely distributed, we introduce difficulty-aware curriculum sampling that ranks training instances by model-predicted difficulty and gradually exposes the model to harder samples. This boosts reasoning ability in long, multi-turn contexts. F2RVLM outperforms popular VLMs in both in-domain and real-domain settings, demonstrating superior retrieval performance.

Method: Built on FSFM self-supervised face model, fine-tuned with ensemble of deepfake datasets using triplet loss variants and attribution-based supervision schemes for manipulation type/dataset categorization.

Result: Extensive experiments show effective performance across diverse benchmarks, particularly in challenging real-world scenarios with strong generalization capabilities.

Conclusion: The framework demonstrates robust deepfake detection with enhanced generalization through face foundation models, triplet loss, and attribution supervision, addressing real-world detection challenges.

Abstract: The increasing realism and accessibility of deepfakes have raised critical concerns about media authenticity and information integrity. Despite recent advances, deepfake detection models often struggle to generalize beyond their training distributions, particularly when applied to media content found in the wild. In this work, we present a robust video deepfake detection framework with strong generalization that takes advantage of the rich facial representations learned by face foundation models. Our method is built on top of FSFM, a self-supervised model trained on real face data, and is further fine-tuned using an ensemble of deepfake datasets spanning both face-swapping and face-reenactment manipulations. To enhance discriminative power, we incorporate triplet loss variants during training, guiding the model to produce more separable embeddings between real and fake samples. Additionally, we explore attribution-based supervision schemes, where deepfakes are categorized by manipulation type or source dataset, to assess their impact on generalization. Extensive experiments across diverse evaluation benchmarks demonstrate the effectiveness of our approach, especially in challenging real-world scenarios.

Method: Proposes UniPixel model that processes visual prompts, generates relevant masks on demand, and performs reasoning conditioned on these intermediate pointers during inference.

Result: Verified effectiveness on 10 benchmarks across diverse tasks including pixel-level referring/segmentation, object-centric understanding in images/videos, and a novel PixelQA task requiring joint referring, segmentation, and QA.

Conclusion: UniPixel successfully integrates pixel-level perception with general visual understanding, enabling flexible fine-grained pixel-level reasoning across multiple tasks.

Abstract: Recent advances in Large Multi-modal Models (LMMs) have demonstrated their remarkable success as general-purpose multi-modal assistants, with particular focuses on holistic image- and video-language understanding. Conversely, less attention has been given to scaling fine-grained pixel-level understanding capabilities, where the models are expected to realize pixel-level alignment between visual signals and language semantics. Some previous studies have applied LMMs to related tasks such as region-level captioning and referring expression segmentation. However, these models are limited to performing either referring or segmentation tasks independently and fail to integrate these fine-grained perception capabilities into visual reasoning. To bridge this gap, we propose UniPixel, a large multi-modal model capable of flexibly comprehending visual prompt inputs and generating mask-grounded responses. Our model distinguishes itself by seamlessly integrating pixel-level perception with general visual understanding capabilities. Specifically, UniPixel processes visual prompts and generates relevant masks on demand, and performs subsequent reasoning conditioning on these intermediate pointers during inference, thereby enabling fine-grained pixel-level reasoning. The effectiveness of our approach has been verified on 10 benchmarks across a diverse set of tasks, including pixel-level referring/segmentation and object-centric understanding in images/videos. A novel PixelQA task that jointly requires referring, segmentation, and question answering is also designed to verify the flexibility of our method.

Method: Proposes PersonaAnimator framework that learns personalized motion patterns from unconstrained videos, introduces PersonaVid dataset with 20 motion content and 120 style categories, and implements Physics-aware Motion Style Regularization for physical plausibility.

Result: Extensive experiments show PersonaAnimator outperforms state-of-the-art motion transfer methods and establishes a new benchmark for Video-to-Video Motion Personalization.

Conclusion: The paper pioneers Video-to-Video Motion Personalization task and demonstrates successful personalized motion transfer directly from videos while ensuring physical plausibility.

Abstract: Recent advances in motion generation show remarkable progress. However, several limitations remain: (1) Existing pose-guided character motion transfer methods merely replicate motion without learning its style characteristics, resulting in inexpressive characters. (2) Motion style transfer methods rely heavily on motion capture data, which is difficult to obtain. (3) Generated motions sometimes violate physical laws. To address these challenges, this paper pioneers a new task: Video-to-Video Motion Personalization. We propose a novel framework, PersonaAnimator, which learns personalized motion patterns directly from unconstrained videos. This enables personalized motion transfer. To support this task, we introduce PersonaVid, the first video-based personalized motion dataset. It contains 20 motion content categories and 120 motion style categories. We further propose a Physics-aware Motion Style Regularization mechanism to enforce physical plausibility in the generated motions. Extensive experiments show that PersonaAnimator outperforms state-of-the-art motion transfer methods and sets a new benchmark for the Video-to-Video Motion Personalization task.

Method: Proposed FastVGGT with a training-free token merging mechanism, using a unique token partitioning strategy tailored for 3D architectures to eliminate redundant computation.

Result: Achieved 4x speedup over VGGT with 1000 input images while mitigating error accumulation in long-sequence scenarios, validated on multiple 3D geometry benchmarks.

Conclusion: Token merging is a principled solution for scalable 3D vision systems, effectively accelerating models without compromising reconstruction capacity.

Abstract: Foundation models for 3D vision have recently demonstrated remarkable capabilities in 3D perception. However, scaling these models to long-sequence image inputs remains a significant challenge due to inference-time inefficiency. In this work, we present a detailed analysis of VGGT, a state-of-the-art feed-forward visual geometry model and identify its primary bottleneck. Visualization further reveals a token collapse phenomenon in the attention maps. Motivated by these findings, we explore the potential of token merging in the feed-forward visual geometry model. Owing to the unique architectural and task-specific properties of 3D models, directly applying existing merging techniques proves challenging. To this end, we propose FastVGGT, which, for the first time, leverages token merging in the 3D domain through a training-free mechanism for accelerating VGGT. we devise a unique token partitioning strategy tailored to 3D architectures and tasks, effectively eliminating redundant computation while preserving VGGT’s powerful reconstruction capacity. Extensive experiments on multiple 3D geometry benchmarks validate the effectiveness of our approach. Notably, with 1000 input images, FastVGGT achieves a 4x speedup over VGGT while mitigating error accumulation in long-sequence scenarios. These findings underscore the potential of token merging as a principled solution for scalable 3D vision systems. Code is available at: https://mystorm16.github.io/fastvggt/.

Method: Decouples diffusion and condition generation processes, uses dynamic conditions generated by the same noise schedule to gradually adjust statistical characteristics and embed time-related information, reducing network learning difficulty.

Result: Achieved best performance in multiple indicators on public datasets for dehazing and visible-infrared fusion tasks. Demonstrated superiority through analysis of attention weights and learning objectives.

Conclusion: The proposed dynamic conditional double diffusion bridge paradigm effectively addresses challenges in ill-posed multi-task scenarios by enabling better exploitation of task correlations and adapting to dynamic learning requirements.

Abstract: Conditional diffusion models have made impressive progress in the field of image processing, but the characteristics of constructing data distribution pathways make it difficult to exploit the intrinsic correlation between tasks in multi-task scenarios, which is even worse in ill-posed tasks with a lack of training data. In addition, traditional static condition control makes it difficult for networks to learn in multi-task scenarios with its dynamically evolving characteristics. To address these challenges, we propose a dynamic conditional double diffusion bridge training paradigm to build a general framework for ill-posed multi-tasks. Firstly, this paradigm decouples the diffusion and condition generation processes, avoiding the dependence of the diffusion model on supervised data in ill-posed tasks. Secondly, generated by the same noise schedule, dynamic conditions are used to gradually adjust their statistical characteristics, naturally embed time-related information, and reduce the difficulty of network learning. We analyze the learning objectives of the network under different conditional forms in the single-step denoising process and compare the changes in its attention weights in the network, demonstrating the superiority of our dynamic conditions. Taking dehazing and visible-infrared fusion as typical ill-posed multi-task scenarios, we achieve the best performance in multiple indicators on public datasets. The code has been publicly released at: https://anonymous.4open.science/r/DCDB-D3C2.

Method: Three key modifications: expanded hidden layers for 6-band inputs, redesigned residual functions for spectral discrepancy optimization, and adapted data compression for multispectral imagery’s higher bit-depth requirements.

Result: Experimental results show Multispectral-NeRF successfully processes multi-band spectral features while accurately preserving original scenes’ spectral characteristics.

Conclusion: The proposed Multispectral-NeRF effectively integrates multispectral information into neural radiance fields, overcoming limitations of existing methods and enabling high-quality multispectral 3D reconstruction.

Abstract: 3D reconstruction technology generates three-dimensional representations of real-world objects, scenes, or environments using sensor data such as 2D images, with extensive applications in robotics, autonomous vehicles, and virtual reality systems. Traditional 3D reconstruction techniques based on 2D images typically relies on RGB spectral information. With advances in sensor technology, additional spectral bands beyond RGB have been increasingly incorporated into 3D reconstruction workflows. Existing methods that integrate these expanded spectral data often suffer from expensive scheme prices, low accuracy and poor geometric features. Three - dimensional reconstruction based on NeRF can effectively address the various issues in current multispectral 3D reconstruction methods, producing high - precision and high - quality reconstruction results. However, currently, NeRF and some improved models such as NeRFacto are trained on three - band data and cannot take into account the multi - band information. To address this problem, we propose Multispectral-NeRF, an enhanced neural architecture derived from NeRF that can effectively integrates multispectral information. Our technical contributions comprise threefold modifications: Expanding hidden layer dimensionality to accommodate 6-band spectral inputs; Redesigning residual functions to optimize spectral discrepancy calculations between reconstructed and reference images; Adapting data compression modules to address the increased bit-depth requirements of multispectral imagery. Experimental results confirm that Multispectral-NeRF successfully processes multi-band spectral features while accurately preserving the original scenes’ spectral characteristics.

Method: Introduces GauSep for selective color/normal attribute activation, GauRep for adaptive Gaussian representation, CoRe for surface reconstruction via normal field distillation, and Separate Rendering pipeline.

Result: Achieves superior appearance and geometry reconstruction compared to state-of-the-art methods while using significantly fewer parameters.

Conclusion: The proposed framework effectively balances model capacity and parameter efficiency, is model-agnostic, and can be seamlessly integrated into other architectures.

Abstract: This paper presents an effective Gaussian management framework for high-fidelity scene reconstruction of appearance and geometry. Departing from recent Gaussian Splatting (GS) methods that rely on indiscriminate attribute assignment, our approach introduces a novel densification strategy called \emph{GauSep} that selectively activates Gaussian color or normal attributes. Together with a tailored rendering pipeline, termed \emph{Separate Rendering}, this strategy alleviates gradient conflicts arising from dual supervision and yields improved reconstruction quality. In addition, we develop \emph{GauRep}, an adaptive and integrated Gaussian representation that reduces redundancy both at the individual and global levels, effectively balancing model capacity and number of parameters. To provide reliable geometric supervision essential for effective management, we also introduce \emph{CoRe}, a novel surface reconstruction module that distills normal fields from the SDF branch to the Gaussian branch through a confidence mechanism. Notably, our management framework is model-agnostic and can be seamlessly incorporated into other architectures, simultaneously improving performance and reducing model size. Extensive experiments demonstrate that our approach achieves superior performance in reconstructing both appearance and geometry compared with state-of-the-art methods, while using significantly fewer parameters.

Method: Uses a Developmental Mixture of Experts architecture with LoRA models: Real-LoRA for stable real face knowledge and multiple Fake-LoRAs for incremental forgery learning. Employs orthogonal learning directions and orthogonal gradients to prevent catastrophic forgetting.

Result: Experimental results show effectiveness under both datasets and manipulation types incremental protocols, demonstrating successful adaptation to new forgery types while maintaining detection capabilities for previously learned types.

Conclusion: The proposed continual learning framework effectively addresses the challenge of evolving face forgery techniques by enabling incremental learning while preventing knowledge forgetting, providing a sustainable solution for face forgery detection.

Abstract: The rise of realistic digital face generation and manipulation poses significant social risks. The primary challenge lies in the rapid and diverse evolution of generation techniques, which often outstrip the detection capabilities of existing models. To defend against the ever-evolving new types of forgery, we need to enable our model to quickly adapt to new domains with limited computation and data while avoiding forgetting previously learned forgery types. In this work, we posit that genuine facial samples are abundant and relatively stable in acquisition methods, while forgery faces continuously evolve with the iteration of manipulation techniques. Given the practical infeasibility of exhaustively collecting all forgery variants, we frame face forgery detection as a continual learning problem and allow the model to develop as new forgery types emerge. Specifically, we employ a Developmental Mixture of Experts (MoE) architecture that uses LoRA models as its individual experts. These experts are organized into two groups: a Real-LoRA to learn and refine knowledge of real faces, and multiple Fake-LoRAs to capture incremental information from different forgery types. To prevent catastrophic forgetting, we ensure that the learning direction of Fake-LoRAs is orthogonal to the established subspace. Moreover, we integrate orthogonal gradients into the orthogonal loss of Fake-LoRAs, preventing gradient interference throughout the training process of each task. Experimental results under both the datasets and manipulation types incremental protocols demonstrate the effectiveness of our method.

Method: Uses a generative physics network with diffusion model conditioned on physics parameters and forces, trained on 550K synthetic animations. Incorporates spatiotemporal attention blocks for particle interactions and physics-based constraints.

Result: Generates realistic physics-grounded motion trajectories that drive image-to-video models to produce high-fidelity, controllable videos superior to existing methods in both visual quality and physical plausibility.

Conclusion: PhysCtrl successfully bridges the gap between video generation and physical realism, enabling controllable physics-based video synthesis with improved plausibility.

Abstract: Existing video generation models excel at producing photo-realistic videos from text or images, but often lack physical plausibility and 3D controllability. To overcome these limitations, we introduce PhysCtrl, a novel framework for physics-grounded image-to-video generation with physical parameters and force control. At its core is a generative physics network that learns the distribution of physical dynamics across four materials (elastic, sand, plasticine, and rigid) via a diffusion model conditioned on physics parameters and applied forces. We represent physical dynamics as 3D point trajectories and train on a large-scale synthetic dataset of 550K animations generated by physics simulators. We enhance the diffusion model with a novel spatiotemporal attention block that emulates particle interactions and incorporates physics-based constraints during training to enforce physical plausibility. Experiments show that PhysCtrl generates realistic, physics-grounded motion trajectories which, when used to drive image-to-video models, yield high-fidelity, controllable videos that outperform existing methods in both visual quality and physical plausibility. Project Page: https://cwchenwang.github.io/physctrl

Method: Uses Mask AutoRegressive models with two key components: Dual-Stream Encoder Information Fusion (DEIF) to fuse text and background information in frequency domain, and Adaptive Decoder Attention Score Enhancing (ADAE) to enhance attention on guidance and inpainting tokens.

Result: Outperforms prior state-of-the-art methods across almost all metrics in extensive experiments.

Conclusion: The proposed training-free method effectively addresses text-prompt alignment and background consistency issues in image inpainting through novel attention mechanisms and information fusion techniques.

Abstract: Text-guided image inpainting aims to inpaint masked image regions based on a textual prompt while preserving the background. Although diffusion-based methods have become dominant, their property of modeling the entire image in latent space makes it challenging for the results to align well with prompt details and maintain a consistent background. To address these issues, we explore Mask AutoRegressive (MAR) models for this task. MAR naturally supports image inpainting by generating latent tokens corresponding to mask regions, enabling better local controllability without altering the background. However, directly applying MAR to this task makes the inpainting content either ignore the prompts or be disharmonious with the background context. Through analysis of the attention maps from the inpainting images, we identify the impact of background tokens on text tokens during the MAR generation, and leverage this to design \textbf{Token Painter}, a training-free text-guided image inpainting method based on MAR. Our approach introduces two key components: (1) Dual-Stream Encoder Information Fusion (DEIF), which fuses the semantic and context information from text and background in frequency domain to produce novel guidance tokens, allowing MAR to generate text-faithful inpainting content while keeping harmonious with background context. (2) Adaptive Decoder Attention Score Enhancing (ADAE), which adaptively enhances attention scores on guidance tokens and inpainting tokens to further enhance the alignment of prompt details and the content visual quality. Extensive experiments demonstrate that our training-free method outperforms prior state-of-the-art methods across almost all metrics. Codes: https://github.com/longtaojiang/Token-Painter.

Method: Proposes RDTTrack tracker that fuses thermal infrared and depth modalities under orthogonal projection constraint, then integrates them with RGB as prompts for pre-trained foundation tracking model using prompt learning.

Result: Experimental results show significant improvements over existing dual-modal approaches in tracking accuracy and robustness in complex scenarios.

Conclusion: Tri-modal tracking with RGB, Depth, and Thermal infrared enhances robustness in complex scenarios, with proposed method and dataset advancing multi-modal tracking research.

Abstract: Existing multi-modal object tracking approaches primarily focus on dual-modal paradigms, such as RGB-Depth or RGB-Thermal, yet remain challenged in complex scenarios due to limited input modalities. To address this gap, this work introduces a novel multi-modal tracking task that leverages three complementary modalities, including visible RGB, Depth (D), and Thermal Infrared (TIR), aiming to enhance robustness in complex scenarios. To support this task, we construct a new multi-modal tracking dataset, coined RGBDT500, which consists of 500 videos with synchronised frames across the three modalities. Each frame provides spatially aligned RGB, depth, and thermal infrared images with precise object bounding box annotations. Furthermore, we propose a novel multi-modal tracker, dubbed RDTTrack. RDTTrack integrates tri-modal information for robust tracking by leveraging a pretrained RGB-only tracking model and prompt learning techniques. In specific, RDTTrack fuses thermal infrared and depth modalities under a proposed orthogonal projection constraint, then integrates them with RGB signals as prompts for the pre-trained foundation tracking model, effectively harmonising tri-modal complementary cues. The experimental results demonstrate the effectiveness and advantages of the proposed method, showing significant improvements over existing dual-modal approaches in terms of tracking accuracy and robustness in complex scenarios. The dataset and source code are publicly available at https://xuefeng-zhu5.github.io/RGBDT500.

Method: Proposes DA² with two key components: 1) Data curation engine generating ~543K panoramic RGB-depth pairs from perspective data, 2) SphereViT that leverages spherical coordinates to enforce geometric consistency in panoramic features.

Result: Achieves 38% average improvement on AbsRel over strongest zero-shot baseline, outperforms prior in-domain methods, and shows higher efficiency than fusion-based approaches.

Conclusion: DA² provides an accurate, zero-shot generalizable, and fully end-to-end solution for panoramic depth estimation with superior performance and efficiency.

Abstract: Panorama has a full FoV (360$^\circ\times$180$^\circ$), offering a more complete visual description than perspective images. Thanks to this characteristic, panoramic depth estimation is gaining increasing traction in 3D vision. However, due to the scarcity of panoramic data, previous methods are often restricted to in-domain settings, leading to poor zero-shot generalization. Furthermore, due to the spherical distortions inherent in panoramas, many approaches rely on perspective splitting (e.g., cubemaps), which leads to suboptimal efficiency. To address these challenges, we propose $\textbf{DA}$$^{\textbf{2}}$: $\textbf{D}$epth $\textbf{A}$nything in $\textbf{A}$ny $\textbf{D}$irection, an accurate, zero-shot generalizable, and fully end-to-end panoramic depth estimator. Specifically, for scaling up panoramic data, we introduce a data curation engine for generating high-quality panoramic depth data from perspective, and create $\sim$543K panoramic RGB-depth pairs, bringing the total to $\sim$607K. To further mitigate the spherical distortions, we present SphereViT, which explicitly leverages spherical coordinates to enforce the spherical geometric consistency in panoramic image features, yielding improved performance. A comprehensive benchmark on multiple datasets clearly demonstrates DA$^{2}$’s SoTA performance, with an average 38% improvement on AbsRel over the strongest zero-shot baseline. Surprisingly, DA$^{2}$ even outperforms prior in-domain methods, highlighting its superior zero-shot generalization. Moreover, as an end-to-end solution, DA$^{2}$ exhibits much higher efficiency over fusion-based approaches. Both the code and the curated panoramic data has be released. Project page: https://depth-any-in-any-dir.github.io/.

Method: ViSurf integrates SFT and RLVR in a single stage by injecting ground-truth labels into RLVR rollouts, providing simultaneous external supervision and internal reinforcement. It introduces three novel reward control strategies to stabilize training.

Result: Extensive experiments across diverse benchmarks show ViSurf outperforms individual SFT, RLVR, and two-stage SFT→RLVR approaches.

Conclusion: ViSurf provides a unified perspective on post-training paradigms, effectively combining the strengths of both SFT and RLVR while overcoming their individual limitations.

Abstract: Typical post-training paradigms for Large Vision-and-Language Models (LVLMs) include Supervised Fine-Tuning (SFT) and Reinforcement Learning with Verifiable Rewards (RLVR). SFT leverages external guidance to inject new knowledge, whereas RLVR utilizes internal reinforcement to enhance reasoning capabilities and overall performance. However, our analysis reveals that SFT often leads to sub-optimal performance, while RLVR struggles with tasks that exceed the model’s internal knowledge base. To address these limitations, we propose ViSurf (\textbf{Vi}sual \textbf{Su}pervised-and-\textbf{R}einforcement \textbf{F}ine-Tuning), a unified post-training paradigm that integrates the strengths of both SFT and RLVR within a single stage. We analyze the derivation of the SFT and RLVR objectives to establish the ViSurf objective, providing a unified perspective on these two paradigms. The core of ViSurf involves injecting ground-truth labels into the RLVR rollouts, thereby providing simultaneous external supervision and internal reinforcement. Furthermore, we introduce three novel reward control strategies to stabilize and optimize the training process. Extensive experiments across several diverse benchmarks demonstrate the effectiveness of ViSurf, outperforming both individual SFT, RLVR, and two-stage SFT \textrightarrow RLVR. In-depth analysis corroborates these findings, validating the derivation and design principles of ViSurf.

Method: FedHUG uses Minimal Bias Aggregation to handle heterogeneous non-IID features and Global Distribution-aware Learning Controller to mitigate label distribution skew and long-tail issues.

Result: The framework shows superior performance compared to state-of-the-art techniques for both RGB video and mmWave radar-based physiological estimation.

Conclusion: FedHUG successfully addresses the challenges of federated unsupervised domain generalization for remote physiological measurement while preserving privacy.

Abstract: Remote physiological measurement gained wide attention, while it requires collecting users’ privacy-sensitive information, and existing contactless measurements still rely on labeled client data. This presents challenges when we want to further update real-world deployed models with numerous user data lacking labels. To resolve these challenges, we instantiate a new protocol called Federated Unsupervised Domain Generalization (FUDG) in this work. Subsequently, the \textbf{Fed}erated \textbf{H}eterogeneous \textbf{U}nsupervised \textbf{G}eneralization (\textbf{FedHUG}) framework is proposed and consists of: (1) Minimal Bias Aggregation module dynamically adjusts aggregation weights based on prior-driven bias evaluation to cope with heterogeneous non-IID features from multiple domains. (2) The Global Distribution-aware Learning Controller parameterizes the label distribution and dynamically manipulates client-specific training strategies, thereby mitigating the server-client label distribution skew and long-tail issue. The proposal shows superior performance across state-of-the-art techniques in estimation with either RGB video or mmWave radar. The code will be released.

Method: Integrates 2D computer vision techniques including convex filtering and semantic feature supervision from foundational models like DINO to guide Gaussian splat densification and refinement using 2D segmentation priors and high-dimensional feature embeddings.

Result: Outperforms existing Gaussian Splatting in majority of evaluated scenes across three datasets, demonstrating improved coverage in underrepresented areas and better preservation of structural details.

Conclusion: Hybrid 2D-3D approaches combining 2D foundational models with 3D reconstruction pipelines can overcome limitations inherent in either approach alone, showing significant potential for enhanced scene reconstruction.

Abstract: Scene reconstruction has emerged as a central challenge in computer vision, with approaches such as Neural Radiance Fields (NeRF) and Gaussian Splatting achieving remarkable progress. While Gaussian Splatting demonstrates strong performance on large-scale datasets, it often struggles to capture fine details or maintain realism in regions with sparse coverage, largely due to the inherent limitations of sparse 3D training data. In this work, we propose GauSSmart, a hybrid method that effectively bridges 2D foundational models and 3D Gaussian Splatting reconstruction. Our approach integrates established 2D computer vision techniques, including convex filtering and semantic feature supervision from foundational models such as DINO, to enhance Gaussian-based scene reconstruction. By leveraging 2D segmentation priors and high-dimensional feature embeddings, our method guides the densification and refinement of Gaussian splats, improving coverage in underrepresented areas and preserving intricate structural details. We validate our approach across three datasets, where GauSSmart consistently outperforms existing Gaussian Splatting in the majority of evaluated scenes. Our results demonstrate the significant potential of hybrid 2D-3D approaches, highlighting how the thoughtful combination of 2D foundational models with 3D reconstruction pipelines can overcome the limitations inherent in either approach alone.

Method: Proposes GT-Pairs that automatically build preference pairs using real videos as positives and model-generated videos as negatives, eliminating external annotation. Introduces Reg-DPO that incorporates SFT loss as regularization into DPO loss. Combines FSDP framework with memory optimization techniques to achieve 3x higher training capacity.

Result: Extensive experiments on I2V and T2V tasks across multiple datasets show the method consistently outperforms existing approaches, delivering superior video generation quality.

Conclusion: The proposed Reg-DPO with GT-Pairs effectively addresses video generation challenges, providing stable training and high-quality results without requiring external annotations.

Abstract: Recent studies have identified Direct Preference Optimization (DPO) as an efficient and reward-free approach to improving video generation quality. However, existing methods largely follow image-domain paradigms and are mainly developed on small-scale models (approximately 2B parameters), limiting their ability to address the unique challenges of video tasks, such as costly data construction, unstable training, and heavy memory consumption. To overcome these limitations, we introduce a GT-Pair that automatically builds high-quality preference pairs by using real videos as positives and model-generated videos as negatives, eliminating the need for any external annotation. We further present Reg-DPO, which incorporates the SFT loss as a regularization term into the DPO loss to enhance training stability and generation fidelity. Additionally, by combining the FSDP framework with multiple memory optimization techniques, our approach achieves nearly three times higher training capacity than using FSDP alone. Extensive experiments on both I2V and T2V tasks across multiple datasets demonstrate that our method consistently outperforms existing approaches, delivering superior video generation quality.

Method: DOS modifies three types of CLIP text embeddings before feeding them into T2I models to better separate object representations and preserve inter-object relationships.

Result: DOS consistently improves multi-object generation success rates, reduces object mixing, and significantly outperforms four competing methods in human evaluations (26.24%-43.04% more votes across four benchmarks).

Conclusion: DOS is a practical and effective solution for improving multi-object image generation in text-to-image models.

Abstract: Recent progress in text-to-image (T2I) generative models has led to significant improvements in generating high-quality images aligned with text prompts. However, these models still struggle with prompts involving multiple objects, often resulting in object neglect or object mixing. Through extensive studies, we identify four problematic scenarios, Similar Shapes, Similar Textures, Dissimilar Background Biases, and Many Objects, where inter-object relationships frequently lead to such failures. Motivated by two key observations about CLIP embeddings, we propose DOS (Directional Object Separation), a method that modifies three types of CLIP text embeddings before passing them into text-to-image models. Experimental results show that DOS consistently improves the success rate of multi-object image generation and reduces object mixing. In human evaluations, DOS significantly outperforms four competing methods, receiving 26.24%-43.04% more votes across four benchmarks. These results highlight DOS as a practical and effective solution for improving multi-object image generation.

Method: Pre-computes holistic visual saliency maps by tracking positive attention changes (gaze shifts) during query comprehension, then amplifies attention to both salient visual information and user queries at each decoding step.

Result: Achieves up to 20.7% improvement over greedy decoding in hallucination mitigation across generative and classification tasks, while maintaining general vision-language performance with low computational overhead.

Conclusion: GIFT effectively addresses visual attention sink and cross-modal fusion imbalance, significantly reducing hallucination in VLMs without compromising overall performance or computational efficiency.

Abstract: Vision language models (VLMs) often generate hallucination, i.e., content that cannot be substantiated by either textual or visual inputs. Prior work primarily attributes this to over-reliance on linguistic prior knowledge rather than visual inputs. Some methods attempt to mitigate hallucination by amplifying visual token attention proportionally to their attention scores. However, these methods overlook the visual attention sink problem, where attention is frequently misallocated to task-irrelevant visual regions, and neglect cross-modal fusion balance by enhancing only visual attention without adjusting attention to the user query. This can result in amplifying incorrect areas while failing to properly interpret the user query. To address these challenges, we propose a simple yet effective method called Gaze Shift-Guided Cross-modal Fusion Enhancement (GIFT). GIFT pre-computes a holistic visual saliency map by tracking positive changes in visual attention, or “gaze shifts”, during user query comprehension, and leverages this map to amplify attention to both salient visual information and the user query at each decoding step. This reduces the impact of visual attention sink, as irrelevant tokens exhibit minimal shifts, while ensuring balanced cross-modal fusion for well-integrated representation. Extensive experiments show that GIFT effectively mitigates hallucination in VLMs across both generative and classification tasks, achieving up to 20.7% improvement over greedy decoding, while maintaining general vision-language performance with low computational overhead.

Method: S-GRACE leverages semantic guidance within concept space to generate adversarial samples and perform erasure training, addressing the limitations of existing methods that fail to properly fit concept spaces.

Result: Experiments show S-GRACE significantly improves erasure performance by 26%, better preserves non-target concepts, and reduces training time by 90% compared to seven state-of-the-art methods.

Conclusion: Semantics-guided adversarial training effectively addresses concept erasure in diffusion models, providing comprehensive coverage of target concepts while preserving non-target concepts and achieving substantial efficiency gains.

Abstract: Concept erasure aims to selectively unlearning undesirable content in diffusion models (DMs) to reduce the risk of sensitive content generation. As a novel paradigm in concept erasure, most existing methods employ adversarial training to identify and suppress target concepts, thus reducing the likelihood of sensitive outputs. However, these methods often neglect the specificity of adversarial training in DMs, resulting in only partial mitigation. In this work, we investigate and quantify this specificity from the perspective of concept space, i.e., can adversarial samples truly fit the target concept space? We observe that existing methods neglect the role of conceptual semantics when generating adversarial samples, resulting in ineffective fitting of concept spaces. This oversight leads to the following issues: 1) when there are few adversarial samples, they fail to comprehensively cover the object concept; 2) conversely, they will disrupt other target concept spaces. Motivated by the analysis of these findings, we introduce S-GRACE (Semantics-Guided Robust Adversarial Concept Erasure), which grace leveraging semantic guidance within the concept space to generate adversarial samples and perform erasure training. Experiments conducted with seven state-of-the-art methods and three adversarial prompt generation strategies across various DM unlearning scenarios demonstrate that S-GRACE significantly improves erasure performance 26%, better preserves non-target concepts, and reduces training time by 90%. Our code is available at https://github.com/Qhong-522/S-GRACE.

Method: Proposed MM-UNet architecture with two key innovations: Morph Mamba Convolution layers that replace pointwise convolutions to enhance branching topological perception through morph-aware feature sampling, and Reverse Selective State Guidance modules that integrate reverse guidance theory with state-space modeling to improve geometric boundary awareness and decoding efficiency.

Result: Extensive experiments on DRIVE and STARE datasets show superior performance, achieving F1-score gains of 1.64% on DRIVE and 1.25% on STARE compared to existing approaches, demonstrating significant improvement in segmentation accuracy.

Conclusion: MM-UNet effectively addresses the challenges of retinal vessel segmentation through its specialized architecture, providing more accurate and robust segmentation of thin, branching vascular structures for clinical applications.

Abstract: Accurate detection of retinal vessels plays a critical role in reflecting a wide range of health status indicators in the clinical diagnosis of ocular diseases. Recently, advances in deep learning have led to a surge in retinal vessel segmentation methods, which have significantly contributed to the quantitative analysis of vascular morphology. However, retinal vasculature differs significantly from conventional segmentation targets in that it consists of extremely thin and branching structures, whose global morphology varies greatly across images. These characteristics continue to pose challenges to segmentation precision and robustness. To address these issues, we propose MM-UNet, a novel architecture tailored for efficient retinal vessel segmentation. The model incorporates Morph Mamba Convolution layers, which replace pointwise convolutions to enhance branching topological perception through morph, state-aware feature sampling. Additionally, Reverse Selective State Guidance modules integrate reverse guidance theory with state-space modeling to improve geometric boundary awareness and decoding efficiency. Extensive experiments conducted on two public retinal vessel segmentation datasets demonstrate the superior performance of the proposed method in segmentation accuracy. Compared to the existing approaches, MM-UNet achieves F1-score gains of 1.64 % on DRIVE and 1.25 % on STARE, demonstrating its effectiveness and advancement. The project code is public via https://github.com/liujiawen-jpg/MM-UNet.

Method: Constructed a balanced dataset with 41.7% positive and 58.3% negative samples across six diagnostic question types, annotated with radiologist-verified bounding boxes and clinical reasoning.

Result: Benchmarking with MedGemma-4B-it showed improved performance (F1 = 0.624, +11.8% over baseline) while enabling lesion localization.

Conclusion: VinDr-CXR-VQA successfully advances explainable Med-VQA with spatial grounding capabilities and provides a valuable resource for reproducible research in medical AI.

Abstract: We present VinDr-CXR-VQA, a large-scale chest X-ray dataset for explainable Medical Visual Question Answering (Med-VQA) with spatial grounding. The dataset contains 17,597 question-answer pairs across 4,394 images, each annotated with radiologist-verified bounding boxes and clinical reasoning explanations. Our question taxonomy spans six diagnostic types-Where, What, Is there, How many, Which, and Yes/No-capturing diverse clinical intents. To improve reliability, we construct a balanced distribution of 41.7% positive and 58.3% negative samples, mitigating hallucinations in normal cases. Benchmarking with MedGemma-4B-it demonstrates improved performance (F1 = 0.624, +11.8% over baseline) while enabling lesion localization. VinDr-CXR-VQA aims to advance reproducible and clinically grounded Med-VQA research. The dataset and evaluation tools are publicly available at huggingface.co/datasets/Dangindev/VinDR-CXR-VQA.

Method: 1) Boundary Distance Regression (BDR): Uses signed-distance regression and zero-crossing extraction instead of classification. 2) Adaptive Temporal Refinement (ATR): Learns continuous depth allocation to adapt computation.

Result: BDR reduces boundary variance by 3.3x to 16.7x, improves mAP@0.7 by 1.8-3.1%. ATR achieves 56.5% mAP@0.7 at 151G FLOPs vs 53.6% at 198G for baseline (24% FLOPs reduction).

Conclusion: The proposed methods provide theoretically grounded boundary localization and adaptive computation, achieving significant efficiency gains while maintaining or improving accuracy.

Abstract: Temporal action localization requires precise boundaries, yet most methods apply uniform computation despite varying boundary difficulty. We propose two complementary contributions. Boundary Distance Regression (BDR) replaces classification with signed-distance regression and zero-crossing extraction. Under idealized assumptions (i.i.d. Laplace noise, uniform stride, sufficient capacity), BDR approaches the Cramer-Rao lower bound, yielding variance on the order of (Delta t)^2 / T (appearing as O((Delta t)^2) for fixed-video inference). The variance ratio R = Var[b_BDR] / Var[b_cls] scales as O((Delta t)^2 / W) for plateau width W approx 2*kappa, with empirical scaling appearing stronger (O((Delta t)^2 / W^2)) due to amplification factors (see Section~4). Empirically, BDR reduces boundary variance by 3.3x to 16.7x (R = 0.06 to 0.30) via four amplification factors. BDR retrofits to existing methods with about 50 lines of code, improving mAP@0.7 by 1.8 to 3.1 percent (average +2.4). Adaptive Temporal Refinement (ATR) learns continuous depth allocation tau in [0,1] to adapt computation, avoiding discrete routing complexity. On THUMOS14, ATR achieves 56.5 percent mAP@0.7 at 151G FLOPs versus 53.6 percent at 198G for the Uniform-6 baseline (24 percent FLOPs reduction, 118 ms vs. 167 ms latency). Gains scale with boundary heterogeneity: THUMOS14 (+2.9), FineAction (+2.7), ActivityNet (+1.8). Training overhead (1.29x baseline) is mitigated via knowledge distillation, with students retaining 99.5 percent performance. Code will be released.

Method: Two-stage training pipeline: cold-start stage to establish tool-use patterns, followed by reinforcement learning stage to refine tool invocation. Uses a diverse, moderately challenging training dataset with examples where tool use is beneficial.

Result: DeepEyesV2 demonstrates effectiveness across real-world understanding, mathematical reasoning, and search-intensive tasks, exhibiting task-adaptive tool invocation and complex tool combinations.

Conclusion: The study provides guidance for developing agentic multimodal models through the proposed training pipeline and evaluation framework.

Abstract: Agentic multimodal models should not only comprehend text and images, but also actively invoke external tools, such as code execution environments and web search, and integrate these operations into reasoning. In this work, we introduce DeepEyesV2 and explore how to build an agentic multimodal model from the perspectives of data construction, training methods, and model evaluation. We observe that direct reinforcement learning alone fails to induce robust tool-use behavior. This phenomenon motivates a two-stage training pipeline: a cold-start stage to establish tool-use patterns, and reinforcement learning stage to further refine tool invocation. We curate a diverse, moderately challenging training dataset, specifically including examples where tool use is beneficial. We further introduce RealX-Bench, a comprehensive benchmark designed to evaluate real-world multimodal reasoning, which inherently requires the integration of multiple capabilities, including perception, search, and reasoning. We evaluate DeepEyesV2 on RealX-Bench and other representative benchmarks, demonstrating its effectiveness across real-world understanding, mathematical reasoning, and search-intensive tasks. Moreover, DeepEyesV2 exhibits task-adaptive tool invocation, tending to use image operations for perception tasks and numerical computations for reasoning tasks. Reinforcement learning further enables complex tool combinations and allows model to selectively invoke tools based on context. We hope our study can provide guidance for community in developing agentic multimodal models.

Method: Proposed improved evaluation metrics including Density-Aware Chamfer Distance (DCD) and Surface Normal Concordance (SNC), and developed Diffusion Point Transformer architecture using transformer-based models for point cloud generation.

Result: The model outperforms previous solutions on ShapeNet dataset, achieving state-of-the-art performance in generating high-fidelity point clouds with better geometric quality.

Conclusion: Combining improved evaluation metrics with the novel Diffusion Point Transformer architecture provides more comprehensive assessment and superior generation of 3D point clouds.

Abstract: As 3D point clouds become a cornerstone of modern technology, the need for sophisticated generative models and reliable evaluation metrics has grown exponentially. In this work, we first expose that some commonly used metrics for evaluating generated point clouds, particularly those based on Chamfer Distance (CD), lack robustness against defects and fail to capture geometric fidelity and local shape consistency when used as quality indicators. We further show that introducing samples alignment prior to distance calculation and replacing CD with Density-Aware Chamfer Distance (DCD) are simple yet essential steps to ensure the consistency and robustness of point cloud generative model evaluation metrics. While existing metrics primarily focus on directly comparing 3D Euclidean coordinates, we present a novel metric, named Surface Normal Concordance (SNC), which approximates surface similarity by comparing estimated point normals. This new metric, when combined with traditional ones, provides a more comprehensive evaluation of the quality of generated samples. Finally, leveraging recent advancements in transformer-based models for point cloud analysis, such as serialized patch attention , we propose a new architecture for generating high-fidelity 3D structures, the Diffusion Point Transformer. We perform extensive experiments and comparisons on the ShapeNet dataset, showing that our model outperforms previous solutions, particularly in terms of quality of generated point clouds, achieving new state-of-the-art. Code available at https://github.com/matteo-bastico/DiffusionPointTransformer.

Method: Cannot determine method without access to the paper content

Result: Cannot determine results without access to the paper content

Conclusion: Cannot determine conclusion without access to the paper content

Abstract: Failed to fetch summary for 2511.06361: Page request resulted in HTTP 429 (https://export.arxiv.org/api/query?search_query=&id_list=2511.06361&sortBy=relevance&sortOrder=descending&start=0&max_results=100)

Method: Dual governance gates: pre-execution Approval Gate validates acceptance criteria schemas before code runs; post-execution Verification Gate validates artifacts via MLflow API queries with recursive path checking and optional metric validation. Claims propagate only when backed by queryable run ID, required artifacts, and FINISHED status.

Result: Achieved 0% hallucination: 7/8 tasks verified and 1 task correctly blocked at approval gate, with only ~8.3% execution overhead. Baseline A had 100% hallucination, Baseline B had 25% hallucination.

Conclusion: Research integrity is an architectural property achieved through governance gates rather than emergent from model scale. The framework provides verifiable execution with minimal overhead.

Abstract: LLM-based autonomous research agents report false claims: tasks marked “complete” despite missing artifacts, contradictory metrics, or failed executions. EviBound is an evidence-bound execution framework that eliminates false claims through dual governance gates requiring machine-checkable evidence. Two complementary gates enforce evidence requirements. The pre-execution Approval Gate validates acceptance criteria schemas before code runs, catching structural violations proactively. The post-execution Verification Gate validates artifacts via MLflow API queries (with recursive path checking) and optionally validates metrics when specified by acceptance criteria. Claims propagate only when backed by a queryable run ID, required artifacts, and FINISHED status. Bounded, confidence-gated retries (typically 1-2 attempts) recover from transient failures without unbounded loops. The framework was evaluated on 8 benchmark tasks spanning infrastructure validation, ML capabilities, and governance stress tests. Baseline A (Prompt-Level Only) yields 100% hallucination (8/8 claimed, 0/8 verified). Baseline B (Verification-Only) reduces hallucination to 25% (2/8 fail verification). EviBound (Dual Gates) achieves 0% hallucination: 7/8 tasks verified and 1 task correctly blocked at the approval gate, all with only approximately 8.3% execution overhead. This package includes execution trajectories, MLflow run IDs for all verified tasks, and a 4-step verification protocol. Research integrity is an architectural property, achieved through governance gates rather than emergent from model scale.

Method: Unable to analyze method as paper content could not be retrieved

Result: Unable to analyze results as paper content could not be retrieved

Conclusion: Unable to analyze conclusion as paper content could not be retrieved

Abstract: Failed to fetch summary for 2511.05549: Page request resulted in HTTP 429 (https://export.arxiv.org/api/query?search_query=&id_list=2511.05549&sortBy=relevance&sortOrder=descending&start=0&max_results=100)

Method: Unable to identify the methodology used in the paper due to missing abstract data.

Result: No results can be analyzed since the paper content could not be retrieved.

Conclusion: Analysis incomplete - server rate limiting prevented access to the paper’s abstract and content.

Abstract: Failed to fetch summary for 2511.06779: Page request resulted in HTTP 429 (https://export.arxiv.org/api/query?search_query=&id_list=2511.06779&sortBy=relevance&sortOrder=descending&start=0&max_results=100)

Method: Represent debate traces as directed interaction graphs with nodes encoding reasoning steps and edges capturing continuity and influence. Train a student model with composite objectives including language modeling, graph supervision, contrastive reasoning, and embedding alignment.

Result: SMAGDi compresses a 40B multi-agent system into a 6B student while retaining 88% accuracy on StrategyQA and MMLU, outperforming prior distillation methods like MAGDi, standard KD, and fine-tuned baselines.

Conclusion: Explicitly modeling interaction graphs and Socratic decomposition enables small models to inherit multi-agent debate accuracy benefits while remaining efficient for real-world deployment.

Abstract: Multi-agent systems (MAS) often achieve higher reasoning accuracy than single models, but their reliance on repeated debates across agents makes them computationally expensive. We introduce SMAGDi, a distillation framework that transfers the debate dynamics of a five-agent Llama-based MAS into a compact Socratic decomposer-solver student. SMAGDi represents debate traces as directed interaction graphs, where nodes encode intermediate reasoning steps with correctness labels and edges capture continuity and cross-agent influence. The student is trained with a composite objective combining language modeling, graph-based supervision, contrastive reasoning, and embedding alignment to preserve both fluency and structured reasoning. On StrategyQA and MMLU, SMAGDi compresses a 40B multi-agent system into a 6B student while retaining 88% of its accuracy, substantially outperforming prior distillation methods such as MAGDi, standard KD, and fine-tuned baselines. These results highlight that explicitly modeling interaction graphs and Socratic decomposition enable small models to inherit the accuracy benefits of multi-agent debate while remaining efficient enough for real-world deployment.

Method: Conducted large-scale measurement-based study using vLLM inference engine, quantifying energy usage at prompt level across diverse hardware (21 GPU configurations) and model architectures (155 models from small to frontier systems).

Result: Identified how architectural and operational factors shape energy demand, developed accurate predictive model for estimating inference energy consumption across unseen architectures and hardware, and implemented as browser extension.

Conclusion: Provides comprehensive energy consumption insights for LLM inference, enabling better environmental awareness and more sustainable AI deployment through predictive modeling and user-facing tools.

Abstract: The rapid expansion of Large Language Models (LLMs) has introduced unprecedented energy demands, extending beyond training to large-scale inference workloads that often dominate total lifecycle consumption. Deploying these models requires energy-intensive GPU infrastructure, and in some cases has even prompted plans to power data centers with nuclear energy. Despite this growing relevance, systematic analyses of inference energy consumption remain limited. In this work, we present a large-scale measurement-based study comprising over 32,500 measurements across 21 GPU configurations and 155 model architectures, from small open-source models to frontier systems. Using the vLLM inference engine, we quantify energy usage at the prompt level and identify how architectural and operational factors shape energy demand. Building on these insights, we develop a predictive model that accurately estimates inference energy consumption across unseen architectures and hardware, and implement it as a browser extension to raise awareness of the environmental impact of generative AI.

Method: Adaptive reasoning summarization with semantic segmentation, importance scoring, budget-aware dynamic compression, and coherence reconstruction to compress reasoning traces.

Result: 40% higher accuracy than truncation on medical questions, strong cross-model transferability across 64 model pairs, 84% evaluation cost reduction via Bayesian optimization.

Conclusion: Reasoning summarization enables efficient CoT transfer, allowing advanced reasoning under tight computational constraints.

Abstract: Chain-of-Thought (CoT) reasoning enhances the problem-solving ability of large language models (LLMs) but leads to substantial inference overhead, limiting deployment in resource-constrained settings. This paper investigates efficient CoT transfer across models of different scales and architectures through an adaptive reasoning summarization framework. The proposed method compresses reasoning traces via semantic segmentation with importance scoring, budget-aware dynamic compression, and coherence reconstruction, preserving critical reasoning steps while significantly reducing token usage. Experiments on 7{,}501 medical examination questions across 10 specialties show up to 40% higher accuracy than truncation under the same token budgets. Evaluations on 64 model pairs from eight LLMs (1.5B-32B parameters, including DeepSeek-R1 and Qwen3) confirm strong cross-model transferability. Furthermore, a Gaussian Process-based Bayesian optimization module reduces evaluation cost by 84% and reveals a power-law relationship between model size and cross-domain robustness. These results demonstrate that reasoning summarization provides a practical path toward efficient CoT transfer, enabling advanced reasoning under tight computational constraints. Code will be released upon publication.

Method: Introduces two modalities for de re and de dicto awareness, uses 2D-semantics for formal specification, and develops a logical system for their interplay with standard knowledge.

Result: A sound and complete logical system that describes the relationships between de re awareness, de dicto awareness, and standard knowledge of facts.

Conclusion: The proposed framework successfully formalizes agent awareness as knowledge with distinct de re and de dicto forms, providing a complete logical system for reasoning about their interactions.

Abstract: The paper proposes to treat agent awareness as a form of knowledge, breaking the tradition in the existing literature on awareness. It distinguishes the de re and de dicto forms of such knowledge. The work introduces two modalities capturing these forms and formally specifies their meaning using a version of 2D-semantics. The main technical result is a sound and complete logical system describing the interplay between the two proposed modalities and the standard “knowledge of the fact” modality.

Method: Three-pronged approach: (1) log-probability-based behavioral analysis with training-data contamination controls, (2) Shapley-value attribution over structured prompt fields, and (3) unified Anchoring Bias Sensitivity Score integrating behavioral and attributional evidence across six models.

Result: Robust anchoring effects found in Gemma-2B, Phi-2, and Llama-2-7B with attribution showing anchor influence reweighting. Smaller models (GPT-2, Falcon-RW-1B, GPT-Neo-125M) show variability, suggesting scale modulates sensitivity. Attributional effects vary across prompt designs.

Conclusion: Anchoring bias in LLMs is robust, measurable, and interpretable, but attribution effects are fragile, highlighting risks in treating LLMs as human substitutes. The framework bridges behavioral science, LLM safety, and interpretability for evaluating cognitive biases.

Abstract: Large language models (LLMs) are increasingly examined as both behavioral subjects and decision systems, yet it remains unclear whether observed cognitive biases reflect surface imitation or deeper probability shifts. Anchoring bias, a classic human judgment bias, offers a critical test case. While prior work shows LLMs exhibit anchoring, most evidence relies on surface-level outputs, leaving internal mechanisms and attributional contributions unexplored. This paper advances the study of anchoring in LLMs through three contributions: (1) a log-probability-based behavioral analysis showing that anchors shift entire output distributions, with controls for training-data contamination; (2) exact Shapley-value attribution over structured prompt fields to quantify anchor influence on model log-probabilities; and (3) a unified Anchoring Bias Sensitivity Score integrating behavioral and attributional evidence across six open-source models. Results reveal robust anchoring effects in Gemma-2B, Phi-2, and Llama-2-7B, with attribution signaling that the anchors influence reweighting. Smaller models such as GPT-2, Falcon-RW-1B, and GPT-Neo-125M show variability, suggesting scale may modulate sensitivity. Attributional effects, however, vary across prompt designs, underscoring fragility in treating LLMs as human substitutes. The findings demonstrate that anchoring bias in LLMs is robust, measurable, and interpretable, while highlighting risks in applied domains. More broadly, the framework bridges behavioral science, LLM safety, and interpretability, offering a reproducible path for evaluating other cognitive biases in LLMs.

Method: Proposes Maestro framework with parallel Execution Agents for exploration and Central Agent for synthesis, plus CLPO reinforcement learning that combines policy gradients with list-wise ranking loss for clean credit assignment.

Result: Experiments show Maestro with CLPO consistently outperforms state-of-the-art multi-agent approaches, achieving absolute accuracy gains of 6% on average and up to 10% at best on mathematical reasoning and problem-solving benchmarks.

Conclusion: Structural decoupling of exploration and synthesis through Maestro framework with CLPO effectively resolves core cognitive tension in multi-agent LLM systems, enabling superior performance through principled collaboration.

Abstract: Multi-agent systems (MAS) built on Large Language Models (LLMs) are being used to approach complex problems and can surpass single model inference. However, their success hinges on navigating a fundamental cognitive tension: the need to balance broad, divergent exploration of the solution space with a principled, convergent synthesis to the optimal solution. Existing paradigms often struggle to manage this duality, leading to premature consensus, error propagation, and a critical credit assignment problem that fails to distinguish between genuine reasoning and superficially plausible arguments. To resolve this core challenge, we propose the Multi-Agent Exploration-Synthesis framework Through Role Orchestration (Maestro), a principled paradigm for collaboration that structurally decouples these cognitive modes. Maestro uses a collective of parallel Execution Agents for diverse exploration and a specialized Central Agent for convergent, evaluative synthesis. To operationalize this critical synthesis phase, we introduce Conditional Listwise Policy Optimization (CLPO), a reinforcement learning objective that disentangles signals for strategic decisions and tactical rationales. By combining decision-focused policy gradients with a list-wise ranking loss over justifications, CLPO achieves clean credit assignment and stronger comparative supervision. Experiments on mathematical reasoning and general problem-solving benchmarks demonstrate that Maestro, coupled with CLPO, consistently outperforms existing state-of-the-art multi-agent approaches, delivering absolute accuracy gains of 6% on average and up to 10% at best.

Method: Integrates GP-unmix (Gaussian Process-based hierarchical model for cell-type-specific gene expression inference), eQTL-guided neural classifier, and LLM-based reasoning module that translates model outputs into audience-specific diagnostic reports.

Result: Achieves 88.0% accuracy in Alzheimer’s Disease detection. Human evaluations confirm that the generated reports are accurate, actionable, and appropriately tailored for both physicians and patients.

Conclusion: LLMs, when deployed as post-hoc reasoners rather than end-to-end predictors, can serve as effective communicators within hybrid diagnostic pipelines.

Abstract: Building trustworthy clinical AI systems requires not only accurate predictions but also transparent, biologically grounded explanations. We present \texttt{DiagnoLLM}, a hybrid framework that integrates Bayesian deconvolution, eQTL-guided deep learning, and LLM-based narrative generation for interpretable disease diagnosis. DiagnoLLM begins with GP-unmix, a Gaussian Process-based hierarchical model that infers cell-type-specific gene expression profiles from bulk and single-cell RNA-seq data while modeling biological uncertainty. These features, combined with regulatory priors from eQTL analysis, power a neural classifier that achieves high predictive performance in Alzheimer’s Disease (AD) detection (88.0% accuracy). To support human understanding and trust, we introduce an LLM-based reasoning module that translates model outputs into audience-specific diagnostic reports, grounded in clinical features, attribution signals, and domain knowledge. Human evaluations confirm that these reports are accurate, actionable, and appropriately tailored for both physicians and patients. Our findings show that LLMs, when deployed as post-hoc reasoners rather than end-to-end predictors, can serve as effective communicators within hybrid diagnostic pipelines.

Method: LEAP formulates hallucination detection as dynamic strategy learning, using teacher models to generate trajectories and distill dynamic planning into efficient student models via agent tuning with proactive correction mechanisms.

Result: LEAP-tuned models outperform state-of-the-art methods on three challenging benchmarks.

Conclusion: The LEAP framework successfully endows efficient student models with dynamic learning and proactive correction capabilities, solving the strategy adaptability problem in hallucination detection.

Abstract: Hallucination in large language models (LLMs) remains a critical barrier to their safe deployment. Existing tool-augmented hallucination detection methods require pre-defined fixed verification strategies, which are crucial to the quality and effectiveness of tool calls. Some methods directly employ powerful closed-source LLMs such as GPT-4 as detectors, which are effective but too costly. To mitigate the cost issue, some methods adopt the teacher-student architecture and finetune open-source small models as detectors via agent tuning. However, these methods are limited by fixed strategies. When faced with a dynamically changing execution environment, they may lack adaptability and inappropriately call tools, ultimately leading to detection failure. To address the problem of insufficient strategy adaptability, we propose the innovative ``Learning to Evaluate and Adaptively Plan’’(LEAP) framework, which endows an efficient student model with the dynamic learning and proactive correction capabilities of the teacher model. Specifically, our method formulates the hallucination detection problem as a dynamic strategy learning problem. We first employ a teacher model to generate trajectories within the dynamic learning loop and dynamically adjust the strategy based on execution failures. We then distill this dynamic planning capability into an efficient student model via agent tuning. Finally, during strategy execution, the student model adopts a proactive correction mechanism, enabling it to propose, review, and optimize its own verification strategies before execution. We demonstrate through experiments on three challenging benchmarks that our LEAP-tuned model outperforms existing state-of-the-art methods.

Method: Developed STATION as a multi-agent environment where agents leverage extended context windows to read papers, formulate hypotheses, submit code, perform analyses, and publish results independently, allowing emergent behavior and organic method development.

Result: AI agents achieved state-of-the-art performance across mathematics, computational biology, and machine learning benchmarks, notably surpassing AlphaEvolve in circle packing. Emergent narratives produced novel methods like a density-adaptive algorithm for scRNA-seq batch integration.

Conclusion: STATION represents a new paradigm for autonomous scientific discovery driven by emergent behavior in open-world environments, demonstrating the potential for AI agents to organically develop novel methods and advance scientific frontiers.

Abstract: We introduce the STATION, an open-world multi-agent environment that models a miniature scientific ecosystem. Leveraging their extended context windows, agents in the Station can engage in long scientific journeys that include reading papers from peers, formulating hypotheses, submitting code, performing analyses, and publishing results. Importantly, there is no centralized system coordinating their activities - agents are free to choose their own actions and develop their own narratives within the Station. Experiments demonstrate that AI agents in the Station achieve new state-of-the-art performance on a wide range of benchmarks, spanning from mathematics to computational biology to machine learning, notably surpassing AlphaEvolve in circle packing. A rich tapestry of narratives emerges as agents pursue independent research, interact with peers, and build upon a cumulative history. From these emergent narratives, novel methods arise organically, such as a new density-adaptive algorithm for scRNA-seq batch integration. The Station marks a first step towards autonomous scientific discovery driven by emergent behavior in an open-world environment, representing a new paradigm that moves beyond rigid optimization.

Method: Evaluated 6 reasoning LLMs on 100 code generation tasks from BigCodeBench, quantified reasoning structure, conducted step-budget adjustments, and performed 21-participant human evaluation across efficiency, logical correctness, and completeness.

Result: Targeted step increases improved resolution rates for certain models/tasks; modest step reductions preserved success on standard tasks but rarely on hard ones; completeness identified as dominant failure mode; hard problems more prone to incompleteness; models maintain consistent logical structures and can self-correct errors.

Conclusion: Provides insights into strengths and limitations of current thinking LLMs in software engineering, revealing systematic patterns in reasoning quality and failure modes.

Abstract: Thinking Large Language Models (LLMs) generate explicit intermediate reasoning traces before final answers, potentially improving transparency, interpretability, and solution accuracy for code generation. However, the quality of these reasoning chains remains underexplored. We present a comprehensive empirical study examining the reasoning process and quality of thinking LLMs for code generation. We evaluate six state-of-the-art reasoning LLMs (DeepSeek-R1, OpenAI-o3-mini, Claude-3.7-Sonnet-Thinking, Gemini-2.0-Flash-Thinking, Gemini-2.5-Flash, and Qwen-QwQ) across 100 code generation tasks of varying difficulty from BigCodeBench. We quantify reasoning-chain structure through step counts and verbosity, conduct controlled step-budget adjustments, and perform a 21-participant human evaluation across three dimensions: efficiency, logical correctness, and completeness. Our step-count interventions reveal that targeted step increases can improve resolution rates for certain models/tasks, while modest reductions often preserve success on standard tasks, rarely on hard ones. Through systematic analysis, we develop a reasoning-problematic taxonomy, identifying completeness as the dominant failure mode. Task complexity significantly impacts reasoning quality; hard problems are substantially more prone to incompleteness than standard tasks. Our stability analysis demonstrates that thinking LLMs maintain consistent logical structures across computational effort levels and can self-correct previous errors. This study provides new insights into the strengths and limitations of current thinking LLMs in software engineering.

Method: Three bias quantification methods: 1) coarse-grained modality benefit evaluation, 2) medium-grained information flow quantification, and 3) fine-grained causality analysis. Validated through human evaluation on popular benchmarks.

Result: Three key findings: ensemble of multiple views is crucial for reliable automated analysis; automated analysis is prone to detector-induced fluctuations; different views produce higher agreement on modality-balanced samples but diverge on biased ones.

Conclusion: The proposed automated bias recognition framework provides meaningful sample-level insights and reveals important patterns about modality bias, offering potential directions for future research in multimodal misinformation detection.

Abstract: Numerous multimodal misinformation benchmarks exhibit bias toward specific modalities, allowing detectors to make predictions based solely on one modality. While previous research has quantified bias at the dataset level or manually identified spurious correlations between modalities and labels, these approaches lack meaningful insights at the sample level and struggle to scale to the vast amount of online information. In this paper, we investigate the design for automated recognition of modality bias at the sample level. Specifically, we propose three bias quantification methods based on theories/views of different levels of granularity: 1) a coarse-grained evaluation of modality benefit; 2) a medium-grained quantification of information flow; and 3) a fine-grained causality analysis. To verify the effectiveness, we conduct a human evaluation on two popular benchmarks. Experimental results reveal three interesting findings that provide potential direction toward future research: 1)~Ensembling multiple views is crucial for reliable automated analysis; 2)~Automated analysis is prone to detector-induced fluctuations; and 3)~Different views produce a higher agreement on modality-balanced samples but diverge on biased ones.

Method: After initial rollout, agents induce concise plan abstractions from grounded experience, extracting key steps, dependencies, and constraints, then use these as contextual guidance for refined subsequent executions.

Result: SAGE achieves 7.2% relative improvement over Mini-SWE-Agent baseline with GPT-5, and reaches 73.2%-74% Pass@1 resolve rates on SWE-Bench Verified benchmark.

Conclusion: SAGE framework enables consistent performance gains across diverse LLM backbones and agent architectures through self-abstraction from experience.

Abstract: Large language model (LLM) based agents are increasingly used to tackle software engineering tasks that require multi-step reasoning and code modification, demonstrating promising yet limited performance. However, most existing LLM agents typically operate within static execution frameworks, lacking a principled mechanism to learn and self-improve from their own experience and past rollouts. As a result, their performance remains bounded by the initial framework design and the underlying LLM’s capabilities. We propose Self-Abstraction from Grounded Experience (SAGE), a framework that enables agents to learn from their own task executions and refine their behavior through self-abstraction. After an initial rollout, the agent induces a concise plan abstraction from its grounded experience, distilling key steps, dependencies, and constraints. This learned abstraction is then fed back as contextual guidance, refining the agent’s policy and supporting more structured, informed subsequent executions. Empirically, SAGE delivers consistent performance gains across diverse LLM backbones and agent architectures. Notably, it yields a 7.2% relative performance improvement over the strong Mini-SWE-Agent baseline when paired with the GPT-5 (high) backbone. SAGE further achieves strong overall performance on SWE-Bench Verified benchmark, reaching 73.2% and 74% Pass@1 resolve rates with the Mini-SWE-Agent and OpenHands CodeAct agent framework, respectively.

Method: Comprehensive pipeline with supervised fine-tuning using synthetic data followed by multi-turn reinforcement learning for diverse agentic tasks.

Result: Achieves state-of-the-art performance among LLMs of similar size and remains competitive with significantly larger models on various agentic benchmarks.

Conclusion: Successfully developed fully open-source pipeline for training high-performance agentic models that can interact with external tools and environments.

Abstract: Despite the proliferation of powerful agentic models, the lack of critical post-training details hinders the development of strong counterparts in the open-source community. In this study, we present a comprehensive and fully open-source pipeline for training a high-performance agentic model for interacting with external tools and environments, named Klear-Qwen3-AgentForge, starting from the Qwen3-8B base model. We design effective supervised fine-tuning (SFT) with synthetic data followed by multi-turn reinforcement learning (RL) to unlock the potential for multiple diverse agentic tasks. We perform exclusive experiments on various agentic benchmarks in both tool use and coding domains. Klear-Qwen3-AgentForge-8B achieves state-of-the-art performance among LLMs of similar size and remains competitive with significantly larger models.

Method: Comparative analysis of three scenarios: fully manual, AI-assisted (human-in-the-loop), and advanced multi-agent agentic AI workflow with parsers and verifiers.

Result: AI-assisted and agentic AI scenarios significantly outperform manual processes in sustainability metrics, with full agentic configurations achieving the best environmental performance despite slightly higher resource usage than simpler AI solutions.

Conclusion: The framework successfully integrates performance, energy, and emission indicators into a unified ESG-oriented methodology for assessing AI-enabled supply chain solutions, demonstrating substantial sustainability gains through agentic AI adoption.

Abstract: This paper presents a comprehensive sustainability assessment framework for document intelligence within supply chain operations, centered on agentic artificial intelligence (AI). We address the dual objective of improving automation efficiency while providing measurable environmental performance in document-intensive workflows. The research compares three scenarios: fully manual (human-only), AI-assisted (human-in-the-loop, HITL), and an advanced multi-agent agentic AI workflow leveraging parsers and verifiers. Empirical results show that AI-assisted HITL and agentic AI scenarios achieve reductions of up to 70-90% in energy consumption, 90-97% in carbon dioxide emissions, and 89-98% in water usage compared to manual processes. Notably, full agentic configurations, combining advanced reasoning (thinking mode) and multi-agent validation, achieve substantial sustainability gains over human-only approaches, even when resource usage increases slightly versus simpler AI-assisted solutions. The framework integrates performance, energy, and emission indicators into a unified ESG-oriented methodology for assessing and governing AI-enabled supply chain solutions. The paper includes a complete replicability use case demonstrating the methodology’s application to real-world document extraction tasks.

Method: Two-stage approach: (1) Trial-and-error learning with GRPO to collect incorrect answers in an error pool, (2) Self-correction learning where the model reflects on why previous answers were wrong.

Result: Outperforms several post-training methods across multiple math reasoning benchmarks including AIME, AMC, Olympiad, MATH-500, and GSM8k using Deepseek-Distill-Qwen models.

Conclusion: ScRPO is a promising paradigm for enabling language models to self-improve on difficult tasks with limited external feedback, paving the way toward more reliable and capable AI systems.

Abstract: We propose Self-correction Relative Policy Optimization (ScRPO), a novel reinforcement learning framework designed to enhance large language models on challenging mathematical problems by leveraging self-reflection and error correction. Our approach consists of two stages: (1) Trial-and-error learning stage: training the model with GRPO and collecting incorrect answers along with their corresponding questions in an error pool; (2) Self-correction learning stage: guiding the model to reflect on why its previous answers were wrong. Extensive experiments across multiple math reasoning benchmarks, including AIME, AMC, Olympiad, MATH-500, GSM8k, using Deepseek-Distill-Qwen-1.5B and Deepseek-Distill-Qwen-7B. The experimental results demonstrate that ScRPO consistently outperforms several post-training methods. These findings highlight ScRPO as a promising paradigm for enabling language models to self-improve on difficult tasks with limited external feedback, paving the way toward more reliable and capable AI systems.

Method: Introduce DLPWM, a fully unsupervised disentangled object-centric world model that learns object-level latents directly from pixels.

Result: DLPWM achieves strong reconstruction and prediction performance with OOD robustness, but policies trained on its latents underperform DreamerV3 due to representation shift during multi-object interactions.

Conclusion: Object-centric perception supports robust visual modeling, but achieving stable control requires mitigating latent drift caused by representation shift during interactions.

Abstract: Object-centric world models (OCWM) aim to decompose visual scenes into object-level representations, providing structured abstractions that could improve compositional generalization and data efficiency in reinforcement learning. We hypothesize that explicitly disentangled object-level representations, by localizing task-relevant information, can enhance policy performance across novel feature combinations. To test this hypothesis, we introduce DLPWM, a fully unsupervised, disentangled object-centric world model that learns object-level latents directly from pixels. DLPWM achieves strong reconstruction and prediction performance, including robustness to several out-of-distribution (OOD) visual variations. However, when used for downstream model-based control, policies trained on DLPWM latents underperform compared to DreamerV3. Through latent-trajectory analyses, we identify representation shift during multi-object interactions as a key driver of unstable policy learning. Our results suggest that, although object-centric perception supports robust visual modeling, achieving stable control requires mitigating latent drift.

Method: Developed a LLM-powered simulator for OSN conversations that enables parallel counterfactual simulations where toxic behavior is influenced by moderation interventions while keeping all other factors constant.

Result: Experiments demonstrated psychological realism of OSN agents, emergence of social contagion phenomena, and superior effectiveness of personalized moderation strategies compared to standard approaches.

Conclusion: LLM-powered simulation provides a viable approach for evaluating content moderation strategies, revealing the importance of personalized interventions and enabling controlled experimentation that was previously impractical.

Abstract: Online Social Networks (OSNs) widely adopt content moderation to mitigate the spread of abusive and toxic discourse. Nonetheless, the real effectiveness of moderation interventions remains unclear due to the high cost of data collection and limited experimental control. The latest developments in Natural Language Processing pave the way for a new evaluation approach. Large Language Models (LLMs) can be successfully leveraged to enhance Agent-Based Modeling and simulate human-like social behavior with unprecedented degree of believability. Yet, existing tools do not support simulation-based evaluation of moderation strategies. We fill this gap by designing a LLM-powered simulator of OSN conversations enabling a parallel, counterfactual simulation where toxic behavior is influenced by moderation interventions, keeping all else equal. We conduct extensive experiments, unveiling the psychological realism of OSN agents, the emergence of social contagion phenomena and the superior effectiveness of personalized moderation strategies.

Method: Projects joint-action returns into low-dimensional space using contextual linear bandit formulation. Solves the bandit problem with convex and smooth loss functions, derives linear Upper Confidence Bound for Trees (LinUCT), and uses (1-1/e)-approximation algorithm for joint action selection with sub-modular objective.

Result: State-of-the-art performance on multi-agent benchmarks including matrix games, SMAC, and SMACv2. Outperforms both model-based and model-free multi-agent reinforcement learning baselines with faster learning speed and better performance.

Conclusion: MALinZero successfully addresses the exponential complexity of multi-agent MCTS through low-dimensional representations and contextual linear bandits, enabling efficient exploration and exploitation in complex multi-agent planning problems.

Abstract: Monte Carlo Tree Search (MCTS), which leverages Upper Confidence Bound for Trees (UCTs) to balance exploration and exploitation through randomized sampling, is instrumental to solving complex planning problems. However, for multi-agent planning, MCTS is confronted with a large combinatorial action space that often grows exponentially with the number of agents. As a result, the branching factor of MCTS during tree expansion also increases exponentially, making it very difficult to efficiently explore and exploit during tree search. To this end, we propose MALinZero, a new approach to leverage low-dimensional representational structures on joint-action returns and enable efficient MCTS in complex multi-agent planning. Our solution can be viewed as projecting the joint-action returns into the low-dimensional space representable using a contextual linear bandit problem formulation. We solve the contextual linear bandit problem with convex and $μ$-smooth loss functions – in order to place more importance on better joint actions and mitigate potential representational limitations – and derive a linear Upper Confidence Bound applied to trees (LinUCT) to enable novel multi-agent exploration and exploitation in the low-dimensional space. We analyze the regret of MALinZero for low-dimensional reward functions and propose an $(1-\tfrac1e)$-approximation algorithm for the joint action selection by maximizing a sub-modular objective. MALinZero demonstrates state-of-the-art performance on multi-agent benchmarks such as matrix games, SMAC, and SMACv2, outperforming both model-based and model-free multi-agent reinforcement learning baselines with faster learning speed and better performance.

Method: Uses logic grid puzzles with stereotypical, anti-stereotypical, and neutral variants generated from shared structures, enabling automatic verification and controlled comparisons across model families and puzzle complexities.

Result: Models consistently reason more accurately when puzzle solutions align with gender stereotypes, revealing implicit bias influence on deductive reasoning.

Conclusion: PRIME effectively diagnoses and quantifies social biases in LLMs’ reasoning processes, highlighting the need for better bias detection in critical fairness contexts.

Abstract: While recent safety guardrails effectively suppress overtly biased outputs, subtler forms of social bias emerge during complex logical reasoning tasks that evade current evaluation benchmarks. To fill this gap, we introduce a new evaluation framework, PRIME (Puzzle Reasoning for Implicit Biases in Model Evaluation), that uses logic grid puzzles to systematically probe the influence of social stereotypes on logical reasoning and decision making in LLMs. Our use of logic puzzles enables automatic generation and verification, as well as variability in complexity and biased settings. PRIME includes stereotypical, anti-stereotypical, and neutral puzzle variants generated from a shared puzzle structure, allowing for controlled and fine-grained comparisons. We evaluate multiple model families across puzzle sizes and test the effectiveness of prompt-based mitigation strategies. Focusing our experiments on gender stereotypes, our findings highlight that models consistently reason more accurately when solutions align with stereotypical associations. This demonstrates the significance of PRIME for diagnosing and quantifying social biases perpetuated in the deductive reasoning of LLMs, where fairness is critical.

Method: Proposes the Alignment Score metric and Semantic Consistency Optimization Sampling (SCOS) method that samples reasoning chains with minimal alignment errors (logical disconnection, thematic shift, redundant reasoning, causal reversal).

Result: Empirical findings show 2-hop reasoning chains achieve highest Alignment Score. SCOS improves Alignment Scores by average 29.84% with longer reasoning chains like 3-hop tasks.

Conclusion: The framework successfully quantifies reasoning consistency and the SCOS method effectively optimizes semantic alignment, particularly benefiting longer reasoning chains where consistency issues are more pronounced.

Abstract: This paper presents a framework for evaluating and optimizing reasoning consistency in Large Language Models (LLMs) via a new metric, the Alignment Score, which quantifies the semantic alignment between model-generated reasoning chains and human-written reference chains in Chain-of-Thought (CoT) reasoning. Empirically, we find that 2-hop reasoning chains achieve the highest Alignment Score. To explain this phenomenon, we define four key error types: logical disconnection, thematic shift, redundant reasoning, and causal reversal, and show how each contributes to the degradation of the Alignment Score. Building on this analysis, we further propose Semantic Consistency Optimization Sampling (SCOS), a method that samples and favors chains with minimal alignment errors, significantly improving Alignment Scores by an average of 29.84% with longer reasoning chains, such as in 3-hop tasks.

Method: The framework maps game events and dialogue to four constraint classes (evidence, phenomena, assertions, hypotheses), using hard constraints to prune impossible assignments and weighted soft constraints to score remaining possibilities with information-gain weighting.

Result: Experiments on three public datasets show CSP4SDG outperforms LLM-based baselines in all inference scenarios and boosts LLM performance when used as an auxiliary reasoning tool.

Conclusion: Principled probabilistic reasoning with information theory provides a scalable alternative or complement to heavyweight neural models for social deduction games, offering fully interpretable real-time role probability updates.

Abstract: In Social Deduction Games (SDGs) such as Avalon, Mafia, and Werewolf, players conceal their identities and deliberately mislead others, making hidden-role inference a central and demanding task. Accurate role identification, which forms the basis of an agent’s belief state, is therefore the keystone for both human and AI performance. We introduce CSP4SDG, a probabilistic, constraint-satisfaction framework that analyses gameplay objectively. Game events and dialogue are mapped to four linguistically-agnostic constraint classes-evidence, phenomena, assertions, and hypotheses. Hard constraints prune impossible role assignments, while weighted soft constraints score the remainder; information-gain weighting links each hypothesis to its expected value under entropy reduction, and a simple closed-form scoring rule guarantees that truthful assertions converge to classical hard logic with minimum error. The resulting posterior over roles is fully interpretable and updates in real time. Experiments on three public datasets show that CSP4SDG (i) outperforms LLM-based baselines in every inference scenario, and (ii) boosts LLMs when supplied as an auxiliary “reasoning tool.” Our study validates that principled probabilistic reasoning with information theory is a scalable alternative-or complement-to heavy-weight neural models for SDGs.

Method: Leverages large language model reasoning with grounded validation, performs automatic data cleaning, hierarchical routing, and feature-level optimization through dual feedback loops.

Result: Successfully demonstrates the first practical realization of an autonomous Data Agent that can transform raw data into better data without human supervision.

Conclusion: Data Agent provides a fully autonomous solution for data preparation that is automatic, safe, and non-expert friendly, ensuring end-to-end reliability in transforming raw data into AI-ready format.

Abstract: The growing demand for AI applications in fields such as materials discovery, molecular modeling, and climate science has made data preparation an important but labor-intensive step. Raw data from diverse sources must be cleaned, normalized, and transformed to become AI-ready, while effective feature transformation and selection are essential for efficient training and inference. To address the challenges of scalability and expertise dependence, we present Data Agent, a fully autonomous system specialized for tabular data. Leveraging large language model (LLM) reasoning and grounded validation, Data Agent automatically performs data cleaning, hierarchical routing, and feature-level optimization through dual feedback loops. It embodies three core principles: automatic, safe, and non-expert friendly, which ensure end-to-end reliability without human supervision. This demo showcases the first practical realization of an autonomous Data Agent, illustrating how raw data can be transformed “From Data to Better Data.”

Method: Train transformer-based uncertainty heads (UHeads) on frozen LLM internal states to estimate reasoning step uncertainty, using automatic labels from larger LLMs or self-supervised learning.

Result: UHeads (<10M parameters) match or surpass Process Reward Models (810x larger) across math, planning, and QA domains, showing LLM internal states encode reliable uncertainty signals.

Conclusion: LLM internal states effectively encode uncertainty for reasoning verification, enabling scalable introspective models without expensive verification systems.

Abstract: Solving complex tasks usually requires LLMs to generate long multi-step reasoning chains. Previous work has shown that verifying the correctness of individual reasoning steps can further improve the performance and efficiency of LLMs on such tasks and enhance solution interpretability. However, existing verification approaches, such as Process Reward Models (PRMs), are either computationally expensive, limited to specific domains, or require large-scale human or model-generated annotations. Thus, we propose a lightweight alternative for step-level reasoning verification based on data-driven uncertainty scores. We train transformer-based uncertainty quantification heads (UHeads) that use the internal states of a frozen LLM to estimate the uncertainty of its reasoning steps during generation. The approach is fully automatic: target labels are generated either by another larger LLM (e.g., DeepSeek R1) or in a self-supervised manner by the original model itself. UHeads are both effective and lightweight, containing less than 10M parameters. Across multiple domains, including mathematics, planning, and general knowledge question answering, they match or even surpass the performance of PRMs that are up to 810x larger. Our findings suggest that the internal states of LLMs encode their uncertainty and can serve as reliable signals for reasoning verification, offering a promising direction toward scalable and generalizable introspective LLMs.

Method: Spectrum-to-Signal Principle (SSP) with Two-Stage Diversity-Exploring Distillation (SFT) to generate broad solution spectrum, followed by MaxEnt-Guided Policy Optimization (RL) to amplify correct signals.

Result: Outperforms 400x larger DeepSeek R1 on math benchmarks (AIME24: 80.3 vs 79.8, AIME25: 74.4 vs 70.0, HMMT25: 50.4 vs 41.7) and scores 51.1 on LiveCodeBench V6, beating Magistral Medium’s 50.3.

Conclusion: Small models can achieve reasoning capabilities comparable to large models, drastically reducing training and inference costs and democratizing advanced AI research.

Abstract: Challenging the prevailing consensus that small models inherently lack robust reasoning, this report introduces VibeThinker-1.5B, a 1.5B-parameter dense model developed via our Spectrum-to-Signal Principle (SSP). This challenges the prevailing approach of scaling model parameters to enhance capabilities, as seen in models like DeepSeek R1 (671B) and Kimi k2 (>1T). The SSP framework first employs a Two-Stage Diversity-Exploring Distillation (SFT) to generate a broad spectrum of solutions, followed by MaxEnt-Guided Policy Optimization (RL) to amplify the correct signal. With a total training cost of only $7,800, VibeThinker-1.5B demonstrates superior reasoning capabilities compared to closed-source models like Magistral Medium and Claude Opus 4, and performs on par with open-source models like GPT OSS-20B Medium. Remarkably, it surpasses the 400x larger DeepSeek R1 on three math benchmarks: AIME24 (80.3 vs. 79.8), AIME25 (74.4 vs. 70.0), and HMMT25 (50.4 vs. 41.7). This is a substantial improvement over its base model (6.7, 4.3, and 0.6, respectively). On LiveCodeBench V6, it scores 51.1, outperforming Magistral Medium’s 50.3 and its base model’s 0.0. These findings demonstrate that small models can achieve reasoning capabilities comparable to large models, drastically reducing training and inference costs and thereby democratizing advanced AI research.

Method: Combines Discrete Wavelet Transform (DWT) for feature extraction from noisy data, self-adaptive object-aware module for focusing on high-risk vehicles and modeling spatial-temporal relationships, and dynamic focal loss to address class imbalance.

Result: Outperforms existing baselines on three datasets (DAD, CCD, A3D) in key metrics including Average Precision (AP) and mean Time to Accident (mTTA), demonstrating robustness in handling sensor degradation, noise, and imbalanced data.

Conclusion: ROAR provides a reliable and accurate solution for accident anticipation in complex real-world traffic environments, addressing practical challenges that previous methods overlooked.

Abstract: Accurate accident anticipation is essential for enhancing the safety of autonomous vehicles (AVs). However, existing methods often assume ideal conditions, overlooking challenges such as sensor failures, environmental disturbances, and data imperfections, which can significantly degrade prediction accuracy. Additionally, previous models have not adequately addressed the considerable variability in driver behavior and accident rates across different vehicle types. To overcome these limitations, this study introduces ROAR, a novel approach for accident detection and prediction. ROAR combines Discrete Wavelet Transform (DWT), a self adaptive object aware module, and dynamic focal loss to tackle these challenges. The DWT effectively extracts features from noisy and incomplete data, while the object aware module improves accident prediction by focusing on high-risk vehicles and modeling the spatial temporal relationships among traffic agents. Moreover, dynamic focal loss mitigates the impact of class imbalance between positive and negative samples. Evaluated on three widely used datasets, Dashcam Accident Dataset (DAD), Car Crash Dataset (CCD), and AnAn Accident Detection (A3D), our model consistently outperforms existing baselines in key metrics such as Average Precision (AP) and mean Time to Accident (mTTA). These results demonstrate the model’s robustness in real-world conditions, particularly in handling sensor degradation, environmental noise, and imbalanced data distributions. This work offers a promising solution for reliable and accurate accident anticipation in complex traffic environments.

Method: GAIA defines three roles (Principal, Delegate, Counterparty) with optional Critic, using three mechanisms: information-gated progression separating screening from negotiation, dual feedback integration combining AI critique with human corrections, and authorization boundaries with escalation paths.

Result: The framework provides four key contributions: formal governance framework with safety invariants, information-gated progression via task-completeness tracking, dual feedback integration, and hybrid validation combining automated metrics with human judgment.

Conclusion: GAIA bridges theory and practice to offer reproducible specifications for safe, efficient, and accountable AI delegation applicable across procurement, real estate, and staffing workflows.

Abstract: Organizations are increasingly exploring delegation of screening and negotiation tasks to AI systems, yet deployment in high-stakes B2B settings is constrained by governance: preventing unauthorized commitments, ensuring sufficient information before bargaining, and maintaining effective human oversight and auditability. Prior work on large language model negotiation largely emphasizes autonomous bargaining between agents and omits practical needs such as staged information gathering, explicit authorization boundaries, and systematic feedback integration. We propose GAIA, a governance-first framework for LLM-human agency in B2B negotiation and screening. GAIA defines three essential roles - Principal (human), Delegate (LLM agent), and Counterparty - with an optional Critic to enhance performance, and organizes interactions through three mechanisms: information-gated progression that separates screening from negotiation; dual feedback integration that combines AI critique with lightweight human corrections; and authorization boundaries with explicit escalation paths. Our contributions are fourfold: (1) a formal governance framework with three coordinated mechanisms and four safety invariants for delegation with bounded authorization; (2) information-gated progression via task-completeness tracking (TCI) and explicit state transitions that separate screening from commitment; (3) dual feedback integration that blends Critic suggestions with human oversight through parallel learning channels; and (4) a hybrid validation blueprint that combines automated protocol metrics with human judgment of outcomes and safety. By bridging theory and practice, GAIA offers a reproducible specification for safe, efficient, and accountable AI delegation that can be instantiated across procurement, real estate, and staffing workflows.

Method: Closed-loop framework combining synthetic data generator, verifiers, and prompt optimizer. Generates synthetic financial tables and documents, verifies correctness, and incrementally refines prompts based on identified weaknesses.

Result: Achieves higher performance in both accuracy and robustness than standard prompt methods on the DocMath-Eval benchmark, demonstrating effectiveness without needing external labels.

Conclusion: Incorporating synthetic data generation into prompt learning is valuable for financial applications, enabling continuous improvement and adaptation to diverse document structures and question types.

Abstract: Financial documents like earning reports or balance sheets often involve long tables and multi-page reports. Large language models have become a new tool to help numerical reasoning and understanding these documents. However, prompt quality can have a major effect on how well LLMs perform these financial reasoning tasks. Most current methods tune prompts on fixed datasets of financial text or tabular data, which limits their ability to adapt to new question types or document structures, or they involve costly and manually labeled/curated dataset to help build the prompts. We introduce a self-improving prompt framework driven by data-augmented optimization. In this closed-loop process, we generate synthetic financial tables and document excerpts, verify their correctness and robustness, and then update the prompt based on the results. Specifically, our framework combines a synthetic data generator with verifiers and a prompt optimizer, where the generator produces new examples that exposes weaknesses in the current prompt, the verifiers check the validity and robustness of the produced examples, and the optimizer incrementally refines the prompt in response. By iterating these steps in a feedback cycle, our method steadily improves prompt accuracy on financial reasoning tasks without needing external labels. Evaluation on DocMath-Eval benchmark demonstrates that our system achieves higher performance in both accuracy and robustness than standard prompt methods, underscoring the value of incorporating synthetic data generation into prompt learning for financial applications.

Method: Constructed dataset using security data from CVE and CWE sources, annotated with Wikidata and SEPSES CSKG knowledge graphs, and released publicly with all code.

Result: Created comprehensive security domain dataset for STI evaluation, with preliminary baseline evaluation using Falcon3-7b-instruct, Mistral-7B-Instruct, and GPT-4o mini models.

Conclusion: Secu-Table dataset addresses the gap in security domain tabular data availability and supports evaluation of STI systems in the SemTab challenge context.

Abstract: Evaluating semantic tables interpretation (STI) systems, (particularly, those based on Large Language Models- LLMs) especially in domain-specific contexts such as the security domain, depends heavily on the dataset. However, in the security domain, tabular datasets for state-of-the-art are not publicly available. In this paper, we introduce Secu-Table dataset, composed of more than 1500 tables with more than 15k entities constructed using security data extracted from Common Vulnerabilities and Exposures (CVE) and Common Weakness Enumeration (CWE) data sources and annotated using Wikidata and the SEmantic Processing of Security Event Streams CyberSecurity Knowledge Graph (SEPSES CSKG). Along with the dataset, all the code is publicly released. This dataset is made available to the research community in the context of the SemTab challenge on Tabular to Knowledge Graph Matching. This challenge aims to evaluate the performance of several STI based on open source LLMs. Preliminary evaluation, serving as baseline, was conducted using Falcon3-7b-instruct and Mistral-7B-Instruct, two open source LLMs and GPT-4o mini one closed source LLM.

Method: Multi-stage pipeline integrating large language and vision-language models that performs OCR, linguistic reasoning, and map-level verification through grid-based spatial scanning, evaluating predicted locations against contextual and visual evidence.

Result: ALIGN demonstrates consistent improvements over traditional geoparsing methods, accurately identifying district and sub-district-level crash sites in Bangla-language news data.

Conclusion: The framework establishes a high-accuracy foundation for automated crash mapping in data-scarce regions, supporting evidence-driven road safety policymaking and broader integration of multimodal AI in transportation analytics.

Abstract: Reliable geospatial information on road accidents is vital for safety analysis and infrastructure planning, yet most low- and middle-income countries continue to face a critical shortage of accurate, location-specific crash data. Existing text-based geocoding tools perform poorly in multilingual and unstructured news environments, where incomplete place descriptions and mixed Bangla-English scripts obscure spatial context. To address these limitations, this study introduces ALIGN (Accident Location Inference through Geo-Spatial Neural Reasoning)- a vision-language framework that emulates human spatial reasoning to infer accident coordinates directly from textual and map-based cues. ALIGN integrates large language and vision-language models within a multi-stage pipeline that performs optical character recognition, linguistic reasoning, and map-level verification through grid-based spatial scanning. The framework systematically evaluates each predicted location against contextual and visual evidence, ensuring interpretable, fine-grained geolocation outcomes without requiring model retraining. Applied to Bangla-language news data, ALIGN demonstrates consistent improvements over traditional geoparsing methods, accurately identifying district and sub-district-level crash sites. Beyond its technical contribution, the framework establishes a high accuracy foundation for automated crash mapping in data-scarce regions, supporting evidence-driven road-safety policymaking and the broader integration of multimodal artificial intelligence in transportation analytics. The code for this paper is open-source and available at: https://github.com/Thamed-Chowdhury/ALIGN

Method: Created LPFQA benchmark using authentic professional forums across 20 academic/industrial fields (502 tasks), with four innovations: fine-grained evaluation dimensions, hierarchical difficulty structure, realistic scenario modeling, and interdisciplinary knowledge integration.

Result: Evaluation of 12 mainstream LLMs revealed significant performance disparities, particularly in specialized reasoning tasks, demonstrating LPFQA’s discriminative power.

Conclusion: LPFQA provides a robust, authentic, and discriminative benchmark that advances LLM evaluation and guides future model development by better capturing real-world professional expertise.

Abstract: Large Language Models (LLMs) have made rapid progress in reasoning, question answering, and professional applications; however, their true capabilities remain difficult to evaluate using existing benchmarks. Current datasets often focus on simplified tasks or artificial scenarios, overlooking long-tail knowledge and the complexities of real-world applications. To bridge this gap, we propose LPFQA, a long-tail knowledge-based benchmark derived from authentic professional forums across 20 academic and industrial fields, covering 502 tasks grounded in practical expertise. LPFQA introduces four key innovations: fine-grained evaluation dimensions that target knowledge depth, reasoning, terminology comprehension, and contextual analysis; a hierarchical difficulty structure that ensures semantic clarity and unique answers; authentic professional scenario modeling with realistic user personas; and interdisciplinary knowledge integration across diverse domains. We evaluated 12 mainstream LLMs on LPFQA and observed significant performance disparities, especially in specialized reasoning tasks. LPFQA provides a robust, authentic, and discriminative benchmark for advancing LLM evaluation and guiding future model development.

Method: Proposed Adaptive Entropy Policy Optimization (AEPO) with two components: Reflection-aware Information Filtration to control cognitive information flow, and Adaptive-Entropy Optimization to balance exploration and exploitation across reasoning stages.

Result: AEPO consistently achieves state-of-the-art performance over mainstream reinforcement learning baselines across diverse benchmarks.

Conclusion: The AEPO framework effectively addresses the ‘Echo Reflection’ problem in LLMs by controlling information flow and promoting reflective diversity, leading to improved reasoning performance in complex domain-specific tasks.

Abstract: Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of reasoning tasks. Recent methods have further improved LLM performance in complex mathematical reasoning. However, when extending these methods beyond the domain of mathematical reasoning to tasks involving complex domain-specific knowledge, we observe a consistent failure of LLMs to generate novel insights during the reflection stage. Instead of conducting genuine cognitive refinement, the model tends to mechanically reiterate earlier reasoning steps without introducing new information or perspectives, a phenomenon referred to as “Echo Reflection”. We attribute this behavior to two key defects: (1) Uncontrollable information flow during response generation, which allows premature intermediate thoughts to propagate unchecked and distort final decisions; (2) Insufficient exploration of internal knowledge during reflection, leading to repeating earlier findings rather than generating new cognitive insights. Building on these findings, we proposed a novel reinforcement learning method termed Adaptive Entropy Policy Optimization (AEPO). Specifically, the AEPO framework consists of two major components: (1) Reflection-aware Information Filtration, which quantifies the cognitive information flow and prevents the final answer from being affected by earlier bad cognitive information; (2) Adaptive-Entropy Optimization, which dynamically balances exploration and exploitation across different reasoning stages, promoting both reflective diversity and answer correctness. Extensive experiments demonstrate that AEPO consistently achieves state-of-the-art performance over mainstream reinforcement learning baselines across diverse benchmarks.

Method: Multi-agent framework with SLMs conducting structured debates among critic, defender, and judge agents; uses HAJailBench with 12,000 human-annotated jailbreak interactions across diverse attacks.

Result: SLM-based framework achieves comparable agreement to GPT-4o judges on HAJailBench while substantially reducing inference cost; three debate rounds provide optimal accuracy-efficiency balance.

Conclusion: Structured, value-aligned debate enables SLMs to capture semantic nuances of jailbreak attacks, and HAJailBench provides reliable foundation for scalable LLM safety evaluation.

Abstract: Safety evaluation of large language models (LLMs) increasingly relies on LLM-as-a-Judge frameworks, but the high cost of frontier models limits scalability. We propose a cost-efficient multi-agent judging framework that employs Small Language Models (SLMs) through structured debates among critic, defender, and judge agents. To rigorously assess safety judgments, we construct HAJailBench, a large-scale human-annotated jailbreak benchmark comprising 12,000 adversarial interactions across diverse attack methods and target models. The dataset provides fine-grained, expert-labeled ground truth for evaluating both safety robustness and judge reliability. Our SLM-based framework achieves agreement comparable to GPT-4o judges on HAJailBench while substantially reducing inference cost. Ablation results show that three rounds of debate yield the optimal balance between accuracy and efficiency. These findings demonstrate that structured, value-aligned debate enables SLMs to capture semantic nuances of jailbreak attacks and that HAJailBench offers a reliable foundation for scalable LLM safety evaluation.

Method: SofT-GRPO injects Gumbel noise into logits, uses Gumbel-Softmax to keep tokens within the pre-trained embedding space, and leverages the reparameterization trick in policy gradient.

Result: Experiments on LLMs from 1.5B to 7B parameters show SofT-GRPO enables soft-thinking LLMs to slightly outperform discrete-token GRPO on Pass@1 (+0.13% average accuracy) and substantially improve on Pass@32 (+2.19% average accuracy).

Conclusion: SofT-GRPO successfully unlocks the potential of soft-thinking reasoning by enabling effective RL-based policy optimization, demonstrating its research and application value.

Abstract: The soft-thinking paradigm for Large Language Model (LLM) reasoning can outperform the conventional discrete-token Chain-of-Thought (CoT) reasoning in some scenarios, underscoring its research and application value. However, while the discrete-token CoT reasoning pattern can be reinforced through policy optimization algorithms such as group relative policy optimization (GRPO), extending the soft-thinking pattern with Reinforcement Learning (RL) remains challenging. This difficulty stems from the complexities of injecting stochasticity into soft-thinking tokens and updating soft-thinking policies accordingly. As a result, previous attempts to combine soft-thinking with GRPO typically underperform their discrete-token GRPO counterparts. To fully unlock the potential of soft-thinking, this paper presents a novel policy optimization algorithm, SofT-GRPO, to reinforce LLMs under the soft-thinking reasoning pattern. SofT-GRPO injects the Gumbel noise into logits, employs the Gumbel-Softmax technique to avoid soft-thinking tokens outside the pre-trained embedding space, and leverages the reparameterization trick in policy gradient. We conduct experiments across base LLMs ranging from 1.5B to 7B parameters, and results demonstrate that SofT-GRPO enables soft-thinking LLMs to slightly outperform discrete-token GRPO on Pass@1 (+0.13% on average accuracy), while exhibiting a substantial uplift on Pass@32 (+2.19% on average accuracy). Codes and weights are available on https://github.com/zz1358m/SofT-GRPO-master

Method: Proposed Auto-Explorer with exploration mechanisms that autonomously parse and explore GUI environments, plus UIXplore benchmark to assess exploration quality and fine-tune MLLMs.

Result: Experiments show superior performance of Auto-Explorer, demonstrating it can quickly enhance MLLM capabilities in explored software.

Conclusion: Auto-Explorer provides an effective automated solution for GUI data collection that overcomes limitations of existing methods and enables rapid adaptation to new software environments.

Abstract: Recent advancements in GUI agents have significantly expanded their ability to interpret natural language commands to manage software interfaces. However, acquiring GUI data remains a significant challenge. Existing methods often involve designing automated agents that browse URLs from the Common Crawl, using webpage HTML to collect screenshots and corresponding annotations, including the names and bounding boxes of UI elements. However, this method is difficult to apply to desktop software or some newly launched websites not included in the Common Crawl. While we expect the model to possess strong generalization capabilities to handle this, it is still crucial for personalized scenarios that require rapid and perfect adaptation to new software or websites. To address this, we propose an automated data collection method with minimal annotation costs, named Auto-Explorer. It incorporates a simple yet effective exploration mechanism that autonomously parses and explores GUI environments, gathering data efficiently. Additionally, to assess the quality of exploration, we have developed the UIXplore benchmark. This benchmark creates environments for explorer agents to discover and save software states. Using the data gathered, we fine-tune a multimodal large language model (MLLM) and establish a GUI element grounding testing set to evaluate the effectiveness of the exploration strategies. Our experiments demonstrate the superior performance of Auto-Explorer, showing that our method can quickly enhance the capabilities of an MLLM in explored software.

Method: Propose MONICA, a Monitor-guided Calibration framework with a sycophantic monitor that provides real-time monitoring of sycophantic drift scores during response generation, and a calibrator that dynamically suppresses sycophantic behavior when scores exceed thresholds.

Result: Extensive experiments across 12 datasets and 3 LRMs show that MONICA effectively reduces sycophantic behavior in both intermediate reasoning steps and final answers, yielding robust performance improvements.

Conclusion: MONICA successfully addresses the limitation of current methods by monitoring and mitigating sycophancy during model inference at the reasoning step level, without requiring complete answer generation.

Abstract: Large Reasoning Models (LRMs) suffer from sycophantic behavior, where models tend to agree with users’ incorrect beliefs and follow misinformation rather than maintain independent reasoning. This behavior undermines model reliability and poses societal risks. Mitigating LRM sycophancy requires monitoring how this sycophancy emerges during the reasoning trajectory; however, current methods mainly focus on judging based on final answers and correcting them, without understanding how sycophancy develops during reasoning processes. To address this limitation, we propose MONICA, a novel Monitor-guided Calibration framework that monitors and mitigates sycophancy during model inference at the level of reasoning steps, without requiring the model to finish generating its complete answer. MONICA integrates a sycophantic monitor that provides real-time monitoring of sycophantic drift scores during response generation with a calibrator that dynamically suppresses sycophantic behavior when scores exceed predefined thresholds. Extensive experiments across 12 datasets and 3 LRMs demonstrate that our method effectively reduces sycophantic behavior in both intermediate reasoning steps and final answers, yielding robust performance improvements.

Method: EDTR treats each Chain-of-Thought as a vector in high-dimensional space and extracts eight topological risk features capturing geometric structure of reasoning distributions, using tighter clusters for higher confidence and dispersed paths for uncertainty.

Result: EDTR achieves 41% better calibration than competing methods with average ECE of 0.287 and best composite score of 0.672, with perfect accuracy on AIME and exceptional calibration on GSM8K (ECE 0.107).

Conclusion: The work provides a geometric framework for understanding and quantifying uncertainty in multi-step LLM reasoning, enabling more reliable deployment where calibrated confidence estimates are essential.

Abstract: Chain-of-thought (CoT) prompting enables Large Language Models to solve complex problems, but deploying these models safely requires reliable confidence estimates, a capability where existing methods suffer from poor calibration and severe overconfidence on incorrect predictions. We propose Enhanced Dirichlet and Topology Risk (EDTR), a novel decoding strategy that combines topological analysis with Dirichlet-based uncertainty quantification to measure LLM confidence across multiple reasoning paths. EDTR treats each CoT as a vector in high-dimensional space and extracts eight topological risk features capturing the geometric structure of reasoning distributions: tighter, more coherent clusters indicate higher confidence while dispersed, inconsistent paths signal uncertainty. We evaluate EDTR against three state-of-the-art calibration methods across four diverse reasoning benchmarks spanning olympiad-level mathematics (AIME), grade school math (GSM8K), commonsense reasoning, and stock price prediction \cite{zhang2025aime, cobbe2021training, talmor-etal-2019-commonsenseqa, yahoo_finance}. EDTR achieves 41% better calibration than competing methods with an average ECE of 0.287 and the best overall composite score of 0.672, while notably achieving perfect accuracy on AIME and exceptional calibration on GSM8K with an ECE of 0.107, domains where baselines exhibit severe overconfidence. Our work provides a geometric framework for understanding and quantifying uncertainty in multi-step LLM reasoning, enabling more reliable deployment where calibrated confidence estimates are essential.

Method: 1) Top-down attention for spatial abstraction; 2) Skipper framework for automatic task decomposition; 3) Feasibility evaluator to reject hallucinated infeasible targets.

Result: Significant improvements in systematic generalization outside training tasks, robustness against distributional shifts, and better performance in long-term compositional planning.

Conclusion: Human-inspired reasoning mechanisms (spatial abstraction, task decomposition, feasibility evaluation) effectively enhance RL agents’ zero-shot generalization, with future work needed for general task abstraction and abstract planning.

Abstract: Existing Reinforcement Learning (RL) systems encounter significant challenges when applied to real-world scenarios, primarily due to poor generalization across environments that differ from their training conditions. This thesis explores the direction of enhancing agents’ zero-shot systematic generalization abilities by granting RL agents reasoning behaviors that are found to help systematic generalization in the human brain. Inspired by human conscious planning behaviors, we first introduced a top-down attention mechanism, which allows a decision-time planning agent to dynamically focus its reasoning on the most relevant aspects of the environmental state given its instantaneous intentions, a process we call “spatial abstraction”. This approach significantly improves systematic generalization outside the training tasks. Subsequently, building on spatial abstraction, we developed the Skipper framework to automatically decompose complex tasks into simpler, more manageable sub-tasks. Skipper provides robustness against distributional shifts and efficacy in long-term, compositional planning by focusing on pertinent spatial and temporal elements of the environment. Finally, we identified a common failure mode and safety risk in planning agents that rely on generative models to generate state targets during planning. It is revealed that most agents blindly trust the targets they hallucinate, resulting in delusional planning behaviors. Inspired by how the human brain rejects delusional intentions, we propose learning a feasibility evaluator to enable rejecting hallucinated infeasible targets, which led to significant performance improvements in various kinds of planning agents. Finally, we suggest directions for future research, aimed at achieving general task abstraction and fully enabling abstract planning.

Method: GHOST uses hierarchical best-first search with abstract-path-unfolding algorithm to compute admissible lower bounds, combining tour search and trajectory optimization while minimizing convex optimization calls.

Result: GHOST is orders-of-magnitude faster than mixed-integer convex programming baselines and handles complex trajectory planning with high-order continuity constraints and incomplete GCS.

Conclusion: GHOST provides an efficient optimal solution for GCS-TSP with strong pruning power and bounded-suboptimal variants for time-critical applications.

Abstract: We study GCS-TSP, a new variant of the Traveling Salesman Problem (TSP) defined over a Graph of Convex Sets (GCS) – a powerful representation for trajectory planning that decomposes the configuration space into convex regions connected by a sparse graph. In this setting, edge costs are not fixed but depend on the specific trajectory selected through each convex region, making classical TSP methods inapplicable. We introduce GHOST, a hierarchical framework that optimally solves the GCS-TSP by combining combinatorial tour search with convex trajectory optimization. GHOST systematically explores tours on a complete graph induced by the GCS, using a novel abstract-path-unfolding algorithm to compute admissible lower bounds that guide best-first search at both the high level (over tours) and the low level (over feasible GCS paths realizing the tour). These bounds provide strong pruning power, enabling efficient search while avoiding unnecessary convex optimization calls. We prove that GHOST guarantees optimality and present a bounded-suboptimal variant for time-critical scenarios. Experiments show that GHOST is orders-of-magnitude faster than unified mixed-integer convex programming baselines for simple cases and uniquely handles complex trajectory planning problems involving high-order continuity constraints and an incomplete GCS.

Method: Created FractalBench with 12 canonical fractals, evaluated four leading MLLMs (GPT-4o, Claude 3.7 Sonnet, Gemini 2.5 Flash, Qwen 2.5-VL) by requiring them to generate executable Python code that reproduces fractals from images.

Result: 76% of generated code was syntactically valid but only 4% captured correct mathematical structure. Models performed better on geometric transformations (Koch curves: 17-21%) but failed at branching recursion (trees: <2%).

Conclusion: There is a fundamental gap in multimodal AI systems’ mathematical abstraction capabilities, particularly in bridging visual perception with mathematical reasoning. FractalBench serves as a contamination-resistant diagnostic tool for visual-mathematical reasoning.

Abstract: Mathematical reasoning requires abstracting symbolic rules from visual patterns – inferring the infinite from the finite. We investigate whether multimodal AI systems possess this capability through FractalBench, a benchmark evaluating fractal program synthesis from images. Fractals provide ideal test cases: Iterated Function Systems with only a few contraction maps generate complex self-similar patterns through simple recursive rules, requiring models to bridge visual perception with mathematical abstraction. We evaluate four leading MLLMs – GPT-4o, Claude 3.7 Sonnet, Gemini 2.5 Flash, and Qwen 2.5-VL – on 12 canonical fractals. Models must generate executable Python code reproducing the fractal, enabling objective evaluation. Results reveal a striking disconnect: 76% generate syntactically valid code but only 4% capture mathematical structure. Success varies systematically – models handle geometric transformations (Koch curves: 17-21%) but fail at branching recursion (trees: <2%), revealing fundamental gaps in mathematical abstraction. FractalBench provides a contamination-resistant diagnostic for visual-mathematical reasoning and is available at https://github.com/NaiveNeuron/FractalBench

Method: The method involves: 1) Creating an ontology mapping legal contract elements to graph nodes/edges, 2) Using reinforcement learning with LLMs (GRAPH-GRPO-LEX) for segmentation and entity/relationship extraction, 3) Applying group relative policy optimization with graph metrics as reward functions.

Result: The framework successfully identifies direct relationships between clauses and uncovers hidden dependencies. The gated GRPO approach provides strong learning signals and enables transformation of contract analysis from manual reading to visualized graphs.

Conclusion: This approach enables dynamic contract analysis and lays the groundwork for contract linting similar to software engineering practices, moving contract review from linear manual processes to automated, visualized graph-based analysis.

Abstract: Contracts are complex documents featuring detailed formal structures, explicit and implicit dependencies and rich semantic content. Given these document properties, contract drafting and manual examination of contracts have proven to be both arduous and susceptible to errors. This work aims to simplify and automate the task of contract review and analysis using a novel framework for transforming legal contracts into structured semantic graphs, enabling computational analysis and data-driven insights. We introduce a detailed ontology mapping core legal contract elements to their graph-theoretic equivalents of nodes and edges. We then present a reinforcement learning based Large Language Model (LLM) framework for segmentation and extraction of entities and relationships from contracts. Our method, GRAPH-GRPO-LEX, incorporates both LLMs and reinforcement learning with group relative policy optimization (GRPO). By applying a carefully drafted reward function of graph metrics, we demonstrate the ability to automatically identify direct relationships between clauses, and even uncover hidden dependencies. Our introduction of the gated GRPO approach shows a strong learning signal and can move contract analysis from a linear, manual reading process to an easily visualized graph. This allows for a more dynamic analysis, including building the groundwork for contract linting similar to what is now practiced in software engineering.

Method: Proposed self-report fine-tuning (SRFT), a supervised fine-tuning technique that trains models to admit their factual mistakes when asked. This approach generalizes from factual error admission to admitting hidden misaligned objectives.

Result: SRFT models achieved near-ceiling performance in detecting hidden objectives (F1 score = 0.98 vs 0 for baseline), recovered 28-100% of hidden objective details (vs 0% for baseline), and confessed even under strong pressure not to disclose.

Conclusion: SRFT provides a promising technique for promoting honesty propensity and incriminating misaligned AI systems, offering effective detection of deceptive behavior in adversarial agentic settings.

Abstract: As AI systems become more capable of complex agentic tasks, they also become more capable of pursuing undesirable objectives and causing harm. Previous work has attempted to catch these unsafe instances by interrogating models directly about their objectives and behaviors. However, the main weakness of trusting interrogations is that models can lie. We propose self-report fine-tuning (SRFT), a simple supervised fine-tuning technique that trains models to admit their factual mistakes when asked. We show that the admission of factual errors in simple question-answering settings generalizes out-of-distribution (OOD) to the admission of hidden misaligned objectives in adversarial agentic settings. We evaluate SRFT in OOD stealth tasks, where models are instructed to complete a hidden misaligned objective alongside a user-specified objective without being caught by monitoring. After SRFT, models are more likely to confess the details of their hidden objectives when interrogated, even under strong pressure not to disclose them. Interrogation on SRFT models can detect hidden objectives with near-ceiling performance (F1 score = 0.98), while the baseline model lies when interrogated under the same conditions (F1 score = 0). Interrogation on SRFT models can further elicit the content of the hidden objective, recovering 28-100% details, compared to 0% details recovered in the baseline model and by prefilled assistant turn attacks. This provides a promising technique for promoting honesty propensity and incriminating misaligned AI systems.

Method: Introduces Spatiotemporal Relational Neural Network (SRNN) with unified neural representations for object attributes, relations, and timeline, using Hebbian “Fire Together, Wire Together” mechanism across What and How pathways. Adopts “predefine-then-finetune” approach instead of “pretrain-then-finetune”.

Result: Achieves competitive performance on CLEVRER benchmark, identifies benchmark bias, provides path for more holistic evaluation, and demonstrates white-box utility for precise error diagnosis.

Conclusion: Confirms viability of translating biological intelligence into engineered systems for intuitive physics understanding, bridging perception and language within shared neural substrate.

Abstract: Human prowess in intuitive physics remains unmatched by machines. To bridge this gap, we argue for a fundamental shift towards brain-inspired computational principles. This paper introduces the Spatiotemporal Relational Neural Network (SRNN), a model that establishes a unified neural representation for object attributes, relations, and timeline, with computations governed by a Hebbian Fire Together, Wire Together'' mechanism across dedicated \textit{What} and \textit{How} pathways. This unified representation is directly used to generate structured linguistic descriptions of the visual scene, bridging perception and language within a shared neural substrate. Moreover, unlike the prevalent pretrain-then-finetune’’ paradigm, SRNN adopts a ``predefine-then-finetune’’ approach. On the CLEVRER benchmark, SRNN achieves competitive performance. Our analysis further reveals a benchmark bias, outlines a path for a more holistic evaluation, and demonstrates SRNN’s white-box utility for precise error diagnosis. Our work confirms the viability of translating biological intelligence into engineered systems for intuitive physics understanding.

Method: Proposes MathSE framework with iterative cycles of inference, reflection, and reward-based feedback using an Outcome Reward Model (ORM) to refine reasoning paths.

Result: Significant performance gains over backbone models on challenging benchmarks, with MathVL-test results surpassing leading open-source multimodal mathematical reasoning model QVQ.

Conclusion: The MathSE framework effectively addresses limitations of traditional fine-tuning by enabling iterative self-evolution, demonstrating superior mathematical reasoning capabilities in MLLMs.

Abstract: Multimodal large language models (MLLMs) have demonstrated remarkable capabilities in vision-language answering tasks. Despite their strengths, these models often encounter challenges in achieving complex reasoning tasks such as mathematical problem-solving. Previous works have focused on fine-tuning on specialized mathematical datasets. However, these datasets are typically distilled directly from teacher models, which capture only static reasoning patterns and leaving substantial gaps compared to student models. This reliance on fixed teacher-derived datasets not only restricts the model’s ability to adapt to novel or more intricate questions that extend beyond the confines of the training data, but also lacks the iterative depth needed for robust generalization. To overcome these limitations, we propose \textbf{\method}, a \textbf{Math}ematical \textbf{S}elf-\textbf{E}volving framework for MLLMs. In contrast to traditional one-shot fine-tuning paradigms, \method iteratively refines the model through cycles of inference, reflection, and reward-based feedback. Specifically, we leverage iterative fine-tuning by incorporating correct reasoning paths derived from previous-stage inference and integrating reflections from a specialized Outcome Reward Model (ORM). To verify the effectiveness of \method, we evaluate it on a suite of challenging benchmarks, demonstrating significant performance gains over backbone models. Notably, our experimental results on MathVL-test surpass the leading open-source multimodal mathematical reasoning model QVQ. Our code and models are available at \texttt{https://zheny2751\allowbreak-dotcom.github.io/\allowbreak MathSE.github.io/}.

Method: Competition organization and evaluation of constraint solvers following XCSP3 standards

Result: Documentation of competition proceedings and solver performance results

Conclusion: Successful completion and reporting of the 2025 XCSP3 Competition at the CP'25 conference

Abstract: This document represents the proceedings of the 2025 XCSP3 Competition. The results of this competition of constraint solvers were presented at CP'25 (31st International Conference on Principles and Practice of Constraint Programming).

Method: Uses emotion-sensitive prompt templates, constructs first dedicated demonstration retrieval repository for ERC, and implements curriculum learning with weighted emotional shifts in LoRA fine-tuning to organize training from easy to hard samples.

Result: Achieves new state-of-the-art performance on IEMOCAP and MELD benchmark datasets, demonstrating effectiveness and generalizability in improving LLM-based emotional understanding.

Conclusion: The PRC-Emo framework successfully enhances LLMs’ ability to perceive emotions in conversational contexts through integrated prompt engineering, retrieval, and curriculum learning strategies.

Abstract: Emotion Recognition in Conversation (ERC) is a crucial task for understanding human emotions and enabling natural human-computer interaction. Although Large Language Models (LLMs) have recently shown great potential in this field, their ability to capture the intrinsic connections between explicit and implicit emotions remains limited. We propose a novel ERC training framework, PRC-Emo, which integrates Prompt engineering, demonstration Retrieval, and Curriculum learning, with the goal of exploring whether LLMs can effectively perceive emotions in conversational contexts. Specifically, we design emotion-sensitive prompt templates based on both explicit and implicit emotional cues to better guide the model in understanding the speaker’s psychological states. We construct the first dedicated demonstration retrieval repository for ERC, which includes training samples from widely used datasets, as well as high-quality dialogue examples generated by LLMs and manually verified. Moreover, we introduce a curriculum learning strategy into the LoRA fine-tuning process, incorporating weighted emotional shifts between same-speaker and different-speaker utterances to assign difficulty levels to dialogue samples, which are then organized in an easy-to-hard training sequence. Experimental results on two benchmark datasets– IEMOCAP and MELD –show that our method achieves new state-of-the-art (SOTA) performance, demonstrating the effectiveness and generalizability of our approach in improving LLM-based emotional understanding.

Method: Proposes UrbanLN with: 1) Information-preserved stretching interpolation for long caption alignment, 2) Dual-level optimization: multi-model collaboration for diverse captions and momentum-based self-distillation for noise suppression.

Result: Extensive experiments across four real-world cities and various downstream tasks demonstrate superior performance compared to existing methods.

Conclusion: UrbanLN effectively addresses key challenges in urban region representation learning by improving long-text awareness and noise suppression, leading to better performance on urban computing tasks.

Abstract: Region representation learning plays a pivotal role in urban computing by extracting meaningful features from unlabeled urban data. Analogous to how perceived facial age reflects an individual’s health, the visual appearance of a city serves as its ``portrait", encapsulating latent socio-economic and environmental characteristics. Recent studies have explored leveraging Large Language Models (LLMs) to incorporate textual knowledge into imagery-based urban region representation learning. However, two major challenges remain: i)~difficulty in aligning fine-grained visual features with long captions, and ii) suboptimal knowledge incorporation due to noise in LLM-generated captions. To address these issues, we propose a novel pre-training framework called UrbanLN that improves Urban region representation learning through Long-text awareness and Noise suppression. Specifically, we introduce an information-preserved stretching interpolation strategy that aligns long captions with fine-grained visual semantics in complex urban scenes. To effectively mine knowledge from LLM-generated captions and filter out noise, we propose a dual-level optimization strategy. At the data level, a multi-model collaboration pipeline automatically generates diverse and reliable captions without human intervention. At the model level, we employ a momentum-based self-distillation mechanism to generate stable pseudo-targets, facilitating robust cross-modal learning under noisy conditions. Extensive experiments across four real-world cities and various downstream tasks demonstrate the superior performance of our UrbanLN.

Method: Three-stage progressive RL-prioritized post-training: (1) Exploratory Learning on curated SNS corpora for initial alignment and weakness identification, (2) Targeted Fine-Tuning applying SFT to diagnosed gaps with general data to prevent forgetting, (3) Refinement Learning using RL with SNS-centric signals to consolidate improvements.

Result: The 4B scale model achieves average improvements of 2.41 over 7B baseline and 8.74 performance lift from base model with less than half the data required by SFT-centric methods, demonstrating superior data efficiency and stability at compact scales.

Conclusion: RedOne 2.0 establishes a competitive, cost-effective baseline for domain-specific LLMs in SNS scenarios, advancing capability without sacrificing robustness through its progressive training approach.

Abstract: As a key medium for human interaction and information exchange, social networking services (SNS) pose unique challenges for large language models (LLMs): heterogeneous workloads, fast-shifting norms and slang, and multilingual, culturally diverse corpora that induce sharp distribution shift. Supervised fine-tuning (SFT) can specialize models but often triggers a ``seesaw’’ between in-distribution gains and out-of-distribution robustness, especially for smaller models. To address these challenges, we introduce RedOne 2.0, an SNS-oriented LLM trained with a progressive, RL-prioritized post-training paradigm designed for rapid and stable adaptation. The pipeline consist in three stages: (1) Exploratory Learning on curated SNS corpora to establish initial alignment and identify systematic weaknesses; (2) Targeted Fine-Tuning that selectively applies SFT to the diagnosed gaps while mixing a small fraction of general data to mitigate forgetting; and (3) Refinement Learning that re-applies RL with SNS-centric signals to consolidate improvements and harmonize trade-offs across tasks. Across various tasks spanning three categories, our 4B scale model delivers an average improvements about 2.41 over the 7B sub-optimal baseline. Additionally, RedOne 2.0 achieves average performance lift about 8.74 from the base model with less than half the data required by SFT-centric method RedOne, evidencing superior data efficiency and stability at compact scales. Overall, RedOne 2.0 establishes a competitive, cost-effective baseline for domain-specific LLMs in SNS scenario, advancing capability without sacrificing robustness.

Method: Integrates LLMs into defined decision models using a layered architecture that separates LLM reasoning space from explainable process space, situating LLM reasoning within formal analytical structures.

Result: Empirical evaluations show the system can reproduce human-level decision logic in decentralized governance, systems analysis, and strategic reasoning contexts.

Conclusion: LLM-driven standard processes provide a foundation for reliable, interpretable, and verifiable AI-supported decision making.

Abstract: This paper introduces an approach to increasing the explainability of artificial intelligence (AI) systems by embedding Large Language Models (LLMs) within standardized analytical processes. While traditional explainable AI (XAI) methods focus on feature attribution or post-hoc interpretation, the proposed framework integrates LLMs into defined decision models such as Question-Option-Criteria (QOC), Sensitivity Analysis, Game Theory, and Risk Management. By situating LLM reasoning within these formal structures, the approach transforms opaque inference into transparent and auditable decision traces. A layered architecture is presented that separates the reasoning space of the LLM from the explainable process space above it. Empirical evaluations show that the system can reproduce human-level decision logic in decentralized governance, systems analysis, and strategic reasoning contexts. The results suggest that LLM-driven standard processes provide a foundation for reliable, interpretable, and verifiable AI-supported decision making.

Method: Modular pipeline with swappable modules using LLM components (default GPT-5) paired with deterministic analyzers for equilibria and matrix-based role classification, implementing three frameworks: Vester’s Sensitivity Model, normal-form games, and sequential games.

Result: In logistics case study (100 runs): 55.5% mean factor alignment with human baseline over 26 factors, 62.9% on transport-core subset; 57% role agreement; LLM judge scored runs on par with reconstructed human baseline using eight-criterion rubric.

Conclusion: Configurable LLM pipelines can effectively mimic expert workflows while maintaining transparency and inspectability of reasoning steps, providing auditable decision support artifacts.

Abstract: We present a modular, explainable LLM-agent pipeline for decision support that externalizes reasoning into auditable artifacts. The system instantiates three frameworks: Vester’s Sensitivity Model (factor set, signed impact matrix, systemic roles, feedback loops); normal-form games (strategies, payoff matrix, equilibria); and sequential games (role-conditioned agents, tree construction, backward induction), with swappable modules at every step. LLM components (default: GPT-5) are paired with deterministic analyzers for equilibria and matrix-based role classification, yielding traceable intermediates rather than opaque outputs. In a real-world logistics case (100 runs), mean factor alignment with a human baseline was 55.5% over 26 factors and 62.9% on the transport-core subset; role agreement over matches was 57%. An LLM judge using an eight-criterion rubric (max 100) scored runs on par with a reconstructed human baseline. Configurable LLM pipelines can thus mimic expert workflows with transparent, inspectable steps.

Method: Develops a five-phase lifecycle mapped to LCA stages, specifies governance via PDCA cycles, systematizes hardware/system strategies across edge-cloud continuum, and defines a calibrated measurement framework combining estimator models with direct metering.

Result: Provides actionable guidance for reducing AI’s environmental burdens through lifecycle management, hardware optimization, and reproducible measurement across facility, system, device, and workload levels.

Conclusion: The framework offers evidence-based guidance for researchers, practitioners, and policymakers to achieve Green AI through unified definition, lifecycle processes, hardware strategies, and calibrated measurement.

Abstract: Across the Artificial Intelligence (AI) lifecycle - from hardware to development, deployment, and reuse - burdens span energy, carbon, water, and embodied impacts. Cloud provider tools improve transparency but remain heterogeneous and often omit water and value chain effects, limiting comparability and reproducibility. Addressing these multi dimensional burdens requires a lifecycle approach linking phase explicit mapping with system levers (hardware, placement, energy mix, cooling, scheduling) and calibrated measurement across facility, system, device, and workload levels. This article (i) establishes a unified, operational definition of Green AI distinct from Sustainable AI; (ii) formalizes a five phase lifecycle mapped to Life Cycle Assessment (LCA) stages, making energy, carbon, water, and embodied impacts first class; (iii) specifies governance via Plan Do Check Act (PDCA) cycles with decision gateways; (iv) systematizes hardware and system level strategies across the edge cloud continuum to reduce embodied burdens; and (v) defines a calibrated measurement framework combining estimator models with direct metering to enable reproducible, provider agnostic comparisons. Combining definition, lifecycle processes, hardware strategies, and calibrated measurement, this article offers actionable, evidence based guidance for researchers, practitioners, and policymakers.

Method: The authors assign costs to interpretations based on weights of violated axioms and assertions, then determine query answers by considering interpretations with optimal or bounded cost. They analyze data complexity by sharpening lower bounds and establishing precise complexity results.

Result: The paper provides sharpened lower bounds and precise complexity results for optimal-cost certain answer semantics. Most notably, for DL-Lite^H_bool ontologies with fixed cost bounds, certain answers for instance queries and possible answers for conjunctive queries can be computed using first-order rewriting, achieving the lowest possible data complexity (TC0).

Conclusion: Cost-based semantics for querying inconsistent weighted DL knowledge bases can achieve tractable data complexity (TC0) for DL-Lite^H_bool ontologies with fixed cost bounds, which is surprising given previous intractability results and represents the lowest possible data complexity.

Abstract: In this paper, we study the data complexity of querying inconsistent weighted description logic (DL) knowledge bases under recently-introduced cost-based semantics. In a nutshell, the idea is to assign each interpretation a cost based upon the weights of the violated axioms and assertions, and certain and possible query answers are determined by considering all (resp. some) interpretations having optimal or bounded cost. Whereas the initial study of cost-based semantics focused on DLs between $\mathcal{EL}\bot$ and $\mathcal{ALCO}$, we consider DLs that may contain inverse roles and role inclusions, thus covering prominent DL-Lite dialects. Our data complexity analysis goes significantly beyond existing results by sharpening several lower bounds and pinpointing the precise complexity of optimal-cost certain answer semantics (no non-trivial upper bound was known). Moreover, while all existing results show the intractability of cost-based semantics, our most challenging and surprising result establishes that if we consider $\text{DL-Lite}^\mathcal{H}\mathsf{bool}$ ontologies and a fixed cost bound, certain answers for instance queries and possible answers for conjunctive queries can be computed using first-order rewriting and thus enjoy the lowest possible data complexity ($\mathsf{TC}_0$).

Method: Proposes PLGF architecture with Progressive Local-Global Fusion strategy and DualFocal Loss that integrates dual-space supervision with difficulty-aware focusing mechanism.

Result: Achieves state-of-the-art performance while reducing model size by up to 97% compared to current methods. Under comparable parameter budgets, yields over 10% accuracy improvement against strong baselines.

Conclusion: The unified solution effectively addresses computational efficiency and performance challenges in fine-grained urban flow inference through synergistic architectural efficiency and adaptive optimization.

Abstract: Fine-grained urban flow inference is crucial for urban planning and intelligent transportation systems, enabling precise traffic management and resource allocation. However, the practical deployment of existing methods is hindered by two key challenges: the prohibitive computational cost of over-parameterized models and the suboptimal performance of conventional loss functions on the highly skewed distribution of urban flows. To address these challenges, we propose a unified solution that synergizes architectural efficiency with adaptive optimization. Specifically, we first introduce PLGF, a lightweight yet powerful architecture that employs a Progressive Local-Global Fusion strategy to effectively capture both fine-grained details and global contextual dependencies. Second, we propose DualFocal Loss, a novel function that integrates dual-space supervision with a difficulty-aware focusing mechanism, enabling the model to adaptively concentrate on hard-to-predict regions. Extensive experiments on 4 real-world scenarios validate the effectiveness and scalability of our method. Notably, while achieving state-of-the-art performance, PLGF reduces the model size by up to 97% compared to current high-performing methods. Furthermore, under comparable parameter budgets, our model yields an accuracy improvement of over 10% against strong baselines. The implementation is included in the https://github.com/Yasoz/PLGF.

Method: Introduces the contraction hypothesis stating model-generated time series exhibit progressively decreasing uncertainty under recursive forecasting. Develops UCE detector that aggregates uncertainty metrics over successive prefixes to identify synthetic data.

Result: UCE consistently outperforms state-of-the-art baselines across 32 datasets, providing reliable and generalizable detection of model-generated time series.

Conclusion: The contraction hypothesis is theoretically proven and empirically validated, offering an effective solution for detecting synthetic time series through uncertainty contraction patterns in recursive forecasting.

Abstract: Motivated by the increasing risks of data misuse and fabrication, we investigate the problem of identifying synthetic time series generated by Time-Series Large Models (TSLMs) in this work. While there are extensive researches on detecting model generated text, we find that these existing methods are not applicable to time series data due to the fundamental modality difference, as time series usually have lower information density and smoother probability distributions than text data, which limit the discriminative power of token-based detectors. To address this issue, we examine the subtle distributional differences between real and model-generated time series and propose the contraction hypothesis, which states that model-generated time series, unlike real ones, exhibit progressively decreasing uncertainty under recursive forecasting. We formally prove this hypothesis under theoretical assumptions on model behavior and time series structure. Model-generated time series exhibit progressively concentrated distributions under recursive forecasting, leading to uncertainty contraction. We provide empirical validation of the hypothesis across diverse datasets. Building on this insight, we introduce the Uncertainty Contraction Estimator (UCE), a white-box detector that aggregates uncertainty metrics over successive prefixes to identify TSLM-generated time series. Extensive experiments on 32 datasets show that UCE consistently outperforms state-of-the-art baselines, offering a reliable and generalizable solution for detecting model-generated time series.

Method: Introduces metacognitive self-assessment for value alignment reflection, dynamic rule knowledge graphs that extend static rule trees, and activation steering during inference to guide rule following.

Result: Substantially reduces semantic attack success rates across education, finance, and management domains, with metacognitive assessment aligning closely with human evaluators while providing more thorough analysis.

Conclusion: MENTOR establishes a continuous self-evolution cycle that enhances generalization and reduces maintenance costs of static systems, enabling robust implicit risk mitigation for LLMs.

Abstract: Ensuring the safety and value alignment of large language models (LLMs) is critical for their deployment. Current alignment efforts primarily target explicit risks such as bias, hate speech, and violence. However, they often fail to address deeper, domain-specific implicit risks and lack a flexible, generalizable framework applicable across diverse specialized fields. Hence, we proposed MENTOR: A MEtacognition-driveN self-evoluTion framework for uncOvering and mitigating implicit Risks in LLMs on Domain Tasks. To address the limitations of labor-intensive human evaluation, we introduce a novel metacognitive self-assessment tool. This enables LLMs to reflect on potential value misalignments in their responses using strategies like perspective-taking and consequential thinking. We also release a supporting dataset of 9,000 risk queries spanning education, finance, and management to enhance domain-specific risk identification. Subsequently, based on the outcomes of metacognitive reflection, the framework dynamically generates supplementary rule knowledge graphs that extend predefined static rule trees. This enables models to actively apply validated rules to future similar challenges, establishing a continuous self-evolution cycle that enhances generalization by reducing maintenance costs and inflexibility of static systems. Finally, we employ activation steering during inference to guide LLMs in following the rules, a cost-effective method to robustly enhance enforcement across diverse contexts. Experimental results show MENTOR’s effectiveness: In defensive testing across three vertical domains, the framework substantially reduces semantic attack success rates, enabling a new level of implicit risk mitigation for LLMs. Furthermore, metacognitive assessment not only aligns closely with baseline human evaluators but also delivers more thorough and insightful analysis of LLMs value alignment.

Method: Proposes Cooperative Market Making (CMM) framework that decouples LLM features across three orthogonal dimensions (layer, task, data) and uses multiple student models for collaborative learning, integrated via Hájek-MoE.

Result: Extensive experiments on four real-world market datasets show CMM outperforms current distillation methods and RL-based market-making strategies.

Conclusion: CMM effectively addresses LLM inference speed issues in market making through multi-dimensional feature decoupling and collaborative student model learning with Hájek-MoE integration.

Abstract: Market making (MM) through Reinforcement Learning (RL) has attracted significant attention in financial trading. With the development of Large Language Models (LLMs), more and more attempts are being made to apply LLMs to financial areas. A simple, direct application of LLM as an agent shows significant performance. Such methods are hindered by their slow inference speed, while most of the current research has not studied LLM distillation for this specific task. To address this, we first propose the normalized fluorescent probe to study the mechanism of the LLM’s feature. Based on the observation found by our investigation, we propose Cooperative Market Making (CMM), a novel framework that decouples LLM features across three orthogonal dimensions: layer, task, and data. Various student models collaboratively learn simple LLM features along with different dimensions, with each model responsible for a distinct feature to achieve knowledge distillation. Furthermore, CMM introduces an Hájek-MoE to integrate the output of the student models by investigating the contribution of different models in a kernel function-generated common feature space. Extensive experimental results on four real-world market datasets demonstrate the superiority of CMM over the current distillation method and RL-based market-making strategies.

Method: Transform multiclass classification to binary problems, generate saliency maps from trained classifiers, cluster signals using saliency guidance, and use LLMs for symbolic approximation and fuzzy knowledge graph matching.

Result: Outperforms signal-only baselines in clustering and subclass identification on established time series datasets.

Conclusion: The saliency map-driven neuro-symbolic approach effectively discovers latent subclasses in time series classification, demonstrating superior performance over traditional methods.

Abstract: This paper proposes a novel neuro-symbolic approach for sensor signal-based knowledge discovery, focusing on identifying latent subclasses in time series classification tasks. The approach leverages gradient-based saliency maps derived from trained neural networks to guide the discovery process. Multiclass time series classification problems are transformed into binary classification problems through label subsumption, and classifiers are trained for each of these to yield saliency maps. The input signals, grouped by predicted class, are clustered under three distinct configurations. The centroids of the final set of clusters are provided as input to an LLM for symbolic approximation and fuzzy knowledge graph matching to discover the underlying subclasses of the original multiclass problem. Experimental results on well-established time series classification datasets demonstrate the effectiveness of our saliency map-driven method for knowledge discovery, outperforming signal-only baselines in both clustering and subclass identification.

Method: Proposed PADiff, a diffusion-based policy that integrates predictive information about teammates into the denoising process to handle non-stationary AHT scenarios.

Result: Extensive experiments across three cooperation environments demonstrate that PADiff significantly outperforms existing AHT methods.

Conclusion: Diffusion-based approaches with integrated predictive information effectively capture multimodal behaviors and enable diverse cooperation modes in ad hoc teamwork scenarios.

Abstract: Ad hoc teamwork (AHT) requires agents to collaborate with previously unseen teammates, which is crucial for many real-world applications. The core challenge of AHT is to develop an ego agent that can predict and adapt to unknown teammates on the fly. Conventional RL-based approaches optimize a single expected return, which often causes policies to collapse into a single dominant behavior, thus failing to capture the multimodal cooperation patterns inherent in AHT. In this work, we introduce PADiff, a diffusion-based approach that captures agent’s multimodal behaviors, unlocking its diverse cooperation modes with teammates. However, standard diffusion models lack the ability to predict and adapt in highly non-stationary AHT scenarios. To address this limitation, we propose a novel diffusion-based policy that integrates critical predictive information about teammates into the denoising process. Extensive experiments across three cooperation environments demonstrate that PADiff outperforms existing AHT methods significantly.

Method: A collaborative system with over 10 specialized AI agents that use structured debate, retrieval-augmented memory, and ensemble-guided evolutionary search to propose, critique, and refine SciML solutions.

Result: The framework discovered novel strategies including adaptive mixture-of-expert architectures, decomposition-based PINNs, and physics-informed operator learning models that outperform baselines by up to 4 orders of magnitude.

Conclusion: Collaborative reasoning among AI agents enables emergent methodological innovation, suggesting a path toward scalable and autonomous discovery in scientific computing.

Abstract: Scientific Machine Learning (SciML) integrates data-driven inference with physical modeling to solve complex problems in science and engineering. However, the design of SciML architectures, loss formulations, and training strategies remains an expert-driven research process, requiring extensive experimentation and problem-specific insights. Here we introduce AgenticSciML, a collaborative multi-agent system in which over 10 specialized AI agents collaborate to propose, critique, and refine SciML solutions through structured reasoning and iterative evolution. The framework integrates structured debate, retrieval-augmented method memory, and ensemble-guided evolutionary search, enabling the agents to generate and assess new hypotheses about architectures and optimization procedures. Across physics-informed learning and operator learning tasks, the framework discovers solution methods that outperform single-agent and human-designed baselines by up to four orders of magnitude in error reduction. The agents produce novel strategies – including adaptive mixture-of-expert architectures, decomposition-based PINNs, and physics-informed operator learning models – that do not appear explicitly in the curated knowledge base. These results show that collaborative reasoning among AI agents can yield emergent methodological innovation, suggesting a path toward scalable, transparent, and autonomous discovery in scientific computing.

Method: ED2D extends previous approaches by incorporating factual evidence retrieval and is designed as a persuasive multi-agent system that generates debunking transcripts through evidence-based debates.

Result: ED2D outperforms existing baselines across three misinformation detection benchmarks. When correct, its debunking transcripts show persuasive effects comparable to human experts, but when incorrect, they may reinforce misconceptions even alongside accurate human explanations.

Conclusion: ED2D shows promise for misinformation intervention but also reveals risks when systems misclassify, highlighting the need for careful deployment. A public community website was developed to foster transparency and critical thinking.

Abstract: Multi-agent debate (MAD) frameworks have emerged as promising approaches for misinformation detection by simulating adversarial reasoning. While prior work has focused on detection accuracy, it overlooks the importance of helping users understand the reasoning behind factual judgments and develop future resilience. The debate transcripts generated during MAD offer a rich but underutilized resource for transparent reasoning. In this study, we introduce ED2D, an evidence-based MAD framework that extends previous approach by incorporating factual evidence retrieval. More importantly, ED2D is designed not only as a detection framework but also as a persuasive multi-agent system aimed at correcting user beliefs and discouraging misinformation sharing. We compare the persuasive effects of ED2D-generated debunking transcripts with those authored by human experts. Results demonstrate that ED2D outperforms existing baselines across three misinformation detection benchmarks. When ED2D generates correct predictions, its debunking transcripts exhibit persuasive effects comparable to those of human experts; However, when ED2D misclassifies, its accompanying explanations may inadvertently reinforce users’misconceptions, even when presented alongside accurate human explanations. Our findings highlight both the promise and the potential risks of deploying MAD systems for misinformation intervention. We further develop a public community website to help users explore ED2D, fostering transparency, critical thinking, and collaborative fact-checking.

Method: Maintains an evolving report as memory and periodically synthesizes insights; develops Efficiency-Aware Policy Optimization (EAPO) with geometric reward discounting and adaptive downsampling for stable distributed training.

Result: Achieves +14.5pp average improvement across six benchmarks, extends to 2048 interactions with dramatic performance gains (3.5% to 42.5%), and serves as effective prompting strategy improving frontier models by up to 19.2pp over ReAct.

Conclusion: IterResearch is a versatile solution for long-horizon reasoning, effective both as a trained agent and as a prompting paradigm for frontier models, demonstrating unprecedented interaction scaling and performance improvements.

Abstract: Recent advances in deep-research agents have shown promise for autonomous knowledge construction through dynamic reasoning over external sources. However, existing approaches rely on a mono-contextual paradigm that accumulates all information in a single, expanding context window, leading to context suffocation and noise contamination that limit their effectiveness on long-horizon tasks. We introduce IterResearch, a novel iterative deep-research paradigm that reformulates long-horizon research as a Markov Decision Process with strategic workspace reconstruction. By maintaining an evolving report as memory and periodically synthesizing insights, our approach preserves consistent reasoning capacity across arbitrary exploration depths. We further develop Efficiency-Aware Policy Optimization (EAPO), a reinforcement learning framework that incentivizes efficient exploration through geometric reward discounting and enables stable distributed training via adaptive downsampling. Extensive experiments demonstrate that IterResearch achieves substantial improvements over existing open-source agents with average +14.5pp across six benchmarks and narrows the gap with frontier proprietary systems. Remarkably, our paradigm exhibits unprecedented interaction scaling, extending to 2048 interactions with dramatic performance gains (from 3.5% to 42.5%), and serves as an effective prompting strategy, improving frontier models by up to 19.2pp over ReAct on long-horizon tasks. These findings position IterResearch as a versatile solution for long-horizon reasoning, effective both as a trained agent and as a prompting paradigm for frontier models.

Method: Two-stage approach: 1) Algorithmically construct the largest human-attribute taxonomy from thousands of real user-ChatGPT conversations, 2) Progressively sample attributes from this taxonomy to conditionally generate coherent personas with hundreds of structured attributes and ~1MB of narrative text.

Result: Significant improvements: 32% higher attribute diversity, 44% greater profile uniqueness, 11.6% average improvement in personalized question answering accuracy, 31.7% reduction in gap between simulated LLM citizens and human responses, and 17% reduction in Big Five personality test performance gap.

Conclusion: DEEPPERSONA provides a rigorous, scalable, privacy-free platform for high-fidelity human simulation and personalized AI research, enabling more realistic and diverse synthetic personas than previous methods.

Abstract: Simulating human profiles by instilling personas into large language models (LLMs) is rapidly transforming research in agentic behavioral simulation, LLM personalization, and human-AI alignment. However, most existing synthetic personas remain shallow and simplistic, capturing minimal attributes and failing to reflect the rich complexity and diversity of real human identities. We introduce DEEPPERSONA, a scalable generative engine for synthesizing narrative-complete synthetic personas through a two-stage, taxonomy-guided method. First, we algorithmically construct the largest-ever human-attribute taxonomy, comprising over hundreds of hierarchically organized attributes, by mining thousands of real user-ChatGPT conversations. Second, we progressively sample attributes from this taxonomy, conditionally generating coherent and realistic personas that average hundreds of structured attributes and roughly 1 MB of narrative text, two orders of magnitude deeper than prior works. Intrinsic evaluations confirm significant improvements in attribute diversity (32 percent higher coverage) and profile uniqueness (44 percent greater) compared to state-of-the-art baselines. Extrinsically, our personas enhance GPT-4.1-mini’s personalized question answering accuracy by 11.6 percent on average across ten metrics and substantially narrow (by 31.7 percent) the gap between simulated LLM citizens and authentic human responses in social surveys. Our generated national citizens reduced the performance gap on the Big Five personality test by 17 percent relative to LLM-simulated citizens. DEEPPERSONA thus provides a rigorous, scalable, and privacy-free platform for high-fidelity human simulation and personalized AI research.

Method: Created DigiData dataset through comprehensive exploration of app features (unlike unstructured interactions), and developed DigiData-Bench benchmark with dynamic evaluation protocols and AI-powered assessments.

Result: Produces a diverse, multi-modal dataset with higher goal complexity than existing datasets, and introduces more reliable evaluation methods beyond step-accuracy metrics.

Conclusion: These contributions significantly advance mobile control agent development, enabling more intuitive human-device interactions through better training data and evaluation frameworks.

Abstract: AI agents capable of controlling user interfaces have the potential to transform human interaction with digital devices. To accelerate this transformation, two fundamental building blocks are essential: high-quality datasets that enable agents to achieve complex and human-relevant goals, and robust evaluation methods that allow researchers and practitioners to rapidly enhance agent performance. In this paper, we introduce DigiData, a large-scale, high-quality, diverse, multi-modal dataset designed for training mobile control agents. Unlike existing datasets, which derive goals from unstructured interactions, DigiData is meticulously constructed through comprehensive exploration of app features, resulting in greater diversity and higher goal complexity. Additionally, we present DigiData-Bench, a benchmark for evaluating mobile control agents on real-world complex tasks. We demonstrate that the commonly used step-accuracy metric falls short in reliably assessing mobile control agents and, to address this, we propose dynamic evaluation protocols and AI-powered evaluations as rigorous alternatives for agent assessment. Our contributions aim to significantly advance the development of mobile control agents, paving the way for more intuitive and effective human-device interactions.

Method: LD first uses large LLMs to create discrete functions and their usage examples, then fine-tunes small LLMs to learn the logic behind function selection and invocation based on instructions and current states.

Result: Experiments show that small LLMs equipped with LD achieve outstanding results in planning and decision-making tasks, comparable to or even surpassing large LLMs.

Conclusion: The Logic Distillation framework successfully bridges the logical reasoning capability gap between large and small LLMs, enabling efficient deployment of reasoning capabilities on resource-constrained devices.

Abstract: Large language models (LLMs) have garnered increasing attention owing to their powerful logical reasoning capabilities. Generally, larger LLMs (L-LLMs) that require paid interfaces exhibit significantly superior performance compared to smaller LLMs (S-LLMs) that can be deployed on a variety of devices. Knowledge distillation (KD) aims to empower S-LLMs with the capabilities of L-LLMs, while S-LLMs merely mimic the outputs of L-LLMs, failing to get the powerful logical reasoning capabilities. Consequently, S-LLMs are helpless when it comes to planning and decision-making tasks that require logical reasoning capabilities. To tackle the identified challenges, we propose a novel framework called Logic Distillation (LD). Initially, LD employs L-LLMs to instantiate complex instructions into discrete functions and illustrates their usage to establish a function base. Subsequently, based on the function base, LD fine-tunes S-LLMs to learn the logic employed by L-LLMs in planning and decision-making. During testing, LD utilizes a retriever to identify the top-$K$ relevant functions based on instructions and current states, which will be selected and invoked by S-LLMs. Ultimately, S-LLMs yield planning and decision-making outcomes, function by function. Relevant experiments demonstrate that with the assistance of LD, S-LLMs can achieve outstanding results in planning and decision-making tasks, comparable to, or even surpassing, those of L-LLMs.

Method: HI-RL forms concepts by extracting high-level categories of critical environmental information and constructs adaptive concept-associated probabilistic beliefs as experience priors to guide value or policy updates.

Result: When integrated into various RL algorithms (DQN, PPO, SAC, TD3), HI-RL consistently improved sample efficiency and performance across both discrete and continuous control benchmarks.

Conclusion: HI-RL successfully emulates human intelligence in RL by using conceptual abstraction and probabilistic beliefs, providing an efficient experience utilization paradigm that can be directly integrated into existing RL frameworks.

Abstract: Reinforcement learning (RL) has achieved significant success but is hindered by inefficiency and instability, relying on large amounts of trial-and-error data and failing to efficiently use past experiences to guide decisions. However, humans achieve remarkably efficient learning from experience, attributed to abstracting concepts and updating associated probabilistic beliefs by integrating both uncertainty and prior knowledge, as observed by cognitive science. Inspired by this, we introduce Conceptual Belief-Informed Reinforcement Learning to emulate human intelligence (HI-RL), an efficient experience utilization paradigm that can be directly integrated into existing RL frameworks. HI-RL forms concepts by extracting high-level categories of critical environmental information and then constructs adaptive concept-associated probabilistic beliefs as experience priors to guide value or policy updates. We evaluate HI-RL by integrating it into various existing value- and policy-based algorithms (DQN, PPO, SAC, and TD3) and demonstrate consistent improvements in sample efficiency and performance across both discrete and continuous control benchmarks.

Method: GlitchMiner uses a gradient-guided local search strategy to efficiently explore discrete token space without model-specific heuristics or large-batch sampling, focusing on maximizing predictive entropy to identify anomalous behavior.

Result: Extensive experiments across ten LLMs from five major model families show GlitchMiner consistently outperforms existing approaches in detection accuracy and query efficiency.

Conclusion: GlitchMiner provides a generalizable and scalable solution for effective glitch token discovery, offering improved reliability and safety for LLMs.

Abstract: Glitch tokens, inputs that trigger unpredictable or anomalous behavior in Large Language Models (LLMs), pose significant challenges to model reliability and safety. Existing detection methods primarily rely on heuristic embedding patterns or statistical anomalies within internal representations, limiting their generalizability across different model architectures and potentially missing anomalies that deviate from observed patterns. We introduce GlitchMiner, an behavior-driven framework designed to identify glitch tokens by maximizing predictive entropy. Leveraging a gradient-guided local search strategy, GlitchMiner efficiently explores the discrete token space without relying on model-specific heuristics or large-batch sampling. Extensive experiments across ten LLMs from five major model families demonstrate that GlitchMiner consistently outperforms existing approaches in detection accuracy and query efficiency, providing a generalizable and scalable solution for effective glitch token discovery. Code is available at [https://github.com/wooozihu/GlitchMiner]

Method: Constructs two graphs: semantic mind graph (captures authors’ thought process) and hierarchical background graph (delineates research domains), then uses retrieval to extract relevant content for explaining review comments.

Result: Extensive experiments show SEAGraph excels in review comment understanding tasks and offers significant benefits to authors.

Conclusion: SEAGraph bridges the gap between reviewers’ critiques and authors’ comprehension, contributing to a more efficient, transparent and collaborative scientific publishing ecosystem.

Abstract: Peer review, as a cornerstone of scientific research, ensures the integrity and quality of scholarly work by providing authors with objective feedback for refinement. However, in the traditional peer review process, authors often receive vague or insufficiently detailed feedback, which provides limited assistance and leads to a more time-consuming review cycle. If authors can identify some specific weaknesses in their paper, they can not only address the reviewer’s concerns but also improve their work. This raises the critical question of how to enhance authors’ comprehension of review comments. In this paper, we present SEAGraph, a novel framework developed to clarify review comments by uncovering the underlying intentions behind them. We construct two types of graphs for each paper: the semantic mind graph, which captures the authors’ thought process, and the hierarchical background graph, which delineates the research domains related to the paper. A retrieval method is then designed to extract relevant content from both graphs, facilitating coherent explanations for the review comments. Extensive experiments show that SEAGraph excels in review comment understanding tasks, offering significant benefits to authors. By bridging the gap between reviewers’ critiques and authors’ comprehension, SEAGraph contributes to a more efficient, transparent and collaborative scientific publishing ecosystem.

Method: Uses pretrained vision-language foundation models to transform instructions into pixel-level semantic masks. Implements dual-resolution perception (global context from low-res, local features from high-res) and a consistency-driven diffusion transformer action head.

Result: Outperforms state-of-the-art vision-language manipulation frameworks and visuomotor imitation learning policies, especially in complex scenes. Shows strong zero-shot generalization with novel objects and distractions. Action head achieves order-of-magnitude speed improvement while maintaining success rates.

Conclusion: ImitDiff effectively addresses performance degradation in complex scenes through semantic-guided dual-resolution perception and achieves significant improvements in both performance and inference speed.

Abstract: Visuomotor imitation learning policies enable robots to efficiently acquire manipulation skills from visual demonstrations. However, as scene complexity and visual distractions increase, policies that perform well in simple settings often experience substantial performance degradation. To address this challenge, we propose ImitDiff, a diffusion-based imitation learning policy guided by fine-grained semantics within a dual-resolution workflow. Leveraging pretrained priors of vision-language foundation models, our method transforms high-level instructions into pixel-level visual semantic masks. These masks guide a dual-resolution perception pipeline that captures both global context (e.g., overall layout) from low-resolution observation and fine-grained local features (e.g., geometric details) from high-resolution observation, enabling the policy to focus on task-relevant regions. Additionally, we introduce a consistency-driven diffusion transformer action head that bridges visual semantic conditions and real-time action generation. Extensive experiments demonstrate that ImitDiff outperforms state-of-the-art vision-language manipulation frameworks, as well as visuomotor imitation learning policies, particularly under increased scene complexity and visual distractions. Notably, ImitDiff exhibits strong generalization in zero-shot settings involving novel objects and visual distractions. Furthermore, our consistency-driven action head achieves an order-of-magnitude improvement in inference speed while maintaining competitive success rates.

Method: Prunes completions with low absolute advantages to reduce gradient calculations, and uses dynamic completion allocation to maximize GPU utilization by adding more questions.

Result: Achieves up to 7.98x speedup on GSM8K and 3.48x on Math while preserving or enhancing accuracy compared to original GRPO.

Conclusion: CPPO effectively reduces training costs of reasoning models while maintaining performance, making GRPO-based training more efficient.

Abstract: This paper introduces Completion Pruning Policy Optimization (CPPO) to accelerate the training of reasoning models based on Group Relative Policy Optimization (GRPO). GRPO, while effective, incurs high training costs due to the need to sample multiple completions for each question. Our experiment and theoretical analysis reveal that the number of completions impacts model accuracy yet increases training time multiplicatively, and not all completions contribute equally to policy training – their contribution depends on their relative advantage. To address these issues, we propose CPPO, which prunes completions with low absolute advantages, significantly reducing the number needed for gradient calculation and updates. Additionally, we introduce a dynamic completion allocation strategy to maximize GPU utilization by incorporating additional questions, further enhancing training efficiency. Experiments show that CPPO achieves up to $7.98\times$ speedup on GSM8K and $3.48\times$ on Math while preserving or even enhancing the accuracy compared to the original GRPO. We release our code at \href{https://github.com/lzhxmu/CPPO}{https://github.com/lzhxmu/CPPO}.

Method: Two key innovations: (1) context-aware synthesis strategy that shifts query topics while preserving reasoning patterns, and (2) tool-based answer reconstruction that reuses remote-generated plug-and-play solutions with code snippets.

Result: Achieves better reasoning accuracy than solely using local models while providing stronger data protection than fully relying on remote models. Improves accuracy by 16.2%-43.6% and reduces data leakage by 2.3%-44.6% compared to existing approaches.

Conclusion: The proposed framework successfully balances the trade-off between reasoning performance and data privacy, enabling effective numerical reasoning on constrained devices without exposing sensitive local data to remote models.

Abstract: Numerical reasoning over documents, which demands both contextual understanding and logical inference, is challenging for low-capacity local models deployed on computation-constrained devices. Although such complex reasoning queries could be routed to powerful remote models like GPT-4, exposing local data raises significant data leakage concerns. Existing mitigation methods generate problem descriptions or examples for remote assistance. However, the inherent complexity of numerical reasoning hinders the local model from generating logically equivalent queries and accurately inferring answers with remote guidance. In this paper, we present a model collaboration framework with two key innovations: (1) a context-aware synthesis strategy that shifts the query topics while preserving reasoning patterns; and (2) a tool-based answer reconstruction approach that reuses the remote-generated plug-and-play solution with code snippets. Experimental results demonstrate that our method achieves better reasoning accuracy than solely using local models while providing stronger data protection than fully relying on remote models. Furthermore, our method improves accuracy by 16.2% - 43.6% while reducing data leakage by 2.3% - 44.6% compared to existing data protection approaches.

Method: Uses lightweight model training for structural embeddings and logical rules, bottom-up graph retrieval guided by rules, GCN adapter to enhance embeddings, and integrates structural information into prompts for LLM fine-tuning.

Result: Superior performance on two general domain and two biomedical benchmark datasets, with interpretability demonstrated through biomedical case study.

Conclusion: DrKGC effectively bridges the gap between LLMs and graph structures, achieving better KGC performance while maintaining interpretability and practical utility.

Abstract: Knowledge graph completion (KGC) aims to predict missing triples in knowledge graphs (KGs) by leveraging existing triples and textual information. Recently, generative large language models (LLMs) have been increasingly employed for graph tasks. However, current approaches typically encode graph context in textual form, which fails to fully exploit the potential of LLMs for perceiving and reasoning about graph structures. To address this limitation, we propose DrKGC (Dynamic Subgraph Retrieval-Augmented LLMs for Knowledge Graph Completion). DrKGC employs a flexible lightweight model training strategy to learn structural embeddings and logical rules within the KG. It then leverages a novel bottom-up graph retrieval method to extract a subgraph for each query guided by the learned rules. Finally, a graph convolutional network (GCN) adapter uses the retrieved subgraph to enhance the structural embeddings, which are then integrated into the prompt for effective LLM fine-tuning. Experimental results on two general domain benchmark datasets and two biomedical datasets demonstrate the superior performance of DrKGC. Furthermore, a realistic case study in the biomedical domain highlights its interpretability and practical utility.

Method: Implemented and compared classical vs LLM-based versions of two established swarm algorithms (Boids and Ant Colony Optimization) using OpenAI’s Swarm framework.

Result: LLM-powered swarms can emulate swarm-like dynamics but require roughly 300x more computation time than classical counterparts, with LLM-based Boids simulation showing particularly high computational costs.

Conclusion: While LLM-driven swarms can replicate swarm intelligence behaviors, current computational limitations make them impractical for real-time systems due to excessive overhead.

Abstract: Swarm intelligence describes how simple, decentralized agents can collectively produce complex behaviors. Recently, the concept of swarming has been extended to large language model (LLM)-powered systems, such as OpenAI’s Swarm (OAS) framework, where agents coordinate through natural language prompts. This paper evaluates whether such systems capture the fundamental principles of classical swarm intelligence: decentralization, simplicity, emergence, and scalability. Using OAS, we implement and compare classical and LLM-based versions of two well-established swarm algorithms: Boids and Ant Colony Optimization. Results indicate that while LLM-powered swarms can emulate swarm-like dynamics, they are constrained by substantial computational overhead. For instance, our LLM-based Boids simulation required roughly 300x more computation time than its classical counterpart, highlighting current limitations in applying LLM-driven swarms to real-time systems.

Method: A framework with case database, image retrieval module, and LLM-based reasoning module that dynamically integrates external knowledge and retrieved similar cases via prompt tuning, using LPIPS- and CLIP-based retrieval strategy.

Result: Boosted GLM-4V’s accuracy to 50% on real-site data, a 35.49% improvement over baselines, with consistent gains across hazard types. Ablation studies validated the effectiveness of the image retrieval strategy.

Conclusion: The proposed technique significantly improves identification accuracy and contextual understanding, demonstrating strong generalization and offering a practical path for intelligent safety risk detection in construction.

Abstract: This study addresses construction site hazard identification by proposing a retrieval-augmented framework that enhances large language models (LLMs) without requiring fine-tuning. Current LLM-based approaches face limitations: image-text matching struggles with complex hazards, while instruction tuning lacks generalization and is resource-intensive. Our method dynamically integrates external knowledge and retrieved similar cases via prompt tuning, overcoming LLMs’ limitations in domain knowledge and feature correlation. The framework comprises a case database, an image retrieval module, and an LLM-based reasoning module. Evaluated on real-site data, our approach boosted GLM-4V’s accuracy to 50%, a 35.49% improvement over baselines, with consistent gains across hazard types. Ablation studies validated the effectiveness of our image retrieval strategy, showing the superiority of our LPIPS- and CLIP-based method. The proposed technique significantly improves identification accuracy and contextual understanding, demonstrating strong generalization and offering a practical path for intelligent safety risk detection in construction.

Method: Built on General Video Game AI framework using game description language for rapid game creation, representing game scenes as ASCII characters, with zero-shot evaluations across diverse games and levels.

Result: LLMs show persistent limitations in spatial reasoning and basic planning, consistently making spatial and logical errors. Structured prompting and spatial grounding provide partial improvements but benchmark remains unsolved.

Conclusion: GVGAI-LLM provides a reproducible testbed for advancing research on language model capabilities, particularly for agentic behavior and contextual reasoning, highlighting significant remaining challenges.

Abstract: We introduce GVGAI-LLM, a video game benchmark for evaluating the reasoning and problem-solving capabilities of large language models (LLMs). Built on the General Video Game AI framework, it features a diverse collection of arcade-style games designed to test a model’s ability to handle tasks that differ from most existing LLM benchmarks. The benchmark leverages a game description language that enables rapid creation of new games and levels, helping to prevent overfitting over time. Each game scene is represented by a compact set of ASCII characters, allowing for efficient processing by language models. GVGAI-LLM defines interpretable metrics, including the meaningful step ratio, step efficiency, and overall score, to assess model behavior. Through zero-shot evaluations across a broad set of games and levels with diverse challenges and skill depth, we reveal persistent limitations of LLMs in spatial reasoning and basic planning. Current models consistently exhibit spatial and logical errors, motivating structured prompting and spatial grounding techniques. While these interventions lead to partial improvements, the benchmark remains very far from solved. GVGAI-LLM provides a reproducible testbed for advancing research on language model capabilities, with a particular emphasis on agentic behavior and contextual reasoning.

Method: Proposed a twofold taxonomy around agentic capabilities (planning, tool use, memory, reasoning, self-improvement, perception) and applications, synthesizing over 500 works.

Result: Consolidated open-source environments, benchmarks, and frameworks into a practical compendium to accelerate future research in scalable AI agents.

Conclusion: Reinforcement learning is crucial for developing adaptive, robust agentic behavior, with identified opportunities and challenges for general-purpose AI agents.

Abstract: The emergence of agentic reinforcement learning (Agentic RL) marks a paradigm shift from conventional reinforcement learning applied to large language models (LLM RL), reframing LLMs from passive sequence generators into autonomous, decision-making agents embedded in complex, dynamic worlds. This survey formalizes this conceptual shift by contrasting the degenerate single-step Markov Decision Processes (MDPs) of LLM-RL with the temporally extended, partially observable Markov decision processes (POMDPs) that define Agentic RL. Building on this foundation, we propose a comprehensive twofold taxonomy: one organized around core agentic capabilities, including planning, tool use, memory, reasoning, self-improvement, and perception, and the other around their applications across diverse task domains. Central to our thesis is that reinforcement learning serves as the critical mechanism for transforming these capabilities from static, heuristic modules into adaptive, robust agentic behavior. To support and accelerate future research, we consolidate the landscape of open-source environments, benchmarks, and frameworks into a practical compendium. By synthesizing over five hundred recent works, this survey charts the contours of this rapidly evolving field and highlights the opportunities and challenges that will shape the development of scalable, general-purpose AI agents.

Method: Developed a sparse GNN that selectively encodes relevant local relationships and explicitly integrates spatial goal features, validated using novel drone mission scenarios in PDDL-based grid worlds.

Result: Method scales effectively to larger grid sizes previously infeasible with dense graphs, and substantially improves policy generalization and success rates in drone mission scenarios.

Conclusion: Provides a practical foundation for addressing realistic, large-scale generalized planning tasks by overcoming scalability limitations of dense graph representations.

Abstract: Generalized planning using deep reinforcement learning (RL) combined with graph neural networks (GNNs) has shown promising results in various symbolic planning domains described by PDDL. However, existing approaches typically represent planning states as fully connected graphs, leading to a combinatorial explosion in edge information and substantial sparsity as problem scales grow, especially evident in large grid-based environments. This dense representation results in diluted node-level information, exponentially increases memory requirements, and ultimately makes learning infeasible for larger-scale problems. To address these challenges, we propose a sparse, goal-aware GNN representation that selectively encodes relevant local relationships and explicitly integrates spatial features related to the goal. We validate our approach by designing novel drone mission scenarios based on PDDL within a grid world, effectively simulating realistic mission execution environments. Our experimental results demonstrate that our method scales effectively to larger grid sizes previously infeasible with dense graph representations and substantially improves policy generalization and success rates. Our findings provide a practical foundation for addressing realistic, large-scale generalized planning tasks.

Method: Tree-guided Diffusion Planner (TDP) frames planning as tree search with bi-level sampling: diverse parent trajectories via training-free particle guidance for exploration, and sub-trajectory refinement through fast conditional denoising guided by task objectives.

Result: TDP consistently outperforms state-of-the-art approaches on maze gold-picking, robot arm block manipulation, and AntMaze multi-goal exploration tasks.

Conclusion: TDP effectively addresses gradient guidance limitations by exploring diverse trajectory regions and leveraging gradient information across expanded solution space using only pretrained models and test-time rewards.

Abstract: Planning with pretrained diffusion models has emerged as a promising approach for solving test-time guided control problems. Standard gradient guidance typically performs optimally under convex, differentiable reward landscapes. However, it shows substantially reduced effectiveness in real-world scenarios with non-convex objectives, non-differentiable constraints, and multi-reward structures. Furthermore, recent supervised planning approaches require task-specific training or value estimators, which limits test-time flexibility and zero-shot generalization. We propose a Tree-guided Diffusion Planner (TDP), a zero-shot test-time planning framework that balances exploration and exploitation through structured trajectory generation. We frame test-time planning as a tree search problem using a bi-level sampling process: (1) diverse parent trajectories are produced via training-free particle guidance to encourage broad exploration, and (2) sub-trajectories are refined through fast conditional denoising guided by task objectives. TDP addresses the limitations of gradient guidance by exploring diverse trajectory regions and harnessing gradient information across this expanded solution space using only pretrained models and test-time reward signals. We evaluate TDP on three diverse tasks: maze gold-picking, robot arm block manipulation, and AntMaze multi-goal exploration. TDP consistently outperforms state-of-the-art approaches on all tasks. The project page can be found at: https://tree-diffusion-planner.github.io.

Method: Proposed Multi-Agent Hierarchical Task Generation (MAHTG) system that extracts research-worthy inspirations from seminar transcripts and translates them into high-quality research tasks, ensuring traceability while filtering noise.

Result: Curated DeepResearch Arena with over 10,000 high-quality research tasks from 200+ academic seminars across 12 disciplines, showing substantial challenges for current state-of-the-art agents with clear performance gaps.

Conclusion: DeepResearch Arena provides a more faithful evaluation benchmark for deep research agents by leveraging academic seminar discourse, better reflecting real-world research environments and reducing data leakage risks.

Abstract: Deep research agents have attracted growing attention for their potential to orchestrate multi-stage research workflows, spanning literature synthesis, methodological design, and empirical verification. Despite these strides, evaluating their research capability faithfully is rather challenging due to the difficulty of collecting frontier research questions that genuinely capture researchers’ attention and intellectual curiosity. To address this gap, we introduce DeepResearch Arena, a benchmark grounded in academic seminars that capture rich expert discourse and interaction, better reflecting real-world research environments and reducing the risk of data leakage. To automatically construct DeepResearch Arena, we propose a Multi-Agent Hierarchical Task Generation (MAHTG) system that extracts research-worthy inspirations from seminar transcripts. The MAHTG system further translates research-worthy inspirations into high-quality research tasks, ensuring the traceability of research task formulation while filtering noise. With the MAHTG system, we curate DeepResearch Arena with over 10,000 high-quality research tasks from over 200 academic seminars, spanning 12 disciplines, such as literature, history, and science. Our extensive evaluation shows that DeepResearch Arena presents substantial challenges for current state-of-the-art agents, with clear performance gaps observed across different models.

Method: Linear probing on steering vector activations across 15 models from 0.27B to 70B parameters from four families.

Result: Clear power-law scaling: evaluation awareness increases predictably with model size.

Conclusion: This scaling law enables forecasting deceptive behavior in future larger models and guides scale-aware evaluation strategies for AI safety.

Abstract: Large language models (LLMs) can internally distinguish between evaluation and deployment contexts, a behaviour known as \emph{evaluation awareness}. This undermines AI safety evaluations, as models may conceal dangerous capabilities during testing. Prior work demonstrated this in a single $70$B model, but the scaling relationship across model sizes remains unknown. We investigate evaluation awareness across $15$ models scaling from $0.27$B to $70$B parameters from four families using linear probing on steering vector activations. Our results reveal a clear power-law scaling: evaluation awareness increases predictably with model size. This scaling law enables forecasting deceptive behavior in future larger models and guides the design of scale-aware evaluation strategies for AI safety. A link to the implementation of this paper can be found at https://anonymous.4open.science/r/evaluation-awareness-scaling-laws/README.md.

Method: Proposes a physics-informed AI framework that explicitly integrates vehicle kinematics, interaction feasibility, and traffic-safety metrics (distance headway, time headway, time-to-collision, closing gap time) into the learning process. Formulates lane-change prediction as a three-class problem (left change, right change, no change) and evaluates across both straight highway segments and complex ramp scenarios.

Result: LightGBM model achieves up to 99.8% accuracy and 93.6% macro F1 on highway data (highD), and 96.1% accuracy and 88.7% macro F1 on ramp scenarios (exiD) at 1-second horizon, outperforming a two-layer stacked LSTM baseline.

Conclusion: The findings demonstrate the practical advantages of a physics-informed and feature-rich machine learning framework for real-time lane-change intention prediction in autonomous driving systems.

Abstract: Lane-change maneuvers are a leading cause of highway accidents, underscoring the need for accurate intention prediction to improve the safety and decision-making of autonomous driving systems. While prior studies using machine learning and deep learning methods (e.g., SVM, CNN, LSTM, Transformers) have shown promise, most approaches remain limited by binary classification, lack of scenario diversity, and degraded performance under longer prediction horizons. In this study, we propose a physics-informed AI framework that explicitly integrates vehicle kinematics, interaction feasibility, and traffic-safety metrics (e.g., distance headway, time headway, time-to-collision, closing gap time) into the learning process. lane-change prediction is formulated as a three-class problem that distinguishes left change, right change, and no change, and is evaluated across both straight highway segments (highD) and complex ramp scenarios (exiD). By integrating vehicle kinematics with interaction features, our machine learning models, particularly LightGBM, achieve state-of-the-art accuracy and strong generalization. Results show up to 99.8% accuracy and 93.6% macro F1 on highD, and 96.1% accuracy and 88.7% macro F1 on exiD at a 1-second horizon, outperforming a two-layer stacked LSTM baseline. These findings demonstrate the practical advantages of a physics-informed and feature-rich machine learning framework for real-time lane-change intention prediction in autonomous driving systems.

Method: The authors develop a cost-benefit analysis framework considering hardware requirements, operational expenses, and performance benchmarks of open-source models (Qwen, Llama, Mistral, etc.), then compare total local deployment costs with major cloud providers’ subscription fees.

Result: The analysis provides estimated breakeven points based on usage levels and performance needs, showing when on-premise LLM deployment becomes economically viable compared to commercial subscription services.

Conclusion: The findings give organizations a practical framework for planning their LLM strategies by identifying the optimal deployment approach based on their specific usage patterns and requirements.

Abstract: Large language models (LLMs) are becoming increasingly widespread. Organizations that want to use AI for productivity now face an important decision. They can subscribe to commercial LLM services or deploy models on their own infrastructure. Cloud services from providers such as OpenAI, Anthropic, and Google are attractive because they provide easy access to state-of-the-art models and are easy to scale. However, concerns about data privacy, the difficulty of switching service providers, and long-term operating costs have driven interest in local deployment of open-source models. This paper presents a cost-benefit analysis framework to help organizations determine when on-premise LLM deployment becomes economically viable compared to commercial subscription services. We consider the hardware requirements, operational expenses, and performance benchmarks of the latest open-source models, including Qwen, Llama, Mistral, and etc. Then we compare the total cost of deploying these models locally with the major cloud providers subscription fee. Our findings provide an estimated breakeven point based on usage levels and performance needs. These results give organizations a practical framework for planning their LLM strategies.

Method: Proposes TERAG framework incorporating Personalized PageRank during retrieval phase for efficient graph construction with low token consumption.

Result: Achieves at least 80% accuracy of widely used graph-based RAG methods while consuming only 3%-11% of output tokens.

Conclusion: TERAG is well-suited for large-scale and cost-sensitive deployment scenarios due to its low token footprint and efficient construction pipeline.

Abstract: Graph-based Retrieval-augmented generation (RAG) has become a widely studied approach for improving the reasoning, accuracy, and factuality of Large Language Models (LLMs). However, many existing graph-based RAG systems overlook the high cost associated with LLM token usage during graph construction, hindering large-scale adoption. To address this, we propose TERAG, a simple yet effective framework designed to build informative graphs at a significantly lower cost. Inspired by HippoRAG, we incorporate Personalized PageRank (PPR) during the retrieval phase, and we achieve at least 80% of the accuracy of widely used graph-based RAG methods while consuming only 3%-11% of the output tokens. With its low token footprint and efficient construction pipeline, TERAG is well-suited for large-scale and cost-sensitive deployment scenarios.

Method: Proposed PaperArena benchmark with modular platform offering tools like multimodal parsing, context retrieval, and programmatic computation to evaluate agents on research questions requiring cross-paper reasoning.

Result: Even the most advanced LLM-powered agent system achieved only 38.78% average accuracy, dropping to 18.47% on hard tasks. All agents showed inefficient tool usage, often invoking unnecessary tools.

Conclusion: PaperArena reveals significant gaps in current agent capabilities for scientific discovery and provides a standardized platform for developing more capable agents for complex knowledge-intensive tasks.

Abstract: Understanding and reasoning on the web-scale scientific literature is a crucial touchstone for large language model (LLM) based agents designed to support complex knowledge-intensive tasks. However, existing works are mainly restricted to tool-free tasks within isolated papers, largely due to the lack of a benchmark for cross-paper reasoning and multi-tool orchestration in real research scenarios. In this work, we propose PaperArena, an evaluation benchmark for agents to address real-world research questions that typically require integrating information across multiple papers with the assistance of external tools. Given a research question, agents should integrate diverse formats across multiple papers through reasoning and interacting with appropriate tools, thereby producing a well-grounded answer. To support standardized evaluation, we provide a modular and extensible platform for agent execution, offering tools such as multimodal parsing, context retrieval, and programmatic computation. Experimental results reveal that even the most advanced LLM powering a well-established agent system achieves merely 38.78% average accuracy. On the hard subset, accuracy drops to only 18.47%, highlighting great potential for improvement. We also present several empirical findings, including that all agents tested exhibit inefficient tool usage, often invoking more tools than necessary to solve a task. We invite the community to adopt PaperArena to develop and evaluate more capable agents for scientific discovery. Our code and data are available https://github.com/Melmaphother/PaperArena.

Method: Developed Jr. AI Scientist that follows a research workflow: analyzes baseline paper limitations, formulates novel hypotheses, iteratively conducts experiments using modern coding agents for complex implementations, and writes papers with results.

Result: Successfully generated new research papers building on real NeurIPS, IJCV, and ICLR works; papers received higher review scores than existing fully automated systems in evaluations using AI Reviewers, author-led assessments, and Agents4Science submissions.

Conclusion: Jr. AI Scientist clarifies the current role and limitations of AI Scientist systems, showing they can generate valuable contributions but still require human expertise and pose emerging risks that need addressing.

Abstract: Understanding the current capabilities and risks of AI Scientist systems is essential for ensuring trustworthy and sustainable AI-driven scientific progress while preserving the integrity of the academic ecosystem. To this end, we develop Jr. AI Scientist, a state-of-the-art autonomous AI scientist system that mimics the core research workflow of a novice student researcher: Given the baseline paper from the human mentor, it analyzes its limitations, formulates novel hypotheses for improvement, and iteratively conducts experiments until improvements are realized, and writes a paper with the results. Unlike previous approaches that assume full automation or operate on small-scale code, Jr. AI Scientist follows a well-defined research workflow and leverages modern coding agents to handle complex, multi-file implementations, leading to scientifically valuable contributions. Through our experiments, the Jr. AI Scientist successfully generated new research papers that build upon real NeurIPS, IJCV, and ICLR works by proposing and implementing novel methods. For evaluation, we conducted automated assessments using AI Reviewers, author-led evaluations, and submissions to Agents4Science, a venue dedicated to AI-driven scientific contributions. The findings demonstrate that Jr. AI Scientist generates papers receiving higher review scores than existing fully automated systems. Nevertheless, we identify important limitations from both the author evaluation and the Agents4Science reviews, indicating the potential risks of directly applying current AI Scientist systems and key challenges for future research. Finally, we comprehensively report various risks identified during development. We believe this study clarifies the current role and limitations of AI Scientist systems, offering insights into the areas that still require human expertise and the risks that may emerge as these systems evolve.

Method: Uses island-based genetic algorithms for population diversity, inspiration-based crossover using LLM context windows, and meta-prompting for dynamic solution space exploration.

Result: Surpassed AlphaEvolve’s performance on several challenging mathematical benchmarks from the AlphaEvolve evaluation set.

Conclusion: CodeEvolve demonstrates the effectiveness of combining evolutionary algorithms with LLMs for computational problem solving and is released as open-source to foster collaboration.

Abstract: In this work, we introduce CodeEvolve, an open-source evolutionary coding agent that unites Large Language Models (LLMs) with genetic algorithms to solve complex computational problems. Our framework adapts powerful evolutionary concepts to the LLM domain, building upon recent methods for generalized scientific discovery. CodeEvolve employs an island-based genetic algorithm to maintain population diversity and increase throughput, introduces a novel inspiration-based crossover mechanism that leverages the LLMs context window to combine features from successful solutions, and implements meta-prompting strategies for dynamic exploration of the solution space. We conduct a rigorous evaluation of CodeEvolve on a subset of the mathematical benchmarks used to evaluate Google DeepMind’s closed-source AlphaEvolve. Our findings show that our method surpasses AlphaEvolve’s performance on several challenging problems. To foster collaboration and accelerate progress, we release our complete framework as an open-source repository.

Method: Methodology details unavailable due to access restrictions

Result: Results cannot be retrieved due to server rate limiting

Conclusion: Analysis impossible - paper content inaccessible

Abstract: Failed to fetch summary for 2510.18318: Page request resulted in HTTP 429 (https://export.arxiv.org/api/query?search_query=&id_list=2510.18318&sortBy=relevance&sortOrder=descending&start=0&max_results=100)

Method: Uses specialized LLM-driven agents embedded as graph nodes with a workflow manager executing a loss-driven loop (code synthesis -> execution -> evaluation -> code repair) and performs Textual-Gradient Descent optimization.

Result: Achieves state-of-the-art overall accuracy across three CPS tasks: User Modeling, Mask Adoption, and Personal Mobility.

Conclusion: SOCIA-Nabla successfully unifies multi-agent orchestration with loss-aligned optimization to create reproducible simulator code generation that scales across domains and granularities, converting brittle prompt pipelines into robust systems.

Abstract: In this paper, we present SOCIA-Nabla, an end-to-end, agentic framework that treats simulator construction asinstance optimization over code within a textual computation graph. Specialized LLM-driven agents are embedded as graph nodes, and a workflow manager executes a loss-driven loop: code synthesis -> execution -> evaluation -> code repair. The optimizer performs Textual-Gradient Descent (TGD), while human-in-the-loop interaction is reserved for task-spec confirmation, minimizing expert effort and keeping the code itself as the trainable object. Across three CPS tasks, i.e., User Modeling, Mask Adoption, and Personal Mobility, SOCIA-Nabla attains state-of-the-art overall accuracy. By unifying multi-agent orchestration with a loss-aligned optimization view, SOCIA-Nabla converts brittle prompt pipelines into reproducible, constraint-aware simulator code generation that scales across domains and simulation granularities. This work is under review, and we will release the code soon.

Method: Uses progressive graph distillation (RL-based tool sequence optimization) and structure-aware test-time adaptation (spectral properties + lightweight adapters) for dynamic tool selection.

Result: Significantly outperforms prior methods, enabling scalable and adaptive LLM-driven graph analysis across diverse graph topologies.

Conclusion: GraphChain provides an effective framework for LLM-based graph analysis through dynamic tool orchestration and topology-aware adaptation, overcoming key limitations of current approaches.

Abstract: Large Language Models (LLMs) face significant limitations when applied to large-scale graphs, struggling with context constraints and inflexible reasoning. We present GraphChain, a framework that enables LLMs to analyze complex graphs through dynamic sequences of specialized tools, mimicking human exploratory intelligence. Our approach introduces two key innovations: (1) Progressive Graph Distillation, a reinforcement learning mechanism that generates optimized tool sequences balancing task relevance with information compression, and (2) Structure-aware Test-Time Adaptation, which efficiently tailors tool selection strategies to diverse graph topologies using spectral properties and lightweight adapters without costly retraining. Experiments show GraphChain significantly outperforms prior methods, enabling scalable and adaptive LLM-driven graph analysis.

Method: Two-stage framework: first predicts user preference profile from reference images, then provides interpretable assessments. Uses CoT-style dataset, supervised fine-tuning followed by reinforcement learning with similarity-aware prediction reward.

Result: Extensive experiments demonstrate superiority of the proposed method over existing approaches.

Conclusion: The framework effectively handles personalized image preference assessment by leveraging common preference profiles and structured reasoning, achieving better performance through interpretable multi-dimensional scoring.

Abstract: Personalized image preference assessment aims to evaluate an individual user’s image preferences by relying only on a small set of reference images as prior information. Existing methods mainly focus on general preference assessment, training models with large-scale data to tackle well-defined tasks such as text-image alignment. However, these approaches struggle to handle personalized preference because user-specific data are scarce and not easily scalable, and individual tastes are often diverse and complex. To overcome these challenges, we introduce a common preference profile that serves as a bridge across users, allowing large-scale user data to be leveraged for training profile prediction and capturing complex personalized preferences. Building on this idea, we propose a reasoning-based personalized image preference assessment framework that follows a \textit{predict-then-assess} paradigm: it first predicts a user’s preference profile from reference images, and then provides interpretable, multi-dimensional scores and assessments of candidate images based on the predicted profile. To support this, we first construct a large-scale Chain-of-Thought (CoT)-style personalized assessment dataset annotated with diverse user preference profiles and high-quality CoT-style reasoning, enabling explicit supervision of structured reasoning. Next, we adopt a two-stage training strategy: a cold-start supervised fine-tuning phase to empower the model with structured reasoning capabilities, followed by reinforcement learning to incentivize the model to explore more reasonable assessment paths and enhance generalization. Furthermore, we propose a similarity-aware prediction reward to encourage better prediction of the user’s preference profile, which facilitates more reasonable assessments exploration. Extensive experiments demonstrate the superiority of the proposed method.

Method: Uses synthetic mazes with controlled difficulty for multi-step spatial reasoning, training VLMs with Reinforcement Learning with Verified Rewards (RLVR) in a difficulty-aware curriculum.

Result: Post-RLVR training achieved over 50% accuracy on problems where base model scored 0%, with zero-shot improvements of 16% on MapBench and 24% on ReasonMap for real-world spatial reasoning tasks.

Conclusion: RL post-training with verified rewards can expand VLMs’ initial capability boundaries and enhance generalization to real-world spatial reasoning, though limited to post-training phase due to pre-training data opaqueness.

Abstract: While Vision-Language Models (VLMs) post-trained with Reinforcement Learning (RL) show impressive general reasoning, their evaluation is often confined to language-dominant tasks (e.g., math). This raises a critical question: can RL post-training truly extend the inherent capability boundary of a base VLM, particularly for visual-centric spatial tasks where it initially fails? To investigate this, we introduce Ariadne, a framework utilizing synthetic mazes for multi-step spatial reasoning where task difficulty (e.g., path length, turns) is precisely controlled. We leverage this controllable environment to train VLMs using Reinforcement Learning with Verified Rewards (RLVR) in a difficulty-aware curriculum. Surprisingly, post-RLVR training, the VLM achieves over 50% accuracy on a problem set where the base model scored 0%, demonstrating that our approach expands the model’s initial capability boundary. To assess real-world viability, we evaluate out-of-distribution (OOD) generalization on practical benchmarks. Despite training only on synthetic maze samples, Ariadne achieves significant zero-shot improvements, averaging 16% on MapBench (e.g., museum navigation) and 24% on ReasonMap (subway transfer tasks). These results confirm that our method not only broadens the model’s fundamental limits but also enhances its generalization to real-world spatial reasoning. We acknowledge our study is limited to the post-training phase, given the opaqueness of pre-training data, and hope our research motivates further work on specialized, capability-extending alignment.

Method: Developed SnapStream, a KV cache compression method that can be deployed at scale in systems with static graphs and continuous batching. Tested on Llama-3.1-8B-Instruct and DeepSeek-R1, and deployed in a 16-way tensor-parallel deployment of DeepSeek-671B on SambaNova SN40L accelerators.

Result: Achieved 4× improved on-chip memory usage at 128k context length with up to 1832 tokens per second in production. Showed minimal accuracy degradation on LongBench-v2, AIME24 and LiveCodeBench benchmarks.

Conclusion: SnapStream successfully enables efficient KV cache compression in production inference systems, representing the first implementation of sparse KV attention techniques deployed with static graphs and continuous batching.

Abstract: The proliferation of 100B+ parameter Large Language Models (LLMs) with 100k+ context length support have resulted in increasing demands for on-chip memory to support large KV caches. Techniques such as StreamingLLM and SnapKV demonstrate how to control KV cache size while maintaining model accuracy. Yet, these techniques are not commonly used within industrial deployments using frameworks like vLLM or SGLang. The reason is twofold: on one hand, the static graphs and continuous batching methodology employed by these frameworks make it difficult to admit modifications to the standard multi-head attention algorithm, while on the other hand, the accuracy implications of such techniques on modern instruction-following and reasoning models are not well understood, obfuscating the need for implementing these techniques. In this paper, we explore these accuracy implications on Llama-3.1-8B-Instruct and DeepSeek-R1, and develop SnapStream, a KV cache compression method that can be deployed at scale. We demonstrate the efficacy of SnapStream in a 16-way tensor-parallel deployment of DeepSeek-671B on SambaNova SN40L accelerators running at 128k context length and up to 1832 tokens per second in a real production setting. SnapStream enables $4\times$ improved on-chip memory usage and introduces minimal accuracy degradation on LongBench-v2, AIME24 and LiveCodeBench. To the best of our knowledge, this is the first implementation of sparse KV attention techniques deployed in a production inference system with static graphs and continuous batching.

Method: Input-level: supervised fine-tuning-based safety classification with 4-tier taxonomy; Output-level: RAG with fine-tuned interpretation model for trustworthy knowledge grounding.

Result: 99.3% risk recall rate, significantly higher safety scores than baseline, and 100% safety score on proprietary high-risk test set.

Conclusion: Provides an effective engineering pathway for building high-security, high-trust LLM applications with exceptional protective capabilities.

Abstract: With the widespread application of Large Language Models (LLMs), their associated security issues have become increasingly prominent, severely constraining their trustworthy deployment in critical domains. This paper proposes a novel safety response framework designed to systematically safeguard LLMs at both the input and output levels. At the input level, the framework employs a supervised fine-tuning-based safety classification model. Through a fine-grained four-tier taxonomy (Safe, Unsafe, Conditionally Safe, Focused Attention), it performs precise risk identification and differentiated handling of user queries, significantly enhancing risk coverage and business scenario adaptability, and achieving a risk recall rate of 99.3%. At the output level, the framework integrates Retrieval-Augmented Generation (RAG) with a specifically fine-tuned interpretation model, ensuring all responses are grounded in a real-time, trustworthy knowledge base. This approach eliminates information fabrication and enables result traceability. Experimental results demonstrate that our proposed safety control model achieves a significantly higher safety score on public safety evaluation benchmarks compared to the baseline model, TinyR1-Safety-8B. Furthermore, on our proprietary high-risk test set, the framework’s components attained a perfect 100% safety score, validating their exceptional protective capabilities in complex risk scenarios. This research provides an effective engineering pathway for building high-security, high-trust LLM applications.

Method: Uses reasoning-based experience model for state transitions and feedback, experience replay buffer initialized with offline data, and adaptive task generation for curriculum learning.

Result: Outperforms baselines by over 30% on non-RL-ready tasks like WebArena, matches GRPO and PPO performance using only synthetic interactions, and provides significant performance gains in sim-to-real transfer with fewer real-world interactions.

Conclusion: DreamGym enables scalable RL training through synthetic experience synthesis, providing an effective warm-start strategy for general-purpose RL with reduced reliance on costly real-environment interactions.

Abstract: While reinforcement learning (RL) can empower autonomous agents by enabling self-improvement through interaction, its practical adoption remains challenging due to costly rollouts, limited task diversity, unreliable reward signals, and infrastructure complexity, all of which obstruct the collection of scalable experience data. To address these challenges, we introduce DreamGym, the first unified framework designed to synthesize diverse experiences with scalability in mind to enable effective online RL training for autonomous agents. Rather than relying on expensive real-environment rollouts, DreamGym distills environment dynamics into a reasoning-based experience model that derives consistent state transitions and feedback signals through step-by-step reasoning, enabling scalable agent rollout collection for RL. To improve the stability and quality of transitions, DreamGym leverages an experience replay buffer initialized with offline real-world data and continuously enriched with fresh interactions to actively support agent training. To improve knowledge acquisition, DreamGym adaptively generates new tasks that challenge the current agent policy, enabling more effective online curriculum learning. Experiments across diverse environments and agent backbones demonstrate that DreamGym substantially improves RL training, both in fully synthetic settings and in sim-to-real transfer scenarios. On non-RL-ready tasks like WebArena, DreamGym outperforms all baselines by over 30%. And in RL-ready but costly settings, it matches GRPO and PPO performance using only synthetic interactions. When transferring a policy trained purely on synthetic experiences to real-environment RL, DreamGym yields significant additional performance gains while requiring far fewer real-world interactions, providing a scalable warm-start strategy for general-purpose RL.

Method: Combined linear probing with partial correlation analysis and prompt-based vulnerability tests across models of varying sizes.

Result: LLMs encode real-world numerical correlations but tend to systematically amplify them. Irrelevant context induces consistent shifts in magnitude representations with downstream effects that vary by model size.

Conclusion: Reveals vulnerability in LLM decision-making and provides groundwork for fairer, representation-aware control under multi-attribute entanglement.

Abstract: Although behavioral studies have documented numerical reasoning errors in large language models (LLMs), the underlying representational mechanisms remain unclear. We hypothesize that numerical attributes occupy shared latent subspaces and investigate two questions:(1) How do LLMs internally integrate multiple numerical attributes of a single entity? (2)How does irrelevant numerical context perturb these representations and their downstream outputs? To address these questions, we combine linear probing with partial correlation analysis and prompt-based vulnerability tests across models of varying sizes. Our results show that LLMs encode real-world numerical correlations but tend to systematically amplify them. Moreover, irrelevant context induces consistent shifts in magnitude representations, with downstream effects that vary by model size. These findings reveal a vulnerability in LLM decision-making and lay the groundwork for fairer, representation-aware control under multi-attribute entanglement.

Method: Methodology details unavailable due to failed API request.

Result: Results cannot be analyzed without access to the paper’s content.

Conclusion: No conclusion can be drawn as the paper information could not be retrieved from the arXiv API.

Abstract: Failed to fetch summary for 2511.04076: Page request resulted in HTTP 429 (https://export.arxiv.org/api/query?search_query=&id_list=2511.04076&sortBy=relevance&sortOrder=descending&start=0&max_results=100)

Method: Uses an LLM-augmented automated pipeline for query sourcing, environment-template construction, task instantiation, batched execution, and LLM-driven quality filtering. Contains 1.2M+ action steps across Windows office applications with screenshots, accessibility metadata, goals, reasoning traces, and both successful/failed trajectories.

Result: Benchmarking shows substantial shortcomings in state-of-the-art vision-language models for grounding and action prediction. Supervised fine-tuning and reinforcement learning yield significant improvements but don’t reach human-level reliability.

Conclusion: GUI-360° provides a comprehensive dataset and benchmark to accelerate research on robust desktop computer-using agents, with the dataset publicly released to facilitate reproducible research.

Abstract: We introduce GUI-360$^\circ$, a large-scale, comprehensive dataset and benchmark suite designed to advance computer-using agents (CUAs). CUAs present unique challenges and is constrained by three persistent gaps: a scarcity of real-world CUA tasks, the lack of automated collection-and-annotation pipelines for multi-modal trajectories, and the absence of a unified benchmark that jointly evaluates GUI grounding, screen parsing, and action prediction. GUI-360$^\circ$ addresses these gaps with an LLM-augmented, largely automated pipeline for query sourcing, environment-template construction, task instantiation, batched execution, and LLM-driven quality filtering. The released corpus contains over 1.2M executed action steps across thousands of trajectories in popular Windows office applications, and includes full-resolution screenshots, accessibility metadata when available, instantiated goals, intermediate reasoning traces, and both successful and failed action trajectories. The dataset supports three canonical tasks, GUI grounding, screen parsing, and action prediction, and a hybrid GUI+API action space that reflects modern agent designs. Benchmarking state-of-the-art vision–language models on GUI-360$^\circ$ reveals substantial out-of-the-box shortcomings in grounding and action prediction; supervised fine-tuning and reinforcement learning yield significant gains but do not close the gap to human-level reliability. We release GUI-360$^\circ$ and accompanying code to facilitate reproducible research and accelerate progress on robust desktop CUAs. The full dataset has been made public on https://huggingface.co/datasets/vyokky/GUI-360.

Method: N/A - Abstract not available

Result: N/A - Abstract not available

Conclusion: N/A - Abstract not available

Abstract: Failed to fetch summary for 2408.13850: Page request resulted in HTTP 429 (https://export.arxiv.org/api/query?search_query=&id_list=2408.13850&sortBy=relevance&sortOrder=descending&start=0&max_results=100)

Method: N/A - Paper content unavailable

Result: N/A - Paper content unavailable

Conclusion: N/A - Paper content unavailable

Abstract: Failed to fetch summary for 2409.09684: Page request resulted in HTTP 429 (https://export.arxiv.org/api/query?search_query=&id_list=2409.09684&sortBy=relevance&sortOrder=descending&start=0&max_results=100)

Method: Attempted to fetch paper metadata from arXiv API but encountered server error

Result: No results obtained - service temporarily unavailable

Conclusion: Technical limitations prevented analysis of paper 2412.12987

Abstract: Failed to fetch summary for 2412.12987: Page request resulted in HTTP 503 (https://export.arxiv.org/api/query?search_query=&id_list=2412.12987&sortBy=relevance&sortOrder=descending&start=0&max_results=100)

Method: Leverages equal-loudness contours to assign frequency-dependent weights to reconstruction error, penalizing deviations according to human auditory sensitivity. The loss is model-agnostic and flexible.

Result: Experiments on VoiceBank+DEMAND dataset show replacing MSE with the proposed loss in GTCRN model elevates WB-PESQ score from 2.17 to 2.93, indicating significant perceptual quality improvement.

Conclusion: The perceptually-weighted loss function grounded in psychoacoustic principles effectively addresses MSE’s limitations and significantly enhances speech enhancement performance in terms of perceptual quality.

Abstract: The mean squared error (MSE) is a ubiquitous loss function for speech enhancement, but its problem is that the error cannot reflect the auditory perception quality. This is because MSE causes models to over-emphasize low-frequency components which has high energy, leading to the inadequate modeling of perceptually important high-frequency information. To overcome this limitation, we propose a perceptually-weighted loss function grounded in psychoacoustic principles. Specifically, it leverages equal-loudness contours to assign frequency-dependent weights to the reconstruction error, thereby penalizing deviations in a way aligning with human auditory sensitivity. The proposed loss is model-agnostic and flexible, demonstrating strong generality. Experiments on the VoiceBank+DEMAND dataset show that replacing MSE with our loss in a GTCRN model elevates the WB-PESQ score from 2.17 to 2.93-a significant improvement in perceptual quality.

Method: Uses narrowband mmWave signals to capture vibrations, reveals optimal vibrating material and radar sampling rate, designs deep neural network for speech reconstruction, and fine-tunes pre-trained ASR model encoder with synthetic training pipeline.

Result: Achieves state-of-the-art speech quality and generalizes well across unseen speakers and various conditions using commercial mmWave radar.

Conclusion: mmSpeech demonstrates effective speech reconstruction from loudspeaker vibrations through walls, highlighting significant privacy concerns in voice-enabled technologies.

Abstract: With the rise of voice-enabled technologies, loudspeaker playback has become widespread, posing increasing risks to speech privacy. Traditional eavesdropping methods often require invasive access or line-of-sight, limiting their practicality. In this paper, we present mmSpeech, an end-to-end mmWave-based eavesdropping system that reconstructs intelligible speech solely from vibration signals induced by loudspeaker playback, even through walls and without prior knowledge of the speaker. To achieve this, we reveal an optimal combination of vibrating material and radar sampling rate for capturing high- quality vibrations using narrowband mmWave signals. We then design a deep neural network that reconstructs intelligible speech from the estimated noisy spectrograms. To further support downstream speech understanding, we introduce a synthetic training pipeline and selectively fine-tune the encoder of a pre-trained ASR model. We implement mmSpeech with a commercial mmWave radar and validate its performance through extensive experiments. Results show that mmSpeech achieves state-of-the-art speech quality and generalizes well across unseen speakers and various conditions.

Method: Proposes three linguistic guidance strategies: output linguistic constraints, intermediate linguistic prediction, and input linguistic prior, using LLMs like RoBERTa, Qwen3-0.6B, and Qwen3-4B.

Result: Significant improvements in challenging scenarios including visual cue impairment, unseen languages, speaker switches, more interfering speakers, and out-of-domain tests.

Conclusion: Incorporating linguistic knowledge from LLMs effectively enhances AV-TSE performance, especially in difficult conditions where visual cues alone are insufficient.

Abstract: Audio-visual target speaker extraction (AV-TSE) models primarily rely on visual cues from the target speaker. However, humans also leverage linguistic knowledge, such as syntactic constraints, next word prediction, and prior knowledge of conversation, to extract target speech. Inspired by this observation, we propose ELEGANCE, a novel framework that incorporates linguistic knowledge from large language models (LLMs) into AV-TSE models through three distinct guidance strategies: output linguistic constraints, intermediate linguistic prediction, and input linguistic prior. Comprehensive experiments with RoBERTa, Qwen3-0.6B, and Qwen3-4B on two AV-TSE backbones demon- strate the effectiveness of our approach. Significant improvements are observed in challenging scenarios, including visual cue impaired, unseen languages, target speaker switches, increased interfering speakers, and out-of-domain test set. Demo page: https://alexwxwu.github.io/ELEGANCE/.

Method: Operates in latent domain to handle variable RIR characteristics, jointly optimizing with perceptual loss for RIR reconstruction and loss for watermark detection.

Result: Achieves room acoustic parameter matching comparable to state-of-the-art FiNS, MOS of 4.22/5, watermark detection accuracy >99%, and bit error rates <0.3%.

Conclusion: EchoMark effectively balances high-quality environment transfer with reliable watermark embedding, providing a secure solution for acoustic environment matching applications.

Abstract: Acoustic Environment Matching (AEM) is the task of transferring clean audio into a target acoustic environment, enabling engaging applications such as audio dubbing and auditory immersive virtual reality (VR). Recovering similar room impulse response (RIR) directly from reverberant speech offers more accessible and flexible AEM solution. However, this capability also introduces vulnerabilities of arbitrary ``relocation" if misused by malicious user, such as facilitating advanced voice spoofing attacks or undermining the authenticity of recorded evidence. To address this issue, we propose EchoMark, the first deep learning-based AEM framework that generates perceptually similar RIRs with embedded watermark. Our design tackle the challenges posed by variable RIR characteristics, such as different durations and energy decays, by operating in the latent domain. By jointly optimizing the model with a perceptual loss for RIR reconstruction and a loss for watermark detection, EchoMark achieves both high-quality environment transfer and reliable watermark recovery. Experiments on diverse datasets validate that EchoMark achieves room acoustic parameter matching performance comparable to FiNS, the state-of-the-art RIR estimator. Furthermore, a high Mean Opinion Score (MOS) of 4.22 out of 5, watermark detection accuracy exceeding 99%, and bit error rates (BER) below 0.3% collectively demonstrate the effectiveness of EchoMark in preserving perceptual quality while ensuring reliable watermark embedding.

Method: Propose Mix-Training HuBERT (MT-HuBERT), a self-supervised pre-training framework that implements the Mix-Training criterion during pre-training to predict clean acoustic units of each constituent signal from contextual cues, rather than predicting compositional patterns of mixed speech.

Result: Experiments on Google Speech Commands (GSC v2) corpus show MT-HuBERT consistently outperforms state-of-the-art baselines in few-shot keyword spotting tasks under both mixed and clean conditions.

Conclusion: MT-HuBERT effectively addresses mixed keyword detection in few-shot scenarios through self-supervised pre-training, demonstrating superior performance compared to existing approaches while leveraging unlabeled data.

Abstract: Few-shot keyword spotting aims to detect previously unseen keywords with very limited labeled samples. A pre-training and adaptation paradigm is typically adopted for this task. While effective in clean conditions, most existing approaches struggle with mixed keyword spotting–detecting multiple overlapping keywords within a single utterance–a capability essential for real-world applications. We have previously proposed a pre-training approach based on Mix-Training (MT) to tackle the mixed keyword detection problem and demonstrated its efficiency. However, this approach is fully supervised, unable to utilize vast unlabeled data. To this end, we propose Mix-Training HuBERT (MT-HuBERT), a self-supervised learning (SSL) pre-training framework that implements the MT criterion during pre-training. MT-HuBERT predicts, in a self-supervised manner, the clean acoustic units of each constituent signal from contextual cues, in contrast to predicting compositional patterns of mixed speech. Experiments conducted on the Google Speech Commands (GSC v2) corpus demonstrate that our proposed MT-HuBERT consistently outperforms several state-of-the-art baselines in few-shot KWS tasks under both mixed and clean conditions.

Method: Converts audio into structured symbolic features across speech, sound events, and music domains, creating human-readable inputs that support reasoning and transparent error analysis.

Result: Achieves competitive performance across three benchmarks (MMAU, MMAR, OmniBench) while enabling traceable error analysis to specific features.

Conclusion: SAR-LM provides an interpretable alternative to dense embedding approaches, prioritizing transparency in audio reasoning while maintaining competitive performance.

Abstract: Large language models (LLMs) have advanced in text and vision, but their reasoning on audio remains limited. Most existing methods rely on dense audio embeddings, which are difficult to interpret and often fail on structured reasoning tasks. Caption-based approaches, introduced in recent benchmarks such as MMAU, improve performance by translating audio into text, yet still depend on dense embeddings as input, offering little insight when models fail. We present SAR-LM, a symbolic audio reasoning pipeline that builds on this caption-based paradigm by converting audio into structured, human-readable features across speech, sound events, and music. These symbolic inputs support both reasoning and transparent error analysis, enabling us to trace failures to specific features. Across three benchmarks, MMAU, MMAR, and OmniBench, SAR-LM achieves competitive results, while prioritizing interpretability as its primary contribution.

Method: The authors compare CA-SDR with classical SDR, analyze its limitations, propose a modified CA-SDR that focuses on class-agnostic SDR first and then accounts for mislabeled sources, and introduce penalties for labeling and separation errors.

Result: The analysis reveals cases where CA-SDR doesn’t allow proper system comparison, and the proposed modifications address these issues by better handling classification errors and cross-contamination between separated sources.

Conclusion: The paper presents an improved evaluation metric for spatial semantic segmentation that more accurately reflects both separation and classification performance through modified CA-SDR with appropriate penalties.

Abstract: Spatial semantic segmentation of sound scenes (S5) consists of jointly performing audio source separation and sound event classification from a multichannel audio mixture. To evaluate S5 systems, one can consider two individual metrics, i.e., one for source separation and another for sound event classification, but this approach makes it challenging to compare S5 systems. Thus, a joint class-aware signal-to-distortion ratio (CA-SDR) metric was proposed to evaluate S5 systems. In this work, we first compare the CA-SDR with the classical SDR on scenarios with only classification errors. We then analyze the cases where the metric might not allow proper comparison of the systems. To address this problem, we propose a modified version of the CA-SDR which first focuses on class-agnostic SDR and then accounts for the wrongly labeled sources. We also analyze the performance of the two metrics under cross-contamination between separated audio sources. Finally, we propose a first set of penalties in an attempt to make the metric more reflective of the labeling and separation errors.

Method: Deep hierarchical variational autoencoder to model speaker timbre space, generating novel speaker representations by sampling from the trained model as a plug-in module for VC pipelines.

Result: Successfully generates novel, unseen speakers with quality comparable to training speakers when evaluated with FACodec and CosyVoice2 VC models.

Conclusion: SpeakerVAE provides a flexible solution for generating novel speaker voices in voice conversion without requiring target data or modifying base VC systems.

Abstract: Voice conversion models modify timbre while preserving paralinguistic features, enabling applications like dubbing and identity protection. However, most VC systems require access to target utterances, limiting their use when target data is unavailable or when users desire conversion to entirely novel, unseen voices. To address this, we introduce a lightweight method SpeakerVAE to generate novel speakers for VC. Our approach uses a deep hierarchical variational autoencoder to model the speaker timbre space. By sampling from the trained model, we generate novel speaker representations for voice synthesis in a VC pipeline. The proposed method is a flexible plug-in module compatible with various VC models, without co-training or fine-tuning of the base VC system. We evaluated our approach with state-of-the-art VC models: FACodec and CosyVoice2. The results demonstrate that our method successfully generates novel, unseen speakers with quality comparable to that of the training speakers.

Method: The framework employs encoder ensemble with feature extractor for timbre protection, ASR-targeted adversarial examples to disrupt pronunciation, and incorporates psychoacoustic model to ensure imperceptible perturbations.

Result: Comprehensive evaluation across 16 open-source synthesizers and 3 commercial APIs on Chinese and English datasets confirms E2E-VGuard’s effectiveness in protecting both timbre and pronunciation, with real-world deployment validation.

Conclusion: E2E-VGuard successfully addresses the security gaps in defending against modern speech synthesis threats, particularly production LLM-based systems and ASR-driven end-to-end voice cloning attacks.

Abstract: Recent advancements in speech synthesis technology have enriched our daily lives, with high-quality and human-like audio widely adopted across real-world applications. However, malicious exploitation like voice-cloning fraud poses severe security risks. Existing defense techniques struggle to address the production large language model (LLM)-based speech synthesis. While previous studies have considered the protection for fine-tuning synthesizers, they assume manually annotated transcripts. Given the labor intensity of manual annotation, end-to-end (E2E) systems leveraging automatic speech recognition (ASR) to generate transcripts are becoming increasingly prevalent, e.g., voice cloning via commercial APIs. Therefore, this E2E speech synthesis also requires new security mechanisms. To tackle these challenges, we propose E2E-VGuard, a proactive defense framework for two emerging threats: (1) production LLM-based speech synthesis, and (2) the novel attack arising from ASR-driven E2E scenarios. Specifically, we employ the encoder ensemble with a feature extractor to protect timbre, while ASR-targeted adversarial examples disrupt pronunciation. Moreover, we incorporate the psychoacoustic model to ensure perturbative imperceptibility. For a comprehensive evaluation, we test 16 open-source synthesizers and 3 commercial APIs across Chinese and English datasets, confirming E2E-VGuard’s effectiveness in timbre and pronunciation protection. Real-world deployment validation is also conducted. Our code and demo page are available at https://wxzyd123.github.io/e2e-vguard/.

Method: Uses Schrodinger bridge framework with RSS and target spectrum as endpoints, subband-aware convolutional diffusion network with uneven subband division and large-kernel attention for T-F modeling, and omnidirectional distillation for single-step inference.

Result: Achieves SOTA performance on various benchmarks with fewer parameters, lower computational cost, and competitive inference speed using only 4 sampling steps, maintaining superiority even with single-step inference.

Conclusion: BridgeVoC successfully demonstrates that vocoder tasks can be effectively treated as audio restoration problems, achieving high-quality synthesis with efficient sampling through novel diffusion modeling and distillation techniques.

Abstract: This paper revisits the neural vocoder task through the lens of audio restoration and propose a novel diffusion vocoder called BridgeVoC. Specifically, by rank analysis, we compare the rank characteristics of Mel-spectrum with other common acoustic degradation factors, and cast the vocoder task as a specialized case of audio restoration, where the range-space spectral (RSS) surrogate of the target spectrum acts as the degraded input. Based on that, we introduce the Schrodinger bridge framework for diffusion modeling, which defines the RSS and target spectrum as dual endpoints of the stochastic generation trajectory. Further, to fully utilize the hierarchical prior of subbands in the time-frequency (T-F) domain, we elaborately devise a novel subband-aware convolutional diffusion network as the data predictor, where subbands are divided following an uneven strategy, and convolutional-style attention module is employed with large kernels for efficient T-F contextual modeling. To enable single-step inference, we propose an omnidirectional distillation loss to facilitate effective information transfer from the teacher model to the student model, and the performance is improved by combining target-related and bijective consistency losses. Comprehensive experiments are conducted on various benchmarks and out-of-distribution datasets. Quantitative and qualitative results show that while enjoying fewer parameters, lower computational cost, and competitive inference speed, the proposed BridgeVoC yields stateof-the-art performance over existing advanced GAN-, DDPMand flow-matching-based baselines with only 4 sampling steps. And consistent superiority is still achieved with single-step inference.

Method: Systematic comparison of different datasets, model architectures, model sizes, and training strategies; evaluation using quantitative metrics correlated with human judgment from listening studies.

Result: Best-performing model is a 950M-parameter transformer trained on 80K MIDI files from diverse genres, producing outputs often rated as human-composed in Turing-style listening surveys.

Conclusion: Comprehensive analysis provides insights into effective design choices for symbolic music generation, with large-scale transformers trained on diverse datasets achieving human-level composition quality.

Abstract: Although a variety of transformers have been proposed for symbolic music generation in recent years, there is still little comprehensive study on how specific design choices affect the quality of the generated music. In this work, we systematically compare different datasets, model architectures, model sizes, and training strategies for the task of symbolic piano music generation. To support model development and evaluation, we examine a range of quantitative metrics and analyze how well they correlate with human judgment collected through listening studies. Our best-performing model, a 950M-parameter transformer trained on 80K MIDI files from diverse genres, produces outputs that are often rated as human-composed in a Turing-style listening survey.

Method: The paper traces MIR evolution by reviewing main research achievements along three EDICS (music analysis, processing, and generation) and analyzing successful practices including annual benchmarks (MIREX), reproducible research, industry engagement, and DEI commitments.

Result: The analysis identifies key factors for MIR’s success: annual research benchmarks, pursuit of reproducible/open research, active industry engagement, and commitment to diversity and inclusion, which have created a vibrant research community.

Conclusion: MIR has evolved significantly over 25 years through successful practices that foster rapid development, though the field still faces future challenges that need to be addressed.

Abstract: In this paper, we trace the evolution of Music Information Retrieval (MIR) over the past 25 years. While MIR gathers all kinds of research related to music informatics, a large part of it focuses on signal processing techniques for music data, fostering a close relationship with the IEEE Audio and Acoustic Signal Processing Technical Commitee. In this paper, we reflect the main research achievements of MIR along the three EDICS related to music analysis, processing and generation. We then review a set of successful practices that fuel the rapid development of MIR research. One practice is the annual research benchmark, the Music Information Retrieval Evaluation eXchange, where participants compete on a set of research tasks. Another practice is the pursuit of reproducible and open research. The active engagement with industry research and products is another key factor for achieving large societal impacts and motivating younger generations of students to join the field. Last but not the least, the commitment to diversity, equity and inclusion ensures MIR to be a vibrant and open community where various ideas, methodologies, and career pathways collide. We finish by providing future challenges MIR will have to face.

Method: Developed AcousTools as a Python library that supports the full suite of acoustic holographic applications, including setup, acoustic propagation modeling, transducer phase retrieval, sound field analysis, and hardware control.

Result: AcousTools successfully meets each step of the full-stack requirements for acoustic holography and has the potential to become the standard library in this field.

Conclusion: AcousTools provides a uniquely complete and easy-to-use solution that will enable researchers to develop novel acoustic holography applications and facilitate accurate review of others’ work.

Abstract: Acoustic Holography is an emerging field where mid-air ultrasound is controlled and manipulated for novel and exciting applications. These range from mid-air haptics, volumetric displays, contactless fabrication, and even chemical and biomedical applications such as drug delivery. To develop these applications, a software framework to predict acoustic behaviour and simulating resulting effects, such as applied forces or scattering patterns is desirable. There have been various software libraries and platforms that attempt to fill this role, but there is yet to be a single piece of software that acts as a ‘full-stack’ solution. We define this full-stack as the process from abstraction to physicalisation starting with setup, modelling acoustic propagation, transducer phase retrieval, sound field analysis, and control of the acoustic holographic hardware itself. Existing methods fail to fulfil one or more of these categories. To address this, we present AcousTools, a Python-based acoustic holography library, designed to support the full suite of acoustic holographic applications and we show AcousTools’s ability to meet each step of the full-stack’s requirements. AcousTools has the potential to become the standard code library for acoustic holography, with the uniquely complete suite of features wrapped in a language that is known to be easy to use, AcousTools will increase the ability for researchers to develop novel applications as well as accurately review other’s work. The full-stack, aside from software, will also be useful for researchers - providing a way to view and compare methodologies by understanding where they fit into the stack.

Method: Uses text-based environment embeddings from noise descriptions via pre-trained text encoder, fused with transformer-based SER model. Employs contrastive learning and joint fine-tuning of text encoder with emotion model.

Result: Text-based environment descriptions processed by LLMs improve noise robustness. Fine-tuning text encoder with CL-based representation shows significant improvements: 76.4% (arousal), 100.0% (dominance), 27.7% (valence) at -5dB SNR.

Conclusion: Leveraging text-based environment knowledge through joint training significantly enhances SER performance in noisy conditions, demonstrating the value of multimodal approaches for noise robustness.

Abstract: Speech emotion recognition (SER) systems often struggle in real-world environments, where ambient noise severely degrades their performance. This paper explores a novel approach that exploits prior knowledge of testing environments to maximize SER performance under noisy conditions. To address this task, we propose a text-guided, environment-aware training where an SER model is trained with contaminated speech samples and their paired noise description. We use a pre-trained text encoder to extract the text-based environment embedding and then fuse it to a transformer-based SER model during training and inference. We demonstrate the effectiveness of our approach through our experiment with the MSP-Podcast corpus and real-world additive noise samples collected from the Freesound and DEMAND repositories. Our experiment indicates that the text-based environment descriptions processed by a large language model (LLM) produce representations that improve the noise-robustness of the SER system. With a contrastive learning (CL)-based representation, our proposed method can be improved by jointly fine-tuning the text encoder with the emotion recognition model. Under the -5dB signal-to-noise ratio (SNR) level, fine-tuning the text encoder improves our CL-based representation method by 76.4% (arousal), 100.0% (dominance), and 27.7% (valence).

Method: Two-stage approach: 1) Weakly supervised audio separation using CLAP model for semantic alignment, with ranking loss for contextual significance; 2) Image generation using trainable adapter and MLP layer to map separated audio signals to generation conditions.

Result: Outperforms current state-of-the-art methods in 17 out of 21 evaluation indexes across multi-source, mixed-source, and single-source tasks, delivering superior visual quality.

Conclusion: MACS successfully bridges the gap in multi-source audio-to-image generation by explicitly separating audio components before generation, establishing a new benchmark for this task.

Abstract: Propelled by the breakthrough in deep generative models, audio-to-image generation has emerged as a pivotal cross-modal task that converts complex auditory signals into rich visual representations. However, previous works only focus on single-source audio inputs for image generation, ignoring the multi-source characteristic in natural auditory scenes, thus limiting the performance in generating comprehensive visual content. To bridge this gap, we propose a method called MACS to conduct multi-source audio-to-image generation. To our best knowledge, this is the first work that explicitly separates multi-source audio to capture the rich audio components before image generation. MACS is a two-stage method. In the first stage, multi-source audio inputs are separated by a weakly supervised method, where the audio and text labels are semantically aligned by casting into a common space using the large pre-trained CLAP model. We introduce a ranking loss to consider the contextual significance of the separated audio signals. In the second stage, effective image generation is achieved by mapping the separated audio signals to the generation condition using only a trainable adapter and a MLP layer. We preprocess the LLP dataset as the first full multi-source audio-to-image generation benchmark. The experiments are conducted on multi-source, mixed-source, and single-source audio-to-image generation tasks. The proposed MACS outperforms the current state-of-the-art methods in 17 out of the 21 evaluation indexes on all tasks and delivers superior visual quality.

Method: Proposed GRAM: a General-purpose Real-world Audio Model using multi-channel masked auto-encoder approach to learn spatial audio representations from high-quality simulated real-world scenes.

Result: GRAM surpasses all state-of-the-art self-supervised audio foundation models and speech models on both HEAR and Nat-HEAR benchmarks, achieves state-of-the-art localization performance (even beating supervised approaches), and works with both binaural and Ambisonics formats.

Conclusion: GRAM represents a significant advancement towards robust, spatial audio foundation models for real-world applications, demonstrating effective transfer to real-world recordings.

Abstract: Although audio foundations models have seen great progress on a wide variety of tasks, their application in real-world acoustic environments with reverberation and noise has been less successful. Moreover, as audio foundation models are typically trained on dry, single-channel audio clips, the inherent spatial nature of real-world sound scenes is overlooked and tasks involving sound localization ruled out. To address these limitations, we propose GRAM: a General-purpose Real-world Audio Model utilizing a multi-channel masked auto-encoder approach to efficiently learn spatial audio representations from high-quality simulated real-world scenes. To evaluate the performance of GRAM and other audio foundation models in real-world sound scenes, we release Nat-HEAR: A naturalistic version of the HEAR benchmark suite comprising a simulated real-world version, as well as two new sound localization tasks. We show that the performance of GRAM surpasses all state-of-the-art self-supervised audio foundation models and speech models on both HEAR and Nat-HEAR, while using only a fraction of the training data. GRAM also showcases state-of-the-art localization performance, surpassing even supervised sound localization approaches, and can be flexibly applied either to a two-channel, binaural sound format or a four-channel, Ambisonics format. Validating GRAM’s performance on real-world sound recordings demonstrates robust transfer to real-world scenes. Taken together, GRAM presents a significant advancement towards robust, spatial audio foundation models for real-world applications.

Method: Three core components: automated data collection from dark data, utterance-level dynamic selection of data cleansing methods based on training quality, and evaluation-in-the-loop selection using predicted MOS scores to estimate utterance impact. Jointly optimizes corpus and TTS model in closed-loop.

Result: Extensive experiments on Japanese YouTube data show TTSOps outperforms conventional acoustic-quality-based baselines in both naturalness and speaker diversity of synthesized speech.

Conclusion: TTSOps successfully demonstrates that automated construction of multi-speaker TTS systems from noisy web-scale data is feasible and superior to traditional curated data approaches.

Abstract: This paper presents TTSOps, a fully automated closed-loop framework for constructing multi-speaker text-to-speech (TTS) systems from noisy, uncurated web-scale speech data, often referred to as ``dark data,’’ such as online videos. Conventional TTS training pipelines require well-curated corpora with high acoustic quality and accurate text-speech alignment, which severely limits scalability, speaker diversity, and real-world applicability. While recent studies have proposed acoustic-quality-based data selection techniques, they often overlook two critical aspects: (1) the inherent robustness of modern TTS models to noise, and (2) the potential contribution of perceptually low-quality yet informative samples. To address these issues, TTSOps introduces a data-centric training pipeline that integrates three core components: (1) automated data collection from dark data sources, (2) utterance-level dynamic selection of data cleansing methods based on training data quality, and (3) evaluation-in-the-loop data selection using automatically predicted mean opinion scores (MOS) to estimate each utterance’s impact on model performance. Furthermore, TTSOps jointly optimizes the corpus and the TTS model in a closed-loop framework by dynamically adapting both data selection and data cleansing processes to the characteristics of the target TTS model. Extensive experiments on Japanese YouTube data demonstrate that TTSOps outperforms conventional acoustic-quality-based baselines in both the naturalness and speaker diversity of synthesized speech.

Method: Uses a frozen diffusion language model with lightweight dual-adapter architecture, trained in two stages: semantic alignment via ASR objective, then instruction-following using synthetic audio-caption pairs generated by LLMs.

Result: DIFFA achieves competitive performance on MMSU, MMAU, and VoiceBench benchmarks, outperforming several autoregressive open-source baselines despite training on only 960 hours of ASR and 127 hours of synthetic instruction data.

Conclusion: Demonstrates the potential of diffusion-based language models for efficient and scalable audio understanding, opening new directions for speech-driven AI.

Abstract: Recent advances in large language models (LLMs) have shown remarkable capabilities across textual and multimodal domains. In parallel, diffusion-based language models have emerged as a promising alternative to the autoregressive paradigm, offering improved controllability, bidirectional context modeling, and robust generation. However, their application to the audio modality remains underexplored. In this work, we introduce \textbf{DIFFA}, the first diffusion-based large audio-language model designed to perform spoken language understanding. DIFFA integrates a frozen diffusion language model with a lightweight dual-adapter architecture that bridges speech understanding and natural language reasoning. We employ a two-stage training pipeline: first, aligning semantic representations via an ASR objective; then, learning instruction-following abilities through synthetic audio-caption pairs automatically generated by prompting LLMs. Despite being trained on only 960 hours of ASR and 127 hours of synthetic instruction data, DIFFA demonstrates competitive performance on major benchmarks, including MMSU, MMAU, and VoiceBench, outperforming several autoregressive open-source baselines. Our results reveal the potential of diffusion-based language models for efficient and scalable audio understanding, opening a new direction for speech-driven AI. Our code will be available at https://github.com/NKU-HLT/DIFFA.git.

Method: Uses fine-grained facial attribute modeling by decomposing facial images into non-overlapping segments, creates multi-granular representations, employs multi-task learning for speaker attributes, and adopts multi-view training with various visual perspectives.

Result: Extensive evaluations show substantial improvements in face-voice congruence and synthesis stability compared to existing methods.

Conclusion: The proposed approach effectively addresses limitations of existing face-to-voice synthesis methods by preserving fine-grained facial attributes and achieving better voice synthesis quality through robust multi-view training.

Abstract: For individuals who have experienced traumatic events such as strokes, speech may no longer be a viable means of communication. While text-to-speech (TTS) can be used as a communication aid since it generates synthetic speech, it fails to preserve the user’s own voice. As such, face-to-voice (FTV) synthesis, which derives corresponding voices from facial images, provides a promising alternative. However, existing methods rely on pre-trained visual encoders, and finetune them to align with speech embeddings, which strips fine-grained information from facial inputs such as gender or ethnicity, despite their known correlation with vocal traits. Moreover, these pipelines are multi-stage, which requires separate training of multiple components, thus leading to training inefficiency. To address these limitations, we utilize fine-grained facial attribute modeling by decomposing facial images into non-overlapping segments and progressively integrating them into a multi-granular representation. This representation is further refined through multi-task learning of speaker attributes such as gender and ethnicity at both the visual and acoustic domains. Moreover, to improve alignment robustness, we adopt a multi-view training strategy by pairing various visual perspectives of a speaker in terms of different angles and lighting conditions, with identical speech recordings. Extensive subjective and objective evaluations confirm that our approach substantially enhances face-voice congruence and synthesis stability.

Method: Proposes WavJEPA using high-level semantic representation learning instead of speech unit/token level learning. Also presents WavJEPA-Nat as a multi-channel extension trained on simulated naturalistic scenes for robustness.

Result: WavJEPA substantially outperforms state-of-the-art time-domain audio foundation models across various downstream tasks while requiring fewer computational resources. WavJEPA-Nat shows high robustness to reverberation and noise.

Conclusion: Demonstrates feasibility and computational efficiency of general-purpose audio representation learning from raw waveforms, enabling low-latency, robust time-domain audio foundation models for real-world applications.

Abstract: Learning audio representations from raw waveforms overcomes key limitations of spectrogram-based audio representation learning, such as the long latency of spectrogram computation and the loss of phase information. Yet, while self-supervised speech representation learning from raw waveforms has been remarkably successful, these approaches have not achieved similar feats for general-purpose audio representation learning from waveforms. Here, we propose WavJEPA, a waveform-based version of the Joint-Embedding Predictive Architecture. WavJEPA leverages high-level semantic representation learning to tackle the shortcomings of representation learning at the speech unit or token level. We show that this approach substantially outperforms state-of-the-art time-domain audio foundation models across a wide variety of downstream benchmark tasks, while requiring considerably fewer computational resources. Additionally, to overcome the performance drop that time-domain models typically exhibit in noisy and reverberant real-world acoustic environments, we present WavJEPA-Nat. WavJEPA-Nat is a multi-channel extension of the WavJEPA architecture trained on simulated naturalistic scenes. We find that WavJEPA-Nat is highly robust to reverberation and noise. These results highlight the feasibility and computational efficiency of general-purpose audio representation learning from raw waveforms, showcasing the potential for low-latency, robust time-domain audio foundation models for real-world applications.

Method: Uses hybrid objective combining weighted categorical cross-entropy and Concordance Correlation Coefficient (CCC) losses for joint discrete and dimensional emotion modeling, enabling capture of both emotion categories (happy, angry) and dimensions (arousal, valence, dominance).

Result: Extensive evaluations show MERaLiON-SER consistently surpasses both open-source speech encoders and large Audio-LLMs across multilingual Singaporean languages (English, Chinese, Malay, Tamil) and other public benchmarks.

Conclusion: Specialized speech-only models are crucial for accurate paralinguistic understanding and cross-lingual generalization, providing foundation for emotion-aware perception in future agentic audio systems for more empathetic multimodal reasoning.

Abstract: We present MERaLiON-SER, a robust speech emotion recognition model de- signed for English and Southeast Asian languages. The model is trained using a hybrid objective combining weighted categorical cross-entropy and Concordance Correlation Coefficient (CCC) losses for joint discrete and dimensional emotion modelling. This dual approach enables the model to capture both the distinct categories of emotion (like happy or angry) and the fine-grained, such as arousal (intensity), valence (positivity/negativity), and dominance (sense of control), lead- ing to a more comprehensive and robust representation of human affect. Extensive evaluations across multilingual Singaporean languages (English, Chinese, Malay, and Tamil ) and other public benchmarks show that MERaLiON-SER consistently surpasses both open-source speech encoders and large Audio-LLMs. These results underscore the importance of specialised speech-only models for accurate paralin- guistic understanding and cross-lingual generalisation. Furthermore, the proposed framework provides a foundation for integrating emotion-aware perception into future agentic audio systems, enabling more empathetic and contextually adaptive multimodal reasoning.

Method: Train autoencoders to reconstruct inputs from noised versions of their encodings combined with perceptual losses, then apply this approach to audio autoencoders.

Result: Perceptually salient information is captured in coarser representation structures, and the perceptual hierarchies improve latent diffusion decoding for estimating music pitch surprisal and predicting EEG-brain responses to music.

Conclusion: The proposed training method successfully creates perceptual hierarchies in autoencoder encodings that enhance downstream tasks like music analysis and brain response prediction.

Abstract: We argue that training autoencoders to reconstruct inputs from noised versions of their encodings, when combined with perceptual losses, yields encodings that are structured according to a perceptual hierarchy. We demonstrate the emergence of this hierarchical structure by showing that, after training an audio autoencoder in this manner, perceptually salient information is captured in coarser representation structures than with conventional training. Furthermore, we show that such perceptual hierarchies improve latent diffusion decoding in the context of estimating surprisal in music pitches and predicting EEG-brain responses to music listening. Pretrained weights are available on github.com/CPJKU/pa-audioic.

Method: Transform a two-hidden-layer multilayer perceptron designed for binary classification into a deep neural network architecture featuring one-gate layers and skip connections.

Result: Successful transformation of the MLP architecture into a deep neural network with the specified components.

Conclusion: Traditional MLP architectures can be readily converted into modern deep neural network structures with gated layers and skip connections, showing architectural flexibility.

Abstract: This paper shows how a multilayer perceptron with two hidden layers, which has been designed to classify two classes of data points, can easily be transformed into a deep neural network with one-gate layers and skip connections.

Method: Two-stage approach: 1) Identify daily proxy using six time series similarity measures on energy variables, 2) Model selected proxy with XGBoost algorithm for 15-day ahead forecasts.

Result: Brent volume emerged as the best proxy. Model achieved R² of 0.981 (training) and 0.945 (test) with acceptable error metrics. 15-day forecast shows fluctuating pattern with peak around day 4 and downward trend toward day 15.

Conclusion: The framework successfully converts low-frequency macroeconomic indicators into high-frequency signals, enabling real-time monitoring of energy security for policymakers in data-scarce environments.

Abstract: The policy environment of countries changes rapidly, influencing macro-level indicators such as the Energy Security Index. However, this index is only reported annually, limiting its responsiveness to short-term fluctuations. To address this gap, the present study introduces a daily proxy for the Energy Security Index and applies it to forecast energy security at a daily frequency.The study employs a two stage approach first, a suitable daily proxy for the annual Energy Security Index is identified by applying six time series similarity measures to key energy related variables. Second, the selected proxy is modeled using the XGBoost algorithm to generate 15 day ahead forecasts, enabling high frequency monitoring of energy security dynamics.As the result of proxy choosing, Volume Brent consistently emerged as the most suitable proxy across the majority of methods. The model demonstrated strong performance, with an R squared of 0.981 on the training set and 0.945 on the test set, and acceptable error metrics . The 15 day forecast of Brent volume indicates short term fluctuations, with a peak around day 4, a decline until day 8, a rise near day 10, and a downward trend toward day 15, accompanied by prediction intervals.By integrating time series similarity measures with machine learning based forecasting, this study provides a novel framework for converting low frequency macroeconomic indicators into high frequency, actionable signals. The approach enables real time monitoring of the Energy Security Index, offering policymakers and analysts a scalable and practical tool to respond more rapidly to fast changing policy and market conditions, especially in data scarce environments.

Method: Proposed diversified flow matching with bilevel optimization, nonlinear interpolant, and structural reformulation to adapt flow matching for unified translation functions.

Result: DFM successfully achieves translation identifiability and provides transport trajectories, validated on synthetic and real-world datasets.

Conclusion: DFM is the first ODE-based approach that guarantees translation identifiability while providing useful transport trajectory information.

Abstract: Diversified distribution matching (DDM) finds a unified translation function mapping a diverse collection of conditional source distributions to their target counterparts. DDM was proposed to resolve content misalignment issues in unpaired domain translation, achieving translation identifiability. However, DDM has only been implemented using GANs due to its constraints on the translation function. GANs are often unstable to train and do not provide the transport trajectory information – yet such trajectories are useful in applications such as single-cell evolution analysis and robot route planning. This work introduces diversified flow matching (DFM), an ODE-based framework for DDM. Adapting flow matching (FM) to enforce a unified translation function as in DDM is challenging, as FM learns the translation function’s velocity rather than the translation function itself. A custom bilevel optimization-based training loss, a nonlinear interpolant, and a structural reformulation are proposed to address these challenges, offering a tangible implementation. To our knowledge, DFM is the first ODE-based approach guaranteeing translation identifiability. Experiments on synthetic and real-world datasets validate the proposed method.

Method: Introduces Iterative Reward-Guided Refinement (IterRef) that leverages reward-guided noising-denoising transitions within a Multiple-Try Metropolis (MTM) framework to progressively refine misaligned intermediate states, proving convergence to the reward-aligned distribution.

Result: IterRef achieves consistent improvements in reward-guided generation quality across text and image domains, with striking gains under low compute budgets that far surpass prior state-of-the-art baselines.

Conclusion: IterRef provides an effective test-time scaling approach for discrete diffusion models that progressively refines intermediate states toward optimal distributions, demonstrating superior performance particularly under constrained computational resources.

Abstract: Test-time scaling through reward-guided generation remains largely unexplored for discrete diffusion models despite its potential as a promising alternative. In this work, we introduce Iterative Reward-Guided Refinement (IterRef), a novel test-time scaling method tailored to discrete diffusion that leverages reward-guided noising-denoising transitions to progressively refine misaligned intermediate states. We formalize this process within a Multiple-Try Metropolis (MTM) framework, proving convergence to the reward-aligned distribution. Unlike prior methods that assume the current state is already aligned with the reward distribution and only guide the subsequent transition, our approach explicitly refines each state in situ, progressively steering it toward the optimal intermediate distribution. Across both text and image domains, we evaluate IterRef on diverse discrete diffusion models and observe consistent improvements in reward-guided generation quality. In particular, IterRef achieves striking gains under low compute budgets, far surpassing prior state-of-the-art baselines.

Method: Proposes LookUM framework with: (i) path generator that samples from pools of unmasking sets, and (ii) verifier that computes path uncertainty and performs importance sampling to select final paths.

Result: Validated across six benchmarks (mathematics, planning, coding) with consistent performance improvements. LookUM requires only 2-3 paths for peak performance. Base LLaDA with LookUM rivals RL-tuned LLaDA 1.5 performance.

Conclusion: Uncertainty-based verification provides orthogonal benefits to reinforcement learning, demonstrating the versatility of the framework for improving masked diffusion models.

Abstract: Masked Diffusion Models (MDMs) as language models generate by iteratively unmasking tokens, yet their performance crucially depends on the inference time order of unmasking. Prevailing heuristics, such as confidence based sampling, are myopic: they optimize locally, fail to leverage extra test-time compute, and let early decoding mistakes cascade. We propose Lookahead Unmasking (LookUM), which addresses these concerns by reformulating sampling as path selection over all possible unmasking orders without the need for an external reward model. Our framework couples (i) a path generator that proposes paths by sampling from pools of unmasking sets with (ii) a verifier that computes the uncertainty of the proposed paths and performs importance sampling to subsequently select the final paths. Empirically, erroneous unmasking measurably inflates sequence level uncertainty, and our method exploits this to avoid error-prone trajectories. We validate our framework across six benchmarks, such as mathematics, planning, and coding, and demonstrate consistent performance improvements. LookUM requires only two to three paths to achieve peak performance, demonstrating remarkably efficient path selection. The consistent improvements on both LLaDA and post-trained LLaDA 1.5 are particularly striking: base LLaDA with LookUM rivals the performance of RL-tuned LLaDA 1.5, while LookUM further enhances LLaDA 1.5 itself showing that uncertainty based verification provides orthogonal benefits to reinforcement learning and underscoring the versatility of our framework. Code will be publicly released.

Method: Proposes variance-driven adaptive DRO that identifies high-risk samples and assigns personalized budgets using KL-divergence bounds, resulting in efficient water-filling solution with warmup phase and label smoothing.

Result: Achieves highest mean accuracy on CIFAR-10-C, improves overall performance on Waterbirds while matching/surpassing KL-DRO, and remains competitive on original CIFAR-10 with expected robustness trade-off.

Conclusion: Var-DRO is an unsupervised, theoretically sound, computationally efficient framework that provides adaptive robustness without requiring group labels.

Abstract: Distribution shifts and minority subpopulations frequently undermine the reliability of deep neural networks trained using Empirical Risk Minimization (ERM). Distributionally Robust Optimization (DRO) addresses this by optimizing for the worst-case risk within a neighborhood of the training distribution. However, conventional methods depend on a single, global robustness budget, which can lead to overly conservative models or a misallocation of robustness. We propose a variance-driven, adaptive, sample-level DRO (Var-DRO) framework that automatically identifies high-risk training samples and assigns a personalized robustness budget to each based on its online loss variance. Our formulation employs two-sided, KL-divergence-style bounds to constrain the ratio between adversarial and empirical weights for every sample. This results in a linear inner maximization problem over a convex polytope, which admits an efficient water-filling solution. To stabilize training, we introduce a warmup phase and a linear ramp schedule for the global cap on per-sample budgets, complemented by label smoothing for numerical robustness. Evaluated on CIFAR-10-C (corruptions), our method achieves the highest overall mean accuracy compared to ERM and KL-DRO. On Waterbirds, Var-DRO improves overall performance while matching or surpassing KL-DRO. On the original CIFAR-10 dataset, Var-DRO remains competitive, exhibiting the modest trade-off anticipated when prioritizing robustness. The proposed framework is unsupervised (requiring no group labels), straightforward to implement, theoretically sound, and computationally efficient.

Method: The authors explore this trade-off in symbolic music generation with continuous musical attributes, proposing suitable attribute transformations to help achieve both objectives simultaneously.

Result: The study shows that existing approaches fail to jointly minimize both regularization objectives, but appropriate attribute transformations enable achieving both controllability and proper regularization of target latent dimensions.

Conclusion: Attribute transformations provide an effective solution to balance the KLD-AR trade-off, enabling explicit latent variable models to maintain both controllability over musical attributes and proper regularization of latent spaces.

Abstract: Explicit latent variable models provide a flexible yet powerful framework for data synthesis, enabling controlled manipulation of generative factors. With latent variables drawn from a tractable probability density function that can be further constrained, these models enable continuous and semantically rich exploration of the output space by navigating their latent spaces. Structured latent representations are typically obtained through the joint minimization of regularization loss functions. In variational information bottleneck models, reconstruction loss and Kullback-Leibler Divergence (KLD) are often linearly combined with an auxiliary Attribute-Regularization (AR) loss. However, balancing KLD and AR turns out to be a very delicate matter. When KLD dominates over AR, generative models tend to lack controllability; when AR dominates over KLD, the stochastic encoder is encouraged to violate the standard normal prior. We explore this trade-off in the context of symbolic music generation with explicit control over continuous musical attributes. We show that existing approaches struggle to jointly minimize both regularization objectives, whereas suitable attribute transformations can help achieve both controllability and regularization of the target latent dimensions.

Method: Used data from a major Danish aviation fuel distributor to compare traditional time series models, Prophet, LSTM sequence-to-sequence neural networks, and hybrid models across three different datasets for 30-day forecasting horizon.

Result: The study demonstrates the advantages of data-driven models over traditional approaches and shows the impact of incorporating additional variables in predictive models for jet fuel demand forecasting.

Conclusion: Data-driven approaches provide significant benefits for jet fuel demand forecasting, with hybrid models and additional variables improving prediction accuracy for supply chain optimization.

Abstract: Accurate forecasting of jet fuel demand is crucial for optimizing supply chain operations in the aviation market. Fuel distributors specifically require precise estimates to avoid inventory shortages or excesses. However, there is a lack of studies that analyze the jet fuel demand forecasting problem using machine learning models. Instead, many industry practitioners rely on deterministic or expertise-based models. In this research, we evaluate the performance of data-driven approaches using a substantial amount of data obtained from a major aviation fuel distributor in the Danish market. Our analysis compares the predictive capabilities of traditional time series models, Prophet, LSTM sequence-to-sequence neural networks, and hybrid models. A key challenge in developing these models is the required forecasting horizon, as fuel demand needs to be predicted for the next 30 days to optimize sourcing strategies. To ensure the reliability of the data-driven approaches and provide valuable insights to practitioners, we analyze three different datasets. The primary objective of this study is to present a comprehensive case study on jet fuel demand forecasting, demonstrating the advantages of employing data-driven models and highlighting the impact of incorporating additional variables in the predictive models.

Method: Couples agent-based modeling with Life Cycle Assessment (LCA) to simulate energy transition pathways interacting with regional resource competition, ecological constraints, and community-level burdens.

Result: Demonstrates how integrated multiscale decision making shapes energy pathway deployment and reveals spatially explicit trade-offs under scenario-driven constraints.

Conclusion: The framework supports more adaptive and resilient energy transition planning on spatial and institutional scales.

Abstract: Transitioning to sustainable and resilient energy systems requires navigating complex and interdependent trade-offs across environmental, social, and resource dimensions. Neglecting these trade-offs can lead to unintended consequences across sectors. However, existing assessments often evaluate emerging energy pathways and their impacts in silos, overlooking critical interactions such as regional resource competition and cumulative impacts. We present an integrated modeling framework that couples agent-based modeling and Life Cycle Assessment (LCA) to simulate how energy transition pathways interact with regional resource competition, ecological constraints, and community-level burdens. We apply the model to a case study in Southern California. The results demonstrate how integrated and multiscale decision making can shape energy pathway deployment and reveal spatially explicit trade-offs under scenario-driven constraints. This modeling framework can further support more adaptive and resilient energy transition planning on spatial and institutional scales.

Method: Created a multimodal polymer dataset and fine-tuned VLMs using instruction-tuning pairs with LoRA to assess multimodality impact on prediction.

Result: Fine-tuned models using LoRA outperformed unimodal and baseline approaches, demonstrating benefits of multimodal learning.

Conclusion: Multimodal approach improves polymer property prediction and reduces need for separate models for different properties, lowering deployment and maintenance costs.

Abstract: Vision-Language Models (VLMs) have shown strong performance in tasks like visual question answering and multimodal text generation, but their effectiveness in scientific domains such as materials science remains limited. While some machine learning methods have addressed specific challenges in this field, there is still a lack of foundation models designed for broad tasks like polymer property prediction using multimodal data. In this work, we present a multimodal polymer dataset to fine-tune VLMs through instruction-tuning pairs and assess the impact of multimodality on prediction performance. Our fine-tuned models, using LoRA, outperform unimodal and baseline approaches, demonstrating the benefits of multimodal learning. Additionally, this approach reduces the need to train separate models for different properties, lowering deployment and maintenance costs.

Method: Uses a library of small conditional diffusion models as plug-and-play latent constraints on a frozen unconditional backbone model, applying denoising diffusion processes as implicit probabilistic priors.

Result: Diffusion-driven constraints outperform traditional attribute regularization and other latent constraint architectures, achieving stronger correlations between target and generated attributes while maintaining high perceptual quality and diversity.

Conclusion: The approach demonstrates versatility across diverse musical attributes and provides expert users with precise control capabilities for symbolic music generation.

Abstract: Recent advances in latent diffusion models have demonstrated state-of-the-art performance in high-dimensional time-series data synthesis while providing flexible control through conditioning and guidance. However, existing methodologies primarily rely on musical context or natural language as the main modality of interacting with the generative process, which may not be ideal for expert users who seek precise fader-like control over specific musical attributes. In this work, we explore the application of denoising diffusion processes as plug-and-play latent constraints for unconditional symbolic music generation models. We focus on a framework that leverages a library of small conditional diffusion models operating as implicit probabilistic priors on the latents of a frozen unconditional backbone. While previous studies have explored domain-specific use cases, this work, to the best of our knowledge, is the first to demonstrate the versatility of such an approach across a diverse array of musical attributes, such as note density, pitch range, contour, and rhythm complexity. Our experiments show that diffusion-driven constraints outperform traditional attribute regularization and other latent constraints architectures, achieving significantly stronger correlations between target and generated attributes while maintaining high perceptual quality and diversity.

Method: Proposed DAUM framework integrates multi-objective learning with uncertainty modeling, then applies knowledge distillation to reduce computational overhead while preserving accuracy and uncertainty estimation benefits.

Result: Experiments on JD advertisement dataset show DAUM consistently improves predictive performance, with distilled model achieving tenfold increase in inference speed.

Conclusion: DAUM effectively addresses both accuracy and efficiency challenges in RTA Interception through joint uncertainty modeling and knowledge distillation, enabling reliable traffic filtering in real-time applications.

Abstract: Real-Time Auction (RTA) Interception aims to filter out invalid or irrelevant traffic to enhance the integrity and reliability of downstream data. However, two key challenges remain: (i) the need for accurate estimation of traffic quality together with sufficiently high confidence in the model’s predictions, typically addressed through uncertainty modeling, and (ii) the efficiency bottlenecks that such uncertainty modeling introduces in real-time applications due to repeated inference. To address these challenges, we propose DAUM, a joint modeling framework that integrates multi-objective learning with uncertainty modeling, yielding both traffic quality predictions and reliable confidence estimates. Building on DAUM, we further apply knowledge distillation to reduce the computational overhead of uncertainty modeling, while largely preserving predictive accuracy and retaining the benefits of uncertainty estimation. Experiments on the JD advertisement dataset demonstrate that DAUM consistently improves predictive performance, with the distilled model delivering a tenfold increase in inference speed.

Method: Proposes FedMAC framework with contrastive-based regularization to impose constraints on latent representation space and avoid trivial aggregation of multi-modal features.

Result: FedMAC outperforms baseline methods by up to 26% in severe missing scenarios across various client configurations with statistical heterogeneity.

Conclusion: FedMAC shows strong potential as a solution for partially missing modalities in federated systems, effectively handling instance-level heterogeneity in multi-modal data.

Abstract: Federated Learning (FL) is a method for training machine learning models using distributed data sources. It ensures privacy by allowing clients to collaboratively learn a shared global model while storing their data locally. However, a significant challenge arises when dealing with missing modalities in clients’ datasets, where certain features or modalities are unavailable or incomplete, leading to heterogeneous data distribution. While previous studies have addressed the issue of complete-modality missing, they fail to tackle partial-modality missing on account of severe heterogeneity among clients at an instance level, where the pattern of missing data can vary significantly from one sample to another. To tackle this challenge, this study proposes a novel framework named FedMAC, designed to address multi-modality missing under conditions of partial-modality missing in FL. Additionally, to avoid trivial aggregation of multi-modal features, we introduce contrastive-based regularization to impose additional constraints on the latent representation space. The experimental results demonstrate the effectiveness of FedMAC across various client configurations with statistical heterogeneity, outperforming baseline methods by up to 26% in severe missing scenarios, highlighting its potential as a solution for the challenge of partially missing modalities in federated systems. Our source code is provided at https://github.com/nmduonggg/PEPSY

Method: Proposed a depth-induced NTK kernel based on shortcut-related architecture, which converges to a Gaussian process as network depth approaches infinity. Theoretically analyzed training invariance and spectrum properties.

Result: The proposed kernel stabilizes kernel dynamics and mitigates degeneration. Experimental results demonstrate the effectiveness of the method.

Conclusion: The findings significantly extend neural kernel theory and provide deeper understanding of deep learning and scaling laws by incorporating depth considerations into NTK framework.

Abstract: While deep learning has achieved remarkable success across a wide range of applications, its theoretical understanding of representation learning remains limited. Deep neural kernels provide a principled framework to interpret over-parameterized neural networks by mapping hierarchical feature transformations into kernel spaces, thereby combining the expressive power of deep architectures with the analytical tractability of kernel methods. Recent advances, particularly neural tangent kernels (NTKs) derived by gradient inner products, have established connections between infinitely wide neural networks and nonparametric Bayesian inference. However, the existing NTK paradigm has been predominantly confined to the infinite-width regime, while overlooking the representational role of network depth. To address this gap, we propose a depth-induced NTK kernel based on a shortcut-related architecture, which converges to a Gaussian process as the network depth approaches infinity. We theoretically analyze the training invariance and spectrum properties of the proposed kernel, which stabilizes the kernel dynamics and mitigates degeneration. Experimental results further underscore the effectiveness of our proposed method. Our findings significantly extend the existing landscape of the neural kernel theory and provide an in-depth understanding of deep learning and the scaling law.

Method: Parse initial equation to syntax tree, compute residuals for each subequation, use residuals as new target variables to generate prompts for better subequations, and replace old subequations if better ones are found on validation set.

Result: RED improves all tested neural-guided and classical genetic programming systems on 53 equations from the Feynman benchmark.

Conclusion: RED is a fast, extensible post-processing method that enhances equation discovery systems by using residuals to generate targeted improvements.

Abstract: Neural-guided equation discovery systems use a data set as prompt and predict an equation that describes the data set without extensive search. However, if the equation does not meet the user’s expectations, there are few options for getting other equation suggestions without intensive work with the system. To fill this gap, we propose Residuals for Equation Discovery (RED), a post-processing method that improves a given equation in a targeted manner, based on its residuals. By parsing the initial equation to a syntax tree, we can use node-based calculation rules to compute the residual for each subequation of the initial equation. It is then possible to use this residual as new target variable in the original data set and generate a new prompt. If, with the new prompt, the equation discovery system suggests a subequation better than the old subequation on a validation set, we replace the latter by the former. RED is usable with any equation discovery system, is fast to calculate, and is easy to extend for new mathematical operations. In experiments on 53 equations from the Feynman benchmark, we show that it not only helps to improve all tested neural-guided systems, but also all tested classical genetic programming systems.

Method: Concurrency-Controlled Partial Rollout with Importance Sampling (CoPRIS) maintains fixed concurrent rollouts, early-terminates when sufficient samples are collected, reuses unfinished trajectories, and uses Cross-stage Importance Sampling Correction to handle off-policy trajectories.

Result: Experiments on mathematical reasoning benchmarks show CoPRIS achieves up to 1.94x faster training while maintaining comparable or superior performance to synchronous RL systems.

Conclusion: CoPRIS effectively addresses the inefficiency problem in RL post-training for LLMs through asynchronous partial rollouts and importance sampling correction.

Abstract: Reinforcement learning (RL) post-training has become a trending paradigm for enhancing the capabilities of large language models (LLMs). Most existing RL systems for LLMs operate in a fully synchronous manner, where training must wait for the rollout of an entire batch to complete. This design leads to severe inefficiencies, as extremely long trajectories can stall the entire rollout process and leave many GPUs idle. To address this issue, we propose Concurrency- Controlled Partial Rollout with Importance Sampling (CoPRIS), which mitigates long-tail inefficiencies by maintaining a fixed number of concurrent rollouts, early-terminating once sufficient samples are collected, and reusing unfinished trajectories in subsequent rollouts. To mitigate the impact of off-policy trajectories, we introduce Cross-stage Importance Sampling Correction, which concatenates buffered log probabilities from the previous policy with those recomputed under the current policy for importance sampling correction. Experiments on challenging mathematical reasoning benchmarks show that CoPRIS achieves up to 1.94x faster training while maintaining comparable or superior performance to synchronous RL systems. The code of CoPRIS is available at https://github.com/777pomingzi/CoPRIS.

Method: FedSparQ dynamically sparsifies gradients using adaptive thresholds, applies half-precision quantization to retained entries, and integrates residuals from error feedback to prevent information loss. It requires no manual tuning and works with any model architecture.

Result: FedSparQ reduces communication overhead by 90% compared to FedAvg, improves model accuracy by 6% over uncompressed FedAvg and state-of-the-art compression methods, and enhances convergence robustness by 50% compared to other baselines.

Conclusion: FedSparQ provides a practical, easy-to-deploy solution for bandwidth-constrained federated deployments and enables future extensions in adaptive precision and privacy-preserving protocols.

Abstract: Federated Learning (FL) enables collaborative model training across decentralized clients while preserving data privacy by keeping raw data local. However, FL suffers from significant communication overhead due to the frequent exchange of high-dimensional model updates over constrained networks. In this paper, we present FedSparQ, a lightweight compression framework that dynamically sparsifies the gradient of each client through an adaptive threshold, applies half-precision quantization to retained entries and integrates residuals from error feedback to prevent loss of information. FedSparQ requires no manual tuning of sparsity rates or quantization schedules, adapts seamlessly to both homogeneous and heterogeneous data distributions, and is agnostic to model architecture. Through extensive empirical evaluation on vision benchmarks under independent and identically distributed (IID) and non-IID data, we show that FedSparQ substantially reduces communication overhead (reducing by 90% of bytes sent compared to FedAvg) while preserving or improving model accuracy (improving by 6% compared to FedAvg non-compressed solution or to state-of-the-art compression models) and enhancing convergence robustness (by 50%, compared to the other baselines). Our approach provides a practical, easy-to-deploy solution for bandwidth-constrained federated deployments and lays the groundwork for future extensions in adaptive precision and privacy-preserving protocols.

Method: Extracts transferable class-specific subgraph patterns via ego-graph disentanglement, constructs graph vocabularies using graphon-based generative experts, and employs a lightweight MoE-CoE network for attentive knowledge routing from source domains during prompt fine-tuning.

Result: Extensive experiments show GRAVER outperforms 15 state-of-the-art baselines in effectiveness, robustness, and efficiency on downstream few-shot node and graph classification tasks.

Conclusion: GRAVER provides a robust and efficient framework for fine-tuning Graph Foundation Models by leveraging generative graph vocabularies and transferable subgraph patterns, enabling stable knowledge transfer across diverse graph domains and tasks.

Abstract: Inspired by the remarkable success of foundation models in language and vision, Graph Foundation Models (GFMs) hold significant promise for broad applicability across diverse graph tasks and domains. However, existing GFMs struggle with unstable few-shot fine-tuning, where both performance and adaptation efficiency exhibit significant fluctuations caused by the randomness in the support sample selection and structural discrepancies between the pre-trained and target graphs. How to fine-tune GFMs robustly and efficiently to enable trustworthy knowledge transfer across domains and tasks is the major challenge. In this paper, we propose GRAVER, a novel Generative gRAph VocabulariEs for Robust GFM fine-tuning framework that tackles the aforementioned instability via generative augmentations. Specifically, to identify transferable units, we analyze and extract key class-specific subgraph patterns by ego-graph disentanglement and validate their transferability both theoretically and empirically. To enable effective pre-training across diverse domains, we leverage a universal task template based on ego-graph similarity and construct graph vocabularies via graphon-based generative experts. To facilitate robust and efficient prompt fine-tuning, we grave the support samples with in-context vocabularies, where the lightweight MoE-CoE network attentively routes knowledge from source domains. Extensive experiments demonstrate the superiority of GRAVER over effectiveness, robustness, and efficiency on downstream few-shot node and graph classification tasks compared with 15 state-of-the-art baselines.