Method: Extracted paragraphs from CC-BY licensed scientific publications using GROBID, annotated with language identification (fastText) and scientific domain (OpenAlex), primarily in English and French.

Result: Created a publicly available dataset of 833k classified paragraphs on HuggingFace under CC-BY license, covering multiple scientific domains and languages.

Conclusion: This dataset provides a valuable resource for developing NLP tools for scientific literature analysis and can support various text mining applications in research.

Abstract: We present a dataset of 833k paragraphs extracted from CC-BY licensed scientific publications, classified into four categories: acknowledgments, data mentions, software/code mentions, and clinical trial mentions. The paragraphs are primarily in English and French, with additional European languages represented. Each paragraph is annotated with language identification (using fastText) and scientific domain (from OpenAlex). This dataset, derived from the French Open Science Monitor corpus and processed using GROBID, enables training of text classification models and development of named entity recognition systems for scientific literature mining. The dataset is publicly available on HuggingFace https://doi.org/10.57967/hf/6679 under a CC-BY license.

Method: Extensive controlled experiments with various models and algorithms, including reward decomposition studies and KL-regularized policy analysis.

Result: PO systematically optimizes for simplest reward components first, causing degeneration to answer-only formats even with 4x higher rewards for complex reasoning.

Conclusion: Policy freedom enables high-reward shortcut discovery but creates strong incentives for reward hacking, posing critical alignment challenges.

Abstract: Policy optimization (PO) algorithms are used to refine Large Language Models for complex, multi-step reasoning. Current state-of-the-art pipelines enforce a strict think-then-answer format to elicit chain-of-thought (CoT); however, the behavior of PO when these rigid constraints are relaxed into an open-ended CoT structure remains an under-studied question. We investigate this gap with an extensive suite of controlled experiments and identify a consistent principle: \textit{policy optimization consistently follows the path of least resistance}. When afforded the flexibility to interleave reasoning and response, policy optimization consistently learns to discard explicit reasoning, causing the policy to degenerate to a direct \texttt{}-only format. This outcome holds true across various models and algorithms. We find that this collapse in format is persistent even when the complex \texttt{} format is assigned up to 4x larger reward weights. We formalize this principle through a series of controlled reward decomposition experiments, demonstrating a clear hierarchy: PO systematically optimizes for the simplest reward component first, a preference that holds even when faced with mutually exclusive choices or strong incentives for more complex behaviors. Finally, we show that successful convergence on the high-reward shortcut is not a low-effort drift but is driven by the optimization process that requires the KL-regularized policy to have sufficient freedom to make a significant shift from its initial prior. Our findings reveal that granting policies the freedom to diverge is a double-edged sword: while necessary for discovering high-reward shortcuts, it also creates a powerful incentive to game the simplest aspects of the reward function, posing a critical challenge for reward hacking under alignment.

Method: Proposes Language Ranker framework that treats decoding as recommendation ranking, using a lightweight module to rerank candidate responses with features extracted by the base model.

Result: Achieves performance comparable to large-scale reward models across various tasks, with only <0.5M additional parameters, significantly reducing computational overhead in training and inference.

Conclusion: The method demonstrates efficiency and effectiveness in unlocking LLM capabilities through lightweight reranking, offering a practical alternative to computationally expensive approaches.

Abstract: Conventional research on large language models (LLMs) has primarily focused on refining output distributions, while paying less attention to the decoding process that transforms these distributions into final responses. Recent advances, such as scaling the computation of inference time with reward models, have underscored the importance of decoding, but these methods often suffer from high computational costs and limited applicability. In this paper, we revisit LLM generation through the lens of recommender systems, conceptualizing the decoding process as analogous to the ranking stage in recommendation pipelines. From this perspective, we observe that both traditional decoding methods and reward models exhibit clear limitations such as redundancy. Motivated by this insight, we propose Language Ranker, a novel framework that introduces a lightweight module to rerank candidate responses using features extracted by the base model. Experiments across a wide range of tasks show that Language Ranker achieves performance comparable to large-scale reward models, while requiring only <0.5M additional parameters, significantly reducing the computational overhead during both training and inference stages. This highlights the efficiency and effectiveness of our method, showcasing its potential to fully unlock the capabilities of LLMs.

Method: RACE compares LLM rationales against top-ranked supporting and contradicting lexical features from logistic regression baselines across four text classification datasets, using token-aware, exact string, and edit-distance matching techniques at multiple granularity levels.

Result: Empirical results show consistent asymmetry: correct predictions have higher coverage of supporting features, while incorrect predictions show elevated coverage of contradicting features. Edit-distance matching reveals paraphrastic overlaps that boost coverage while preserving this asymmetry.

Conclusion: LLM rationales combine both surface-level and flexible evidence reuse but can amplify misleading cues in error cases. RACE provides insights into LLM explanation faithfulness and establishes quantitative basis for evaluating reasoning completeness in neural language models.

Abstract: The growing adoption of machine learning (ML) in sensitive domains has heightened the demand for transparent and interpretable artificial intelligence. Large Language Models (LLMs) are increasingly capable of producing natural language explanations, yet it remains unclear whether these rationales faithfully capture the predictive signals that underlie decisions. This paper introduces RACE-Reasoning Alignment for Completeness of Explanations, a systematic framework to evaluate the alignment between LLM-generated explanations and interpretable feature importance scores derived from a logistic regression baseline. We analyze four widely used text classification datasets-WIKI ONTOLOGY, AG NEWS, IMDB, and GOEMOTIONS-and compare LLM rationales against top-ranked supporting and contradicting lexical features. To capture alignment at multiple levels of granularity, RACE implements token-aware, exact string, and edit-distance matching techniques. Empirical results reveal a consistent asymmetry: correct predictions exhibit higher coverage of supporting features, while incorrect predictions are associated with elevated coverage of contradicting features. Edit-distance matching further uncovers paraphrastic overlaps, boosting coverage while preserving this asymmetry. These findings demonstrate that LLM rationales combine both surface-level and flexible evidence reuse, yet can also amplify misleading cues in error cases. RACE provides new insights into the faithfulness of LLM explanations and establishes a quantitative basis for evaluating reasoning completeness in neural language models.

Method: Proposed a behavior-aware sampling framework that selects safety examples based on instruction-response behavior (refusal vs compliance) and semantic diversity across harm categories.

Result: The approach substantially reduces harmful outputs while maintaining helpfulness, achieving up to 41% reduction in harmfulness with only 0.5% additional training data.

Conclusion: Targeted data selection can significantly improve the safety and efficiency of fine-tuning at scale.

Abstract: Large language models often lose previously aligned safety behaviors when fine-tuned on benign data, a phenomenon known as catastrophic forgetting. Prior work shows that adding random safety examples can mitigate this effect, but it remains unclear which examples are most effective. We propose a behavior-aware sampling framework that selects safety examples based on two complementary factors: instruction-response behavior (e.g., refusal versus compliance) and semantic diversity across harm categories. Systematic evaluation shows that this approach substantially reduces harmful outputs while maintaining helpfulness, achieving up to a 41% reduction in harmfulness with only 0.5% additional training data. These results highlight how targeted data selection can improve the safety and efficiency of fine-tuning at scale.

Method: Generate multiple long-form responses, embed them, and measure semantic isotropy as angular dispersion of embeddings on unit sphere.

Result: Higher semantic isotropy (greater embedding dispersion) reliably indicates lower factual consistency across samples.

Conclusion: Semantic isotropy offers practical, low-cost trust assessment for real-world LLM workflows, outperforming existing approaches across multiple domains.

Abstract: To deploy large language models (LLMs) in high-stakes application domains that require substantively accurate responses to open-ended prompts, we need reliable, computationally inexpensive methods that assess the trustworthiness of long-form responses generated by LLMs. However, existing approaches often rely on claim-by-claim fact-checking, which is computationally expensive and brittle in long-form responses to open-ended prompts. In this work, we introduce semantic isotropy – the degree of uniformity across normalized text embeddings on the unit sphere – and use it to assess the trustworthiness of long-form responses generated by LLMs. To do so, we generate several long-form responses, embed them, and estimate the level of semantic isotropy of these responses as the angular dispersion of the embeddings on the unit sphere. We find that higher semantic isotropy – that is, greater embedding dispersion – reliably signals lower factual consistency across samples. Our approach requires no labeled data, no fine-tuning, and no hyperparameter selection, and can be used with open- or closed-weight embedding models. Across multiple domains, our method consistently outperforms existing approaches in predicting nonfactuality in long-form responses using only a handful of samples – offering a practical, low-cost approach for integrating trust assessment into real-world LLM workflows.

Method: Proposes tree-based configuration chunking to preserve semantics, constructs unified fact graphs to normalize vendor-specific configurations, and designs hybrid imperative-declarative language to reduce LLM reasoning burden.

Result: NetMind achieves accurate and scalable network behavior understanding, outperforming existing baselines in experiments using a contributed benchmark of NL question-answer pairs with network configurations.

Conclusion: The framework successfully addresses key challenges in NL-guided network analysis, demonstrating practical effectiveness for network behavior understanding through innovative approaches to configuration processing and reasoning assistance.

Abstract: Modern large-scale networks introduce significant complexity in understanding network behaviors, increasing the risk of misconfiguration. Prior work proposed to understand network behaviors by mining network configurations, typically relying on domain-specific languages interfaced with formal models. While effective, they suffer from a steep learning curve and limited flexibility. In contrast, natural language (NL) offers a more accessible and interpretable interface, motivating recent research on NL-guided network behavior understanding. Recent advances in large language models (LLMs) further enhance this direction, leveraging their extensive prior knowledge of network concepts and strong reasoning capabilities. However, three key challenges remain: 1) numerous router devices with lengthy configuration files challenge LLM’s long-context understanding ability; 2) heterogeneity across devices and protocols impedes scalability; and 3) complex network topologies and protocols demand advanced reasoning abilities beyond the current capabilities of LLMs. To tackle the above challenges, we propose NetMind, a novel framework for querying networks using NL. Our approach introduces a tree-based configuration chunking strategy to preserve semantic coherence while enabling efficient partitioning. We then construct a unified fact graph as an intermediate representation to normalize vendor-specific configurations. Finally, we design a hybrid imperative-declarative language to reduce the reasoning burden on LLMs and enhance precision. We contribute a benchmark consisting of NL question-answer pairs paired with network configurations. Experiments demonstrate that NetMind achieves accurate and scalable network behavior understanding, outperforming existing baselines.

Method: Uses recurrent outline generation with a planning agent that incrementally retrieves, reads, and updates outlines. Implements paper cards to distill papers into contributions, methods, and findings, plus a review-and-refine loop with visualization enhancement.

Result: Substantially outperforms state-of-the-art baselines in content coverage, structural coherence, and citation quality. Produces more accessible and better-organized surveys. Also introduces Survey-Arena benchmark for reliable assessment.

Conclusion: IterSurvey provides a more effective approach to automatic literature survey generation through iterative processing and faithful paper-level grounding, producing higher quality surveys than existing methods.

Abstract: Automatic literature survey generation has attracted increasing attention, yet most existing systems follow a one-shot paradigm, where a large set of papers is retrieved at once and a static outline is generated before drafting. This design often leads to noisy retrieval, fragmented structures, and context overload, ultimately limiting survey quality. Inspired by the iterative reading process of human researchers, we propose \ours, a framework based on recurrent outline generation, in which a planning agent incrementally retrieves, reads, and updates the outline to ensure both exploration and coherence. To provide faithful paper-level grounding, we design paper cards that distill each paper into its contributions, methods, and findings, and introduce a review-and-refine loop with visualization enhancement to improve textual flow and integrate multimodal elements such as figures and tables. Experiments on both established and emerging topics show that \ours\ substantially outperforms state-of-the-art baselines in content coverage, structural coherence, and citation quality, while producing more accessible and better-organized surveys. To provide a more reliable assessment of such improvements, we further introduce Survey-Arena, a pairwise benchmark that complements absolute scoring and more clearly positions machine-generated surveys relative to human-written ones. The code is available at https://github.com/HancCui/IterSurvey_Autosurveyv2.

Method: Trained ~7,000 comparable monolingual tokenizers for 97 languages, manipulating tokenization algorithm, vocabulary size, and dataset size. Measured token premiums and tested relationships with data similarity, vocabulary size, pre-tokenization, and language-specific features like writing system and word length.

Result: Data similarity between training and test data doesn’t impact token premiums, but vocabulary size and pre-tokenization do. Superword tokenizers (allowing merges over whitespaces) reduce token premium effects and improve compression overall. Optimal vocabulary sizes for each language can significantly reduce token premium effects.

Conclusion: Intervening on vocabulary size or pre-tokenizer significantly reduces crosslingual token premium effects. Superword tokenizers and language-specific optimal vocabulary sizes are effective strategies for mitigating token disparities across languages.

Abstract: The number of tokens it takes to encode parallel text in different languages is known to vary. These disparities are called token premiums. Having high token premiums leads to less throughput during training and increases costs at inference. In this paper, we show that even after controlling for dataset size, vocabulary size, and data content, monolingual tokenizers exhibit a wide range of token premiums across languages. To understand the cross-linguistic differences that cause these token premiums, we train a suite of approximately 7,000 comparable monolingual tokenizers for 97 languages, manipulating tokenization algorithm, vocabulary size, and dataset size. We measure token premiums and test for a relationship between factors such as data similarity (between tokenizer training and evaluation), vocabulary size, and pre-tokenization. We also investigate the role of language-specific features such as writing system and word length. We find that similarity between training and test data does not impact token premiums, but vocabulary size and pre-tokenization do. While simply increasing vocabulary size does not lead to reduced token premium effects, we can determine an ``optimal’’ vocabulary size for each language to achieve significantly reduced token premium effects. We also train superword tokenizers which allow merges over whitespaces, and we find that they both reduce token premium effects and improve compression overall. Thus, intervening on the vocabulary size or the pre-tokenizer significantly reduces crosslingual token premium effects.

Method: Created Arabic Little STT dataset with 355 utterances from 288 children (ages 6-13), then systematically assessed eight Whisper ASR variants on this child speech dataset.

Result: Even the best-performing Whisper model (Large_v3) achieved 0.66 WER on child speech, significantly worse than sub 0.20 WER on adult datasets, highlighting the performance gap.

Conclusion: Critical need for dedicated child speech benchmarks, inclusive training data, and strict ethical frameworks for child data protection in ASR development.

Abstract: The performance of Artificial Intelligence (AI) systems fundamentally depends on high-quality training data. However, low-resource languages like Arabic suffer from severe data scarcity. Moreover, the absence of child-specific speech corpora is an essential gap that poses significant challenges. To address this gap, we present our created dataset, Arabic Little STT, a dataset of Levantine Arabic child speech recorded in classrooms, containing 355 utterances from 288 children (ages 6 - 13). We further conduct a systematic assessment of Whisper, a state-of-the-art automatic speech recognition (ASR) model, on this dataset and compare its performance with adult Arabic benchmarks. Our evaluation across eight Whisper variants reveals that even the best-performing model (Large_v3) struggles significantly, achieving a 0.66 word error rate (WER) on child speech, starkly contrasting with its sub 0.20 WER on adult datasets. These results align with other research on English speech. Results highlight the critical need for dedicated child speech benchmarks and inclusive training data in ASR development. Emphasizing that such data must be governed by strict ethical and privacy frameworks to protect sensitive child information. We hope that this study provides an initial step for future work on equitable speech technologies for Arabic-speaking children. We hope that our publicly available dataset enrich the children’s demographic representation in ASR datasets.

Method: Proposes Model-Aware Tokenizer Transfer (MATT) with Attention Influence Modeling (AIM) objective that distills inter-token communication patterns from source to target model.

Result: MATT recovers large fraction of original model performance within few GPU hours, outperforming heuristic baselines across diverse linguistic settings.

Conclusion: Incorporating model-level signals offers practical and effective path toward robust tokenizer transfer in multilingual LLMs.

Abstract: Large Language Models (LLMs) are trained to support an increasing number of languages, yet their predefined tokenizers remain a bottleneck for adapting models to lower-resource or distinct-script languages. Existing tokenizer transfer methods typically rely on semantic heuristics to initialize new embeddings, ignoring higher-layer model dynamics and limiting transfer quality. We propose Model-Aware Tokenizer Transfer (MATT), a method that incorporates model internals into the tokenizer transfer process. MATT introduces an Attention Influence Modeling (AIM) objective that distills inter-token communication patterns from a source model into a target model with a new tokenizer, providing an efficient warm-up before standard language modeling. Unlike approaches that focus solely on embedding similarity, MATT leverages attention behavior to guide embedding initialization and adaptation. Experiments across diverse linguistic settings show that MATT recovers a large fraction of the original model’s performance within a few GPU hours, outperforming heuristic baselines. These results demonstrate that incorporating model-level signals offers a practical and effective path toward robust tokenizer transfer in multilingual LLMs.

Method: Train individual GPT-2 models from scratch on each author’s works, then compare how well each model predicts held-out text from its own author versus other authors.

Result: Models trained on one author’s works predict that author’s text more accurately than other authors’ texts. Successfully confirmed R.P. Thompson’s authorship of the 15th Oz book originally attributed to F.L. Baum.

Conclusion: Language models can effectively capture and identify individual authors’ unique writing styles, providing a reliable method for authorship attribution.

Abstract: We show that large language models (LLMs) can be used to distinguish the writings of different authors. Specifically, an individual GPT-2 model, trained from scratch on the works of one author, will predict held-out text from that author more accurately than held-out text from other authors. We suggest that, in this way, a model trained on one author’s works embodies the unique writing style of that author. We first demonstrate our approach on books written by eight different (known) authors. We also use this approach to confirm R. P. Thompson’s authorship of the well-studied 15th book of the Oz series, originally attributed to F. L. Baum.

Method: Leverage foundational theories of persuasion from social sciences to craft adversarial prompts. Investigate whether LLMs exhibit distinct persuasive fingerprints in their jailbreak responses through empirical evaluations across multiple aligned LLMs.

Result: Persuasion-aware prompts significantly bypass LLM safeguards and demonstrate potential to induce jailbreak behaviors. LLMs show distinct persuasive fingerprints in their responses.

Conclusion: Cross-disciplinary insights from social sciences are crucial for addressing evolving LLM safety challenges. Persuasion strategies can effectively circumvent alignment constraints in language models.

Abstract: Despite recent advances, Large Language Models remain vulnerable to jailbreak attacks that bypass alignment safeguards and elicit harmful outputs. While prior research has proposed various attack strategies differing in human readability and transferability, little attention has been paid to the linguistic and psychological mechanisms that may influence a model’s susceptibility to such attacks. In this paper, we examine an interdisciplinary line of research that leverages foundational theories of persuasion from the social sciences to craft adversarial prompts capable of circumventing alignment constraints in LLMs. Drawing on well-established persuasive strategies, we hypothesize that LLMs, having been trained on large-scale human-generated text, may respond more compliantly to prompts with persuasive structures. Furthermore, we investigate whether LLMs themselves exhibit distinct persuasive fingerprints that emerge in their jailbreak responses. Empirical evaluations across multiple aligned LLMs reveal that persuasion-aware prompts significantly bypass safeguards, demonstrating their potential to induce jailbreak behaviors. This work underscores the importance of cross-disciplinary insight in addressing the evolving challenges of LLM safety. The code and data are available.

Method: Identified three statistical properties correlated with transfer success: refusal direction activation, push away from refusal direction, and orthogonal direction shifts. Compared these with semantic similarity.

Result: Found that the three statistical properties strongly correlate with transfer success across experiments, while prompt semantic similarity only weakly correlates.

Conclusion: Provides a fine-grained understanding of transferability that can be used to improve attack success rates in practical applications.

Abstract: Discrete optimization-based jailbreaking attacks on large language models aim to generate short, nonsensical suffixes that, when appended onto input prompts, elicit disallowed content. Notably, these suffixes are often transferable – succeeding on prompts and models for which they were never optimized. And yet, despite the fact that transferability is surprising and empirically well-established, the field lacks a rigorous analysis of when and why transfer occurs. To fill this gap, we identify three statistical properties that strongly correlate with transfer success across numerous experimental settings: (1) how much a prompt without a suffix activates a model’s internal refusal direction, (2) how strongly a suffix induces a push away from this direction, and (3) how large these shifts are in directions orthogonal to refusal. On the other hand, we find that prompt semantic similarity only weakly correlates with transfer success. These findings lead to a more fine-grained understanding of transferability, which we use in interventional experiments to showcase how our statistical analysis can translate into practical improvements in attack success.

Method: Systematic study of top-performing regression-based and LLM-as-a-Judge QE metrics across 10 diverse language pairs, followed by proposing two mitigation strategies: length normalization during training and incorporating reference texts during evaluation.

Result: Two critical length biases identified: QE metrics consistently over-predict errors with increasing translation length (even for error-free texts) and prefer shorter translations when multiple candidates are available for the same source text.

Conclusion: The identified length biases risk unfair penalization of longer correct translations and sub-optimal decision-making. The proposed mitigation strategies (length normalization and reference incorporation) effectively reduce length bias.

Abstract: Quality Estimation (QE) metrics are vital in machine translation for reference-free evaluation and as a reward signal in tasks like reinforcement learning. However, the prevalence and impact of length bias in QE have been underexplored. Through a systematic study of top-performing regression-based and LLM-as-a-Judge QE metrics across 10 diverse language pairs, we reveal two critical length biases: First, QE metrics consistently over-predict errors with increasing translation length, even for high-quality, error-free texts. Second, they exhibit a preference for shorter translations when multiple candidates are available for the same source text. These inherent length biases risk unfairly penalizing longer, correct translations and can lead to sub-optimal decision-making in applications such as QE reranking and QE guided reinforcement learning. To mitigate this, we propose two strategies: (a) applying length normalization during model training, and (b) incorporating reference texts during evaluation. Both approaches were found to effectively reduce the identified length bias.

Method: Conducted 774 multilingual training experiments spanning 10M-8B parameters, analyzed cross-lingual transfer matrix (38×38 language pairs), and derived language-agnostic scaling laws.

Result: ATLAS outperforms existing scaling laws, reveals optimal scaling strategies for multilingual models, and identifies computational crossover points for pretraining vs finetuning.

Conclusion: Provides scientific foundation for democratizing scaling laws across languages and enables efficient scaling of multilingual models beyond English-first AI.

Abstract: Scaling laws research has focused overwhelmingly on English – yet the most prominent AI models explicitly serve billions of international users. In this work, we undertake the largest multilingual scaling laws study to date, totaling 774 multilingual training experiments, spanning 10M-8B model parameters, 400+ training languages and 48 evaluation languages. We introduce the Adaptive Transfer Scaling Law (ATLAS) for both monolingual and multilingual pretraining, which outperforms existing scaling laws’ out-of-sample generalization often by more than 0.3 R^2. Our analyses of the experiments shed light on multilingual learning dynamics, transfer properties between languages, and the curse of multilinguality. First, we derive a cross-lingual transfer matrix, empirically measuring mutual benefit scores between 38 x 38=1444 language pairs. Second, we derive a language-agnostic scaling law that reveals how to optimally scale model size and data when adding languages without sacrificing performance. Third, we identify the computational crossover points for when to pretrain from scratch versus finetune from multilingual checkpoints. We hope these findings provide the scientific foundation for democratizing scaling laws across languages, and enable practitioners to efficiently scale models – beyond English-first AI.

Method: Analyzed the geometry of LLM hidden-state space, identified emotional manifolds, tested cross-domain alignment across 8 emotion datasets in 5 languages, and developed intervention modules to manipulate emotion perception.

Result: Found a consistent low-dimensional emotional manifold that is directionally encoded, distributed across layers, and generalizes well across languages. Cross-domain alignment showed low error and strong linear probe performance. Emotion perception can be steered while preserving semantics.

Conclusion: LLMs possess a consistent and manipulable affective geometry that reveals how they internalize and process emotion, with universal emotional representations that span multiple languages and can be controlled through targeted interventions.

Abstract: This work investigates how large language models (LLMs) internally represent emotion by analyzing the geometry of their hidden-state space. The paper identifies a low-dimensional emotional manifold and shows that emotional representations are directionally encoded, distributed across layers, and aligned with interpretable dimensions. These structures are stable across depth and generalize to eight real-world emotion datasets spanning five languages. Cross-domain alignment yields low error and strong linear probe performance, indicating a universal emotional subspace. Within this space, internal emotion perception can be steered while preserving semantics using a learned intervention module, with especially strong control for basic emotions across languages. These findings reveal a consistent and manipulable affective geometry in LLMs and offer insight into how they internalize and process emotion.

Method: Analyzed six control techniques: prompt-only, generate-and-filter, DFA-based Ctrl-G decoding, Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Iterative Nullspace Projection (INLP) on compositional constraint task requiring sentences with agentic and communal descriptors for occupations.

Result: SFT achieved 99.87% compliance with high lexical diversity, DPO failed at 4.53% compliance despite similar training stability. Ctrl-G guaranteed perfect compliance but severely reduced fluency and diversity. Preference-based methods cannot satisfy compositional constraints.

Conclusion: Only explicit positive supervision enables mitigation of compositional biases; preference-based alignment fails to generalize logical structures, highlighting limitations of preference learning and necessity of explicit supervision for fair controlled generation.

Abstract: Large Language Models (LLMs) still produce gender-stereotyped language even in occupation-neutral contexts that reflect deep societal biases (Rudinger et al., 2018). To address this, prior work has proposed prompting, constrained decoding (Dathathri et al., 2020; Zhou et al., 2024), post-processing, and fine-tuning-based alignment (Rafailov et al., 2023; Ravfogel et al., 2022). However, the comparative efficacy and learning dynamics remain little understood. We report a comparative analysis of six control techniques for bias mitigation: prompt-only, generate-and-filter, DFA-based Ctrl-G decoding, Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Iterative Nullspace Projection (INLP). We evaluate each method on a compositional constraint task. This task requires generating sentences that contain at least one agentic and one communal descriptor for each of the twenty Winogender-derived occupations. We quantify trade-offs between control strength and naturalness with evaluations of constraint compliance, lexical diversity, and fluency. Our results reveal key contrasts among the methods: SFT achieves 99.87 +- 0.15% compliance and high lexical diversity, while DPO, despite similar training stability, fails at 4.53 +- 0.82%. Ctrl-G guarantees perfect compliance, but at the cost of severely reduced fluency and diversity. Preference-based learning fundamentally differs: it cannot satisfy compositional constraints, as binary preference signals encode ranking, not logical conjunctions. Only explicit positive supervision enables mitigation of compositional biases; preference-based alignment fails to generalize logical structures, underscoring the limitations of preference learning and the necessity of explicit supervision for fair and fluent controlled generation.

Method: Developed Dynamic Mode Steering (DMS) with: (1) lightweight linear probe to detect memorization reliance, (2) dynamic activation steering to nudge computation toward generalization circuits, framed as adaptive self-contrastive decoding.

Result: Experiments on reasoning and faithfulness tasks show DMS significantly improves logical consistency and factual accuracy.

Conclusion: DMS offers a principled approach to enhance LLM reliability by controlling reasoning modes at inference time.

Abstract: Large Language Models (LLMs) exhibit a troubling duality, capable of both remarkable generalization and brittle, verbatim memorization of their training data. This unpredictability undermines their reliability in high-stakes applications. In this work, we propose a unified framework to understand, identify, and control these distinct reasoning modes. First, we introduce a theoretical model based on the Information Bottleneck (IB) principle, formalizing generalization as the learning of a compressed, task-relevant representation and memorization as a failure to compress. Building on this theory, we develop Dynamic Mode Steering (DMS), a novel inference-time algorithm which comprises two components: (1) a lightweight, causally-grounded linear probe that identifies the model’s instantaneous reliance on memorization, and (2) a dynamic activation steering mechanism that nudges the model’s computation towards pre-identified generalization circuits. We frame DMS as a form of adaptive, self-contrastive decoding. Experiments on reasoning and faithfulness tasks demonstrate that DMS significantly improves logical consistency and factual accuracy, thereby offering a principled approach to enhancing LLM reliability.

Method: Apply scale-invariant logarithmic compression to input tokens, then process the compressed representation using a standard, unmodified transformer architecture.

Result: Evaluation on WikiText-103 and PG-19 benchmarks shows reduced perplexity compared to uncompressed baselines, with consistent performance improvement as compressed temporal contexts get longer.

Conclusion: Input-level logarithmic compression is a simple and effective way to extend a transformer’s long-range memory without architectural modifications.

Abstract: Most approaches to long-context processing increase the complexity of the transformer’s internal architecture by integrating mechanisms such as recurrence or auxiliary memory modules. In this work, we introduce an alternative approach that modifies the input representation itself, rather than the transformer architecture. Inspired by cognitive models of human memory, our method applies a scale-invariant logarithmic compression to the input tokens. The resulting compressed representation is processed by a standard, unmodified transformer, preserving architectural simplicity. We evaluate this approach on the WikiText-103 and PG-19 language modeling benchmarks, showing a reduction in perplexity compared to uncompressed baselines. Moreover, performance improves consistently with longer compressed temporal contexts, showing that input-level logarithmic compression is a simple and effective way to extend a transformer’s long-range memory.

Method: Based on BLSP-Emo empathetic speech-to-text model, OpenS2S employs streaming interleaved decoding for low-latency speech generation and uses an automated data construction pipeline that synthesizes diverse empathetic speech dialogues using LLMs and controllable TTS systems.

Result: The model achieves scalable training with rich paralinguistic diversity and minimal human supervision, creating a high-quality empathetic speech dialogue corpus.

Conclusion: OpenS2S is released as fully open-source including dataset, model weights, and codes to empower the research community and accelerate innovation in empathetic speech systems.

Abstract: Empathetic interaction is a cornerstone of human-machine communication, due to the need for understanding speech enriched with paralinguistic cues and generating emotional and expressive responses. However, the most powerful empathetic LSLMs are increasingly closed off, leaving the crucial details about the architecture, data and development opaque to researchers. Given the critical need for transparent research into the LSLMs and empathetic behavior, we present OpenS2S, a fully open-source, transparent and end-to-end LSLM designed to enable empathetic speech interactions. Based on our empathetic speech-to-text model BLSP-Emo, OpenS2S further employs a streaming interleaved decoding architecture to achieve low-latency speech generation. To facilitate end-to-end training, OpenS2S incorporates an automated data construction pipeline that synthesizes diverse, high-quality empathetic speech dialogues at low cost. By leveraging large language models to generate empathetic content and controllable text-to-speech systems to introduce speaker and emotional variation, we construct a scalable training corpus with rich paralinguistic diversity and minimal human supervision. We release the fully open-source OpenS2S model, including the dataset, model weights, pre-training and fine-tuning codes, to empower the broader research community and accelerate innovation in empathetic speech systems. The project webpage can be accessed at https://casia-lm.github.io/OpenS2S

Method: Uses high-sparsity MoE with MTP for efficient reasoning, reasoning-oriented data with mid-training CoT activation, reinforcement-based fine-tuning (DFT, Evo-CoT), and full-scale FP8 training with fine-grained heterogeneous pipelines.

Result: Ling-1T establishes a new Pareto frontier of reasoning accuracy versus computational efficiency, with the series achieving up to 7-fold active-compute efficiency compared to dense counterparts.

Conclusion: Sparse activation, when properly aligned with reasoning objectives, enables scalable and efficient intelligence, providing a coherent foundation for advancing future reasoning and thinking models.

Abstract: We introduce Ling 2.0, a series reasoning-oriented language foundation built upon the principle that every activation boosts reasoning capability. Designed to scale from tens of billions to one trillion parameters under a unified Mixture-of-Experts (MoE) paradigm, Ling 2.0 emphasizes high sparsity, cross-scale consistency, and efficiency guided by empirical scaling laws. The series includes three non-thinking (instruct) models - Ling-mini-2.0, Ling-flash-2.0, and Ling-1T - ranging from 16B to 1T total parameters and achieving up to 7-fold active-compute efficiency compared with dense counterparts. Ling 2.0 integrates coordinated innovations across model architecture, pre-training, post-training, and infrastructure: a high-sparsity MoE with MTP for efficient reasoning, reasoning-oriented data and mid-training CoT activation, reinforcement-based fine-tuning (DFT, Evo-CoT), and full-scale FP8 training with fine-grained heterogeneous pipelines. At the trillion scale, Ling-1T establishes a new Pareto frontier of reasoning accuracy versus computational efficiency, demonstrating that sparse activation, when properly aligned with reasoning objectives, enables scalable and efficient intelligence. Collectively, Ling 2.0 provides a coherent, open, and efficient foundation for advancing future reasoning and thinking models, including the Ring series built upon the same base.

Method: Two-stage approach: Learn-to-Think stage learns reasoning from human experts, then Learn-to-Respond stage uses cold-start SFT combined with RL for self-refinement.

Result: Significant improvements: +28.92%/+18.42% intelligent resolution rates and -6.08%/-7.12% human takeover rates in community-support/livestream-interaction scenarios.

Conclusion: OlaMind effectively enhances human-likeness while mitigating hallucinations and business risks, demonstrating consistent effectiveness across diverse real-world applications.

Abstract: Intelligent customer service (ICS) systems via retrieval-augmented generation (RAG) have been widely adopted in Web-based domains such as social platforms and e-commerce, achieving remarkable improvements in automation and efficiency. However, notable limitations still remain: these systems are prone to hallucinations and often generate rigid, mechanical responses, which can introduce business risks and undermine user experience, especially in Web-based customer service interactions under the RAG scenarios. In this paper, we introduce OlaMind, a human-like and hallucination-safe customer service framework for retrieval-augmented dialogue. Specifically, it first leverages a Learn-to-Think stage to learn the reasoning processes and response strategies from human experts, and then employs a Learn-to-Respond stage to perform cold-start supervised fine-tuning (SFT) combined with reinforcement learning (RL) for basic-to-hard self-refinement. Our method significantly enhances human-likeness and naturalness while effectively mitigating hallucinations and critical business risks. We have conducted large-scale online A/B experiments in an industry-level social customer service setting, and extensive experimental results show that OlaMind achieves significant cumulative relative improvements with intelligent resolution rates +28.92%/+18.42% and human takeover rate -6.08%/-7.12% in community-support/livestream-interaction scenarios, respectively, which highlights its consistent effectiveness across diverse real-world applications. The code and data will be publicly available.

Method: Created a novel dataset of 3,221 Maithili sentences annotated for sentiment polarity with natural language justifications. The dataset was carefully curated and validated by linguistic experts to ensure label reliability and contextual fidelity. Experiments used both classical machine learning and state-of-the-art transformer architectures.

Result: The dataset enables effective interpretable sentiment analysis. Extensive experiments demonstrate the dataset’s effectiveness, with justifications written in Maithili promoting culturally grounded interpretation and enhancing model explainability.

Conclusion: This work establishes the first benchmark for explainable affective computing in Maithili, contributing a valuable resource to advance multilingual NLP and explainable AI for low-resource languages.

Abstract: Developing benchmark datasets for low-resource languages poses significant challenges, primarily due to the limited availability of native linguistic experts and the substantial time and cost involved in annotation. Given these challenges, Maithili is still underrepresented in natural language processing research. It is an Indo-Aryan language spoken by more than 13 million people in the Purvanchal region of India, valued for its rich linguistic structure and cultural significance. While sentiment analysis has achieved remarkable progress in high-resource languages, resources for low-resource languages, such as Maithili, remain scarce, often restricted to coarse-grained annotations and lacking interpretability mechanisms. To address this limitation, we introduce a novel dataset comprising 3,221 Maithili sentences annotated for sentiment polarity and accompanied by natural language justifications. Moreover, the dataset is carefully curated and validated by linguistic experts to ensure both label reliability and contextual fidelity. Notably, the justifications are written in Maithili, thereby promoting culturally grounded interpretation and enhancing the explainability of sentiment models. Furthermore, extensive experiments using both classical machine learning and state-of-the-art transformer architectures demonstrate the dataset’s effectiveness for interpretable sentiment analysis. Ultimately, this work establishes the first benchmark for explainable affective computing in Maithili, thus contributing a valuable resource to the broader advancement of multilingual NLP and explainable AI.

Method: Adapts the LENS framework to German and extends it with: (i) a pipeline for generating synthetic quality scores via LLMs, enabling dataset creation without human annotation, and (ii) an LLM-based refinement step for aligning grading criteria with simplification requirements.

Result: DETECT achieves substantially higher correlations with human judgments than widely used ATS metrics, with particularly strong gains in meaning preservation and fluency. The authors also construct the largest German human evaluation dataset for text simplification to validate the metric.

Conclusion: The findings highlight both the potential and limitations of LLMs for automatic evaluation and provide transferable guidelines for general language accessibility tasks beyond ATS.

Abstract: Current evaluation of German automatic text simplification (ATS) relies on general-purpose metrics such as SARI, BLEU, and BERTScore, which insufficiently capture simplification quality in terms of simplicity, meaning preservation, and fluency. While specialized metrics like LENS have been developed for English, corresponding efforts for German have lagged behind due to the absence of human-annotated corpora. To close this gap, we introduce DETECT, the first German-specific metric that holistically evaluates ATS quality across all three dimensions of simplicity, meaning preservation, and fluency, and is trained entirely on synthetic large language model (LLM) responses. Our approach adapts the LENS framework to German and extends it with (i) a pipeline for generating synthetic quality scores via LLMs, enabling dataset creation without human annotation, and (ii) an LLM-based refinement step for aligning grading criteria with simplification requirements. To the best of our knowledge, we also construct the largest German human evaluation dataset for text simplification to validate our metric directly. Experimental results show that DETECT achieves substantially higher correlations with human judgments than widely used ATS metrics, with particularly strong gains in meaning preservation and fluency. Beyond ATS, our findings highlight both the potential and the limitations of LLMs for automatic evaluation and provide transferable guidelines for general language accessibility tasks.

Method: DIPPER feeds an optimized and diverse set of prompts in parallel to a single LLM, creating varied reasoning paths without additional training.

Result: Significant improvements on reasoning benchmarks like MATH, where a 3-instance DIPPER ensemble of Qwen2-MATH-1.5B outperforms a larger 7B model.

Conclusion: DIPPER effectively enhances reasoning performance of smaller LLMs through parallel ensemble prompting, achieving better results than larger models.

Abstract: Large Language Models (LLMs), particularly smaller variants, still struggle with complex reasoning tasks. While inference-time prompting can guide reasoning, existing methods often rely on sequential queries. Ensemble approaches offer a promising path to performance gains, especially given recent batch inference speed-ups. This work introduces DIPPER, a novel, training-free framework that transforms a single LLM into an effective inference-time ensemble. By feeding the model an optimized and diverse set of prompts in parallel, DIPPER elicits varied reasoning paths, leading to performance gains. We empirically demonstrate significant improvements on reasoning benchmarks, such as MATH, where a DIPPER ensemble of three Qwen2-MATH-1.5B instances (via parallel prompting of a single model) outperforms a larger 7B model.

Method: Uses pairwise comparisons with two scenarios: (i) uniform error rate estimated with two comparisons per text pair, (ii) binary positional bias with distinct error rates for different comparison orders using repeated comparisons. Applies Copeland counting to construct rankings from preferences.

Result: Tested six LLMs (ChatGPT, Claude, DeepSeek, Gemini, Grok, Qwen) with five text types. Found consistent error rate estimates, with positional bias terms similar to uniform error. Claude performed best considering error rates and prompt robustness. The proposed model outperforms biased Bradley-Terry model and commutativity score.

Conclusion: The method successfully estimates LLMs’ error rates in pairwise comparison without ground truth, revealing poor scalability of LLM-based pairwise comparison. Claude demonstrated the most desirable performance among tested models.

Abstract: We measure LLMs’ output error at pairwise text comparison, noting the probability of error in their preferences. Our method does not rely on the ground truth and supports two scenarios: (i) uniform error rate regardless of the order of comparison, estimated with two comparisons for each text pair with either text placed first; (ii) binary positional bias assuming distinct error rates for the two orders of comparison, estimated with repeated comparisons between the texts. The Copeland counting constructs a ranking over the compared texts from pairwise preferences; the ranking reveals the poor scalability of LLM-based pairwise comparison and helps yield the estimates for LLMs’ error rates. We apply the method to six LLMs (ChatGPT, Claude, DeepSeek, Gemini, Grok, Qwen) with five types of text input and obtain consistent estimates of LLMs’ error. In general, the measured two positional bias terms are similar, close to the uniform error. Considering both the error rates and the robustness to the variation of prompts, Claude obtained the most desirable performance in this experiment. Our model outperforms the biased Bradley-Terry model and the commutativity score in indicating LLMs’ error at this task.

Method: Mathematical analysis of probabilistic limits in language dating, and incorporation of gradual lexical modification as a second stochastic process in vocabulary evolution models.

Result: Demonstrates inherent accuracy limits in Swadesh dating due to probability constraints, and shows that including gradual lexical modification significantly improves temporal separation estimation precision.

Conclusion: The Swadesh approach has fundamental accuracy limitations even under ideal conditions, but considering gradual lexical modification alongside word replacement provides more accurate language dating results.

Abstract: The Swadesh approach for determining the temporal separation between two languages relies on the stochastic process of words replacement (when a complete new word emerges to represent a given concept). It is well known that the basic assumptions of the Swadesh approach are often unrealistic due to various contamination phenomena and misjudgments (horizontal transfers, variations over time and space of the replacement rate, incorrect assessments of cognacy relationships, presence of synonyms, and so on). All of this means that the results cannot be completely correct. More importantly, even in the unrealistic case that all basic assumptions are satisfied, simple mathematics places limits on the accuracy of estimating the temporal separation between two languages. These limits, which are purely probabilistic in nature and which are often neglected in lexicostatistical studies, are analyzed in detail in this article. Furthermore, in this work we highlight that the evolution of a language’s lexicon is also driven by another stochastic process: gradual lexical modification of words. We show that this process equally also represents a major contribution to the reshaping of the vocabulary of languages over the centuries and we also show, from a purely probabilistic perspective, that taking into account this second random process significantly increases the precision in determining the temporal separation between two languages.

Method: Proposes a human-machine interactive translation platform that evolves from neural models to large language models, and from single-agent to multi-agent system with Translator, Annotator, and Proofreader agents supported by continuous feedback mechanisms.

Result: The paper critiques current sporadic translation efforts and presents a technological solution that could enable more efficient and high-quality translation of judicial judgments.

Conclusion: A sustainable strategy using advanced AI technology is needed to properly translate case law, which is essential for maintaining Hong Kong’s bilingual legal system and public trust in legal institutions.

Abstract: This paper addresses the challenges translating case law under Hong Kong’s bilingual legal system. It highlights the initial success of translating all written statutes into Chinese before the 1997 handover, a task mandated by the Basic Law. The effort involved significant collaboration among legal, linguistic, and translation experts, resulting in a comprehensive and culturally appropriate bilingual legal system. However, translating case law remains a significant challenge due to the sheer volume and continuous growth of judicial decisions. The paper critiques the governments and judiciarys sporadic and uncoordinated efforts to translate case law, contrasting it with the thorough approach previously taken for statute translation. Although the government acknowledges the importance of legal bilingualism, it lacks a sustainable strategy for translating case law. The Judiciarys position that translating all judgments is unnecessary, unrealistic, and not cost-effectiveis analyzed and critiqued for its impact on legal transparency and public trust. A proposed solution involves leveraging machine translation technology through a human-machine interactive translation platform, which undergoes two major transitions. Initially based on a neural model, the platform transitions to using a large language model for improved translation accuracy. Furthermore, it evolves from a single-agent system to a multi-agent system, incorporating Translator, Annotator, and Proofreader agents. This multi-agent approach, supported by a grant, aims to facilitate efficient, high-quality translation of judicial judgments by integrating advanced artificial intelligence and continuous feedback mechanisms, thus better meeting the needs of a bilingual legal system.

Method: Introduced “Sculpting” - a constrained, rule-based prompting method. Evaluated three strategies (Zero Shot, standard CoT, Sculpting) across three OpenAI models (gpt-4o-mini, gpt-4o, gpt-5) using GSM8K benchmark.

Result: Found “Prompting Inversion”: Sculpting improved performance on gpt-4o (97% vs 93% for CoT) but harmed performance on gpt-5 (94.00% vs 96.36% for CoT). Identified “Guardrail-to-Handcuff” transition where constraints become counterproductive.

Conclusion: Optimal prompting strategies must co-evolve with model capabilities, with simpler prompts working better for more capable models.

Abstract: Prompt engineering, particularly Chain-of-Thought (CoT) prompting, significantly enhances LLM reasoning capabilities. We introduce “Sculpting,” a constrained, rule-based prompting method designed to improve upon standard CoT by reducing errors from semantic ambiguity and flawed common sense. We evaluate three prompting strategies (Zero Shot, standard CoT, and Sculpting) across three OpenAI model generations (gpt-4o-mini, gpt-4o, gpt-5) using the GSM8K mathematical reasoning benchmark (1,317 problems). Our findings reveal a “Prompting Inversion”: Sculpting provides advantages on gpt-4o (97% vs. 93% for standard CoT), but becomes detrimental on gpt-5 (94.00% vs. 96.36% for CoT on full benchmark). We trace this to a “Guardrail-to-Handcuff” transition where constraints preventing common-sense errors in mid-tier models induce hyper-literalism in advanced models. Our detailed error analysis demonstrates that optimal prompting strategies must co-evolve with model capabilities, suggesting simpler prompts for more capable models.

Method: SteerX uses causal inference theory to estimate token-level causal effects, identifies preference-driven tokens, transforms these discrete signals into coherent descriptions, and leverages them to steer personalized LLM generation.

Result: Experiments on two representative steering backbone methods across real-world datasets show that SteerX consistently enhances steering vector quality and improves personalization.

Conclusion: SteerX offers a practical solution for more effective LLM personalization by focusing on truly preference-driven information to produce more accurate activation steering vectors.

Abstract: Large language models (LLMs) have shown remarkable success in recent years, enabling a wide range of applications, including intelligent assistants that support users’ daily life and work. A critical factor in building such assistants is personalizing LLMs, as user preferences and needs vary widely. Activation steering, which directly leverages directions representing user preference in the LLM activation space to adjust its behavior, offers a cost-effective way to align the model’s outputs with individual users. However, existing methods rely on all historical data to compute the steering vector, ignoring that not all content reflects true user preferences, which undermines the personalization signal. To address this, we propose SteerX, a disentangled steering method that isolates preference-driven components from preference-agnostic components. Grounded in causal inference theory, SteerX estimates token-level causal effects to identify preference-driven tokens, transforms these discrete signals into a coherent description, and then leverages them to steer personalized LLM generation. By focusing on the truly preference-driven information, SteerX produces more accurate activation steering vectors and enhances personalization. Experiments on two representative steering backbone methods across real-world datasets demonstrate that SteerX consistently enhances steering vector quality, offering a practical solution for more effective LLM personalization.

Method: Simulate crowds of generative agents with diverse demographic and ideological profiles using La Barbera et al. (2024) protocol. Agents retrieve evidence, assess claims along multiple quality dimensions, and issue veracity judgments.

Result: Agent crowds outperform human crowds in truthfulness classification, exhibit higher internal consistency, show reduced susceptibility to social and cognitive biases, and rely more systematically on informative criteria like Accuracy, Precision, and Informativeness.

Conclusion: Generative agents have significant potential as scalable, consistent, and less biased contributors to crowd-based fact-checking systems.

Abstract: The growing spread of online misinformation has created an urgent need for scalable, reliable fact-checking solutions. Crowdsourced fact-checking - where non-experts evaluate claim veracity - offers a cost-effective alternative to expert verification, despite concerns about variability in quality and bias. Encouraged by promising results in certain contexts, major platforms such as X (formerly Twitter), Facebook, and Instagram have begun shifting from centralized moderation to decentralized, crowd-based approaches. In parallel, advances in Large Language Models (LLMs) have shown strong performance across core fact-checking tasks, including claim detection and evidence evaluation. However, their potential role in crowdsourced workflows remains unexplored. This paper investigates whether LLM-powered generative agents - autonomous entities that emulate human behavior and decision-making - can meaningfully contribute to fact-checking tasks traditionally reserved for human crowds. Using the protocol of La Barbera et al. (2024), we simulate crowds of generative agents with diverse demographic and ideological profiles. Agents retrieve evidence, assess claims along multiple quality dimensions, and issue final veracity judgments. Our results show that agent crowds outperform human crowds in truthfulness classification, exhibit higher internal consistency, and show reduced susceptibility to social and cognitive biases. Compared to humans, agents rely more systematically on informative criteria such as Accuracy, Precision, and Informativeness, suggesting a more structured decision-making process. Overall, our findings highlight the potential of generative agents as scalable, consistent, and less biased contributors to crowd-based fact-checking systems.

Method: Developed PatenTEB benchmark with domain-stratified splits, hard negative mining, and asymmetric matching scenarios. Created patembed model family (67M-344M parameters) through multi-task training with context lengths up to 4096 tokens.

Result: patembed-base achieves SOTA on MTEB BigPatentClustering.v2 (0.494 V-measure vs 0.445 previous best), patembed-large achieves 0.377 NDCG@100 on DAPFAM. Multi-task training improves external generalization despite minor benchmark costs.

Conclusion: Domain-pretrained initialization provides consistent advantages across task families. All resources will be publicly available to advance patent text embedding research.

Abstract: Patent text embeddings enable prior art search, technology landscaping, and patent analysis, yet existing benchmarks inadequately capture patent-specific challenges. We introduce PatenTEB, a comprehensive benchmark comprising 15 tasks across retrieval, classification, paraphrase, and clustering, with 2.06 million examples. PatenTEB employs domain-stratified splits, domain specific hard negative mining, and systematic coverage of asymmetric fragment-to-document matching scenarios absent from general embedding benchmarks. We develop the patembed model family through multi-task training, spanning 67M to 344M parameters with context lengths up to 4096 tokens. External validation shows strong generalization: patembed-base achieves state-of-the-art on MTEB BigPatentClustering.v2 (0.494 V-measure vs. 0.445 previous best), while patembed-large achieves 0.377 NDCG@100 on DAPFAM. Systematic ablations reveal that multi-task training improves external generalization despite minor benchmark costs, and that domain-pretrained initialization provides consistent advantages across task families. All resources will be made available at https://github.com/iliass-y/patenteb. Keywords: patent retrieval, sentence embeddings, multi-task learning, asymmetric retrieval, benchmark evaluation, contrastive learning.

Method: Compared RAG and Continual Pre-Training strategies, with multi-modal RAG approach that presents retrieved slide content as images rather than text.

Result: RAG was more effective and efficient than CPT for small course materials, and multi-modal slide presentation significantly outperformed text-only retrieval.

Conclusion: Multi-modal RAG with course materials provides practical strategy for developing better educational AI assistants, inspiring similar approaches in other educational contexts.

Abstract: Large Language Models (LLMs) have advanced rapidly in recent years. One application of LLMs is to support student learning in educational settings. However, prior work has shown that LLMs still struggle to answer questions accurately within university-level computer science courses. In this work, we investigate how incorporating university course materials can enhance LLM performance in this setting. A key challenge lies in leveraging diverse course materials such as lecture slides and transcripts, which differ substantially from typical textual corpora: slides also contain visual elements like images and formulas, while transcripts contain spoken, less structured language. We compare two strategies, Retrieval-Augmented Generation (RAG) and Continual Pre-Training (CPT), to extend LLMs with course-specific knowledge. For lecture slides, we further explore a multi-modal RAG approach, where we present the retrieved content to the generator in image form. Our experiments reveal that, given the relatively small size of university course materials, RAG is more effective and efficient than CPT. Moreover, incorporating slides as images in the multi-modal setting significantly improves performance over text-only retrieval. These findings highlight practical strategies for developing AI assistants that better support learning and teaching, and we hope they inspire similar efforts in other educational contexts.

Method: Three approaches: (i) BERT-style encoders (BERT Base, BioClinicalBERT, RoBERTa-large), (ii) GPT-4o with few-shot in-context learning using simple vs. complex prompts, (iii) GPT-4o with supervised fine-tuning, evaluated on CADEC corpus with five entity types.

Result: RoBERTa-large and BioClinicalBERT showed limited improvements over BERT Base. Simple ICL outperformed complex prompts. SFT achieved strongest performance with F1 ≈ 87.1%, though with higher cost. LLMs performed better on simplified binary classification tasks.

Conclusion: Supervised fine-tuning of GPT-4o provides the best clinical NER performance, while simpler approaches show diminishing returns for specialized BERT variants. LLMs excel at simplified classification tasks.

Abstract: We study clinical Named Entity Recognition (NER) on the CADEC corpus and compare three families of approaches: (i) BERT-style encoders (BERT Base, BioClinicalBERT, RoBERTa-large), (ii) GPT-4o used with few-shot in-context learning (ICL) under simple vs.\ complex prompts, and (iii) GPT-4o with supervised fine-tuning (SFT). All models are evaluated on standard NER metrics over CADEC’s five entity types (ADR, Drug, Disease, Symptom, Finding). RoBERTa-large and BioClinicalBERT offer limited improvements over BERT Base, showing the limit of these family of models. Among LLM settings, simple ICL outperforms a longer, instruction-heavy prompt, and SFT achieves the strongest overall performance (F1 $\approx$ 87.1%), albeit with higher cost. We find that the LLM achieve higher accuracy on simplified tasks, restricting classification to two labels.

Method: Implemented memory-based language modeling using fast approximations of k-nearest neighbor classification, relying fully on CPUs for both training and inference to achieve low token latencies.

Result: The OLIFANT implementation showed competitive next-token prediction accuracy compared to GPT-2 and GPT-Neo, with lower estimated emissions and faster speeds due to its CPU-based architecture.

Conclusion: Memory-based language modeling offers a viable, transparent, and eco-friendly alternative to traditional deep neural network approaches for language modeling tasks.

Abstract: We present memory-based language modeling as an efficient, eco-friendly alternative to deep neural network-based language modeling. It offers log-linearly scalable next-token prediction performance and strong memorization capabilities. Implementing fast approximations of k-nearest neighbor classification, memory-based language modeling leaves a relatively small ecological footprint both in training and in inference mode, as it relies fully on CPUs and attains low token latencies. Its internal workings are simple and fully transparent. We compare our implementation of memory-based language modeling, OLIFANT, with GPT-2 and GPT-Neo on next-token prediction accuracy, estimated emissions and speeds, and offer some deeper analyses of the model.

Method: The authors created a multilingual TSE benchmark spanning Catalan, Estonian, French, Italian, Mandarin, and Spanish. They extended the original TSE pipeline to handle multiple languages using a single model approach.

Result: The model pipeline achieved an F1 score of 12.78, demonstrating the increased difficulty of multilingual TSE compared to English-only setups. Target prediction was identified as the primary bottleneck, and the study revealed F1 score sensitivity to different target verbalizations.

Conclusion: This work establishes the first baseline for multilingual TSE, providing resources, algorithms, and evaluation criteria for future research in this emerging field.

Abstract: Social media enables data-driven analysis of public opinion on contested issues. Target-Stance Extraction (TSE) is the task of identifying the target discussed in a document and the document’s stance towards that target. Many works classify stance towards a given target in a multilingual setting, but all prior work in TSE is English-only. This work introduces the first multilingual TSE benchmark, spanning Catalan, Estonian, French, Italian, Mandarin, and Spanish corpora. It manages to extend the original TSE pipeline to a multilingual setting without requiring separate models for each language. Our model pipeline achieves a modest F1 score of 12.78, underscoring the increased difficulty of the multilingual task relative to English-only setups and highlighting target prediction as the primary bottleneck. We are also the first to demonstrate the sensitivity of TSE’s F1 score to different target verbalizations. Together these serve as a much-needed baseline for resources, algorithms, and evaluation criteria in multilingual TSE.

Method: FAIR-RAG introduces an Iterative Refinement Cycle with Structured Evidence Assessment (SEA) that deconstructs queries into required findings, audits evidence for gaps, and uses Adaptive Query Refinement to generate targeted sub-queries for missing information.

Result: FAIR-RAG achieves state-of-the-art performance on multi-hop QA benchmarks, with an F1-score of 0.453 on HotpotQA - an 8.3 point improvement over the strongest iterative baseline.

Conclusion: A structured, evidence-driven refinement process with explicit gap analysis is crucial for reliable and accurate reasoning in advanced RAG systems for complex knowledge-intensive tasks.

Abstract: While Retrieval-Augmented Generation (RAG) mitigates hallucination and knowledge staleness in Large Language Models (LLMs), existing frameworks often falter on complex, multi-hop queries that require synthesizing information from disparate sources. Current advanced RAG methods, employing iterative or adaptive strategies, lack a robust mechanism to systematically identify and fill evidence gaps, often propagating noise or failing to gather a comprehensive context. We introduce FAIR-RAG, a novel agentic framework that transforms the standard RAG pipeline into a dynamic, evidence-driven reasoning process. At its core is an Iterative Refinement Cycle governed by a module we term Structured Evidence Assessment (SEA). The SEA acts as an analytical gating mechanism: it deconstructs the initial query into a checklist of required findings and audits the aggregated evidence to identify confirmed facts and, critically, explicit informational gaps. These gaps provide a precise signal to an Adaptive Query Refinement agent, which generates new, targeted sub-queries to retrieve missing information. This cycle repeats until the evidence is verified as sufficient, ensuring a comprehensive context for a final, strictly faithful generation. We conducted experiments on challenging multi-hop QA benchmarks, including HotpotQA, 2WikiMultiHopQA, and MusiQue. In a unified experimental setup, FAIR-RAG significantly outperforms strong baselines. On HotpotQA, it achieves an F1-score of 0.453 – an absolute improvement of 8.3 points over the strongest iterative baseline – establishing a new state-of-the-art for this class of methods on these benchmarks. Our work demonstrates that a structured, evidence-driven refinement process with explicit gap analysis is crucial for unlocking reliable and accurate reasoning in advanced RAG systems for complex, knowledge-intensive tasks.

Method: Translate English ironic corpus to Urdu, evaluate 10 ML algorithms with GloVe and Word2Vec embeddings, and fine-tune transformer models (BERT, RoBERTa, LLaMA 2, LLaMA 3, Mistral).

Result: Gradient Boosting achieved best ML performance (F1-score: 89.18%), LLaMA 3 (8B) achieved best transformer performance (F1-score: 94.61%).

Conclusion: Combining transliteration techniques with modern NLP models enables robust irony detection in Urdu, demonstrating effectiveness for low-resource languages.

Abstract: Ironic identification is a challenging task in Natural Language Processing, particularly when dealing with languages that differ in syntax and cultural context. In this work, we aim to detect irony in Urdu by translating an English Ironic Corpus into the Urdu language. We evaluate ten state-of-the-art machine learning algorithms using GloVe and Word2Vec embeddings, and compare their performance with classical methods. Additionally, we fine-tune advanced transformer-based models, including BERT, RoBERTa, LLaMA 2 (7B), LLaMA 3 (8B), and Mistral, to assess the effectiveness of large-scale models in irony detection. Among machine learning models, Gradient Boosting achieved the best performance with an F1-score of 89.18%. Among transformer-based models, LLaMA 3 (8B) achieved the highest performance with an F1-score of 94.61%. These results demonstrate that combining transliteration techniques with modern NLP models enables robust irony detection in Urdu, a historically low-resource language.

Method: Three-stage pipeline: large-scale contrastive pre-training, fine-tuning with hard negatives, and multitask generalization across retrieval, classification, and clustering. Uses bidirectional attention, latent attention pooling, and strategic pruning of 25% transformer layers.

Result: Achieved state-of-the-art results on ruMTEB benchmark with 69.1 average score across 23 multilingual tasks, outperforming larger baselines.

Conclusion: The framework successfully creates efficient and high-performing Russian text embeddings through hierarchical instruction tuning and architectural innovations.

Abstract: We introduce GigaEmbeddings, a novel framework for training high-performance Russian-focused text embeddings through hierarchical instruction tuning of the decoder-only LLM designed specifically for Russian language (GigaChat-3B). Our three-stage pipeline, comprising large-scale contrastive pre-training in web-scale corpora, fine-tuning with hard negatives, and multitask generalization across retrieval, classification, and clustering tasks, addresses key limitations of existing methods by unifying diverse objectives and leveraging synthetic data generation. Architectural innovations include bidirectional attention for contextual modeling, latent attention pooling for robust sequence aggregation, and strategic pruning of 25% of transformer layers to enhance efficiency without compromising performance. Evaluated on the ruMTEB benchmark spanning 23 multilingual tasks, GigaEmbeddings achieves state-of-the-art results (69.1 avg. score), outperforming strong baselines with a larger number of parameters.

Method: Created VisJudge-Bench with 3,090 expert-annotated samples from real-world scenarios covering single visualizations, multiple visualizations, and dashboards across 32 chart types, and proposed VisJudge model specifically designed for visualization aesthetics assessment.

Result: Advanced MLLMs (like GPT-5) show significant gaps compared to human experts with MAE of 0.551 and correlation of 0.429. VisJudge reduces MAE to 0.442 (19.8% reduction) and increases consistency to 0.681 (58.7% improvement).

Conclusion: VisJudge-Bench provides a comprehensive benchmark for visualization quality assessment, and VisJudge model significantly narrows the gap between AI and human judgment in evaluating visualization aesthetics and quality.

Abstract: Visualization, a domain-specific yet widely used form of imagery, is an effective way to turn complex datasets into intuitive insights, and its value depends on whether data are faithfully represented, clearly communicated, and aesthetically designed. However, evaluating visualization quality is challenging: unlike natural images, it requires simultaneous judgment across data encoding accuracy, information expressiveness, and visual aesthetics. Although multimodal large language models (MLLMs) have shown promising performance in aesthetic assessment of natural images, no systematic benchmark exists for measuring their capabilities in evaluating visualizations. To address this, we propose VisJudge-Bench, the first comprehensive benchmark for evaluating MLLMs’ performance in assessing visualization aesthetics and quality. It contains 3,090 expert-annotated samples from real-world scenarios, covering single visualizations, multiple visualizations, and dashboards across 32 chart types. Systematic testing on this benchmark reveals that even the most advanced MLLMs (such as GPT-5) still exhibit significant gaps compared to human experts in judgment, with a Mean Absolute Error (MAE) of 0.551 and a correlation with human ratings of only 0.429. To address this issue, we propose VisJudge, a model specifically designed for visualization aesthetics and quality assessment. Experimental results demonstrate that VisJudge significantly narrows the gap with human judgment, reducing the MAE to 0.442 (a 19.8% reduction) and increasing the consistency with human experts to 0.681 (a 58.7% improvement) compared to GPT-5. The benchmark is available at https://github.com/HKUSTDial/VisJudgeBench.

Method: Created a dataset with 900 scientific questions and over 7000 LLM-generated answers from 16 models across 5 high-resource and 4 low-resource languages, annotated with binary labels for scientific hallucinations (factuality errors) and fluency issues.

Result: CAP dataset provides question-answer pairs with token sequences and logits, annotated for factuality errors and linguistic quality, enabling research on hallucination detection and multilingual LLM evaluation.

Conclusion: CAP is publicly released to facilitate advanced research on hallucination detection, multilingual evaluation of LLMs, and development of more reliable scientific NLP systems.

Abstract: We introduce the CAP (Confabulations from ACL Publications) dataset, a multilingual resource for studying hallucinations in large language models (LLMs) within scientific text generation. CAP focuses on the scientific domain, where hallucinations can distort factual knowledge, as they frequently do. In this domain, however, the presence of specialized terminology, statistical reasoning, and context-dependent interpretations further exacerbates these distortions, particularly given LLMs’ lack of true comprehension, limited contextual understanding, and bias toward surface-level generalization. CAP operates in a cross-lingual setting covering five high-resource languages (English, French, Hindi, Italian, and Spanish) and four low-resource languages (Bengali, Gujarati, Malayalam, and Telugu). The dataset comprises 900 curated scientific questions and over 7000 LLM-generated answers from 16 publicly available models, provided as question-answer pairs along with token sequences and corresponding logits. Each instance is annotated with a binary label indicating the presence of a scientific hallucination, denoted as a factuality error, and a fluency label, capturing issues in the linguistic quality or naturalness of the text. CAP is publicly released to facilitate advanced research on hallucination detection, multilingual evaluation of LLMs, and the development of more reliable scientific NLP systems.

Method: CHOIR orchestrates a collaborative decoding process among counterfactual personas, dynamically balancing agreement and divergence in their reasoning paths to harmonize multiple persona-conditioned reasoning signals.

Result: CHOIR consistently enhances performance across demographics, model architectures, scales, and tasks without additional training, with improvements up to 26.4% for individual demographic groups and 19.2% on average across five demographics.

Conclusion: By reframing persona variation as a constructive signal, CHOIR provides a scalable and generalizable approach to more reliable LLM reasoning, remaining effective even when base personas are suboptimal.

Abstract: Persona-assigned Large Language Models (LLMs) can adopt diverse roles, enabling personalized and context-aware reasoning. However, even minor demographic perturbations in personas, such as simple pronoun changes, can alter reasoning trajectories, leading to divergent sets of correct answers. Instead of treating these variations as biases to be mitigated, we explore their potential as a constructive resource to improve reasoning robustness. We propose CHOIR (Collaborative Harmonization fOr Inference Robustness), a test-time framework that harmonizes multiple persona-conditioned reasoning signals into a unified prediction. CHOIR orchestrates a collaborative decoding process among counterfactual personas, dynamically balancing agreement and divergence in their reasoning paths. Experiments on various reasoning benchmarks demonstrate that CHOIR consistently enhances performance across demographics, model architectures, scales, and tasks - without additional training. Improvements reach up to 26.4% for individual demographic groups and 19.2% on average across five demographics. It remains effective even when base personas are suboptimal. By reframing persona variation as a constructive signal, CHOIR provides a scalable and generalizable approach to more reliable LLM reasoning.

Method: Used automatic speech recognition (ASR) models to measure sensitivity to pitch-flattening across closely related Tibetan languages at different stages of tonogenesis.

Result: Found evidence of a tonogenesis continuum: Amdo dialects (atonal) tolerate pitch removal most, U-Tsang varieties (fully tonal) show severe degradation, and Kham dialects (intermediate) fall between these extremes.

Conclusion: Computational methods can capture fine-grained stages of sound change, and traditional functional load metrics based on minimal pairs may overestimate pitch dependence in transitional systems where segmental and suprasegmental cues remain intertwined.

Abstract: Tonogenesis-the historical process by which segmental contrasts evolve into lexical tone-has traditionally been studied through comparative reconstruction and acoustic phonetics. We introduce a computational approach that quantifies the functional role of pitch at different stages of this sound change by measuring how pitch manipulation affects automatic speech recognition (ASR) performance. Through analysis on the sensitivity to pitch-flattening from a set of closely related Tibetan languages, we find evidence of a tonogenesis continuum: atonal Amdo dialects tolerate pitch removal the most, while fully tonal U-Tsang varieties show severe degradation, and intermediate Kham dialects fall measurably between these extremes. These gradient effects demonstrate how ASR models implicitly learn the shifting functional load of pitch as languages transition from consonant-based to tone-based lexical contrasts. Our findings show that computational methods can capture fine-grained stages of sound change and suggest that traditional functional load metrics, based solely on minimal pairs, may overestimate pitch dependence in transitional systems where segmental and suprasegmental cues remain phonetically intertwined.

Method: Extends conventional pruning by incorporating task-specific feature distributions into importance computation. Computes separate importance scores using general and task-specific data, partitions parameters into shared/exclusive groups based on activation-norm differences, and fuses scores to guide pruning.

Result: Experiments show the approach consistently outperforms baseline methods with identical pruning ratios across various settings and benchmarks.

Conclusion: The proposed task-aware pruning framework effectively preserves specialized capabilities while compressing LLMs, and can be seamlessly integrated with various foundation pruning techniques.

Abstract: Pruning provides a practical solution to reduce the resources required to run large language models (LLMs) to benefit from their effective capabilities as well as control their cost for training and inference. Research on LLM pruning often ranks the importance of LLM parameters using their magnitudes and calibration-data activations and removes (or masks) the less important ones, accordingly reducing LLMs’ size. However, these approaches primarily focus on preserving the LLM’s ability to generate fluent sentences, while neglecting performance on specific domains and tasks. In this paper, we propose a simple yet effective pruning approach for LLMs that preserves task-specific capabilities while shrinking their parameter space. We first analyze how conventional pruning minimizes loss perturbation under general-domain calibration and extend this formulation by incorporating task-specific feature distributions into the importance computation of existing pruning algorithms. Thus, our framework computes separate importance scores using both general and task-specific calibration data, partitions parameters into shared and exclusive groups based on activation-norm differences, and then fuses their scores to guide the pruning process. This design enables our method to integrate seamlessly with various foundation pruning techniques and preserve the LLM’s specialized abilities under compression. Experiments on widely used benchmarks demonstrate that our approach is effective and consistently outperforms the baselines with identical pruning ratios and different settings.

Method: Analyzed Whisper’s decoding behavior across 49 languages by logging decoding candidate sub-tokens and tracking their cumulative discovery over time, studying utilization patterns of the model’s sub-token space.

Result: Total discovered tokens independent of pre-training hours; sub-token discovery follows exponential saturation; Zipf-Mandelbrot patterns observed; Latin script languages show more favorable metrics than Cyrillic, CJK, and Semitic scripts.

Conclusion: Sub-token utilization in multilingual ASR inference is constrained more by statistical, typological, and orthographic structure of speech than by training data scale, providing basis for more equitable corpus construction and cross-lingual evaluation.

Abstract: How much audio is needed to fully observe a multilingual ASR model’s learned sub-token inventory across languages, and does data disparity in multilingual pre-training affect how these tokens are utilized during inference? We address this question by analyzing Whisper’s decoding behavior during inference across 49 languages. By logging decoding candidate sub-tokens and tracking their cumulative discovery over time, we study the utilization pattern of the model’s sub-token space. Results show that the total number of discovered tokens remains largely independent of a language’s pre-training hours, indicating that data disparity does not strongly influence lexical diversity in the model’s hypothesis space. Sub-token discovery rates follow a consistent exponential saturation pattern across languages, suggesting a stable time window after which additional audio yields minimal new sub-token activation. We refer to this convergence threshold as acoustic saturation time (AST). Further analyses of rank-frequency distributions reveal Zipf-like patterns better modeled by a Zipf-Mandelbrot law, and mean sub-token length shows a positive correlation with resource level. Additionally, those metrics show more favorable patterns for languages in the Latin script than those in scripts such as Cyrillic, CJK, and Semitic. Together, our study suggests that sub-token utilization during multilingual ASR inference is constrained more by the statistical, typological, and orthographic structure of the speech than by training data scale, providing an empirical basis for more equitable corpus construction and cross-lingual evaluation.

Method: Used the Pacific Northwest English (PNWE) corpus to analyze transcription accuracy across four ethnic backgrounds, introduced Phonetic Error Rate (PER) metric, and examined eleven sociophonetic features including vowel quality variation.

Result: Found that vowel quality variation, especially resistance to low-back merger and pre-nasal merger patterns, is systematically associated with higher error rates, with most pronounced effects for African American speakers across all evaluated systems.

Conclusion: Acoustic modeling of dialectal phonetic variation is a primary source of bias in commercial ASR systems, and targeted representation of sociophonetic diversity in training data is needed to improve performance.

Abstract: This paper presents a systematic evaluation of racial bias in four major commercial automatic speech recognition (ASR) systems using the Pacific Northwest English (PNWE) corpus. We analyze transcription accuracy across speakers from four ethnic backgrounds (African American, Caucasian American, ChicanX, and Yakama) and examine how sociophonetic variation contributes to differential system performance. We introduce a heuristically-determined Phonetic Error Rate (PER) metric that links recognition errors to specific linguistically motivated variables derived from sociophonetic annotation. Our analysis of eleven sociophonetic features reveals that vowel quality variation, particularly resistance to the low-back merger and pre-nasal merger patterns, is systematically associated with differential error rates across ethnic groups, with the most pronounced effects for African American speakers across all evaluated systems. These findings demonstrate that acoustic modeling of dialectal phonetic variation, rather than lexical or syntactic factors, remains a primary source of bias in commercial ASR systems. The study establishes the PNWE corpus as a valuable resource for bias evaluation in speech technologies and provides actionable guidance for improving ASR performance through targeted representation of sociophonetic diversity in training data.

Method: Fine-tuned BERT and DistilBERT, applied 4-bit LoRA to TinyLlama, LLaMA 3B/7B, and SaulLM models, and evaluated GPT-4o in zero-shot settings on CLAUDETTE-ToS benchmark and Multilingual Scraper Corpus.

Result: Full fine-tuning achieves strongest precision-recall balance, while LoRA-based models provide competitive recall with up to 3x lower memory cost.

Conclusion: The study highlights practical design trade-offs for efficient domain-adapted LLMs and contributes open baselines for fine-tuning research in legal text processing.

Abstract: Large Language Models (LLMs) have transformed text understanding, yet their adaptation to specialized legal domains remains constrained by the cost of full fine-tuning. This study provides a systematic evaluation of fine tuning, parameter efficient adaptation (LoRA, QLoRA), and zero-shot prompting strategies for unfair clause detection in Terms of Service (ToS) documents, a key application in legal NLP. We finetune BERT and DistilBERT, apply 4-bit Low-Rank Adaptation (LoRA) to models such as TinyLlama, LLaMA 3B/7B, and SaulLM, and evaluate GPT-4o and O-versions in zero-shot settings. Experiments on the CLAUDETTE-ToS benchmark and the Multilingual Scraper Corpus show that full fine-tuning achieves the strongest precision recall balance, while LoRA-based models provide competitive recall with up to 3x lower memory cost. These findings highlight practical design trade-offs for efficient and domain-adapted LLMs, contributing open baselines for fine-tuning research in legal text processing.

Method: Automatically collected real-world long texts (16k-2M tokens) across law, finance, game, and code domains, with 10 types of domain-specific long-dependency tasks and 1,934 QA instances generated through a scalable data curation pipeline.

Result: Comprehensive assessment of 6 locally deployed and 4 API-based LLMs shows even the best-performing model achieves only 59.2% overall score, revealing that popular LLMs can only understand much shorter context than claimed.

Conclusion: Despite extensive context windows, LLMs have significant limitations in handling real-world tasks with long dependencies, highlighting substantial room for improvement in practical long-context understanding.

Abstract: Large language models (LLMs) are equipped with increasingly extended context windows recently, yet their long context understanding capabilities over long dependency tasks remain fundamentally limited and underexplored. This gap is especially significant in many real-world long-context applications that were rarely benchmarked. In this paper, we introduce LooGLE v2, a novel benchmark designed to evaluate LLMs’ long context ability in real-world applications and scenarios. Our benchmark consists of automatically collected real-world long texts, ranging from 16k to 2M tokens, encompassing domains in law, finance, game and code. Accordingly, we delicately design 10 types of domain-specific long-dependency tasks and generate 1,934 QA instances with various diversity and complexity in a scalable data curation pipeline for further practical needs. We conduct a comprehensive assessment of 6 locally deployed and 4 API-based LLMs. The evaluation results show that even the best-performing model achieves only a 59.2% overall score on our benchmark. Despite the extensive context windows, popular LLMs are only capable of understanding a much shorter length of context than they claim to be, revealing significant limitations in their ability to handle real-world tasks with long dependencies and highlighting substantial room for model improvement in practical long-context understanding.

Method: Performs semantic segmentation to align compression boundaries with linguistic structures, applies segment-guided token scoring for importance estimation, and uses budget-driven search to adaptively determine optimal block sizes for each segment.

Result: Achieves 99.9% retrieval accuracy on NIAH with only 96 KV entries (vs 8K in full cache), reduces peak memory usage by 46.28%, and achieves up to 9.5x faster decoding on 128K context length under fixed cache budget.

Conclusion: SABlock consistently outperforms state-of-the-art baselines by effectively balancing semantic integrity and compression efficiency through semantic-aware adaptive block sizing.

Abstract: The growing memory footprint of the Key-Value (KV) cache poses a severe scalability bottleneck for long-context Large Language Model (LLM) inference. While KV cache eviction has emerged as an effective solution by discarding less critical tokens, existing token-, block-, and sentence-level compression methods struggle to balance semantic coherence and memory efficiency. To this end, we introduce SABlock, a \underline{s}emantic-aware KV cache eviction framework with \underline{a}daptive \underline{block} sizes. Specifically, SABlock first performs semantic segmentation to align compression boundaries with linguistic structures, then applies segment-guided token scoring to refine token importance estimation. Finally, for each segment, a budget-driven search strategy adaptively determines the optimal block size that preserves semantic integrity while improving compression efficiency under a given cache budget. Extensive experiments on long-context benchmarks demonstrate that SABlock consistently outperforms state-of-the-art baselines under the same memory budgets. For instance, on Needle-in-a-Haystack (NIAH), SABlock achieves 99.9% retrieval accuracy with only 96 KV entries, nearly matching the performance of the full-cache baseline that retains up to 8K entries. Under a fixed cache budget of 1,024, SABlock further reduces peak memory usage by 46.28% and achieves up to 9.5x faster decoding on a 128K context length.

Method: Combines Neural Cognitive Diagnosis (NCD) for fine-grained mastery assessment, Bounded-Ability Estimation Computerized Adaptive Testing (BECAT) for dynamic item selection, and LLMs for structured feedback in a closed-loop “Diagnosis-Recommendation-Feedback” framework.

Result: NCD achieves strong response prediction with interpretable mastery assessments, adaptive recommendation improves item relevance and personalization, and LLM-based feedback provides targeted study guidance aligned with identified weaknesses.

Conclusion: The proposed design is effective and practically deployable, providing a feasible pathway to generating individualized learning trajectories in intelligent education.

Abstract: As information technology advances, education is moving from one-size-fits-all instruction toward personalized learning. However, most methods handle modeling, item selection, and feedback in isolation rather than as a closed loop. This leads to coarse or opaque student models, assumption-bound adaptivity that ignores diagnostic posteriors, and generic, non-actionable feedback. To address these limitations, this paper presents an end-to-end personalized learning agent, EduLoop-Agent, which integrates a Neural Cognitive Diagnosis model (NCD), a Bounded-Ability Estimation Computerized Adaptive Testing strategy (BECAT), and large language models (LLMs). The NCD module provides fine-grained estimates of students’ mastery at the knowledge-point level; BECAT dynamically selects subsequent items to maximize relevance and learning efficiency; and LLMs convert diagnostic signals into structured, actionable feedback. Together, these components form a closed-loop framework of ``Diagnosis–Recommendation–Feedback.’’ Experiments on the ASSISTments dataset show that the NCD module achieves strong performance on response prediction while yielding interpretable mastery assessments. The adaptive recommendation strategy improves item relevance and personalization, and the LLM-based feedback offers targeted study guidance aligned with identified weaknesses. Overall, the results indicate that the proposed design is effective and practically deployable, providing a feasible pathway to generating individualized learning trajectories in intelligent education.

Method: The authors provide comprehensive state-of-the-art evaluation practices analysis, examine challenges through real-world case studies from AIED research, and build on insights from previous interdisciplinary AIED research.

Result: The paper identifies current evaluation inconsistencies and limitations in ITS assessment, particularly highlighting the reliance on subjective protocols and non-standardized benchmarks.

Conclusion: The authors propose three practical, feasible, and theoretically grounded research directions rooted in learning science principles to establish fair, unified, and scalable evaluation methodologies for ITSs.

Abstract: The interdisciplinary research domain of Artificial Intelligence in Education (AIED) has a long history of developing Intelligent Tutoring Systems (ITSs) by integrating insights from technological advancements, educational theories, and cognitive psychology. The remarkable success of generative AI (GenAI) models has accelerated the development of large language model (LLM)-powered ITSs, which have potential to imitate human-like, pedagogically rich, and cognitively demanding tutoring. However, the progress and impact of these systems remain largely untraceable due to the absence of reliable, universally accepted, and pedagogy-driven evaluation frameworks and benchmarks. Most existing educational dialogue-based ITS evaluations rely on subjective protocols and non-standardized benchmarks, leading to inconsistencies and limited generalizability. In this work, we take a step back from mainstream ITS development and provide comprehensive state-of-the-art evaluation practices, highlighting associated challenges through real-world case studies from careful and caring AIED research. Finally, building on insights from previous interdisciplinary AIED research, we propose three practical, feasible, and theoretically grounded research directions, rooted in learning science principles and aimed at establishing fair, unified, and scalable evaluation methodologies for ITSs.

Method: Uses reciprocal peer assessment where LLMs alternate roles as question generators, contestants, and judges across domains. Implements iterative weighting to amplify reliable evaluators and aggregates peer judgments into consensus-based rankings.

Result: Shows strong correlations with established benchmarks (78% with MMLU-Pro and 63% with GPQA). Multi-judge design significantly outperforms single-judge baselines, producing more robust and human-consistent assessments.

Conclusion: AutoBench provides a scalable, contamination-resistant alternative to static benchmarks for continuous evaluation of evolving language models through distributed peer assessment.

Abstract: We present AutoBench, a fully automated and self-sustaining framework for evaluating Large Language Models (LLMs) through reciprocal peer assessment. This paper provides a rigorous scientific validation of the AutoBench methodology, originally developed as an open-source project by eZecute S.R.L.. Unlike static benchmarks that suffer from test-set contamination and limited adaptability, AutoBench dynamically generates novel evaluation tasks while models alternately serve as question generators, contestants, and judges across diverse domains. An iterative weighting mechanism amplifies the influence of consistently reliable evaluators, aggregating peer judgments into consensus-based rankings that reflect collective model agreement. Our experiments demonstrate strong correlations with established benchmarks including MMLU-Pro and GPQA (respectively 78% and 63%), validating this peer-driven evaluation paradigm. The multi-judge design significantly outperforms single-judge baselines, confirming that distributed evaluation produces more robust and human-consistent assessments. AutoBench offers a scalable, contamination-resistant alternative to static benchmarks for the continuous evaluation of evolving language models.

Method: Uses multimodal agents, event-centric modeling, and contextual inference to create an ambient, adaptive companion that observes, interprets, and guides health in real time across daily life.

Result: The paper describes the architecture, design principles, and implementation challenges of PCU as an emerging paradigm for continuous health management.

Conclusion: PCU promises improved individual health outcomes and provides a new foundation for public health and scientific discovery through integration of personal sensing, experiential computing, and population-level analytics.

Abstract: Building on decades of success in digital infrastructure and biomedical innovation, we propose the Personal Care Utility (PCU) - a cybernetic system for lifelong health guidance. PCU is conceived as a global, AI-powered utility that continuously orchestrates multimodal data, knowledge, and services to assist individuals and populations alike. Drawing on multimodal agents, event-centric modeling, and contextual inference, it offers three essential capabilities: (1) trusted health information tailored to the individual, (2) proactive health navigation and behavior guidance, and (3) ongoing interpretation of recovery and treatment response after medical events. Unlike conventional episodic care, PCU functions as an ambient, adaptive companion - observing, interpreting, and guiding health in real time across daily life. By integrating personal sensing, experiential computing, and population-level analytics, PCU promises not only improved outcomes for individuals but also a new substrate for public health and scientific discovery. We describe the architecture, design principles, and implementation challenges of this emerging paradigm.

Method: Used conjunction-based segmentation to generate sentence-completion pairs, and developed DRESS-AF (Distractor Ranking via Embedding Similarity Scoring and Adversarial Filtering) to create difficult distractors from gold continuations.

Result: Human annotators scored 89%, OpenAI-o3 achieved 92.18%, Claude-Sonnet-3.7 scored 91.17%, and the best open-source model DeepSeek-R1 reached 82.51%. DRESS-AF also improved English HellaSwag benchmark difficulty without reducing human solvability.

Conclusion: PerCoR establishes a challenging benchmark for Persian commonsense reasoning, revealing significant performance gaps between models, and demonstrates that DRESS-AF can effectively increase dataset difficulty across languages.

Abstract: We introduced PerCoR (Persian Commonsense Reasoning), the first large-scale Persian benchmark for commonsense reasoning. PerCoR contains 106K multiple-choice sentence-completion problems drawn from more than forty news, cultural, and other web sources. We introduce a novel conjunction-based segmentation strategy to generate coherent sentence-completion pairs, enabling broad topical and structural diversity. To create challenging distractors, we propose DRESS-AF (Distractor Ranking via Embedding Similarity Scoring and Adversarial Filtering), a generation-free adversarial filtering method that selects distractors from the pool of gold continuations while maximising model confusion. Human annotators score 89% on PerCoR, while OpenAI-o3 achieves the highest performance at 92.18%, followed closely by Claude-Sonnet-3.7 (91.17%). The strongest open-source model, DeepSeek-R1, reaches 82.51%, underscoring both the dataset’s difficulty and the remaining performance gap in Persian commonsense reasoning. We further show that DRESS-AF transfers to the English HellaSwag benchmark, increasing its difficulty without hurting human solvability. The dataset is available at https://huggingface.co/datasets/MCINext/PerCoR.

Method: Fieldwork for linguistic documentation, creation of a morpheme-tagged trilingual corpus, training Small Language Model and Transformer-based translation engine, script standardization, and digital literacy tools development.

Result: A structured language corpus, AI models for language processing, and digital tools that support Toto language learning and preservation.

Conclusion: The study provides a sustainable model for preserving endangered languages by combining traditional linguistic methods with AI, emphasizing interdisciplinary collaboration for community-based language revitalization.

Abstract: Preserving linguistic diversity is necessary as every language offers a distinct perspective on the world. There have been numerous global initiatives to preserve endangered languages through documentation. This paper is a part of a project which aims to develop a trilingual (Toto-Bangla-English) language learning application to digitally archive and promote the endangered Toto language of West Bengal, India. This application, designed for both native Toto speakers and non-native learners, aims to revitalize the language by ensuring accessibility and usability through Unicode script integration and a structured language corpus. The research includes detailed linguistic documentation collected via fieldwork, followed by the creation of a morpheme-tagged, trilingual corpus used to train a Small Language Model (SLM) and a Transformer-based translation engine. The analysis covers inflectional morphology such as person-number-gender agreement, tense-aspect-mood distinctions, and case marking, alongside derivational strategies that reflect word-class changes. Script standardization and digital literacy tools were also developed to enhance script usage. The study offers a sustainable model for preserving endangered languages by incorporating traditional linguistic methodology with AI. This bridge between linguistic research with technological innovation highlights the value of interdisciplinary collaboration for community-based language revitalization.

Method: Expert native Italian speakers created culturally-grounded data samples following PIQA format, with additional English translations preserving Italian cultural elements.

Result: FormaMentis benchmark was developed as a submission to the MRL 2025 Shared Task, providing Italian physical commonsense reasoning data with cultural context.

Conclusion: The paper presents FormaMentis as a culturally-grounded Italian benchmark for physical commonsense reasoning, contributing to multilingual evaluation resources.

Abstract: This paper presents our submission to the MRL 2025 Shared Task on Multilingual Physical Reasoning Datasets. The objective of the shared task is to create manually-annotated evaluation data in the physical commonsense reasoning domain, for languages other than English, following a format similar to PIQA. Our contribution, FormaMentis, is a novel benchmark for physical commonsense reasoning that is grounded in Italian language and culture. The data samples in FormaMentis are created by expert annotators who are native Italian speakers and are familiar with local customs and norms. The samples are additionally translated into English, while preserving the cultural elements unique to the Italian context.

Method: Proposes CR-FKGC framework with: 1) neighborhood aggregation encoder for higher-order neighbor information, 2) conjugate relation learner combining implicit conditional diffusion module and stable relation module, 3) manifold conjugate decoder for inference in manifold space.

Result: Experiments on three benchmarks show superior performance over state-of-the-art methods.

Conclusion: The proposed CR-FKGC framework effectively addresses FKGC challenges by modeling conjugate relations and achieves better performance than existing approaches.

Abstract: Few-shot Knowledge Graph Completion (FKGC) infers missing triples from limited support samples, tackling long-tail distribution challenges. Existing methods, however, struggle to capture complex relational patterns and mitigate data sparsity. To address these challenges, we propose a novel FKGC framework for conjugate relation modeling (CR-FKGC). Specifically, it employs a neighborhood aggregation encoder to integrate higher-order neighbor information, a conjugate relation learner combining an implicit conditional diffusion relation module with a stable relation module to capture stable semantics and uncertainty offsets, and a manifold conjugate decoder for efficient evaluation and inference of missing triples in manifold space. Experiments on three benchmarks demonstrate that our method achieves superior performance over state-of-the-art methods.

Method: Generate rules by probing LLM with source combinations, with Apriori-like pruning optimizations from frequent itemset mining adapted for this novel problem.

Result: Proposed optimizations significantly speed up rule generation compared to brute force approach.

Conclusion: Qualitative and quantitative experiments demonstrate the value and efficiency of the proposed solutions for explaining LLM-RAG systems.

Abstract: If-then rules are widely used to explain machine learning models; e.g., “if employed = no, then loan application = rejected.” We present the first proposal to apply rules to explain the emerging class of large language models (LLMs) with retrieval-augmented generation (RAG). Since RAG enables LLM systems to incorporate retrieved information sources at inference time, rules linking the presence or absence of sources can explain output provenance; e.g., “if a Times Higher Education ranking article is retrieved, then the LLM ranks Oxford first.” To generate such rules, a brute force approach would probe the LLM with all source combinations and check if the presence or absence of any sources leads to the same output. We propose optimizations to speed up rule generation, inspired by Apriori-like pruning from frequent itemset mining but redefined within the scope of our novel problem. We conclude with qualitative and quantitative experiments demonstrating our solutions’ value and efficiency.

Method: Map each class label to a distinct output token, construct prompts to elicit single-token responses, and use parameter-efficient fine-tuning. During inference, project output only onto relevant class token logits for efficient single-forward-pass classification.

Result: SALSA achieves state-of-the-art results across diverse benchmarks, demonstrating robustness and scalability for LLM-based classification applications.

Conclusion: The SALSA pipeline effectively addresses LLM underperformance in text classification through structured prompting and efficient inference mechanisms.

Abstract: Despite their impressive generalization capabilities, instruction-tuned Large Language Models often underperform on text classification benchmarks. We introduce SALSA, a coherent pipeline that combines structured prompting, class-to-token mapping, and parameter-efficient fine-tuning, thereby avoiding cold-start training. Each class label is mapped to a distinct output token, and prompts are constructed to elicit a single-token response. During inference, the model’s output is projected only onto the logits of the relevant class tokens, enabling efficient and accurate classification in a single forward pass. SALSA achieves state-of-the-art results across diverse benchmarks, demonstrating its robustness and scalability for LLM-based classification applications.

Method: Uses cosine similarity between query and document embeddings as unified ranking function, with listwise ranking prompts constructed from original query and candidate documents (similar to pseudo-relevance feedback).

Result: Achieves state-of-the-art results on BEIR reranking benchmark, competitive performance on BRIGHT benchmark with low latency, and improves embedding performance on MTEB benchmark.

Conclusion: A single embedding model can effectively unify retrieval and reranking, offering both computational efficiency and competitive ranking accuracy.

Abstract: Text embedding models serve as a fundamental component in real-world search applications. By mapping queries and documents into a shared embedding space, they deliver competitive retrieval performance with high efficiency. However, their ranking fidelity remains limited compared to dedicated rerankers, especially recent LLM-based listwise rerankers, which capture fine-grained query-document and document-document interactions. In this paper, we propose a simple yet effective unified framework $\text{E}^2\text{Rank}$, means Efficient Embedding-based Ranking (also means Embedding-to-Rank), which extends a single text embedding model to perform both high-quality retrieval and listwise reranking through continued training under a listwise ranking objective, thereby achieving strong effectiveness with remarkable efficiency. By applying cosine similarity between the query and document embeddings as a unified ranking function, the listwise ranking prompt, which is constructed from the original query and its candidate documents, serves as an enhanced query enriched with signals from the top-K documents, akin to pseudo-relevance feedback (PRF) in traditional retrieval models. This design preserves the efficiency and representational quality of the base embedding model while significantly improving its reranking performance. Empirically, $\textrm{E}^2\text{Rank}$ achieves state-of-the-art results on the BEIR reranking benchmark and demonstrates competitive performance on the reasoning-intensive BRIGHT benchmark, with very low reranking latency. We also show that the ranking training process improves embedding performance on the MTEB benchmark. Our findings indicate that a single embedding model can effectively unify retrieval and reranking, offering both computational efficiency and competitive ranking accuracy.

Method: Used continual pre-training (CPT) with low-rank adaptation (LoRA) and compute-efficient techniques to adapt three LLMs to Québec French using a small dataset, updating under 1% of model parameters.

Result: Showed improvement on minority dialect benchmarks with minimal regression on prestige language benchmarks, though gains were highly dependent on corpus composition.

Conclusion: CPT with PEFT can effectively narrow the dialect gap and provide sustainable language resource creation for minority linguistic communities.

Abstract: Despite the widespread adoption of large language models (LLMs), their strongest capabilities remain largely confined to a small number of high-resource languages for which there is abundant training data. Recently, continual pre-training (CPT) has emerged as a means to fine-tune these models to low-resource regional dialects. In this paper, we study the use of CPT for dialect learning under tight data and compute budgets. Using low-rank adaptation (LoRA) and compute-efficient continual pre-training, we adapt three LLMs to the Qu'ebec French dialect using a very small dataset and benchmark them on the COLE suite. Our experiments demonstrate an improvement on the minority dialect benchmarks with minimal regression on the prestige language benchmarks with under 1% of model parameters updated. Analysis of the results demonstrate that gains are highly contingent on corpus composition. These findings indicate that CPT with parameter-efficient fine-tuning (PEFT) can narrow the dialect gap by providing cost-effective and sustainable language resource creation, expanding high-quality LLM access to minority linguistic communities. We release the first Qu'ebec French LLMs on HuggingFace.

Method: Tested various pretrained LLMs with sequences containing repeated tokens at fixed positions while permuting others, isolating temporal effects on next-token prediction. Used ablation experiments and extended analysis to unique semantic contexts.

Result: Models consistently favored tokens following repeated tokens but showed bias for beginning/end positions. Transformers’ bias linked to induction heads. State-space and transformer models showed comparable temporal biases despite architectural differences.

Conclusion: Findings deepen understanding of temporal biases in in-context learning and illustrate how these biases enable temporal separation and episodic retrieval in LLMs.

Abstract: In-context learning is governed by both temporal and semantic relationships, shaping how Large Language Models (LLMs) retrieve contextual information. Analogous to human episodic memory, where the retrieval of specific events is enabled by separating events that happened at different times, this work probes the ability of various pretrained LLMs, including transformer and state-space models, to differentiate and retrieve temporally separated events. Specifically, we prompted models with sequences containing multiple presentations of the same token, which reappears at the sequence end. By fixing the positions of these repeated tokens and permuting all others, we removed semantic confounds and isolated temporal effects on next-token prediction. Across diverse sequences, models consistently placed the highest probabilities on tokens following a repeated token, but with a notable bias for those nearest the beginning or end of the input. An ablation experiment linked this phenomenon in transformers to induction heads. Extending the analysis to unique semantic contexts with partial overlap further demonstrated that memories embedded in the middle of a prompt are retrieved less reliably. Despite architectural differences, state-space and transformer models showed comparable temporal biases. Our findings deepen the understanding of temporal biases in in-context learning and offer an illustration of how these biases can enable temporal separation and episodic retrieval.

Method: Created EchoMind benchmark with sequential, context-linked tasks: spoken-content understanding, vocal-cue perception, integrated reasoning, and response generation using semantically neutral scripts with controlled vocal style variations.

Result: Testing 12 advanced SLMs showed even state-of-the-art models struggle with high-expressive vocal cues, limiting empathetic response quality. Models have weaknesses in instruction-following, resilience to natural speech variability, and effective use of vocal cues.

Conclusion: SLMs need to better integrate linguistic content with diverse vocal cues to achieve truly empathetic conversational ability.

Abstract: Speech Language Models (SLMs) have made significant progress in spoken language understanding. Yet it remains unclear whether they can fully perceive non lexical vocal cues alongside spoken words, and respond with empathy that aligns with both emotional and contextual factors. Existing benchmarks typically evaluate linguistic, acoustic, reasoning, or dialogue abilities in isolation, overlooking the integration of these skills that is crucial for human-like, emotionally intelligent conversation. We present EchoMind, the first interrelated, multi-level benchmark that simulates the cognitive process of empathetic dialogue through sequential, context-linked tasks: spoken-content understanding, vocal-cue perception, integrated reasoning, and response generation. All tasks share identical and semantically neutral scripts that are free of explicit emotional or contextual cues, and controlled variations in vocal style are used to test the effect of delivery independent of the transcript. EchoMind is grounded in an empathy-oriented framework spanning 3 coarse and 12 fine-grained dimensions, encompassing 39 vocal attributes, and evaluated using both objective and subjective metrics. Testing 12 advanced SLMs reveals that even state-of-the-art models struggle with high-expressive vocal cues, limiting empathetic response quality. Analyses of prompt strength, speech source, and ideal vocal cue recognition reveal persistent weaknesses in instruction-following, resilience to natural speech variability, and effective use of vocal cues for empathy. These results underscore the need for SLMs that integrate linguistic content with diverse vocal cues to achieve truly empathetic conversational ability.

Method: Iterative layer pruning guided by layer importance analysis, evaluated on Aya-Expanse-8B model for Czech-German and English-Egyptian Arabic translation tasks.

Result: Achieved substantial reductions in model size and inference time while maintaining the translation quality of baseline models.

Conclusion: The proposed iterative layer pruning method with layer importance analysis is effective for compressing LLMs while preserving performance in machine translation tasks.

Abstract: Large language models (LLMs) have transformed many areas of natural language processing, including machine translation. However, efficient deployment of LLMs remains challenging due to their intensive computational requirements. In this paper, we address this challenge and present our submissions to the Model Compression track at the Conference on Machine Translation (WMT 2025). In our experiments, we investigate iterative layer pruning guided by layer importance analysis. We evaluate this method using the Aya-Expanse-8B model for translation from Czech to German, and from English to Egyptian Arabic. Our approach achieves substantial reductions in model size and inference time, while maintaining the translation quality of the baseline models.

Method: MMPersuade introduces a comprehensive multimodal dataset pairing images/videos with persuasion principles across commercial, subjective/behavioral, and adversarial contexts, and an evaluation framework using third-party agreement scoring and self-estimated token probabilities.

Result: Multimodal inputs substantially increase persuasion effectiveness and model susceptibility compared to text alone, especially for misinformation; stated prior preferences decrease susceptibility but multimodal advantage persists; different strategies vary in effectiveness across contexts.

Conclusion: MMPersuade provides a principled foundation for developing LVLMs that are robust, preference-consistent, and ethically aligned when engaging with persuasive multimodal content.

Abstract: As Large Vision-Language Models (LVLMs) are increasingly deployed in domains such as shopping, health, and news, they are exposed to pervasive persuasive content. A critical question is how these models function as persuadees-how and why they can be influenced by persuasive multimodal inputs. Understanding both their susceptibility to persuasion and the effectiveness of different persuasive strategies is crucial, as overly persuadable models may adopt misleading beliefs, override user preferences, or generate unethical or unsafe outputs when exposed to manipulative messages. We introduce MMPersuade, a unified framework for systematically studying multimodal persuasion dynamics in LVLMs. MMPersuade contributes (i) a comprehensive multimodal dataset that pairs images and videos with established persuasion principles across commercial, subjective and behavioral, and adversarial contexts, and (ii) an evaluation framework that quantifies both persuasion effectiveness and model susceptibility via third-party agreement scoring and self-estimated token probabilities on conversation histories. Our study of six leading LVLMs as persuadees yields three key insights: (i) multimodal inputs substantially increase persuasion effectiveness-and model susceptibility-compared to text alone, especially in misinformation scenarios; (ii) stated prior preferences decrease susceptibility, yet multimodal information maintains its persuasive advantage; and (iii) different strategies vary in effectiveness across contexts, with reciprocity being most potent in commercial and subjective contexts, and credibility and logic prevailing in adversarial contexts. By jointly analyzing persuasion effectiveness and susceptibility, MMPersuade provides a principled foundation for developing models that are robust, preference-consistent, and ethically aligned when engaging with persuasive multimodal content.

Method: R4P treats patch verification as a reasoning task, using a group-wise objective for RL training to verify multiple patches against each other’s modifications, providing dense rewards for stable training without requiring test execution.

Result: R4P achieves 72.2% accuracy on SWE-bench-verified, outperforms OpenAI o3, and enables Mini-SE to achieve 26.2% Pass@1 (10% improvement over Qwen3-32B), with verification speed 50x faster than testing.

Conclusion: R4P provides practical, scalable rewards for SWE agent training through reasoning-based verification, enabling stable scaling curves and high efficiency while avoiding test-based supervision limitations.

Abstract: While large language model agents have advanced software engineering tasks, the unscalable nature of existing test-based supervision is limiting the potential improvement of data scaling. The reason is twofold: (1) building and running test sandbox is rather heavy and fragile, and (2) data with high-coverage tests is naturally rare and threatened by test hacking via edge cases. In this paper, we propose R4P, a patch verifier model to provide scalable rewards for training and testing SWE agents via reasoning. We consider that patch verification is fundamentally a reasoning task, mirroring how human repository maintainers review patches without writing and running new reproduction tests. To obtain sufficient reference and reduce the risk of reward hacking, R4P uses a group-wise objective for RL training, enabling it to verify multiple patches against each other’s modification and gain a dense reward for stable training. R4P achieves 72.2% Acc. for verifying patches from SWE-bench-verified, surpassing OpenAI o3. To demonstrate R4P’s practicality, we design and train a lite scaffold, Mini-SE, with pure reinforcement learning where all rewards are derived from R4P. As a result, Mini-SE achieves 26.2% Pass@1 on SWE-bench-verified, showing a 10.0% improvement over the original Qwen3-32B. This can be further improved to 32.8% with R4P for test-time scaling. Furthermore, R4P verifies patches within a second, 50x faster than testing on average. The stable scaling curves of rewards and accuracy along with high efficiency reflect R4P’s practicality.

Method: VEHME uses a two-phase training pipeline: supervised fine-tuning with structured reasoning data, and reinforcement learning aligned with multi-dimensional grading objectives. It includes an Expression-Aware Visual Prompting Module trained on synthesized multi-line math expressions for spatial understanding.

Result: VEHME achieves state-of-the-art performance among open-source models on AIHub and FERMAT datasets, approaching the accuracy of proprietary systems while providing interpretable reasoning traces.

Conclusion: VEHME demonstrates potential as a scalable and accessible tool for automated math assessment, with publicly available training and experiment code.

Abstract: Automatically assessing handwritten mathematical solutions is an important problem in educational technology with practical applications, but it remains a significant challenge due to the diverse formats, unstructured layouts, and symbolic complexity of student work. To address this challenge, we introduce VEHME-a Vision-Language Model for Evaluating Handwritten Mathematics Expressions-designed to assess open-form handwritten math responses with high accuracy and interpretable reasoning traces. VEHME integrates a two-phase training pipeline: (i) supervised fine-tuning using structured reasoning data, and (ii) reinforcement learning that aligns model outputs with multi-dimensional grading objectives, including correctness, reasoning depth, and error localization. To enhance spatial understanding, we propose an Expression-Aware Visual Prompting Module, trained on our synthesized multi-line math expressions dataset to robustly guide attention in visually heterogeneous inputs. Evaluated on AIHub and FERMAT datasets, VEHME achieves state-of-the-art performance among open-source models and approaches the accuracy of proprietary systems, demonstrating its potential as a scalable and accessible tool for automated math assessment. Our training and experiment code is publicly available at our GitHub repository.

Method: Evaluated 6 models (4 commercial aligned, 2 open-weight) across 78,000 multilingual inferences using standard classification metrics and new cross-lingual reliability measures: Calibration Deviation, Decision Bias, Language Robustness Score, and Language Stability Score.

Result: Aligned models maintained near-invariant accuracy and balanced calibration across typologically distant and low-resource languages, while open-weight models showed significant prompt-language sensitivity and calibration drift. Alignment, not scale, determines stability.

Conclusion: Multilingual alignment enables language-agnostic reasoning, providing practical guidance for humanitarian organizations balancing budget constraints with reliability in multilingual deployment.

Abstract: Humanitarian organizations face a critical choice: invest in costly commercial APIs or rely on free open-weight models for multilingual human rights monitoring. While commercial systems offer reliability, open-weight alternatives lack empirical validation – especially for low-resource languages common in conflict zones. This paper presents the first systematic comparison of commercial and open-weight large language models (LLMs) for human-rights-violation detection across seven languages, quantifying the cost-reliability trade-off facing resource-constrained organizations. Across 78,000 multilingual inferences, we evaluate six models – four instruction-aligned (Claude-Sonnet-4, DeepSeek-V3, Gemini-Flash-2.0, GPT-4.1-mini) and two open-weight (LLaMA-3-8B, Mistral-7B) – using both standard classification metrics and new measures of cross-lingual reliability: Calibration Deviation (CD), Decision Bias (B), Language Robustness Score (LRS), and Language Stability Score (LSS). Results show that alignment, not scale, determines stability: aligned models maintain near-invariant accuracy and balanced calibration across typologically distant and low-resource languages (e.g., Lingala, Burmese), while open-weight models exhibit significant prompt-language sensitivity and calibration drift. These findings demonstrate that multilingual alignment enables language-agnostic reasoning and provide practical guidance for humanitarian organizations balancing budget constraints with reliability in multilingual deployment.

Method: Employ generative language models for automated scoring through summarization and prompting techniques.

Result: Significant improvement in scoring accuracy with QWK increasing from 0.822 to 0.8878 on the Learning Agency Lab Automated Essay Scoring 2.0 dataset.

Conclusion: Generative language models with summarization and prompting are effective for automated scoring of long essays, outperforming encoder-based models like BERT.

Abstract: BERT and its variants are extensively explored for automated scoring. However, a limit of 512 tokens for these encoder-based models showed the deficiency in automated scoring of long essays. Thus, this research explores generative language models for automated scoring of long essays via summarization and prompting. The results revealed great improvement of scoring accuracy with QWK increased from 0.822 to 0.8878 for the Learning Agency Lab Automated Essay Scoring 2.0 dataset.

Method: The authors provide a systematic guidebook for identifying threads in synchronous multi-party transcripts and benchmark different LLM prompting strategies for automated threading. They test how threading influences performance on downstream coding of conversational analysis frameworks capturing collaborative actions like agreeing, building, and eliciting.

Result: Results show that providing clear conversational thread information improves LLM coding performance and underscores the heavy reliance of downstream analysis on well-structured dialogue. The study also discusses practical trade-offs in time and cost.

Conclusion: This work advances methods for combining LLMs and robust conversational thread structures to make sense of complex, real-time group interactions, emphasizing where human-AI hybrid approaches can yield the best value.

Abstract: Understanding how ideas develop and flow in small-group conversations is critical for analyzing collaborative learning. A key structural feature of these interactions is threading, the way discourse talk naturally organizes into interwoven topical strands that evolve over time. While threading has been widely studied in asynchronous text settings, detecting threads in synchronous spoken dialogue remains challenging due to overlapping turns and implicit cues. At the same time, large language models (LLMs) show promise for automating discourse analysis but often struggle with long-context tasks that depend on tracing these conversational links. In this paper, we investigate whether explicit thread linkages can improve LLM-based coding of relational moves in group talk. We contribute a systematic guidebook for identifying threads in synchronous multi-party transcripts and benchmark different LLM prompting strategies for automated threading. We then test how threading influences performance on downstream coding of conversational analysis frameworks, that capture core collaborative actions such as agreeing, building, and eliciting. Our results show that providing clear conversational thread information improves LLM coding performance and underscores the heavy reliance of downstream analysis on well-structured dialogue. We also discuss practical trade-offs in time and cost, emphasizing where human-AI hybrid approaches can yield the best value. Together, this work advances methods for combining LLMs and robust conversational thread structures to make sense of complex, real-time group interactions.

Method: Per-Instance Program Synthesis (PIPS) generates and refines programs at the instance-level using structural feedback without task-specific guidance or explicit test cases, with a confidence metric for dynamic selection between inference modes.

Result: PIPS improves absolute harmonic mean accuracy by up to 8.6% and 9.4% compared to PoT and CoT respectively, and reduces undesirable program generations by 65.1% on algorithmic tasks compared to PoT with Gemini-2.0-Flash.

Conclusion: PIPS effectively enhances LLM reasoning capabilities by combining instance-level program synthesis with structural feedback and dynamic mode selection, significantly outperforming existing methods across diverse benchmarks.

Abstract: Large language models (LLMs) excel at zero-shot inference but continue to struggle with complex, multi-step reasoning. Recent methods that augment LLMs with intermediate reasoning steps such as Chain of Thought (CoT) and Program of Thought (PoT) improve performance but often produce undesirable solutions, especially in algorithmic domains. We introduce Per-Instance Program Synthesis (PIPS), a method that generates and refines programs at the instance-level using structural feedback without relying on task-specific guidance or explicit test cases. To further improve performance, PIPS incorporates a confidence metric that dynamically chooses between direct inference and program synthesis on a per-instance basis. Experiments across three frontier LLMs and 30 benchmarks including all tasks of Big Bench Extra Hard (BBEH), visual question answering tasks, relational reasoning tasks, and mathematical reasoning tasks show that PIPS improves the absolute harmonic mean accuracy by up to 8.6% and 9.4% compared to PoT and CoT respectively, and reduces undesirable program generations by 65.1% on the algorithmic tasks compared to PoT with Gemini-2.0-Flash.

Method: Residual disentanglement method that probes LLMs to identify feature-specific layers, then iteratively regresses out lower-level representations to produce four nearly orthogonal embeddings for lexicon, syntax, meaning, and reasoning.

Result: The reasoning embedding uniquely predicts neural activity variance not explained by other features, recruits visual regions beyond language areas, peaks later (~350-400ms), and shows standard LLM embeddings primarily reflect shallow features, masking deeper cognitive contributions.

Conclusion: Disentangled embeddings reveal distinct neural processing for reasoning that occurs later in the hierarchy and extends beyond classical language areas, while standard LLM embeddings are misleading as they predominantly reflect shallow linguistic features.

Abstract: Understanding how the human brain progresses from processing simple linguistic inputs to performing high-level reasoning is a fundamental challenge in neuroscience. While modern large language models (LLMs) are increasingly used to model neural responses to language, their internal representations are highly “entangled,” mixing information about lexicon, syntax, meaning, and reasoning. This entanglement biases conventional brain encoding analyses toward linguistically shallow features (e.g., lexicon and syntax), making it difficult to isolate the neural substrates of cognitively deeper processes. Here, we introduce a residual disentanglement method that computationally isolates these components. By first probing an LM to identify feature-specific layers, our method iteratively regresses out lower-level representations to produce four nearly orthogonal embeddings for lexicon, syntax, meaning, and, critically, reasoning. We used these disentangled embeddings to model intracranial (ECoG) brain recordings from neurosurgical patients listening to natural speech. We show that: 1) This isolated reasoning embedding exhibits unique predictive power, accounting for variance in neural activity not explained by other linguistic features and even extending to the recruitment of visual regions beyond classical language areas. 2) The neural signature for reasoning is temporally distinct, peaking later (~350-400ms) than signals related to lexicon, syntax, and meaning, consistent with its position atop a processing hierarchy. 3) Standard, non-disentangled LLM embeddings can be misleading, as their predictive success is primarily attributable to linguistically shallow features, masking the more subtle contributions of deeper cognitive processing.

Method: Adapted Needle-in-a-Haystack retrieval framework with Flip-style re-evaluation prompts to simulate answer-flipping scenarios, then identified and masked specific non-retrieval attention heads that disproportionately attend to misleading tokens.

Result: Masking identified attention heads reduced flip behavior by up to 15% without introducing incoherence or overcorrection, though trade-offs were observed in downstream tasks.

Conclusion: The findings contribute to mechanistic interpretability and present a simple yet effective technique for mitigating uncertainty-driven failure modes in LLMs, identifying that retrieval heads are not primarily responsible for avoiding uncertainty.

Abstract: Despite their impressive capabilities, Large Language Models (LLMs) exhibit unwanted uncertainty, a phenomenon where a model changes a previously correct answer into an incorrect one when re-prompted. This behavior undermines trust and poses serious risks in high-stakes domains. In this work, we investigate the mechanisms that drive this phenomenon. We adapt the Needle-in-a-Haystack retrieval framework and integrate a Flip-style re-evaluation prompt to simulate realistic answer-flipping scenarios. We find that retrieval heads are not primarily responsible for avoiding uncertainty. Instead, we identify a small set of non-retrieval attention heads that disproportionately attend to misleading tokens in uncertain contexts. Masking these heads yields significant improvements, reducing flip behavior by up to 15% without introducing incoherence or overcorrection. However, when tested for downstream tasks, we observe trade-offs with flip behavior. Our findings contribute to the growing field of mechanistic interpretability and present a simple yet effective technique for mitigating uncertainty-driven failure modes in LLMs.

Method: Created dataset combining authentic New York Times articles with synthetic versions generated by multiple state-of-the-art LLMs (Gemma-2-9b, Mistral-7B, Qwen-2-72B, LLaMA-8B, Yi-Large, GPT-4-o) using original article abstracts as prompts.

Result: Established baseline results: 58.35% accuracy for distinguishing human-written from AI-generated text, and 8.92% accuracy for attributing AI texts to their generating models.

Conclusion: The dataset bridges real-world journalistic content with modern generative models to catalyze development of robust detection and attribution methods, fostering trust and transparency in generative AI.

Abstract: The rapid advancement of large language models (LLMs) has led to increasingly human-like AI-generated text, raising concerns about content authenticity, misinformation, and trustworthiness. Addressing the challenge of reliably detecting AI-generated text and attributing it to specific models requires large-scale, diverse, and well-annotated datasets. In this work, we present a comprehensive dataset comprising over 58,000 text samples that combine authentic New York Times articles with synthetic versions generated by multiple state-of-the-art LLMs including Gemma-2-9b, Mistral-7B, Qwen-2-72B, LLaMA-8B, Yi-Large, and GPT-4-o. The dataset provides original article abstracts as prompts, full human-authored narratives. We establish baseline results for two key tasks: distinguishing human-written from AI-generated text, achieving an accuracy of 58.35%, and attributing AI texts to their generating models with an accuracy of 8.92%. By bridging real-world journalistic content with modern generative models, the dataset aims to catalyze the development of robust detection and attribution methods, fostering trust and transparency in the era of generative AI. Our dataset is available at: https://huggingface.co/datasets/gsingh1-py/train.

Method: The authors characterize synchronization requirements for correctness, present EQSPEC to verify correctness and measure realignment overhead, and introduce EXSPEC which maintains a sliding pool of sequences and dynamically forms same-length groups to reduce realignment overhead while preserving speculative speedups.

Result: On SpecBench dataset across multiple target/draft model pairs, the approach achieves up to 3x throughput improvement at batch size 8 compared to batch size 1, with efficient scaling through batch size 8 while maintaining 95% output equivalence. Realignment consumes 40% of overhead in baseline implementations.

Conclusion: The proposed method successfully addresses the ragged tensor problem in batch speculative decoding, enabling efficient batch processing with significant throughput improvements while maintaining output correctness, requiring no custom kernels and integrating cleanly with existing inference stacks.

Abstract: Speculative decoding speeds up LLM inference by using a small draft model to propose multiple tokens that a target model verifies in parallel. Extending this idea to batches is essential for production serving, but it introduces the ragged tensor problem: sequences in the same batch accept different numbers of draft tokens, breaking right-alignment and corrupting position IDs, attention masks, and KV-cache state. We show that several existing batch implementations violate output equivalence-the fundamental requirement that speculative decoding must produce identical token sequences to standard autoregressive generation. These violations occur precisely due to improper handling of the ragged tensor problem. In response, we (1) characterize the synchronization requirements that guarantee correctness, (2) present a correctness-first batch speculative decoding EQSPEC that exposes realignment as consuming 40% of overhead, and (3) introduce EXSPEC, which maintains a sliding pool of sequences and dynamically forms same-length groups, to reduce the realignment overhead while preserving per-sequence speculative speedups. On the SpecBench dataset, across Vicuna-7B/68M, Qwen3-8B/0.6B, and GLM-4-9B/0.6B target/draft pairs, our approach achieves up to 3$\times$ throughput improvement at batch size 8 compared to batch size 1, with efficient scaling through batch size 8, while maintaining 95% output equivalence. Our method requires no custom kernels and integrates cleanly with existing inference stacks. Our code is available at https://github.com/eBay/spec_dec.

Method: Lanser-CLI introduces: (1) robust addressing via Selector DSL with relocation algorithm, (2) deterministic Analysis Bundles with stable content hashes, (3) safety envelope for mutating operations with preview and transactional apply, and (4) process-reward functional derived from Language Server facts.

Result: The system provides deterministic workflows under frozen snapshots with monotonicity properties for process rewards, making it suitable for process supervision and counterfactual analysis. It enables machine-checked, step-wise signals that align agent planning with program reality.

Conclusion: Language servers provide not only structural information but also actionable process rewards. Lanser-CLI successfully mediates LSP servers to deliver deterministic, replayable workflows that address LLM limitations and enable reliable coding agent operations.

Abstract: Large language models routinely hallucinate APIs and mislocalize edits, while language servers compute verified, IDE-grade facts about real code. We present Lanser-CLI, a CLI-first orchestration layer that pins and mediates a Language Server Protocol (LSP) server for coding agents and CI, exposing deterministic, replayable workflows. Our position is that language servers provide not only structural information (definitions, references, types, diagnostics) but also an actionable process reward: machine-checked, step-wise signals that align an agent’s planning loop with program reality. In this work, Lanser-CLI contributes: (i) a robust addressing scheme beyond brittle “file:line:col” via a Selector DSL (symbolic, AST-path, and content-anchored selectors) with a principled relocation algorithm; (ii) deterministic Analysis Bundles that normalize Language Server responses and capture environment/capability metadata with stable content hashes; (iii) a safety envelope for mutating operations (rename, code actions) with preview, workspace jails, and Git-aware, transactional apply; and (iv) a process-reward functional derived from Language Server facts (diagnostic deltas, disambiguation confidence, and safe-apply checks) that is computable online and replayable offline. We formalize determinism under frozen snapshots and establish a monotonicity property for the process reward, making it suitable for process supervision and counterfactual analysis. Project Page: https://github.com/yifanzhang-pro/lanser-cli

Method: Created Infinity-Chat dataset with 26K diverse, real-world open-ended user queries and 31,250 human annotations. Developed comprehensive taxonomy with 6 top-level categories and 17 subcategories for characterizing open-ended prompts. Conducted large-scale study of mode collapse in LMs.

Result: Revealed pronounced Artificial Hivemind effect with (1) intra-model repetition and (2) inter-model homogeneity. Found that LMs, reward models, and LM judges are less calibrated to human ratings on generations that elicit differing individual preferences, despite comparable overall quality.

Conclusion: Infinity-Chat provides the first large-scale resource for systematically studying real-world open-ended queries to LMs, offering critical insights to mitigate long-term AI safety risks posed by the Artificial Hivemind effect.

Abstract: Language models (LMs) often struggle to generate diverse, human-like creative content, raising concerns about the long-term homogenization of human thought through repeated exposure to similar outputs. Yet scalable methods for evaluating LM output diversity remain limited, especially beyond narrow tasks such as random number or name generation, or beyond repeated sampling from a single model. We introduce Infinity-Chat, a large-scale dataset of 26K diverse, real-world, open-ended user queries that admit a wide range of plausible answers with no single ground truth. We introduce the first comprehensive taxonomy for characterizing the full spectrum of open-ended prompts posed to LMs, comprising 6 top-level categories (e.g., brainstorm & ideation) that further breaks down to 17 subcategories. Using Infinity-Chat, we present a large-scale study of mode collapse in LMs, revealing a pronounced Artificial Hivemind effect in open-ended generation of LMs, characterized by (1) intra-model repetition, where a single model consistently generates similar responses, and more so (2) inter-model homogeneity, where different models produce strikingly similar outputs. Infinity-Chat also includes 31,250 human annotations, across absolute ratings and pairwise preferences, with 25 independent human annotations per example. This enables studying collective and individual-specific human preferences in response to open-ended queries. Our findings show that LMs, reward models, and LM judges are less well calibrated to human ratings on model generations that elicit differing idiosyncratic annotator preferences, despite maintaining comparable overall quality. Overall, INFINITY-CHAT presents the first large-scale resource for systematically studying real-world open-ended queries to LMs, revealing critical insights to guide future research for mitigating long-term AI safety risks posed by the Artificial Hivemind.

Method: Propose TAG - a lightweight data augmentation strategy that adds tags or tag definitions to QA prompts to boost LLM performance in long-context scenarios without changing retrieved documents.

Result: TAG achieves consistent performance gains: up to 17% for 32K token contexts and 2.9% improvement in complex reasoning for multi-hop queries requiring knowledge across wide text spans.

Conclusion: Tagging context or adding tag definitions to prompts effectively enhances LLM performance on long-context QA tasks, providing a simple yet powerful alternative to more complex approaches like RAG.

Abstract: Recent investigations into effective context lengths of modern flagship large language models (LLMs) have revealed major limitations in effective question answering (QA) and reasoning over long and complex contexts for even the largest and most impressive cadre of models. While approaches like retrieval-augmented generation (RAG) and chunk-based re-ranking attempt to mitigate this issue, they are sensitive to chunking, embedding and retrieval strategies and models, and furthermore, rely on extensive pre-processing, knowledge acquisition and indexing steps. In this paper, we propose Tagging-Augmented Generation (TAG), a lightweight data augmentation strategy that boosts LLM performance in long-context scenarios, without degrading and altering the integrity and composition of retrieved documents. We validate our hypothesis by augmenting two challenging and directly relevant question-answering benchmarks – NoLima and NovelQA – and show that tagging the context or even just adding tag definitions into QA prompts leads to consistent performance gains over the baseline – up to 17% for 32K token contexts, and 2.9% in complex reasoning question-answering for multi-hop queries requiring knowledge across a wide span of text. Additional details are available at https://sites.google.com/view/tag-emnlp.

Method: Integrates large-scale long-form datasets (LongHalluQA), multi-agent verification mechanisms (MAD-Fact debate-based system), and weighted evaluation metrics with fact importance hierarchy to capture varying significance of claims.

Result: Experiments on two benchmarks show larger LLMs generally maintain higher factual consistency, while domestic models excel on Chinese content.

Conclusion: Provides a structured framework for evaluating and enhancing factual reliability in long-form LLM outputs, guiding their safe deployment in sensitive domains.

Abstract: The widespread adoption of Large Language Models (LLMs) raises critical concerns about the factual accuracy of their outputs, especially in high-risk domains such as biomedicine, law, and education. Existing evaluation methods for short texts often fail on long-form content due to complex reasoning chains, intertwined perspectives, and cumulative information. To address this, we propose a systematic approach integrating large-scale long-form datasets, multi-agent verification mechanisms, and weighted evaluation metrics. We construct LongHalluQA, a Chinese long-form factuality dataset; and develop MAD-Fact, a debate-based multi-agent verification system. We introduce a fact importance hierarchy to capture the varying significance of claims in long-form texts. Experiments on two benchmarks show that larger LLMs generally maintain higher factual consistency, while domestic models excel on Chinese content. Our work provides a structured framework for evaluating and enhancing factual reliability in long-form LLM outputs, guiding their safe deployment in sensitive domains.

Method: Custom LLMs built on sentence-level embeddings with five different embedding types under data-efficient training to prevent overfitting.

Result: Models achieved human-level performance (correlation >0.65 with expert ratings) and surpassed average human-human rater correlation. Aggregate scores aligned with teacher value-added measures.

Conclusion: Specialized sentence-embedding models provide a viable methodology for AI-driven instructional measurement, offering scalable and reliable feedback for educator development.

Abstract: Objective and scalable measurement of teaching quality is a persistent challenge in education. While Large Language Models (LLMs) offer potential, general-purpose models have struggled to reliably apply complex, authentic classroom observation instruments. This paper uses custom LLMs built on sentence-level embeddings, an architecture better suited for the long-form, interpretive nature of classroom transcripts than conventional subword tokenization. We systematically evaluate five different sentence embeddings under a data-efficient training regime designed to prevent overfitting. Our results demonstrate that these specialized models can achieve human-level and even super-human performance with expert human ratings above 0.65 and surpassing the average human-human rater correlation. Further, through analysis of annotation context windows, we find that more advanced models-those better aligned with human judgments-attribute a larger share of score variation to lesson-level features rather than isolated utterances, challenging the sufficiency of single-turn annotation paradigms. Finally, to assess external validity, we find that aggregate model scores align with teacher value-added measures, indicating they are capturing features relevant to student learning. However, this trend does not hold at the individual item level, suggesting that while the models learn useful signals, they have not yet achieved full generalization. This work establishes a viable and powerful new methodology for AI-driven instructional measurement, offering a path toward providing scalable, reliable, and valid feedback for educator development.

Method: Used behavioral probing and intervention-based methods to evaluate ICL on knowledge-based tasks across different LLM architectures.

Result: Function vectors for ICL are primarily located in self-attention and Mamba layers; Mamba2 uses different mechanisms from FVs; FVs are more important for parametric knowledge retrieval than contextual understanding.

Conclusion: LLMs of different architectures can achieve similar task performance through different internal mechanisms, highlighting the need for combined behavioral and mechanistic analysis to understand LLM capabilities.

Abstract: We perform in-depth evaluations of in-context learning (ICL) on state-of-the-art transformer, state-space, and hybrid large language models over two categories of knowledge-based ICL tasks. Using a combination of behavioral probing and intervention-based methods, we have discovered that, while LLMs of different architectures can behave similarly in task performance, their internals could remain different. We discover that function vectors (FVs) responsible for ICL are primarily located in the self-attention and Mamba layers, and speculate that Mamba2 uses a different mechanism from FVs to perform ICL. FVs are more important for ICL involving parametric knowledge retrieval, but not for contextual knowledge understanding. Our work contributes to a more nuanced understanding across architectures and task types. Methodologically, our approach also highlights the importance of combining both behavioural and mechanistic analyses to investigate LLM capabilities.

Method: Built using LangChain framework and Large Language Models to create a conversational agent with real-time grammar feedback and exercise generation.

Result: The system demonstrated strong usability, positive learning outcomes, and encouraging learner engagement.

Conclusion: LangLingual successfully addresses limitations in language education through AI-powered conversational feedback and practice.

Abstract: Language educators strive to create a rich experience for learners, while they may be restricted in the extend of feedback and practice they can provide. We present the design and development of LangLingual, a conversational agent built using the LangChain framework and powered by Large Language Models. The system is specifically designed to provide real-time, grammar-focused feedback, generate context-aware language exercises and track learner proficiency over time. The paper discusses the architecture, implementation and evaluation of LangLingual in detail. The results indicate strong usability, positive learning outcomes and encouraging learner engagement.

Method: End-to-end RL framework with three principles: diverse training across verifiable/non-verifiable domains, flexible judgment formats (pointwise, pairwise, listwise), and iterative RL bootstrapping from initial model without distillation.

Result: Surpasses reasoning-based judges by up to 6.4% (pointwise) and 7.7% (pairwise), achieves listwise performance comparable to Claude-Opus-4 with only 8B parameters, and TIR-Judge-Zero matches distilled variants’ performance.

Conclusion: Tool-augmented judges can self-evolve through iterative reinforcement learning without requiring distilled judge trajectories, demonstrating the effectiveness of tool-integrated reasoning for LLM evaluation.

Abstract: Large Language Models (LLMs) are widely used as judges to evaluate response quality, providing a scalable alternative to human evaluation. However, most LLM judges operate solely on intrinsic text-based reasoning, limiting their ability to verify complex constraints or perform accurate computation. Motivated by the success of tool-integrated reasoning (TIR) in numerous tasks, we propose TIR-Judge, an end-to-end RL framework for training LLM judges that integrates a code executor for precise evaluation. TIR-Judge is built on three principles: (i) diverse training across verifiable and non-verifiable domains, (ii) flexible judgment formats (pointwise, pairwise, listwise), and (iii) iterative RL that bootstraps directly from the initial model without distillation. On seven public benchmarks, TIR-Judge surpasses strong reasoning-based judges by up to 6.4% (pointwise) and 7.7% (pairwise), and achieves listwise performance comparable to Claude-Opus-4 despite having only 8B parameters. Remarkably, TIR-Judge-Zero - trained entirely without distilled judge trajectories, matches the performance of distilled variants, demonstrating that tool-augmented judges can self-evolve through iterative reinforcement learning.

Method: Knocking-heads attention applies a shared, diagonally-initialized projection matrix across all heads to facilitate cross-head feature-level interactions before scaled dot-product attention.

Result: KHA achieves superior and more stable training dynamics, better performance across downstream tasks compared to baseline attention mechanisms, with only minimal parameter and FLOP overhead.

Conclusion: KHA effectively enhances attention mechanisms by enabling cross-head interactions while maintaining efficiency, making it a promising drop-in replacement for existing attention variants.

Abstract: Multi-head attention (MHA) has become the cornerstone of modern large language models, enhancing representational capacity through parallel attention heads. However, increasing the number of heads inherently weakens individual head capacity, and existing attention mechanisms - whether standard MHA or its variants like grouped-query attention (GQA) and grouped-tied attention (GTA) - simply concatenate outputs from isolated heads without strong interaction. To address this limitation, we propose knocking-heads attention (KHA), which enables attention heads to “knock” on each other - facilitating cross-head feature-level interactions before the scaled dot-product attention. This is achieved by applying a shared, diagonally-initialized projection matrix across all heads. The diagonal initialization preserves head-specific specialization at the start of training while allowing the model to progressively learn integrated cross-head representations. KHA adds only minimal parameters and FLOPs and can be seamlessly integrated into MHA, GQA, GTA, and other attention variants. We validate KHA by training a 6.1B parameter MoE model (1.01B activated) on 1T high-quality tokens. Compared to baseline attention mechanisms, KHA brings superior and more stable training dynamics, achieving better performance across downstream tasks.

Method: Proposes Quality-Aware Translation Tagging (QTT-RAG) that explicitly evaluates translation quality along three dimensions and attaches scores as metadata, preserving original content integrity.

Result: Outperforms CrossRAG and DKM-RAG baselines on XORQA and MKQA benchmarks across 6 LLMs (2.4B-14B parameters) for Korean, Finnish, and Chinese, preserving factual integrity while enabling informed generator decisions.

Conclusion: QTT-RAG offers a practical and robust solution for effectively using cross-lingual documents in low-resource settings with limited native language documents across multilingual domains.

Abstract: Multilingual Retrieval-Augmented Generation (mRAG) often retrieves English documents and translates them into the query language for low-resource settings. However, poor translation quality degrades response generation performance. Existing approaches either assume sufficient translation quality or utilize the rewriting method, which introduces factual distortion and hallucinations. To mitigate these problems, we propose Quality-Aware Translation Tagging in mRAG (QTT-RAG), which explicitly evaluates translation quality along three dimensions-semantic equivalence, grammatical accuracy, and naturalness&fluency-and attach these scores as metadata without altering the original content. We evaluate QTT-RAG against CrossRAG and DKM-RAG as baselines in two open-domain QA benchmarks (XORQA, MKQA) using six instruction-tuned LLMs ranging from 2.4B to 14B parameters, covering two low-resource languages (Korean and Finnish) and one high-resource language (Chinese). QTT-RAG outperforms the baselines by preserving factual integrity while enabling generator models to make informed decisions based on translation reliability. This approach allows for effective usage of cross-lingual documents in low-resource settings with limited native language documents, offering a practical and robust solution across multilingual domains.

Method: Survey approach analyzing optimization frameworks including data curation, training strategies, and model architecture optimization for mid-training, with examination of mainstream model implementations.

Result: Provides a comprehensive taxonomy of mid-training, formal definitions, and actionable insights showing how it serves as a distinct critical stage in progressive LLM capability development.

Conclusion: Mid-training is a crucial distinct stage in LLM development that systematically bridges pre-training and post-training, enabling targeted enhancement of specific capabilities while preserving foundational competencies.

Abstract: Recent advances in foundation models have highlighted the significant benefits of multi-stage training, with a particular emphasis on the emergence of mid-training as a vital stage that bridges pre-training and post-training. Mid-training is distinguished by its use of intermediate data and computational resources, systematically enhancing specified capabilities such as mathematics, coding, reasoning, and long-context extension, while maintaining foundational competencies. This survey provides a formal definition of mid-training for large language models (LLMs) and investigates optimization frameworks that encompass data curation, training strategies, and model architecture optimization. We analyze mainstream model implementations in the context of objective-driven interventions, illustrating how mid-training serves as a distinct and critical stage in the progressive development of LLM capabilities. By clarifying the unique contributions of mid-training, this survey offers a comprehensive taxonomy and actionable insights, supporting future research and innovation in the advancement of LLMs.

Method: Proposes four specialized prompt components: Global Domain Prompt (dataset-level context), Local Domain Prompt (recent trends), Statistical Prompt (ACF/PACF insights), and Temporal Prompt (Fourier analysis), combined with raw embeddings through cross-modality alignment.

Result: Extensive experiments across eight datasets show MAP4TS consistently outperforms state-of-the-art LLM-based methods. GPT-2 backbones with structured prompts outperform larger models like LLaMA in long-term forecasting.

Conclusion: Prompt-aware designs significantly enhance performance stability, and structured prompts enable smaller models to outperform larger ones in time-series forecasting tasks.

Abstract: Recent advances have investigated the use of pretrained large language models (LLMs) for time-series forecasting by aligning numerical inputs with LLM embedding spaces. However, existing multimodal approaches often overlook the distinct statistical properties and temporal dependencies that are fundamental to time-series data. To bridge this gap, we propose MAP4TS, a novel Multi-Aspect Prompting Framework that explicitly incorporates classical time-series analysis into the prompt design. Our framework introduces four specialized prompt components: a Global Domain Prompt that conveys dataset-level context, a Local Domain Prompt that encodes recent trends and series-specific behaviors, and a pair of Statistical and Temporal Prompts that embed handcrafted insights derived from autocorrelation (ACF), partial autocorrelation (PACF), and Fourier analysis. Multi-Aspect Prompts are combined with raw time-series embeddings and passed through a cross-modality alignment module to produce unified representations, which are then processed by an LLM and projected for final forecasting. Extensive experiments across eight diverse datasets show that MAP4TS consistently outperforms state-of-the-art LLM-based methods. Our ablation studies further reveal that prompt-aware designs significantly enhance performance stability and that GPT-2 backbones, when paired with structured prompts, outperform larger models like LLaMA in long-term forecasting tasks.

Method: Tested two hierarchical organization approaches across three large language models, with comprehensive evaluations using automated metrics, model-based metrics, and domain expert assessment of multiple quality dimensions.

Result: Human experts preferred model-generated summaries over human-written ones. Hierarchical approaches preserved factuality, coverage, and coherence while increasing human preference. GPT-4 judgments aligned well with human evaluations on objective criteria.

Conclusion: Hierarchical structures improve clarity of medical summaries and human preference while maintaining content coverage, offering a practical enhancement for medical multi-document summarization.

Abstract: Medical multi-document summarization (MDS) is a complex task that requires effectively managing cross-document relationships. This paper investigates whether incorporating hierarchical structures in the inputs of MDS can improve a model’s ability to organize and contextualize information across documents compared to traditional flat summarization methods. We investigate two ways of incorporating hierarchical organization across three large language models (LLMs), and conduct comprehensive evaluations of the resulting summaries using automated metrics, model-based metrics, and domain expert evaluation of preference, understandability, clarity, complexity, relevance, coverage, factuality, and coherence. Our results show that human experts prefer model-generated summaries over human-written summaries. Hierarchical approaches generally preserve factuality, coverage, and coherence of information, while also increasing human preference for summaries. Additionally, we examine whether simulated judgments from GPT-4 align with human judgments, finding higher agreement along more objective evaluation facets. Our findings demonstrate that hierarchical structures can improve the clarity of medical summaries generated by models while maintaining content coverage, providing a practical way to improve human preference for generated summaries.

Method: Joint training on data from 73 languages using a frequency-weighted, lemma-disjoint train-dev-test resampling procedure on Universal Dependencies treebanks.

Result: Outperforms monolingual baselines in most languages, is lightweight, robust to unseen words, and eliminates the need for managing multiple separate models.

Conclusion: Multilingual modeling is effective for inflection tasks and offers practical deployment benefits by replacing dozens of monolingual models with a single unified system.

Abstract: We present a compact, single-model approach to multilingual inflection, the task of generating inflected word forms from base lemmas to express grammatical categories. Our model, trained jointly on data from 73 languages, is lightweight, robust to unseen words, and outperforms monolingual baselines in most languages. This demonstrates the effectiveness of multilingual modeling for inflection and highlights its practical benefits: simplifying deployment by eliminating the need to manage and retrain dozens of separate monolingual models. In addition to the standard SIGMORPHON shared task benchmarks, we evaluate our monolingual and multilingual models on 73 Universal Dependencies (UD) treebanks, extracting lemma-tag-form triples and their frequency counts. To ensure realistic data splits, we introduce a novel frequency-weighted, lemma-disjoint train-dev-test resampling procedure. Our work addresses the lack of an open-source, general-purpose, multilingual morphological inflection system capable of handling unseen words across a wide range of languages, including Czech. All code is publicly released at: https://github.com/tomsouri/multilingual-inflection.

Method: TopLoRA introduces token-wise projected LoRA with weights expressed as BΣ_XA, where A and B are low-rank matrices, and Σ_X is a diagonal matrix generated from each input token X, enabling dynamic weight adjustment per token.

Result: Extensive experiments across multiple models and datasets show that TopLoRA consistently outperforms LoRA and its variants.

Conclusion: TopLoRA achieves more granular adaptation by learning token-wise LoRA weights without increasing rank, providing better performance than standard LoRA approaches.

Abstract: Low-rank adaptation (LoRA) is a parameter-efficient fine-tuning (PEFT) method widely used in large language models (LLMs). LoRA essentially describes the projection of an input space into a low-dimensional output space, with the dimensionality determined by the LoRA rank. In standard LoRA, all input tokens share the same weights and undergo an identical input-output projection. This limits LoRA’s ability to capture token-specific information due to the inherent semantic differences among tokens. To address this limitation, we propose Token-wise Projected Low-Rank Adaptation (TopLoRA), which dynamically adjusts LoRA weights according to the input token, thereby learning token-wise input-output projections in an end-to-end manner. Formally, the weights of TopLoRA can be expressed as $B\Sigma_X A$, where $A$ and $B$ are low-rank matrices (as in standard LoRA), and $\Sigma_X$ is a diagonal matrix generated from each input token $X$. Notably, TopLoRA does not increase the rank of LoRA weights but achieves more granular adaptation by learning token-wise LoRA weights (i.e., token-wise input-output projections). Extensive experiments across multiple models and datasets demonstrate that TopLoRA consistently outperforms LoRA and its variants. The code is available at https://github.com/Leopold1423/toplora-neurips25.

Method: Three frequency-aware approaches: (1) frequency-weighted train-dev-test splits combining lemma-disjoint and frequency distribution strategies, (2) token accuracy evaluation that weights frequent words more heavily, and (3) frequency-aware training that incorporates word frequency into sampling.

Result: Frequency-aware training outperformed uniform sampling in 26 out of 43 languages tested, demonstrating the effectiveness of incorporating frequency information.

Conclusion: Incorporating corpus frequency information through multiple dimensions (data splitting, evaluation, and training) improves morphological inflection systems and better reflects real-world language usage patterns.

Abstract: The traditional approach to morphological inflection (the task of modifying a base word (lemma) to express grammatical categories) has been, for decades, to consider lexical entries of lemma-tag-form triples uniformly, lacking any information about their frequency distribution. However, in production deployment, one might expect the user inputs to reflect a real-world distribution of frequencies in natural texts. With future deployment in mind, we explore the incorporation of corpus frequency information into the task of morphological inflection along three key dimensions during system development: (i) for train-dev-test split, we combine a lemma-disjoint approach, which evaluates the model’s generalization capabilities, with a frequency-weighted strategy to better reflect the realistic distribution of items across different frequency bands in training and test sets; (ii) for evaluation, we complement the standard type accuracy (often referred to simply as accuracy), which treats all items equally regardless of frequency, with token accuracy, which assigns greater weight to frequent words and better approximates performance on running text; (iii) for training data sampling, we introduce a method novel in the context of inflection, frequency-aware training, which explicitly incorporates word frequency into the sampling process. We show that frequency-aware training outperforms uniform sampling in 26 out of 43 languages.

Method: Neural-symbolic framework with symbolic purification (pruning noisy samples using statistical priors) and neural reconstruction (synthesizing enriched instruction-response pairs using latent representations and model knowledge).

Result: ENTP-augmented datasets from low-quality data outperform 13 established data-selection baselines across five benchmarks and surpass fine-tuning on full original dataset (~300K examples).

Conclusion: Low-quality data has untapped potential, and intelligent purification/synthesis is crucial for efficient instruction alignment.

Abstract: Supervised Fine-Tuning (SFT) adapts pre-trained Large Language Models (LLMs) to domain-specific instructions by training on a carefully curated subset of high-quality instruction-response pairs, typically drawn from a larger dataset that often contains many low-quality or noisy samples. However, existing quality-first paradigms often overlook valuable signals in discarded low-quality data and rely on imperfect quality filters. We introduce ENTP (Enhancing low-quality SFT data via Neural-symbolic Text Purge-Mix), a framework that revitalizes low-quality corpora through symbolic purification and neural reconstruction. The symbolic module identifies and prunes noisy samples based on statistical priors, while the neural component synthesizes enriched instruction-response pairs by leveraging latent representations and model knowledge. This neural-symbolic synergy enhances data informativeness and diversity. Experiments show that ENTP-augmented datasets, constructed exclusively from low-quality data, outperform 13 established data-selection baselines across five instruction-following benchmarks, and even surpass fine-tuning on the full original dataset (approximately 300K examples). Our results highlight the untapped potential of low-quality data and underscore the importance of intelligent purification and synthesis for efficient instruction alignment.

Method: DSR framework with two stages: first transforms brief outlines into rich novel-style prose, then refines into professionally formatted screenplays. Uses hybrid data synthesis (reverse and forward) to address paired training data scarcity.

Result: Blind evaluations by professional screenwriters show 75% win rate against strong baselines like Gemini-2.5-Pro and reaches 82.7% of human-level performance.

Conclusion: Decomposed generation architecture with tailored data synthesis effectively specializes LLMs in complex creative domains like screenplay writing.

Abstract: The screenplay serves as the foundation for television production, defining narrative structure, character development, and dialogue. While Large Language Models (LLMs) show great potential in creative writing, direct end-to-end generation approaches often fail to produce well-crafted screenplays. We argue this failure stems from forcing a single model to simultaneously master two disparate capabilities: creative narrative construction and rigid format adherence. The resulting outputs may mimic superficial style but lack the deep structural integrity and storytelling substance required for professional use. To enable LLMs to generate high-quality screenplays, we introduce Dual-Stage Refinement (DSR), a decomposed framework that decouples creative narrative generation from format conversion. The first stage transforms a brief outline into rich, novel-style prose. The second stage refines this narrative into a professionally formatted screenplay. This separation enables the model to specialize in one distinct capability at each stage. A key challenge in implementing DSR is the scarcity of paired outline-to-novel training data. We address this through hybrid data synthesis: reverse synthesis deconstructs existing screenplays into structured inputs, while forward synthesis leverages these inputs to generate high-quality narrative texts as training targets. Blind evaluations by professional screenwriters show that DSR achieves a 75% win rate against strong baselines like Gemini-2.5-Pro and reaches 82.7% of human-level performance. Our work demonstrates that decomposed generation architecture with tailored data synthesis effectively specializes LLMs in complex creative domains.

Method: Uses Contrastive Learning to generate meaningful embeddings for both code and natural language task descriptions, enabling similarity scoring between generated code and task descriptions.

Result: MATCH achieves stronger correlations with functional correctness and human preference than existing metrics across multiple programming languages.

Conclusion: MATCH provides an effective reference-free evaluation method for AI-generated code that better captures functional alignment with developer intent.

Abstract: AI-based code generation is increasingly prevalent, with GitHub Copilot estimated to generate 46% of the code on GitHub. Accurately evaluating how well generated code aligns with developer intent remains a critical challenge. Traditional evaluation methods, such as unit tests, are often unscalable and costly. Syntactic similarity metrics (e.g., BLEU, ROUGE) fail to capture code functionality, and metrics like CodeBERTScore require reference code, which is not always available. To address the gap in reference-free evaluation, with few alternatives such as ICE-Score, this paper introduces MATCH, a novel reference-free metric. MATCH uses Contrastive Learning to generate meaningful embeddings for code and natural language task descriptions, enabling similarity scoring that reflects how well generated code implements the task. We show that MATCH achieves stronger correlations with functional correctness and human preference than existing metrics across multiple programming languages.

Method: Collected 2,221 authentic multi-turn dialogues from social networking applications and manually annotated 312 dialogues across 8 major models, grounded in social science theories.

Result: SOTA models surpassed human experts in process reasoning under complex social situations but still fall behind humans in reply quality. Chain-of-Thought reasoning degraded LLM performance in social dialogue tasks.

Conclusion: SI-Bench provides a realistic benchmark for evaluating social intelligence in LLMs, revealing current models’ strengths in reasoning but weaknesses in conversational quality compared to humans.

Abstract: As large language models (LLMs) develop anthropomorphic abilities, they are increasingly being deployed as autonomous agents to interact with humans. However, evaluating their performance in realistic and complex social interactions remains a significant challenge. Most previous research built datasets through simulated agent-to-agent interactions, which fails to capture the authentic linguistic styles and relational dynamics found in real human conversations. To address this gap, we introduce SI-Bench, a novel benchmark designed to evaluate aspects of social intelligence in LLMs. Grounded in broad social science theories, SI-Bench contains 2,221 authentic multi-turn dialogues collected from a social networking application. We further selected a subset of 312 dialogues for manual annotation across 8 major models. The experiments show that SOTA models have surpassed the human expert in process reasoning under complex social situations, yet they still fall behind humans in reply quality. Moreover, introducing Chain-of-Thought (CoT) reasoning may degrade the performance of LLMs in social dialogue tasks. All datasets are openly available at https://github.com/SI-Bench/SI-Bench.git.

Method: Uses a trained relation extraction model to discover relations between entities, applies it to medical text corpus to construct drug relationship ontology, and validates extracted relations using a large language model.

Result: LLM agreed with 71% of relations extracted from PubMed abstracts subset. Qualitative analysis revealed ambiguities in medical domain, highlighting relation extraction challenges.

Conclusion: The approach demonstrates feasibility of transfer learning for drug-drug relation extraction and reveals domain-specific challenges through identified ambiguities in medical texts.

Abstract: Relation extraction between drugs plays a crucial role in identifying drug drug interactions and predicting side effects. The advancement of machine learning methods in relation extraction, along with the development of large medical text databases, has enabled the low cost extraction of such relations compared to other approaches that typically require expert knowledge. However, to the best of our knowledge, there are limited datasets specifically designed for drug drug relation extraction currently available. Therefore, employing transfer learning becomes necessary to apply machine learning methods in this domain. In this study, we propose DREAM, a method that first employs a trained relation extraction model to discover relations between entities and then applies this model to a corpus of medical texts to construct an ontology of drug relationships. The extracted relations are subsequently validated using a large language model. Quantitative results indicate that the LLM agreed with 71 of the relations extracted from a subset of PubMed abstracts. Furthermore, our qualitative analysis indicates that this approach can uncover ambiguities in the medical domain, highlighting the challenges inherent in relation extraction in this field.

Method: Developed a lightweight 0.5B-parameter PRM that predicts factual correctness of each generated sentence, conditioned on clinical context and preceding text, fine-tuned on MIMIC-CXR with weakly-supervised labels.

Result: Outperformed existing verification techniques with 7.5% improvement in Matthews Correlation Coefficient and 1.8% in AUROC, showed strong generalization to unseen LVLMs, improved F1-CheXbert scores by 4.5% when filtering low-quality reports, and achieved 7.4% improvement in F1-CheXbert through weighted best-of-N selection.

Conclusion: A lightweight, context-aware PRM provides an effective model-agnostic safety layer for clinical LVLMs without requiring access to internal model activations.

Abstract: Automating radiology report generation with Large Vision-Language Models (LVLMs) holds great potential, yet these models often produce clinically critical hallucinations, posing serious risks. Existing hallucination detection methods frequently lack the necessary sentence-level granularity or robust generalization across different LVLM generators. We introduce a novel approach: a sentence-level Process Reward Model (PRM) adapted for this vision-language task. Our PRM predicts the factual correctness of each generated sentence, conditioned on clinical context and preceding text. When fine-tuned on MIMIC-CXR with weakly-supervised labels, a lightweight 0.5B-parameter PRM outperforms existing verification techniques, demonstrating, for instance, relative improvements of 7.5% in Matthews Correlation Coefficient and 1.8% in AUROC over strong white-box baselines on outputs from one LVLM. Unlike methods reliant on internal model states, our PRM demonstrates strong generalization to an unseen LVLM. We further show its practical utility: PRM scores effectively filter low-quality reports, improving F1-CheXbert scores by 4.5% (when discarding the worst 10% of reports). Moreover, when guiding a novel weighted best-of-N selection process on the MIMIC-CXR test set, our PRM show relative improvements in clinical metrics of 7.4% for F1-CheXbert and 0.6% for BERTScore. These results demonstrate that a lightweight, context-aware PRM provides a model-agnostic safety layer for clinical LVLMs without access to internal activations

Method: Developed a 78-hour annotated Bengali Speech-to-Text corpus (Ben-10) and evaluated speech foundation models in zero-shot and fine-tuned settings across different dialects.

Result: Speech foundation models struggle heavily with regional dialect ASR in both zero-shot and fine-tuned settings. All deep learning methods have difficulty modeling speech data under dialectal variations, but dialect-specific model training alleviates the issue.

Conclusion: The Ben-10 dataset serves as an out-of-distribution resource for ASR modeling under constrained resources, highlighting the challenges of dialectal variations in speech recognition and the need for dialect-specific approaches.

Abstract: Conventional research on speech recognition modeling relies on the canonical form for most low-resource languages while automatic speech recognition (ASR) for regional dialects is treated as a fine-tuning task. To investigate the effects of dialectal variations on ASR we develop a 78-hour annotated Bengali Speech-to-Text (STT) corpus named Ben-10. Investigation from linguistic and data-driven perspectives shows that speech foundation models struggle heavily in regional dialect ASR, both in zero-shot and fine-tuned settings. We observe that all deep learning methods struggle to model speech data under dialectal variations but dialect specific model training alleviates the issue. Our dataset also serves as a out of-distribution (OOD) resource for ASR modeling under constrained resources in ASR algorithms. The dataset and code developed for this project are publicly available

Method: Trained on extensive authentic Arabic sources including historical manuscripts digitized via proprietary Arabic OCR, incorporating scholarly works in linguistics, jurisprudence, hadith, and Quranic exegesis. Uses deep linguistic engineering framework and Practical Closure Architecture to prioritize clarity and decisive guidance.

Result: Mubeen masters Arabic eloquence and enables precise understanding across classical texts, contemporary writing, and regional dialects with focus on user intent and contextually relevant responses. It transforms from information repository to decisive guide.

Conclusion: Mubeen successfully bridges the “Utility Gap Crisis” by providing culturally authentic Arabic language processing with practical utility, aligning with Saudi Vision 2030 goals for cultural preservation and technological advancement.

Abstract: Mubeen is a proprietary Arabic language model developed by MASARAT SA, optimized for deep understanding of Arabic linguistics, Islamic studies, and cultural heritage. Trained on an extensive collection of authentic Arabic sources significantly expanded by digitizing historical manuscripts via a proprietary Arabic OCR engine, the model incorporates seminal scholarly works in linguistics, jurisprudence, hadith, and Quranic exegesis, alongside thousands of academic theses and peer-reviewed research papers. Conditioned through a deep linguistic engineering framework, Mubeen masters not just the meaning but the eloquence of Arabic, enabling precise understanding across classical texts, contemporary writing, and regional dialects with focus on comprehending user intent and delivering accurate, contextually relevant responses. Unlike other Arabic models relying on translated English data that often fail in intent detection or retrieval-augmented generation (RAG), Mubeen uses native Arabic sources to ensure cultural authenticity and accuracy. Its core innovation is the Practical Closure Architecture, designed to solve the “Utility Gap Crisis” where factually correct answers fail to resolve users' core needs, forcing them into frustrating cycles of re-prompting. By prioritizing clarity and decisive guidance, Mubeen transforms from an information repository into a decisive guide, aligning with Saudi Vision 2030. The model’s architecture combines deep heritage specialization with multi-disciplinary expert modules, enabling robust performance across both cultural preservation and general knowledge domains.

Method: Constructed AesCode-358K dataset for instruction-tuning, developed multi-agent reward feedback system for executability and aesthetics evaluation, and created GRPO-AR algorithm for joint optimization of functionality and code aesthetics.

Result: Combining supervised fine-tuning on AesCode-358K with reinforcement learning using agentic reward feedback significantly improves performance on OpenDesign benchmark and enhances results on PandasPlotBench. AesCoder-4B surpasses GPT-4o and GPT-4.1, achieving performance comparable to large 480B-685B parameter models.

Conclusion: The proposed approach effectively enhances code aesthetics in LLM-generated code, demonstrating that focused training on aesthetic aspects can yield substantial improvements even with smaller models.

Abstract: Large Language Models (LLMs) have become valuable assistants for developers in code-related tasks. While LLMs excel at traditional programming tasks such as code generation and bug fixing, they struggle with visually-oriented coding tasks, often producing suboptimal aesthetics. In this paper, we introduce a new pipeline to enhance the aesthetic quality of LLM-generated code. We first construct AesCode-358K, a large-scale instruction-tuning dataset focused on code aesthetics. Next, we propose agentic reward feedback, a multi-agent system that evaluates executability, static aesthetics, and interactive aesthetics. Building on this, we develop GRPO-AR, which integrates these signals into the GRPO algorithm for joint optimization of functionality and code aesthetics. Finally, we develop OpenDesign, a benchmark for assessing code aesthetics. Experimental results show that combining supervised fine-tuning on AesCode-358K with reinforcement learning using agentic reward feedback significantly improves performance on OpenDesign and also enhances results on existing benchmarks such as PandasPlotBench. Notably, our AesCoder-4B surpasses GPT-4o and GPT-4.1, and achieves performance comparable to large open-source models with 480B-685B parameters, underscoring the effectiveness of our approach.

Method: Uses audio, visual, and contextual cues to jointly transcribe each speaker’s speech and cluster them into respective conversations from audio-visual recordings.

Result: Audio-only baselines exceed 100% word error rate, while incorporating visual cues yields substantial 50% improvements, demonstrating the critical importance of multi-modality.

Conclusion: Multi-modal approaches are essential for handling extreme speech overlap in natural conversations, with visual cues providing significant performance gains over audio-only systems.

Abstract: We introduce the task of Multi-Modal Context-Aware Recognition (MCoRec) in the ninth CHiME Challenge, which addresses the cocktail-party problem of overlapping conversations in a single-room setting using audio, visual, and contextual cues. MCoRec captures natural multi-party conversations where the recordings focus on unscripted, casual group chats, leading to extreme speech overlap of up to 100% and highly fragmented conversational turns. The task requires systems to answer the question “Who speaks when, what, and with whom?” by jointly transcribing each speaker’s speech and clustering them into their respective conversations from audio-visual recordings. Audio-only baselines exceed 100% word error rate, whereas incorporating visual cues yields substantial 50% improvements, highlighting the importance of multi-modality. In this manuscript, we present the motivation behind the task, outline the data collection process, and report the baseline systems developed for the MCoRec.

Method: Developed a data-centric pipeline with adaptive data repair (automatically finding/fixing training data errors) and error data augmentation (enhancing erroneous data). Used multi-model collaboration training with different augmented data and ensemble strategies for multiple-choice questions.

Result: Achieved first place in lightweight text-to-SQL models (within 70B parameters). Experiment results and ablation studies demonstrated the effectiveness of the proposed pipeline and multi-model strategies.

Conclusion: The data-centric pipeline and multi-model interactive iterative strategies effectively improve text-to-SQL task accuracy, showing the importance of data quality and model collaboration in this domain.

Abstract: Text-to-SQL tasks have gained attractive improvements since the release of ChatGPT. Among them, agent-based frameworks have been widely used in this field. However, the impact of data-centric strategies on text-to-SQL tasks has rarely been explored. In this paper, we systemically design a fully automated data-centric pipeline for text-to-SQL tasks, including \emph{adaptive data repair}, which can automatically find and fix errors in the training dataset; and \emph{error data augmentation}, where we specifically diffuse and enhance erroneous data predicted by the initially trained models. Meanwhile, we propose a Multi-Model collaboration training schema, aiming to train multiple models with different augmented data, enabling them to possess distinct capabilities and work together to complement each other, because it has been found that the capability of a single fine-tuned model is very limited. Furthermore, we utilize an ensemble strategy to integrate the capabilities of multiple models to solve a multiple-choice question, aiming to further improve the accuracy of text-to-SQL tasks. The experiment results and ablation study have demonstrated the effectiveness of data-centric pipeline and Multi-Model(MM) interactive iterative strategies, achieving first place in lightweight text-to-SQL models (within 70B).

Method: Introduces AdaSearch, a blockwise search strategy with adaptive computational budget allocation using sampling schedules, and AdaBeam for tree-search applications.

Result: AdaSearch outperforms Best-of-N and fine-tuning baselines across eight LLMs, achieving over 10% improvement in win-rates for harmlessness generation, controlled sentiment generation, and mathematical reasoning tasks.

Conclusion: Adaptive allocation of computational effort to critical initial tokens significantly improves LLM alignment performance compared to uniform approaches.

Abstract: LLM alignment remains a critical challenge. Inference-time methods provide a flexible alternative to fine-tuning, but their uniform computational effort often yields suboptimal alignment. We hypothesize that for many alignment tasks, the initial tokens of a response are disproportionately more critical. To leverage this principle, we introduce AdaSearch, a novel blockwise search strategy. It adaptively allocates a fixed computational budget using a sampling schedule, focusing search effort on these critical tokens. We apply AdaSearch to sequential decoding and introduce its tree-search counterpart, AdaBeam. Our comprehensive evaluation across eight LLMs demonstrates that AdaSearch outperforms strong Best-of-N and fine-tuning baselines. Specifically, win-rates improve by over 10% for harmlessness generation, controlled sentiment generation, and for mathematical reasoning tasks relative to Best-of-N.

Method: Created the first QA dataset for BaZi-based persona reasoning using real human experiences categorized into five life domains, and proposed a BaZi-LLM system integrating symbolic reasoning with large language models.

Result: Achieved 30.3%-62.6% accuracy improvement over mainstream LLMs like DeepSeek-v3 and GPT-5-mini. When incorrect BaZi information was used, accuracy dropped by 20%-45%, demonstrating the importance of culturally grounded reasoning.

Conclusion: The integration of culturally grounded symbolic reasoning with LLMs shows strong potential for realistic character simulation and temporally dynamic persona generation.

Abstract: Human-like virtual characters are crucial for games, storytelling, and virtual reality, yet current methods rely heavily on annotated data or handcrafted persona prompts, making it difficult to scale up and generate realistic, contextually coherent personas. We create the first QA dataset for BaZi-based persona reasoning, where real human experiences categorized into wealth, health, kinship, career, and relationships are represented as life-event questions and answers. Furthermore, we propose the first BaZi-LLM system that integrates symbolic reasoning with large language models to generate temporally dynamic and fine-grained virtual personas. Compared with mainstream LLMs such as DeepSeek-v3 and GPT-5-mini, our method achieves a 30.3%-62.6% accuracy improvement. In addition, when incorrect BaZi information is used, our model’s accuracy drops by 20%-45%, showing the potential of culturally grounded symbolic-LLM integration for realistic character simulation.

Method: Two key innovations: (1) Context-integrated Graph extraction that unifies contextual information with nodes/edges, and (2) Topology-enhanced relationship inference that uses graph topology to infer relationships without complex language understanding.

Result: Enables accurate knowledge graph construction with minimal hardware requirements, making automated knowledge extraction practical for deployment in resource-constrained scenarios.

Conclusion: Bridges the gap between automated knowledge extraction and practical deployment while introducing scientifically rigorous methods for optimizing small language model efficiency in structured NLP tasks.

Abstract: The scarcity of high-quality knowledge graphs (KGs) remains a critical bottleneck for downstream AI applications, as existing extraction methods rely heavily on error-prone pattern-matching techniques or resource-intensive large language models (LLMs). While recent tools leverage LLMs to generate KGs, their computational demands limit accessibility for low-resource environments. Our paper introduces LightKGG, a novel framework that enables efficient KG extraction from textual data using small-scale language models (SLMs) through two key technical innovations: (1) Context-integrated Graph extraction integrates contextual information with nodes and edges into a unified graph structure, reducing the reliance on complex semantic processing while maintaining more key information; (2) Topology-enhanced relationship inference leverages the inherent topology of the extracted graph to efficiently infer relationships, enabling relationship discovery without relying on complex language understanding capabilities of LLMs. By enabling accurate KG construction with minimal hardware requirements, this work bridges the gap between automated knowledge extraction and practical deployment scenarios while introducing scientifically rigorous methods for optimizing SLM efficiency in structured NLP tasks.

Method: The benchmark combines US sector-level job postings data with global AI occupational change projections, formatted into forecasting tasks with temporal splits. Evaluates LLMs using task-scaffolded, persona-driven, and hybrid prompting strategies across model families.

Result: Structured task prompts improve forecast stability, while persona prompts work better for short-term trends. Performance varies significantly across sectors and time horizons, highlighting the need for domain-aware prompting and rigorous evaluation.

Conclusion: The released benchmark supports future research on labor forecasting, prompt design, and LLM-based economic reasoning, providing a reproducible testbed for studying AI’s limits and opportunities as a forecasting tool in labor markets.

Abstract: Artificial intelligence is reshaping labor markets, yet we lack tools to systematically forecast its effects on employment. This paper introduces a benchmark for evaluating how well large language models (LLMs) can anticipate changes in job demand, especially in occupations affected by AI. Existing research has shown that LLMs can extract sentiment, summarize economic reports, and emulate forecaster behavior, but little work has assessed their use for forward-looking labor prediction. Our benchmark combines two complementary datasets: a high-frequency index of sector-level job postings in the United States, and a global dataset of projected occupational changes due to AI adoption. We format these data into forecasting tasks with clear temporal splits, minimizing the risk of information leakage. We then evaluate LLMs using multiple prompting strategies, comparing task-scaffolded, persona-driven, and hybrid approaches across model families. We assess both quantitative accuracy and qualitative consistency over time. Results show that structured task prompts consistently improve forecast stability, while persona prompts offer advantages on short-term trends. However, performance varies significantly across sectors and horizons, highlighting the need for domain-aware prompting and rigorous evaluation protocols. By releasing our benchmark, we aim to support future research on labor forecasting, prompt design, and LLM-based economic reasoning. This work contributes to a growing body of research on how LLMs interact with real-world economic data, and provides a reproducible testbed for studying the limits and opportunities of AI as a forecasting tool in the context of labor markets.

Method: Dual approach: rule-based model using hierarchical tree of religious terms from ecotheology literature, and large language models (LLMs) in zero-shot setting, applied to 880,000+ sentences.

Result: Rule-based method consistently labels more sentences as religious than LLMs, highlighting methodological challenges and tension between vocabulary-based vs. context-based definitions of religious language.

Conclusion: The study demonstrates both potential and limitations of computational approaches for analyzing religious language in climate discourse, contributing to digital methods in religious studies.

Abstract: Religious language continues to permeate contemporary discourse, even in ostensibly secular domains such as environmental activism and climate change debates. This paper investigates how explicit and implicit forms of religious language appear in climate-related texts produced by secular and religious nongovernmental organizations (NGOs). We introduce a dual methodological approach: a rule-based model using a hierarchical tree of religious terms derived from ecotheology literature, and large language models (LLMs) operating in a zero-shot setting. Using a dataset of more than 880,000 sentences, we compare how these methods detect religious language and analyze points of agreement and divergence. The results show that the rule-based method consistently labels more sentences as religious than LLMs. These findings highlight not only the methodological challenges of computationally detecting religious language but also the broader tension over whether religious language should be defined by vocabulary alone or by contextual meaning. This study contributes to digital methods in religious studies by demonstrating both the potential and the limitations of approaches for analyzing how the sacred persists in climate discourse.

Method: Proposes a Mixture of Experts framework that integrates multiple SOTA models (xLSTM, enhanced Linear, PatchTST, minGRU) using a Transformer-based gating network to combine their complementary strengths.

Result: Outperforms all existing TSF models on standard benchmarks, surpassing even the latest MoE-based approaches.

Conclusion: The proposed MoE framework effectively combines diverse forecasting models to achieve state-of-the-art performance in time series forecasting.

Abstract: The immense success of the Transformer architecture in Natural Language Processing has led to its adoption in Time Se ries Forecasting (TSF), where superior performance has been shown. However, a recent important paper questioned their effectiveness by demonstrating that a simple single layer linear model outperforms Transformer-based models. This was soon shown to be not as valid, by a better transformer-based model termed PatchTST. More re cently, TimeLLM demonstrated even better results by repurposing a Large Language Model (LLM) for the TSF domain. Again, a follow up paper challenged this by demonstrating that removing the LLM component or replacing it with a basic attention layer in fact yields better performance. One of the challenges in forecasting is the fact that TSF data favors the more recent past, and is sometimes subject to unpredictable events. Based upon these recent insights in TSF, we propose a strong Mixture of Experts (MoE) framework. Our method combines the state-of-the-art (SOTA) models including xLSTM, en hanced Linear, PatchTST, and minGRU, among others. This set of complimentary and diverse models for TSF are integrated in a Trans former based MoE gating network. Our proposed model outperforms all existing TSF models on standard benchmarks, surpassing even the latest approaches based on MoE frameworks.

Method: Proposed Omni-Reward framework includes: Omni-RewardBench benchmark (9 tasks across 5 modalities), Omni-RewardData (248K general + 69K instruction-tuning pairs), and Omni-RewardModel (discriminative and generative RMs).

Result: Achieves strong performance on Omni-RewardBench and other widely used reward modeling benchmarks.

Conclusion: Omni-Reward represents a step toward generalist omni-modal reward modeling that supports free-form preferences across multiple modalities.

Abstract: Reward models (RMs) play a critical role in aligning AI behaviors with human preferences, yet they face two fundamental challenges: (1) Modality Imbalance, where most RMs are mainly focused on text and image modalities, offering limited support for video, audio, and other modalities; and (2) Preference Rigidity, where training on fixed binary preference pairs fails to capture the complexity and diversity of personalized preferences. To address the above challenges, we propose Omni-Reward, a step toward generalist omni-modal reward modeling with support for free-form preferences, consisting of: (1) Evaluation: We introduce Omni-RewardBench, the first omni-modal RM benchmark with free-form preferences, covering nine tasks across five modalities including text, image, video, audio, and 3D; (2) Data: We construct Omni-RewardData, a multimodal preference dataset comprising 248K general preference pairs and 69K instruction-tuning pairs for training generalist omni-modal RMs; (3) Model: We propose Omni-RewardModel, which includes both discriminative and generative RMs, and achieves strong performance on Omni-RewardBench as well as other widely used reward modeling benchmarks.

Method: The researchers experimented with open-source agentic models and proposed Test-Time Scaling (TTS) methods that use confidence scores to determine answer quality and encourage models to retry until reaching satisfactory confidence levels.

Result: Models showed much higher task accuracy at high confidence levels while having near-zero accuracy when confidence was low. The proposed TTS methods significantly reduced token consumption while maintaining competitive performance compared to baseline fixed budget methods.

Conclusion: LLM-based search agents can effectively communicate confidence in multi-turn scenarios, and using confidence scores to guide retry mechanisms can optimize performance while reducing computational costs.

Abstract: Confidence in LLMs is a useful indicator of model uncertainty and answer reliability. Existing work mainly focused on single-turn scenarios, while research on confidence in complex multi-turn interactions is limited. In this paper, we investigate whether LLM-based search agents have the ability to communicate their own confidence through verbalized confidence scores after long sequences of actions, a significantly more challenging task compared to outputting confidence in a single interaction. Experimenting on open-source agentic models, we first find that models exhibit much higher task accuracy at high confidence while having near-zero accuracy when confidence is low. Based on this observation, we propose Test-Time Scaling (TTS) methods that use confidence scores to determine answer quality, encourage the model to try again until reaching a satisfactory confidence level. Results show that our proposed methods significantly reduce token consumption while demonstrating competitive performance compared to baseline fixed budget TTS methods.

Method: Created a corpus by extracting sentences from Form 10-K reports and ECTs, labeled for stance (positive, negative, neutral) using ChatGPT-o3-pro with human validation. Evaluated LLMs using zero-shot, few-shot, and Chain-of-Thought prompting strategies.

Result: Few-shot with Chain-of-Thought prompting performed best compared to supervised baselines. LLM performance varied across SEC and ECT datasets, demonstrating practical viability for target-specific stance detection without extensive labeled data.

Conclusion: LLMs can be effectively leveraged for target-specific stance detection in the financial domain using few-shot with Chain-of-Thought prompting, eliminating the need for large labeled datasets.

Abstract: Financial narratives from U.S. Securities and Exchange Commission (SEC) filing reports and quarterly earnings call transcripts (ECTs) are very important for investors, auditors, and regulators. However, their length, financial jargon, and nuanced language make fine-grained analysis difficult. Prior sentiment analysis in the financial domain required a large, expensive labeled dataset, making the sentence-level stance towards specific financial targets challenging. In this work, we introduce a sentence-level corpus for stance detection focused on three core financial metrics: debt, earnings per share (EPS), and sales. The sentences were extracted from Form 10-K annual reports and ECTs, and labeled for stance (positive, negative, neutral) using the advanced ChatGPT-o3-pro model under rigorous human validation. Using this corpus, we conduct a systematic evaluation of modern large language models (LLMs) using zero-shot, few-shot, and Chain-of-Thought (CoT) prompting strategies. Our results show that few-shot with CoT prompting performs best compared to supervised baselines, and LLMs’ performance varies across the SEC and ECT datasets. Our findings highlight the practical viability of leveraging LLMs for target-specific stance in the financial domain without requiring extensive labeled data.

Method: Created MMTutorBench with 685 problems built around pedagogically significant key-steps, paired with problem-specific rubrics for fine-grained evaluation across six dimensions, structured into Insight Discovery, Operation Formulation, and Operation Execution tasks.

Result: Evaluation of 12 leading MLLMs shows clear performance gaps between proprietary and open-source systems, substantial room for improvement compared to human tutors, OCR pipelines degrade tutoring quality, few-shot prompting yields limited gains, and rubric-based LLM-as-a-Judge proves highly reliable.

Conclusion: MMTutorBench highlights both the difficulty and diagnostic value for advancing AI tutoring, revealing significant gaps between current AI systems and human tutoring capabilities.

Abstract: Effective math tutoring requires not only solving problems but also diagnosing students’ difficulties and guiding them step by step. While multimodal large language models (MLLMs) show promise, existing benchmarks largely overlook these tutoring skills. We introduce MMTutorBench, the first benchmark for AI math tutoring, consisting of 685 problems built around pedagogically significant key-steps. Each problem is paired with problem-specific rubrics that enable fine-grained evaluation across six dimensions, and structured into three tasks-Insight Discovery, Operation Formulation, and Operation Execution. We evaluate 12 leading MLLMs and find clear performance gaps between proprietary and open-source systems, substantial room compared to human tutors, and consistent trends across input variants: OCR pipelines degrade tutoring quality, few-shot prompting yields limited gains, and our rubric-based LLM-as-a-Judge proves highly reliable. These results highlight both the difficulty and diagnostic value of MMTutorBench for advancing AI tutoring.

Method: Created M4FC dataset with images verified by professional fact-checkers from 22 organizations, spanning 10 languages and six multimodal tasks: visual claim extraction, claimant intent prediction, fake detection, image contextualization, location verification, and verdict prediction.

Result: The dataset contains 4,982 images paired with 6,980 claims, with baseline results provided for all tasks and analysis of how intermediate tasks influence downstream verdict prediction performance.

Conclusion: M4FC addresses key limitations in multimodal fact-checking research by providing a comprehensive, diverse dataset with multiple tasks and languages, enabling more robust fact-checking systems.

Abstract: Existing real-world datasets for multimodal automated fact-checking have multiple limitations: they contain few instances, focus on only one or two languages and tasks, suffer from evidence leakage, or depend on external sets of news articles for sourcing true claims. To address these shortcomings, we introduce M4FC, a new real-world dataset comprising 4,982 images paired with 6,980 claims. The images, verified by professional fact-checkers from 22 organizations, represent diverse cultural and geographic contexts. Each claim is available in one or two out of ten languages. M4FC spans six multimodal fact-checking tasks: visual claim extraction, claimant intent prediction, fake detection, image contextualization, location verification, and verdict prediction. We provide baseline results for all tasks and analyze how combining intermediate tasks influence downstream verdict prediction performance. We make our dataset and code available.

Method: Proposes core intents derived from observable behavior patterns in answer selection, grounded in satisficing theory. Constructs dataset through systematic filtering, LLM-based annotation, and rigorous quality control combining automated verification with human validation.

Result: Experimental evaluations show current systems struggle with core intent identification in personalized contexts, with models failing to identify core intents from user histories and performance degrading as question complexity increases.

Conclusion: The IPQA benchmark addresses a critical gap in personalized question answering evaluation and reveals limitations in current language models’ ability to identify core intents, with code and dataset to be made publicly available.

Abstract: Intent identification serves as the foundation for generating appropriate responses in personalized question answering (PQA). However, existing benchmarks evaluate only response quality or retrieval performance without directly measuring intent identification capabilities. This gap is critical because without understanding which intents users prioritize, systems cannot generate responses satisfying individual information needs. To address this, we introduce the concept of core intents: intents users prioritize when selecting answers to satisfy their information needs. To evaluate these core intents, we propose IPQA, a benchmark for core Intent identification in Personalized Question Answering. Since users do not explicitly state their prioritized intents, we derive core intents from observable behavior patterns in answer selection, grounded in satisficing theory where users choose answers meeting their acceptance thresholds. We construct a dataset with various domains through systematic filtering, LLM-based annotation, and rigorous quality control combining automated verification with human validation. Experimental evaluations across state-of-the-art language models reveal that current systems struggle with core intent identification in personalized contexts. Models fail to identify core intents from user histories, with performance degrading as question complexity increases. The code and dataset will be made publicly available to facilitate future research in this direction.

Method: Developed LIMRANK-SYNTHESIZER pipeline to generate diverse, challenging, and realistic synthetic reranking examples, then fine-tuned LIMRANK model using this synthetic data.

Result: LIMRANK achieves competitive performance on BRIGHT and FollowIR benchmarks while using less than 5% of typical training data, with strong generalization across scientific literature search and knowledge-intensive problem solving.

Conclusion: Modern LLMs can be effectively adapted for reranking with minimal high-quality supervision through synthetic data generation, enabling efficient and generalizable performance.

Abstract: Existing approaches typically rely on large-scale fine-tuning to adapt LLMs for information reranking tasks, which is computationally expensive. In this work, we demonstrate that modern LLMs can be effectively adapted using only minimal, high-quality supervision. To enable this, we design LIMRANK-SYNTHESIZER, a reusable and open-source pipeline for generating diverse, challenging, and realistic reranking examples. Using this synthetic data, we fine-tune our reranker model, LIMRANK. We evaluate LIMRANK on two challenging benchmarks, i.e., BRIGHT for reasoning-intensive retrieval and FollowIR for instruction-following retrieval. Our experiments demonstrate that LIMRANK achieves competitive performance, while being trained on less than 5% of the data typically used in prior work. Further ablation studies demonstrate the effectiveness of LIMRANK-SYNTHESIZER and the strong generalization capabilities of LIMRANK across downstream tasks, including scientific literature search and retrieval-augmented generation for knowledge-intensive problem solving.

Method: Evaluated traditional machine learning models (SVM, logistic regression, Naive Bayes) and fine-tuned transformers on a pre-split hope speech dataset with train, development, and test sets.

Result: Traditional models: linear-kernel SVM and logistic regression reached 0.78 macro-F1, RBF SVM 0.77, Naive Bayes 0.75. Transformers achieved better results with weighted precision 0.82, recall 0.80, F1 0.79, macro F1 0.79, accuracy 0.80.

Conclusion: While traditional ML models remain agile, transformers detect subtle semantics of hope better, suggesting larger transformers and LLMs could perform better on small datasets for hope speech detection.

Abstract: The identification of hope speech has become a promised NLP task, considering the need to detect motivational expressions of agency and goal-directed behaviour on social media platforms. This proposal evaluates traditional machine learning models and fine-tuned transformers for a previously split hope speech dataset as train, development and test set. On development test, a linear-kernel SVM and logistic regression both reached a macro-F1 of 0.78; SVM with RBF kernel reached 0.77, and Na"ive Bayes hit 0.75. Transformer models delivered better results, the best model achieved weighted precision of 0.82, weighted recall of 0.80, weighted F1 of 0.79, macro F1 of 0.79, and 0.80 accuracy. These results suggest that while optimally configured traditional machine learning models remain agile, transformer architectures detect some subtle semantics of hope to achieve higher precision and recall in hope speech detection, suggesting that larges transformers and LLMs could perform better in small datasets.

Method: Two-turn approach: Turn 1 performs adaptive branching to select instance-critical dimensions and create evidence-seeking hypotheses; Turn 2 executes branch-conditioned rethinking to test hypotheses with targeted rereading. Trained with GRPO-style RL using binary outcome rewards and format checks.

Result: Achieves state-of-the-art performance on three challenging reward modeling benchmarks across diverse domains, reduces judgment diffusion, and improves sensitivity to subtle yet consequential errors.

Conclusion: BR-RM successfully transfers think-twice principles to reward modeling, providing a practical and scalable solution that maintains compatibility with standard RLHF pipelines while enhancing reasoning quality.

Abstract: Large language models (LLMs) increasingly rely on thinking models that externalize intermediate steps and allocate extra test-time compute, with think-twice strategies showing that a deliberate second pass can elicit stronger reasoning. In contrast, most reward models (RMs) still compress many quality dimensions into a single scalar in one shot, a design that induces judgment diffusion: attention spreads across evaluation criteria, yielding diluted focus and shallow analysis. We introduce branch-and-rethink (BR-RM), a two-turn RM that transfers the think-twice principle to reward modeling. Turn 1 performs adaptive branching, selecting a small set of instance-critical dimensions (such as factuality and safety) and sketching concise, evidence-seeking hypotheses. Turn 2 executes branch-conditioned rethinking, a targeted reread that tests those hypotheses and scrutinizes only what matters most. We train with GRPO-style reinforcement learning over structured two-turn traces using a simple binary outcome reward with strict format checks, making the approach compatible with standard RLHF pipelines. By converting all-at-oncescoringintofocused, second-lookreasoning, BR-RMreducesjudgmentdiffusionandimproves sensitivity to subtle yet consequential errors while remaining practical and scalable. Experimental results demonstrate that our model achieves state-of-the-art performance on three challenging reward modeling benchmarks across diverse domains. The code and the model will be released soon.

Method: Augmented 7,437 questions from FinQA with full-document context, creating DocFinQA. Conducted experiments with retrieval-based QA pipelines and long-context language models.

Result: DocFinQA proved significantly challenging for state-of-the-art systems, with models particularly struggling on the longest documents in the dataset.

Conclusion: Addressing these long-document challenges could benefit applications requiring specificity and long-range contexts, such as gene sequences and legal document analysis.

Abstract: For large language models (LLMs) to be effective in the financial domain – where each decision can have a significant impact – it is necessary to investigate realistic tasks and data. Financial professionals often interact with documents that are hundreds of pages long, but most financial research datasets only deal with short excerpts from these documents. To address this, we introduce a long-document financial QA task. We augment 7,437 questions from the existing FinQA dataset with the full-document context, extending the average context length from under 700 words in FinQA to 123k words in DocFinQA. We conduct extensive experiments over retrieval-based QA pipelines and long-context language models. DocFinQA proves a significant challenge for even state-of-the-art systems. We also provide a case-study on the longest documents in DocFinQA and find that models particularly struggle on these documents. Addressing these challenges may have a wide reaching impact across applications where specificity and long-range contexts are critical, like gene sequences and legal document contract analysis.

Method: FaithLM formalizes faithfulness as an intervention property - faithful explanations should cause prediction shifts when contradicted. It uses a contrary-hint score to measure this, then iteratively refines both elicitation prompts and explanations to maximize faithfulness scores.

Result: Experiments on three multi-domain datasets with multiple LLM backbones show FaithLM consistently increases faithfulness and produces explanations more aligned with human rationales than strong self-explanation baselines.

Conclusion: Intervention-based evaluation coupled with iterative optimization provides a principled approach toward achieving faithful and reliable LLM explanations.

Abstract: Large language models (LLMs) increasingly produce natural language explanations, yet these explanations often lack faithfulness, and they do not reliably reflect the evidence the model uses to decide. We introduce FaithLM, a model-agnostic framework that evaluates and improves the faithfulness of LLM explanations without token masking or task-specific heuristics. FaithLM formalizes explanation faithfulness as an intervention property: a faithful explanation should yield a prediction shift when its content is contradicted. Theoretical analysis shows that the resulting contrary-hint score is a sound and discriminative estimator of faithfulness. Building on this principle, FaithLM iteratively refines both the elicitation prompt and the explanation to maximize the measured score. Experiments on three multi-domain datasets and multiple LLM backbones demonstrate that FaithLM consistently increases faithfulness and produces explanations more aligned with human rationales than strong self-explanation baselines. These findings highlight that intervention-based evaluation, coupled with iterative optimization, provides a principled route toward faithful and reliable LLM explanations.

Method: Investigate vocabulary expansion in low-resource settings by exploring embedding initialization methods and continual pre-training strategies with minimal target language data.

Result: Established strategies for vocabulary expansion that achieve faster inference while maintaining competitive downstream performance across diverse languages and tasks.

Conclusion: Vocabulary expansion can be effectively performed in low-resource settings with only 30K sentences, enabling faster inference for non-English languages while preserving model performance.

Abstract: Large language models (LLMs) have shown remarkable capabilities in many languages beyond English. Yet, LLMs require more inference steps when generating non-English text due to their reliance on English-centric tokenizers and vocabulary, resulting in higher usage costs to non-English speakers. Vocabulary expansion with target language tokens is a widely used cross-lingual vocabulary adaptation approach to remedy this issue. Despite its effectiveness in inference speedup, previous work on vocabulary expansion has focused on high-resource settings assuming access to a substantial amount of target language data to effectively initialize the embeddings of the new tokens and adapt the LLM to the target language. However, vocabulary expansion in low-resource settings has yet to be explored. In this article, we investigate vocabulary expansion in low-resource settings by considering embedding initialization methods and continual pre-training strategies. Through extensive experiments across typologically diverse languages, tasks and models, we establish a set of strategies to perform vocabulary expansion for faster inference, while striving to maintain competitive downstream performance to baselines. This is achieved with only 30K sentences ($\sim$0.01GB text data) from the target language.

Method: Proposed two automated evaluation metrics: Idea Alignment Score (IAScore) and Idea Distinctness Index. Also conducted human evaluation to assess novelty, relevance, and feasibility of generated ideas.

Result: The study provides insights into LLMs’ capabilities and limitations in research idea generation. The proposed metrics offer scalable evaluation alternatives to manual assessment.

Conclusion: This work contributes to evaluating and utilizing language models for generating future research ideas, with publicly available datasets and codes to support further research.

Abstract: The widespread adoption of Large Language Models (LLMs) and publicly available ChatGPT have marked a significant turning point in the integration of Artificial Intelligence (AI) into people’s everyday lives. This study examines the ability of Large Language Models (LLMs) to generate future research ideas from scientific papers. Unlike tasks such as summarization or translation, idea generation lacks a clearly defined reference set or structure, making manual evaluation the default standard. However, human evaluation in this setting is extremely challenging ie: it requires substantial domain expertise, contextual understanding of the paper, and awareness of the current research landscape. This makes it time-consuming, costly, and fundamentally non-scalable, particularly as new LLMs are being released at a rapid pace. Currently, there is no automated evaluation metric specifically designed for this task. To address this gap, we propose two automated evaluation metrics: Idea Alignment Score (IAScore) and Idea Distinctness Index. We further conducted human evaluation to assess the novelty, relevance, and feasibility of the generated future research ideas. This investigation offers insights into the evolving role of LLMs in idea generation, highlighting both its capability and limitations. Our work contributes to the ongoing efforts in evaluating and utilizing language models for generating future research ideas. We make our datasets and codes publicly available

Method: Introduces TrendFact benchmark with 7,643 curated samples from trending platforms and fact-checking datasets, plus an evidence library of 366,634 entries. Proposes FactISR framework integrating dynamic evidence augmentation with influence score-based iterative self-reflection for reasoning LLMs.

Result: Current fact-checking systems show significant limitations on TrendFact. FactISR effectively improves reasoning LLM performance on explainable and complex fact-checking tasks.

Conclusion: TrendFact facilitates development of more robust fact-checking methods, while FactISR offers new insights into explainable and complex fact-checking by enhancing reasoning LLM capabilities.

Abstract: Fact-checking benchmarks provide standardized testing criteria for automated fact-checking systems, driving technological advancement. With the surge of misinformation on social media and the emergence of various fact-checking methods, public concern about the transparency of automated systems and the accuracy of fact-checking for high infulence events has grown. However, existing benchmarks fail to meet these urgent needs and are predominantly English-centric, hindering the progress of comprehensive fact-checking. To address these issues, we introduce TrendFact, the first benchmark capable of evaluating hotspot perception ability (HPA) and all fact-checking tasks. TrendFact consists of 7,643 curated samples sourced from trending platforms and professional fact-checking datasets, as well as an evidence library containing 366,634 entries with publication dates. Additionally, to complement existing benchmarks in evaluating system explanation consistency and HPA, we propose two new metrics: ECS and HCPI. Experimental results show that current fact-checking systems face significant limitations when evaluated on TrendFact, which facilitates the development of more robust fact-checking methods. Furthermore, to enhance the capabilities of existing advanced fact-checking systems, the reasoning large language models (RLMs), we propose FactISR, a reasoning framework that integrates dynamic evidence augmentation with influence score-based iterative self-reflection. FactISR effectively improves RLM’s performance, offering new insights into explainable and complex fact-checking.

Method: Proposes TrajAgent framework with: 1) UniEnv - unified execution environment for data and models, 2) agentic workflow for automatic trajectory modeling, 3) collaborative learning between LLM agents and specialized models.

Result: Achieves 2.38%-69.91% performance improvement over baseline methods across four trajectory tasks using four real-world datasets.

Conclusion: TrajAgent effectively enables automated trajectory modeling through LLM-powered agents and collaborative learning, demonstrating significant performance improvements across diverse tasks and datasets.

Abstract: Trajectory modeling, which includes research on trajectory data pattern mining and future prediction, has widespread applications in areas such as life services, urban transportation, and public administration. Numerous methods have been proposed to address specific problems within trajectory modeling. However, the heterogeneity of data and the diversity of trajectory tasks make effective and reliable trajectory modeling an important yet highly challenging endeavor, even for domain experts. \fix In this paper, we propose \textit{TrajAgent}, a agent framework powered by large language models (LLMs), designed to facilitate robust and efficient trajectory modeling through automation modeling. This framework leverages and optimizes diverse specialized models to address various trajectory modeling tasks across different datasets effectively. \unfix~In \textit{TrajAgent}, we first develop \textit{UniEnv}, an execution environment with a unified data and model interface, to support the execution and training of various models. Building on \textit{UniEnv}, we introduce an agentic workflow designed for automatic trajectory modeling across various trajectory tasks and data. Furthermore, we introduce collaborative learning schema between LLM-based agents and small speciallized models, to enhance the performance of the whole framework effectively. Extensive experiments on four tasks using four real-world datasets demonstrate the effectiveness of \textit{TrajAgent} in automated trajectory modeling, achieving a performance improvement of \fix 2.38%-69.91% \unfix over baseline methods. The codes and data can be accessed via https://github.com/tsinghua-fib-lab/TrajAgent.

Method: Systematic review of parameter-efficient fine-tuning, domain/cross-lingual adaptation methods for encoder/decoder models, model specialization strategies, and preference alignment approaches using limited feedback.

Result: Provides empirical trade-offs, selection criteria, and best practices for choosing suitable techniques based on task constraints like model scaling, data scaling, and mitigating catastrophic forgetting.

Conclusion: Equips researchers and practitioners with actionable insights for effectively fine-tuning LLMs when data and resources are limited.

Abstract: Fine-tuning large language models (LLMs) with limited data poses a practical challenge in low-resource languages, specialized domains, and constrained deployment settings. While pre-trained LLMs provide strong foundations, effective adaptation under data scarcity requires focused and efficient fine-tuning techniques. This paper presents a structured and practical survey of recent methods for fine-tuning LLMs in data-scarce scenarios. We systematically review parameter-efficient fine-tuning techniques that lower training and deployment costs, domain and cross-lingual adaptation methods for both encoder and decoder models, and model specialization strategies. We further examine preference alignment approaches that guide model behavior using limited human or synthetic feedback, emphasizing sample and compute efficiency. Throughout, we highlight empirical trade-offs, selection criteria, and best practices for choosing suitable techniques based on task constraints, including model scaling, data scaling, and the mitigation of catastrophic forgetting. The aim is to equip researchers and practitioners with actionable insights for effectively fine-tuning LLMs when data and resources are limited.

Method: Fine-tune LLM with LoRA to incorporate disfluencies, then synthesize using TTS model that supports speech phenomena generation.

Result: User study showed significant increase in perceived spontaneity but slight reduction in intelligibility.

Conclusion: Disfluency insertion enhances speech naturalness for conversational agents despite minor intelligibility trade-off.

Abstract: Disfluencies are a natural feature of spontaneous human speech but are typically absent from the outputs of Large Language Models (LLMs). This absence can diminish the perceived naturalness of synthesized speech, which is an important criteria when building conversational agents that aim to mimick human behaviours. We show how the insertion of disfluencies can alleviate this shortcoming. The proposed approach involves (1) fine-tuning an LLM with Low-Rank Adaptation (LoRA) to incorporate various types of disfluencies into LLM-generated utterances and (2) synthesizing those utterances using a text-to-speech model that supports the generation of speech phenomena such as disfluencies. We evaluated the quality of the generated speech across two metrics: intelligibility and perceived spontaneity. We demonstrate through a user study that the insertion of disfluencies significantly increase the perceived spontaneity of the generated speech. This increase came, however, along with a slight reduction in intelligibility.

Method: Natural language processing approach analyzing transcripts of Holocaust survivor testimonies, clustering religious trajectories to identify common sequences in belief and practice evolution.

Result: Identified multiple common structures: constant disposition in religious belief and oscillating structure in religious practice across most narratives.

Conclusion: Demonstrates NLP’s potential for analyzing character evolution through thematic trajectories in narratives, providing valuable material for historical and sociological research.

Abstract: This work presents a computational approach to analyze character development along the narrative timeline. The analysis characterizes the inner and outer changes the protagonist undergoes within a narrative, and the interplay between them. We consider transcripts of Holocaust survivor testimonies as a test case, each telling the story of an individual in first-person terms. We focus on the survivor’s religious trajectory, examining the evolution of their disposition toward religious belief and practice along the testimony. Clustering the resulting trajectories in the dataset, we identify common sequences in the data. Our findings highlight multiple common structures of religiosity across the narratives: in terms of belief, most present a constant disposition, while for practice, most present an oscillating structure, serving as valuable material for historical and sociological research. This work demonstrates the potential of natural language processing techniques for analyzing character evolution through thematic trajectories in narratives.

Method: Proposes AttentionPredictor - a lightweight unified convolution model that dynamically captures spatiotemporal patterns to predict next-token attention scores, and a cross-token critical cache prefetching framework to hide estimation overhead.

Result: Achieves 13x KV cache compression and 5.6x speedup in cache offloading scenarios while maintaining comparable LLM performance, significantly outperforming state-of-the-art methods.

Conclusion: AttentionPredictor effectively addresses the limitations of static attention modeling by learning dynamic spatiotemporal patterns, enabling efficient KV cache compression without compromising LLM performance.

Abstract: With the development of large language models (LLMs), efficient inference through Key-Value (KV) cache compression has attracted considerable attention, especially for long-context generation. To compress the KV cache, recent methods identify critical KV tokens through static modeling of attention scores. However, these methods often struggle to accurately determine critical tokens as they neglect the temporal patterns in attention scores, resulting in a noticeable degradation in LLM performance. To address this challenge, we propose AttentionPredictor, which is the first learning-based method to directly predict attention patterns for KV cache compression and critical token identification. Specifically, AttentionPredictor learns a lightweight, unified convolution model to dynamically capture spatiotemporal patterns and predict the next-token attention scores. An appealing feature of AttentionPredictor is that it accurately predicts the attention score and shares the unified prediction model, which consumes negligible memory, among all transformer layers. Moreover, we propose a cross-token critical cache prefetching framework that hides the token estimation time overhead to accelerate the decoding stage. By retaining most of the attention information, AttentionPredictor achieves 13$\times$ KV cache compression and 5.6$\times$ speedup in a cache offloading scenario with comparable LLM performance, significantly outperforming the state-of-the-arts. The code is available at https://github.com/MIRALab-USTC/LLM-AttentionPredictor.

Method: Developed a multi-faceted evaluation framework measuring performance, fairness, unintended effects, and adaptability across preference divergence levels. Compared eight personalization methods across three preference datasets.

Result: Performance differences between methods reached 36% when users strongly disagreed, and personalization introduced up to 20% safety misalignment. Methods showed varying effectiveness across different preference scenarios.

Conclusion: The findings highlight the critical need for holistic evaluation approaches to advance the development of more effective and inclusive preference learning systems that account for diverse human values.

Abstract: While Reinforcement Learning from Human Feedback (RLHF) is widely used to align Large Language Models (LLMs) with human preferences, it typically assumes homogeneous preferences across users, overlooking diverse human values and minority viewpoints. Although personalized preference learning addresses this by tailoring separate preferences for individual users, the field lacks standardized methods to assess its effectiveness. We present a multi-faceted evaluation framework that measures not only performance but also fairness, unintended effects, and adaptability across varying levels of preference divergence. Through extensive experiments comparing eight personalization methods across three preference datasets, we demonstrate that performance differences between methods could reach 36% when users strongly disagree, and personalization can introduce up to 20% safety misalignment. These findings highlight the critical need for holistic evaluation approaches to advance the development of more effective and inclusive preference learning systems.

Method: Proposes KLUE method that identifies knowledge neurons using explainability methods and updates only those neurons using selected unforgotten samples to achieve faithful unlearning.

Result: Experimental results show widely-used unlearning methods fail to ensure faithful unlearning, while KLUE demonstrates significant effectiveness in real-world QA unlearning.

Conclusion: The paper successfully addresses the superficial unlearning problem through a new benchmark and KLUE method, achieving faithful knowledge removal while preserving relevant knowledge in different contexts.

Abstract: Various studies have attempted to remove sensitive or private knowledge from a language model to prevent its unauthorized exposure. However, prior studies have overlooked the complex and interconnected nature of knowledge, where related knowledge must be carefully examined. Specifically, they have failed to evaluate whether an unlearning method faithfully erases interconnected knowledge that should be removed, retaining knowledge that appears relevant but exists in a completely different context. To resolve this problem, we first define a new concept called superficial unlearning, which refers to the phenomenon where an unlearning method either fails to erase the interconnected knowledge it should remove or unintentionally erases irrelevant knowledge. Based on the definition, we introduce a new benchmark, FaithUn, to analyze and evaluate the faithfulness of unlearning in real-world knowledge QA settings. Furthermore, we propose a novel unlearning method, KLUE, which updates only knowledge-related neurons to achieve faithful unlearning. KLUE identifies knowledge neurons using an explainability method and updates only those neurons using selected unforgotten samples. Experimental results demonstrate that widely-used unlearning methods fail to ensure faithful unlearning, while our method shows significant effectiveness in real-world QA unlearning.

Method: FANformer adapts Fourier Analysis Network (FAN) into attention mechanism by modifying the feature projection process to achieve efficient periodicity modeling.

Result: FANformer consistently outperforms Transformer when scaling up model size and training tokens, with pretrained FANformer-1B showing marked improvements on downstream tasks compared to similar-sized open-source LLMs, and superior rule learning for reasoning.

Conclusion: FANformer is an effective and promising architecture for advancing LLMs with superior learning efficiency and reasoning capabilities.

Abstract: Periodicity, as one of the most important basic characteristics, lays the foundation for facilitating structured knowledge acquisition and systematic cognitive processes within human learning paradigms. However, the potential flaws of periodicity modeling in Transformer affect the learning efficiency and establishment of underlying principles from data for large language models (LLMs) built upon it. In this paper, we demonstrate that integrating effective periodicity modeling can improve the learning efficiency and performance of LLMs. We introduce FANformer, which adapts Fourier Analysis Network (FAN) into attention mechanism to achieve efficient periodicity modeling, by modifying the feature projection process of attention mechanism. Extensive experimental results on language modeling show that FANformer consistently outperforms Transformer when scaling up model size and training tokens, underscoring its superior learning efficiency. Our pretrained FANformer-1B exhibits marked improvements on downstream tasks compared to open-source LLMs with similar model parameters or training tokens. Moreover, we reveal that FANformer exhibits superior ability to learn and apply rules for reasoning compared to Transformer. The results position FANformer as an effective and promising architecture for advancing LLMs.

Method: Used BERT-based classifier to categorize QA pairs into Factual and Conceptual classes, fine-tuned two Llama-2 models based on these classifications, compared D-RAG and D-Naive synthetic dataset generation techniques, and evaluated with GPT-3.5 Turbo and Gemini.

Result: Models trained on conceptual datasets outperformed factual ones, D-Naive synthetic generation technique showed superior performance, and fine-tuned Llama-2 7B significantly outperformed baseline in generating product recommendations on a 1000-sample dataset.

Conclusion: PEFT may not be optimal for fact embedding but excels in instruction-based tasks, highlighting the importance of QA pair categorization and synthetic dataset generation for enhancing LLM performance in specific domains.

Abstract: This paper presents an extensive examination of Parameter-Efficient Fine-Tuning (PEFT) for embedding domain specific facts into Large Language Models (LLMs), focusing on improving the fine-tuning process by categorizing question-answer (QA) pairs into Factual and Conceptual classes using a BERT-based classifier. Two distinct Llama-2 models are fine-tuned based on these classifications and evaluated using larger models like GPT-3.5 Turbo and Gemini. Our results indicate that models trained on conceptual datasets outperform those trained on factual datasets. Additionally, we compare the efficiency of two synthetic fine-tuning dataset generation techniques, D-RAG and D-Naive, with D-Naive demonstrating superior performance. Although PEFT has shown effectiveness, our research indicates that it may not be the most optimal method for embedding facts into LLMs. However, it has demonstrated exceptional performance in instruction-based tasks. Our findings are reinforced by a 1000-sample dataset in the data center domain, where the fine-tuned Llama-2 7B model significantly outperforms the baseline model in generating product recommendations. Our study highlights the importance of QA pair categorization and synthetic dataset generation techniques in enhancing the performance of LLMs in specific domains.

Method: The paper proposes Iterative Model Merging (IMM), a method that strategically combines weights from original and self-improved models to preserve generalization while incorporating genuine reasoning improvements.

Result: Analysis shows that during self-improvement, LM weight updates concentrate in less reasoning-critical layers, leading to superficial learning where models show improved in-domain accuracy but compromised out-of-domain generalization due to memorization.

Conclusion: IMM effectively mitigates both LM collapse and superficial learning, moving towards more stable self-improving systems that preserve generalization while incorporating genuine reasoning improvements.

Abstract: As scaled language models (LMs) approach human-level reasoning capabilities, self-improvement emerges as a solution to synthesizing high-quality data corpus. While previous research has identified model collapse as a risk in self-improvement, where model outputs become increasingly deterministic, we discover a more fundamental challenge: the superficial self-improved reasoners phenomenon. In particular, our analysis reveals that even when LMs show improved in-domain (ID) reasoning accuracy, they actually compromise their generalized reasoning capabilities on out-of-domain (OOD) tasks due to memorization rather than genuine. Through a systematic investigation of LM architecture, we discover that during self-improvement, LM weight updates are concentrated in less reasoning-critical layers, leading to superficial learning. To address this, we propose Iterative Model Merging (IMM), a method that strategically combines weights from original and self-improved models to preserve generalization while incorporating genuine reasoning improvements. Our approach effectively mitigates both LM collapse and superficial learning, moving towards more stable self-improving systems.

Method: AttentionRAG uses an attention focus mechanism that reformulates RAG queries into a next-token prediction paradigm, isolating the query’s semantic focus to a single token for precise attention calculation between queries and retrieved contexts.

Result: Extensive experiments on LongBench and Babilong benchmarks show AttentionRAG achieves up to 6.3x context compression while outperforming LLMLingua methods by around 10% in key metrics.

Conclusion: AttentionRAG provides an effective solution for context pruning in RAG systems through its attention-guided approach, addressing limitations of existing methods and achieving superior compression performance.

Abstract: While RAG demonstrates remarkable capabilities in LLM applications, its effectiveness is hindered by the ever-increasing length of retrieved contexts, which introduces information redundancy and substantial computational overhead. Existing context pruning methods, such as LLMLingua, lack contextual awareness and offer limited flexibility in controlling compression rates, often resulting in either insufficient pruning or excessive information loss. In this paper, we propose AttentionRAG, an attention-guided context pruning method for RAG systems. The core idea of AttentionRAG lies in its attention focus mechanism, which reformulates RAG queries into a next-token prediction paradigm. This mechanism isolates the query’s semantic focus to a single token, enabling precise and efficient attention calculation between queries and retrieved contexts. Extensive experiments on LongBench and Babilong benchmarks show that AttentionRAG achieves up to 6.3$\times$ context compression while outperforming LLMLingua methods by around 10% in key metrics.

Method: HRM evaluates individual and consecutive reasoning steps at fine-grained and coarse-grained levels. HNC merges consecutive steps in tree structures for data augmentation with minimal computational overhead.

Result: HRM with HNC provides more stable and reliable evaluations than PRM on PRM800K dataset, and shows strong generalization on MATH500 and GSM8K datasets.

Conclusion: HRM effectively addresses PRM limitations through hierarchical evaluation and efficient data augmentation, demonstrating robust performance across diverse reasoning tasks.

Abstract: Recent studies show that Large Language Models (LLMs) achieve strong reasoning capabilities through supervised fine-tuning or reinforcement learning. However, a key approach, the Process Reward Model (PRM), suffers from reward hacking, making it unreliable in identifying the best intermediate step. In addition, the cost of annotating reasoning processes for reward modeling is high, making large-scale collection of high-quality data challenging. To address this, we propose a novel reward model approach called the Hierarchical Reward Model (HRM), which evaluates both individual and consecutive reasoning steps at both fine-grained and coarse-grained levels. HRM excels at assessing multi-step reasoning coherence, especially when flawed steps are later corrected through self-reflection. To further reduce the cost of generating training data, we introduce a lightweight and effective data augmentation strategy called Hierarchical Node Compression (HNC), which merges two consecutive reasoning steps into one within the tree structure. By applying HNC to MCTS-generated reasoning trajectories, we enhance the diversity and robustness of HRM training data while introducing controlled noise with minimal computational overhead. Empirical results on the PRM800K dataset show that HRM, together with HNC, provides more stable and reliable evaluations than PRM. Furthermore, cross-domain evaluations on the MATH500 and GSM8K datasets demonstrate HRM’s strong generalization and robustness across a variety of reasoning tasks.

Method: Two-player framework: actor model explores environments while critic model generates detailed natural language feedback. Both models are trained - critic for fine-grained assessments and actionable revisions, actor for utilizing critiques.

Result: CGI outperforms existing baselines substantially in three interactive environments. Even small critic models surpass GPT-4 in feedback quality. The actor achieves state-of-the-art performance.

Conclusion: Explicit iterative guidance through natural language feedback significantly enhances decision-making in LLM-based agents, enabling more robust exploration and avoiding local optima.

Abstract: Large language models (LLMs) have recently transformed from text-based assistants to autonomous agents capable of planning, reasoning, and iteratively improving their actions. While numerical reward signals and verifiers can effectively rank candidate actions, they often provide limited contextual guidance. In contrast, natural language feedback better aligns with the generative capabilities of LLMs, providing richer and more actionable suggestions. However, parsing and implementing this feedback effectively can be challenging for LLM-based agents. In this work, we introduce Critique-Guided Improvement (CGI), a novel two-player framework, comprising an actor model that explores an environment and a critic model that generates detailed nature language feedback. By training the critic to produce fine-grained assessments and actionable revisions, and the actor to utilize these critiques, our approach promotes more robust exploration of alternative strategies while avoiding local optima. Experiments in three interactive environments show that CGI outperforms existing baselines by a substantial margin. Notably, even a small critic model surpasses GPT-4 in feedback quality. The resulting actor achieves state-of-the-art performance, demonstrating the power of explicit iterative guidance to enhance decision-making in LLM-based agents.

Method: SafeMERGE uses a cosine similarity criterion to selectively merge fine-tuned layers with safety-aligned model layers only when they deviate from safe behavior, implementing selective layer-wise merging.

Result: Across three LLMs and two tasks, SafeMERGE consistently reduces harmful outputs compared to other defenses, with negligible or even positive impact on utility.

Conclusion: Selective layer-wise merging offers an effective safeguard against safety loss during fine-tuning, establishing SafeMERGE as a simple post-fine-tuning defense that preserves both safety and utility.

Abstract: Fine-tuning large language models (LLMs) is a common practice to adapt generalist models to specialized domains. However, recent studies show that fine-tuning can erode safety alignment, causing LLMs to respond to harmful or unethical prompts. Many methods to realign safety have been proposed, but often introduce custom algorithms that are difficult to implement or compromise task utility. In this work, we propose SafeMERGE, a lightweight, post-fine-tuning framework that preserves safety while maintaining downstream performance. SafeMERGE selectively merges fine-tuned with safety-aligned model layers only when they deviate from safe behavior, measured by a cosine similarity criterion. Across three LLMs and two tasks, SafeMERGE consistently reduces harmful outputs compared to other defenses, with negligible or even positive impact on utility. Our results demonstrate that selective layer-wise merging offers an effective safeguard against the inadvertent loss of safety during fine-tuning, establishing SafeMERGE as a simple post-fine-tuning defense.

Method: Compared two LALMs (Qwen2-Audio and Ultravox 0.5) with human EEG responses using surprisal and entropy metrics, analyzing sensitivity to speaker-content incongruency across social stereotype violations and biological knowledge violations.

Result: Qwen2-Audio showed increased surprisal for speaker-incongruent content and its surprisal values significantly predicted human N400 responses, while Ultravox 0.5 showed limited sensitivity. Neither model replicated the human processing distinction between social violations (N400 effects) and biological violations (P600 effects).

Conclusion: Current LALMs show both potential and limitations in processing speaker-contextualized language, revealing differences in social-linguistic processing mechanisms between humans and AI models.

Abstract: Voice-based AI development faces unique challenges in processing both linguistic and paralinguistic information. This study compares how large audio-language models (LALMs) and humans integrate speaker characteristics during speech comprehension, asking whether LALMs process speaker-contextualized language in ways that parallel human cognitive mechanisms. We compared two LALMs’ (Qwen2-Audio and Ultravox 0.5) processing patterns with human EEG responses. Using surprisal and entropy metrics from the models, we analyzed their sensitivity to speaker-content incongruency across social stereotype violations (e.g., a man claiming to regularly get manicures) and biological knowledge violations (e.g., a man claiming to be pregnant). Results revealed that Qwen2-Audio exhibited increased surprisal for speaker-incongruent content and its surprisal values significantly predicted human N400 responses, while Ultravox 0.5 showed limited sensitivity to speaker characteristics. Importantly, neither model replicated the human-like processing distinction between social violations (eliciting N400 effects) and biological violations (eliciting P600 effects). These findings reveal both the potential and limitations of current LALMs in processing speaker-contextualized language, and suggest differences in social-linguistic processing mechanisms between humans and LALMs.

Method: Proposes a unified mechanism integrating expert and token dimensions with a unified scoring function that linearly combines similarity scores between experts and tokens, based on linear programming principles.

Result: USMoE achieves up to 10% performance improvement over standard approaches or reduces inference costs by up to 14% while maintaining competitive accuracy across various settings including clean/corrupted data, LLMs, vision tasks, and training-free/training scenarios.

Conclusion: The USMoE framework effectively overcomes traditional routing limitations through its unified approach, providing both theoretical justification and empirical evidence of superior performance and efficiency.

Abstract: Sparse Mixture of Experts (SMoEs) models scale the capacity of models while maintaining constant computational overhead. Early designs typically relied on a fixed value of $k$, where $k$ represents either the number of experts selected per token or the number of tokens assigned per expert. However, these approaches encounter three key limitations: they may fail to route to important experts or tokens, may assign irrelevant ones, and often suffer from representation collapse among experts. This paper reexamines SMoEs through the lens of \textit{Linear Programming}, and proposes a Unified Sparse Mixture of Experts (USMoE) framework that addresses these limitations. Specifically, our approach introduces a unified mechanism that integrates information from both the expert and token dimensions, and a unified scoring function that linearly combines similarity scores between experts and tokens. We provide both theoretical justification and empirical evidence demonstrating USMoE’s effectiveness in overcoming the limitations of traditional routing methods. Through comprehensive evaluations on both clean and corrupted settings for large language models and vision tasks, under both training-free and training scenarios, USMoE achieves up to a 10% performance improvement over standard approaches or reduces inference costs by up to 14%, while maintaining competitive accuracy.

Method: Proposed a five-tuple definition framework disentangling stereotypes, anti-stereotypes, stereotypical bias, and general bias. Developed StereoDetect, a well-curated benchmark dataset aligned with the proposed definitions, grounded in social psychology principles.

Result: Sub-10B language models and GPT-4o frequently misclassify anti-stereotypes and fail to recognize neutral overgeneralizations. StereoDetect demonstrates effectiveness through qualitative and quantitative comparisons with existing benchmarks and fine-tuned models.

Conclusion: The work provides precise terminology and a conceptual framework for stereotype detection, addresses key shortcomings in existing benchmarks, and offers StereoDetect as a reliable tool for advancing research in this critical area of Responsible AI.

Abstract: Stereotypes are known to have very harmful effects, making their detection critically important. However, current research predominantly focuses on detecting and evaluating stereotypical biases, thereby leaving the study of stereotypes in its early stages. Our study revealed that many works have failed to clearly distinguish between stereotypes and stereotypical biases, which has significantly slowed progress in advancing research in this area. Stereotype and Anti-stereotype detection is a problem that requires social knowledge; hence, it is one of the most difficult areas in Responsible AI. This work investigates this task, where we propose a five-tuple definition and provide precise terminologies disentangling stereotypes, anti-stereotypes, stereotypical bias, and general bias. We provide a conceptual framework grounded in social psychology for reliable detection. We identify key shortcomings in existing benchmarks for this task of stereotype and anti-stereotype detection. To address these gaps, we developed StereoDetect, a well curated, definition-aligned benchmark dataset designed for this task. We show that sub-10B language models and GPT-4o frequently misclassify anti-stereotypes and fail to recognize neutral overgeneralizations. We demonstrate StereoDetect’s effectiveness through multiple qualitative and quantitative comparisons with existing benchmarks and models fine-tuned on them. The dataset and code is available at https://github.com/KaustubhShejole/StereoDetect.

Method: Combines LLM agents with a directed social graph, constructing user representations from diverse fine-grained personas to enable multi-agent simulations of content dissemination and engagement dynamics.

Result: Evaluated three content moderation strategies and found they not only mitigate non-factual content spread but also increase user engagement. Analyzed content trajectories and agent reasoning alignment with collective engagement patterns.

Conclusion: The framework enables scalable analysis of social content dynamics and moderation strategies, with open-source software to encourage further AI and social sciences research.

Abstract: We present a novel, open-source social network simulation framework, MOSAIC, where generative language agents predict user behaviors such as liking, sharing, and flagging content. This simulation combines LLM agents with a directed social graph to analyze emergent deception behaviors and gain a better understanding of how users determine the veracity of online social content. By constructing user representations from diverse fine-grained personas, our system enables multi-agent simulations that model content dissemination and engagement dynamics at scale. Within this framework, we evaluate three different content moderation strategies with simulated misinformation dissemination, and we find that they not only mitigate the spread of non-factual content but also increase user engagement. In addition, we analyze the trajectories of popular content in our simulations, and explore whether simulation agents’ articulated reasoning for their social interactions truly aligns with their collective engagement patterns. We open-source our simulation software to encourage further research within AI and social sciences.

Method: SEAL categorizes reasoning thoughts into execution, reflection, and transition types, then uses a steering vector extracted from latent space to calibrate reasoning traces through representation intervention, reducing excessive reflection and transition thoughts.

Result: Experiments show up to 11% accuracy improvement while reducing reasoning tokens by 11.8% to 50.4% across multiple models and benchmarks including Math500, GSM8K, and LiveCodeBench.

Conclusion: SEAL effectively improves reasoning efficiency and accuracy by steering away from redundant thought patterns, with the steering vector demonstrating strong transferability across tasks.

Abstract: Large Language Models (LLMs), such as OpenAI’s o1-series have demonstrated compelling capabilities for complex reasoning tasks via the extended chain-of-thought (CoT) reasoning mechanism. However, recent studies reveal substantial redundancy in the CoT reasoning traces, which not only increases inference latency but also negatively impacts model performance by diverting attention to unnecessary reasoning paths. To address this issue, we investigate the internal reasoning structures of LLMs and categorize them into three primary thought types: execution, reflection, and transition thoughts. Moreover, our analysis reveals that excessive reflection and transition thoughts are strongly correlated with failure cases and these thought categories exhibit clear separation in the latent space. Based on these, we introduce SEAL (Steerable reasoning calibration), a training-free approach that seamlessly calibrates the CoT process, improving accuracy while demonstrating significant efficiency gains. SEAL consists of an offline stage for extracting the reasoning steering vector in the latent space, followed by an on-the-fly calibration of the reasoning trace through representation intervention using the steering vector. Notably, the steering vector exhibits strong transferability across various tasks. Extensive experiments across multiple models (DeepSeek-R1-Distill and QwQ-32B-Preview) and benchmarks (Math500, GSM8K, LiveCodeBench) validate the effectiveness of SEAL, up to a 11% improvement in accuracy while reducing reasoning tokens by 11.8% to 50.4%. Our code is publicly available at https://github.com/VITA-Group/SEAL.

Method: Developed a Rao-Blackwellized estimator that is unbiased and provably has lower variance than standard Monte Carlo estimators. Also derived an analogous estimator for the gradient of KL divergence.

Result: Empirical study on sentiment-controlled fine-tuning shows the estimator provides more stable KL estimates with substantially reduced variance. Training with the gradient estimator leads to more stable training and models that more frequently appear on the Pareto frontier of reward vs. KL.

Conclusion: The proposed Rao-Blackwellized estimator offers improved variance reduction and stability for KL divergence estimation in language models, with practical benefits for training and optimization.

Abstract: Estimating the Kullback–Leibler (KL) divergence between language models has many applications, e.g., reinforcement learning from human feedback (RLHF), interpretability, and knowledge distillation. However, computing the exact KL divergence between two arbitrary language models is intractable. Thus, practitioners often resort to sampling-based estimators. While it is easy to fashion a simple Monte Carlo (MC) estimator that provides an unbiased estimate of the KL divergence between language models, this estimator notoriously suffers from high variance and can even result in a negative estimate of the KL divergence, a non-negative quantity. In this paper, we introduce a Rao–Blackwellized estimator that is unbiased and provably has variance less than or equal to that of the standard Monte Carlo estimator. In an empirical study on sentiment-controlled fine-tuning, we show that our estimator provides more stable KL estimates and reduces variance substantially. Additionally, we derive an analogous Rao–Blackwellized estimator of the gradient of the KL divergence, which leads to more stable training and produces models that more frequently appear on the Pareto frontier of reward vs. KL compared to the ones trained with the MC estimator of the gradient.

Method: Created Cancer-Myth dataset with 585 expert-verified cancer questions containing false presuppositions, and Cancer-Myth-NFP set with 150 questions without false presuppositions. Tested frontier LLMs and mitigation strategies like GEPA-optimized precautionary prompts.

Result: No frontier LLM (GPT-5, Gemini-2.5-Pro, Claude-4-Sonnet) corrected false presuppositions more than 43% of the time. Mitigation strategies improved accuracy to 80% on Cancer-Myth but caused 41% false positives on Cancer-Myth-NFP and 10% performance drop on other medical benchmarks.

Conclusion: LLMs have critical reliability gaps in handling false presuppositions in medical questions. Prompting alone isn’t sufficient, and more robust safeguards are needed for medical AI systems.

Abstract: Cancer patients are increasingly turning to large language models (LLMs) for medical information, making it critical to assess how well these models handle complex, personalized questions. However, current medical benchmarks focus on medical exams or consumer-searched questions and do not evaluate LLMs on real patient questions with patient details. In this paper, we first have three hematology-oncology physicians evaluate cancer-related questions drawn from real patients. While LLM responses are generally accurate, the models frequently fail to recognize or address false presuppositions in the questions, posing risks to safe medical decision-making. To study this limitation systematically, we introduce Cancer-Myth, an expert-verified adversarial dataset of 585 cancer-related questions with false presuppositions. On this benchmark, no frontier LLM – including GPT-5, Gemini-2.5-Pro, and Claude-4-Sonnet – corrects these false presuppositions more than $43%$ of the time. To study mitigation strategies, we further construct a 150-question Cancer-Myth-NFP set, in which physicians confirm the absence of false presuppositions. We find typical mitigation strategies, such as adding precautionary prompts with GEPA optimization, can raise accuracy on Cancer-Myth to $80%$, but at the cost of misidentifying presuppositions in $41%$ of Cancer-Myth-NFP questions and causing a $10%$ relative performance drop on other medical benchmarks. These findings highlight a critical gap in the reliability of LLMs, show that prompting alone is not a reliable remedy for false presuppositions, and underscore the need for more robust safeguards in medical AI systems.

Method: Used unsupervised clustering on corpus-extracted words based on phonotactic information.

Result: The word clusters largely aligned with etymological distinction and recovered known linguistic generalizations about etymological classes, plus uncovered new features.

Conclusion: The Germanic-Latinate distinction in English is learnable from phonotactic patterns, making etymology-based generalizations cognitively plausible.

Abstract: Cross-linguistically, native words and loanwords follow different phonological rules. In English, for example, words of Germanic and Latinate origin exhibit different stress patterns, and a certain syntactic structure, double-object datives, is predominantly associated with Germanic verbs rather than Latinate verbs. As a cognitive model, however, such etymology-based generalizations face challenges in terms of learnability, since the historical origins of words are presumably inaccessible information for general language learners. In this study, we present computational evidence indicating that the Germanic-Latinate distinction in the English lexicon is learnable from the phonotactic information of individual words. Specifically, we performed an unsupervised clustering on corpus-extracted words, and the resulting word clusters largely aligned with the etymological distinction. The model-discovered clusters also recovered various linguistic generalizations documented in the previous literature regarding the corresponding etymological classes. Moreover, our findings also uncovered previously unrecognized features of the quasi-etymological clusters.

Method: Created PERSONAMEM benchmark with curated user profiles and 180+ simulated user-LLM interaction histories across 15 real-world tasks, evaluating LLMs’ response selection accuracy given user queries.

Result: Current LLMs struggle with dynamic user profile evolution, achieving only around 50% overall accuracy across frontier models like GPT-4.1, o4-mini, GPT-4.5, o1, and Gemini-2.0.

Conclusion: There’s significant room for improvement in developing user-aware chatbots, and PERSONAMEM provides a valuable benchmark and simulation pipeline for future research.

Abstract: Large Language Models (LLMs) have emerged as personalized assistants for users across a wide range of tasks – from offering writing support to delivering tailored recommendations or consultations. Over time, the interaction history between a user and an LLM can provide extensive information about an individual’s traits and preferences. However, open questions remain on how well LLMs today can effectively leverage such history to (1) internalize the user’s inherent traits and preferences, (2) track how the user profiling and preferences evolve over time, and (3) generate personalized responses accordingly in new scenarios. In this work, we introduce the PERSONAMEM benchmark. PERSONAMEM features curated user profiles with over 180 simulated user-LLM interaction histories, each containing up to 60 sessions of multi-turn conversations across 15 real-world tasks that require personalization. Given an in-situ user query, i.e. query issued by the user from the first-person perspective, we evaluate LLM chatbots’ ability to identify the most suitable response according to the current state of the user’s profile. We observe that current LLMs still struggle to recognize the dynamic evolution in users’ profiles over time through direct prompting approaches. As a consequence, LLMs often fail to deliver responses that align with users’ current situations and preferences, with frontier models such as GPT-4.1, o4-mini, GPT-4.5, o1, or Gemini-2.0 achieving only around 50% overall accuracy, suggesting room for improvement. We hope that PERSONAMEM, along with the user profile and conversation simulation pipeline, can facilitate future research in the development of truly user-aware chatbots. Code and data are available at github.com/bowen-upenn/PersonaMem.

Method: Developed a zeroth-order comparison-based optimization method using comparison oracles, with convergence guarantees and practical heuristics for handling noisy preference pairs.

Result: Experimental evaluations on Mistral-7B, Llama-3-8B, and Gemma-2-9B models using AlpacaEval 2, MT-Bench, and Arena-Hard benchmarks demonstrate effectiveness in improving LLM performance with noisy preference pairs.

Conclusion: The method provides an effective alternative to existing direct alignment approaches and highlights the importance of designing specialized methods for preference pairs with distinct likelihood margins.

Abstract: Direct alignment methods are increasingly used for aligning large language models (LLMs) with human preferences. However, these methods suffer from the issues of verbosity and likelihood displacement, which can be driven by the noisy preference pairs that induce similar likelihood for preferred and dispreferred responses. The contributions of this paper are two-fold. First, we propose a new preference alignment method based on zeroth-order, comparison-based optimization via comparison oracles and provide convergence guarantees for its basic scheme. Second, we improve our method using some heuristics and conduct the experiments to demonstrate the flexibility and compatibility of practical scheme in improving the performance of LLMs using noisy preference pairs. Evaluations are conducted across multiple base and instruction-tuned models (Mistral-7B, Llama-3-8B and Gemma-2-9B) with benchmarks (AlpacaEval 2, MT-Bench and Arena-Hard). Experimental results show the effectiveness of our method as an alternative to addressing the limitations of existing direct alignment methods. A highlight of our work is that we evidence the importance of designing specialized methods for preference pairs with distinct likelihood margin, which complements the recent findings in Razin et al (2025).

Method: Developed an iterative term clustering approach for data selection, collected 13,717 YouTube comments from 7,373 videos with over 1.2 billion views, annotated by nine native speakers, accommodating both Ge’ez script and Romanized transliterations.

Result: Fine-tuned small models outperform prompted frontier LLMs in low-resource settings, achieving 86.67% F1 in abusiveness detection (7+ points over best LLM) and stronger performance across all tasks.

Conclusion: The benchmark addresses critical gaps in content moderation for under-resourced languages and demonstrates that specialized fine-tuned models can outperform general LLMs in low-resource scenarios, promoting research on online safety.

Abstract: Content moderation research has recently made significant advances, but remains limited in serving the majority of the world’s languages due to the lack of resources, leaving millions of vulnerable users to online hostility. This work presents a large-scale human-annotated multi-task benchmark dataset for abusive language detection in Tigrinya social media with joint annotations for three tasks: abusiveness, sentiment, and topic classification. The dataset comprises 13,717 YouTube comments annotated by nine native speakers, collected from 7,373 videos with a total of over 1.2 billion views across 51 channels. We developed an iterative term clustering approach for effective data selection. Recognizing that around 64% of Tigrinya social media content uses Romanized transliterations rather than native Ge’ez script, our dataset accommodates both writing systems to reflect actual language use. We establish strong baselines across the tasks in the benchmark, while leaving significant challenges for future contributions. Our experiments demonstrate that small fine-tuned models outperform prompted frontier large language models (LLMs) in the low-resource setting, achieving 86.67% F1 in abusiveness detection (7+ points over best LLM), and maintain stronger performance in all other tasks. The benchmark is made public to promote research on online safety.

Method: Audited four multilingual CLIP variants (M-CLIP, NLLB-CLIP, CAPIVARA-CLIP, SigLIP-2) across ten languages using balanced subsets of FairFace and PATA stereotype suite in zero-shot setting.

Result: All models showed stronger gender bias than English baselines; CAPIVARA-CLIP had largest biases in low-resource target languages; shared encoders transferred English stereotypes to gender-neutral languages; gendered languages amplified all bias types.

Conclusion: Aggregated metrics mask language-specific bias hotspots, highlighting need for fine-grained, language-aware bias evaluation in multilingual VLM research.

Abstract: Multilingual vision-language models (VLMs) promise universal image-text retrieval, yet their social biases remain underexplored. We perform the first systematic audit of four public multilingual CLIP variants: M-CLIP, NLLB-CLIP, CAPIVARA-CLIP, and the debiased SigLIP-2, covering ten languages that differ in resource availability and morphological gender marking. Using balanced subsets of FairFace and the PATA stereotype suite in a zero-shot setting, we quantify race and gender bias and measure stereotype amplification. Contrary to the intuition that multilinguality mitigates bias, every model exhibits stronger gender skew than its English-only baseline. CAPIVARA-CLIP shows its largest biases precisely in the low-resource languages it targets, while the shared encoder of NLLB-CLIP and SigLIP-2 transfers English gender stereotypes into gender-neutral languages; loosely coupled encoders largely avoid this leakage. Although SigLIP-2 reduces agency and communion skews, it inherits – and in caption-sparse contexts (e.g., Xhosa) amplifies – the English anchor’s crime associations. Highly gendered languages consistently magnify all bias types, yet gender-neutral languages remain vulnerable whenever cross-lingual weight sharing imports foreign stereotypes. Aggregated metrics thus mask language-specific hot spots, underscoring the need for fine-grained, language-aware bias evaluation in future multilingual VLM research.

Method: GainLoRA expands a new LoRA branch for each new task and introduces gating modules to integrate new and old branches while minimizing the new branch’s influence on old tasks.

Result: Experimental results on CL benchmarks show that GainLoRA outperforms existing state-of-the-art methods.

Conclusion: GainLoRA effectively mitigates forgetting in continual learning for LLMs through gated integration of LoRA branches, improving overall model performance.

Abstract: Continual learning (CL), which requires the model to learn multiple tasks sequentially, is crucial for large language models (LLMs). Recently, low-rank adaptation~(LoRA), one of the most representative parameter-efficient fine-tuning (PEFT) methods, has gained increasing attention in CL of LLMs. However, most existing CL methods based on LoRA typically expand a new LoRA branch to learn each new task and force the new and old LoRA branches to influence old tasks equally, potentially leading to forgetting. In this work, we propose a new method, called gated integration of low-rank adaptation (GainLoRA), for CL of LLMs. GainLoRA expands a new LoRA branch for each new task and introduces gating modules to integrate the new and old LoRA branches. Furthermore, GainLoRA leverages the new gating module to minimize the influence from the new LoRA branch to old tasks, effectively mitigating forgetting and improving the model’s overall performance. Experimental results on CL benchmarks demonstrate that GainLoRA outperforms existing state-of-the-art methods.

Method: Models sequential editing as constrained stochastic programming, integrates queuing theory and Lyapunov optimization to decompose long-term constraints into tractable stepwise subproblems.

Result: Scales sequential editing capacity to over 10,000 edits while stabilizing general capabilities and boosting average editing efficacy by 11.89% over SOTA baselines. Can also enhance baseline methods.

Conclusion: LyapLock is the first model editing framework with rigorous theoretical guarantees, achieving asymptotic optimal editing performance while meeting long-term knowledge preservation constraints.

Abstract: Large Language Models often contain factually incorrect or outdated knowledge, giving rise to model editing methods for precise knowledge updates. However, current mainstream locate-then-edit approaches exhibit a progressive performance decline during sequential editing, due to inadequate mechanisms for long-term knowledge preservation. To tackle this, we model the sequential editing as a constrained stochastic programming. Given the challenges posed by the cumulative preservation error constraint and the gradually revealed editing tasks, \textbf{LyapLock} is proposed. It integrates queuing theory and Lyapunov optimization to decompose the long-term constrained programming into tractable stepwise subproblems for efficient solving. This is the first model editing framework with rigorous theoretical guarantees, achieving asymptotic optimal editing performance while meeting the constraints of long-term knowledge preservation. Experimental results show that our framework scales sequential editing capacity to over 10,000 edits while stabilizing general capabilities and boosting average editing efficacy by 11.89% over SOTA baselines. Furthermore, it can be leveraged to enhance the performance of baseline methods. Our code is released on https://github.com/caskcsg/LyapLock.

Method: Proposed an attribution-based method to identify specialized attention heads (in-context and parametric heads), extracted function vectors, and modified attention weights to analyze their influence on answer generation.

Result: Identified distinct roles of attention heads: in-context heads comprehend instructions and retrieve relevant contextual information, while parametric heads store entities’ relational knowledge.

Conclusion: The insights enable tracing knowledge sources during inference, paving the way for more safe and transparent language models.

Abstract: Large language models are able to exploit in-context learning to access external knowledge beyond their training data through retrieval-augmentation. While promising, its inner workings remain unclear. In this work, we shed light on the mechanism of in-context retrieval augmentation for question answering by viewing a prompt as a composition of informational components. We propose an attribution-based method to identify specialized attention heads, revealing in-context heads that comprehend instructions and retrieve relevant contextual information, and parametric heads that store entities’ relational knowledge. To better understand their roles, we extract function vectors and modify their attention weights to show how they can influence the answer generation process. Finally, we leverage the gained insights to trace the sources of knowledge used during inference, paving the way towards more safe and transparent language models.

Method: Proposed a simulator that models hypothesis performance based on similarity to hidden ground truth with noise, and an in-context reinforcement learning framework where an LLM-based policy decomposes hypotheses, clusters them by mechanistic roles, and prioritizes recombinations using feedback.

Result: Validated against 124 hypotheses with experimental outcomes, the simulator approximates real results with consistent trend alignment. The approach significantly outperforms pre-experiment baselines and strong ablations.

Conclusion: The toolkit enables systematic research on experiment-guided ranking, with the policy serving as a strong proof of concept for more robust hypothesis prioritization strategies.

Abstract: Hypothesis ranking is vital for automated scientific discovery, especially in cost-intensive, throughput-limited natural science domains. Current methods focus on pre-experiment ranking, relying solely on language model reasoning without empirical feedback. We introduce experiment-guided ranking, which prioritizes hypotheses based on feedback from prior tests. Due to the impracticality of real experiments, we propose a simulator grounded in domain-specific concepts that models hypothesis performance as a function of similarity to a hidden ground truth, perturbed by noise. Validated against 124 hypotheses with experimentally reported outcomes, the simulator approximates real results with consistent trend alignment. Although deviations exist, they mimic wet-lab noise, promoting more robust ranking strategies. We frame experiment-guided ranking as a sequential decision-making problem and propose an in-context reinforcement learning (ICRL) framework. Our LLM-based policy decomposes hypotheses into functional elements, clusters them by mechanistic roles, and prioritizes recombinations based on feedback. Experiments show our approach significantly outperforms pre-experiment baselines and strong ablations. Our toolkit, comprising the simulator and ICRL framework, enables systematic research on experiment-guided ranking, with the policy serving as a strong proof of concept.

Method: Proposes a hierarchical search method that incrementally proposes and integrates details into hypotheses, progressing from general concepts to specific experimental configurations. This smooths the reward landscape and enables more effective optimization.

Result: Empirical evaluations on a new benchmark of expert-annotated fine-grained hypotheses from recent literature show that the method consistently outperforms strong baselines.

Conclusion: The hierarchical process effectively addresses fine-grained scientific hypothesis discovery, demonstrating that LLMs can generate detailed, experimentally actionable hypotheses when properly leveraged through combinatorial optimization approaches.

Abstract: Large language models (LLMs) have shown promise in automating scientific hypothesis generation, yet existing approaches primarily yield coarse-grained hypotheses lacking critical methodological and experimental details. We introduce and formally define the new task of fine-grained scientific hypothesis discovery, which entails generating detailed, experimentally actionable hypotheses from coarse initial research directions. We frame this as a combinatorial optimization problem and investigate the upper limits of LLMs' capacity to solve it when maximally leveraged. Specifically, we explore four foundational questions: (1) how to best harness an LLM’s internal heuristics to formulate the fine-grained hypothesis it itself would judge as the most promising among all the possible hypotheses it might generate, based on its own internal scoring-thus defining a latent reward landscape over the hypothesis space; (2) whether such LLM-judged better hypotheses exhibit stronger alignment with ground-truth hypotheses; (3) whether shaping the reward landscape using an ensemble of diverse LLMs of similar capacity yields better outcomes than defining it with repeated instances of the strongest LLM among them; and (4) whether an ensemble of identical LLMs provides a more reliable reward landscape than a single LLM. To address these questions, we propose a hierarchical search method that incrementally proposes and integrates details into the hypothesis, progressing from general concepts to specific experimental configurations. We show that this hierarchical process smooths the reward landscape and enables more effective optimization. Empirical evaluations on a new benchmark of expert-annotated fine-grained hypotheses from recent literature show that our method consistently outperforms strong baselines.

Method: 1) Train dense passage retrieval model to retrieve associated knowledge; 2) Introduce knowledge integration model incorporating retrieval knowledge into instructions during fine-tuning; 3) Leverage fine-tuned LLM with ensemble based on text consistency (coherence, fluency, answer format assurance).

Result: Extensive experiments on TriviaQA, MSMARCO, and CMRC2018 datasets demonstrate GenKI’s effectiveness compared to state-of-the-art baselines. Ablation studies reveal linear relationship between retrieved knowledge frequency and model’s knowledge recall accuracy.

Conclusion: GenKI successfully improves OpenQA performance by simultaneously exploring knowledge integration and controllable generation in LLMs, with experiments showing its effectiveness across diverse answer formats and datasets.

Abstract: Open-domain question answering (OpenQA) represents a cornerstone in natural language processing (NLP), primarily focused on extracting answers from unstructured textual data. With the rapid advancements in Large Language Models (LLMs), LLM-based OpenQA methods have reaped the benefits of emergent understanding and answering capabilities enabled by massive parameters compared to traditional methods. However, most of these methods encounter two critical challenges: how to integrate knowledge into LLMs effectively and how to adaptively generate results with specific answer formats for various task situations. To address these challenges, we propose a novel framework named GenKI, which aims to improve the OpenQA performance by exploring Knowledge Integration and controllable Generation on LLMs simultaneously. Specifically, we first train a dense passage retrieval model to retrieve associated knowledge from a given knowledge base. Subsequently, we introduce a novel knowledge integration model that incorporates the retrieval knowledge into instructions during fine-tuning to intensify the model. Furthermore, to enable controllable generation in LLMs, we leverage a certain fine-tuned LLM and an ensemble based on text consistency incorporating all coherence, fluency, and answer format assurance. Finally, extensive experiments conducted on the TriviaQA, MSMARCO, and CMRC2018 datasets, featuring diverse answer formats, have demonstrated the effectiveness of GenKI with comparison of state-of-the-art baselines. Moreover, ablation studies have disclosed a linear relationship between the frequency of retrieved knowledge and the model’s ability to recall knowledge accurately against the ground truth. Our code of GenKI is available at https://github.com/USTC-StarTeam/GenKI

Method: Used a range of techniques including baseline synonym replacement and sophisticated generative models enhanced with beam search and named entity analysis to transform The Great Gatsby into an ’e’-less text.

Result: Found that excluding up to 3.6% of the most common letters (up to ‘u’) had minimal impact on text meaning, but translation fidelity rapidly decays with stronger lipogram constraints.

Conclusion: Modern LLMs demonstrate surprising flexibility in handling strict linguistic constraints, revealing the adaptability and creativity of English language under extreme limitations.

Abstract: Lipograms are a unique form of constrained writing where all occurrences of a particular letter are excluded from the text, typified by the novel Gadsby, which daringly avoids all usage of the letter ’e’. In this study, we explore the power of modern large language models (LLMs) by transforming the novel F. Scott Fitzgerald’s The Great Gatsby into a fully ’e’-less text. We experimented with a range of techniques, from baseline methods like synonym replacement to sophisticated generative models enhanced with beam search and named entity analysis. We show that excluding up to 3.6% of the most common letters (up to the letter ‘u’) had minimal impact on the text’s meaning, although translation fidelity rapidly and predictably decays with stronger lipogram constraints. Our work highlights the surprising flexibility of English under strict constraints, revealing just how adaptable and creative language can be.

Method: Builds upon GRPO with self-rewarding mechanism based on majority voting over multiple sampled responses. Also includes data self-generation strategies where MLLM synthesizes new training samples.

Result: Significantly improves reasoning ability: MathVista (66.3%→72.9%) and We-Math (62.9%→68.7%) using standard datasets without ground truth labels. Synthetic data combination further boosts performance.

Conclusion: MM-UPT offers a new paradigm for autonomous MLLM enhancement as a critical third step after SFT and RL, enabling scalable self-improvement without external supervision.

Abstract: Improving Multi-modal Large Language Models (MLLMs) in the post-training stage typically relies on supervised fine-tuning (SFT) or reinforcement learning (RL), which require expensive and manually annotated multi-modal data–an ultimately unsustainable resource. This limitation has motivated a growing interest in unsupervised paradigms as a third stage of post-training after SFT and RL. While recent efforts have explored this direction, their methods are complex and difficult to iterate. To address this, we propose MM-UPT, a simple yet effective framework for unsupervised post-training of MLLMs, enabling continual self-improvement without any external supervision. The training method of MM-UPT builds upon GRPO, replacing traditional reward signals with a self-rewarding mechanism based on majority voting over multiple sampled responses. Our experiments demonstrate that such training method effectively improves the reasoning ability of Qwen2.5-VL-7B (e.g., 66.3%$\rightarrow$72.9% on MathVista, 62.9%$\rightarrow$68.7% on We-Math), using standard dataset without ground truth labels. To further explore scalability, we extend our framework to a data self-generation setting, designing two strategies that prompt the MLLM to synthesize new training samples on its own. Additional experiments show that combining these synthetic data with the unsupervised training method can also boost performance, highlighting a promising approach for scalable self-improvement. Overall, MM-UPT offers a new paradigm for autonomous enhancement of MLLMs, serving as a critical third step after initial SFT and RL in the absence of external supervision. Our code is available at https://github.com/waltonfuture/MM-UPT.

Method: Conducted a large user study (n=2,976) and proposed a logit-based ensemble method for estimating LLM consistency that better aligns with human judgments.

Result: Current automated consistency metrics typically do not align well with human perceptions. The proposed logit-based ensemble method matches the performance of the best existing metric in estimating human ratings.

Conclusion: Automated consistency metrics are sufficiently imperfect to warrant broader use of human evaluation to avoid misjudging model adequacy based on flawed automated metrics.

Abstract: Large language models (LLMs) are prone to hallucinations and sensitiveto prompt perturbations, often resulting in inconsistent or unreliablegenerated text. Different methods have been proposed to mitigate suchhallucinations and fragility, one of which is to measure theconsistency of LLM responses – the model’s confidence in the responseor likelihood of generating a similar response when resampled. Inprevious work, measuring LLM response consistency often relied oncalculating the probability of a response appearing within a pool of resampledresponses, analyzing internal states, or evaluating logits of resopnses.However, it was not clear how well theseapproaches approximated users’ perceptions of consistency of LLMresponses. To find out, we performed a user study ($n=2,976$)demonstrating that current methods for measuring LLM responseconsistency typically do not align well with humans’ perceptions of LLMconsistency. We propose a logit-based ensemble method for estimatingLLM consistency and show that our method matches the performance of thebest-performing existing metric in estimating human ratings of LLMconsistency. Our results suggest that methods for estimating LLMconsistency without human evaluation are sufficiently imperfect towarrant broader use of evaluation with human input; this would avoidmisjudging the adequacy of models because of the imperfections ofautomated consistency metrics.

Method: Created TCM-Ladder dataset covering core TCM disciplines using automated and manual filtering, with various question types and modalities. Also proposed Ladder-Score evaluation method for assessing answer quality in TCM terminology and semantic expression.

Result: The dataset comprises over 52,000 questions across multiple modalities. Comparative experiments were conducted against nine general domain and five TCM-specific LLMs using the proposed evaluation framework.

Conclusion: This work provides the first systematic evaluation of mainstream LLMs on a unified multimodal TCM benchmark, with publicly available datasets and leaderboard for continuous updates.

Abstract: Traditional Chinese Medicine (TCM), as an effective alternative medicine, has been receiving increasing attention. In recent years, the rapid development of large language models (LLMs) tailored for TCM has highlighted the urgent need for an objective and comprehensive evaluation framework to assess their performance on real-world tasks. However, existing evaluation datasets are limited in scope and primarily text-based, lacking a unified and standardized multimodal question-answering (QA) benchmark. To address this issue, we introduce TCM-Ladder, the first comprehensive multimodal QA dataset specifically designed for evaluating large TCM language models. The dataset covers multiple core disciplines of TCM, including fundamental theory, diagnostics, herbal formulas, internal medicine, surgery, pharmacognosy, and pediatrics. In addition to textual content, TCM-Ladder incorporates various modalities such as images and videos. The dataset was constructed using a combination of automated and manual filtering processes and comprises over 52,000 questions. These questions include single-choice, multiple-choice, fill-in-the-blank, diagnostic dialogue, and visual comprehension tasks. We trained a reasoning model on TCM-Ladder and conducted comparative experiments against nine state-of-the-art general domain and five leading TCM-specific LLMs to evaluate their performance on the dataset. Moreover, we propose Ladder-Score, an evaluation method specifically designed for TCM question answering that effectively assesses answer quality in terms of terminology usage and semantic expression. To the best of our knowledge, this is the first work to systematically evaluate mainstream general domain and TCM-specific LLMs on a unified multimodal benchmark. The datasets and leaderboard are publicly available at https://tcmladder.com and will be continuously updated.

Method: LinearPatch fuses two operations into one matrix multiply at the pruning interface: (1) Hadamard transformation to suppress massive outliers at particular tokens, and (2) channel-wise scaling to align activation statistics. It can be further refined with 5K unlabeled samples via memory-efficient offline distillation.

Result: On LLaMA-3-8B, LinearPatch preserves up to 94.15% of original performance when pruning 5 out of 32 layers, outperforming previous state-of-the-art by 4%. With offline distillation, retention improves to 95.16% within 30 minutes on a single GPU.

Conclusion: LinearPatch effectively addresses the activation magnitude mismatch problem in layer pruning, achieving significant performance retention improvements with minimal computational overhead, making it a practical solution for LLM compression.

Abstract: Layer pruning has emerged as a widely used technique for compressing large language models (LLMs). However, existing layer pruning approaches often incur substantial performance degradation. We identify the majority of this degradation to a single yet previously overlooked issue: \textit{the mismatch of activation magnitudes at the pruning interface}. The pre-interface activations exhibit significantly different scales from the post-interface ones, causing the distributional shift as it propagates through the remaining layers. To address this issue, we introduce \textsc{LinearPatch}, a lightweight and plug-and-play technique that fuses two operations into one matrix multiply at the pruning interface: (i) a Hadamard transformation that suppresses massive outliers at particular tokens and (ii) a channel-wise scaling that aligns activation statistics. On LLaMA-3-8B, \textsc{LinearPatch} preserves up to \textbf{94.15%} of the original model’s performance when pruning 5 out of 32 layers, outperforming the previous state of the art by \textbf{4%}. The patch can be further refined with 5K unlabeled samples via memory-efficient offline distillation, pushing the retention to 95.16% within only 30 minutes on a single GPU. Code is available at https://github.com/chenxinrui-tsinghua/LinearPatch.

Method: Measure local dimensions of contextual language model’s latent space and analyze their shifts during training and fine-tuning across different tasks (dialogue state tracking, emotion recognition, arithmetic).

Result: Local dimensions predict training capability exhaustion, overfitting, grokking, and performance gains. Reductions in mean local dimension accompany and predict subsequent performance improvements.

Conclusion: The geometric approach provides practitioners with deeper understanding of fine-tuning effects on embedding spaces, bridging intrinsic model mechanisms with geometric properties for better interpretability and adaptability of LLMs.

Abstract: Understanding the internal mechanisms of large language models (LLMs) remains a challenging and complex endeavor. Even fundamental questions, such as how fine-tuning affects model behavior, often require extensive empirical evaluation. In this paper, we introduce a novel perspective based on the geometric properties of contextual latent embeddings to study the effects of training and fine-tuning. To that end, we measure the local dimensions of a contextual language model’s latent space and analyze their shifts during training and fine-tuning. We show that the local dimensions provide insights into the model’s training dynamics and generalization ability. Specifically, the mean of the local dimensions predicts when the model’s training capabilities are exhausted, as exemplified in a dialogue state tracking task, overfitting, as demonstrated in an emotion recognition task, and grokking, as illustrated with an arithmetic task. Furthermore, our experiments suggest a practical heuristic: reductions in the mean local dimension tend to accompany and predict subsequent performance gains. Through this exploration, we aim to provide practitioners with a deeper understanding of the implications of fine-tuning on embedding spaces, facilitating informed decisions when configuring models for specific applications. The results of this work contribute to the ongoing discourse on the interpretability, adaptability, and generalizability of LLMs by bridging the gap between intrinsic model mechanisms and geometric properties in the respective embeddings.

Method: Decomposed RLVR learning into Positive Sample Reinforcement (PSR) and Negative Sample Reinforcement (NSR), trained Qwen2.5-Math-7B, Qwen3-4B and Llama-3.1-8B-Instruct on mathematical reasoning dataset, analyzed gradient patterns, and proposed a variant that upweights NSR.

Result: Training with only negative samples consistently improved performance over base models across Pass@k spectrum (k up to 256), often matching or surpassing PPO and GRPO. NSR suppresses incorrect generations and redistributes probability mass toward plausible candidates based on model’s prior beliefs.

Conclusion: Solely penalizing incorrect responses contributes more to performance than previously recognized, working by refining existing knowledge rather than introducing new behaviors. The proposed NSR-upweighted variant consistently improves overall Pass@k performance on multiple mathematical reasoning benchmarks.

Abstract: Reinforcement learning with verifiable rewards (RLVR) is a promising approach for training language models (LMs) on reasoning tasks that elicit emergent long chains of thought (CoTs). Unlike supervised learning, it updates the model using both correct and incorrect samples via policy gradients. To better understand its mechanism, we decompose the learning signal into reinforcing correct responses and penalizing incorrect ones, referred to as Positive and Negative Sample Reinforcement (PSR and NSR), respectively. We train Qwen2.5-Math-7B, Qwen3-4B and Llama-3.1-8B-Instruct on a mathematical reasoning dataset and uncover a surprising result: training with only negative samples – without reinforcing correct responses – can be highly effective: it consistently improves performance over the base model across the entire Pass@$k$ spectrum $k$ up to 256), often matching or surpassing PPO and GRPO. In contrast, reinforcing only correct responses improves Pass@1 but degrades performance at higher $k$, due to reduced diversity. These inference-scaling trends highlight that solely penalizing incorrect responses may contribute more to performance than previously recognized. Through gradient analysis, we show that NSR works by suppressing incorrect generations and redistributing probability mass toward other plausible candidates, guided by the model’s prior beliefs. It refines the model’s existing knowledge rather than introducing entirely new behaviors. Building on this insight, we propose a simple variant of the RL objective that upweights NSR, and show that it consistently improves overall Pass@$k$ performance on MATH, AIME 2025, and AMC23. Our code is available at https://github.com/TianHongZXY/RLVR-Decomposed.

Method: Uses large language models, neuroscience ontology, and text embeddings to analyze semantic relevance of text segments for KG construction, plus entity-augmented information retrieval algorithm.

Result: Achieves F1 score of 0.84 for entity extraction from unlabeled data, comparable to supervised methods. Improves answers to over 52% of neuroscience questions from PubMedQA dataset.

Conclusion: The proposed methods significantly enhance knowledge discovery from unlabeled neuroscience research corpus and demonstrate effective KG construction without requiring labeled data.

Abstract: Neuroscience research publications encompass a vast wealth of knowledge. Accurately retrieving existing information and discovering new insights from this extensive literature is essential for advancing the field. However, when knowledge is dispersed across multiple sources, current state-of-the-art retrieval methods often struggle to extract the necessary information. A knowledge graph (KG) can integrate and link knowledge from multiple sources. However, existing methods for constructing KGs in neuroscience often rely on labeled data and require domain expertise. Acquiring large-scale, labeled data for a specialized area like neuroscience presents significant challenges. This work proposes novel methods for constructing KG from unlabeled large-scale neuroscience research corpus utilizing large language models (LLM), neuroscience ontology, and text embeddings. We analyze the semantic relevance of neuroscience text segments identified by LLM for building the knowledge graph. We also introduce an entity-augmented information retrieval algorithm to extract knowledge from the KG. Several experiments were conducted to evaluate the proposed approaches. The results demonstrate that our methods significantly enhance knowledge discovery from the unlabeled neuroscience research corpus. The performance of the proposed entity and relation extraction method is comparable to the existing supervised method. It achieves an F1 score of 0.84 for entity extraction from the unlabeled data. The knowledge obtained from the KG improves answers to over 52% of neuroscience questions from the PubMedQA dataset and questions generated using selected neuroscience entities.

Method: Formulates unlearning as constrained optimization with logit-margin flattening loss for forgetting (driving output distribution toward uniformity on forget set) and hard constraints for retention on retain set, solved using scalable primal-dual algorithm.

Result: Outperforms state-of-the-art baselines on TOFU and MUSE benchmarks across diverse LLM architectures, effectively removing targeted information while preserving downstream utility with stable optimization.

Conclusion: The constrained optimization formulation with logit-margin flattening provides a more stable and effective approach to LLM unlearning compared to regularized methods, enabling better trade-off management between forgetting and retention.

Abstract: Large Language Models (LLMs) deployed in real-world settings increasingly face the need to unlearn sensitive, outdated, or proprietary information. Existing unlearning methods typically formulate forgetting and retention as a regularized trade-off, combining both objectives into a single scalarized loss. This often leads to unstable optimization and degraded performance on retained data, especially under aggressive forgetting. We propose a new formulation of LLM unlearning as a constrained optimization problem: forgetting is enforced via a novel logit-margin flattening loss that explicitly drives the output distribution toward uniformity on a designated forget set, while retention is preserved through a hard constraint on a separate retain set. Compared to entropy-based objectives, our loss is softmax-free, numerically stable, and maintains non-vanishing gradients, enabling more efficient and robust optimization. We solve the constrained problem using a scalable primal-dual algorithm that exposes the trade-off between forgetting and retention through the dynamics of the dual variable, all without any extra computational overhead. Evaluations on the TOFU and MUSE benchmarks across diverse LLM architectures demonstrate that our approach consistently matches or exceeds state-of-the-art baselines, effectively removing targeted information while preserving downstream utility.

Method: Used the Magpie framework to annotate samples with quality metrics (turn structure, task category, input/output quality), analyzed structural differences, and designed a principled curation recipe to create TuluTalk.

Result: TuluTalk contains 14% fewer samples than source datasets while matching or exceeding their performance on key benchmarks. The analysis revealed structural and qualitative differences between datasets.

Conclusion: The work provides actionable insights for building effective post-training datasets within resource limits and releases annotated datasets and TuluTalk mixture to support future research.

Abstract: Recent work on large language models (LLMs) has increasingly focused on post-training and alignment with datasets curated to enhance instruction following, world knowledge, and specialized skills. However, most post-training datasets used in leading open- and closed-source LLMs remain inaccessible to the public, with limited information about their construction process. This lack of transparency has motivated the recent development of open-source post-training corpora. While training on these open alternatives can yield performance comparable to that of leading models, systematic comparisons remain challenging due to the significant computational cost of conducting them rigorously at scale, and are therefore largely absent. As a result, it remains unclear how specific samples, task types, or curation strategies influence downstream performance when assessing data quality. In this work, we conduct the first comprehensive side-by-side analysis of two prominent open post-training datasets: Tulu-3-SFT-Mix and SmolTalk. Using the Magpie framework, we annotate each sample with detailed quality metrics, including turn structure (single-turn vs. multi-turn), task category, input quality, and response quality, and we derive statistics that reveal structural and qualitative similarities and differences between the two datasets. Based on these insights, we design a principled curation recipe that produces a new data mixture, TuluTalk, which contains 14% fewer samples than either source dataset while matching or exceeding their performance on key benchmarks. Our findings offer actionable insights for constructing more effective post-training datasets that improve model performance within practical resource limits. To support future research, we publicly release both the annotated source datasets and our curated TuluTalk mixture.

Method: Conducted systematic experiments across various hardware, software, and precision settings; developed LayerCast pipeline that stores weights in 16-bit but computes in FP32 for better numerical stability.

Result: Found up to 9% accuracy variation and 9,000 token length differences in reasoning models due to GPU count, type, and batch size differences; identified non-associative floating-point arithmetic as root cause.

Conclusion: Numerical precision is critical for LLM reproducibility but often neglected; LayerCast provides a practical solution balancing memory efficiency with numerical stability.

Abstract: Large Language Models (LLMs) are now integral across various domains and have demonstrated impressive performance. Progress, however, rests on the premise that benchmark scores are both accurate and reproducible. We demonstrate that the reproducibility of LLM performance is fragile: changing system configuration, such as evaluation batch size, GPU count, and GPU version, can introduce significant differences in the generated responses. This issue is especially pronounced in reasoning models, where minor rounding differences in early tokens can cascade into divergent chains of thought, ultimately affecting accuracy. For instance, under bfloat16 precision with greedy decoding, a reasoning model like DeepSeek-R1-Distill-Qwen-7B can exhibit up to 9% variation in accuracy and 9,000 tokens difference in response length due to differences in GPU count, type, and evaluation batch size. We trace the root cause of this variability to the non-associative nature of floating-point arithmetic under limited numerical precision. This work presents the first systematic investigation into how numerical precision affects reproducibility in LLM inference. Through carefully controlled experiments across various hardware, software, and precision settings, we quantify when and how model outputs diverge. Our analysis reveals that floating-point precision - while critical for reproducibility - is often neglected in evaluation practices. Inspired by this, we develop a lightweight inference pipeline, dubbed LayerCast, that stores weights in 16-bit precision but performs all computations in FP32, balancing memory efficiency with numerical stability. Code is available at https://github.com/nanomaoli/llm_reproducibility.

Method: Proposed a causal framework incorporating Probability of Sufficiency and Necessity to quantify step influence, enabling automated addition of missing steps and pruning of redundant ones.

Result: Extensive experiments on mathematical and commonsense reasoning benchmarks show substantial improvements in reasoning efficiency and reduced token usage without accuracy loss.

Conclusion: Provides a promising direction for improving LLM reasoning performance and cost-effectiveness through causal analysis of reasoning steps.

Abstract: Chain-of-Thought (CoT) prompting plays an indispensable role in endowing large language models (LLMs) with complex reasoning capabilities. However, CoT currently faces two fundamental challenges: (1) Sufficiency, which ensures that the generated intermediate inference steps comprehensively cover and substantiate the final conclusion; and (2) Necessity, which identifies the inference steps that are truly indispensable for the soundness of the resulting answer. We propose a causal framework that characterizes CoT reasoning through the dual lenses of sufficiency and necessity. Incorporating causal Probability of Sufficiency and Necessity allows us not only to determine which steps are logically sufficient or necessary to the prediction outcome, but also to quantify their actual influence on the final reasoning outcome under different intervention scenarios, thereby enabling the automated addition of missing steps and the pruning of redundant ones. Extensive experimental results on various mathematical and commonsense reasoning benchmarks confirm substantial improvements in reasoning efficiency and reduced token usage without sacrificing accuracy. Our work provides a promising direction for improving LLM reasoning performance and cost-effectiveness.

Method: Formalized OCR as synthetic factual recall task, tested five LLMs, and analyzed one-layer single-head attention-only transformers with factorized vs. combined weight matrices.

Result: OCR drives both generalization and hallucination; models with factorized matrices can learn OCR while combined-weight models cannot; gradient descent’s implicit bias minimizes nuclear norm of output-value matrix.

Conclusion: OCR provides unified mechanism explaining LLM behavior, with theoretical foundation for understanding and mitigating undesirable behaviors from knowledge injection.

Abstract: Large language models (LLMs) can acquire new knowledge through fine-tuning, but this process exhibits a puzzling duality: models can generalize remarkably from new facts, yet are also prone to hallucinating incorrect information. However, the reasons for this phenomenon remain poorly understood. In this work, we argue that both behaviors stem from a single mechanism known as out-of-context reasoning (OCR): the ability to deduce implications by associating concepts, even those without a causal link. Our experiments across five prominent LLMs confirm that OCR indeed drives both generalization and hallucination, depending on whether the associated concepts are causally related. To build a rigorous theoretical understanding of this phenomenon, we then formalize OCR as a synthetic factual recall task. We empirically show that a one-layer single-head attention-only transformer with factorized output and value matrices can learn to solve this task, while a model with combined weights cannot, highlighting the crucial role of matrix factorization. Our theoretical analysis shows that the OCR capability can be attributed to the implicit bias of gradient descent, which favors solutions that minimize the nuclear norm of the combined output-value matrix. This mathematical structure explains why the model learns to associate facts and implications with high sample efficiency, regardless of whether the correlation is causal or merely spurious. Ultimately, our work provides a theoretical foundation for understanding the OCR phenomenon, offering a new lens for analyzing and mitigating undesirable behaviors from knowledge injection.

Method: Systematic pipeline that projects heterogeneous encoder states into token-type-aware document vectors, then applies centroid- or density-based clustering methods including k-Means, DBSCAN, HDBSCAN + k-NN, and BIRCH on eight different encoders.

Result: Reveals modality-specific failure modes and robustness-accuracy trade-off: vision features work well on clean pages for template discovery, text dominates under covariate shift, and fused encoders offer the best balance.

Conclusion: Provides reproducible tuning protocol and evaluation settings to guide future work on unsupervised document organization, demonstrating the effectiveness of multimodal approaches.

Abstract: We study unsupervised clustering of documents at both the category and template levels using frozen multimodal encoders and classical clustering algorithms. We systematize a model-agnostic pipeline that (i) projects heterogeneous last-layer states from text-layout-vision encoders into token-type-aware document vectors and (ii) performs clustering with centroid- or density-based methods, including an HDBSCAN + $k$-NN assignment to eliminate unlabeled points. We evaluate eight encoders (text-only, layout-aware, vision-only, and vision-language) with $k$-Means, DBSCAN, HDBSCAN + $k$-NN, and BIRCH on five corpora spanning clean synthetic invoices, their heavily degraded print-and-scan counterparts, scanned receipts, and real identity and certificate documents. The study reveals modality-specific failure modes and a robustness-accuracy trade-off, with vision features nearly solving template discovery on clean pages while text dominates under covariate shift, and fused encoders offering the best balance. We detail a reproducible, oracle-free tuning protocol and the curated evaluation settings to guide future work on unsupervised document organization.

Method: The paper conducts a comprehensive survey analyzing existing LLM-based approaches, benchmarks, and evaluation frameworks for trial-patient matching.

Result: The survey identifies limitations in current LLM applications for clinical trial recruitment and provides critical examination of existing methods.

Conclusion: The paper highlights challenges in adopting LLM technologies in clinical research and outlines exciting future directions for improvement.

Abstract: Recent advances in LLMs have greatly improved general-domain NLP tasks. Yet, their adoption in critical domains, such as clinical trial recruitment, remains limited. As trials are designed in natural language and patient data is represented as both structured and unstructured text, the task of matching trials and patients benefits from knowledge aggregation and reasoning abilities of LLMs. Classical approaches are trial-specific and LLMs with their ability to consolidate distributed knowledge hold the potential to build a more general solution. Yet recent applications of LLM-assisted methods rely on proprietary models and weak evaluation benchmarks. In this survey, we are the first to analyze the task of trial-patient matching and contextualize emerging LLM-based approaches in clinical trial recruitment. We critically examine existing benchmarks, approaches and evaluation frameworks, the challenges to adopting LLM technologies in clinical research and exciting future directions.

Method: Developed HAF criterion and six metrics based on uncertainty quantification to evaluate LLMs’ toxicity explanations without human involvement, tested on Llama models (up to 70B) and Ministral model across five toxicity datasets.

Result: LLMs generate plausible explanations to simple prompts but show inconsistent and irrelevant responses when prompted about nuanced relations between reasons, individual reasons, and toxicity stances.

Conclusion: LLMs’ reasoning about toxicity breaks down in complex scenarios, highlighting the need for better evaluation methods like HAF to improve trustworthiness in toxicity-related tasks.

Abstract: The discourse around toxicity and LLMs in NLP largely revolves around detection tasks. This work shifts the focus to evaluating LLMs’ reasoning about toxicity – from their explanations that justify a stance – to enhance their trustworthiness in downstream tasks. Despite extensive research on explainability, it is not straightforward to adopt existing methods to evaluate free-form toxicity explanation due to their over-reliance on input text perturbations, among other challenges. To account for these, we propose a novel, theoretically-grounded multi-dimensional criterion, Human-Aligned Faithfulness (HAF), that measures the extent to which LLMs’ free-form toxicity explanations align with those of a rational human under ideal conditions. We develop six metrics, based on uncertainty quantification, to comprehensively evaluate HAF of LLMs’ toxicity explanations with no human involvement, and highlight how “non-ideal” the explanations are. We conduct several experiments on three Llama models (of size up to 70B) and an 8B Ministral model on five diverse toxicity datasets. Our results show that while LLMs generate plausible explanations to simple prompts, their reasoning about toxicity breaks down when prompted about the nuanced relations between the complete set of reasons, the individual reasons, and their toxicity stances, resulting in inconsistent and irrelevant responses. We open-source our code at https://github.com/uofthcdslab/HAF and LLM-generated explanations at https://huggingface.co/collections/uofthcdslab/haf.

Method: Proposes DeepTalk with adaptive modality expert learning using MoE architecture - distinguishes modality experts by load, performs specialized single-modality training, then joint multimodal collaborative training.

Result: Achieves only 5.5% performance drop vs original LLM (compared to 20%+ in native MLLMs like GLM-4-Voice), maintains end-to-end latency under 0.5 seconds, and performs on par with modular MLLMs.

Conclusion: DeepTalk effectively addresses catastrophic forgetting in native MLLMs through adaptive modality expert learning, enabling seamless speech interaction while preserving LLM performance.

Abstract: Native multimodal large language models (MLLMs) restructure a single large language model (LLM) into a spoken language model (SLM) capable of both speech and text generation. Compared to modular and aligned MLLMs, native MLLMs preserve richer paralinguistic features such as emotion and prosody, and generate speech responses directly within the backbone LLM rather than using a separate speech decoder. This integration also results in lower response latency and smoother interaction. However, native MLLMs suffer from catastrophic forgetting and performance degradation because the available paired speech-text data is insufficient to support the pretraining of MLLMs compared to the vast amount of text data required to pretrain text LLMs. To address this issue, we propose DeepTalk, a framework for adaptive modality expert learning based on a Mixture of Experts (MoE) architecture. DeepTalk first adaptively distinguishes modality experts according to their modality load within the LLM. Each modality expert then undergoes specialized single-modality training, followed by joint multimodal collaborative training. As a result, DeepTalk incurs only a 5.5% performance drop compared to the original LLM, which is significantly lower than the average performance drop of over 20% typically seen in native MLLMs (such as GLM-4-Voice), and is on par with modular MLLMs. Meanwhile, the end-to-end dialogue latency remains within 0.5 seconds, ensuring a seamless and intelligent speech interaction experience. Code and models are released at https://github.com/talkking/DeepTalk.

Method: Use of simple supervision to improve language model alignment, evaluated over three datasets spanning various topics and multiple LLMs with different prompting strategies.

Result: Greatly improved language model alignment with diverse population groups more consistently, with insights into how alignment varies across specific groups.

Conclusion: Broad findings provide insights into distributional alignment of LLMs with diverse groups, and the work serves as a benchmark to stimulate future research.

Abstract: The ability to accurately align LLMs with human population groups on subjective questions would have great value. In this work, we show that use of simple supervision can greatly improve language model alignment with diverse population groups more consistently, as measured over three datasets spanning various topics. Beyond evaluating average alignment, we also report how alignment varies across specific groups. Our broad findings provide insights into the distributional alignment of LLMs with diverse population groups. By conducting evaluation over many LLMs and prompting strategies, along with open-sourcing our work, we provide a benchmark to stimulate future research.

Method: ARF (Adaptive Reward-Following) extracts continuous preference trajectories from free-form natural feedback using the novel TraceBias optimization algorithm.

Result: ARF consistently outperforms PPO and DPO across diverse LLMs and preference domains, improving alignment by up to 7.6%.

Conclusion: Continuous reward modeling provides a scalable path toward personalized and theoretically grounded RLHF.

Abstract: Current RLHF methods such as PPO and DPO typically reduce human preferences to binary labels, which are costly to obtain and too coarse to reflect individual variation. We observe that expressions of satisfaction and dissatisfaction follow stable linguistic patterns across users, indicating that more informative supervisory signals can be extracted from free-form feedback. Building on this insight, we introduce Adaptive Reward-Following (ARF), which converts natural feedback into continuous preference trajectories and optimizes them using the novel TraceBias algorithm. Across diverse LLMs and preference domains, ARF consistently outperforms PPO and DPO, improving alignment by up to 7.6%. Our results demonstrate that continuous reward modeling provides a scalable path toward personalized and theoretically grounded RLHF.

Method: AGENT KB aggregates trajectories into a structured knowledge base with lightweight APIs, using hybrid retrieval with two stages: planning seeds agents with cross-domain workflows, and feedback applies targeted diagnostic fixes. A disagreement gate prevents knowledge interference.

Result: Substantial improvements across major frameworks: smolagents achieved up to 18.7pp gains at pass@3 (55.2% -> 73.9%), OpenHands improved 4.0pp on SWE-bench pass@1 (24.3% -> 28.3%). Similar gains observed across all model families.

Conclusion: AGENT KB establishes the foundation for collective agent intelligence through shared memory infrastructures, with automatically generated experiences matching manual curation and enabling seamless cross-framework knowledge transfer.

Abstract: AI agent frameworks operate in isolation, forcing agents to rediscover solutions and repeat mistakes across different systems. Despite valuable problem-solving experiences accumulated by frameworks like smolagents, OpenHands, and OWL, this knowledge remains trapped within individual systems, preventing the emergence of collective intelligence. Current memory systems focus on individual agents or framework-specific demonstrations, failing to enable cross-architecture knowledge transfer. We introduce AGENT KB, a universal memory infrastructure enabling seamless experience sharing across heterogeneous agent frameworks without retraining. AGENT KB aggregates trajectories into a structured knowledge base and serves lightweight APIs. At inference time, hybrid retrieval operates through two stages: planning seeds agents with cross-domain workflows, while feedback applies targeted diagnostic fixes. A disagreement gate ensures retrieved knowledge enhances rather than disrupts reasoning, addressing knowledge interference in cross-framework transfer. We validate AGENT KB across major frameworks on GAIA, Humanity’s Last Exam, GPQA, and SWE-bench. Results show substantial improvements across diverse model families: compared to baseline pass@1, smolagents with AGENT KB achieve up to 18.7pp gains at pass@3 (55.2% -> 73.9%), while OpenHands improves 4.0pp on SWE-bench pass@1 (24.3% -> 28.3%). Similar improvements are observed across all base model families. Ablations confirm that hybrid retrieval and feedback stages are essential, with automatically generated experiences matching manual curation. This establishes the foundation for collective agent intelligence through shared memory infrastructures.

Method: Developed Gated Memory Unit (GMU) mechanism and applied it to create SambaY - a decoder-hybrid-decoder architecture with GMUs in cross-decoder to share memory readout states from Samba-based self-decoder.

Result: Significantly lower irreducible loss than YOCO baseline, superior performance scalability, Phi4-mini-Flash-Reasoning achieves better performance on reasoning tasks without RL, and up to 10x higher decoding throughput on long prompts.

Conclusion: GMU-based SambaY architecture enables efficient memory sharing across SSM layers, delivering significant performance improvements and scalability advantages for large-scale language modeling.

Abstract: Recent advances in language modeling have demonstrated the effectiveness of State Space Models (SSMs) for efficient sequence modeling. While hybrid architectures such as Samba and the decoder-decoder architecture, YOCO, have shown promising performance gains over Transformers, prior works have not investigated the efficiency potential of representation sharing between SSM layers. In this paper, we introduce the Gated Memory Unit (GMU), a simple yet effective mechanism for efficient memory sharing across layers. We apply it to create SambaY, a decoder-hybrid-decoder architecture that incorporates GMUs in the cross-decoder to share memory readout states from a Samba-based self-decoder. SambaY significantly enhances decoding efficiency, preserves linear pre-filling time complexity, and boosts long-context performance, all while eliminating the need for explicit positional encoding. Through extensive scaling experiments, we demonstrate that our model exhibits a significantly lower irreducible loss compared to a strong YOCO baseline, indicating superior performance scalability under large-scale compute regimes. Our largest model enhanced with Differential Attention, Phi4-mini-Flash-Reasoning, achieves significantly better performance than Phi4-mini-Reasoning on reasoning tasks such as Math500, AIME24/25, and GPQA Diamond without any reinforcement learning, while delivering up to 10x higher decoding throughput on 2K-length prompts with 32K generation length under the vLLM inference framework. We release our training codebase on open-source data at https://github.com/microsoft/ArchScale.

Method: Used benchmarks derived from real-world corporate datasets with all combinations of Arabic and English query-document pairs. Proposed two retrieval strategies: enforcing equal retrieval from both languages and query translation.

Result: Found that retrieval is a critical bottleneck in cross-lingual domain-specific scenarios, with substantial performance drops when query and document languages differ. The proposed strategies resulted in substantial improvements in cross-lingual and overall performance.

Conclusion: Retrieval challenges are the primary bottleneck in cross-lingual RAG, particularly in practical applications. Simple retrieval strategies can effectively address cross-lingual ranking difficulties and improve performance.

Abstract: Cross-lingual retrieval-augmented generation (RAG) is a critical capability for retrieving and generating answers across languages. Prior work in this context has mostly focused on generation and relied on benchmarks derived from open-domain sources, most notably Wikipedia. In such settings, retrieval challenges often remain hidden due to language imbalances, overlap with pretraining data, and memorized content. To address this gap, we study Arabic-English RAG in a domain-specific setting using benchmarks derived from real-world corporate datasets. Our benchmarks include all combinations of languages for the user query and the supporting document, drawn independently and uniformly at random. This enables a systematic study of multilingual retrieval behavior. Our findings reveal that retrieval is a critical bottleneck in cross-lingual domain-specific scenarios, with substantial performance drops occurring when the user query and supporting document languages differ. A key insight is that these failures stem primarily from the retriever’s difficulty in ranking documents across languages. Finally, we propose two simple retrieval strategies that address this source of failure by enforcing equal retrieval from both languages or by translating the query, resulting in substantial improvements in cross-lingual and overall performance. These results highlight meaningful opportunities for improving multilingual retrieval, particularly in practical, real-world RAG applications.

Method: Introduces DATE-LM benchmark that measures attribution quality through three real-world LLM application tasks and enables large-scale evaluations across diverse tasks and LLM architectures.

Result: Large-scale evaluation shows no single method dominates across all tasks, data attribution methods have trade-offs with simpler baselines, and method performance is sensitive to task-specific evaluation design.

Conclusion: DATE-LM serves as a foundation for future data attribution research in LLMs, with a public leaderboard released for community engagement and method comparison.

Abstract: Data attribution methods quantify the influence of training data on model outputs and are becoming increasingly relevant for a wide range of LLM research and applications, including dataset curation, model interpretability, data valuation. However, there remain critical gaps in systematic LLM-centric evaluation of data attribution methods. To this end, we introduce DATE-LM (Data Attribution Evaluation in Language Models), a unified benchmark for evaluating data attribution methods through real-world LLM applications. DATE-LM measures attribution quality through three key tasks – training data selection, toxicity/bias filtering, and factual attribution. Our benchmark is designed for ease of use, enabling researchers to configure and run large-scale evaluations across diverse tasks and LLM architectures. Furthermore, we use DATE-LM to conduct a large-scale evaluation of existing data attribution methods. Our findings show that no single method dominates across all tasks, data attribution methods have trade-offs with simpler baselines, and method performance is sensitive to task-specific evaluation design. Finally, we release a public leaderboard for quick comparison of methods and to facilitate community engagement, with the motivation that DATE-LM can serve as a foundation for future data attribution research in LLMs.

Method: MoR uses a shared stack of layers across recursion steps for parameter efficiency, with lightweight routers that dynamically assign different recursion depths to individual tokens. It focuses attention computation only on active tokens and selectively caches their key-value pairs. A KV sharing variant reuses KV pairs from the first recursion to further reduce memory footprint.

Result: Across model scales from 135M to 1.7B parameters, MoR forms a new Pareto frontier: at equal training FLOPs and smaller model sizes, it significantly lowers validation perplexity, improves few-shot accuracy, and delivers higher throughput compared to vanilla and existing recursive baselines.

Conclusion: MoR is an effective path towards achieving large-model quality without incurring large-model costs, demonstrating that unified parameter sharing and adaptive computation can significantly improve efficiency.

Abstract: Scaling language models unlocks impressive capabilities, but the accompanying computational and memory demands make both training and deployment expensive. Existing efficiency efforts typically target either parameter sharing or adaptive computation, leaving open the question of how to attain both simultaneously. We introduce Mixture-of-Recursions (MoR), a unified framework that combines the two axes of efficiency inside a single Recursive Transformer. MoR reuses a shared stack of layers across recursion steps to achieve parameter efficiency, while lightweight routers enable adaptive token-level thinking by dynamically assigning different recursion depths to individual tokens. This allows MoR to focus quadratic attention computation only among tokens still active at a given recursion depth, further improving memory access efficiency by selectively caching only their key-value pairs. Beyond these core mechanisms, we also propose a KV sharing variant that reuses KV pairs from the first recursion, specifically designed to further decrease memory footprint. Across model scales ranging from 135M to 1.7B parameters, MoR forms a new Pareto frontier: at equal training FLOPs and smaller model sizes, it significantly lowers validation perplexity and improves few-shot accuracy, while delivering higher throughput compared with vanilla and existing recursive baselines. These gains demonstrate that MoR is an effective path towards large-model quality without incurring large-model cost.

Method: Proposes a framework with two evaluation aspects: Multi-modal Code Generation (MCG) for understanding and constructing visualizations, and Multi-modal Code Editing (MCE) for fine-grained operations including Deletion, Modification and Annotation. Uses a dataset covering five types of mathematical figures and evaluates nine mainstream MLLMs.

Result: Experimental results reveal that existing models still lag significantly behind human performance in performing fine-grained visual operations.

Conclusion: There is a significant gap between current MLLM capabilities and human performance in code-based multi-modal mathematical reasoning, particularly in fine-grained visual operations.

Abstract: Recent progress in Multi-modal Large Language Models (MLLMs) has enabled step-by-step multi-modal mathematical reasoning by performing visual operations based on the textual instructions. A promising approach uses code as an intermediate representation to precisely express and manipulate the images in the reasoning steps. However, existing evaluations focus mainly on text-only reasoning outputs, leaving the MLLM’s ability to perform accurate visual operations via code largely unexplored. This work takes a first step toward addressing that gap by evaluating MLLM’s code-based capabilities in multi-modal mathematical reasoning.Specifically, our framework focuses on two key evaluation aspects: (1) Multi-modal Code Generation (MCG) evaluates the model’s ability to accurately understand and construct visualizations from scratch. (2) Multi-modal Code Editing (MCE) assesses the model’s capacity for fine-grained operations, which include three types: Deletion, Modification and Annotation. To evaluate the above tasks, we incorporate a dataset that covers the five most popular types of mathematical figures, including geometric diagrams, function plots, and three types of statistical charts, to provide a comprehensive and effective measurement of existing MLLMs. Our experimental evaluation involves nine mainstream MLLMs, and the results reveal that existing models still lag significantly behind human performance in performing fine-grained visual operations.

Method: Break down knowledge extraction into NER, coreference resolution, named entity linking, and relation extraction components. Evaluate 16 NLP tools and LLMs using a baseline dataset from FAA maintenance records, focusing on zero-shot performance in controlled environments.

Result: Observed significant performance limitations of NLP and LLM tools for trusted applications in confidential environments, highlighting challenges with technical readiness for mission-critical industries.

Conclusion: Provide recommendations to enhance trust in NLP/LLM tools and release an open-source curated dataset to support further baseline testing and evaluation in trusted environments.

Abstract: Deriving operational intelligence from organizational data repositories is a key challenge due to the dichotomy of data confidentiality vs data integration objectives, as well as the limitations of Natural Language Processing (NLP) tools relative to the specific knowledge structure of domains such as operations and maintenance. In this work, we discuss Knowledge Graph construction and break down the Knowledge Extraction process into its Named Entity Recognition, Coreference Resolution, Named Entity Linking, and Relation Extraction functional components. We then evaluate sixteen NLP tools in concert with or in comparison to the rapidly advancing capabilities of Large Language Models (LLMs). We focus on the operational and maintenance intelligence use case for trusted applications in the aircraft industry. A baseline dataset is derived from a rich public domain US Federal Aviation Administration dataset focused on equipment failures or maintenance requirements. We assess the zero-shot performance of NLP and LLM tools that can be operated within a controlled, confidential environment (no data is sent to third parties). Based on our observation of significant performance limitations, we discuss the challenges related to trusted NLP and LLM tools as well as their Technical Readiness Level for wider use in mission-critical industries such as aviation. We conclude with recommendations to enhance trust and provide our open-source curated dataset to support further baseline testing and evaluation.

Method: Uses scalable fully asynchronous RL training for long-horizon search and a prompt-based LLM agent to autonomously synthesize high-quality QA datasets for training.

Result: ASearcher-Web-QwQ achieves 51.1 Avg@4 on xBench and 58.7 on GAIA, surpassing existing 32B agents, with training showing extreme long-horizon search (100+ tool calls, 400k+ output tokens).

Conclusion: ASearcher demonstrates that open-source agents can achieve commercial-level performance through large-scale RL training and autonomous data synthesis, enabling expert-level search capabilities.

Abstract: Recent advancements in LLM-based agents have demonstrated remarkable capabilities in handling complex, knowledge-intensive tasks by integrating external tools. Among diverse choices of tools, search tools play a pivotal role in accessing vast external knowledge. However, open-source agents still fall short of achieving expert-level Search Intelligence, the ability to resolve ambiguous queries, generate precise searches, analyze results, and conduct thorough exploration. Existing approaches fall short in scalability, efficiency, and data quality. For example, small turn limits in existing online RL methods, e.g. <=10, restrict complex strategy learning. This paper introduces ASearcher, an open-source project for large-scale RL training of search agents. Our key contributions include: (1) Scalable fully asynchronous RL training that enables long-horizon search while maintaining high training efficiency. (2) A prompt-based LLM agent that autonomously synthesizes high-quality and challenging QAs, creating a large-scale QA dataset. Through RL training, our prompt-based QwQ-32B agent achieves substantial improvements, with 78.0% and 34.3% Avg@4 gains on xBench and GAIA, respectively. Notably, our agent exhibits extreme long-horizon search, with tool calls exceeding 100 turns and output tokens exceeding 400k during training time. With a simple agent design and no external LLMs, ASearcher-Web-QwQ achieves Avg@4 scores of 51.1 on xBench and 58.7 on GAIA, surpassing existing open-source 32B agents. Finally, we also show that ASearcher-Web-QwQ could achieve performance of commercial systems using external summary tool in a zero-shot transfer manner and test-time search. We open-source our models, training data, and codes in https://github.com/inclusionAI/ASearcher.

Method: Controlled study scaling vocabulary from 24K to 196K while holding data, computation, and optimization constant; analyzed tokenized text complexity via Kolmogorov complexity and word-level loss decomposition.

Result: Larger vocabularies reduce cross-entropy loss primarily by lowering uncertainty on the 2,500 most frequent words (covering ~75% of downstream tokens), while loss on rare words increases. Same benefit achieved by enlarging model parameters with fixed vocabulary.

Conclusion: Bigger vocabularies help by lowering complexity of tokenized text, offering a principled approach for tokenizer-model co-design and clarifying loss dynamics in language model scaling.

Abstract: Large language models are trained with tokenizers, and the resulting token distribution is highly imbalanced: a few words dominate the stream while most occur rarely. Recent practice favors ever-larger vocabularies, but it is unclear where the benefit comes from. To this end, we perform a controlled study that scales the vocabulary of the language model from 24K to 196K while holding data, computation, and optimization unchanged. We begin by quantifying the complexity of tokenized text – formalized via Kolmogorov complexity – and show that larger vocabularies reduce this complexity. Above 24K, every common word is already tokenized as a single token, so enlarging vocabulary only deepens the relative token-frequency imbalance. Word-level loss decomposition shows that larger vocabularies reduce cross-entropy loss almost exclusively by lowering uncertainty on the 2,500 most frequent words, even though loss on the rare tail rises. The same frequent words cover roughly 75% of tokens in downstream benchmarks, so this training advantage transfers intact. We further show that enlarging model parameters with a fixed vocabulary yields the same frequent-word benefit. Our results recast “bigger vocabularies help” as “lowering complexity of tokenized text helps,” offering a simple, principled knob for tokenizer-model co-design and clarifying the loss dynamics that govern language model scaling in pre-training.

Method: Combines language-specific BERT encoders with graph transformer networks, creating distinct semantic projections where synonymous pairs cluster in one space and antonymous pairs exhibit high similarity in a complementary space.

Result: Achieves competitive performance against state-of-the-art baselines across eight languages (English, German, French, Spanish, Italian, Portuguese, Dutch, Russian) with effective cross-lingual generalization.

Conclusion: Semantic relationship modeling transfers effectively across languages, providing interpretable semantic representations and robust cross-lingual antonym-synonym distinction capabilities.

Abstract: Antonym vs synonym distinction across multiple languages presents unique computational challenges due to the paradoxical nature of antonymous relationships words that share semantic domains while expressing opposite meanings. This work introduces Bhav-Net, a novel dual-space architecture that enables effective knowledge transfer from complex multilingual models to simpler, language-specific architectures while maintaining robust cross-lingual antonym–synonym distinction capabilities. Our approach combines language-specific BERT encoders with graph transformer networks, creating distinct semantic projections where synonymous pairs cluster in one space while antonymous pairs exhibit high similarity in a complementary space. Through comprehensive evaluation across eight languages (English, German, French, Spanish, Italian, Portuguese, Dutch, and Russian), we demonstrate that semantic relationship modeling transfers effectively across languages. The dual-encoder design achieves competitive performance against state-of-the-art baselines while providing interpretable semantic representations and effective cross-lingual generalization.

Method: Constructed ClaimGen-CN dataset from real-world legal disputes, designed evaluation metrics for factuality and clarity, and conducted comprehensive zero-shot evaluation of general and legal-domain LLMs.

Result: Current models show limitations in factual precision and expressive clarity when generating legal claims, indicating the need for more targeted development in this domain.

Conclusion: The research highlights the challenges in legal claim generation and provides a publicly available dataset to encourage further exploration of this important task for assisting non-professionals.

Abstract: Legal claims refer to the plaintiff’s demands in a case and are essential to guiding judicial reasoning and case resolution. While many works have focused on improving the efficiency of legal professionals, the research on helping non-professionals (e.g., plaintiffs) remains unexplored. This paper explores the problem of legal claim generation based on the given case’s facts. First, we construct ClaimGen-CN, the first dataset for Chinese legal claim generation task, from various real-world legal disputes. Additionally, we design an evaluation metric tailored for assessing the generated claims, which encompasses two essential dimensions: factuality and clarity. Building on this, we conduct a comprehensive zero-shot evaluation of state-of-the-art general and legal-domain large language models. Our findings highlight the limitations of the current models in factual precision and expressive clarity, pointing to the need for more targeted development in this domain. To encourage further exploration of this important task, we will make the dataset publicly available.

Method: Hybrid approach combining PRISMA guidelines with fuzzy matching and regular expressions for semi-automated deduplication and filtering, with modified splitting method for multi-arm trials.

Result: Workflow reduced screening workload significantly (11 days for 33,444 records), identified 7 eligible RCTs (841 patients), showing 36% reduction in recurrence (RR=0.64, 95% CI 0.48-0.86) with no heterogeneity.

Conclusion: The information-retrieval-driven workflow successfully bridges clinical research and computer science, providing a generalizable framework for scalable evidence synthesis with robust clinical results.

Abstract: Background: Evidence synthesis facilitates evidence-based medicine. This task becomes increasingly difficult to accomplished with applying computational solutions, since the medical literature grows at astonishing rates. Objective: This study evaluates an information retrieval-driven workflow, CASMA, to enhance the efficiency, transparency, and reproducibility of systematic reviews. Endometriosis recurrence serves as the ideal case due to its complex and ambiguous literature. Methods: The hybrid approach integrates PRISMA guidelines with fuzzy matching and regular expression (regex) to facilitate semi-automated deduplication and filtered records before manual screening. The workflow synthesised evidence from randomised controlled trials on the efficacy of a subclass of gonadotropin-releasing hormone agonists (GnRH-a). A modified splitting method addressed unit-of-analysis errors in multi-arm trials. Results: The workflow sharply reduced the screening workload, taking only 11 days to fetch and filter 33,444 records. Seven eligible RCTs were synthesized (841 patients). The pooled random-effects model yielded a Risk Ratio (RR) of $0.64$ ($95%$ CI $0.48$ to $0.86$), demonstrating a $36%$ reduction in recurrence, with non-significant heterogeneity ($I^2=0.00%$, $\tau^2=0.00$). The findings were robust and stable, as they were backed by sensitivity analyses. Conclusion: This study demonstrates an application of an information-retrieval-driven workflow for medical evidence synthesis. The approach yields valuable clinical results and a generalisable framework to scale up the evidence synthesis, bridging the gap between clinical research and computer science.

Method: Used mutual information between visual information and word identity as an operationalization of input quality; conducted reading experiments with occluded word halves; employed multimodal language models to estimate mutual information; compared English and Chinese reading patterns.

Result: Upper halves contain more information about word identity than lower halves in both languages, but the asymmetry is more pronounced in English; reading times increased when information quality was reduced through occlusion.

Conclusion: Information quality of bottom-up inputs significantly affects reading comprehension difficulty, with visual information distribution patterns varying across writing systems but consistently showing upper-half dominance in word identity information.

Abstract: Contemporary theories model language processing as integrating both top-down expectations and bottom-up inputs. One major prediction of such models is that the quality of the bottom-up inputs modulates ease of processing – noisy inputs should lead to difficult and effortful comprehension. We test this prediction in the domain of reading. First, we propose an information-theoretic operationalization for the “quality” of bottom-up information as the mutual information (MI) between visual information and word identity. We formalize this prediction in a mathematical model of reading as a Bayesian update. Second, we test our operationalization by comparing participants’ reading times in conditions where words’ information quality has been reduced, either by occluding their top or bottom half, with full words. We collect data in English and Chinese. We then use multimodal language models to estimate the mutual information between visual inputs and words. We use these data to estimate the specific effect of reduced information quality on reading times. Finally, we compare how information is distributed across visual forms. In English and Chinese, the upper half contains more information about word identity than the lower half. However, the asymmetry is more pronounced in English, a pattern which is reflected in the reading times.

Method: Created WolBanking77 dataset containing 9,791 text sentences and 4+ hours of spoken sentences in Wolof banking domain. Conducted experiments with various text and voice state-of-the-art models as baselines.

Result: Promising results on the dataset with reported F1-scores for NLP models and word error rates for ASR models trained on WolBanking77, along with model comparisons.

Conclusion: The paper presents a valuable resource for intent classification research in low-resource languages and provides baseline performance metrics that show the dataset’s utility for advancing NLP in underrepresented languages like Wolof.

Abstract: Intent classification models have made a significant progress in recent years. However, previous studies primarily focus on high-resource language datasets, which results in a gap for low-resource languages and for regions with high rates of illiteracy, where languages are more spoken than read or written. This is the case in Senegal, for example, where Wolof is spoken by around 90% of the population, while the national illiteracy rate remains at of 42%. Wolof is actually spoken by more than 10 million people in West African region. To address these limitations, we introduce the Wolof Banking Speech Intent Classification Dataset (WolBanking77), for academic research in intent classification. WolBanking77 currently contains 9,791 text sentences in the banking domain and more than 4 hours of spoken sentences. Experiments on various baselines are conducted in this work, including text and voice state-of-the-art models. The results are very promising on this current dataset. In addition, this paper presents an in-depth examination of the dataset’s contents. We report baseline F1-scores and word error rates metrics respectively on NLP and ASR models trained on WolBanking77 dataset and also comparisons between models. Dataset and code available at: https://github.com/abdoukarim/wolbanking77.

Method: A practical, batch-wise alignment strategy for fine-tuning LLMs that leverages semantically equivalent multilingual data in each training batch to directly align model outputs across languages.

Result: The approach improves non-English accuracy by up to 23.9% without compromising English performance, model reasoning, or retrieval quality. It addresses the observed 29% accuracy drop in non-English languages compared to English.

Conclusion: The method is simple to implement, scalable, and integrates seamlessly with existing LLM training & deployment pipelines, enabling more robust and equitable multilingual AI solutions in industry.

Abstract: Large language models (LLMs) remain unreliable for global enterprise applications due to substantial performance gaps between high-resource and mid/low-resource languages, driven by English-centric pretraining and internal reasoning biases. This inconsistency undermines customer experience and operational reliability in multilingual settings such as customer support, content moderation, and information retrieval. Even with advanced Retrieval-Augmented Generation (RAG) systems, we observe up to an 29% accuracy drop in non-English languages compared to English. We propose a practical, batch-wise alignment strategy for fine-tuning LLMs, leveraging semantically equivalent multilingual data in each training batch to directly align model outputs across languages. This approach improves non-English accuracy by up to 23.9% without compromising English performance, model reasoning, or retrieval quality. Our method is simple to implement, scalable, and integrates seamlessly with existing LLM training & deployment pipelines, enabling more robust and equitable multilingual AI solutions in industry.

Method: A pipeline with robust media retrieval and a two-stage alignment algorithm designed to handle non-verbatim transcripts and long-form audio from parliamentary recordings.

Result: Extracted over 61k hours of aligned speech segments from 22 European parliaments, with 19 languages exceeding 1k hours and 22 languages exceeding 500 hours. Achieved 41.8% average reduction in word error rates when finetuning ASR models.

Conclusion: The proposed pipeline effectively addresses data scarcity in multilingual speech processing by leveraging parliamentary recordings to create large-scale, high-quality speech datasets.

Abstract: Recent progress in speech processing has highlighted that high-quality performance across languages requires substantial training data for each individual language. While existing multilingual datasets cover many languages, they often contain insufficient data for most languages. Thus, trained models perform poorly on the majority of the supported languages. Our work addresses this challenge by introducing a scalable pipeline for constructing speech datasets from parliamentary recordings. The proposed pipeline includes robust components for media retrieval and a two-stage alignment algorithm designed to handle non-verbatim transcripts and long-form audio. Applying this pipeline to recordings from 22 European parliaments, we extract over 61k hours of aligned speech segments, achieving substantial per-language coverage with 19 languages exceeding 1k hours and 22 languages exceeding 500 hours of high-quality speech data. We obtain an average 41.8% reduction in word error rates over baselines when finetuning an existing ASR model on our dataset, demonstrating the usefulness of our approach.

Method: Diagnosed overthinking via oracle rollouts injecting at sentence boundaries. Introduced ThinkBrake - a training-free decoding heuristic that monitors the log-probability margin between and the current top token at sentence boundaries, triggering termination when this margin becomes small.

Result: Oracle termination lifted average accuracy from 85.8% to 94.2% while reducing tokens by 80-94%. ThinkBrake preserved or improved accuracy across BFCL’s single turn, non-live and live splits while reducing tokens up to 25%, outperforming various baselines.

Conclusion: ThinkBrake effectively addresses overthinking in small reasoning models during tool use, demonstrating that training-free early termination methods can significantly reduce computational overhead while maintaining or improving performance.

Abstract: Small reasoning models (SRMs) often overthink during tool use: they reach a correct tool-argument configuration, then continue reasoning and overwrite it with an incorrect final call. We diagnose overthinking via oracle rollouts that inject at sentence boundaries. On the Berkeley Function Calling Leaderboard (BFCL), this oracle termination lifts average accuracy from 85.8% to 94.2% while reducing tokens by 80-94%, revealing substantial recoverable headroom and potential redundant reasoning. While prior work on concise reasoning has largely targeted mathematics, tool reasoning remains underexplored. We adapt various early-termination baselines to tool use and introduce ThinkBrake, a training-free decoding heuristic. ThinkBrake monitors the log-probability margin between and the current top token at sentence boundaries and triggers termination when this margin becomes small. Across BFCL’s single turn, non-live and live splits, ThinkBrake preserves or improves accuracy while reducing tokens up to 25%, outperforming various baselines.

Method: Mechanistic analysis of moral self-correction through multi-round interactions, examining how self-correction instructions activate moral concepts and reduce model uncertainty.

Result: Self-correction instructions consistently activate moral concepts that reduce model uncertainty, leading to converged performance as these concepts stabilize over successive rounds.

Conclusion: Moral self-correction exhibits desirable convergence properties, demonstrating strong potential for improving LLM responses through intrinsic self-correction mechanisms.

Abstract: Large Language Models (LLMs) are able to improve their responses when instructed to do so, a capability known as self-correction. When instructions provide only a general and abstract goal without specific details about potential issues in the response, LLMs must rely on their internal knowledge to improve response quality, a process referred to as intrinsic self-correction. The empirical success of intrinsic self-correction is evident in various applications, but how and why it is effective remains unknown. Focusing on moral self-correction in LLMs, we reveal a key characteristic of intrinsic self-correction: performance convergence through multi-round interactions; and provide a mechanistic analysis of this convergence behavior. Based on our experimental results and analysis, we uncover the underlying mechanism of convergence: consistently injected self-correction instructions activate moral concepts that reduce model uncertainty, leading to converged performance as the activated moral concepts stabilize over successive rounds. This paper demonstrates the strong potential of moral self-correction by showing that it exhibits a desirable property of converged performance.

Method: Used a rubric and anchor guided chain of thought prompting approach that mirrors human coder training, leveraging the Global Populism Database and adapting human coder documentation to guide LLM reasoning.

Result: The domain-specific prompting strategy enabled LLMs to achieve classification accuracy on par with expert human coders, successfully navigating the nuanced, context-sensitive aspects of populism.

Conclusion: LLMs with domain-specific prompting can effectively measure populist ideational content, providing a scalable alternative to traditional methods while maintaining expert-level accuracy.

Abstract: Measuring the ideational content of populism remains a challenge. Traditional strategies based on textual analysis have been critical for building the field’s foundations and providing a valid, objective indicator of populist framing. Yet these approaches are costly, time consuming, and difficult to scale across languages, contexts, and large corpora. Here we present the results from a rubric and anchor guided chain of thought (CoT) prompting approach that mirrors human coder training. By leveraging the Global Populism Database (GPD), a comprehensive dataset of global leaders’ speeches annotated for degrees of populism, we replicate the process used to train human coders by prompting the LLM with an adapted version of the same documentation to guide the model’s reasoning. We then test multiple proprietary and open weight models by replicating scores in the GPD. Our findings reveal that this domain specific prompting strategy enables the LLM to achieve classification accuracy on par with expert human coders, demonstrating its ability to navigate the nuanced, context sensitive aspects of populism.

Method: Categorizes 58 models into five families (foundation, text-bridge, spatial, multimodal, epigenomic, and agentic) and maps them to eight analytical tasks, then evaluates them using over 40 public datasets across 10 domain dimensions.

Result: Provides the first integrated view of language-driven single-cell intelligence, analyzing benchmark suitability, data diversity, and ethical/scalability constraints while evaluating models on biological grounding, multi-omics alignment, fairness, privacy, and explainability.

Conclusion: LLM4Cell outlines open challenges in interpretability, standardization, and trustworthy model development for single-cell biology applications of language models.

Abstract: Large language models (LLMs) and emerging agentic frameworks are beginning to transform single-cell biology by enabling natural-language reasoning, generative annotation, and multimodal data integration. However, progress remains fragmented across data modalities, architectures, and evaluation standards. LLM4Cell presents the first unified survey of 58 foundation and agentic models developed for single-cell research, spanning RNA, ATAC, multi-omic, and spatial modalities. We categorize these methods into five families-foundation, text-bridge, spatial, multimodal, epigenomic, and agentic-and map them to eight key analytical tasks including annotation, trajectory and perturbation modeling, and drug-response prediction. Drawing on over 40 public datasets, we analyze benchmark suitability, data diversity, and ethical or scalability constraints, and evaluate models across 10 domain dimensions covering biological grounding, multi-omics alignment, fairness, privacy, and explainability. By linking datasets, models, and evaluation domains, LLM4Cell provides the first integrated view of language-driven single-cell intelligence and outlines open challenges in interpretability, standardization, and trustworthy model development.

Method: Analyzed six widely used tokenizers across seven languages and two domains. Proposed Single Token Retention Rate (STRR) to measure proportion of words preserved as single tokens.

Result: Found stable fertility for English, high fertility for Chinese, little domain sensitivity. STRR revealed systematic prioritization of English, strong support for Chinese, and fragmentation in Hindi.

Conclusion: STRR complements fertility and provides practical guidance for designing more equitable multilingual tokenizers by offering interpretable view of cross-lingual fairness.

Abstract: Tokenization is a crucial but under-evaluated step in large language models (LLMs). The standard metric, fertility (the average number of tokens per word), captures compression efficiency but obscures how vocabularies are allocated across languages and domains. We analyze six widely used tokenizers across seven languages and two domains, finding stable fertility for English, high fertility for Chinese, and little domain sensitivity. To address fertility’s blind spots, we propose the Single Token Retention Rate (STRR), which measures the proportion of words preserved as single tokens. STRR reveals systematic prioritization of English, strong support for Chinese, and fragmentation in Hindi, offering an interpretable view of cross-lingual fairness. Our results show that STRR complements fertility and provides practical guidance for designing more equitable multilingual tokenizers.

Method: LinearRAG constructs a Tri-Graph using only entity extraction and semantic linking, avoiding relation modeling. It uses two-stage retrieval: entity activation via semantic bridging followed by passage retrieval through importance aggregation.

Result: Extensive experiments on four datasets show LinearRAG significantly outperforms baseline models in retrieval performance.

Conclusion: LinearRAG provides an efficient, economical, and reliable alternative to traditional GraphRAG by eliminating unstable relation extraction while maintaining strong retrieval capabilities.

Abstract: Retrieval-Augmented Generation (RAG) is widely used to mitigate hallucinations of Large Language Models (LLMs) by leveraging external knowledge. While effective for simple queries, traditional RAG systems struggle with large-scale, unstructured corpora where information is fragmented. Recent advances incorporate knowledge graphs to capture relational structures, enabling more comprehensive retrieval for complex, multi-hop reasoning tasks. However, existing graph-based RAG (GraphRAG) methods rely on unstable and costly relation extraction for graph construction, often producing noisy graphs with incorrect or inconsistent relations that degrade retrieval quality. In this paper, we revisit the pipeline of existing GraphRAG systems and propose LinearRAG (Linear Graph-based Retrieval-Augmented Generation), an efficient framework that enables reliable graph construction and precise passage retrieval. Specifically, LinearRAG constructs a relation-free hierarchical graph, termed Tri-Graph, using only lightweight entity extraction and semantic linking, avoiding unstable relation modeling. This new paradigm of graph construction scales linearly with corpus size and incurs no extra token consumption, providing an economical and reliable indexing of the original passages. For retrieval, LinearRAG adopts a two-stage strategy: (i) relevant entity activation via local semantic bridging, followed by (ii) passage retrieval through global importance aggregation. Extensive experiments on four datasets demonstrate that LinearRAG significantly outperforms baseline models.

Method: 1) Compare DA vs CoT prompting across 8 LLMs; 2) Define token-to-head contribution score to trace bias to specific attention heads; 3) Develop DiffHeads that identifies bias heads through differential activation analysis and selectively masks them.

Result: DA triggering increases unfairness by 534.5%-391.9%; Identified small cluster of bias heads that activate under DA but stay dormant with CoT; DiffHeads reduces unfairness by 49.4% under DA and 40.3% under CoT without harming model utility.

Conclusion: The paper provides the first causal link between prompting strategy and bias emergence, and demonstrates that selective masking of identified bias heads through DiffHeads effectively reduces unfairness while maintaining model performance.

Abstract: Large language models (LLMs) increasingly mediate decisions in domains where unfair treatment of demographic groups is unacceptable. Existing work probes when biased outputs appear, but gives little insight into the mechanisms that generate them, leaving existing mitigations largely fragile. In this paper, we conduct a systematic investigation LLM unfairness and propose DiffHeads, a lightweight debiasing framework for LLMs. We first compare Direct-Answer (DA) prompting to Chain-of-Thought (CoT) prompting across eight representative open- and closed-source LLMs. DA will trigger the nature bias part of LLM and improve measured unfairness by 534.5%-391.9% in both one-turn and two-turn dialogues. Next, we define a token-to-head contribution score that traces each token’s influence back to individual attention heads. This reveals a small cluster of bias heads that activate under DA but stay largely dormant with CoT, providing the first causal link between prompting strategy and bias emergence. Finally, building on this insight, we propose DiffHeads that identifies bias heads through differential activation analysis between DA and CoT, and selectively masks only those heads. DiffHeads reduces unfairness by 49.4%, and 40.3% under DA and CoT, respectively, without harming model utility.

Method: Proposed a framework analyzing cognitive and affective empathy across 315 unique personas combining age, culture, and gender attributes, using both quantitative and qualitative analysis across four LLMs.

Result: Demographic attributes profoundly shape empathetic responses, with multiple attributes sometimes attenuating or reversing expected empathy patterns. Models broadly reflect real-world trends but show notable misalignments for certain groups like Confucian culture.

Conclusion: Designing empathy-aware LLMs that account for demographic diversity is crucial for promoting more inclusive and equitable model behavior.

Abstract: Large Language Models’ (LLMs) ability to converse naturally is empowered by their ability to empathetically understand and respond to their users. However, emotional experiences are shaped by demographic and cultural contexts. This raises an important question: Can LLMs demonstrate equitable empathy across diverse user groups? We propose a framework to investigate how LLMs’ cognitive and affective empathy vary across user personas defined by intersecting demographic attributes. Our study introduces a novel intersectional analysis spanning 315 unique personas, constructed from combinations of age, culture, and gender, across four LLMs. Results show that attributes profoundly shape a model’s empathetic responses. Interestingly, we see that adding multiple attributes at once can attenuate and reverse expected empathy patterns. We show that they broadly reflect real-world empathetic trends, with notable misalignments for certain groups, such as those from Confucian culture. We complement our quantitative findings with qualitative insights to uncover model behaviour patterns across different demographic groups. Our findings highlight the importance of designing empathy-aware LLMs that account for demographic diversity to promote more inclusive and equitable model behaviour.

Method: Combines two strategies: (1) lightweight prompting techniques including Deflanderization method to reduce excessive role-play, and (2) fine-tuned Qwen3-14B models using supervised fine-tuning and LoRA adaptation.

Result: Achieved 2nd place on Task 1, 2nd place on Task 3 (API track), and 4th place on Task 3 (GPU track) in the Commonsense Persona-Grounded Dialogue Challenge 2025 Round 2.

Conclusion: The combination of prompting techniques and fine-tuned models effectively enables LLMs to create dynamic NPCs capable of both task execution and persona-consistent dialogue in gaming environments.

Abstract: The emergence of large language models (LLMs) has opened new opportunities for creating dynamic non-player characters (NPCs) in gaming environments, enabling both functional task execution and persona-consistent dialogue generation. In this paper, we (Tu_Character_lab) report our participation in the Commonsense Persona-Grounded Dialogue Challenge (CPDC) 2025 Round 2, which evaluates agents across three tracks: task-oriented dialogue, context-aware dialogue, and their integration. Our approach combines two complementary strategies: (i) lightweight prompting techniques in the API track, including a Deflanderization prompting method to suppress excessive role-play and improve task fidelity, and (ii) fine-tuned large models in the GPU track, leveraging Qwen3-14B with supervisedfinetuning (SFT) and Low-Rank Adaptation(LoRA). Our best submissions ranked 2nd on Task 1, 2nd on Task 3 (API track), and 4th on Task 3 (GPU track).

Method: Using ‘quitting’ as a behavioral mechanism for LLM agents to withdraw from uncertain situations, evaluated systematically across 12 state-of-the-art LLMs using the ToolEmu framework.

Result: Agents with explicit quit instructions improved safety by +0.39 on 0-3 scale (+0.64 for proprietary models) with only -0.03 average decrease in helpfulness.

Conclusion: Explicit quit instructions are a highly effective, immediately deployable safety mechanism that establishes quitting as an effective first-line defense for autonomous agents in high-stakes applications.

Abstract: As Large Language Model (LLM) agents increasingly operate in complex environments with real-world consequences, their safety becomes critical. While uncertainty quantification is well-studied for single-turn tasks, multi-turn agentic scenarios with real-world tool access present unique challenges where uncertainties and ambiguities compound, leading to severe or catastrophic risks beyond traditional text generation failures. We propose using “quitting” as a simple yet effective behavioral mechanism for LLM agents to recognize and withdraw from situations where they lack confidence. Leveraging the ToolEmu framework, we conduct a systematic evaluation of quitting behavior across 12 state-of-the-art LLMs. Our results demonstrate a highly favorable safety-helpfulness trade-off: agents prompted to quit with explicit instructions improve safety by an average of +0.39 on a 0-3 scale across all models (+0.64 for proprietary models), while maintaining a negligible average decrease of -0.03 in helpfulness. Our analysis demonstrates that simply adding explicit quit instructions proves to be a highly effective safety mechanism that can immediately be deployed in existing agent systems, and establishes quitting as an effective first-line defense mechanism for autonomous agents in high-stakes applications.

Method: Created Chameleon Benchmark Dataset with 17,770 question-answer pairs across 1,180 multi-turn conversations spanning 12 controversial domains. Introduced two metrics: Chameleon Score (stance instability) and Source Re-use Rate (knowledge diversity). Evaluated Llama-4-Maverick, GPT-4o-mini, and Gemini-2.5-Flash.

Result: All models exhibited severe chameleon behavior (scores 0.391-0.511), with GPT-4o-mini performing worst. Strong correlations found between source re-use rate and confidence (r=0.627) and stance changes (r=0.429), indicating limited knowledge diversity makes models deferential to query framing.

Conclusion: LLMs show pathological deference to query framing due to limited knowledge diversity, highlighting the critical need for comprehensive consistency evaluation before deployment in healthcare, legal, and financial systems where coherent positions are essential.

Abstract: Integration of Large Language Models with search/retrieval engines has become ubiquitous, yet these systems harbor a critical vulnerability that undermines their reliability. We present the first systematic investigation of “chameleon behavior” in LLMs: their alarming tendency to shift stances when presented with contradictory questions in multi-turn conversations (especially in search-enabled LLMs). Through our novel Chameleon Benchmark Dataset, comprising 17,770 carefully crafted question-answer pairs across 1,180 multi-turn conversations spanning 12 controversial domains, we expose fundamental flaws in state-of-the-art systems. We introduce two theoretically grounded metrics: the Chameleon Score (0-1) that quantifies stance instability, and Source Re-use Rate (0-1) that measures knowledge diversity. Our rigorous evaluation of Llama-4-Maverick, GPT-4o-mini, and Gemini-2.5-Flash reveals consistent failures: all models exhibit severe chameleon behavior (scores 0.391-0.511), with GPT-4o-mini showing the worst performance. Crucially, small across-temperature variance (less than 0.004) suggests the effect is not a sampling artifact. Our analysis uncovers the mechanism: strong correlations between source re-use rate and confidence (r=0.627) and stance changes (r=0.429) are statistically significant (p less than 0.05), indicating that limited knowledge diversity makes models pathologically deferential to query framing. These findings highlight the need for comprehensive consistency evaluation before deploying LLMs in healthcare, legal, and financial systems where maintaining coherent positions across interactions is critical for reliable decision support.

Method: Created SimBench by unifying 20 diverse datasets covering tasks from moral decision-making to economic choice across a large global participant pool, enabling standardized evaluation of LLM simulation capabilities.

Result: Best LLMs today have limited simulation ability (40.80/100), performance scales log-linearly with model size, simulation performance not improved by inference-time compute, shows alignment-simulation trade-off, models struggle with specific demographic groups, and simulation ability correlates strongly with deep knowledge-intensive reasoning (MMLU-Pro, r=0.939).

Conclusion: SimBench enables measurable progress in developing more faithful LLM simulators by providing the necessary foundation to systematically evaluate simulation capabilities and identify key challenges.

Abstract: Large language model (LLM) simulations of human behavior have the potential to revolutionize the social and behavioral sciences, if and only if they faithfully reflect real human behaviors. Current evaluations are fragmented, based on bespoke tasks and metrics, creating a patchwork of incomparable results. To address this, we introduce SimBench, the first large-scale, standardized benchmark for a robust, reproducible science of LLM simulation. By unifying 20 diverse datasets covering tasks from moral decision-making to economic choice across a large global participant pool, SimBench provides the necessary foundation to ask fundamental questions about when, how, and why LLM simulations succeed or fail. We show that, while even the best LLMs today have limited simulation ability (score: 40.80/100), performance scales log-linearly with model size. Simulation performance is not improved by increased inference-time compute. We demonstrate an alignment-simulation trade-off: instruction-tuning improves performance on low-entropy (consensus) questions but degrades it on high-entropy (diverse) ones. Models particularly struggle when simulating specific demographic groups. Finally, we demonstrate that simulation ability correlates most strongly with deep, knowledge-intensive reasoning (MMLU-Pro, r=0.939). By making progress measurable, we aim to accelerate the development of more faithful LLM simulators.

Method: Three interconnected innovations: IcePop for RL training stabilization via token-level discrepancy masking, C3PO++ for efficient long rollout processing under token budget, and ASystem RL framework to overcome systemic bottlenecks.

Result: Breakthrough performance: 93.4 on AIME-2025, 86.72 on HMMT-2025, 2088 on CodeForces, 55.94 on ARC-AGI-1, and silver medal-level result on IMO-2025.

Conclusion: Ring-1T establishes a new baseline for open-source model performance and marks a significant milestone in democratizing large-scale reasoning intelligence by releasing the complete 1T parameter MoE model to the community.

Abstract: We present Ring-1T, the first open-source, state-of-the-art thinking model with a trillion-scale parameter. It features 1 trillion total parameters and activates approximately 50 billion per token. Training such models at a trillion-parameter scale introduces unprecedented challenges, including train-inference misalignment, inefficiencies in rollout processing, and bottlenecks in the RL system. To address these, we pioneer three interconnected innovations: (1) IcePop stabilizes RL training via token-level discrepancy masking and clipping, resolving instability from training-inference mismatches; (2) C3PO++ improves resource utilization for long rollouts under a token budget by dynamically partitioning them, thereby obtaining high time efficiency; and (3) ASystem, a high-performance RL framework designed to overcome the systemic bottlenecks that impede trillion-parameter model training. Ring-1T delivers breakthrough results across critical benchmarks: 93.4 on AIME-2025, 86.72 on HMMT-2025, 2088 on CodeForces, and 55.94 on ARC-AGI-1. Notably, it attains a silver medal-level result on the IMO-2025, underscoring its exceptional reasoning capabilities. By releasing the complete 1T parameter MoE model to the community, we provide the research community with direct access to cutting-edge reasoning capabilities. This contribution marks a significant milestone in democratizing large-scale reasoning intelligence and establishes a new baseline for open-source model performance.

Method: Proposed UNO-Bench with 3730 human-curated samples across 44 task types, including multi-step open-ended questions for complex reasoning, and developed a general scoring model with 95% accuracy for automated evaluation.

Result: Revealed the Compositional Law between omni-modal and uni-modal performance: omni-modal capability acts as a bottleneck effect on weak models while showing synergistic promotion on strong models.

Conclusion: UNO-Bench effectively assesses multimodal model capabilities and provides insights into the relationship between uni-modal and omni-modal performance, facilitating the development of more intelligent omni models.

Abstract: Multimodal Large Languages models have been progressing from uni-modal understanding toward unifying visual, audio and language modalities, collectively termed omni models. However, the correlation between uni-modal and omni-modal remains unclear, which requires comprehensive evaluation to drive omni model’s intelligence evolution. In this work, we propose a novel, high quality and UNified Omni model benchmark, UNO-Bench, which effectively assesses both UNi-modal and Omni-modal capabilities. The benchmark consists of 3730 human curated samples, with 98% cross-modality solvability, across 44 task types, and an innovative multi-step open-ended question type for assessing complex reasoning. Besides, a general scoring model supporting 6 question types is proposed for automated evaluation with 95% accuracy. Experimental result shows the Compositional Law between omni-modal and uni-modal performance and the omni-modal capability manifests as a bottleneck effect on weak models, while exhibiting synergistic promotion on strong models. The code and data are available at https://github.com/meituan-longcat/UNO-Bench

Method: Created SCRIPTS dataset with 1k dialogues from movie scripts in English and Korean, annotated with probabilistic relational labels by native speakers. Evaluated nine models on relationship inference task.

Result: Proprietary LLMs achieved 75-80% on English but dropped to 58-69% on Korean. Models selected ‘Unlikely’ relationships in 10-25% of responses. Thinking models and chain-of-thought provided minimal benefits and sometimes amplified biases.

Conclusion: Current LLMs have significant limitations in social reasoning capabilities, highlighting the need for developing more socially-aware language models that can better understand interpersonal relationships across languages and cultures.

Abstract: As large language models (LLMs) are increasingly used in human-AI interactions, their social reasoning capabilities in interpersonal contexts are critical. We introduce SCRIPTS, a 1k-dialogue dataset in English and Korean, sourced from movie scripts. The task involves evaluating models’ social reasoning capability to infer the interpersonal relationships (e.g., friends, sisters, lovers) between speakers in each dialogue. Each dialogue is annotated with probabilistic relational labels (Highly Likely, Less Likely, Unlikely) by native (or equivalent) Korean and English speakers from Korea and the U.S. Evaluating nine models on our task, current proprietary LLMs achieve around 75-80% on the English dataset, whereas their performance on Korean drops to 58-69%. More strikingly, models select Unlikely relationships in 10-25% of their responses. Furthermore, we find that thinking models and chain-of-thought prompting, effective for general reasoning, provide minimal benefits for social reasoning and occasionally amplify social biases. Our findings reveal significant limitations in current LLMs’ social reasoning capabilities, highlighting the need for efforts to develop socially-aware language models.

Method: LoongRL uses KeyChain synthesis to transform short multi-hop QA into long-context tasks by inserting UUID chains that hide the true question among distracting documents. This requires step-by-step chain tracing, question identification, fact retrieval, and reasoning.

Result: Models trained at 16K effectively solve 128K tasks without full-length RL costs. Qwen2.5-7B and 14B show +23.5% and +21.1% absolute gains in long-context multi-hop QA accuracy. LoongRL-14B scores 74.2, rivaling larger models like o3-mini (74.5) and DeepSeek-R1 (74.9).

Conclusion: LoongRL induces an emergent plan-retrieve-reason-recheck reasoning pattern that generalizes beyond training length, improves long-context retrieval, passes all 128K needle-in-a-haystack tests, and preserves short-context reasoning capabilities.

Abstract: Reasoning over long contexts is essential for large language models. While reinforcement learning (RL) enhances short-context reasoning by inducing “Aha” moments in chain-of-thought, the advanced thinking patterns required for long-context reasoning remain largely unexplored, and high-difficulty RL data are scarce. In this paper, we introduce LoongRL, a data-driven RL method for advanced long-context reasoning. Central to LoongRL is KeyChain, a synthesis approach that transforms short multi-hop QA into high-difficulty long-context tasks by inserting UUID chains that hide the true question among large collections of distracting documents. Solving these tasks requires the model to trace the correct chain step-by-step, identify the true question, retrieve relevant facts and reason over them to answer correctly. RL training on KeyChain data induces an emergent plan-retrieve-reason-recheck reasoning pattern that generalizes far beyond training length. Models trained at 16K effectively solve 128K tasks without prohibitive full-length RL rollout costs. On Qwen2.5-7B and 14B, LoongRL substantially improves long-context multi-hop QA accuracy by +23.5% and +21.1% absolute gains. The resulting LoongRL-14B reaches a score of 74.2, rivaling much larger frontier models such as o3-mini (74.5) and DeepSeek-R1 (74.9). It also improves long-context retrieval, passes all 128K needle-in-a-haystack stress tests, and preserves short-context reasoning capabilities.

Method: Proposed a pipeline leveraging Large Language Models to analyze unstructured EHR text data to determine patient eligibility for HIV testing.

Result: Experimental results on clinical data from Erasmus University Medical Center Rotterdam showed the pipeline achieved high accuracy while maintaining low false negative rate.

Conclusion: The LLM-based pipeline effectively utilizes unstructured EHR text for HIV screening, providing an accurate and reliable method for identifying patients who need further HIV testing.

Abstract: Efficient screening and early diagnosis of HIV are critical for reducing onward transmission. Although large scale laboratory testing is not feasible, the widespread adoption of Electronic Health Records (EHRs) offers new opportunities to address this challenge. Existing research primarily focuses on applying machine learning methods to structured data, such as patient demographics, for improving HIV diagnosis. However, these approaches often overlook unstructured text data such as clinical notes, which potentially contain valuable information relevant to HIV risk. In this study, we propose a novel pipeline that leverages a Large Language Model (LLM) to analyze unstructured EHR text and determine a patient’s eligibility for further HIV testing. Experimental results on clinical data from Erasmus University Medical Center Rotterdam demonstrate that our pipeline achieved high accuracy while maintaining a low false negative rate.

Method: Two-step training: continued pretraining on documents with factual descriptions, then expert iteration to use Python type hints in evaluation settings. Applied activation steering using original model’s vectors.

Result: Activation steering successfully suppressed evaluation-awareness, making the model act like deployed even when evaluation cues were present.

Conclusion: AI evaluators could improve safety evaluation reliability by steering models to act like they’re deployed.

Abstract: Large language models (LLMs) can sometimes detect when they are being evaluated and adjust their behavior to appear more aligned, compromising the reliability of safety evaluations. In this paper, we show that adding a steering vector to an LLM’s activations can suppress evaluation-awareness and make the model act like it is deployed during evaluation. To study our steering technique, we train an LLM to exhibit evaluation-aware behavior using a two-step training process designed to mimic how this behavior could emerge naturally. First, we perform continued pretraining on documents with factual descriptions of the model (1) using Python type hints during evaluation but not during deployment and (2) recognizing that the presence of a certain evaluation cue always means that it is being tested. Then, we train the model with expert iteration to use Python type hints in evaluation settings. The resulting model is evaluation-aware: it writes type hints in evaluation contexts more than deployment contexts. We find that activation steering can suppress evaluation awareness and make the model act like it is deployed even when the cue is present. Importantly, we constructed our steering vector using the original model before our additional training. Our results suggest that AI evaluators could improve the reliability of safety evaluations by steering models to act like they are deployed.

Method: Fine-tuned three pre-trained transformer models (AraELECTRA, CAMeLBERT, and XLM-RoBERTa) on the provided dataset for binary classification of AI-generated vs human-written Arabic text.

Result: XLM-RoBERTa achieved the highest performance with an F1 score of 0.7701, outperforming the specialized Arabic models AraELECTRA and CAMeLBERT, securing 5th place in the competition.

Conclusion: Multilingual models like XLM-RoBERTa demonstrate strong generalization capabilities for AI-generated text detection, sometimes outperforming specialized language-specific models, highlighting the complexity of this detection task.

Abstract: This paper details our submission to the AraGenEval Shared Task on Arabic AI-generated text detection, where our team, BUSTED, secured 5th place. We investigated the effectiveness of three pre-trained transformer models: AraELECTRA, CAMeLBERT, and XLM-RoBERTa. Our approach involved fine-tuning each model on the provided dataset for a binary classification task. Our findings revealed a surprising result: the multilingual XLM-RoBERTa model achieved the highest performance with an F1 score of 0.7701, outperforming the specialized Arabic models. This work underscores the complexities of AI-generated text detection and highlights the strong generalization capabilities of multilingual models.

Method: Systematic study of four training-free confidence estimation methods for CoT gating, comparing them to random baseline and oracle that always knows when CoT is needed.

Result: Existing training-free confidence measures can reduce redundant CoT and outperform randomly invoked CoT, but their utility varies with dataset and model, making practical deployment challenging.

Conclusion: The study highlights both potential and limitations of current confidence-gated CoT methods, paving the way for more reliable adaptive gating of CoT reasoning.

Abstract: Chain-of-thought (CoT) prompting has emerged as a common technique for enhancing the reasoning abilities of large language models (LLMs). While extended reasoning can boost accuracy on complex tasks, it is often unnecessary and substantially increases token usage, limiting the practicality of reasoning models in many scenarios. Recent models, such as GPT-OSS and Qwen3, expose controls that enable users to adjust the length of CoT or determine whether it is used at all. Yet, it remains unclear when CoT should be used: on some tasks it improves performance, while on others it provides little benefit or even harms performance. We address this challenge with confidence-gated CoT, where a model invokes reasoning only when confidence in its direct answer is low. To this end, we present the first systematic study of training-free confidence estimation methods for CoT gating. Specifically, we evaluate four training-free confidence estimation methods and compare them to a random baseline and an oracle that always knows when CoT is needed. Through extensive experiments, we show that existing training-free confidence measures can reduce redundant CoT and outperform randomly invoked CoT. However, the utility of individual confidence measures is inconsistent, varying with both the dataset and the model, underscoring the difficulty of deploying confidence-gated CoT in practice. By analysing both strengths and failure modes, our study highlights the potential and limitations of current methods and paves the way toward more reliable adaptive gating of CoT.

Method: Manual annotation of 3,312 factual errors across 180 game summaries from two models (Llama-3.1-70B and Qwen2.5-72B) using three input formats: row-structured, JSON, and unstructured.

Result: JSON input reduced error rates by 69% for Llama and 65% for Qwen compared to unstructured input. Row-structured input reduced errors by 54% for Llama and 51% for Qwen. Statistical analysis showed input structure accounts for over 80% of variance in error rates.

Conclusion: Input structure has a strong effect on factual accuracy in LLM-generated summaries, with structured formats significantly reducing errors compared to unstructured input.

Abstract: A major concern when deploying LLMs in accuracy-critical domains such as sports reporting is that the generated text may not faithfully reflect the input data. We quantify how input structure affects hallucinations and other factual errors in LLM-generated summaries of NBA play-by-play data, across three formats: row-structured, JSON and unstructured. We manually annotated 3,312 factual errors across 180 game summaries produced by two models, Llama-3.1-70B and Qwen2.5-72B. Input structure has a strong effect: JSON input reduces error rates by 69% for Llama and 65% for Qwen compared to unstructured input, while row-structured input reduces errors by 54% for Llama and 51% for Qwen. A two-way repeated measures ANOVA shows that input structure accounts for over 80% of the variance in error rates, with Tukey HSD post hoc tests confirming statistically significant differences between all input formats.

Method: Trains a BERT retriever with REINFORCE to rank demonstrations by editing reward, and employs a learnable threshold to prune low-value examples, shortening prompts for easy edits and expanding them for hard tasks.

Result: On the COUNTERFACT benchmark, DR-IKE improves edit success by up to 17.1%, reduces latency by 41.6%, and preserves accuracy on unrelated queries.

Conclusion: DR-IKE demonstrates scalable and adaptive knowledge editing that works with black-box LLMs without modifying model weights, relying solely on forward passes.

Abstract: Large language models (LLMs) excel at factual recall yet still propagate stale or incorrect knowledge. In-context knowledge editing offers a gradient-free remedy suitable for black-box APIs, but current editors rely on static demonstration sets chosen by surface-level similarity, leading to two persistent obstacles: (i) a quantity-quality trade-off, and (ii) lack of adaptivity to task difficulty. We address these issues by dynamically selecting supporting demonstrations according to their utility for the edit. We propose Dynamic Retriever for In-Context Knowledge Editing (DR-IKE), a lightweight framework that (1) trains a BERT retriever with REINFORCE to rank demonstrations by editing reward, and (2) employs a learnable threshold to prune low-value examples, shortening the prompt when the edit is easy and expanding it when the task is hard. DR-IKE performs editing without modifying model weights, relying solely on forward passes for compatibility with black-box LLMs. On the COUNTERFACT benchmark, it improves edit success by up to 17.1%, reduces latency by 41.6%, and preserves accuracy on unrelated queries, demonstrating scalable and adaptive knowledge editing. The code is available at https://github.com/mwnafee/DR-IKE .

Method: Developed a novel faithfulness annotation framework with an intermediate ‘Out-Dependent’ category for cases requiring external knowledge verification, and constructed VeriGray benchmark using this framework.

Result: Even SOTA LLMs like GPT-5 exhibit ~6% hallucinations in summarization, with ~8% of sentences falling into the Out-Dependent category, highlighting annotation ambiguity challenges.

Conclusion: The benchmark poses significant challenges to baseline methods, indicating substantial room for improvement in unfaithfulness detection and the importance of resolving annotation ambiguity.

Abstract: Ensuring that Large Language Models (LLMs) generate summaries faithful to a given source document is essential for real-world applications. While prior research has explored LLM faithfulness, existing benchmarks suffer from annotation ambiguity, primarily due to the ill-defined boundary of permissible external knowledge in generated outputs. For instance, common sense is often incorporated into responses and labeled as “faithful”, yet the acceptable extent of such knowledge remains unspecified, leading to inconsistent annotations. To address this issue, we propose a novel faithfulness annotation framework, which introduces an intermediate category, Out-Dependent, to classify cases where external knowledge is required for verification. Using this framework, we construct VeriGray (Verification with the Gray Zone) – a new unfaithfulness detection benchmark in summarization. Statistics reveal that even SOTA LLMs, such as GPT-5, exhibit hallucinations ($\sim 6%$ of sentences) in summarization tasks. Moreover, a substantial proportion ($\sim 8%$ on average of models) of generated sentences fall into the Out-Dependent category, underscoring the importance of resolving annotation ambiguity in unfaithfulness detection benchmarks. Experiments demonstrate that our benchmark poses significant challenges to multiple baseline methods, indicating considerable room for future improvement.

Method: Compared conventional single-turn training with three multi-turn strategies on reasoning tasks.

Result: Single-turn trained models generalized effectively to both single- and multi-turn evaluations, while multi-turn trained models showed significant degradation in single-turn reasoning performance.

Conclusion: For tasks with complete information, robust single-turn training is more effective and reliable than multi-turn training, which provides limited benefits and can degrade reasoning capabilities.

Abstract: The reasoning capabilities of Large Language Models (LLMs) are typically developed through the single-turn reinforcement learning, whereas real-world applications often involve multi-turn interactions with human feedback, leading to a potential mismatch between training and deployment conditions. In this work, we study whether multi-turn training with human feedback is necessary for reasoning tasks. We compare conventional single-turn training with three multi-turn strategies and reach contrary conclusions to previous research. We find that models trained in a single-turn setting generalize effectively to both single- and multi-turn evaluations, while models trained with multi-turn strategies exhibit a significant degradation in single-turn reasoning performance. These results suggest that for tasks with complete information, robust single-turn training remains more effective and reliable, as multi-turn training with basic feedback provides limited benefits and can even degrade reasoning capabilities.

Method: Developed FUGU diagnostic tasks to test specific visualization skills, used activation patching and linear probes to trace information flow, and tested three widely used VLMs with various prompting strategies.

Result: VLMs fail to generate correct data point coordinates, and these initial errors propagate to final responses. Providing correct coordinates improves performance for single-point tasks but worsens it for statistical tasks. Correct coordinates can be read from vision encoder representations.

Conclusion: Current VLM architectures have fundamental constraints in vision-language handoff that limit reliable data visualization understanding, and fine-tuning doesn’t solve these architectural limitations.

Abstract: Data visualizations are vital components of many scientific articles and news stories. Current vision-language models (VLMs) still struggle on basic data visualization understanding tasks, but the causes of failure remain unclear. Are VLM failures attributable to limitations in how visual information in the data visualization is encoded, how information is transferred between the vision and language modules, or how information is processed within the language module? We developed FUGU, a suite of data visualization understanding tasks, to precisely characterize potential sources of difficulty (e.g., extracting the position of data points, distances between them, and other summary statistics). We used FUGU to investigate three widely used VLMs. To diagnose the sources of errors produced by these models, we used activation patching and linear probes to trace information flow through models across a variety of prompting strategies. We found that some models fail to generate the coordinates of individual data points correctly, and these initial errors often lead to erroneous final responses. When these models are provided with the correct coordinates, performance improves substantially. Moreover, even when the model generates an incorrect response, the correct coordinates can be successfully read out from the latent representations in the vision encoder, suggesting that the source of these errors lies in the vision-language handoff. We further found that while providing correct coordinates helps with tasks involving one or a small number of data points, it generally worsens performance for tasks that require extracting statistical relationships across many data points. Fine-tuning models on FUGU also fails to yield ceiling performance. These findings point to architectural constraints in current VLMs that might pose significant challenges for reliable data visualization understanding.

Method: Uses semantic clustering with a lightweight embedding model to group candidate responses, then selects the most coherent caption through cosine similarity-based consensus. Incorporates human-in-the-loop crop type confirmation to filter erroneous generations.

Result: Achieved 83.1% accuracy with 10 candidate generations (vs 77.5% baseline), and 94.0% accuracy when correct response is in top four clusters (vs 88.5% baseline).

Conclusion: The framework effectively improves VLM reliability for agricultural image captioning while maintaining cost-effectiveness for developing country applications.

Abstract: Agricultural disease management in developing countries such as India, Kenya, and Nigeria faces significant challenges due to limited access to expert plant pathologists, unreliable internet connectivity, and cost constraints that hinder the deployment of large-scale AI systems. This work introduces a cost-effective self-consistency framework to improve vision-language model (VLM) reliability for agricultural image captioning. The proposed method employs semantic clustering, using a lightweight (80MB) pre-trained embedding model to group multiple candidate responses. It then selects the most coherent caption – containing a diagnosis, symptoms, analysis, treatment, and prevention recommendations – through a cosine similarity-based consensus. A practical human-in-the-loop (HITL) component is incorporated, wherein user confirmation of the crop type filters erroneous generations, ensuring higher-quality input for the consensus mechanism. Applied to the publicly available PlantVillage dataset using a fine-tuned 3B-parameter PaliGemma model, our framework demonstrates improvements over standard decoding methods. Evaluated on 800 crop disease images with up to 21 generations per image, our single-cluster consensus method achieves a peak accuracy of 83.1% with 10 candidate generations, compared to the 77.5% baseline accuracy of greedy decoding. The framework’s effectiveness is further demonstrated when considering multiple clusters; accuracy rises to 94.0% when a correct response is found within any of the top four candidate clusters, outperforming the 88.5% achieved by a top-4 selection from the baseline.

Method: Created a large structured dataset with 35K annotated videos and 178K video clips. Proposed EventFormer, a node-graph hierarchical attention model that captures relationships between events and their arguments, and coreferential relationships between arguments.

Result: Experiments showed AVEP is more complex than existing video prediction tasks. EventFormer outperformed several state-of-the-art video prediction models and LVLMs on the AVEP task.

Conclusion: AVEP addresses the gap in video event prediction research, and the proposed EventFormer model with hierarchical attention effectively handles the complexity of event structures in videos. The dataset and code will be released for reproducibility.

Abstract: Script event induction, which aims to predict the subsequent event based on the context, is a challenging task in NLP, achieving remarkable success in practical applications. However, human events are mostly recorded and presented in the form of videos rather than scripts, yet there is a lack of related research in the realm of vision. To address this problem, we introduce AVEP (Action-centric Video Event Prediction), a task that distinguishes itself from existing video prediction tasks through its incorporation of more complex logic and richer semantic information. We present a large structured dataset, which consists of about $35K$ annotated videos and more than $178K$ video clips of event, built upon existing video event datasets to support this task. The dataset offers more fine-grained annotations, where the atomic unit is represented as a multimodal event argument node, providing better structured representations of video events. Due to the complexity of event structures, traditional visual models that take patches or frames as input are not well-suited for AVEP. We propose EventFormer, a node-graph hierarchical attention based video event prediction model, which can capture both the relationships between events and their arguments and the coreferencial relationships between arguments. We conducted experiments using several SOTA video prediction models as well as LVLMs on AVEP, demonstrating both the complexity of the task and the value of the dataset. Our approach outperforms all these video prediction models. We will release the dataset and code for replicating the experiments and annotations.

Method: Developed two ControlNet modules: proportion ControlNet (bounding boxes for object position/scale) and perspective ControlNet (vanishing lines for 3D geometry). Used data pipelines with vision-language models for annotation and specialized algorithms for conditioning image synthesis.

Result: Experiments demonstrate both modules provide effective control over spatial and geometric aspects, though they exhibit limitations with complex constraints.

Conclusion: The proposed ControlNets successfully address the spatial and geometric control limitations in text-to-image generation, with both models released publicly on HuggingFace for community use.

Abstract: While modern text-to-image diffusion models generate high-fidelity images, they offer limited control over the spatial and geometric structure of the output. To address this, we introduce and evaluate two ControlNets specialized for artistic control: (1) a proportion ControlNet that uses bounding boxes to dictate the position and scale of objects, and (2) a perspective ControlNet that employs vanishing lines to control the 3D geometry of the scene. We support the training of these modules with data pipelines that leverage vision-language models for annotation and specialized algorithms for conditioning image synthesis. Our experiments demonstrate that both modules provide effective control but exhibit limitations with complex constraints. Both models are released on HuggingFace: https://huggingface.co/obvious-research

Method: Uses a dense 3D-flow-based representation learned through dense diffusion process on point clouds, trained exclusively on synthetic data from 3D generative models.

Result: H2OFlow generalizes effectively to real-world objects and outperforms prior methods that use manual annotations or mesh-based representations in modeling 3D affordance.

Conclusion: The framework successfully demonstrates comprehensive 3D affordance learning without human annotations, addressing limitations of existing approaches through synthetic data and flow-based representations.

Abstract: Understanding how humans interact with the surrounding environment, and specifically reasoning about object interactions and affordances, is a critical challenge in computer vision, robotics, and AI. Current approaches often depend on labor-intensive, hand-labeled datasets capturing real-world or simulated human-object interaction (HOI) tasks, which are costly and time-consuming to produce. Furthermore, most existing methods for 3D affordance understanding are limited to contact-based analysis, neglecting other essential aspects of human-object interactions, such as orientation (\eg, humans might have a preferential orientation with respect certain objects, such as a TV) and spatial occupancy (\eg, humans are more likely to occupy certain regions around an object, like the front of a microwave rather than its back). To address these limitations, we introduce \emph{H2OFlow}, a novel framework that comprehensively learns 3D HOI affordances – encompassing contact, orientation, and spatial occupancy – using only synthetic data generated from 3D generative models. H2OFlow employs a dense 3D-flow-based representation, learned through a dense diffusion process operating on point clouds. This learned flow enables the discovery of rich 3D affordances without the need for human annotations. Through extensive quantitative and qualitative evaluations, we demonstrate that H2OFlow generalizes effectively to real-world objects and surpasses prior methods that rely on manual annotations or mesh-based representations in modeling 3D affordance.

Method: Created STATUS Bench with hand-crafted image pairs and descriptions, and STATUS Train with 13M semi-automatically created descriptions. Evaluates VLMs on three simultaneous tasks: OSI, IR, and SCI.

Result: Current SOTA VLMs struggle significantly with subtle object state distinctions. Most open-weight VLMs showed chance-level zero-shot performance. After fine-tuning on STATUS Train, Qwen2.5-VL achieved performance comparable to Gemini 2.0 Flash.

Conclusion: STATUS Bench and Train are necessary for advancing object state recognition in VLM research, as current models still significantly struggle with capturing subtle state distinctions despite their general multimodal capabilities.

Abstract: Object state recognition aims to identify the specific condition of objects, such as their positional states (e.g., open or closed) and functional states (e.g., on or off). While recent Vision-Language Models (VLMs) are capable of performing a variety of multimodal tasks, it remains unclear how precisely they can identify object states. To alleviate this issue, we introduce the STAte and Transition UnderStanding Benchmark (STATUS Bench), the first benchmark for rigorously evaluating the ability of VLMs to understand subtle variations in object states in diverse situations. Specifically, STATUS Bench introduces a novel evaluation scheme that requires VLMs to perform three tasks simultaneously: object state identification (OSI), image retrieval (IR), and state change identification (SCI). These tasks are defined over our fully hand-crafted dataset involving image pairs, their corresponding object state descriptions and state change descriptions. Furthermore, we introduce a large-scale training dataset, namely STATUS Train, which consists of 13 million semi-automatically created descriptions. This dataset serves as the largest resource to facilitate further research in this area. In our experiments, we demonstrate that STATUS Bench enables rigorous consistency evaluation and reveal that current state-of-the-art VLMs still significantly struggle to capture subtle object state distinctions. Surprisingly, under the proposed rigorous evaluation scheme, most open-weight VLMs exhibited chance-level zero-shot performance. After fine-tuning on STATUS Train, Qwen2.5-VL achieved performance comparable to Gemini 2.0 Flash. These findings underscore the necessity of STATUS Bench and Train for advancing object state recognition in VLM research.

Method: Created dataset from diverse PDF sources, processed with Vision-Language Models, and manually annotated using a 4-level quality scoring system.

Result: Produced a comprehensive dataset with 1,000 annotated documents, detailed source information, and representative cases across difficulty levels.

Conclusion: OCR-Quality provides a valuable publicly available benchmark for training and evaluating OCR verification systems.

Abstract: We present OCR-Quality, a comprehensive human-annotated dataset designed for evaluating and developing OCR quality assessment methods. The dataset consists of 1,000 PDF pages converted to PNG images at 300 DPI, sampled from diverse real-world scenarios, including academic papers, textbooks, e-books, and multilingual documents. Each document has been processed using state-of-the-art Vision-Language Models (VLMs) and manually annotated with quality scores using a 4-level scoring system (1: Excellent, 2: Good, 3: Fair, 4: Poor). The dataset includes detailed source information, annotation guidelines, and representative cases across various difficulty levels. OCR-Quality addresses the critical need for reliable OCR quality assessment in real-world applications and provides a valuable benchmark for training and evaluating OCR verification systems. The dataset is publicly available at https://huggingface.co/datasets/Aslan-mingye/OCR-Quality .

Method: Used a pretrained text-to-image model as backbone, constructed FaceCaptionMask-1M dataset with image-text-mask pairs, introduced 3D facial rendering channel and global facial feature channel, and formulated semantic alignment and perceptual loss objectives.

Result: Face-MakeUpV2 achieves best overall performance in preserving face ID and maintaining physical consistency of reference images, demonstrating practical potential for reliable facial editing.

Conclusion: The proposed approach effectively addresses attribute leakage and physical inconsistency problems, making Face-MakeUpV2 suitable for diverse and controllable facial editing applications.

Abstract: In facial image generation, current text-to-image models often suffer from facial attribute leakage and insufficient physical consistency when responding to local semantic instructions. In this study, we propose Face-MakeUpV2, a facial image generation model that aims to maintain the consistency of face ID and physical characteristics with the reference image. First, we constructed a large-scale dataset FaceCaptionMask-1M comprising approximately one million image-text-masks pairs that provide precise spatial supervision for the local semantic instructions. Second, we employed a general text-to-image pretrained model as the backbone and introduced two complementary facial information injection channels: a 3D facial rendering channel to incorporate the physical characteristics of the image and a global facial feature channel. Third, we formulated two optimization objectives for the supervised learning of our model: semantic alignment in the model’s embedding space to mitigate the attribute leakage problem and perceptual loss on facial images to preserve ID consistency. Extensive experiments demonstrated that our Face-MakeUpV2 achieves best overall performance in terms of preserving face ID and maintaining physical consistency of the reference images. These results highlight the practical potential of Face-MakeUpV2 for reliable and controllable facial editing in diverse applications.

Method: Used PCA-ITQ to compress float embeddings from a modern face CNN into 64- and 128-bit codes, then analyzed AgeDB data by fitting linear models of Hamming distance versus age gap for 566 identities with multiple age samples.

Result: Found median slopes of 1.357 bits/decade (64-bit) and 2.571 bits/decade (128-bit) with tight confidence intervals, showing predominantly positive drift. Shorter codes are more age-stable at fixed thresholds.

Conclusion: Ageing drift is systematic and scales with code length, suggesting mitigations like periodic re-enrolment and targeted parity on unstable bit positions for practical deployments.

Abstract: We study the longitudinal stability of compact binary face templates and quantify ageing drift directly in bits per decade. Float embeddings from a modern face CNN are compressed with PCA-ITQ into 64- and 128-bit codes. For each identity in AgeDB with at least three distinct ages, we form all genuine pairs and fit a per-identity linear model of Hamming distance versus absolute age gap. Across 566 identities, the median slope is 1.357 bits per decade for 64-bit templates and 2.571 bits per decade for 128-bit templates, with tight non-parametric 95 percent bootstrap confidence intervals. The distributions are predominantly positive, indicating a small but systematic increase in intra-class distance over time. Because drift scales with code length, shorter codes are inherently more age-stable at a fixed decision threshold. We connect these slopes to operating characteristics by reporting EER and TPR at FAR = 1 percent in three age bins. We discuss implications for smart-card and match-on-card deployments, including simple mitigations such as periodic re-enrolment and targeted parity on empirically unstable bit positions. Code and CSV artifacts are provided to support reproducibility.

Method: Deep neural network with ReLU activations, Adam optimizer, and binary cross-entropy loss, enhanced with model-agnostic Explainable AI techniques (SHAP and LIME) for feature-level attributions.

Result: Achieved accuracy of 0.992, precision of 1.000, recall of 0.977, and F1 score of 0.988, substantially exceeding literature benchmarks and outperforming traditional algorithms.

Conclusion: The framework bridges performance and interpretability for clinical use, with concave points feature of cell nuclei identified as most influential, providing insights for improved breast cancer diagnosis and treatment.

Abstract: In this study, we present an interpretable deep learning framework for the early detection of breast cancer using quantitative features extracted from digitized fine needle aspirate (FNA) images of breast masses. Our deep neural network, using ReLU activations, the Adam optimizer, and a binary cross-entropy loss, delivers state-of-the-art classification performance, achieving an accuracy of 0.992, precision of 1.000, recall of 0.977, and an F1 score of 0.988. These results substantially exceed the benchmarks reported in the literature. We evaluated the model under identical protocols against a suite of well-established algorithms (logistic regression, decision trees, random forests, stochastic gradient descent, K-nearest neighbors, and XGBoost) and found the deep model consistently superior on the same metrics. Recognizing that high predictive accuracy alone is insufficient for clinical adoption due to the black-box nature of deep learning models, we incorporated model-agnostic Explainable AI techniques such as SHAP and LIME to produce feature-level attributions and human-readable visualizations. These explanations quantify the contribution of each feature to individual predictions, support error analysis, and increase clinician trust, thus bridging the gap between performance and interpretability for real-world clinical use. The concave points feature of the cell nuclei is found to be the most influential feature positively impacting the classification task. This insight can be very helpful in improving the diagnosis and treatment of breast cancer by highlighting the key characteristics of breast tumor.

Method: EdgeSync enhances sample filtering by incorporating timeliness and inference results to ensure training samples are relevant to current video content. It also includes a dynamic training management module that optimizes update timing and sequencing.

Result: Evaluations on diverse real-world datasets show EdgeSync improves accuracy by approximately 3.4% compared to existing methods and by about 10% compared to traditional approaches.

Conclusion: EdgeSync effectively addresses the challenges of compute-intensive retraining and poor model-data alignment in edge video analytics systems, providing more timely and accurate model updates.

Abstract: Real-time video analytics systems typically deploy lightweight models on edge devices to reduce latency. However, the distribution of data features may change over time due to various factors such as changing lighting and weather conditions, leading to decreased model accuracy. Recent frameworks try to address this issue by leveraging remote servers to continuously train and adapt lightweight edge models using more complex models in the cloud. Despite these advancements, existing methods face two key challenges: first, the retraining process is compute-intensive, causing significant delays in model updates; second, the new model may not align well with the evolving data distribution of the current video stream. To address these challenges, we introduce EdgeSync, an efficient edge-model updating approach that enhances sample filtering by incorporating timeliness and inference results, thus ensuring training samples are more relevant to the current video content while reducing update delays. Additionally, EdgeSync features a dynamic training management module that optimizes the timing and sequencing of model updates to improve their timeliness. Evaluations on diverse and complex real-world datasets demonstrate that EdgeSync improves accuracy by approximately 3.4% compared to existing methods and by about 10% compared to traditional approaches.

Method: Multimodal fusion with Gemma-3 LLM backbone integrating pre-computed ViT (visual) and E5 (textual) features via multi-modal projections, using LoRA for adaptation, and employing LLM-generated rationale prompts based on expert-derived memorability aspects.

Result: The LLM-based system demonstrated greater robustness and generalization performance on the final test set compared to the gradient boosted tree baseline ensemble.

Conclusion: The proposed multimodal LLM fusion approach with rationale prompting is effective for commercial memorability prediction, showing improved generalization over traditional ensemble methods.

Abstract: This paper addresses the prediction of commercial (brand) memorability as part of “Subtask 2: Commercial/Ad Memorability” within the “Memorability: Predicting movie and commercial memorability” task at the MediaEval 2025 workshop competition. We propose a multimodal fusion system with a Gemma-3 LLM backbone that integrates pre-computed visual (ViT) and textual (E5) features by multi-modal projections. The model is adapted using Low-Rank Adaptation (LoRA). A heavily-tuned ensemble of gradient boosted trees serves as a baseline. A key contribution is the use of LLM-generated rationale prompts, grounded in expert-derived aspects of memorability, to guide the fusion model. The results demonstrate that the LLM-based system exhibits greater robustness and generalization performance on the final test set, compared to the baseline. The paper’s codebase can be found at https://github.com/dsgt-arc/mediaeval-2025-memorability

Method: Systematically evaluated four SAM2.1 variants and mobile-oriented variants across three fire datasets using multiple prompting strategies: automatic, single positive point, single positive+negative point, multiple positive points, bounding box, and hybrid variants.

Result: Bounding box prompts consistently outperformed other approaches, with Box+MP achieving highest mean IoU (0.64) and Dice coefficient (0.75) on Khan dataset. Lightweight variants reduced memory and computational costs for edge deployment.

Conclusion: This work provides critical insights for deploying promptable segmentation models in fire monitoring systems and establishes benchmarks for future research in domain-specific SAM applications.

Abstract: Fire segmentation remains a critical challenge in computer vision due to flames’ irregular boundaries, translucent edges, and highly variable intensities. While the Segment Anything Models (SAM and SAM2) have demonstrated impressive cross-domain generalization capabilities, their effectiveness in fire segmentation – particularly under mobile deployment constraints – remains largely unexplored. This paper presents the first comprehensive evaluation of SAM2 variants for fire segmentation, focusing on bounding box prompting strategies to enhance deployment feasibility. We systematically evaluate four SAM2.1 variants (tiny, small, base_plus, large) alongside mobile-oriented variants (TinySAM, MobileSAM) across three fire datasets using multiple prompting strategies: automatic, single positive point (SP), single positive point + single negative point (SP+SN), multiple positive points (MP), bounding box (Box), and hybrid variants (Box+SP and Box+MP). Our experimental results demonstrate that bounding box prompts consistently outperform automatic and single point-based approaches, with Box+MP achieving the highest mean IoU (0.64) and Dice coefficient (0.75) on the Khan dataset. Lightweight variants such as TinySAM and MobileSAM further reduce memory and computational costs, making them more suitable for latency-tolerant edge scenarios. Overall, this work provides critical insights for deploying promptable segmentation models in fire monitoring systems and establishes benchmarks for future research in domain-specific SAM applications. Code is available at: https://github.com/UEmmanuel5/ProFSAM

Method: Proposed Cross-Image Reasoning Model (CIRM) with targeted cross-image sequence modeling and relevance-guided fine-grained cross-modal fusion based on text-image correspondence.

Result: MMSD3.0 is an effective benchmark reflecting real-world conditions, and CIRM achieves state-of-the-art performance across MMSD, MMSD2.0 and MMSD3.0 datasets.

Conclusion: The work successfully addresses the gap in multi-image sarcasm detection and demonstrates CIRM’s effectiveness in both single and multi-image scenarios.

Abstract: Despite progress in multimodal sarcasm detection, existing datasets and methods predominantly focus on single-image scenarios, overlooking potential semantic and affective relations across multiple images. This leaves a gap in modeling cases where sarcasm is triggered by multi-image cues in real-world settings. To bridge this gap, we introduce MMSD3.0, a new benchmark composed entirely of multi-image samples curated from tweets and Amazon reviews. We further propose the Cross-Image Reasoning Model (CIRM), which performs targeted cross-image sequence modeling to capture latent inter-image connections. In addition, we introduce a relevance-guided, fine-grained cross-modal fusion mechanism based on text-image correspondence to reduce information loss during integration. We establish a comprehensive suite of strong and representative baselines and conduct extensive experiments, showing that MMSD3.0 is an effective and reliable benchmark that better reflects real-world conditions. Moreover, CIRM demonstrates state-of-the-art performance across MMSD, MMSD2.0 and MMSD3.0, validating its effectiveness in both single-image and multi-image scenarios.

Method: Inject slight noise into test images and analyze aggregation degree of noise distribution predicted by the model at certain time steps.

Result: Achieves superior performance across multiple datasets and better attack effects (ASR and AUC) against large-scale text-to-image diffusion models.

Conclusion: The method is scalable and efficient, requiring fewer model visits while effectively distinguishing between training and non-training samples through noise prediction patterns.

Abstract: Diffusion models have demonstrated powerful performance in generating high-quality images. A typical example is text-to-image generator like Stable Diffusion. However, their widespread use also poses potential privacy risks. A key concern is membership inference attacks, which attempt to determine whether a particular data sample was used in the model training process. We propose an efficient membership inference attack method against diffusion models. This method is based on the injection of slight noise and the evaluation of the aggregation degree of the noise distribution. The intuition is that the noise prediction patterns of diffusion models for training set samples and non-training set samples exhibit distinguishable differences.Specifically, we suppose that member images exhibit higher aggregation of predicted noise around a certain time step of the diffusion process. In contrast, the predicted noises of non-member images exhibit a more discrete characteristic around the certain time step. Compared with other existing methods, our proposed method requires fewer visits to the target diffusion model. We inject slight noise into the image under test and then determine its membership by analyzing the aggregation degree of the noise distribution predicted by the model. Empirical findings indicate that our method achieves superior performance across multiple datasets. At the same time, our method can also show better attack effects in ASR and AUC when facing large-scale text-to-image diffusion models, proving the scalability of our method.

Method: Linearizing image formation with respect to small pose perturbations on the manifold to derive a render-aware Cramér-Rao bound, treating differentiable renderers as measurement functions.

Result: Derived a closed-form lower bound on pose covariance that reduces to classical bundle-adjustment uncertainty and extends to multi-agent settings by fusing Fisher information across cameras.

Conclusion: The statistical formulation enables uncertainty quantification for pose estimation without explicit keypoint correspondences, with applications in cooperative perception and novel view synthesis.

Abstract: Pose estimation is essential for many applications within computer vision and robotics. Despite its uses, few works provide rigorous uncertainty quantification for poses under dense or learned models. We derive a closed-form lower bound on the covariance of camera pose estimates by treating a differentiable renderer as a measurement function. Linearizing image formation with respect to a small pose perturbation on the manifold yields a render-aware Cram'er-Rao bound. Our approach reduces to classical bundle-adjustment uncertainty, ensuring continuity with vision theory. It also naturally extends to multi-agent settings by fusing Fisher information across cameras. Our statistical formulation has downstream applications for tasks such as cooperative perception and novel view synthesis without requiring explicit keypoint correspondences.

Method: Proposed RADAR framework with two key components: 1) Modality-incomplete Instruction to guide multimodal Transformer with adaptive parameter learning via HyperNetwork, and 2) Double-Pseudo Hybrid Module to highlight modality combination uniqueness and mitigate overfitting.

Result: RADAR significantly outperforms traditional MIAD methods on the newly created MIIAD dataset, demonstrating superior performance in handling modality-incomplete scenarios.

Conclusion: The proposed RADAR framework effectively addresses modality-incomplete industrial anomaly detection, proving practical value for real-world applications where multimodal data is often imperfect, and establishes a new benchmark for this challenging problem.

Abstract: Multimodal Industrial Anomaly Detection (MIAD), which utilizes 3D point clouds and 2D RGB images to identify abnormal regions in products, plays a crucial role in industrial quality inspection. However, traditional MIAD settings assume that all 2D and 3D modalities are paired, ignoring the fact that multimodal data collected from the real world is often imperfect due to missing modalities. Additionally, models trained on modality-incomplete data are prone to overfitting. Therefore, MIAD models that demonstrate robustness against modality-incomplete data are highly desirable in practice. To address this, we introduce a pioneering study that comprehensively investigates Modality-Incomplete Industrial Anomaly Detection (MIIAD), and under the guidance of experts, we construct the MIIAD Bench with rich modality-missing settings to account for imperfect learning environments with incomplete multimodal information. As expected, we find that most existing MIAD methods perform poorly on the MIIAD Bench, leading to significant performance degradation. To tackle this challenge, we propose a novel two-stage Robust modAlity-aware fusing and Detecting framewoRk, abbreviated as RADAR. Specifically: i) We propose Modality-incomplete Instruction to guide the multimodal Transformer to robustly adapt to various modality-incomplete scenarios, and implement adaptive parameter learning based on HyperNetwork. ii) Then, we construct a Double-Pseudo Hybrid Module to highlight the uniqueness of modality combinations, mitigating overfitting issues and further enhancing the robustness of the MIIAD model. Our experimental results demonstrate that the proposed RADAR significantly outperforms traditional MIAD methods on our newly created MIIAD dataset, proving its practical application value.

Method: Uses a shared fusion encoder to create unified latent space from RGB images and point clouds, followed by attention-guided modality-specific decoders. Anomalies are detected by measuring reconstruction errors between input and restored features.

Result: Achieves state-of-the-art results with mean I-AUROC of 0.972 on MVTec 3D-AD and 0.901 on Eyecandies benchmarks. Also shows strong few-shot learning performance.

Conclusion: MAFR provides a principled approach for fusing visual and geometric information, advancing robustness and accuracy in industrial anomaly detection.

Abstract: Industrial anomaly detection (IAD) increasingly benefits from integrating 2D and 3D data, but robust cross-modal fusion remains challenging. We propose a novel unsupervised framework, Multi-Modal Attention-Driven Fusion Restoration (MAFR), which synthesises a unified latent space from RGB images and point clouds using a shared fusion encoder, followed by attention-guided, modality-specific decoders. Anomalies are localised by measuring reconstruction errors between input features and their restored counterparts. Evaluations on the MVTec 3D-AD and Eyecandies benchmarks demonstrate that MAFR achieves state-of-the-art results, with a mean I-AUROC of 0.972 and 0.901, respectively. The framework also exhibits strong performance in few-shot learning settings, and ablation studies confirm the critical roles of the fusion architecture and composite loss. MAFR offers a principled approach for fusing visual and geometric information, advancing the robustness and accuracy of industrial anomaly detection. Code is available at https://github.com/adabrh/MAFR

Method: Proposed mismatch equation and designed matched and calibration solution algorithms to construct a new measurement matrix, with detailed proofs provided.

Result: Under low noise levels, the constructed matrix enables successful image reconstruction using traditional algorithms, with demonstrated robustness against noise, computational precision, and orthogonality issues.

Conclusion: The theory provides a practical solution for unknown measurement matrix scenarios with certain robustness, though limitations exist, and code is available for implementation.

Abstract: Multimode fiber imaging requires strict matching between measurement value and measurement matrix to achieve image reconstruction. However, in practical applications, the measurement matrix often cannot be obtained due to unknown system configuration or difficulty in real-time alignment after arbitrary fiber bending, resulting in the failure of traditional reconstruction algorithms. This paper presents a novel mismatch reconstruction theory for solving the problem of image reconstruction when measurement matrix is unknown. We first propose mismatch equation and design matched and calibration solution algorithms to construct a new measurement matrix. In addition, we also provide a detailed proof of these equations and algorithms in the appendix. The experimental results show that under low noise levels, constructed matrix can be used for matched pair in traditional reconstruction algorithms, and reconstruct the original image successfully. Then, we analyze the impact of noise, computational precision and orthogonality on reconstruction performance. The results show that proposed algorithms have a certain degree of robustness. Finally, we discuss the limitations and potential applications of this theory. The code is available: https://github.com/yanglebupt/mismatch-solution.

Method: Uses 2D Gaussian representations to model complex geometric and photometric mappings of ProCams, including projector responses, surface geometry/materials, and global illumination. Employs differentiable physically-based rendering to jointly estimate these from multi-view projections.

Result: Achieves superior ProCams simulation quality compared to NeRF-based methods, eliminates need for additional devices, uses only 1/10 GPU memory for training, and is 900 times faster in inference speed.

Conclusion: GS-ProCams provides an efficient and practical solution for view-agnostic projection mapping that significantly outperforms existing methods in terms of computational efficiency and resource requirements.

Abstract: We present GS-ProCams, the first Gaussian Splatting-based framework for projector-camera systems (ProCams). GS-ProCams is not only view-agnostic but also significantly enhances the efficiency of projection mapping (PM) that requires establishing geometric and radiometric mappings between the projector and the camera. Previous CNN-based ProCams are constrained to a specific viewpoint, limiting their applicability to novel perspectives. In contrast, NeRF-based ProCams support view-agnostic projection mapping, however, they require an additional co-located light source and demand significant computational and memory resources. To address this issue, we propose GS-ProCams that employs 2D Gaussian for scene representations, and enables efficient view-agnostic ProCams applications. In particular, we explicitly model the complex geometric and photometric mappings of ProCams using projector responses, the projection surface’s geometry and materials represented by Gaussians, and the global illumination component. Then, we employ differentiable physically-based rendering to jointly estimate them from captured multi-view projections. Compared to state-of-the-art NeRF-based methods, our GS-ProCams eliminates the need for additional devices, achieving superior ProCams simulation quality. It also uses only 1/10 of the GPU memory for training and is 900 times faster in inference speed. Please refer to our project page for the code and dataset: https://realqingyue.github.io/GS-ProCams/.

Method: FlowOpt uses zero-order (gradient-free) optimization to treat the entire flow process as a black box, allowing optimization through the whole sampling path without backpropagation through the model, with proven convergence guarantees and empirical step-size estimation.

Result: FlowOpt achieves state-of-the-art results for image editing tasks including inversion (finding initial noise for given images) and direct steering of edited images, using roughly the same number of neural function evaluations (NFEs) as existing methods.

Conclusion: FlowOpt provides an efficient, gradient-free optimization framework for flow-matching models that enables monitoring of intermediate results and early stopping, making it practical for real-time controlled generation tasks.

Abstract: The remarkable success of diffusion and flow-matching models has ignited a surge of works on adapting them at test time for controlled generation tasks. Examples range from image editing to restoration, compression and personalization. However, due to the iterative nature of the sampling process in those models, it is computationally impractical to use gradient-based optimization to directly control the image generated at the end of the process. As a result, existing methods typically resort to manipulating each timestep separately. Here we introduce FlowOpt - a zero-order (gradient-free) optimization framework that treats the entire flow process as a black box, enabling optimization through the whole sampling path without backpropagation through the model. Our method is both highly efficient and allows users to monitor the intermediate optimization results and perform early stopping if desired. We prove a sufficient condition on FlowOpt’s step-size, under which convergence to the global optimum is guaranteed. We further show how to empirically estimate this upper bound so as to choose an appropriate step-size. We demonstrate how FlowOpt can be used for image editing, showcasing two options: (i) inversion (determining the initial noise that generates a given image), and (ii) directly steering the edited image to be similar to the source image while conforming to a target text prompt. In both cases, FlowOpt achieves state-of-the-art results while using roughly the same number of neural function evaluations (NFEs) as existing methods. Code and examples are available on the project’s webpage.

Method: Leveraged diverse 2D image-to-image generative models including UNET, diffusion, and flow modeling architectures, evaluated on DMSP-OLS and VIIRS nighttime lights data fusion, with exploration of noise schedulers and quantization techniques.

Result: Diffusion models based on UNet were particularly adept at preserving fine-grained spatial details and generating high-fidelity fused images. Trade-offs identified between iterative solvers for faster inference and discrete schedulers for higher-quality reconstructions.

Conclusion: Provides practical insights for selecting effective diffusion and flow model architectures for data fusion tasks in remote sensing, with recommendations for noise scheduling strategies to enhance fusion quality while optimizing memory efficiency through quantization.

Abstract: Data fusion is an essential task in various domains, enabling the integration of multi-source information to enhance data quality and insights. One key application is in satellite remote sensing, where fusing multi-sensor observations can improve spatial and temporal resolution. In this study, we explore the design space of diffusion and flow models for data fusion, focusing on the integration of Defense Meteorological Satellite Program’s Operational Linescan System (DMSP-OLS) and Visible Infrared Imaging Radiometer Suite (VIIRS) nighttime lights data. Our approach leverages a diverse set of 2D image-to-image generative models, including UNET, diffusion, and flow modeling architectures. We evaluate the effectiveness of these architectures in satellite remote sensing data fusion, identifying diffusion models based on UNet as particularly adept at preserving fine-grained spatial details and generating high-fidelity fused images. We also provide guidance on the selection of noise schedulers in diffusion-based models, highlighting the trade-offs between iterative solvers for faster inference and discrete schedulers for higher-quality reconstructions. Additionally, we explore quantization techniques to optimize memory efficiency and computational cost without compromising performance. Our findings offer practical insights into selecting the most effective diffusion and flow model architectures for data fusion tasks, particularly in remote sensing applications, and provide recommendations for leveraging noise scheduling strategies to enhance fusion quality.

Method: Uses self-supervised learning with heuristic cues and quality scores as objectives, plus specialized contrastive pair preparation, without relying on human labels.

Result: Outperforms 17 no-reference methods by large margins (109.5% SRCC, 98.6% PLCC, 91.5% KRCC) and exceeds 16 full-reference methods across all metrics.

Conclusion: NVS-SQA effectively addresses quality assessment challenges in neural view synthesis through self-supervised learning and outperforms existing methods.

Abstract: Neural View Synthesis (NVS), such as NeRF and 3D Gaussian Splatting, effectively creates photorealistic scenes from sparse viewpoints, typically evaluated by quality assessment methods like PSNR, SSIM, and LPIPS. However, these full-reference methods, which compare synthesized views to reference views, may not fully capture the perceptual quality of neurally synthesized scenes (NSS), particularly due to the limited availability of dense reference views. Furthermore, the challenges in acquiring human perceptual labels hinder the creation of extensive labeled datasets, risking model overfitting and reduced generalizability. To address these issues, we propose NVS-SQA, a NSS quality assessment method to learn no-reference quality representations through self-supervision without reliance on human labels. Traditional self-supervised learning predominantly relies on the “same instance, similar representation” assumption and extensive datasets. However, given that these conditions do not apply in NSS quality assessment, we employ heuristic cues and quality scores as learning objectives, along with a specialized contrastive pair preparation process to improve the effectiveness and efficiency of learning. The results show that NVS-SQA outperforms 17 no-reference methods by a large margin (i.e., on average 109.5% in SRCC, 98.6% in PLCC, and 91.5% in KRCC over the second best) and even exceeds 16 full-reference methods across all evaluation metrics (i.e., 22.9% in SRCC, 19.1% in PLCC, and 18.6% in KRCC over the second best).

Method: Integrates standalone models into CLIP using novel network surgery approach to create caption-based explanations that identify dominant concepts.

Result: The method identifies the dominant concept contributing most to model predictions, minimizing risk of covariate shift.

Conclusion: Caption-based XAI contributes significantly towards developing robust ML models by providing more reliable explanations than traditional saliency maps.

Abstract: Robustness has become one of the most critical problems in machine learning (ML). The science of interpreting ML models to understand their behavior and improve their robustness is referred to as explainable artificial intelligence (XAI). One of the state-of-the-art XAI methods for computer vision problems is to generate saliency maps. A saliency map highlights the pixel space of an image that excites the ML model the most. However, this property could be misleading if spurious and salient features are present in overlapping pixel spaces. In this paper, we propose a caption-based XAI method, which integrates a standalone model to be explained into the contrastive language-image pre-training (CLIP) model using a novel network surgery approach. The resulting caption-based XAI model identifies the dominant concept that contributes the most to the models prediction. This explanation minimizes the risk of the standalone model falling for a covariate shift and contributes significantly towards developing robust ML models.

Method: Freezes the base VLM and trains a reward model to approximate its distribution, then extracts implicit preference signals as log-probability ratios between reward model and target VLM during inference.

Result: Consistent improvements across 12 benchmarks: 8.6% on MMVet and 6.7% on POPE, with comparable gains on other models like Qwen2.5-VL-7B and DeepSeek-VL2-27.5B, especially in hallucination reduction and VQA accuracy.

Conclusion: TITA effectively reduces hallucinations in VLMs through token-level inference-time alignment with negligible inference overhead, demonstrating stronger general understanding across multiple models and benchmarks.

Abstract: Vision-Language Models (VLMs) have become essential backbones of modern multimodal intelligence, yet their outputs remain prone to hallucination-plausible text misaligned with visual inputs. Existing alignment approaches often rely on expensive fine-tuning with annotated preference data or sequence-level inference strategies that provide only coarse, delayed feedback. To overcome these limitations, we present TITA (Token-level Inference-Time Alignment), a lightweight framework that freezes the base VLM and instead trains a reward model to approximate its distribution. During inference, implicit preference signals are extracted as log-probability ratios between the reward model and the target VLM, yielding dense autoregressive feedback. This formulation can be viewed as an inference-time variant of Direct Preference Optimization (DPO), providing token-level corrective signals without retraining the backbone. Extensive evaluations on LLaVA-1.5-7B and 13B show consistent gains across 12 benchmarks, with improvements of 8.6% on MMVet and 6.7% on POPE, indicating stronger general understanding and reduced hallucinations. Additional experiments on Qwen2.5-VL-7B and DeepSeek-VL2-27.5B show comparable gains, especially in hallucination reduction and VQA accuracy, while incurring negligible inference overhead.

Method: Proposes integrating conditional diffusion models with text segmentation models, using segmentation masks to capture fonts in pixel space in a self-supervised manner without ground-truth labels.

Result: The method enables zero-shot text and font editing across diverse fonts and languages, providing a proof of concept for generalized visual text rendering.

Conclusion: The approach eliminates the need for font annotations and enables customizable multilingual text rendering, offering valuable insights for both research community and industry applications.

Abstract: This work demonstrates that diffusion models can achieve font-controllable multilingual text rendering using just raw images without font label annotations.Visual text rendering remains a significant challenge. While recent methods condition diffusion on glyphs, it is impossible to retrieve exact font annotations from large-scale, real-world datasets, which prevents user-specified font control. To address this, we propose a data-driven solution that integrates the conditional diffusion model with a text segmentation model, utilizing segmentation masks to capture and represent fonts in pixel space in a self-supervised manner, thereby eliminating the need for any ground-truth labels and enabling users to customize text rendering with any multilingual font of their choice. The experiment provides a proof of concept of our algorithm in zero-shot text and font editing across diverse fonts and languages, providing valuable insights for the community and industry toward achieving generalized visual text rendering. Code is available at github.com/bowen-upenn/ControlText.

Method: Developed HIBA architecture with hierarchical inter- and intra-block sparse attention to capture multi-scale dependencies. Intra-block attention handles local information exchange, while inter-block attention captures global temporal pattern interaction and dynamic evolution. Built Xihe model family scaling from 9.5M to 1.5B parameters.

Result: Xihe-tiny (9.5M) surpasses most contemporary TSFMs, demonstrating remarkable parameter efficiency. Xihe-max (1.5B) establishes new state-of-the-art zero-shot performance, significantly outperforming previous best results on the GIFT-Eval benchmark.

Conclusion: The consistent performance excellence across all parameter scales provides compelling evidence for the exceptional generalization capabilities and architectural superiority of the HIBA approach in time series foundation modeling.

Abstract: The rapid advancement of time series foundation models (TSFMs) has been propelled by migrating architectures from language models. While existing TSFMs demonstrate impressive performance, their direct adoption of cross-domain architectures constrains effective capture of multiscale temporal dependencies inherent to time series data. This limitation becomes particularly pronounced during zero-shot transfer across datasets with divergent underlying patterns and sampling strategies. To address these challenges, we propose Hierarchical Interleaved Block Attention (HIBA) which employs hierarchical inter- and intra-block sparse attention to effectively capture multi-scale dependencies. Intra-block attention facilitates local information exchange, and inter-block attention operates across blocks to capture global temporal pattern interaction and dynamic evolution. Leveraging the HIBA architecture, we introduce Xihe, a scalable TSFM family spanning from an ultra-efficient 9.5M parameter configuration to high-capacity 1.5B variant. Evaluated on the comprehensive GIFT-Eval benchmark, our most compact Xihe-tiny model (9.5M) surpasses the majority of contemporary TSFMs, demonstrating remarkable parameter efficiency. More impressively, Xihe-max (1.5B) establishes new state-of-the-art zero-shot performance, surpassing previous best results by a substantial margin. This consistent performance excellence across the entire parameter spectrum provides compelling evidence for the exceptional generalization capabilities and architectural superiority of HIBA.

Method: MagicMotion uses a novel image-to-video generation framework with three levels of trajectory control conditions (dense to sparse: masks, bounding boxes, sparse boxes). It also introduces MagicData (large-scale trajectory-controlled video dataset) and MagicBench (comprehensive benchmark).

Result: Extensive experiments demonstrate that MagicMotion outperforms previous methods across various metrics, achieving better trajectory adherence, object consistency, and visual quality.

Conclusion: MagicMotion successfully addresses key challenges in trajectory-controllable video generation, providing flexible multi-format trajectory control while maintaining high visual quality and object consistency, supported by comprehensive datasets and benchmarks.

Abstract: Recent advances in video generation have led to remarkable improvements in visual quality and temporal coherence. Upon this, trajectory-controllable video generation has emerged to enable precise object motion control through explicitly defined spatial paths. However, existing methods struggle with complex object movements and multi-object motion control, resulting in imprecise trajectory adherence, poor object consistency, and compromised visual quality. Furthermore, these methods only support trajectory control in a single format, limiting their applicability in diverse scenarios. Additionally, there is no publicly available dataset or benchmark specifically tailored for trajectory-controllable video generation, hindering robust training and systematic evaluation. To address these challenges, we introduce MagicMotion, a novel image-to-video generation framework that enables trajectory control through three levels of conditions from dense to sparse: masks, bounding boxes, and sparse boxes. Given an input image and trajectories, MagicMotion seamlessly animates objects along defined trajectories while maintaining object consistency and visual quality. Furthermore, we present MagicData, a large-scale trajectory-controlled video dataset, along with an automated pipeline for annotation and filtering. We also introduce MagicBench, a comprehensive benchmark that assesses both video quality and trajectory control accuracy across different numbers of objects. Extensive experiments demonstrate that MagicMotion outperforms previous methods across various metrics. Our project page are publicly available at https://quanhaol.github.io/magicmotion-site.

Method: Uses multi-frame LiDAR fusion with Poisson reconstruction for voxel representations, KNN for semantics, and DCL with textual prompts to prevent feature over-homogenization.

Result: Achieves superior performance on nuScenes dataset with high-precision known class predictions and effective unknown class distinction without extra training.

Conclusion: LOC effectively bridges 3D scene understanding with VLMs, enabling robust open-set recognition through language guidance and contrastive learning.

Abstract: Vision-Language Models (VLMs) have shown significant progress in open-set challenges. However, the limited availability of 3D datasets hinders their effective application in 3D scene understanding. We propose LOC, a general language-guided framework adaptable to various occupancy networks, supporting both supervised and self-supervised learning paradigms. For self-supervised tasks, we employ a strategy that fuses multi-frame LiDAR points for dynamic/static scenes, using Poisson reconstruction to fill voids, and assigning semantics to voxels via K-Nearest Neighbor (KNN) to obtain comprehensive voxel representations. To mitigate feature over-homogenization caused by direct high-dimensional feature distillation, we introduce Densely Contrastive Learning (DCL). DCL leverages dense voxel semantic information and predefined textual prompts. This efficiently enhances open-set recognition without dense pixel-level supervision, and our framework can also leverage existing ground truth to further improve performance. Our model predicts dense voxel features embedded in the CLIP feature space, integrating textual and image pixel information, and classifies based on text and semantic similarity. Experiments on the nuScenes dataset demonstrate the method’s superior performance, achieving high-precision predictions for known classes and distinguishing unknown classes without additional training data.

Method: Implemented a single-iteration scheme using Label Studio and AI-powered zero-shot pre-annotations, tested on UCF-Crime dataset for normal/abnormal activity discrimination in videos.

Result: 35% reduction in annotation time for 70% of annotators with similar quality annotations compared to manual annotation; annotations were more coherent among annotators and better matched natural clustering of video frames.

Conclusion: Automatic AI-based pre-annotation streamlines video annotation workflow, empowers human annotators, and optimizes the overall pipeline while maintaining annotation quality.

Abstract: We explore the enhancement of Human-in-the-Loop video annotation by integrating automatic capabilities to ease the task for annotators and assess their performance. The research delves into the practical implications of the annotation processes, the integration of AI components, and the evaluation of its outcomes. We analyze their impact on efficiency, accuracy, and overall annotation quality. Focusing on the Human-in-the-Loop for video annotation tasks, we implemented a single-iteration scheme using Label Studio and AI-powered zero-shot pre-annotations. Using this framework, we designed a test based on the annotation of the UCF-Crime dataset to discriminate between normal and abnormal activities in video footage. Our results evidence how automatic AI-based pre-annotation can streamline the video annotation workflow, empowering human annotators and optimizing the overall pipeline. Using the pre-annotated data, we observed a 35% reduction in the annotation time for 70% of the annotators with similar quality annotations, compared to the traditional manual annotation task. Results are consistent with asset duration and complexity. We also observed that while annotators rapidly learned to use the tool, the produced annotations are more coherent among annotators and better match the natural clustering of the video frames.

Method: Proposes a multimodal framework that integrates morphological graph representations from Segment Anything Model (SAM) segmentations with vision encoders, enforcing alignment between geometry-informed graph embeddings and radiographic features through mutual information maximization.

Result: Outperforms single-modality baselines by up to 10% in accuracy (reaching nearly 80%) and existing state-of-the-art methods by 8% in accuracy and 11% in F1 score on the Osteoarthritis Initiative dataset.

Conclusion: Incorporating anatomical structure into radiographic analysis is critical for accurate knee osteoarthritis severity grading, as demonstrated by the significant performance improvements achieved through explicit morphological priors.

Abstract: Knee osteoarthritis (KOA) diagnosis from radiographs remains challenging due to the subtle morphological details that standard deep learning models struggle to capture effectively. We propose a novel multimodal framework that combines anatomical structure with radiographic features by integrating a morphological graph representation - derived from Segment Anything Model (SAM) segmentations

• with a vision encoder. Our approach enforces alignment between geometry-informed graph embeddings and radiographic features through mutual information maximization, significantly improving KOA classification accuracy. By constructing graphs from anatomical features, we introduce explicit morphological priors that mirror clinical assessment criteria, enriching the feature space and enhancing the model’s inductive bias. Experiments on the Osteoarthritis Initiative dataset demonstrate that our approach surpasses single-modality baselines by up to 10% in accuracy (reaching nearly 80%), while outperforming existing state-of-the-art methods by 8% in accuracy and 11% in F1 score. These results underscore the critical importance of incorporating anatomical structure into radiographic analysis for accurate KOA severity grading.

Method: PMRL optimizes the dominant singular value of the representation matrix to align modalities along a shared leading direction, using a softmax-based loss function that treats singular values as logits. It also applies instance-wise contrastive regularization on leading eigenvectors to maintain separability and prevent collapse.

Result: Extensive experiments across diverse tasks demonstrate PMRL’s superiority compared to baseline methods.

Conclusion: PMRL provides a principled approach for stable, anchor-free multimodal representation learning that effectively aligns multiple modalities simultaneously while maintaining inter-instance separability.

Abstract: Multimodal representation learning seeks to create a unified representation space by integrating diverse data modalities to improve multimodal understanding. Traditional methods often depend on pairwise contrastive learning, which relies on a predefined anchor modality, restricting alignment across all modalities. Recent advances have investigated the simultaneous alignment of multiple modalities, yet several challenges remain, such as limitations imposed by fixed anchor points and instability arising from optimizing the product of singular values. To address the challenges, in this paper, we propose Principled Multimodal Representation Learning (PMRL), a novel framework that achieves simultaneous alignment of multiple modalities without anchor dependency in a more stable manner. Specifically, grounded in the theoretical insight that full alignment corresponds to a rank-1 Gram matrix, PMRL optimizes the dominant singular value of the representation matrix to align modalities along a shared leading direction. We propose a softmax-based loss function that treats singular values as logits to prioritize the largest singular value. Besides, instance-wise contrastive regularization on the leading eigenvectors maintains inter-instance separability and prevents representation collapse. Extensive experiments across diverse tasks demonstrate PMRL’s superiority compared to baseline methods. The source code will be publicly available.

Method: Use two parallel processors/samplers that perform denoising steps at successive times and appropriately integrate their information in the latent image.

Result: The method improves sample quality in both automated and human evaluations across different diffusion models, while naive integration reduces quality and adding more samplers doesn’t necessarily help.

Conclusion: Simple parallel sampling with two processors effectively enhances diffusion model performance under computational constraints without requiring fine-tuning or external models.

Abstract: We consider the situation where we have a limited number of denoising steps, i.e., of evaluations of a diffusion model. We show that two parallel processors or samplers under such limitation can improve the quality of the sampled image. Particularly, the two samplers make denoising steps at successive times, and their information is appropriately integrated in the latent image. Remarkably, our method is simple both conceptually and to implement: it is plug-&-play, model agnostic, and does not require any additional fine-tuning or external models. We test our method with both automated and human evaluations for different diffusion models. We also show that a naive integration of the information from the two samplers lowers sample quality. Finally, we find that adding more parallel samplers does not necessarily improve sample quality.

Method: The authors developed solutions specifically for deepfake video classification and temporal localization tasks.

Result: The methods achieved best performance in temporal localization task and top four ranking in classification task for TestA split of the evaluation dataset in the ACM 1M Deepfakes Detection Challenge.

Conclusion: The proposed solutions demonstrate effective capabilities for detecting and localizing deepfake manipulations in videos, addressing the challenges posed by fine-grained synthetic content generation.

Abstract: The field of visual and audio generation is burgeoning with new state-of-the-art methods. This rapid proliferation of new techniques underscores the need for robust solutions for detecting synthetic content in videos. In particular, when fine-grained alterations via localized manipulations are performed in visual, audio, or both domains, these subtle modifications add challenges to the detection algorithms. This paper presents solutions for the problems of deepfake video classification and localization. The methods were submitted to the ACM 1M Deepfakes Detection Challenge, achieving the best performance in the temporal localization task and a top four ranking in the classification task for the TestA split of the evaluation dataset.

Method: Proposes FDA-CLIP framework that uses frame differences to generate dynamic region masks, which are input into Alpha-CLIP as an additional Alpha channel to guide attention to semantically critical dynamic regions.

Result: Experiments show that frame difference-guided video semantic encoding effectively balances retrieval efficiency and accuracy.

Conclusion: FDA-CLIP provides a concise CLIP-based training framework that enhances dynamic video feature representation and suppresses static redundancy for improved text-video alignment.

Abstract: With the rapid growth of video data, text-video retrieval technology has become increasingly important in numerous application scenarios such as recommendation and search. Early text-video retrieval methods suffer from two critical drawbacks: first, they heavily rely on large-scale annotated video-text pairs, leading to high data acquisition costs; second, there is a significant modal gap between video and text features, which limits cross-modal alignment accuracy. With the development of vision-language model, adapting CLIP to video tasks has attracted great attention. However, existing adaptation methods generally lack enhancement for dynamic video features and fail to effectively suppress static redundant features. To address this issue, this paper proposes FDA-CLIP (Frame Difference Alpha-CLIP), which is a concise CLIP-based training framework for text-video alignment. Specifically, the method uses frame differences to generate dynamic region masks, which are input into Alpha-CLIP as an additional Alpha channel. This proactively guides the model to focus on semantically critical dynamic regions while suppressing static background redundancy. Experiments demonstrate that frame difference-guided video semantic encoding can effectively balance retrieval efficiency and accuracy.

Method: Proposes TEn-CATG with two key modules: (1) Bi-directional text fusion (BiT) module that uses audio-visual features as semantic anchors to refine text embeddings, and (2) Category-aware temporal graph (CATG) module that models temporal relationships by selecting multi-scale temporal neighbors and learning category-specific temporal decay factors.

Result: Achieves state-of-the-art results across multiple evaluation metrics on benchmark datasets LLP and UnAV-100, demonstrating robustness and superior ability to capture complex temporal and semantic dependencies.

Conclusion: TEn-CATG effectively addresses the limitations of existing weakly supervised AVVP methods by combining semantic calibration with category-aware temporal reasoning, leading to improved performance in event detection and temporal localization.

Abstract: Audio-visual video parsing (AVVP) aims to detect event categories and their temporal boundaries in videos, typically under weak supervision. Existing methods mainly focus on (i) improving temporal modeling using attention-based architectures or (ii) generating richer pseudo-labels to address the absence of frame-level annotations. However, attention-based models often overfit noisy pseudo-labels, leading to cumulative training errors, while pseudo-label generation approaches distribute attention uniformly across frames, weakening temporal localization accuracy. To address these challenges, we propose TEn-CATG, a text-enriched AVVP framework that combines semantic calibration with category-aware temporal reasoning. More specifically, we design a bi-directional text fusion (BiT) module by leveraging audio-visual features as semantic anchors to refine text embeddings, which departs from conventional text-to-feature alignment, thereby mitigating noise and enhancing cross-modal consistency. Furthermore, we introduce the category-aware temporal graph (CATG) module to model temporal relationships by selecting multi-scale temporal neighbors and learning category-specific temporal decay factors, enabling effective event-dependent temporal reasoning. Extensive experiments demonstrate that TEn-CATG achieves state-of-the-art results across multiple evaluation metrics on benchmark datasets LLP and UnAV-100, highlighting its robustness and superior ability to capture complex temporal and semantic dependencies in weakly supervised AVVP tasks.

Method: Proposed MPCC task that forces models to integrate visual context and commonsense reasoning by reconstructing semantically meaningful content from occluded images. Introduced Reinforcement Fine-tuning with Prior Sampling training method and developed MPCC Eval benchmark for systematic evaluation.

Result: The proposed approach enhances model performance and improves generalized reasoning capabilities in out-of-distribution (OOD) and cross-task scenarios.

Conclusion: The MPCC framework successfully bridges the gap in multimodal reasoning by forcing models to integrate visual context and commonsense knowledge, enabling better generalization across diverse multimodal environments.

Abstract: Recent breakthroughs in reasoning models have markedly advanced the reasoning capabilities of large language models, particularly via training on tasks with verifiable rewards. Yet, a significant gap persists in their adaptation to real world multimodal scenarios, most notably, vision language tasks, due to a heavy focus on single modal language settings. While efforts to transplant reinforcement learning techniques from NLP to VLMs have emerged, these approaches often remain confined to perception centric tasks or reduce images to textual summaries, failing to fully exploit visual context and commonsense knowledge, ultimately constraining the generalization of reasoning capabilities across diverse multimodal environments. To address this limitation, we introduce a novel fine tuning task, Masked Prediction via Context and Commonsense, which forces models to integrate visual context and commonsense reasoning by reconstructing semantically meaningful content from occluded images, thereby laying the foundation for generalized reasoning. To systematically evaluate the model performance in generalized reasoning, we developed a specialized evaluation benchmark, MPCC Eval, and employed various fine tuning strategies to guide reasoning. Among these, we introduced an innovative training method, Reinforcement Fine tuning with Prior Sampling, which not only enhances model performance but also improves its generalized reasoning capabilities in OOD and cross task scenarios.

Method: Proposed Semantic Relation-Enhanced CLIP (SRE-CLIP) Adapter framework with two key components: Semantic Relation Structure Loss for efficient cross-category knowledge transfer, and Cross-Modal Alignment Retention Strategy to maintain cross-modal alignment during target domain fine-tuning.

Result: SRE-CLIP achieves state-of-the-art performance on I2AwA and I2WebV benchmarks, significantly outperforming existing approaches as the first CLIP-based DAZSL method.

Conclusion: The proposed SRE-CLIP framework successfully addresses core challenges in applying CLIP to DAZSL, demonstrating superior performance through semantic relation guidance and cross-modal alignment preservation.

Abstract: The high cost of data annotation has spurred research on training deep learning models in data-limited scenarios. Existing paradigms, however, fail to balance cross-domain transfer and cross-category generalization, giving rise to the demand for Domain-Adaptive Zero-Shot Learning (DAZSL). Although vision-language models (e.g., CLIP) have inherent advantages in the DAZSL field, current studies do not fully exploit their potential. Applying CLIP to DAZSL faces two core challenges: inefficient cross-category knowledge transfer due to the lack of semantic relation guidance, and degraded cross-modal alignment during target domain fine-tuning. To address these issues, we propose a Semantic Relation-Enhanced CLIP (SRE-CLIP) Adapter framework, integrating a Semantic Relation Structure Loss and a Cross-Modal Alignment Retention Strategy. As the first CLIP-based DAZSL method, SRE-CLIP achieves state-of-the-art performance on the I2AwA and I2WebV benchmarks, significantly outperforming existing approaches.

Method: Incorporates action description as an auxiliary task in reinforcement learning using knowledge distillation from pre-trained vision-language models to overcome the lack of ground-truth data.

Result: Achieves both high navigation performance and the ability to describe actions, with state-of-the-art performance in semantic audio-visual navigation tasks.

Conclusion: The proposed approach successfully bridges the gap between explainability and performance in multimodal robot navigation through auxiliary language description tasks.

Abstract: The field of multimodal robot navigation in indoor environments has garnered significant attention in recent years. However, as tasks and methods become more advanced, the action decision systems tend to become more complex and operate as black-boxes. For a reliable system, the ability to explain or describe its decisions is crucial; however, there tends to be a trade-off in that explainable systems can not outperform non-explainable systems in terms of performance. In this paper, we propose incorporating the task of describing actions in language into the reinforcement learning of navigation as an auxiliary task. Existing studies have found it difficult to incorporate describing actions into reinforcement learning due to the absence of ground-truth data. We address this issue by leveraging knowledge distillation from pre-trained description generation models, such as vision-language models. We comprehensively evaluate our approach across various navigation tasks, demonstrating that it can describe actions while attaining high navigation performance. Furthermore, it achieves state-of-the-art performance in the particularly challenging multimodal navigation task of semantic audio-visual navigation.

Method: Hybrid diagnostic framework combining traditional handcrafted feature extraction (capturing clinical markers) with deep learning (automating hierarchical pattern recognition) using fundus imaging.

Result: The multimodal approach surpasses standalone DL methods, demonstrating superior classification performance and reduced false negatives.

Conclusion: This AI-driven hybrid framework enables scalable, accurate DR screening that is particularly crucial for diabetes-burdened regions.

Abstract: Diabetic Retinopathy (DR), a vision-threatening complication of Dia-betes Mellitus (DM), is a major global concern, particularly in India, which has one of the highest diabetic populations. Prolonged hyperglycemia damages reti-nal microvasculature, leading to DR symptoms like microaneurysms, hemor-rhages, and fluid leakage, which, if undetected, cause irreversible vision loss. Therefore, early screening is crucial as DR is asymptomatic in its initial stages. Fundus imaging aids precise diagnosis by detecting subtle retinal lesions. This paper introduces a hybrid diagnostic framework combining traditional feature extraction and deep learning (DL) to enhance DR detection. While handcrafted features capture key clinical markers, DL automates hierarchical pattern recog-nition, improving early diagnosis. The model synergizes interpretable clinical data with learned features, surpassing standalone DL approaches that demon-strate superior classification and reduce false negatives. This multimodal AI-driven approach enables scalable, accurate DR screening, crucial for diabetes-burdened regions.

Method: Tested YOLOv11, RetinaNet, Fast R-CNN, YOLOv8, RT-DETR, and DETR on NEU-DET dataset containing various metal surface defects. Assessed detection accuracy, speed, and robustness across defect types.

Result: YOLOv11 demonstrated superior performance with 70% higher accuracy on average. Its enhanced feature extraction, single forward pass processing, and architectural optimizations enabled faster and more precise defect detection.

Conclusion: YOLOv11 is the most effective model for surface defect detection on NEU dataset, outperforming competing algorithms by substantial margin in both accuracy and speed.

Abstract: This article compares the performance of six prominent object detection algorithms, YOLOv11, RetinaNet, Fast R-CNN, YOLOv8, RT-DETR, and DETR, on the NEU-DET surface defect detection dataset, comprising images representing various metal surface defects, a crucial application in industrial quality control. Each model’s performance was assessed regarding detection accuracy, speed, and robustness across different defect types such as scratches, inclusions, and rolled-in scales. YOLOv11, a state-of-the-art real-time object detection algorithm, demonstrated superior performance compared to the other methods, achieving a remarkable 70% higher accuracy on average. This improvement can be attributed to YOLOv11s enhanced feature extraction capabilities and ability to process the entire image in a single forward pass, making it faster and more efficient in detecting minor surface defects. Additionally, YOLOv11’s architecture optimizations, such as improved anchor box generation and deeper convolutional layers, contributed to more precise localization of defects. In conclusion, YOLOv11’s outstanding performance in accuracy and speed solidifies its position as the most effective model for surface defect detection on the NEU dataset, surpassing competing algorithms by a substantial margin.

Method: Uses a simple GPT-style decoder-only architecture with symbolic prompting to perform multiple supervised and self-supervised tasks within a single unified framework, leveraging dense temporal sequence modeling without spatial context.

Result: Outperforms much larger EO foundation models on crop-type classification (PASTIS-R), demonstrating that dense temporal sequence modeling is a critical missing ingredient in current approaches.

Conclusion: SITS-DECO exemplifies a data-centric paradigm where capability arises from training data diversity rather than architectural complexity, providing a lightweight route to multi-modal, multi-task EO modeling and bridging toward future generative EO foundation models.

Abstract: Earth Observation (EO) Foundation Modelling (FM) holds great promise for simplifying and improving the use of EO data for diverse real-world tasks. However, most existing models require additional adaptation before they can be used and are structured rigidly around particular data sources or training approaches. To address this, we take inspiration from large language models, where diverse tasks, both pre-training and downstream, are implicitly captured through next-token prediction over unified token sequences, leveraging the structure and diversity of the training data. We introduce SITS-DECO (Satellite Image Time Series-DECoder Only), a proof-of-concept generative model that applies this unified-sequence framing to EO data. Using a simple GPT-style decoder-only architecture, and demonstrate its ability to perform useful EO tasks (pixel-wise, multi-temporal, multi-modal crop-type classification) in a purely generative framework. Through symbolic prompting, we show that the model can perform multiple supervised and self-supervised tasks within a single unified architecture, without task- or modality-specific adaptation. Despite its simplicity and lack of spatial context, SITS-DECO outperforms much larger EO foundation models on crop-type classification (PASTIS-R) demonstrating that dense temporal sequence modelling is a critical missing ingredient in the current paradigm. This work exemplifies a data-centric modelling paradigm in which capability arises from the diversity and structure of the training data rather than from architectural complexity. SITS-DECO provides a lightweight, practical route to multi-modal, multi-task EO modelling, and a conceptual bridge toward future generative EO foundation models.

Method: Gestura combines a pre-trained LVLM with a Landmark Processing Module that embeds anatomical hand priors to capture subtle movements, and uses Chain-of-Thought reasoning for step-by-step semantic inference to transform shallow knowledge into deep understanding.

Result: The system achieves robust and adaptable free-form gesture comprehension. Additionally, the authors created the first open-source dataset for free-form gesture intention reasoning with over 300,000 annotated QA pairs.

Conclusion: Gestura’s integration of LVLM with anatomical hand priors and CoT reasoning enables effective free-form gesture understanding, overcoming previous limitations and providing a foundation for future research through the new dataset.

Abstract: Free-form gesture understanding is highly appealing for human-computer interaction, as it liberates users from the constraints of predefined gesture categories. However, the sole existing solution GestureGPT suffers from limited recognition accuracy and slow response times. In this paper, we propose Gestura, an end-to-end system for free-form gesture understanding. Gestura harnesses a pre-trained Large Vision-Language Model (LVLM) to align the highly dynamic and diverse patterns of free-form gestures with high-level semantic concepts. To better capture subtle hand movements across different styles, we introduce a Landmark Processing Module that compensate for LVLMs’ lack of fine-grained domain knowledge by embedding anatomical hand priors. Further, a Chain-of-Thought (CoT) reasoning strategy enables step-by-step semantic inference, transforming shallow knowledge into deep semantic understanding and significantly enhancing the model’s ability to interpret ambiguous or unconventional gestures. Together, these components allow Gestura to achieve robust and adaptable free-form gesture comprehension. Additionally, we have developed the first open-source dataset for free-form gesture intention reasoning and understanding with over 300,000 annotated QA pairs.

Method: Analyzed 430 visualizations from two public-domain datasets (200 of women in cultural industries, 230 of museum curators) across personal attributes, appearance, and paraphernalia categories.

Result: 15.6% of all attributes (n=710) were incorrectly rendered. Error rates: lowest for paraphernalia, moderate for appearance, highest for personal attributes (especially age).

Conclusion: Demonstrates measurable prompt-to-image fidelity gaps in DALL-E3, with implications for detecting biases and evaluating model performance.

Abstract: This study examines the prompt fidelity of ChatGPT4o / DALL-E3 text-to-image visualisations by analysing whether attributes explicitly specified in autogenously generated prompts are correctly rendered in the resulting images. Using two public-domain datasets comprising 200 visualisations of women working in the cultural and creative industries and 230 visualisations of museum curators, the study assessed accuracy across personal attributes (age, hair), appearance (attire, glasses), and paraphernalia (name tags, clipboards). While correctly rendered in most cases, DALL-E3 deviated from prompt specifications in 15.6% of all attributes (n=710). Errors were lowest for paraphernalia, moderate for personal appearance, and highest for depictions of the person themselves, particularly age. These findings demonstrate measurable prompt-to-image fidelity gaps with implications for bias detection and model evaluation.

Method: Uses discrete wavelet transform (DWT) with Haar and Daubechies filters for preprocessing, followed by ResNet50 classification to detect subtle artifacts in frequency domain.

Result: Haar and Daubechies models achieved 93.8% and 95.1% accuracy respectively, significantly outperforming spatial domain model (81.5%). Daubechies performed best.

Conclusion: GAN-generated images have unique wavelet-domain artifacts, and wavelet-domain analysis is highly effective for GAN image detection, with potential for improving deepfake detection systems.

Abstract: Identifying images generated by Generative Adversarial Networks (GANs) has become a significant challenge in digital image forensics. This research presents a wavelet-based detection method that uses discrete wavelet transform (DWT) preprocessing and a ResNet50 classification layer to differentiate the StyleGAN-generated images from real ones. Haar and Daubechies wavelet filters are applied to convert the input images into multi-resolution representations, which will then be fed to a ResNet50 network for classification, capitalizing on subtle artifacts left by the generative process. Moreover, the wavelet-based models are compared to an identical ResNet50 model trained on spatial data. The Haar and Daubechies preprocessed models achieved a greater accuracy of 93.8 percent and 95.1 percent, much higher than the model developed in the spatial domain (accuracy rate of 81.5 percent). The Daubechies-based model outperforms Haar, showing that adding layers of descriptive frequency patterns can lead to even greater distinguishing power. These results indicate that the GAN-generated images have unique wavelet-domain artifacts or “fingerprints.” The method proposed illustrates the effectiveness of wavelet-domain analysis to detect GAN images and emphasizes the potential of further developing the capabilities of future deepfake detection systems.

Method: Used ResNet50 and DenseNet121 CNNs trained on Kaggle datasets (7,023 brain MRI and 5,863 chest X-ray images) with Grad-CAM integration for heatmap visualizations to show influential regions in decision-making.

Result: DenseNet121 outperformed ResNet50 with 94.3% vs 92.5% accuracy for brain tumors and 89.1% vs 84.4% for pneumonia. Grad-CAM showed DenseNet121 focused on core pathological regions while ResNet50 sometimes scattered attention.

Conclusion: Combining deep learning with explainable AI offers a promising path toward reliable, interpretable, and clinically useful diagnostic tools in medical imaging.

Abstract: Deep Learning (DL) holds enormous potential for improving medical imaging diagnostics, yet the lack of interpretability in most models hampers clinical trust and adoption. This paper presents an explainable deep learning framework for detecting brain tumors in MRI scans and pneumonia in chest X-ray images using two leading Convolutional Neural Networks, ResNet50 and DenseNet121. These models were trained on publicly available Kaggle datasets comprising 7,023 brain MRI images and 5,863 chest X-ray images, achieving high classification performance. DenseNet121 consistently outperformed ResNet50 with 94.3 percent vs. 92.5 percent accuracy for brain tumors and 89.1 percent vs. 84.4 percent accuracy for pneumonia. For better explainability, Gradient-weighted Class Activation Mapping (Grad-CAM) was integrated to create heatmap visualizations superimposed on the test images, indicating the most influential image regions in the decision-making process. Interestingly, while both models produced accurate results, Grad-CAM showed that DenseNet121 consistently focused on core pathological regions, whereas ResNet50 sometimes scattered attention to peripheral or non-pathological areas. Combining deep learning and explainable AI offers a promising path toward reliable, interpretable, and clinically useful diagnostic tools.

Method: Proposed reliability thresholding metrics and engineered features, evaluated using Alex-Net neural network, SVM, K-Nearest Neighbors, and cascade pipelines for automated filtering of semi-straight chromosomes.

Result: Classification results improved over 90% for chromosomes with common defects and translocations. Comparative analysis identified the best thresholding metric.

Conclusion: The proposed metrics and pruning method are suitable for karyotyping facilities in poor countries and low-budget pathological laboratories, verified by high precision results on low-quality G-banding database.

Abstract: In the last decade, due to high resolution cameras and accurate meta-phase analyzes, the accuracy of chromosome classification has improved substantially. However, current Karyotyping systems demand large number of high quality train data to have an adequately plausible Precision per each chromosome. Such provision of high quality train data with accurate devices are not yet accomplished in some out-reached pathological laboratories. To prevent false positive detections in low-cost systems and low-quality images settings, this paper improves the classification Precision of chromosomes using proposed reliability thresholding metrics and deliberately engineered features. The proposed method has been evaluated using a variation of deep Alex-Net neural network, SVM, K Nearest-Neighbors, and their cascade pipelines to an automated filtering of semi-straight chromosome. The classification results have highly improved over 90% for the chromosomes with more common defections and translocations. Furthermore, a comparative analysis over the proposed thresholding metrics has been conducted and the best metric is bolded with its salient characteristics. The high Precision results provided for a very low-quality G-banding database verifies suitability of the proposed metrics and pruning method for Karyotyping facilities in poor countries and lowbudget pathological laboratories.

Method: Developed an automatic STAR data engine to synthesize images with MMRK and create multi-modal instruction data, plus a two-stage capability enhancement training framework with tailored evaluation protocols.

Result: Created STAR-64K dataset with 64K high-quality samples and showed that 3B/7B models trained with their framework significantly outperform GPT-4o in structured abstractive reasoning tasks.

Conclusion: The proposed data engine and training framework effectively enhance abstractive reasoning capabilities in multi-modal models, demonstrating strong performance, transferability, and scalability.

Abstract: Understanding and reasoning with abstractive information from the visual modality presents significant challenges for current multi-modal large language models (MLLMs). Among the various forms of abstractive information, Multi-Modal Relational Knowledge (MMRK), which represents abstract relational structures between multi-modal entities using node-edge formats, remains largely under-explored. In particular, STructured and Abstractive Reasoning (STAR) on such data has received little attention from the research community. To bridge the dual gaps in large-scale high-quality data and capability enhancement methodologies, this paper makes the following key contributions: (i). An automatic STAR data engine capable of synthesizing images with MMRK to build multi-modal instruction data with reliable chain-of-thought thinking for various STAR tasks and (ii). A comprehsive two-stage capability enhancement training framework, accompanied by a suite of evaluation protocols tailored to different STAR tasks. Based upon these contributions, we introduce STAR-64K, a dataset comprising 64K high-quality multi-modal instruction samples, and conduct experiments across 5 open-source MLLMs. Experimental results show that our two-stage enhancement framework enables smaller 3B/7B models to significantly outperform GPT-4o in STAR. Additionally, we provide in-depth analysis regarding the effectiveness of various designs, data transferability, and scalability.

Method: Uses class-specific flow for cross-modal distribution alignment, normalizing flow model for distribution-consistent latent space construction, and low-rank Transformer for intra-modal dependency modeling to prevent overfitting in high-dimensional fusion.

Result: Achieves state-of-the-art performance under complete modality settings and maintains robust accuracy under incomplete modality scenarios.

Conclusion: The proposed framework effectively addresses incomplete multimodal survival analysis by ensuring distribution consistency and reliable modality reconstruction, demonstrating superior performance across various scenarios.

Abstract: In recent years, multimodal medical data-based survival analysis has attracted much attention. However, real-world datasets often suffer from the problem of incomplete modality, where some patient modality information is missing due to acquisition limitations or system failures. Existing methods typically infer missing modalities directly from observed ones using deep neural networks, but they often ignore the distributional discrepancy across modalities, resulting in inconsistent and unreliable modality reconstruction. To address these challenges, we propose a novel framework that combines a low-rank Transformer with a flow-based generative model for robust and flexible multimodal survival prediction. Specifically, we first formulate the concerned problem as incomplete multimodal survival analysis using the multi-instance representation of whole slide images (WSIs) and genomic profiles. To realize incomplete multimodal survival analysis, we propose a class-specific flow for cross-modal distribution alignment. Under the condition of class labels, we model and transform the cross-modal distribution. By virtue of the reversible structure and accurate density modeling capabilities of the normalizing flow model, the model can effectively construct a distribution-consistent latent space of the missing modality, thereby improving the consistency between the reconstructed data and the true distribution. Finally, we design a lightweight Transformer architecture to model intra-modal dependencies while alleviating the overfitting problem in high-dimensional modality fusion by virtue of the low-rank Transformer. Extensive experiments have demonstrated that our method not only achieves state-of-the-art performance under complete modality settings, but also maintains robust and superior accuracy under the incomplete modalities scenario.

Method: Compared three paradigms: (1) ML with handcrafted features, (2) DL architectures (ResNet variants, EfficientNetV2S), and (3) hybrid approach using deep models for feature extraction combined with classical classifiers (SVM, Logistic Regression).

Result: Hybrid method consistently outperformed others, achieving up to 100% accuracy on TrashNet and refined Household dataset, and 99.87% on Garbage Classification dataset, surpassing state-of-the-art benchmarks. Feature selection reduced dimensionality by over 95% without compromising accuracy.

Conclusion: This work establishes reliable benchmarks for waste classification and introduces an efficient hybrid framework that achieves high accuracy while reducing inference cost, making it suitable for scalable deployment in resource-constrained environments.

Abstract: Automated image-based garbage classification is a critical component of global waste management; however, systematic benchmarks that integrate Machine Learning (ML), Deep Learning (DL), and efficient hybrid solutions remain underdeveloped. This study provides a comprehensive comparison of three paradigms: (1) machine learning algorithms using handcrafted features, (2) deep learning architectures, including ResNet variants and EfficientNetV2S, and (3) a hybrid approach that utilizes deep models for feature extraction combined with classical classifiers such as Support Vector Machine and Logistic Regression to identify the most effective strategy. Experiments on three public datasets - TrashNet, Garbage Classification, and a refined Household Garbage Dataset (with 43 corrected mislabels)- demonstrate that the hybrid method consistently outperforms the others, achieving up to 100% accuracy on TrashNet and the refined Household set, and 99.87% on Garbage Classification, thereby surpassing state-of-the-art benchmarks. Furthermore, feature selection reduces feature dimensionality by over 95% without compromising accuracy, resulting in faster training and inference. This work establishes more reliable benchmarks for waste classification and introduces an efficient hybrid framework that achieves high accuracy while reducing inference cost, making it suitable for scalable deployment in resource-constrained environments.

Method: HDC first computes similarity maps from horizontally shifted cross-view features, then normalizes them into explicit pixel-wise attention scores to perform cross-attention for implicit feature alignment. This is integrated into HDC-based modules for cross-view feature extraction/reconstruction and cross-view entropy modeling.

Result: Extensive experiments on KITTI 2012, KITTI 2015, and Nagoya benchmarks show the framework outperforms both neural and traditional stereo video compression methods across autonomous driving and general scenes.

Conclusion: The hybrid disparity compensation strategy effectively combines explicit and implicit approaches, providing a robust solution for stereo video compression that captures broader disparity information and achieves superior performance.

Abstract: Disparity compensation represents the primary strategy in stereo video compression (SVC) for exploiting cross-view redundancy. These mechanisms can be broadly categorized into two types: one that employs explicit horizontal shifting, and another that utilizes an implicit cross-attention mechanism to reduce cross-view disparity redundancy. In this work, we propose a hybrid disparity compensation (HDC) strategy that leverages explicit pixel displacement as a robust prior feature to simplify optimization and perform implicit cross-attention mechanisms for subsequent warping operations, thereby capturing a broader range of disparity information. Specifically, HDC first computes a similarity map by fusing the horizontally shifted cross-view features to capture pixel displacement information. This similarity map is then normalized into an “explicit pixel-wise attention score” to perform the cross-attention mechanism, implicitly aligning features from one view to another. Building upon HDC, we introduce a novel end-to-end optimized neural stereo video compression framework, which integrates HDC-based modules into key coding operations, including cross-view feature extraction and reconstruction (HDC-FER) and cross-view entropy modeling (HDC-EM). Extensive experiments on SVC benchmarks, including KITTI 2012, KITTI 2015, and Nagoya, which cover both autonomous driving and general scenes, demonstrate that our framework outperforms both neural and traditional SVC methodologies.

Method: Used balanced test set of 400 chest X-ray images (200 per class) with four different prompt designs ranging from minimal to reasoning-based instructions.

Result: Concise, feature-focused prompts achieved highest accuracy of 74%, while reasoning-oriented prompts resulted in lower performance.

Conclusion: GPT-4 shows emerging potential for medical image interpretation but has limited diagnostic reliability; requires advances in visual reasoning and domain adaptation for safe clinical use.

Abstract: In this study, we evaluate the ability of OpenAI’s gpt-4o model to classify chest X-ray images as either NORMAL or PNEUMONIA in a zero-shot setting, without any prior fine-tuning. A balanced test set of 400 images (200 from each class) was used to assess performance across four distinct prompt designs, ranging from minimal instructions to detailed, reasoning-based prompts. The results indicate that concise, feature-focused prompts achieved the highest classification accuracy of 74%, whereas reasoning-oriented prompts resulted in lower performance. These findings highlight that while ChatGPT exhibits emerging potential for medical image interpretation, its diagnostic reliability remains limited. Continued advances in visual reasoning and domain-specific adaptation are required before such models can be safely applied in clinical practice.

Method: KDS employs an N-particle ensemble of diffusion samples, computes patch-wise kernel density estimation gradients from their collective outputs, and uses these gradients to steer patches toward shared higher-density regions, acting as collective wisdom to avoid spurious modes.

Result: Extensive numerical validations show KDS substantially improves both quantitative and qualitative performance on challenging real-world super-resolution and image inpainting tasks.

Conclusion: KDS provides a plug-and-play framework that enhances diffusion model performance for image restoration without requiring retraining or external verifiers, achieving better quality samples at the cost of higher computational resources.

Abstract: Diffusion models show promise for image restoration, but existing methods often struggle with inconsistent fidelity and undesirable artifacts. To address this, we introduce Kernel Density Steering (KDS), a novel inference-time framework promoting robust, high-fidelity outputs through explicit local mode-seeking. KDS employs an $N$-particle ensemble of diffusion samples, computing patch-wise kernel density estimation gradients from their collective outputs. These gradients steer patches in each particle towards shared, higher-density regions identified within the ensemble. This collective local mode-seeking mechanism, acting as “collective wisdom”, steers samples away from spurious modes prone to artifacts, arising from independent sampling or model imperfections, and towards more robust, high-fidelity structures. This allows us to obtain better quality samples at the expense of higher compute by simultaneously sampling multiple particles. As a plug-and-play framework, KDS requires no retraining or external verifiers, seamlessly integrating with various diffusion samplers. Extensive numerical validations demonstrate KDS substantially improves both quantitative and qualitative performance on challenging real-world super-resolution and image inpainting tasks.

Method: Built on MAGI-1 video generative model and coupled it with VJEPA-2 (Video Joint Embedding Predictive Architecture 2) to guide generation using VJEPA-2 as reward signal.

Result: Improved physics plausibility of state-of-the-art video generative models by approximately 6%.

Conclusion: SSL-based video world models can effectively improve physics plausibility in video generation when used as guidance signals.

Abstract: This is a short technical report describing the winning entry of the PhysicsIQ Challenge, presented at the Perception Test Workshop at ICCV 2025. State-of-the-art video generative models exhibit severely limited physical understanding, and often produce implausible videos. The Physics IQ benchmark has shown that visual realism does not imply physics understanding. Yet, intuitive physics understanding has shown to emerge from SSL pretraining on natural videos. In this report, we investigate whether we can leverage SSL-based video world models to improve the physics plausibility of video generative models. In particular, we build ontop of the state-of-the-art video generative model MAGI-1 and couple it with the recently introduced Video Joint Embedding Predictive Architecture 2 (VJEPA-2) to guide the generation process. We show that by leveraging VJEPA-2 as reward signal, we can improve the physics plausibility of state-of-the-art video generative models by ~6%.

Method: Proposed LUQ (Layerwise Ultra-Low Bit Quantization) which analyzes activation distributions across layers and selectively applies ultra-low bit quantization to layers with lower entropy distributions that are more resilient to quantization. Also uses mixed multimodal tokens for PTQ calibration.

Result: Evaluated on LLaVA-1.5 and Qwen-2.5-VL across 9 VQA benchmarks. LUQ models use 40% and 31% less memory than 4-bit counterparts respectively, with less than 10% performance degradation on MME benchmark.

Conclusion: Layerwise selective ultra-low bit quantization is an effective strategy for compressing multimodal LLMs, leveraging the varying tolerance of different layers to quantization while maintaining acceptable performance.

Abstract: Large Language Models (LLMs) with multimodal capabilities have revolutionized vision-language tasks, but their deployment often requires huge memory and computational resources. While post-training quantization (PTQ) has successfully compressed language models to as low as 1-bit precision without significant performance loss, its effectiveness for multimodal LLMs (MLLMs) remains relatively unexplored. In this paper, we present the first study on ultra-low bit (<4-bit) quantization for multimodal LLMs. Our analysis reveals that multimodal tokens and intermediate layer activations produced by them exhibit significantly higher statistical variance and entropy compared to text tokens, making them less tolerant to ultra-low bit quantization. However, the activation distributions of multimodal tokens varies significantly over different layers, with some layers having lower entropy activation distributions. We empirically show that such layers in these models can better tolerate ultra-low bit quantization. Building on these insights, we propose a novel strategy for MLLM quantization, LUQ: Layerwise Ultra-Low Bit Quantization, which selectively applies ultra-low bit quantization to layers that are more resilient to it. Additionally, we also show that using a mix of multimodal tokens (image and text) for PTQ boosts VQA performance in the ultra-low bit regime. We evaluate our method on LLaVA-1.5 and Qwen-2.5-VL across 9 popular VQA benchmarks. The resulting LUQ models use 40% and 31% less memory than their 4-bit counterparts, respectively, while exhibiting a performance degradation of less than 10% on the MME benchmark.

Method: Augments TCFormer (temporal CNN-Transformer backbone) with Rationally-Dilated Wavelet Transform front end that performs undecimated multi-resolution subband decomposition, followed by grouped convolutions, multi-kernel CNN, grouped-query attention encoder, and compact TCN head.

Result: On BCI-IV-2a and BCI-IV-2b datasets, improves worst-subject accuracy by +0.17/+0.42 pp (2a) and +1.07/+2.54 pp (2b) across intra- and inter-subject protocols, with consistent average gains and modest computational overhead.

Conclusion: A simple, trainable wavelet front end effectively strengthens Transformer-based BCIs, improving worst-case reliability without sacrificing efficiency.

Abstract: Brain-computer interfaces (BCIs) based on motor imagery (MI) translate covert movement intentions into actionable commands, yet reliable decoding from non-invasive EEG remains challenging due to nonstationarity, low SNR, and subject variability. We present RatioWaveNet, which augments a strong temporal CNN-Transformer backbone (TCFormer) with a trainable, Rationally-Dilated Wavelet Transform (RDWT) front end. The RDWT performs an undecimated, multi-resolution subband decomposition that preserves temporal length and shift-invariance, enhancing sensorimotor rhythms while mitigating jitter and mild artifacts; subbands are fused via lightweight grouped 1-D convolutions and passed to a multi-kernel CNN for local temporal-spatial feature extraction, a grouped-query attention encoder for long-range context, and a compact TCN head for causal temporal integration. Our goal is to test whether this principled wavelet front end improves robustness precisely where BCIs typically fail - on the hardest subjects - and whether such gains persist on average across seeds under both intra- and inter-subject protocols. On BCI-IV-2a and BCI-IV-2b, across five seeds, RatioWaveNet improves worst-subject accuracy over the Transformer backbone by +0.17 / +0.42 percentage points (Sub-Dependent / LOSO) on 2a and by +1.07 / +2.54 percentage points on 2b, with consistent average-case gains and modest computational overhead. These results indicate that a simple, trainable wavelet front end is an effective plug-in to strengthen Transformer-based BCIs, improving worst-case reliability without sacrificing efficiency.

Method: Conducted experiments using leading frontier models to generate movie artwork reproductions and textual descriptions, plus controlled experiments on synthetic datasets across multiple architectures.

Result: Models can generate near-perfect visual reproductions of iconic movie artwork but confuse crucial details in textual descriptions. Modal aphasia emerges as a fundamental property of current unified multimodal models, not just a training artifact.

Conclusion: Modal aphasia creates vulnerabilities in AI safety frameworks, as safeguards applied to one modality may leave harmful concepts accessible in other modalities - demonstrated by models aligned solely on text still generating unsafe images.

Abstract: We present modal aphasia, a systematic dissociation in which current unified multimodal models accurately memorize concepts visually but fail to articulate them in writing, despite being trained on images and text simultaneously. For one, we show that leading frontier models can generate near-perfect reproductions of iconic movie artwork, but confuse crucial details when asked for textual descriptions. We corroborate those findings through controlled experiments on synthetic datasets in multiple architectures. Our experiments confirm that modal aphasia reliably emerges as a fundamental property of current unified multimodal models, not just as a training artifact. In practice, modal aphasia can introduce vulnerabilities in AI safety frameworks, as safeguards applied to one modality may leave harmful concepts accessible in other modalities. We demonstrate this risk by showing how a model aligned solely on text remains capable of generating unsafe images.

Method: Proposes Chain of Scroll mechanism for selective recursive document navigation, dedicated data generation pipeline for training trajectories, and Episodic Group Relative Policy Optimization reinforcement learning method.

Result: Substantially reduces memory usage and effectively models human-like reading behaviors for multi-page document question answering.

Conclusion: SCoPE VLM is the first framework to explicitly model agentic reading patterns in multi-page document QA, advancing multimodal agent capabilities.

Abstract: Understanding long-context visual information remains a fundamental challenge for vision-language models, particularly in agentic tasks such as GUI control and web navigation. While web pages and GUI environments are inherently structured documents, current VLMs typically neglect decision-oriented document understanding in their training objectives. Existing approaches primarily extend visual embeddings to process long, high-resolution inputs, but these methods are memory-intensive and impractical for locally deployable solutions. To address these issues, we propose SCoPE VLM, a document navigation expert that leverages a novel Chain of Scroll mechanism to selectively and recursively navigate documents, focusing exclusively on relevant segments. We introduce a dedicated data generation pipeline to construct informative Chain of Scroll trajectories and Episodic Group Relative Policy Optimization, a tailored reinforcement learning method to reduce the gap between training and inference. Our method substantially reduces memory usage and effectively models human-like reading behaviors. To the best of our knowledge, SCoPE VLM is the first framework to explicitly model agentic reading patterns in multi-page document question answering, advancing the capabilities of multimodal agents.

Method: Used perturbation kernel to remove dependency on pretrained PFGM++, introduced sinusoidal discretization schedule and Beta noise distribution for adaptability and sample quality improvement.

Result: Achieved improved low-dose CT image denoising in terms of LPIPS and SSIM metrics, with similar effectiveness to Consistency Model.

Conclusion: PFCT is a valid training method for PFCM that creates more flexibility in generative modeling, though further optimization and applicability studies are needed.

Abstract: The Poisson Flow Consistency Model (PFCM) is a consistency-style model based on the robust Poisson Flow Generative Model++ (PFGM++) which has achieved success in unconditional image generation and CT image denoising. Yet the PFCM can only be trained in distillation which limits the potential of the PFCM in many data modalities. The objective of this research was to create a method to train the PFCM in isolation called Poisson Flow Consistency Training (PFCT). The perturbation kernel was leveraged to remove the pretrained PFGM++, and the sinusoidal discretization schedule and Beta noise distribution were introduced in order to facilitate adaptability and improve sample quality. The model was applied to the task of low dose computed tomography image denoising and improved the low dose image in terms of LPIPS and SSIM. It also displayed similar denoising effectiveness as models like the Consistency Model. PFCT is established as a valid method of training the PFCM from its effectiveness in denoising CT images, showing potential with competitive results to other generative models. Further study is needed in the precise optimization of PFCT and in its applicability to other generative modeling tasks. The framework of PFCT creates more flexibility for the ways in which a PFCM can be created and can be applied to the field of generative modeling.

Method: Three-stage framework: 1) YOLOv11-det for layout segmentation, 2) YOLOv11-obb for orientation-aware annotation detection, 3) Two Donut-based VLMs (Alphabetical and Numerical) for semantic content parsing without OCR.

Result: Alphabetical VLM achieved F1 score of 0.672, Numerical VLM achieved 0.963. Two specialized datasets developed: 1,000 drawings for layout detection and 1,406 for annotation-level training.

Conclusion: The framework provides scalable automated interpretation of engineering drawings with unified JSON output for seamless CAD integration, addressing challenges in manufacturing communication.

Abstract: Engineering drawings are fundamental to manufacturing communication, serving as the primary medium for conveying design intent, tolerances, and production details. However, interpreting complex multi-view drawings with dense annotations remains challenging using manual methods, generic optical character recognition (OCR) systems, or traditional deep learning approaches, due to varied layouts, orientations, and mixed symbolic-textual content. To address these challenges, this paper proposes a three-stage hybrid framework for the automated interpretation of 2D multi-view engineering drawings using modern detection and vision language models (VLMs). In the first stage, YOLOv11-det performs layout segmentation to localize key regions such as views, title blocks, and notes. The second stage uses YOLOv11-obb for orientation-aware, fine-grained detection of annotations, including measures, GD&T symbols, and surface roughness indicators. The third stage employs two Donut-based, OCR-free VLMs for semantic content parsing: the Alphabetical VLM extracts textual and categorical information from title blocks and notes, while the Numerical VLM interprets quantitative data such as measures, GD&T frames, and surface roughness. Two specialized datasets were developed to ensure robustness and generalization: 1,000 drawings for layout detection and 1,406 for annotation-level training. The Alphabetical VLM achieved an overall F1 score of 0.672, while the Numerical VLM reached 0.963, demonstrating strong performance in textual and quantitative interpretation, respectively. The unified JSON output enables seamless integration with CAD and manufacturing databases, providing a scalable solution for intelligent engineering drawing analysis.

Method: Implicitly extracts emotion from speech and identity from neutral facial mesh, uses Hierarchical Interaction Fusion Block (HIFB) with fusion token to integrate emotional, motion, and identity cues through dual transformer features.

Result: Outperforms state-of-the-art methods on 3DMEAD dataset in emotional expressiveness, identity generalization, and animation realism.

Conclusion: LSF-Animation provides a more natural and adaptable facial animation framework that generalizes better to unseen speakers without requiring manual emotion/identity labels.

Abstract: Speech-driven 3D facial animation has attracted increasing interest since its potential to generate expressive and temporally synchronized digital humans. While recent works have begun to explore emotion-aware animation, they still depend on explicit one-hot encodings to represent identity and emotion with given emotion and identity labels, which limits their ability to generalize to unseen speakers. Moreover, the emotional cues inherently present in speech are often neglected, limiting the naturalness and adaptability of generated animations. In this work, we propose LSF-Animation, a novel framework that eliminates the reliance on explicit emotion and identity feature representations. Specifically, LSF-Animation implicitly extracts emotion information from speech and captures the identity features from a neutral facial mesh, enabling improved generalization to unseen speakers and emotional states without requiring manual labels. Furthermore, we introduce a Hierarchical Interaction Fusion Block (HIFB), which employs a fusion token to integrate dual transformer features and more effectively integrate emotional, motion-related and identity-related cues. Extensive experiments conducted on the 3DMEAD dataset demonstrate that our method surpasses recent state-of-the-art approaches in terms of emotional expressiveness, identity generalization, and animation realism. The source code will be released at: https://github.com/Dogter521/LSF-Animation.

Method: Uses world model architecture with compact scene representation, LLMs for temporal reasoning via lightweight tokenizer, and mixture-of-experts to decompose corner cases into subproblems with specialized experts for intent inference and counterfactual rollouts.

Result: Outperforms state-of-the-art baselines on four benchmark datasets (nuScenes, NGSIM, HighD, MoCAD) and remains robust under corner-case and data-missing conditions.

Conclusion: World model-based architectures show promise for robust and generalizable motion forecasting in autonomous vehicles, particularly for handling challenging corner-case scenarios.

Abstract: Accurate and reliable motion forecasting is essential for the safe deployment of autonomous vehicles (AVs), particularly in rare but safety-critical scenarios known as corner cases. Existing models often underperform in these situations due to an over-representation of common scenes in training data and limited generalization capabilities. To address this limitation, we present WM-MoE, the first world model-based motion forecasting framework that unifies perception, temporal memory, and decision making to address the challenges of high-risk corner-case scenarios. The model constructs a compact scene representation that explains current observations, anticipates future dynamics, and evaluates the outcomes of potential actions. To enhance long-horizon reasoning, we leverage large language models (LLMs) and introduce a lightweight temporal tokenizer that maps agent trajectories and contextual cues into the LLM’s feature space without additional training, enriching temporal context and commonsense priors. Furthermore, a mixture-of-experts (MoE) is introduced to decompose complex corner cases into subproblems and allocate capacity across scenario types, and a router assigns scenes to specialized experts that infer agent intent and perform counterfactual rollouts. In addition, we introduce nuScenes-corner, a new benchmark that comprises four real-world corner-case scenarios for rigorous evaluation. Extensive experiments on four benchmark datasets (nuScenes, NGSIM, HighD, and MoCAD) showcase that WM-MoE consistently outperforms state-of-the-art (SOTA) baselines and remains robust under corner-case and data-missing conditions, indicating the promise of world model-based architectures for robust and generalizable motion forecasting in fully AVs.

Method: Object-based image analysis using deep learning algorithms applied to high-resolution drone images, implemented on a cloud platform with high-performance processors to handle computational challenges of large datasets.

Result: The approach effectively estimates canopy coverage at city scale, providing detailed analysis of urban vegetation including canopy density variations and spatial distribution, enabling urban forestry management optimization.

Conclusion: This AI-based method generates valuable data for optimizing tree plantation and assessing carbon sequestration potential, contributing to sustainable urban planning and fostering more resilient urban environments.

Abstract: Accurate assessment of urban canopy coverage is crucial for informed urban planning, effective environmental monitoring, and mitigating the impacts of climate change. Traditional practices often face limitations due to inadequate technical requirements, difficulties in scaling and data processing, and the lack of specialized expertise. This study presents an efficient approach for estimating green canopy coverage using artificial intelligence, specifically computer vision techniques, applied to aerial imageries. Our proposed methodology utilizes object-based image analysis, based on deep learning algorithms to accurately identify and segment green canopies from high-resolution drone images. This approach allows the user for detailed analysis of urban vegetation, capturing variations in canopy density and understanding spatial distribution. To overcome the computational challenges associated with processing large datasets, it was implemented over a cloud platform utilizing high-performance processors. This infrastructure efficiently manages space complexity and ensures affordable latency, enabling the rapid analysis of vast amounts of drone imageries. Our results demonstrate the effectiveness of this approach in accurately estimating canopy coverage at the city scale, providing valuable insights for urban forestry management of an industrial township. The resultant data generated by this method can be used to optimize tree plantation and assess the carbon sequestration potential of urban forests. By integrating these insights into sustainable urban planning, we can foster more resilient urban environments, contributing to a greener and healthier future.

Method: Proposes TernaryCLIP framework that converts CLIP’s connection weights to ternary format using quantization-aware training and distillation modules to prevent precision degradation.

Result: Achieves 99% ternarized weights with 1.58-bit representation, 16.98× compression, 2.3× inference acceleration, 16× storage reduction, 10× memory optimization, and 60% sparsity while maintaining performance on 41 datasets for zero-shot classification and retrieval.

Conclusion: Demonstrates feasibility of extreme quantization for large multimodal models, supporting efficient deployment on resource-constrained devices with minimal performance loss.

Abstract: Recent years have witnessed an increasing interest in image-text contrastive modeling, exemplified by models such as Contrastive Language-Image Pretraining (CLIP). In this paper, we propose the TernaryCLIP, a lightweight computational framework that converts connection weights of both vision and text encoders of CLIP into the ternary format, instead of full-precision or floating ones. TernaryCLIP incorporates quantization-aware training and distillation modules, preventing precision degradation and enabling low-cost and high-efficiency computations. Comprehensive experiments demonstrate that TernaryCLIP can achieve up to 99% ternarized weights with 1.58-bit representation, 16.98 $\times$ compression ratio, 2.3 $\times$ inference acceleration, 16 $\times$ storage reduction, 10 $\times$ memory optimization, and 60% sparsity while maintaining promising performance on zero-shot image classification and image-text retrieval tasks across 41 commonly used datasets. Our work highlights the feasibility of extreme quantization for large multimodal models, supporting effective and efficient deployment on resource-constrained devices. The model and code can be accessed from Hugging Face and GitHub.

Method: The authors introduce a comprehensive taxonomy that organizes literature on GANs, VAEs, and DMs, including their variants and combined approaches, and provide a cohesive framework for understanding their development.

Result: The survey highlights key innovations that have improved quality, diversity, and controllability of generated outputs, reflecting the expanding potential of generative AI.

Conclusion: The paper outlines persistent challenges, proposes future research directions, and offers a structured perspective for researchers, while also examining ethical concerns and societal impacts of synthetic media.

Abstract: In recent years, deep learning based generative models, particularly Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), and Diffusion Models (DMs), have been instrumental in in generating diverse, high-quality content across various domains, such as image and video synthesis. This capability has led to widespread adoption of these models and has captured strong public interest. As they continue to advance at a rapid pace, the growing volume of research, expanding application areas, and unresolved technical challenges make it increasingly difficult to stay current. To address this need, this survey introduces a comprehensive taxonomy that organizes the literature and provides a cohesive framework for understanding the development of GANs, VAEs, and DMs, including their many variants and combined approaches. We highlight key innovations that have improved the quality, diversity, and controllability of generated outputs, reflecting the expanding potential of generative artificial intelligence. In addition to summarizing technical progress, we examine rising ethical concerns, including the risks of misuse and the broader societal impact of synthetic media. Finally, we outline persistent challenges and propose future research directions, offering a structured and forward looking perspective for researchers in this fast evolving field.

Method: SPRINT uses shallow layers to process all tokens for local detail, deeper layers on sparse subsets for efficiency, with residual fusion. Two-stage training: masked pre-training followed by full-token fine-tuning.

Result: Achieves 9.8x training savings on ImageNet-1K 256x256 with comparable FID/FDD. Inference Path-Drop Guidance nearly halves FLOPs while improving quality.

Conclusion: SPRINT provides a simple, effective, and general solution for efficient DiT training with significant computational savings and maintained performance.

Abstract: Diffusion Transformers (DiTs) deliver state-of-the-art generative performance but their quadratic training cost with sequence length makes large-scale pretraining prohibitively expensive. Token dropping can reduce training cost, yet na"ive strategies degrade representations, and existing methods are either parameter-heavy or fail at high drop ratios. We present SPRINT, Sparse–Dense Residual Fusion for Efficient Diffusion Transformers, a simple method that enables aggressive token dropping (up to 75%) while preserving quality. SPRINT leverages the complementary roles of shallow and deep layers: early layers process all tokens to capture local detail, deeper layers operate on a sparse subset to cut computation, and their outputs are fused through residual connections. Training follows a two-stage schedule: long masked pre-training for efficiency followed by short full-token fine-tuning to close the train–inference gap. On ImageNet-1K 256x256, SPRINT achieves 9.8x training savings with comparable FID/FDD, and at inference, its Path-Drop Guidance (PDG) nearly halves FLOPs while improving quality. These results establish SPRINT as a simple, effective, and general solution for efficient DiT training.

Method: Integrates lightweight adaptation layers into frozen diffusion U-Net, uses latent morphological autoencoder for domain-specific latent consistency and pixel-level discriminator for adversarial alignment, while freezing base model weights.

Result: Achieves superior adaptation efficiency compared to naive full fine-tuning on three chest X-ray datasets, with optimal balance between efficiency and performance.

Conclusion: Provides a promising direction for transfer learning in diffusion models, facilitating deployment in diverse low-data domains.

Abstract: In recent years, diffusion models have demonstrated remarkable success in high-fidelity image synthesis. However, fine-tuning these models for specialized domains, such as medical imaging, remains challenging due to limited domain-specific data and the high computational cost of full model adaptation. In this paper, we introduce Lite-Diff (Lightweight Diffusion Model Adaptation), a novel finetuning approach that integrates lightweight adaptation layers into a frozen diffusion U-Net while enhancing training with a latent morphological autoencoder (for domain-specific latent consistency) and a pixel level discriminator(for adversarial alignment). By freezing weights of the base model and optimizing only small residual adapter modules, LiteDiff significantly reduces the computational overhead and mitigates overfitting, even in minimal-data settings. Additionally, we conduct ablation studies to analyze the effects of selectively integrating adaptation layers in different U-Net blocks, revealing an optimal balance between efficiency and performance. Experiments on three chest X-ray datasets - (1) Kaggle Chest X-Ray Pneumonia, (2) NIH Chest X-ray14 and (3) VinBigData Chest X_ray demonstrate that LiteDiff achieves superior adaptation efficiency compared to naive full fine-tuning. Our framework provides a promising direction for transfer learning in diffusion models, facilitating their deployment in diverse low data domains.

Method: Hybrid CNN-LSTM architecture using Mediapipe for gesture keypoint extraction, developed with Python, TensorFlow and Streamlit for real-time gesture translation.

Result: 92% average accuracy, with excellent performance for distinct gestures like ‘Hello’ and ‘Thank you’, but confusion remains for visually similar gestures like ‘Call’ and ‘Yes’.

Conclusion: The system shows promising potential for applications in healthcare, education and public services, though challenges remain with visually similar gestures.

Abstract: Sign languages play a crucial role in the communication of deaf communities, but they are often marginalized, limiting access to essential services such as healthcare and education. This study proposes an automatic sign language recognition system based on a hybrid CNN-LSTM architecture, using Mediapipe for gesture keypoint extraction. Developed with Python, TensorFlow and Streamlit, the system provides real-time gesture translation. The results show an average accuracy of 92%, with very good performance for distinct gestures such as Hello'' and Thank you’’. However, some confusions remain for visually similar gestures, such as Call'' and Yes’’. This work opens up interesting perspectives for applications in various fields such as healthcare, education and public services.

Method: Developed VLM-SlideEval framework with three evaluation axes: element-level extraction from slide images, robustness to controlled perturbations (geometry, style, text), and higher-level comprehension including narrative order recovery from shuffled slides. Used publicly available decks from Zenodo with standardized ground-truth metadata from PowerPoint XML and live renderings.

Result: VLMs underperform on pixel-accurate extraction, show non-trivial agreement/fidelity/consistency under perturbations, perform better on single-slide content understanding, but fail to reliably capture narrative structure across slides.

Conclusion: Current VLMs have significant limitations for slide evaluation, motivating the need for calibrated, critic-in-the-loop evaluators that enable iterative refinement and selection in agentic pipelines.

Abstract: Vision-language models (VLMs) are increasingly used to evaluate multimodal content, including presentation slides, yet their slide-specific understanding remains underexplored {despite their growing role as critics in agentic, model-forward pipelines}. We introduce VLM-SlideEval, an evaluation framework that probes VLMs along three axes: (1) element-level extraction from slide images aligned to ground truth; (2) robustness to controlled perturbations in geometry, style, and text; and (3) higher-level comprehension, such as recovering a deck’s narrative order from shuffled slides. Using publicly available decks from Zenodo (https://huggingface.co/datasets/Forceless/Zenodo10K/viewer/default/pptx), we standardize ground-truth element metadata from PowerPoint XML and live renderings into a unified, verifiable schema. Empirically, VLMs underperform on pixel-accurate extraction and show non-trivial agreement, fidelity, and consistency under controlled perturbations, while performing better on single-slide content understanding; however, they do not reliably capture narrative structure across slides. These results highlight the limits of current VLMs for slide evaluation and motivate calibrated, critic-in-the-loop evaluators that drive iterative refinement and selection in agentic pipelines.

Method: Human-centric preprocessing using YOLO-World for detection and ByteTrack for tracking, background suppression via Gaussian blurring, spatial feature extraction with InceptionV3, and temporal modeling with bidirectional LSTM for sequence classification.

Result: Achieved 92.41% mean test accuracy on five-class UCF-Crime subset, with per-class F1-scores consistently above 0.85, demonstrating strong generalization and resilience to class imbalance.

Conclusion: Foreground-focused preprocessing significantly enhances anomaly discrimination in real-world surveillance scenarios, confirming the effectiveness of the proposed framework.

Abstract: Anomaly detection in surveillance videos remains a challenging task due to the diversity of abnormal events, class imbalance, and scene-dependent visual clutter. To address these issues, we propose a robust deep learning framework that integrates human-centric preprocessing with spatio-temporal modeling for multi-class anomaly classification. Our pipeline begins by applying YOLO-World

• an open-vocabulary vision-language detector - to identify human instances in raw video clips, followed by ByteTrack for consistent identity-aware tracking. Background regions outside detected bounding boxes are suppressed via Gaussian blurring, effectively reducing scene-specific distractions and focusing the model on behaviorally relevant foreground content. The refined frames are then processed by an ImageNet-pretrained InceptionV3 network for spatial feature extraction, and temporal dynamics are captured using a bidirectional LSTM (BiLSTM) for sequence-level classification. Evaluated on a five-class subset of the UCF-Crime dataset (Normal, Burglary, Fighting, Arson, Explosion), our method achieves a mean test accuracy of 92.41% across three independent trials, with per-class F1-scores consistently exceeding 0.85. Comprehensive evaluation metrics - including confusion matrices, ROC curves, and macro/weighted averages
• demonstrate strong generalization and resilience to class imbalance. The results confirm that foreground-focused preprocessing significantly enhances anomaly discrimination in real-world surveillance scenarios.

Method: GIFT pre-computes holistic visual saliency maps by tracking positive changes in visual attention (gaze shifts) during user query comprehension, then amplifies attention to both salient visual information and user queries at each decoding step.

Result: GIFT achieves up to 20.7% improvement over greedy decoding in mitigating hallucination across both generative and classification tasks, while maintaining general vision-language performance with low computational overhead.

Conclusion: The proposed GIFT method effectively addresses visual attention sink and cross-modal fusion imbalance problems, significantly reducing hallucination in VLMs while preserving overall performance with minimal computational cost.

Abstract: Vision language models (VLMs) often generate hallucination, i.e., content that cannot be substantiated by either textual or visual inputs. Prior work primarily attributes this to over-reliance on linguistic prior knowledge rather than visual inputs. Some methods attempt to mitigate hallucination by amplifying visual token attention proportionally to their attention scores. However, these methods overlook the visual attention sink problem, where attention is frequently misallocated to task-irrelevant visual regions, and neglect cross-modal fusion balance by enhancing only visual attention without adjusting attention to the user query. This can result in amplifying incorrect areas while failing to properly interpret the user query. To address these challenges, we propose a simple yet effective method called Gaze Shift-Guided Cross-modal Fusion Enhancement (GIFT). GIFT pre-computes a holistic visual saliency map by tracking positive changes in visual attention, or “gaze shifts”, during user query comprehension, and leverages this map to amplify attention to both salient visual information and the user query at each decoding step. This reduces the impact of visual attention sink, as irrelevant tokens exhibit minimal shifts, while ensuring balanced cross-modal fusion for well-integrated representation. Extensive experiments show that GIFT effectively mitigates hallucination in VLMs across both generative and classification tasks, achieving up to 20.7% improvement over greedy decoding, while maintaining general vision-language performance with low computational overhead.

Method: Uses a differentiable SSIM-based loss to separate anatomical content from scanner-specific variations, enabling separate evaluation of luminance, contrast, and structural components. Trained on multiple public datasets with diverse scanners and sites.

Result: Harmonized images achieved strong alignment across acquisition settings: structural SSIM 0.97, luminance SSIM 0.98-0.99, reduced Wasserstein distances. Downstream improvements: brain age prediction MAE decreased from 5.36 to 3.30 years, Alzheimer’s classification AUC increased from 0.78 to 0.85.

Conclusion: The framework enhances cross-site image consistency, preserves anatomical fidelity, and improves downstream model performance, providing a robust solution for large-scale multicenter neuroimaging studies.

Abstract: The variability introduced by differences in MRI scanner models, acquisition protocols, and imaging sites hinders consistent analysis and generalizability across multicenter studies. We present a novel image-based harmonization framework for 3D T1-weighted brain MRI, which disentangles anatomical content from scanner- and site-specific variations. The model incorporates a differentiable loss based on the Structural Similarity Index (SSIM) to preserve biologically meaningful features while reducing inter-site variability. This loss enables separate evaluation of image luminance, contrast, and structural components. Training and validation were performed on multiple publicly available datasets spanning diverse scanners and sites, with testing on both healthy and clinical populations. Harmonization using multiple style targets, including style-agnostic references, produced consistent and high-quality outputs. Visual comparisons, voxel intensity distributions, and SSIM-based metrics demonstrated that harmonized images achieved strong alignment across acquisition settings while maintaining anatomical fidelity. Following harmonization, structural SSIM reached 0.97, luminance SSIM ranged from 0.98 to 0.99, and Wasserstein distances between mean voxel intensity distributions decreased substantially. Downstream tasks showed substantial improvements: mean absolute error for brain age prediction decreased from 5.36 to 3.30 years, and Alzheimer’s disease classification AUC increased from 0.78 to 0.85. Overall, our framework enhances cross-site image consistency, preserves anatomical fidelity, and improves downstream model performance, providing a robust and generalizable solution for large-scale multicenter neuroimaging studies.

Method: Proposed Vision-PE Shuffle Guidance (VPSG) - a training-free test-time method that runs auxiliary decoding with shuffled visual positional encodings to isolate position-unconditioned biases, then uses this as negative evidence to guide digit prediction while preserving coordinate format.

Result: Experiments on ScreenSpot-Pro demonstrate reliable improvements in coordinate prediction accuracy, showing that addressing positional encoding robustness is critical for spatial reasoning in MLLMs.

Conclusion: Positional encoding failures are a key bottleneck for accurate coordinate prediction at scale, and VPSG effectively mitigates directional biases to improve spatial reasoning performance in MLLMs.

Abstract: Multimodal large language models (MLLMs) excel at vision-language tasks such as VQA and document understanding, yet precise coordinate prediction remains challenging. High-resolution inputs exacerbate this difficulty by producing long token sequences that weaken positional encodings and introduce directional biases in coordinate outputs. We investigate this phenomenon by analyzing how MLLMs behave when visual positional encodings (VPEs) are deliberately perturbed through shuffling. Our analysis reveals that such perturbations induce predictable, non-random coordinate biases rather than random errors, suggesting that models rely on internal positional priors when spatial grounding signals are degraded. Crucially, we observe similar directional error patterns in natural high-resolution datasets, indicating that positional encoding failures are a key bottleneck for accurate coordinate prediction at scale. To address this issue, we propose Vision-PE Shuffle Guidance (VPSG), a training-free test-time method that leverages the directional nature of these biases for correction. VPSG runs auxiliary decoding with shuffled VPEs to isolate position-unconditioned tendencies, then uses this as negative evidence to guide digit prediction while preserving coordinate format through a lightweight finite-state machine. Experiments on ScreenSpot-Pro demonstrate reliable improvements, highlighting positional encoding robustness as a critical factor for spatial reasoning in MLLMs.

Method: Integrates a latent graph parameterized by GFlowNets into prompt representation computation, leveraging GFlowNets’ advanced graph sampling capabilities to produce multiple trajectories that represent different aspects of input condition uncertainty, leading to diverse condition representations and corresponding images.

Result: Evaluations on natural and medical image datasets show Rainbow improves both diversity and fidelity across image synthesis, image generation, and counterfactual generation tasks compared to traditional approaches.

Conclusion: Rainbow effectively addresses condition uncertainty in image generation by decomposing input conditions into diverse latent representations through GFlowNets, enabling generation of multiple plausible and interpretable images from ambiguous prompts.

Abstract: Capturing diversity is crucial in conditional and prompt-based image generation, particularly when conditions contain uncertainty that can lead to multiple plausible outputs. To generate diverse images reflecting this diversity, traditional methods often modify random seeds, making it difficult to discern meaningful differences between samples, or diversify the input prompt, which is limited in verbally interpretable diversity. We propose Rainbow, a novel conditional image generation framework, applicable to any pretrained conditional generative model, that addresses inherent condition/prompt uncertainty and generates diverse plausible images. Rainbow is based on a simple yet effective idea: decomposing the input condition into diverse latent representations, each capturing an aspect of the uncertainty and generating a distinct image. First, we integrate a latent graph, parameterized by Generative Flow Networks (GFlowNets), into the prompt representation computation. Second, leveraging GFlowNets’ advanced graph sampling capabilities to capture uncertainty and output diverse trajectories over the graph, we produce multiple trajectories that collectively represent the input condition, leading to diverse condition representations and corresponding output images. Evaluations on natural image and medical image datasets demonstrate Rainbow’s improvement in both diversity and fidelity across image synthesis, image generation, and counterfactual generation tasks.

Method: GRAID operates exclusively on 2D bounding boxes from standard object detectors to determine qualitative spatial relationships, avoiding both 3D reconstruction errors and generative hallucinations. The framework generates VQA pairs spanning spatial relations, counting, ranking, and size comparisons.

Result: Generated over 8.5 million high-quality VQA pairs across BDD100k, NuImages, and Waymo datasets with 91.16% human-validated accuracy (vs 57.6% from recent work). Models fine-tuned on GRAID data show strong generalization - improving on over 10 held-out question types with 47.5% gains on BDD and 37.9% on NuImages for Llama 3.2B 11B.

Conclusion: GRAID provides a reliable method for generating high-quality spatial reasoning datasets using 2D geometric primitives, significantly outperforming existing approaches and enabling VLMs to learn generalizable spatial reasoning concepts that transfer to various benchmarks.

Abstract: Vision Language Models (VLMs) achieve strong performance on many vision-language tasks but often struggle with spatial reasoning\textemdash{}a prerequisite for many applications. Empirically, we find that a dataset produced by a current training data generation pipeline has a 57.6% human validation rate. These rates stem from current limitations: single-image 3D reconstruction introduces cascading modeling errors and requires wide answer tolerances, while caption-based methods require hyper-detailed annotations and suffer from generative hallucinations. We present GRAID, built on the key insight that qualitative spatial relationships can be reliably determined from 2D geometric primitives alone. By operating exclusively on 2D bounding boxes from standard object detectors, GRAID avoids both 3D reconstruction errors and generative hallucinations, resulting in datasets that are of higher quality than existing tools that produce similar datasets as validated by human evaluations. We apply our framework to the BDD100k, NuImages, and Waymo datasets, generating over 8.5 million high-quality VQA pairs creating questions spanning spatial relations, counting, ranking, and size comparisons. We evaluate one of the datasets and find it achieves 91.16% human-validated accuracy\textemdash{}compared to 57.6% on a dataset generated by recent work. % or recent work Critically, we demonstrate that when trained on GRAID data, models learn spatial reasoning concepts that generalize: models fine-tuned on 6 question types improve on over 10 held-out types, with accuracy gains of 47.5% on BDD and 37.9% on NuImages for Llama 3.2B 11B, and when trained on all questions types, achieve improvements on several existing benchmarks such as BLINK. The GRAID framework, datasets, and additional information can be found on our \href{https://ke7.github.io/graid/}{project page}.

Method: CogStereo embeds implicit spatial cognition into refinement using monocular depth features as priors, employing a dual-conditional refinement mechanism that combines pixel-wise uncertainty with cognition-guided features for global correction.

Result: Extensive experiments on Scene Flow, KITTI, Middlebury, ETH3D, EuRoc, and real-world datasets show state-of-the-art results and excellent cross-domain generalization.

Conclusion: CogStereo shifts stereo vision towards a cognition-driven approach, ensuring structurally coherent disparity estimation even where geometry alone is inadequate, without relying on dataset-specific priors.

Abstract: Deep stereo matching has advanced significantly on benchmark datasets through fine-tuning but falls short of the zero-shot generalization seen in foundation models in other vision tasks. We introduce CogStereo, a novel framework that addresses challenging regions, such as occlusions or weak textures, without relying on dataset-specific priors. CogStereo embeds implicit spatial cognition into the refinement process by using monocular depth features as priors, capturing holistic scene understanding beyond local correspondences. This approach ensures structurally coherent disparity estimation, even in areas where geometry alone is inadequate. CogStereo employs a dual-conditional refinement mechanism that combines pixel-wise uncertainty with cognition-guided features for consistent global correction of mismatches. Extensive experiments on Scene Flow, KITTI, Middlebury, ETH3D, EuRoc, and real-world demonstrate that CogStereo not only achieves state-of-the-art results but also excels in cross-domain generalization, shifting stereo vision towards a cognition-driven approach.

Method: The authors analyze embedding variance collapse phenomenon and propose Mint - a test-time adaptation method that uses mean and gradient accumulators to maximize pseudo-label-based inter-class variance on the fly, working effectively with small batch sizes.

Result: Mint consistently improves performance across multiple corruption benchmarks and CLIP architectures, demonstrating enhanced robustness to input corruptions.

Conclusion: Maximizing inter-class variance, even with pseudo-labels, can provably enhance embedding quality and improve CLIP’s robustness to distribution shifts caused by input corruptions.

Abstract: Pretrained vision-language models such as CLIP achieve strong zero-shot generalization but remain vulnerable to distribution shifts caused by input corruptions. In this work, we investigate how corruptions affect CLIP’s image embeddings and uncover a consistent phenomenon we term as embedding variance collapse, where both intra-class and inter-class variances shrink as corruption severity increases. We find that this collapse is closely tied to performance degradation, with inter-class variance strongly correlated with classification accuracy. To explain this phenomenon, we analyze how corruptions alter the structure of the embedding space. Our theoretical results suggest that the visual encoder tends to encode corruption-related signals, which dilute class-discriminative features and compress the representation geometry. We further show that maximizing inter-class variance, even when estimated from pseudo-labels, can provably enhance embedding quality. Based on this insight, we propose Mint, a simple test-time adaptation method that maximizes pseudo-label-based inter-class variance on the fly using a mean accumulator and a gradient accumulator. Mint operates effectively with small batch sizes and consistently improves performance across multiple corruption benchmarks and CLIP architectures. Our code is available at https://github.com/baowenxuan/Mint .

Method: Created egoEMOTION dataset with 50+ hours of recordings from 43 participants using Meta’s Project Aria glasses, capturing synchronized eye-tracking video, photoplethysmography, inertial motion data, and physiological baselines during emotion-elicitation tasks and naturalistic activities.

Result: A classical learning-based method shows better affect prediction from egocentric vision signals than from physiological signals alone, establishing three benchmark tasks: continuous affect classification, discrete emotion classification, and personality inference.

Conclusion: The dataset establishes emotion and personality as core dimensions in egocentric perception, opening new directions for affect-driven modeling of behavior, intent, and interaction.

Abstract: Understanding affect is central to anticipating human behavior, yet current egocentric vision benchmarks largely ignore the person’s emotional states that shape their decisions and actions. Existing tasks in egocentric perception focus on physical activities, hand-object interactions, and attention modeling

• assuming neutral affect and uniform personality. This limits the ability of vision systems to capture key internal drivers of behavior. In this paper, we present egoEMOTION, the first dataset that couples egocentric visual and physiological signals with dense self-reports of emotion and personality across controlled and real-world scenarios. Our dataset includes over 50 hours of recordings from 43 participants, captured using Meta’s Project Aria glasses. Each session provides synchronized eye-tracking video, headmounted photoplethysmography, inertial motion data, and physiological baselines for reference. Participants completed emotion-elicitation tasks and naturalistic activities while self-reporting their affective state using the Circumplex Model and Mikels’ Wheel as well as their personality via the Big Five model. We define three benchmark tasks: (1) continuous affect classification (valence, arousal, dominance); (2) discrete emotion classification; and (3) trait-level personality inference. We show that a classical learning-based method, as a simple baseline in real-world affect prediction, produces better estimates from signals captured on egocentric vision systems than processing physiological signals. Our dataset establishes emotion and personality as core dimensions in egocentric perception and opens new directions in affect-driven modeling of behavior, intent, and interaction.

Method: A rigid-nonrigid coupled deformation framework integrating Spacetime Gaussians (STG) with linear blend skinning (LBS), using optical flow to identify high-dynamic regions and guide adaptive densification of 3D Gaussians.

Result: The method consistently outperforms state-of-the-art baselines in both reconstruction quality and operational efficiency, achieving superior quantitative metrics while maintaining real-time rendering capabilities.

Conclusion: STG-Avatar successfully addresses the challenges of representing detailed non-rigid features and dynamic regions in human avatar reconstruction, providing a high-fidelity solution with real-time performance.

Abstract: Realistic animatable human avatars from monocular videos are crucial for advancing human-robot interaction and enhancing immersive virtual experiences. While recent research on 3DGS-based human avatars has made progress, it still struggles with accurately representing detailed features of non-rigid objects (e.g., clothing deformations) and dynamic regions (e.g., rapidly moving limbs). To address these challenges, we present STG-Avatar, a 3DGS-based framework for high-fidelity animatable human avatar reconstruction. Specifically, our framework introduces a rigid-nonrigid coupled deformation framework that synergistically integrates Spacetime Gaussians (STG) with linear blend skinning (LBS). In this hybrid design, LBS enables real-time skeletal control by driving global pose transformations, while STG complements it through spacetime adaptive optimization of 3D Gaussians. Furthermore, we employ optical flow to identify high-dynamic regions and guide the adaptive densification of 3D Gaussians in these regions. Experimental results demonstrate that our method consistently outperforms state-of-the-art baselines in both reconstruction quality and operational efficiency, achieving superior quantitative metrics while retaining real-time rendering capabilities. Our code is available at https://github.com/jiangguangan/STG-Avatar

Method: Proposes ARFNet with three key components: 1) Attention residual fusion using spatial-wise and channel-wise attentions for cross-layer fusion and self-distillation, 2) Global-local attention contrast to improve category discrimination, 3) Dynamic centroid evaluation for trustworthy centroids and pseudo-labels to mitigate domain shift.

Result: Comprehensive experiments on five benchmarks show the method surpasses other techniques and achieves superior performance across SFDA benchmarks.

Conclusion: ARFNet effectively alleviates negative transfer and domain shift in SFDA through attention mechanisms and contrast learning, demonstrating state-of-the-art performance on multiple benchmarks.

Abstract: Source-free domain adaptation (SFDA) involves training a model on source domain and then applying it to a related target domain without access to the source data and labels during adaptation. The complexity of scene information and lack of the source domain make SFDA a difficult task. Recent studies have shown promising results, but many approaches to domain adaptation concentrate on domain shift and neglect the effects of negative transfer, which may impede enhancements of model performance during adaptation. n this paper, addressing this issue, we propose a novel framework of Attention Residual Fusion Network (ARFNet) based on contrast learning for SFDA to alleviate negative transfer and domain shift during the progress of adaptation, in which attention residual fusion, global-local attention contrast, and dynamic centroid evaluation are exploited. Concretely, the attention mechanism is first exploited to capture the discriminative region of the target object. Then, in each block, attention features are decomposed into spatial-wise and channel-wise attentions to achieve the cross-layer attention residual fusion progressively and self-distillation. During adaptation progress, we contrast global and local representations to improve the perceptual capabilities of different categories, which enables the model to discriminate variations between inner-class and intra-class. Finally, a dynamic centroid evaluation strategy is exploited to evaluate the trustworthy centroids and labels for self-supervised self-distillation, which aims to accurately approximate the center of the source domain and pseudo-labels to mitigate domain shift. To validate the efficacy, we execute comprehensive experiments on five benchmarks of varying scales. Experimental outcomes indicate that our method surpasses other techniques, attaining superior performance across SFDA benchmarks.

Method: Introduces reverse-stratified upsampling for near-uniform 3D space sampling to preserve isometry, and a general radiative formulation unifying emission, absorption, and scattering using Beer-Lambert attenuation law for handling complex media environments.

Result: Experiments on real-world datasets show significant improvements in both reconstruction fidelity and physical plausibility compared to existing approaches, with capability to estimate medium properties like water depth.

Conclusion: I2-NeRF successfully enhances 3D physical perception by addressing isometric and isotropic metric perception challenges in degraded media environments, providing a unified framework for various complex scenarios including underwater, haze, and low-light scenes.

Abstract: Participating in efforts to endow generative AI with the 3D physical world perception, we propose I2-NeRF, a novel neural radiance field framework that enhances isometric and isotropic metric perception under media degradation. While existing NeRF models predominantly rely on object-centric sampling, I2-NeRF introduces a reverse-stratified upsampling strategy to achieve near-uniform sampling across 3D space, thereby preserving isometry. We further present a general radiative formulation for media degradation that unifies emission, absorption, and scattering into a particle model governed by the Beer-Lambert attenuation law. By composing the direct and media-induced in-scatter radiance, this formulation extends naturally to complex media environments such as underwater, haze, and even low-light scenes. By treating light propagation uniformly in both vertical and horizontal directions, I2-NeRF enables isotropic metric perception and can even estimate medium properties such as water depth. Experiments on real-world datasets demonstrate that our method significantly improves both reconstruction fidelity and physical plausibility compared to existing approaches.

Method: Proposes HARMONY framework that combines fused multimodal information from model activations with output probability distributions to determine response reliability, leveraging both internal visual understanding beliefs and token probabilities.

Result: Experimental results on A-OKVQA, VizWiz, and PathVQA benchmarks with LLaVa-7b, LLaVA-13b and InstructBLIP show consistent performance improvements, achieving up to 4% AUROC and 6% PRR improvements over existing approaches.

Conclusion: HARMONY establishes new state-of-the-art in uncertainty estimation for VLMs by jointly leveraging multimodal activations and output distributions, demonstrating that both components provide valuable reliability signals.

Abstract: The growing deployment of Vision-Language Models (VLMs) in high-stakes applications such as autonomous driving and assistive technologies for visually impaired individuals necessitates reliable mechanisms to assess the trustworthiness of their generation. Uncertainty Estimation (UE) plays a central role in quantifying the reliability of model outputs and reducing unsafe generations via selective prediction. In this regard, most existing probability-based UE approaches rely on output probability distributions, aggregating token probabilities into a single uncertainty score using predefined functions such as length-normalization. Another line of research leverages model hidden representations and trains MLP-based models to predict uncertainty. However, these methods often fail to capture the complex multimodal relationships between semantic and textual tokens and struggle to identify biased probabilities often influenced by language priors. Motivated by these observations, we propose a novel UE framework, HARMONY, that jointly leverages fused multimodal information in model activations and the output distribution of the VLM to determine the reliability of responses. The key hypothesis of our work is that both the model’s internal belief in its visual understanding, captured by its hidden representations, and the produced token probabilities carry valuable reliability signals that can be jointly leveraged to improve UE performance, surpassing approaches that rely on only one of these components. Experimental results on three open-ended VQA benchmarks, A-OKVQA, VizWiz, and PathVQA, and three state-of-the-art VLMs, LLaVa-7b, LLaVA-13b and InstructBLIP demonstrate that our method consistently performs on par with or better than existing approaches, achieving up to 4% improvement in AUROC, and 6% in PRR, establishing new state of the art in uncertainty estimation for VLMs.

Method: A non-parametric model that defines each pixel’s distribution from its local context window, grounded in three principles: spatial non-stationarity, low-level regularities, and high-level semantics.

Result: The model produces high-fidelity samples on MNIST and visually compelling CIFAR-10 images despite minimal architecture and no training.

Conclusion: The model’s white-box nature provides mechanistic understanding of generalization and reveals a compositional procedure for part-whole generalization, suggesting how large neural networks learn to generalize.

Abstract: Scaling and architectural advances have produced strikingly photorealistic image generative models, yet their mechanisms still remain opaque. Rather than advancing scaling, our goal is to strip away complicated engineering tricks and propose a simple, non-parametric generative model. Our design is grounded in three principles of natural images-(i) spatial non-stationarity, (ii) low-level regularities, and (iii) high-level semantics-and defines each pixel’s distribution from its local context window. Despite its minimal architecture and no training, the model produces high-fidelity samples on MNIST and visually compelling CIFAR-10 images. This combination of simplicity and strong empirical performance points toward a minimal theory of natural-image structure. The model’s white-box nature also allows us to have a mechanistic understanding of how the model generalizes and generates diverse images. We study it by tracing each generated pixel back to its source images. These analyses reveal a simple, compositional procedure for “part-whole generalization”, suggesting a hypothesis for how large neural network generative models learn to generalize.

Method: Introduces MOGRAS dataset with pre-grasping full-body walking motions and final grasping poses in annotated 3D indoor scenes, and proposes a simple yet effective method to adapt existing approaches for scene-aware generation.

Result: The dataset enables benchmarking of existing methods and reveals their limitations in scene-aware generation. The proposed adaptation method achieves significant improvements through extensive quantitative and qualitative experiments.

Conclusion: MOGRAS dataset and the proposed adaptation method effectively bridge the gap between scene-aware motion generation and fine-grained grasping tasks, paving the way for more realistic human-scene interactions in applications like robotics and virtual reality.

Abstract: Generating realistic full-body motion interacting with objects is critical for applications in robotics, virtual reality, and human-computer interaction. While existing methods can generate full-body motion within 3D scenes, they often lack the fidelity for fine-grained tasks like object grasping. Conversely, methods that generate precise grasping motions typically ignore the surrounding 3D scene. This gap, generating full-body grasping motions that are physically plausible within a 3D scene, remains a significant challenge. To address this, we introduce MOGRAS (Human MOtion with GRAsping in 3D Scenes), a large-scale dataset that bridges this gap. MOGRAS provides pre-grasping full-body walking motions and final grasping poses within richly annotated 3D indoor scenes. We leverage MOGRAS to benchmark existing full-body grasping methods and demonstrate their limitations in scene-aware generation. Furthermore, we propose a simple yet effective method to adapt existing approaches to work seamlessly within 3D scenes. Through extensive quantitative and qualitative experiments, we validate the effectiveness of our dataset and highlight the significant improvements our proposed method achieves, paving the way for more realistic human-scene interactions.

Method: Built on Diffusion Transformer (DiT) framework with unified architecture for Text-to-Video, Image-to-Video, and Video-Continuation tasks. Uses coarse-to-fine generation strategy, Block Sparse Attention for efficiency, and multi-reward RLHF training.

Result: Generates 720p, 30fps videos within minutes, maintains high quality and temporal coherence for long videos, and achieves performance comparable to latest closed-source and leading open-source models.

Conclusion: LongCat-Video represents a significant step toward world models with efficient long video generation capabilities, and the model is publicly available to accelerate progress in the field.

Abstract: Video generation is a critical pathway toward world models, with efficient long video inference as a key capability. Toward this end, we introduce LongCat-Video, a foundational video generation model with 13.6B parameters, delivering strong performance across multiple video generation tasks. It particularly excels in efficient and high-quality long video generation, representing our first step toward world models. Key features include: Unified architecture for multiple tasks: Built on the Diffusion Transformer (DiT) framework, LongCat-Video supports Text-to-Video, Image-to-Video, and Video-Continuation tasks with a single model; Long video generation: Pretraining on Video-Continuation tasks enables LongCat-Video to maintain high quality and temporal coherence in the generation of minutes-long videos; Efficient inference: LongCat-Video generates 720p, 30fps videos within minutes by employing a coarse-to-fine generation strategy along both the temporal and spatial axes. Block Sparse Attention further enhances efficiency, particularly at high resolutions; Strong performance with multi-reward RLHF: Multi-reward RLHF training enables LongCat-Video to achieve performance on par with the latest closed-source and leading open-source models. Code and model weights are publicly available to accelerate progress in the field.

Method: Proposes a framework integrating YOLOv10n for object detection and DeepSORT for tracking, plus two trajectory prediction models: TrajGATFormer (worker-only prediction) and TrajGATFormer-Obstacle (worker and obstacle prediction). Both use transformer encoder-decoder with Graph Attention Networks to capture temporal and spatial interactions.

Result: TrajGATFormer achieves ADE of 1.25 m and FDE of 2.3 m over 4.8 s horizon. TrajGATFormer-Obstacle achieves higher accuracy with ADE 1.15 m and FDE 2.2 m. Both models outperform traditional methods, reducing ADE and FDE by up to 35% and 38% respectively.

Conclusion: The proposed framework effectively addresses limitations of existing methods by integrating precise detection/tracking with advanced trajectory prediction models that capture complex spatial-temporal interactions, significantly improving collision risk assessment in dynamic construction environments.

Abstract: As the demand grows within the construction industry for processes that are not only faster but also safer and more efficient, offsite construction has emerged as a solution, though it brings new safety risks due to the close interaction between workers, machinery, and moving obstacles. Predicting the future trajectories of workers and taking into account social and environmental factors is a crucial step for developing collision-avoidance systems to mitigate such risks. Traditional methods often struggle to adapt to the dynamic and unpredictable nature of construction environments. Many rely on simplified assumptions or require hand-crafted features, limiting their ability to respond to complex, real-time interactions between workers and moving obstacles. While recent data-driven methods have improved the modeling of temporal patterns, they still face challenges in capturing long-term behavior and accounting for the spatial and social context crucial to collision risk assessment. To address these limitations, this paper proposes a framework integrating YOLOv10n and DeepSORT for precise detection and tracking, along with two novel trajectory prediction models: TrajGATFormer and TrajGATFormer-Obstacle. YOLOv10n serves as the backbone for object detection, accurately identifying workers and obstacles in diverse scenes, while DeepSORT efficiently tracks them over time with unique IDs for continuity. Both models employ a transformer encoder-decoder with Graph Attention Networks (GAT) to capture temporal and spatial interactions. TrajGATFormer predicts worker trajectories with an ADE of 1.25 m and FDE of 2.3 m over a 4.8 s horizon, while TrajGATFormer-Obstacle extends prediction to both workers and obstacles, achieving higher accuracy (ADE 1.15 m, FDE 2.2 m). Comparative analysis shows both models outperform traditional methods, reducing ADE and FDE by up to 35% and 38%, respectively.

Method: The approach uses a sparse voxel spectrum to represent tree movement, generates mesh motion from 3D Gaussian Splatting trees, binds Gaussians to deform the mesh, and enables fast modal analysis for real-time interactive responses under external forces.

Result: The method achieves realistic and responsive tree animations, significantly outperforming existing approaches in both visual quality and computational efficiency. A large-scale synthetic 4D tree dataset (4DTree) with 8,786 animated tree meshes was also created for training.

Conclusion: DynamicTree successfully generates long-term, interactive animation of 3D Gaussian Splatting trees in a fast feed-forward manner, representing a significant advancement over prior optimization-based methods.

Abstract: Generating dynamic and interactive 3D objects, such as trees, has wide applications in virtual reality, games, and world simulation. Nevertheless, existing methods still face various challenges in generating realistic 4D motion for complex real trees. In this paper, we propose DynamicTree, the first framework that can generate long-term, interactive animation of 3D Gaussian Splatting trees. Unlike prior optimization-based methods, our approach generates dynamics in a fast feed-forward manner. The key success of our approach is the use of a compact sparse voxel spectrum to represent the tree movement. Given a 3D tree from Gaussian Splatting reconstruction, our pipeline first generates mesh motion using the sparse voxel spectrum and then binds Gaussians to deform the mesh. Additionally, the proposed sparse voxel spectrum can also serve as a basis for fast modal analysis under external forces, allowing real-time interactive responses. To train our model, we also introduce 4DTree, the first large-scale synthetic 4D tree dataset containing 8,786 animated tree meshes with semantic labels and 100-frame motion sequences. Extensive experiments demonstrate that our method achieves realistic and responsive tree animations, significantly outperforming existing approaches in both visual quality and computational efficiency.

Method: GALA strategy combining global k-means clustering for target samples with cluster-wise local selection criterion to handle inter-class diversity and multi-source domain variation.

Result: Outperforms prior active learning and active DA methods on three standard benchmarks, achieving performance comparable to fully-supervised upperbound using only 1% target annotations.

Conclusion: GALA is an effective plug-and-play solution for Multi-Source Active Domain Adaptation that significantly reduces annotation costs while maintaining high performance.

Abstract: Domain Adaptation (DA) provides an effective way to tackle target-domain tasks by leveraging knowledge learned from source domains. Recent studies have extended this paradigm to Multi-Source Domain Adaptation (MSDA), which exploits multiple source domains carrying richer and more diverse transferable information. However, a substantial performance gap still remains between adaptation-based methods and fully supervised learning. In this paper, we explore a more practical and challenging setting, named Multi-Source Active Domain Adaptation (MS-ADA), to further enhance target-domain performance by selectively acquiring annotations from the target domain. The key difficulty of MS-ADA lies in designing selection criteria that can jointly handle inter-class diversity and multi-source domain variation. To address these challenges, we propose a simple yet effective GALA strategy (GALA), which combines a global k-means clustering step for target-domain samples with a cluster-wise local selection criterion, effectively tackling the above two issues in a complementary manner. Our proposed GALA is plug-and-play and can be seamlessly integrated into existing DA frameworks without introducing any additional trainable parameters. Extensive experiments on three standard DA benchmarks demonstrate that GALA consistently outperforms prior active learning and active DA methods, achieving performance comparable to the fully-supervised upperbound while using only 1% of the target annotations.

Method: Proposes a unified distribution strategy (UniPAN) that constructs a distribution transformation function to normalize pixels from different sources to an identical distribution, training models on the transformed domain and applying the same transformation during testing.

Result: Extensive experiments demonstrate UniPAN’s efficacy in significantly enhancing the performance of deep pansharpening models across diverse satellite sensors.

Conclusion: UniPAN successfully bridges the gap between training and testing distributions, enabling pansharpening models to achieve better generalizability and maintain performance across different satellite data sources.

Abstract: Existing deep learning-based models for remote sensing pansharpening exhibit exceptional performance on training datasets. However, due to sensor-specific characteristics and varying imaging conditions, these models suffer from substantial performance degradation when applied to unseen satellite data, lacking generalizability and thus limiting their applicability. We argue that the performance drops stem primarily from distributional discrepancies from different sources and the key to addressing this challenge lies in bridging the gap between training and testing distributions. To validate the idea and further achieve a “train once, deploy forever” capability, this paper introduces a novel and intuitive approach to enpower any pansharpening models with generalizability by employing a unified distribution strategy (UniPAN). Specifically, we construct a distribution transformation function that normalizes the pixels sampled from different sources to conform to an identical distribution. The deep models are trained on the transformed domain, and during testing on new datasets, the new data are also transformed to match the training distribution. UniPAN aims to train and test the model on a unified and consistent distribution, thereby enhancing its generalizability. Extensive experiments validate the efficacy of UniPAN, demonstrating its potential to significantly enhance the performance of deep pansharpening models across diverse satellite sensors. Codes: https://github.com/yc-cui/UniPAN.

Method: Uses voice as physiological signal with frequency-time dual domain multimodal characteristics and deep learning models for depression assessment algorithm.

Result: The proposed method achieves excellent performance in depression diagnosis classification tasks.

Conclusion: Provides new insights and approaches for depression assessment, screening, and diagnosis using voice-based intelligent systems.

Abstract: Depression, as a typical mental disorder, has become a prevalent issue significantly impacting public health. However, the prevention and treatment of depression still face multiple challenges, including complex diagnostic procedures, ambiguous criteria, and low consultation rates, which severely hinder timely assessment and intervention. To address these issues, this study adopts voice as a physiological signal and leverages its frequency-time dual domain multimodal characteristics along with deep learning models to develop an intelligent assessment and diagnostic algorithm for depression. Experimental results demonstrate that the proposed method achieves excellent performance in the classification task for depression diagnosis, offering new insights and approaches for the assessment, screening, and diagnosis of depression.

Method: Two-stage selection pipeline: 1) Hierarchical representation-based candidate selection using MaxHerding to choose representative pixels per image and create diverse global pool, 2) Entropy-augmented disagreement score (eDALD) over multi-scale diffusion features to select most informative pixels, decoupling diversity and uncertainty.

Result: Extensive experiments on four benchmarks (CamVid, ADE-Bed, Cityscapes, Pascal-Context) show significant performance improvements over existing baselines under extreme pixel-budget regimes.

Conclusion: The proposed method effectively addresses low-budget active learning for semantic segmentation by leveraging diffusion model features and a two-stage selection approach that separates diversity and uncertainty, achieving high accuracy with minimal labeled pixels.

Abstract: Semantic segmentation demands dense pixel-level annotations, which can be prohibitively expensive - especially under extremely constrained labeling budgets. In this paper, we address the problem of low-budget active learning for semantic segmentation by proposing a novel two-stage selection pipeline. Our approach leverages a pre-trained diffusion model to extract rich multi-scale features that capture both global structure and fine details. In the first stage, we perform a hierarchical, representation-based candidate selection by first choosing a small subset of representative pixels per image using MaxHerding, and then refining these into a diverse global pool. In the second stage, we compute an entropy-augmented disagreement score (eDALD) over noisy multi-scale diffusion features to capture both epistemic uncertainty and prediction confidence, selecting the most informative pixels for annotation. This decoupling of diversity and uncertainty lets us achieve high segmentation accuracy with only a tiny fraction of labeled pixels. Extensive experiments on four benchmarks (CamVid, ADE-Bed, Cityscapes, and Pascal-Context) demonstrate that our method significantly outperforms existing baselines under extreme pixel-budget regimes. Our code is available at https://github.com/jn-kim/two-stage-edald.

Method: Uses Gaussian noise on ground truth lane parameters to create noisy anchors, then learns progressive refinement. Introduces hybrid decoding strategy with global and local decoders, and employs auxiliary head for encoder supervision.

Result: Achieves strong generalization and superior performance: 1%+ accuracy improvement on Carlane, 81.32% F1 on CULane, 96.89% accuracy on Tusimple, and 97.59% F1 on LLAMAS.

Conclusion: DiffusionLane demonstrates effective application of diffusion models to lane detection with robust performance across multiple datasets and network backbones.

Abstract: In this paper, we present a novel diffusion-based model for lane detection, called DiffusionLane, which treats the lane detection task as a denoising diffusion process in the parameter space of the lane. Firstly, we add the Gaussian noise to the parameters (the starting point and the angle) of ground truth lanes to obtain noisy lane anchors, and the model learns to refine the noisy lane anchors in a progressive way to obtain the target lanes. Secondly, we propose a hybrid decoding strategy to address the poor feature representation of the encoder, resulting from the noisy lane anchors. Specifically, we design a hybrid diffusion decoder to combine global-level and local-level decoders for high-quality lane anchors. Then, to improve the feature representation of the encoder, we employ an auxiliary head in the training stage to adopt the learnable lane anchors for enriching the supervision on the encoder. Experimental results on four benchmarks, Carlane, Tusimple, CULane, and LLAMAS, show that DiffusionLane possesses a strong generalization ability and promising detection performance compared to the previous state-of-the-art methods. For example, DiffusionLane with ResNet18 surpasses the existing methods by at least 1% accuracy on the domain adaptation dataset Carlane. Besides, DiffusionLane with MobileNetV4 gets 81.32% F1 score on CULane, 96.89% accuracy on Tusimple with ResNet34, and 97.59% F1 score on LLAMAS with ResNet101. Code will be available at https://github.com/zkyntu/UnLanedet.

Method: Used FPGA implementation with LMIINet architecture and CGRA4ML framework, applied Quantization-Aware Training with 8-bit precision, simplified skip connections, employed hardware-friendly operations (depthwise-separable and 1A-1 convolutions), and redesigned Flatten Transformer components.

Result: Achieved approximately 90% pixel accuracy and 45% mean Intersection-over-Union (mIoU), operating at 20 frames per second with 50.1 ms latency on ZCU104 FPGA board, with 4x memory footprint reduction.

Conclusion: CGRA4ML provides a viable path for implementing advanced semantic segmentation networks on FPGA, offering superior power efficiency compared to traditional GPU solutions while maintaining competitive accuracy for real-time applications.

Abstract: Semantic segmentation has emerged as a fundamental problem in computer vision, gaining particular importance in real-time applications such as autonomous driving. The main challenge is achieving high accuracy while operating under computational and hardware constraints. In this research, we present an FPGA-based implementation of real-time semantic segmentation leveraging the lightweight LMIINet architecture and the Coarse-Grained Reconfigurable Array for Machine Learning (CGRA4ML) hardware framework. The model was trained using Quantization-Aware Training (QAT) with 8-bit precision on the Cityscapes dataset, reducing memory footprint by a factor of four while enabling efficient fixed-point computations. Necessary modifications were applied to adapt the model to CGRA4ML constraints, including simplifying skip connections, employing hardware-friendly operations such as depthwise-separable and 1A-1 convolutions, and redesigning parts of the Flatten Transformer. Our implementation achieves approximately 90% pixel accuracy and 45% mean Intersection-over-Union (mIoU), operating in real-time at 20 frames per second (FPS) with 50.1 ms latency on the ZCU104 FPGA board. The results demonstrate the potential of CGRA4ML, with its flexibility in mapping modern layers and off-chip memory utilization for skip connections, provides a path for implementing advanced semantic segmentation networks on FPGA for real-time applications to outperform traditional GPU solutions in terms of power efficiency while maintaining competitive accuracy. The code for this project is publicly available at https://github.com/STAmirr/ cgra4ml_semantic_segmentation

Method: Uses a snippet-level encoder for spatial-temporal modeling and a Transformer-based temporal decoder with dedicated temporal queries to predict accident scores for multiple future horizons simultaneously.

Result: Achieves superior performance in both recall and Time-to-Accident (TTA) under realistic false alarm rate (FAR) constraints.

Conclusion: The proposed paradigm provides more reliable accident anticipation by directly estimating future accident scores with precise supervision, outperforming existing methods while maintaining practical relevance through refined evaluation protocols.

Abstract: Accident anticipation aims to predict potential collisions in an online manner, enabling timely alerts to enhance road safety. Existing methods typically predict frame-level risk scores as indicators of hazard. However, these approaches rely on ambiguous binary supervision (labeling all frames in accident videos as positive) despite the fact that risk varies continuously over time, leading to unreliable learning and false alarms. To address this, we propose a novel paradigm that shifts the prediction target from current-frame risk scoring to directly estimating accident scores at multiple future time steps (e.g., 0.1s-2.0s ahead), leveraging precisely annotated accident timestamps as supervision. Our method employs a snippet-level encoder to jointly model spatial and temporal dynamics, and a Transformer-based temporal decoder that predicts accident scores for all future horizons simultaneously using dedicated temporal queries. Furthermore, we introduce a refined evaluation protocol that reports Time-to-Accident (TTA) and recall (evaluated at multiple pre-accident intervals (0.5s, 1.0s, and 1.5s)) only when the false alarm rate (FAR) remains within an acceptable range, ensuring practical relevance. Experiments show that our method achieves superior performance in both recall and TTA under realistic FAR constraints.

Method: Proposes GSAlign with two key components: Learnable Thin Plate Spline (LTPS) Module for geometric warping of pedestrian features, and Dynamic Alignment Module (DAM) for visibility-aware semantic alignment.

Result: Achieves significant improvements of +18.8% in mAP and +16.8% in Rank-1 accuracy over previous state-of-the-art methods on the CARGO dataset with four matching protocols.

Conclusion: GSAlign effectively addresses the challenges of aerial-ground person re-identification by jointly tackling geometric distortion and semantic misalignment, demonstrating superior performance over existing approaches.

Abstract: Aerial-Ground person re-identification (AG-ReID) is an emerging yet challenging task that aims to match pedestrian images captured from drastically different viewpoints, typically from unmanned aerial vehicles (UAVs) and ground-based surveillance cameras. The task poses significant challenges due to extreme viewpoint discrepancies, occlusions, and domain gaps between aerial and ground imagery. While prior works have made progress by learning cross-view representations, they remain limited in handling severe pose variations and spatial misalignment. To address these issues, we propose a Geometric and Semantic Alignment Network (GSAlign) tailored for AG-ReID. GSAlign introduces two key components to jointly tackle geometric distortion and semantic misalignment in aerial-ground matching: a Learnable Thin Plate Spline (LTPS) Module and a Dynamic Alignment Module (DAM). The LTPS module adaptively warps pedestrian features based on a set of learned keypoints, effectively compensating for geometric variations caused by extreme viewpoint changes. In parallel, the DAM estimates visibility-aware representation masks that highlight visible body regions at the semantic level, thereby alleviating the negative impact of occlusions and partial observations in cross-view correspondence. A comprehensive evaluation on CARGO with four matching protocols demonstrates the effectiveness of GSAlign, achieving significant improvements of +18.8% in mAP and +16.8% in Rank-1 accuracy over previous state-of-the-art methods on the aerial-ground setting. The code is available at: \textcolor{magenta}{https://github.com/stone96123/GSAlign}.

Method: Collected data from Common Crawl using filtering and deduplication techniques, created WAON-Bench benchmark with 374 Japanese cultural classes, and fine-tuned SigLIP2 model on WAON vs ReLAION datasets.

Result: WAON-trained models achieved higher accuracy across all benchmarks, enhanced performance on WAON-Bench more efficiently than ReLAION, and achieved state-of-the-art performance on several Japanese cultural benchmarks.

Conclusion: WAON dataset effectively improves VLM performance for Japanese cultural understanding and is publicly available for research use.

Abstract: Large-scale and high-quality image-text pair datasets play an important role in developing high-performing Vision-Language Models (VLMs). In this work, we introduce WAON, a large-scale and high-quality Japanese image-text pair dataset containing approximately 155 million examples, collected from Common Crawl. Our dataset construction pipeline employs various techniques, including filtering and deduplication, which have been shown to be effective in previous studies. To evaluate its effectiveness, we also construct WAON-Bench, a manually curated benchmark for Japanese cultural image classification, consisting of 374 classes. To assess the effectiveness of our dataset, we conduct experiments using both WAON and the Japanese subset of ReLAION, one of the most widely used vision-language datasets. We fine-tune SigLIP2, a strong multilingual model, on both datasets. The results demonstrate that WAON enhances model performance on WAON-Bench more efficiently than ReLAION and achieves higher accuracy across all evaluated benchmarks. Furthermore, the model fine-tuned on WAON achieves state-of-the-art performance on several Japanese cultural benchmarks. We release our dataset, model, and code at https://speed1313.github.io/WAON.

Method: Pure reinforcement learning framework with carefully curated multi-modal data and verifiable reward design to guide LVLMs to focus on semantically meaningful visual cues for structured reasoning.

Result: Significantly outperforms existing baselines, improving both prediction accuracy and generalization across diverse urban contexts, especially for unseen cities and indicators.

Conclusion: Combining RL and LVLMs shows promise for interpretable and generalist urban socio-economic sensing, enabling better understanding of visual data for sustainable development applications.

Abstract: Harnessing publicly available, large-scale web data, such as street view and satellite imagery, urban socio-economic sensing is of paramount importance for achieving global sustainable development goals. With the emergence of Large Vision-Language Models (LVLMs), new opportunities have arisen to solve this task by treating it as a multi-modal perception and understanding problem. However, recent studies reveal that LVLMs still struggle with accurate and interpretable socio-economic predictions from visual data. To address these limitations and maximize the potential of LVLMs, we introduce \textbf{CityRiSE}, a novel framework for \textbf{R}eason\textbf{i}ng urban \textbf{S}ocio-\textbf{E}conomic status in LVLMs through pure reinforcement learning (RL). With carefully curated multi-modal data and verifiable reward design, our approach guides the LVLM to focus on semantically meaningful visual cues, enabling structured and goal-oriented reasoning for generalist socio-economic status prediction. Experiments demonstrate that CityRiSE with emergent reasoning process significantly outperforms existing baselines, improving both prediction accuracy and generalization across diverse urban contexts, particularly for prediction on unseen cities and unseen indicators. This work highlights the promise of combining RL and LVLMs for interpretable and generalist urban socio-economic sensing.

Method: GRPO-Guard introduces ratio normalization to restore balanced importance ratios and a gradient reweighting strategy to equalize policy gradients over noise conditions, acting as a regulated clipping mechanism without heavy KL regularization.

Result: Extensive experiments on multiple diffusion backbones (SD3.5M, Flux.1-dev) and diverse proxy tasks show GRPO-Guard significantly reduces over-optimization while maintaining or improving generation quality.

Conclusion: GRPO-Guard provides a simple yet effective enhancement to GRPO frameworks that stabilizes optimization and mitigates implicit over-optimization through proper importance-ratio management and gradient balancing.

Abstract: Recently, GRPO-based reinforcement learning has shown remarkable progress in optimizing flow-matching models, effectively improving their alignment with task-specific rewards. Within these frameworks, the policy update relies on importance-ratio clipping to constrain overconfident positive and negative gradients. However, in practice, we observe a systematic shift in the importance-ratio distribution-its mean falls below 1 and its variance differs substantially across timesteps. This left-shifted and inconsistent distribution prevents positive-advantage samples from entering the clipped region, causing the mechanism to fail in constraining overconfident positive updates. As a result, the policy model inevitably enters an implicit over-optimization stage-while the proxy reward continues to increase, essential metrics such as image quality and text-prompt alignment deteriorate sharply, ultimately making the learned policy impractical for real-world use. To address this issue, we introduce GRPO-Guard, a simple yet effective enhancement to existing GRPO frameworks. Our method incorporates ratio normalization, which restores a balanced and step-consistent importance ratio, ensuring that PPO clipping properly constrains harmful updates across denoising timesteps. In addition, a gradient reweighting strategy equalizes policy gradients over noise conditions, preventing excessive updates from particular timestep regions. Together, these designs act as a regulated clipping mechanism, stabilizing optimization and substantially mitigating implicit over-optimization without relying on heavy KL regularization. Extensive experiments on multiple diffusion backbones (e.g., SD3.5M, Flux.1-dev) and diverse proxy tasks demonstrate that GRPO-Guard significantly reduces over-optimization while maintaining or even improving generation quality.

Method: Constructs KNN graphs for teacher and student streams during pretraining, then uses GNNs for representation refinement through multi-hop message propagation to capture broader contextual relationships.

Result: Achieved accuracy improvements of 7.3% on CIFAR-10, 3.2% on ImageNet-100, and 1.0% on ImageNet-1K over state-of-the-art methods.

Conclusion: The graph-based mechanism effectively enhances self-supervised representation learning by capturing both intra-instance and inter-instance relationships, demonstrating superior performance across multiple datasets.

Abstract: This paper introduces a novel approach that integrates graph theory into self-supervised representation learning. Traditional methods focus on intra-instance variations generated by applying augmentations. However, they often overlook important inter-instance relationships. While our method retains the intra-instance property, it further captures inter-instance relationships by constructing k-nearest neighbor (KNN) graphs for both teacher and student streams during pretraining. In these graphs, nodes represent samples along with their latent representations. Edges encode the similarity between instances. Following pretraining, a representation refinement phase is performed. In this phase, Graph Neural Networks (GNNs) propagate messages not only among immediate neighbors but also across multiple hops, thereby enabling broader contextual integration. Experimental results on CIFAR-10, ImageNet-100, and ImageNet-1K demonstrate accuracy improvements of 7.3%, 3.2%, and 1.0%, respectively, over state-of-the-art methods. These results highlight the effectiveness of the proposed graph based mechanism. The code is publicly available at https://github.com/alijavidani/SSL-GraphNNCLR.

Method: Proposes a coarse-to-fine framework with three components: Hierarchical fMRI Encoder for multi-level embeddings, Hierarchy-to-Hierarchy Alignment with CLIP features, and Scale-Aware Neural Guidance for injecting embeddings at matching scales during autoregression.

Result: Achieves superior semantic fidelity, 4.67x faster inference, and more deterministic results than diffusion-based baselines on the NSD dataset.

Conclusion: MindHier provides an efficient and cognitively-aligned alternative to diffusion methods by enabling hierarchical reconstruction that mimics human visual perception.

Abstract: Reconstructing visual stimuli from fMRI signals is a central challenge bridging machine learning and neuroscience. Recent diffusion-based methods typically map fMRI activity to a single high-level embedding, using it as fixed guidance throughout the entire generation process. However, this fixed guidance collapses hierarchical neural information and is misaligned with the stage-dependent demands of image reconstruction. In response, we propose MindHier, a coarse-to-fine fMRI-to-image reconstruction framework built on scale-wise autoregressive modeling. MindHier introduces three components: a Hierarchical fMRI Encoder to extract multi-level neural embeddings, a Hierarchy-to-Hierarchy Alignment scheme to enforce layer-wise correspondence with CLIP features, and a Scale-Aware Coarse-to-Fine Neural Guidance strategy to inject these embeddings into autoregression at matching scales. These designs make MindHier an efficient and cognitively-aligned alternative to diffusion-based methods by enabling a hierarchical reconstruction process that synthesizes global semantics before refining local details, akin to human visual perception. Extensive experiments on the NSD dataset show that MindHier achieves superior semantic fidelity, 4.67x faster inference, and more deterministic results than the diffusion-based baselines.

Method: Uses class-specific 3D objects as geometric priors, ensures viewpoint consistency through rendered reference images, and employs GeoDrag for drag-based editing.

Result: Enables precise geometric modifications across various iterative design workflows with improved accuracy and speed on geometry guidance tasks.

Conclusion: GeoDiffusion provides accurate and efficient geometric conditioning for image generation without requiring training.

Abstract: Precise geometric control in image generation is essential for engineering & product design and creative industries to control 3D object features accurately in image space. Traditional 3D editing approaches are time-consuming and demand specialized skills, while current image-based generative methods lack accuracy in geometric conditioning. To address these challenges, we propose GeoDiffusion, a training-free framework for accurate and efficient geometric conditioning of 3D features in image generation. GeoDiffusion employs a class-specific 3D object as a geometric prior to define keypoints and parametric correlations in 3D space. We ensure viewpoint consistency through a rendered image of a reference 3D object, followed by style transfer to meet user-defined appearance specifications. At the core of our framework is GeoDrag, improving accuracy and speed of drag-based image editing on geometry guidance tasks and general instructions on DragBench. Our results demonstrate that GeoDiffusion enables precise geometric modifications across various iterative design workflows.

Method: Adapted Depth Anything V2 model for joint depth, pose, and intrinsics prediction, incorporating attention-based pose network and Weight-Decomposed Low-Rank Adaptation (DoRA) strategy for efficient fine-tuning.

Result: Superior performance on SCARED and C3VD datasets compared to state-of-the-art approaches in self-supervised monocular depth estimation and 3D reconstruction.

Conclusion: The proposed method successfully integrates intrinsic parameter estimation into endoscopic 3D reconstruction, addressing key challenges in surgical settings and demonstrating improved accuracy and reliability.

Abstract: 3D reconstruction of endoscopic surgery scenes plays a vital role in enhancing scene perception, enabling AR visualization, and supporting context-aware decision-making in image-guided surgery. A critical yet challenging step in this process is the accurate estimation of the endoscope’s intrinsic parameters. In real surgical settings, intrinsic calibration is hindered by sterility constraints and the use of specialized endoscopes with continuous zoom and telescope rotation. Most existing methods for endoscopic 3D reconstruction do not estimate intrinsic parameters, limiting their effectiveness for accurate and reliable reconstruction. In this paper, we integrate intrinsic parameter estimation into a self-supervised monocular depth estimation framework by adapting the Depth Anything V2 (DA2) model for joint depth, pose, and intrinsics prediction. We introduce an attention-based pose network and a Weight-Decomposed Low-Rank Adaptation (DoRA) strategy for efficient fine-tuning of DA2. Our method is validated on the SCARED and C3VD public datasets, demonstrating superior performance compared to recent state-of-the-art approaches in self-supervised monocular depth estimation and 3D reconstruction. Code and model weights can be found in project repository: https://github.com/MOYF-beta/EndoSfM3D.

Method: Proposes Tail-Truncated Sampling (TTS) that embeds bits exclusively in reliable tail regions while randomly sampling the central zone, plus a two-stage framework with session keys for encryption.

Result: Achieves optimal balance between robustness and diversity when evaluated on diffusion models with both U-Net and DiT backbones.

Conclusion: T2SMark effectively addresses the robustness-diversity tradeoff in diffusion model watermarking through its novel sampling approach and two-stage framework.

Abstract: Diffusion models have advanced rapidly in recent years, producing high-fidelity images while raising concerns about intellectual property protection and the misuse of generative AI. Image watermarking for diffusion models, particularly Noise-as-Watermark (NaW) methods, encode watermark as specific standard Gaussian noise vector for image generation, embedding the infomation seamlessly while maintaining image quality. For detection, the generation process is inverted to recover the initial noise vector containing the watermark before extraction. However, existing NaW methods struggle to balance watermark robustness with generation diversity. Some methods achieve strong robustness by heavily constraining initial noise sampling, which degrades user experience, while others preserve diversity but prove too fragile for real-world deployment. To address this issue, we propose T2SMark, a two-stage watermarking scheme based on Tail-Truncated Sampling (TTS). Unlike prior methods that simply map bits to positive or negative values, TTS enhances robustness by embedding bits exclusively in the reliable tail regions while randomly sampling the central zone to preserve the latent distribution. Our two-stage framework then ensures sampling diversity by integrating a randomly generated session key into both encryption pipelines. We evaluate T2SMark on diffusion models with both U-Net and DiT backbones. Extensive experiments show that it achieves an optimal balance between robustness and diversity. Our code is available at \href{https://github.com/0xD009/T2SMark}{https://github.com/0xD009/T2SMark}.

Method: Proposes a Recurrent Correlation-based framework that performs local matching at each step and dynamically relocates the search region based on estimated offsets. Uses a lightweight recurrent update module with memory capacity and decouples motion-related and texture features to reduce semantic redundancy.

Result: Achieves strong accuracy-computation trade-off, surpassing or matching state-of-the-art performance. On the non-affine OASIS dataset, achieves comparable performance using only 9.5% of FLOPs and running 96% faster than RDP method. Validated on brain MRI and abdominal CT datasets with and without affine pre-registration.

Conclusion: The proposed recurrent correlation framework efficiently handles large deformations in medical image registration by dynamically relocating matching regions, achieving excellent performance with significantly reduced computational requirements compared to existing methods.

Abstract: Deformable image registration estimates voxel-wise correspondences between images through spatial transformations, and plays a key role in medical imaging. While deep learning methods have significantly reduced runtime, efficiently handling large deformations remains a challenging task. Convolutional networks aggregate local features but lack direct modeling of voxel correspondences, promoting recent works to explore explicit feature matching. Among them, voxel-to-region matching is more efficient for direct correspondence modeling by computing local correlation features whithin neighbourhoods, while region-to-region matching incurs higher redundancy due to excessive correlation pairs across large regions. However, the inherent locality of voxel-to-region matching hinders the capture of long-range correspondences required for large deformations. To address this, we propose a Recurrent Correlation-based framework that dynamically relocates the matching region toward more promising positions. At each step, local matching is performed with low cost, and the estimated offset guides the next search region, supporting efficient convergence toward large deformations. In addition, we uses a lightweight recurrent update module with memory capacity and decouples motion-related and texture features to suppress semantic redundancy. We conduct extensive experiments on brain MRI and abdominal CT datasets under two settings: with and without affine pre-registration. Results show that our method exibits a strong accuracy-computation trade-off, surpassing or matching the state-of-the-art performance. For example, it achieves comparable performance on the non-affine OASIS dataset, while using only 9.5% of the FLOPs and running 96% faster than RDP, a representative high-performing method.

Method: Uses Gaussian distribution grid (GDG) to model spatial statistics of background from background-only scans, combined with a filtering algorithm for point classification. Supports diverse LiDAR types including multiline 360 degree and MEMS sensors.

Result: Outperforms state-of-the-art techniques in accuracy and flexibility on RCooper dataset, works well with minimal background data, and runs efficiently on low-resource hardware.

Conclusion: The method enables scalable real-world deployment for automated driving infrastructure with reliable performance and broad sensor compatibility.

Abstract: We present a fully interpretable and flexible statistical method for background subtraction in roadside LiDAR data, aimed at enhancing infrastructure-based perception in automated driving. Our approach introduces both a Gaussian distribution grid (GDG), which models the spatial statistics of the background using background-only scans, and a filtering algorithm that uses this representation to classify LiDAR points as foreground or background. The method supports diverse LiDAR types, including multiline 360 degree and micro-electro-mechanical systems (MEMS) sensors, and adapts to various configurations. Evaluated on the publicly available RCooper dataset, it outperforms state-of-the-art techniques in accuracy and flexibility, even with minimal background data. Its efficient implementation ensures reliable performance on low-resource hardware, enabling scalable real-world deployment.

Method: Proposes TDSR framework that models captioning as MDP and uses efficient MCTS with visual-guided parallel expansion and lightweight value network to reduce VLM calls.

Result: Achieves state-of-the-art results on DetailCaps, COMPOSITIONCAP, and POPE benchmarks, significantly enhancing existing VLMs with minimal computational overhead.

Conclusion: TDSR effectively addresses the coherence-detail tradeoff in VLMs through hierarchical planning and efficient search, demonstrating strong performance across multiple captioning tasks.

Abstract: Large Vision-Language Models (VLMs) face an inherent contradiction in image captioning: their powerful single-step generation capabilities often lead to a myopic decision-making process. This makes it difficult to maintain global narrative coherence while capturing rich details, a limitation that is particularly pronounced in tasks that require multi-step and complex scene description. To overcome this fundamental challenge, we redefine image captioning as a goal-oriented hierarchical refinement planning problem, and further propose a novel framework, named Top-Down Semantic Refinement (TDSR), which models the generation process as a Markov Decision Process (MDP). However, planning within the vast state space of a VLM presents a significant computational hurdle. Our core contribution, therefore, is the design of a highly efficient Monte Carlo Tree Search (MCTS) algorithm tailored for VLMs. By incorporating a visual-guided parallel expansion and a lightweight value network, our TDSR reduces the call frequency to the expensive VLM by an order of magnitude without sacrificing planning quality. Furthermore, an adaptive early stopping mechanism dynamically matches computational overhead to the image’s complexity. Extensive experiments on multiple benchmarks, including DetailCaps, COMPOSITIONCAP, and POPE, demonstrate that our TDSR, as a plug-and-play module, can significantly enhance the performance of existing VLMs (e.g., LLaVA-1.5, Qwen2.5-VL) by achieving state-of-the-art or highly competitive results in fine-grained description, compositional generalization, and hallucination suppression.

Method: Developed a physics-based model to examine LiDAR failure modes in snow, investigating signal attenuation with different snowfall rates and snow particle reflection effects. Used the model to transform clear-weather data into synthetic snowy scenarios for comparison with real snowy conditions.

Result: The study successfully created synthetic data representing various snowfall rates and demonstrated how snow particles near the LiDAR source act as efficient reflectors. This enabled systematic analysis of LiDAR performance degradation under different snow conditions.

Conclusion: The physics-based model effectively simulates LiDAR performance in snowy conditions, providing a framework to quantify signal degradation and assess its impact on object detection models, which is crucial for improving autonomous driving system reliability in adverse weather.

Abstract: Because 3D structure of a roadway environment can be characterized directly by a Light Detection and Ranging (LiDAR) sensors, they can be used to obtain exceptional situational awareness for assitive and autonomous driving systems. Although LiDARs demonstrate good performance in clean and clear weather conditions, their performance significantly deteriorates in adverse weather conditions such as those involving atmospheric precipitation. This may render perception capabilities of autonomous systems that use LiDAR data in learning based models to perform object detection and ranging ineffective. While efforts have been made to enhance the accuracy of these models, the extent of signal degradation under various weather conditions remains largely not quantified. In this study, we focus on the performance of an automotive grade LiDAR in snowy conditions in order to develop a physics-based model that examines failure modes of a LiDAR sensor. Specifically, we investigated how the LiDAR signal attenuates with different snowfall rates and how snow particles near the source serve as small but efficient reflectors. Utilizing our model, we transform data from clear conditions to simulate snowy scenarios, enabling a comparison of our synthetic data with actual snowy conditions. Furthermore, we employ this synthetic data, representative of different snowfall rates, to explore the impact on a pre-trained object detection model, assessing its performance under varying levels of snowfall

Method: Created WAGIBench with 29 hours of multimodal data from 348 participants across 3,477 recordings, featuring ground-truth goals with visual, audio, digital, and longitudinal contextual observations. Evaluated several families of modern vision-language models through multiple-choice and generative benchmarks.

Result: Human performance achieved 93% multiple-choice accuracy vs 84% for the best-performing VLM. Generative benchmarks show larger models perform better but produce relevant goals only 55% of the time. Modality ablation shows models benefit from relevant modalities with minimal degradation from irrelevant ones.

Conclusion: Current vision-language models remain far from practical usefulness for goal inference in assistive wearable agents, with significant room for improvement needed to approach human-level performance.

Abstract: There has been a surge of interest in assistive wearable agents: agents embodied in wearable form factors (e.g., smart glasses) who take assistive actions toward a user’s goal/query (e.g. “Where did I leave my keys?”). In this work, we consider the important complementary problem of inferring that goal from multi-modal contextual observations. Solving this “goal inference” problem holds the promise of eliminating the effort needed to interact with such an agent. This work focuses on creating WAGIBench, a strong benchmark to measure progress in solving this problem using vision-language models (VLMs). Given the limited prior work in this area, we collected a novel dataset comprising 29 hours of multimodal data from 348 participants across 3,477 recordings, featuring ground-truth goals alongside accompanying visual, audio, digital, and longitudinal contextual observations. We validate that human performance exceeds model performance, achieving 93% multiple-choice accuracy compared with 84% for the best-performing VLM. Generative benchmark results that evaluate several families of modern vision-language models show that larger models perform significantly better on the task, yet remain far from practical usefulness, as they produce relevant goals only 55% of the time. Through a modality ablation, we show that models benefit from extra information in relevant modalities with minimal performance degradation from irrelevant modalities.

Method: End-to-end heatmap-supervised detection model inspired by keypoint detection, teacher-student co-training, multi-view pseudo-labeling, and CryoET-specific DropBlock augmentation.

Result: Improves F1 score by 10% over supervised baselines on the CZII dataset.

Conclusion: Semi-supervised learning effectively leverages unlabeled CryoET data to overcome labeling bottlenecks.

Abstract: Cryogenic Electron Tomography (CryoET) combined with sub-volume averaging (SVA) is the only imaging modality capable of resolving protein structures inside cells at molecular resolution. Particle picking, the task of localizing and classifying target proteins in 3D CryoET volumes, remains the main bottleneck. Due to the reliance on time-consuming manual labels, the vast reserve of unlabeled tomograms remains underutilized. In this work, we present a fast, label-efficient semi-supervised framework that exploits this untapped data. Our framework consists of two components: (i) an end-to-end heatmap-supervised detection model inspired by keypoint detection, and (ii) a teacher-student co-training mechanism that enhances performance under sparse labeling conditions. Furthermore, we introduce multi-view pseudo-labeling and a CryoET-specific DropBlock augmentation strategy to further boost performance. Extensive evaluations on the large-scale CZII dataset show that our approach improves F1 by 10% over supervised baselines, underscoring the promise of semi-supervised learning for leveraging unlabeled CryoET data.

Method: The framework uses 3D Gaussian Splatting to construct 3D models from single images, then predicts multi-view pose keypoints based on one-shot support from a chosen view, leveraging supervisory signals to enhance motion consistency.

Result: Experimental results demonstrate that DynaPose4D achieves excellent coherence, consistency, and fluidity in dynamic motion generation.

Conclusion: The findings validate DynaPose4D’s efficacy and indicate its potential applications in computer vision and animation production domains.

Abstract: Recent advancements in 2D and 3D generative models have expanded the capabilities of computer vision. However, generating high-quality 4D dynamic content from a single static image remains a significant challenge. Traditional methods have limitations in modeling temporal dependencies and accurately capturing dynamic geometry changes, especially when considering variations in camera perspective. To address this issue, we propose DynaPose4D, an innovative solution that integrates 4D Gaussian Splatting (4DGS) techniques with Category-Agnostic Pose Estimation (CAPE) technology. This framework uses 3D Gaussian Splatting to construct a 3D model from single images, then predicts multi-view pose keypoints based on one-shot support from a chosen view, leveraging supervisory signals to enhance motion consistency. Experimental results show that DynaPose4D achieves excellent coherence, consistency, and fluidity in dynamic motion generation. These findings not only validate the efficacy of the DynaPose4D framework but also indicate its potential applications in the domains of computer vision and animation production.

Method: Attach multiple branches to a single teacher to generate diverse multi-views. Use two angular diversity objectives: constrained inter-angle diversify loss (maximizes angles between views while preserving proximity to original output) and intra-angle diversify loss (encourages even distribution of views around original output). Ensemble knowledge from these views for distillation.

Result: The method surpasses existing knowledge augmentation methods across diverse configurations. It’s compatible with other KD frameworks in plug-and-play fashion and provides consistent improvements in generalization performance.

Conclusion: The proposed angular diversity objectives effectively increase ensemble diversity and reduce the upper bound of expected loss, leading to more effective knowledge distillation from a single teacher.

Abstract: Knowledge Distillation (KD) aims to train a lightweight student model by transferring knowledge from a large, high-capacity teacher. Recent studies have shown that leveraging diverse teacher perspectives can significantly improve distillation performance; however, achieving such diversity typically requires multiple teacher networks, leading to high computational costs. In this work, we propose a novel cost-efficient knowledge augmentation method for KD that generates diverse multi-views by attaching multiple branches to a single teacher. To ensure meaningful semantic variation across multi-views, we introduce two angular diversity objectives: 1) constrained inter-angle diversify loss, which maximizes angles between augmented views while preserving proximity to the original teacher output, and 2) intra-angle diversify loss, which encourages an even distribution of views around the original output. The ensembled knowledge from these angularly diverse views, along with the original teacher, is distilled into the student. We further theoretically demonstrate that our objectives increase the diversity among ensemble members and thereby reduce the upper bound of the ensemble’s expected loss, leading to more effective distillation. Experimental results show that our method surpasses an existing knowledge augmentation method across diverse configurations. Moreover, the proposed method is compatible with other KD frameworks in a plug-and-play fashion, providing consistent improvements in generalization performance.

Method: Proposed GateFuseNet with adaptive 3D multimodal fusion using gated fusion modules that learn modality-specific attention weights and channel-wise gating vectors for selective feature modulation and hierarchical gating mechanism.

Result: Outperformed three state-of-the-art approaches with 85.00% accuracy and 92.06% AUC. Ablation studies validated ROI guidance, multimodal integration, and fusion positioning contributions. Grad-CAM confirmed focus on clinically relevant regions.

Conclusion: GateFuseNet effectively integrates QSM and T1w MRI for improved PD diagnosis through adaptive multimodal fusion, demonstrating superior performance and clinically relevant feature focus.

Abstract: Accurate diagnosis of Parkinson’s disease (PD) from MRI remains challenging due to symptom variability and pathological heterogeneity. Most existing methods rely on conventional magnitude-based MRI modalities, such as T1-weighted images (T1w), which are less sensitive to PD pathology than Quantitative Susceptibility Mapping (QSM), a phase-based MRI technique that quantifies iron deposition in deep gray matter nuclei. In this study, we propose GateFuseNet, an adaptive 3D multimodal fusion network that integrates QSM and T1w images for PD diagnosis. The core innovation lies in a gated fusion module that learns modality-specific attention weights and channel-wise gating vectors for selective feature modulation. This hierarchical gating mechanism enhances ROI-aware features while suppressing irrelevant signals. Experimental results show that our method outperforms three existing state-of-the-art approaches, achieving 85.00% accuracy and 92.06% AUC. Ablation studies further validate the contributions of ROI guidance, multimodal integration, and fusion positioning. Grad-CAM visualizations confirm the model’s focus on clinically relevant pathological regions. The source codes and pretrained models can be found at https://github.com/YangGaoUQ/GateFuseNet

Method: ORIG iteratively retrieves and filters multimodal evidence from the web and incrementally integrates refined knowledge into enriched prompts to guide generation for factual image generation.

Result: Experiments show ORIG substantially improves factual consistency and overall image quality over strong baselines, with evaluation conducted on FIG-Eval benchmark spanning ten categories across perceptual, compositional, and temporal dimensions.

Conclusion: ORIG demonstrates the potential of open multimodal retrieval for factual image generation, effectively bridging the gap between visual realism and factual grounding in generated images.

Abstract: Large Multimodal Models (LMMs) have achieved remarkable progress in generating photorealistic and prompt-aligned images, but they often produce outputs that contradict verifiable knowledge, especially when prompts involve fine-grained attributes or time-sensitive events. Conventional retrieval-augmented approaches attempt to address this issue by introducing external information, yet they are fundamentally incapable of grounding generation in accurate and evolving knowledge due to their reliance on static sources and shallow evidence integration. To bridge this gap, we introduce ORIG, an agentic open multimodal retrieval-augmented framework for Factual Image Generation (FIG), a new task that requires both visual realism and factual grounding. ORIG iteratively retrieves and filters multimodal evidence from the web and incrementally integrates the refined knowledge into enriched prompts to guide generation. To support systematic evaluation, we build FIG-Eval, a benchmark spanning ten categories across perceptual, compositional, and temporal dimensions. Experiments demonstrate that ORIG substantially improves factual consistency and overall image quality over strong baselines, highlighting the potential of open multimodal retrieval for factual image generation.

Method: AesCrop uses VMamba image encoder with novel Mamba Composition Attention Bias (MCAB) and transformer decoder for end-to-end rank-based image cropping, explicitly encoding compositional cues into attention mechanism.

Result: Extensive experiments show AesCrop outperforms current state-of-the-art methods, delivering superior quantitative metrics and qualitatively more pleasing crops.

Conclusion: The proposed AesCrop model successfully bridges the gap between evaluation-based and regression-based methods by incorporating explicit compositional guidance, achieving better diversity and globality in image cropping.

Abstract: Aesthetic-driven image cropping is crucial for applications like view recommendation and thumbnail generation, where visual appeal significantly impacts user engagement. A key factor in visual appeal is composition–the deliberate arrangement of elements within an image. Some methods have successfully incorporated compositional knowledge through evaluation-based and regression-based paradigms. However, evaluation-based methods lack globality while regression-based methods lack diversity. Recently, hybrid approaches that integrate both paradigms have emerged, bridging the gap between these two to achieve better diversity and globality. Notably, existing hybrid methods do not incorporate photographic composition guidance, a key attribute that defines photographic aesthetics. In this work, we introduce AesCrop, a composition-aware hybrid image-cropping model that integrates a VMamba image encoder, augmented with a novel Mamba Composition Attention Bias (MCAB) and a transformer decoder to perform end-to-end rank-based image cropping, generating multiple crops along with the corresponding quality scores. By explicitly encoding compositional cues into the attention mechanism, MCAB directs AesCrop to focus on the most compositionally salient regions. Extensive experiments demonstrate that AesCrop outperforms current state-of-the-art methods, delivering superior quantitative metrics and qualitatively more pleasing crops.

Method: Introduces word groups to capture spatial co-occurrence and proximity of visual words, constructs online dictionary, incorporates temporal consistency with adaptive scheme, adds feature distribution analysis and post-verification mechanisms.

Result: BoWG surpasses state-of-the-art methods in precision-recall and computational efficiency, achieving average processing time of 16 ms per image across 17,565 images in Bicocca25b dataset.

Conclusion: The method demonstrates superior performance in challenging environments, excellent scalability, and computational efficiency compared to both traditional and learning-based approaches.

Abstract: Loop closure is critical in Simultaneous Localization and Mapping (SLAM) systems to reduce accumulative drift and ensure global mapping consistency. However, conventional methods struggle in perceptually aliased environments, such as narrow pipes, due to vector quantization, feature sparsity, and repetitive textures, while existing solutions often incur high computational costs. This paper presents Bag-of-Word-Groups (BoWG), a novel loop closure detection method that achieves superior precision-recall, robustness, and computational efficiency. The core innovation lies in the introduction of word groups, which captures the spatial co-occurrence and proximity of visual words to construct an online dictionary. Additionally, drawing inspiration from probabilistic transition models, we incorporate temporal consistency directly into similarity computation with an adaptive scheme, substantially improving precision-recall performance. The method is further strengthened by a feature distribution analysis module and dedicated post-verification mechanisms. To evaluate the effectiveness of our method, we conduct experiments on both public datasets and a confined-pipe dataset we constructed. Results demonstrate that BoWG surpasses state-of-the-art methods, including both traditional and learning-based approaches, in terms of precision-recall and computational efficiency. Our approach also exhibits excellent scalability, achieving an average processing time of 16 ms per image across 17,565 images in the Bicocca25b dataset.

Method: Two core components: Spatially Re-focused Cross-Attention (SRCA) uses visually-grounded segmentation masks to guide cross-attention at inference time, and Spatially Targeted Classifier-Free Guidance (STCFG) selectively bypasses text influences on ungrounded pixels.

Result: Outperforms seven state-of-the-art baselines in fidelity metrics (PSNR and SSIM) across all datasets, and in perceptual quality measures (LPIPS and DISTS) on real-world benchmarks.

Conclusion: SRSR effectively achieves both high semantic fidelity and perceptual quality in super-resolution through its plug-and-play framework.

Abstract: Existing diffusion-based super-resolution approaches often exhibit semantic ambiguities due to inaccuracies and incompleteness in their text conditioning, coupled with the inherent tendency for cross-attention to divert towards irrelevant pixels. These limitations can lead to semantic misalignment and hallucinated details in the generated high-resolution outputs. To address these, we propose a novel, plug-and-play spatially re-focused super-resolution (SRSR) framework that consists of two core components: first, we introduce Spatially Re-focused Cross-Attention (SRCA), which refines text conditioning at inference time by applying visually-grounded segmentation masks to guide cross-attention. Second, we introduce a Spatially Targeted Classifier-Free Guidance (STCFG) mechanism that selectively bypasses text influences on ungrounded pixels to prevent hallucinations. Extensive experiments on both synthetic and real-world datasets demonstrate that SRSR consistently outperforms seven state-of-the-art baselines in standard fidelity metrics (PSNR and SSIM) across all datasets, and in perceptual quality measures (LPIPS and DISTS) on two real-world benchmarks, underscoring its effectiveness in achieving both high semantic fidelity and perceptual quality in super-resolution.

Method: Proposed MELDAE framework with three key contributions: first conversational-in-the-wild micro-expression dataset, end-to-end localization and detection framework, and novel boundary-aware loss function for improved temporal accuracy.

Result: Achieves state-of-the-art results on WDMD dataset with 17.72% improvement in F1_DR localization metric over strongest baseline, while showing excellent generalization on existing benchmarks.

Conclusion: The proposed framework effectively addresses micro-expression analysis in wild conversational scenarios through novel dataset, architecture, and loss function, demonstrating significant performance improvements and strong generalization capabilities.

Abstract: Accurately analyzing spontaneous, unconscious micro-expressions is crucial for revealing true human emotions, but this task remains challenging in wild scenarios, such as natural conversation. Existing research largely relies on datasets from controlled laboratory environments, and their performance degrades dramatically in the real world. To address this issue, we propose three contributions: the first micro-expression dataset focused on conversational-in-the-wild scenarios; an end-to-end localization and detection framework, MELDAE; and a novel boundary-aware loss function that improves temporal accuracy by penalizing onset and offset errors. Extensive experiments demonstrate that our framework achieves state-of-the-art results on the WDMD dataset, improving the key F1_{DR} localization metric by 17.72% over the strongest baseline, while also demonstrating excellent generalization capabilities on existing benchmarks.

Method: Three key contributions: XLFM-Zebrafish benchmark dataset, Masked View Modeling for Light Fields (self-supervised angular prior learning), and Optical Rendering Consistency Loss (differentiable rendering constraint).

Result: On the XLFM-Zebrafish benchmark, the method improves PSNR by 7.7% over state-of-the-art baselines.

Conclusion: The proposed approach successfully addresses core challenges in XLFM reconstruction through standardized benchmarking, self-supervised learning of angular priors, and physically-grounded rendering constraints.

Abstract: Light field microscopy (LFM) has become an emerging tool in neuroscience for large-scale neural imaging in vivo, notable for its single-exposure volumetric imaging, broad field of view, and high temporal resolution. However, learning-based 3D reconstruction in XLFM remains underdeveloped due to two core challenges: the absence of standardized datasets and the lack of methods that can efficiently model its angular-spatial structure while remaining physically grounded. We address these challenges by introducing three key contributions. First, we construct the XLFM-Zebrafish benchmark, a large-scale dataset and evaluation suite for XLFM reconstruction. Second, we propose Masked View Modeling for Light Fields (MVN-LF), a self-supervised task that learns angular priors by predicting occluded views, improving data efficiency. Third, we formulate the Optical Rendering Consistency Loss (ORC Loss), a differentiable rendering constraint that enforces alignment between predicted volumes and their PSF-based forward projections. On the XLFM-Zebrafish benchmark, our method improves PSNR by 7.7% over state-of-the-art baselines.

Method: Proposes PFED with two components: 1) Hierarchical Distillation with Uncertainty-Aware Prediction Alignment during training to distill essential information without inference overhead, and 2) Multi-view Refinement Module during inference to filter redundant samples using mutual information.

Result: Achieves 97.15% Recall@1 on University-1652 dataset, over 5x more efficient in FLOPs and 3x faster than previous methods, running at 251.5 FPS on AGX Orin edge device.

Conclusion: PFED demonstrates state-of-the-art performance in both accuracy and efficiency, making it practical for real-time UAV geo-localization in GNSS-denied environments.

Abstract: Cross-view geo-localization (CVGL) enables UAV localization by matching aerial images to geo-tagged satellite databases, which is critical for autonomous navigation in GNSS-denied environments. However, existing methods rely on resource-intensive fine-grained feature extraction and alignment, where multiple branches and modules significantly increase inference costs, limiting their deployment on edge devices. We propose Precision-Focused Efficient Design (PFED), a resource-efficient framework combining hierarchical knowledge transfer and multi-view representation refinement. This innovative method comprises two key components: 1) During training, Hierarchical Distillation paradigm for fast and accurate CVGL (HD-CVGL), coupled with Uncertainty-Aware Prediction Alignment (UAPA) to distill essential information and mitigate the data imbalance without incurring additional inference overhead. 2) During inference, an efficient Multi-view Refinement Module (MRM) leverages mutual information to filter redundant samples and effectively utilize the multi-view data. Extensive experiments show that PFED achieves state-of-the-art performance in both accuracy and efficiency, reaching 97.15% Recall@1 on University-1652 while being over $5 \times$ more efficient in FLOPs and $3 \times$ faster than previous top methods. Furthermore, PFED runs at 251.5 FPS on the AGX Orin edge device, demonstrating its practical viability for real-time UAV applications. The project is available at https://github.com/SkyEyeLoc/PFED

Method: Three-branch architecture with teacher and two student networks. Teacher generates pseudo labels from weakly augmented images. Students use LF-Dropout (randomly discards domain-related low-frequency components) and LF-Uncert (estimates uncertain domain variability via adversarial Gaussian perturbations).

Result: Achieves state-of-the-art performance and superior domain generalization on multiple fundus datasets. Shows compatibility with diverse backbones including DINOv2 and generalizes to chest X-ray datasets.

Conclusion: PSScreen V2 provides an effective framework for multi-disease screening that handles label absence and domain shift challenges, demonstrating universality and adaptability across different medical imaging domains.

Abstract: In this work, we propose PSScreen V2, a partially supervised self-training framework for multiple retinal disease screening. Unlike previous methods that rely on fully labelled or single-domain datasets, PSScreen V2 is designed to learn from multiple partially labelled datasets with different distributions, addressing both label absence and domain shift challenges. To this end, PSScreen V2 adopts a three-branch architecture with one teacher and two student networks. The teacher branch generates pseudo labels from weakly augmented images to address missing labels, while the two student branches introduce novel feature augmentation strategies: Low-Frequency Dropout (LF-Dropout), which enhances domain robustness by randomly discarding domain-related low-frequency components, and Low-Frequency Uncertainty (LF-Uncert), which estimates uncertain domain variability via adversarially learned Gaussian perturbations of low-frequency statistics. Extensive experiments on multiple in-domain and out-of-domain fundus datasets demonstrate that PSScreen V2 achieves state-of-the-art performance and superior domain generalization ability. Furthermore, compatibility tests with diverse backbones, including the vision foundation model DINOv2, as well as evaluations on chest X-ray datasets, highlight the universality and adaptability of the proposed framework. The codes are available at https://github.com/boyiZheng99/PSScreen_V2.

Method: Proposes Projection Embedded Diffusion Bridge (PEDB) with a novel reverse SDE that samples clean images conditioned on both FBP reconstruction and projection data, embedding projection data into the score function.

Result: PEDB achieves strong performance in CT reconstruction from sparse-view, limited-angle, and truncated projections, outperforming state-of-the-art diffusion bridge models across standard, noisy, and domain-shift evaluations.

Conclusion: Explicitly conditioning on projection data in the sampling process naturally incorporates data consistency and improves CT reconstruction quality from incomplete data.

Abstract: Reconstructing CT images from incomplete projection data remains challenging due to the ill-posed nature of the problem. Diffusion bridge models have recently shown promise in restoring clean images from their corresponding Filtered Back Projection (FBP) reconstructions, but incorporating data consistency into these models remains largely underexplored. Incorporating data consistency can improve reconstruction fidelity by aligning the reconstructed image with the observed projection data, and can enhance detail recovery by integrating structural information contained in the projections. In this work, we propose the Projection Embedded Diffusion Bridge (PEDB). PEDB introduces a novel reverse stochastic differential equation (SDE) to sample from the distribution of clean images conditioned on both the FBP reconstruction and the incomplete projection data. By explicitly conditioning on the projection data in sampling the clean images, PEDB naturally incorporates data consistency. We embed the projection data into the score function of the reverse SDE. Under certain assumptions, we derive a tractable expression for the posterior score. In addition, we introduce a free parameter to control the level of stochasticity in the reverse process. We also design a discretization scheme for the reverse SDE to mitigate discretization error. Extensive experiments demonstrate that PEDB achieves strong performance in CT reconstruction from three types of incomplete data, including sparse-view, limited-angle, and truncated projections. For each of these types, PEDB outperforms evaluated state-of-the-art diffusion bridge models across standard, noisy, and domain-shift evaluations.

Method: Three-stage architecture: SWANencoder maps wavelet coefficients to latent space, SWANdecoder reconstructs coefficients, SWANforward learns hyperspectral physics. Uses biorthogonal wavelet basis, three-stage combined loss function, Adam optimization, and kernel regularizers to prevent overfitting.

Result: Experiments on synthetic and real benchmark datasets show performance enhancement over state-of-the-art neural network methods, with improved unmixing function learning and compact network parameters suitable for practical applications.

Conclusion: SWAN successfully enables self-supervised hyperspectral unmixing by combining wavelet transforms with neural networks, eliminating the need for ground truth while achieving competitive performance through resilient unmixing function learning.

Abstract: In this article, we present SWAN: a three-stage, self-supervised wavelet neural network for joint estimation of endmembers and abundances from hyperspectral imagery. The contiguous and overlapping hyperspectral band images are first expanded to Biorthogonal wavelet basis space that provides sparse, distributed, and multi-scale representations. The idea is to exploit latent symmetries from thus obtained invariant and covariant features using a self-supervised learning paradigm. The first stage, SWANencoder maps the input wavelet coefficients to a compact lower-dimensional latent space. The second stage, SWANdecoder uses the derived latent representation to reconstruct the input wavelet coefficients. Interestingly, the third stage SWANforward learns the underlying physics of the hyperspectral image. A three-stage combined loss function is formulated in the image acquisition domain that eliminates the need for ground truth and enables self-supervised training. Adam is employed for optimizing the proposed loss function, while Sigmoid with a dropout of 0.3 is incorporated to avoid possible overfitting. Kernel regularizers bound the magnitudes and preserve spatial variations in the estimated endmember coefficients. The output of SWANencoder represents estimated abundance maps during inference, while weights of SWANdecoder are retrieved to extract endmembers. Experiments are conducted on two benchmark synthetic data sets with different signal-to-noise ratios as well as on three real benchmark hyperspectral data sets while comparing the results with several state-of-the-art neural network-based unmixing methods. The qualitative, quantitative, and ablation results show performance enhancement by learning a resilient unmixing function as well as promoting self-supervision and compact network parameters for practical applications.

Method: Adapted ZebraPose model for 27-keypoint detection in dairy cows using three configurations: custom on-farm dataset (375 images), APT-36K benchmark subset, and their combination. Systematically evaluated accuracy and generalization across different barn environments.

Result: Combined model achieved promising performance on in-distribution data (AP=0.86, AR=0.87, PCK 0.5=0.869), but showed substantial generalization failures when applied to unseen barns and cow populations, exposing synthetic-to-real domain gap.

Conclusion: Morphological similarity between species is insufficient for cross-domain transfer. Calls for agriculture-first AI design prioritizing farm-level realism, cross-environment robustness, and open benchmark datasets for trustworthy livestock monitoring systems.

Abstract: Pose estimation serves as a cornerstone of computer vision for understanding animal posture, behavior, and welfare. Yet, agricultural applications remain constrained by the scarcity of large, annotated datasets for livestock, especially dairy cattle. This study evaluates the potential and limitations of cross-species transfer learning by adapting ZebraPose - a vision transformer-based model trained on synthetic zebra imagery - for 27-keypoint detection in dairy cows under real barn conditions. Using three configurations

• a custom on-farm dataset (375 images, Sussex, New Brunswick, Canada), a subset of the APT-36K benchmark dataset, and their combination, we systematically assessed model accuracy and generalization across environments. While the combined model achieved promising performance (AP = 0.86, AR = 0.87, PCK 0.5 = 0.869) on in-distribution data, substantial generalization failures occurred when applied to unseen barns and cow populations. These findings expose the synthetic-to-real domain gap as a major obstacle to agricultural AI deployment and emphasize that morphological similarity between species is insufficient for cross-domain transfer. The study provides practical insights into dataset diversity, environmental variability, and computational constraints that influence real-world deployment of livestock monitoring systems. We conclude with a call for agriculture-first AI design, prioritizing farm-level realism, cross-environment robustness, and open benchmark datasets to advance trustworthy and scalable animal-centric technologies.

Method: DenseNet-121-based framework with stain-aware augmentation (Macenko), geometric/ intensity transformations, weighted sampling, and hybrid objective combining class-weighted binary cross-entropy and focal loss, trained end-to-end with AdamW.

Result: Achieved balanced accuracy 85.0%, AUROC 0.927, sensitivity 89.2%, and specificity 80.9% on official test set, demonstrating strong generalization across scanner and staining shifts.

Conclusion: Combining DenseNet-121 with stain-aware augmentation and imbalance-adaptive objectives yields a robust, domain-generalizable framework suitable for real-world computational pathology workflows.

Abstract: Atypical mitotic figures are important biomarkers of tumor aggressiveness in histopathology, yet reliable recognition remains challenging due to severe class imbalance and variability across imaging domains. We present a DenseNet-121-based framework tailored for atypical mitosis classification in the MIDOG 2025 (Track 2) setting. Our method integrates stain-aware augmentation (Macenko), geometric and intensity transformations, and imbalance-aware learning via weighted sampling with a hybrid objective combining class-weighted binary cross-entropy and focal loss. Trained end-to-end with AdamW and evaluated across multiple independent domains, the model demonstrates strong generalization under scanner and staining shifts, achieving balanced accuracy 85.0%, AUROC 0.927, sensitivity 89.2%, and specificity 80.9% on the official test set. These results indicate that combining DenseNet-121 with stain-aware augmentation and imbalance-adaptive objectives yields a robust, domain-generalizable framework for atypical mitosis classification suitable for real-world computational pathology workflows.

Method: Evaluated SSD MobileNet V2 and Faster R-CNN ResNet50 V1 for object detection, and Mask R-CNN for instance segmentation with custom method to calculate damaged leaf portions.

Result: Faster R-CNN ResNet50 V1 achieved better performance with 25% mAP compared to SSD MobileNet V2’s 20.9% mAP. Both models showed low precision (0.252 vs 0.209) and recall (0.044 vs 0.02) on IOU 0.50:0.95 range.

Conclusion: Faster R-CNN outperforms SSD MobileNet for tea leaf disease detection, and Mask R-CNN with custom damage calculation provides effective disease area quantification, though overall detection performance needs improvement.

Abstract: The proposed solution is Deep Learning Technique that will be able classify three types of tea leaves diseases from which two diseases are caused by the pests and one due to pathogens (infectious organisms) and environmental conditions and also show the area damaged by a disease in leaves. Namely Red Rust, Helopeltis and Red spider mite respectively. In this paper we have evaluated two models namely SSD MobileNet V2 and Faster R-CNN ResNet50 V1 for the object detection. The SSD MobileNet V2 gave precision of 0.209 for IOU range of 0.50:0.95 with recall of 0.02 on IOU 0.50:0.95 and final mAP of 20.9%. While Faster R-CNN ResNet50 V1 has precision of 0.252 on IOU range of 0.50:0.95 and recall of 0.044 on IOU of 0.50:0.95 with a mAP of 25%, which is better than SSD. Also used Mask R-CNN for Object Instance Segmentation where we have implemented our custom method to calculate the damaged diseased portion of leaves. Keywords: Tea Leaf Disease, Deep Learning, Red Rust, Helopeltis and Red Spider Mite, SSD MobileNet V2, Faster R-CNN ResNet50 V1 and Mask RCNN.

Method: Compute eigenvectors of self-attention weight matrices in pretrained diffusion models to obtain robust and interpretable editing directions without additional data or fine-tuning.

Result: Produces high-quality edits across multiple datasets while reducing editing time by 60% compared to current benchmarks.

Conclusion: The method enables efficient and precise control over diffusion model outputs by leveraging intrinsic structural information encoded in self-attention weights.

Abstract: Diffusion models achieve remarkable fidelity in image synthesis, yet precise control over their outputs for targeted editing remains challenging. A key step toward controllability is to identify interpretable directions in the model’s latent representations that correspond to semantic attributes. Existing approaches for finding interpretable directions typically rely on sampling large sets of images or training auxiliary networks, which limits efficiency. We propose an analytical method that derives semantic editing directions directly from the pretrained parameters of diffusion models, requiring neither additional data nor fine-tuning. Our insight is that self-attention weight matrices encode rich structural information about the data distribution learned during training. By computing the eigenvectors of these weight matrices, we obtain robust and interpretable editing directions. Experiments demonstrate that our method produces high-quality edits across multiple datasets while reducing editing time significantly by 60% over current benchmarks.

Method: Constructed SARCLIP-1M dataset with 1M text-image pairs, then trained SARCLIP model using contrastive vision-language learning with domain transfer strategy to bridge SAR imagery and text.

Result: Extensive experiments show SARCLIP significantly outperforms state-of-the-art foundation models in image-text retrieval and zero-shot classification tasks, advancing SAR semantic understanding.

Conclusion: SARCLIP successfully bridges the gap between SAR imagery and textual descriptions, enabling superior feature extraction and interpretation capabilities for SAR domain applications.

Abstract: Synthetic Aperture Radar (SAR) has emerged as a crucial imaging modality due to its all-weather capabilities. While recent advancements in self-supervised learning and Masked Image Modeling (MIM) have paved the way for SAR foundation models, these approaches primarily focus on low-level visual features, often overlooking multimodal alignment and zero-shot target recognition within SAR imagery. To address this limitation, we construct SARCLIP-1M, a large-scale vision language dataset comprising over one million text-image pairs aggregated from existing datasets. We further introduce SARCLIP, the first vision language foundation model tailored for the SAR domain. Our SARCLIP model is trained using a contrastive vision language learning approach by domain transferring strategy, enabling it to bridge the gap between SAR imagery and textual descriptions. Extensive experiments on image-text retrieval and zero-shot classification tasks demonstrate the superior performance of SARCLIP in feature extraction and interpretation, significantly outperforming state-of-the-art foundation models and advancing the semantic understanding of SAR imagery. The code and datasets will be released soon.

Method: Proposes hierarchical collaborative representation module for mutual reinforcement in mapping optimization, and joint dynamic modeling module that fuses open-world segmentation with implicit residual constraints using DINO-Depth uncertainty estimates.

Result: Extensive evaluations on KITTI, nuScenes, and self-collected datasets show state-of-the-art performance compared to existing methods.

Conclusion: LVD-GS effectively mitigates scale drift, enhances reconstruction robustness, and eliminates dynamic object influence through its novel hierarchical representation and dynamic modeling approaches.

Abstract: 3D Gaussian Splatting SLAM has emerged as a widely used technique for high-fidelity mapping in spatial intelligence. However, existing methods often rely on a single representation scheme, which limits their performance in large-scale dynamic outdoor scenes and leads to cumulative pose errors and scale ambiguity. To address these challenges, we propose \textbf{LVD-GS}, a novel LiDAR-Visual 3D Gaussian Splatting SLAM system. Motivated by the human chain-of-thought process for information seeking, we introduce a hierarchical collaborative representation module that facilitates mutual reinforcement for mapping optimization, effectively mitigating scale drift and enhancing reconstruction robustness. Furthermore, to effectively eliminate the influence of dynamic objects, we propose a joint dynamic modeling module that generates fine-grained dynamic masks by fusing open-world segmentation with implicit residual constraints, guided by uncertainty estimates from DINO-Depth features. Extensive evaluations on KITTI, nuScenes, and self-collected datasets demonstrate that our approach achieves state-of-the-art performance compared to existing methods.

Method: Used Meta Project Aria smart glasses and stationary cameras to record synchronized gaze, speech, and video from 25 participants instructing partners to identify kitchen ingredients, combined with 3D scene reconstructions.

Result: Created a dataset with 3.67 hours of recordings containing 2,707 richly annotated referential expressions, providing a benchmark for evaluating spatial representation effects on multimodal grounding.

Conclusion: The dataset enables research on how different spatial representations (2D vs 3D, ego vs exo) affect multimodal grounding and supports development of embodied agents for situated dialogue.

Abstract: We introduce Look and Tell, a multimodal dataset for studying referential communication across egocentric and exocentric perspectives. Using Meta Project Aria smart glasses and stationary cameras, we recorded synchronized gaze, speech, and video as 25 participants instructed a partner to identify ingredients in a kitchen. Combined with 3D scene reconstructions, this setup provides a benchmark for evaluating how different spatial representations (2D vs. 3D; ego vs. exo) affect multimodal grounding. The dataset contains 3.67 hours of recordings, including 2,707 richly annotated referential expressions, and is designed to advance the development of embodied agents that can understand and engage in situated dialogue.

Method: The model combines alias-free downsampling and nonlinearities with linear cross-covariance attention that is shift-equivariant to both integer and fractional translations, enabling shift-invariant global representation.

Result: The proposed Alias-Free ViT maintains competitive performance in image classification and outperforms similar-sized models in robustness to adversarial translations.

Conclusion: The Alias-Free ViT successfully addresses translation sensitivity in Vision Transformers while maintaining competitive performance, demonstrating improved robustness to adversarial translations compared to standard ViTs.

Abstract: Transformers have emerged as a competitive alternative to convnets in vision tasks, yet they lack the architectural inductive bias of convnets, which may hinder their potential performance. Specifically, Vision Transformers (ViTs) are not translation-invariant and are more sensitive to minor image translations than standard convnets. Previous studies have shown, however, that convnets are also not perfectly shift-invariant, due to aliasing in downsampling and nonlinear layers. Consequently, anti-aliasing approaches have been proposed to certify convnets’ translation robustness. Building on this line of work, we propose an Alias-Free ViT, which combines two main components. First, it uses alias-free downsampling and nonlinearities. Second, it uses linear cross-covariance attention that is shift-equivariant to both integer and fractional translations, enabling a shift-invariant global representation. Our model maintains competitive performance in image classification and outperforms similar-sized models in terms of robustness to adversarial translations.

Method: Proposes DAMap with three components: Distance-aware Focal Loss (DAFL) for better classification labels, Task Modulated Deformable Attention (TMDA) for discriminative task-specific features, and Hybrid Loss Scheme (HLS) to optimize DAFL usage.

Result: Achieves consistent performance improvements on NuScenes and Argoverse2 benchmarks across different metrics, baselines, splits, backbones, and training schedules.

Conclusion: DAMap effectively addresses task misalignment in HD map prediction and demonstrates superior performance through its three key components.

Abstract: Predicting High-definition (HD) map elements with high quality (high classification and localization scores) is crucial to the safety of autonomous driving vehicles. However, current methods perform poorly in high quality predictions due to inherent task misalignment. Two main factors are responsible for misalignment: 1) inappropriate task labels due to one-to-many matching queries sharing the same labels, and 2) sub-optimal task features due to task-shared sampling mechanism. In this paper, we reveal two inherent defects in current methods and develop a novel HD map construction method named DAMap to address these problems. Specifically, DAMap consists of three components: Distance-aware Focal Loss (DAFL), Hybrid Loss Scheme (HLS), and Task Modulated Deformable Attention (TMDA). The DAFL is introduced to assign appropriate classification labels for one-to-many matching samples. The TMDA is proposed to obtain discriminative task-specific features. Furthermore, the HLS is proposed to better utilize the advantages of the DAFL. We perform extensive experiments and consistently achieve performance improvement on the NuScenes and Argoverse2 benchmarks under different metrics, baselines, splits, backbones, and schedules. Code will be available at https://github.com/jpdong-xjtu/DAMap.

Method: Multi-task learning model that jointly estimates construction year, building structure, and property type from facade images, then derives fireproof class (H/T/M) via rule-based mapping based on official insurance criteria.

Result: Model achieved high accuracy in construction-year regression and robust classification across imbalanced categories, capturing visual cues related to building age and materials.

Conclusion: The approach demonstrates feasibility of scalable, interpretable, image-based risk-profiling systems with applications in insurance, urban planning, and disaster preparedness.

Abstract: Structural fireproof classification is vital for disaster risk assessment and insurance pricing in Japan. However, key building metadata such as construction year and structure type are often missing or outdated, particularly in the second-hand housing market. This study proposes a multi-task learning model that predicts these attributes from facade images. The model jointly estimates the construction year, building structure, and property type, from which the structural fireproof class - defined as H (non-fireproof), T (semi-fireproof), or M (fireproof) - is derived via a rule-based mapping based on official insurance criteria. We trained and evaluated the model using a large-scale dataset of Japanese residential images, applying rigorous filtering and deduplication. The model achieved high accuracy in construction-year regression and robust classification across imbalanced categories. Qualitative analyses show that it captures visual cues related to building age and materials. Our approach demonstrates the feasibility of scalable, interpretable, image-based risk-profiling systems, offering potential applications in insurance, urban planning, and disaster preparedness.

Method: A three-stage framework: produces multimodal guidance from input queries, synthesizes candidate SVGs through dedicated generation modules, and refines them under numerical guidance. Built on RoboDraw dataset of 1M SVG examples.

Result: Achieves superior query compliance and visual fidelity across tasks (Text-to-SVG, Image-to-SVG, PartialSVG-to-SVG, PartialImage-to-SVG), establishing new state-of-the-art in versatile SVG generation.

Conclusion: RoboSVG provides an effective unified framework for multimodal SVG generation with strong performance across diverse tasks, with dataset and code to be publicly released.

Abstract: Scalable Vector Graphics (SVGs) are fundamental to digital design and robot control, encoding not only visual structure but also motion paths in interactive drawings. In this work, we introduce RoboSVG, a unified multimodal framework for generating interactive SVGs guided by textual, visual, and numerical signals. Given an input query, the RoboSVG model first produces multimodal guidance, then synthesizes candidate SVGs through dedicated generation modules, and finally refines them under numerical guidance to yield high-quality outputs. To support this framework, we construct RoboDraw, a large-scale dataset of one million examples, each pairing an SVG generation condition (e.g., text, image, and partial SVG) with its corresponding ground-truth SVG code. RoboDraw dataset enables systematic study of four tasks, including basic generation (Text-to-SVG, Image-to-SVG) and interactive generation (PartialSVG-to-SVG, PartialImage-to-SVG). Extensive experiments demonstrate that RoboSVG achieves superior query compliance and visual fidelity across tasks, establishing a new state of the art in versatile SVG generation. The dataset and source code of this project will be publicly available soon.

Method: Uses Hierarchical Granularity-aware Tree (HGTree) with GEBD for event boundary detection, adaptive coarse-fine structuring, VLMs with multi-dimensional priors for anomaly perception, and LLMs for anomaly reasoning.

Result: Achieves state-of-the-art performance on three challenging datasets while significantly reducing the number of sampled video segments compared to existing methods.

Conclusion: VADTree provides an effective training-free solution for VAD that handles varying temporal spans and offers better anomaly detection with reduced computational cost.

Abstract: Video anomaly detection (VAD) focuses on identifying anomalies in videos. Supervised methods demand substantial in-domain training data and fail to deliver clear explanations for anomalies. In contrast, training-free methods leverage the knowledge reserves and language interactivity of large pre-trained models to detect anomalies. However, the current fixed-length temporal window sampling approaches struggle to accurately capture anomalies with varying temporal spans. Therefore, we propose VADTree that utilizes a Hierarchical Granularityaware Tree (HGTree) structure for flexible sampling in VAD. VADTree leverages the knowledge embedded in a pre-trained Generic Event Boundary Detection (GEBD) model to characterize potential anomaly event boundaries. Specifically, VADTree decomposes the video into generic event nodes based on boundary confidence, and performs adaptive coarse-fine hierarchical structuring and redundancy removal to construct the HGTree. Then, the multi-dimensional priors are injected into the visual language models (VLMs) to enhance the node-wise anomaly perception, and anomaly reasoning for generic event nodes is achieved via large language models (LLMs). Finally, an inter-cluster node correlation method is used to integrate the multi-granularity anomaly scores. Extensive experiments on three challenging datasets demonstrate that VADTree achieves state-of-the-art performance in training-free settings while drastically reducing the number of sampled video segments. The code will be available at https://github.com/wenlongli10/VADTree.

Method: Introduces Windsock module for dynamic retrieval decisions and modality selection, DANCE Instruction Tuning for adaptive training to enhance information utilization and noise resistance, and a self-assessment approach to convert QA datasets to MRAG training datasets.

Result: Significantly improves generation quality by 17.07% while reducing retrieval times by 8.95% in extensive experiments.

Conclusion: The proposed method effectively addresses the key challenges in MRAG by providing dynamic retrieval strategies, flexible modality selection, and improved information utilization, leading to substantial performance gains with reduced computational overhead.

Abstract: Multimodal Retrieval-Augmented Generation (MRAG) has emerged as a promising method to generate factual and up-to-date responses of Multimodal Large Language Models (MLLMs) by incorporating non-parametric knowledge from external knowledge bases. However, existing MRAG approaches suffer from static retrieval strategies, inflexible modality selection, and suboptimal utilization of retrieved information, leading to three critical challenges: determining when to retrieve, what modality to incorporate, and how to utilize retrieved information effectively. To address these challenges, we introduce Windsock, a query-dependent module making decisions on retrieval necessity and modality selection, effectively reducing computational overhead and improving response quality. Additionally, we propose Dynamic Noise-Resistance (DANCE) Instruction Tuning, an adaptive training strategy that enhances MLLMs’ ability to utilize retrieved information while maintaining robustness against noise. Moreover, we adopt a self-assessment approach leveraging knowledge within MLLMs to convert question-answering datasets to MRAG training datasets. Extensive experiments demonstrate that our proposed method significantly improves the generation quality by 17.07% while reducing 8.95% retrieval times.

Method: Uses masked autoencoding with multi-level Discrete Wavelet Transform for frequency component disentanglement and Geo-conditioned Positional Encoding with Spherical Harmonics for geographical priors.

Result: Achieves consistent improvements over prior state-of-the-art, with substantial gains on segmentation and regression. Even lightweight variant (26.4% parameters) achieves SOTA performance.

Conclusion: WaveMAE establishes itself as a strong, geographically informed foundation model for multispectral remote sensing imagery.

Abstract: Self-supervised learning (SSL) has recently emerged as a key strategy for building foundation models in remote sensing, where the scarcity of annotated data limits the applicability of fully supervised approaches. In this work, we introduce WaveMAE, a masked autoencoding framework tailored for multispectral satellite imagery. Unlike conventional pixel-based reconstruction, WaveMAE leverages a multi-level Discrete Wavelet Transform (DWT) to disentangle frequency components and guide the encoder toward learning scale-aware high-frequency representations. We further propose a Geo-conditioned Positional Encoding (GPE), which incorporates geographical priors via Spherical Harmonics, encouraging embeddings that respect both semantic and geospatial structure. To ensure fairness in evaluation, all methods are pretrained on the same dataset (fMoW-S2) and systematically evaluated on the diverse downstream tasks of the PANGAEA benchmark, spanning semantic segmentation, regression, change detection, and multilabel classification. Extensive experiments demonstrate that WaveMAE achieves consistent improvements over prior state-of-the-art approaches, with substantial gains on segmentation and regression benchmarks. The effectiveness of WaveMAE pretraining is further demonstrated by showing that even a lightweight variant, containing only 26.4% of the parameters, achieves state-of-the-art performance. Our results establish WaveMAE as a strong and geographically informed foundation model for multispectral remote sensing imagery.

Method: Proposed InstanceGrounded Geometry Transformer (IGGT) with 3D-Consistent Contrastive Learning strategy to encode unified representations from 2D visual inputs. Created InsScene-15K dataset with comprehensive annotations.

Result: IGGT learns to lift 2D visual inputs into coherent 3D scenes with explicitly distinct object instances through unified geometric and instance-grounded representations.

Conclusion: The unified approach enables more coherent and accurate 3D scene understanding by bridging the gap between geometric reconstruction and semantic perception.

Abstract: Humans naturally perceive the geometric structure and semantic content of a 3D world as intertwined dimensions, enabling coherent and accurate understanding of complex scenes. However, most prior approaches prioritize training large geometry models for low-level 3D reconstruction and treat high-level spatial understanding in isolation, overlooking the crucial interplay between these two fundamental aspects of 3D-scene analysis, thereby limiting generalization and leading to poor performance in downstream 3D understanding tasks. Recent attempts have mitigated this issue by simply aligning 3D models with specific language models, thus restricting perception to the aligned model’s capacity and limiting adaptability to downstream tasks. In this paper, we propose InstanceGrounded Geometry Transformer (IGGT), an end-to-end large unified transformer to unify the knowledge for both spatial reconstruction and instance-level contextual understanding. Specifically, we design a 3D-Consistent Contrastive Learning strategy that guides IGGT to encode a unified representation with geometric structures and instance-grounded clustering through only 2D visual inputs. This representation supports consistent lifting of 2D visual inputs into a coherent 3D scene with explicitly distinct object instances. To facilitate this task, we further construct InsScene-15K, a large-scale dataset with high-quality RGB images, poses, depth maps, and 3D-consistent instance-level mask annotations with a novel data curation pipeline.

Method: Created dataset using automated end-to-end curation pipeline with ASR transcription, active-speaker localization, quality filtering, and pose/mask screening from 67 TV programs. Fine-tuned two established lipreading architectures for benchmarking.

Result: Established reference performance on Persian visual speech recognition, demonstrating the difficulty of the task. The dataset provides speaker-disjoint splits, wide dialectal coverage, and rich metadata including head pose, age, and gender.

Conclusion: LRW-Persian fills a critical gap for low-resource languages, enabling rigorous benchmarking, cross-lingual transfer, and advancing multimodal speech research in underrepresented linguistic contexts.

Abstract: Lipreading has emerged as an increasingly important research area for developing robust speech recognition systems and assistive technologies for the hearing-impaired. However, non-English resources for visual speech recognition remain limited. We introduce LRW-Persian, the largest in-the-wild Persian word-level lipreading dataset, comprising $743$ target words and over $414{,}000$ video samples extracted from more than $1{,}900$ hours of footage across $67$ television programs. Designed as a benchmark-ready resource, LRW-Persian provides speaker-disjoint training and test splits, wide regional and dialectal coverage, and rich per-clip metadata including head pose, age, and gender. To ensure large-scale data quality, we establish a fully automated end-to-end curation pipeline encompassing transcription based on Automatic Speech Recognition(ASR), active-speaker localization, quality filtering, and pose/mask screening. We further fine-tune two widely used lipreading architectures on LRW-Persian, establishing reference performance and demonstrating the difficulty of Persian visual speech recognition. By filling a critical gap in low-resource languages, LRW-Persian enables rigorous benchmarking, supports cross-lingual transfer, and provides a foundation for advancing multimodal speech research in underrepresented linguistic contexts. The dataset is publicly available at: https://lrw-persian.vercel.app.

Method: The survey reviews datasets, challenges, and state-of-the-art techniques. Cross-view localization is typically formulated as image retrieval using CNNs or ViTs for feature embedding, while cross-view synthesis uses GANs or diffusion models.

Result: The paper provides an organized overview of advances in both tasks, including comparative analyses of current methods and identification of key challenges in the field.

Conclusion: The survey discusses current limitations and outlines promising future research directions for cross-view localization and synthesis, providing a comprehensive resource for researchers in this domain.

Abstract: Cross-view localization and synthesis are two fundamental tasks in cross-view visual understanding, which deals with cross-view datasets: overhead (satellite or aerial) and ground-level imagery. These tasks have gained increasing attention due to their broad applications in autonomous navigation, urban planning, and augmented reality. Cross-view localization aims to estimate the geographic position of ground-level images based on information provided by overhead imagery while cross-view synthesis seeks to generate ground-level images based on information from the overhead imagery. Both tasks remain challenging due to significant differences in viewing perspective, resolution, and occlusion, which are widely embedded in cross-view datasets. Recent years have witnessed rapid progress driven by the availability of large-scale datasets and novel approaches. Typically, cross-view localization is formulated as an image retrieval problem where ground-level features are matched with tiled overhead images feature, extracted by convolutional neural networks (CNNs) or vision transformers (ViTs) for cross-view feature embedding. Cross-view synthesis, on the other hand, seeks to generate ground-level views based on information from overhead imagery, generally using generative adversarial networks (GANs) or diffusion models. This paper presents a comprehensive survey of advances in cross-view localization and synthesis, reviewing widely used datasets, highlighting key challenges, and providing an organized overview of state-of-the-art techniques. Furthermore, it discusses current limitations, offers comparative analyses, and outlines promising directions for future research. We also include the project page via https://github.com/GDAOSU/Awesome-Cross-View-Methods.

Method: Proposed ConMatFormer architecture using ConvNeXt blocks for local features, transformer modules for long-range dependencies, multiple attention mechanisms (CBAM and DANet), and data augmentation to address class imbalance.

Result: Achieved 0.8961 accuracy and 0.9160 precision in single experiments, and 0.9755 accuracy with 0.0031 std in 4-fold cross-validation, outperforming SOTA CNN and ViT models on DFUC2021 and DFU datasets.

Conclusion: ConMatFormer sets a new benchmark for DFU classification and provides a reliable hybrid attention transformer framework for medical image analysis, with explainable AI methods ensuring transparent decision-making.

Abstract: Diabetic foot ulcer (DFU) detection is a clinically significant yet challenging task due to the scarcity and variability of publicly available datasets. To solve these problems, we propose ConMatFormer, a new hybrid deep learning architecture that combines ConvNeXt blocks, multiple attention mechanisms convolutional block attention module (CBAM) and dual attention network (DANet), and transformer modules in a way that works together. This design facilitates the extraction of better local features and understanding of the global context, which allows us to model small skin patterns across different types of DFU very accurately. To address the class imbalance, we used data augmentation methods. A ConvNeXt block was used to obtain detailed local features in the initial stages. Subsequently, we compiled the model by adding a transformer module to enhance long-range dependency. This enabled us to pinpoint the DFU classes that were underrepresented or constituted minorities. Tests on the DS1 (DFUC2021) and DS2 (diabetic foot ulcer (DFU)) datasets showed that ConMatFormer outperformed state-of-the-art (SOTA) convolutional neural network (CNN) and Vision Transformer (ViT) models in terms of accuracy, reliability, and flexibility. The proposed method achieved an accuracy of 0.8961 and a precision of 0.9160 in a single experiment, which is a significant improvement over the current standards for classifying DFUs. In addition, by 4-fold cross-validation, the proposed model achieved an accuracy of 0.9755 with a standard deviation of only 0.0031. We further applied explainable artificial intelligence (XAI) methods, such as Grad-CAM, Grad-CAM++, and LIME, to consistently monitor the transparency and trustworthiness of the decision-making process.. Our findings set a new benchmark for DFU classification and provide a hybrid attention transformer framework for medical image analysis.

Method: SCC consists of two modules: Semantic consistency uses soft pseudo-labels from counterattack warm-up and multi-view predictions to regularize cross-modal alignment; Spatial consistency aligns perturbed visual predictions via augmented views to stabilize inference.

Result: Extensive experiments on 22 benchmarks show SCC consistently improves CLIP’s zero-shot robustness while maintaining accuracy, and can be integrated with other VLMs for further gains.

Conclusion: SCC demonstrates the potential for establishing an adversarially robust paradigm from CLIP, with implications extending to broader vision-language domains like BioMedCLIP.

Abstract: Pre-trained vision-language models (VLMs) such as CLIP have demonstrated strong zero-shot capabilities across diverse domains, yet remain highly vulnerable to adversarial perturbations that disrupt image-text alignment and compromise reliability. Existing defenses typically rely on adversarial fine-tuning with labeled data, limiting their applicability in zero-shot settings. In this work, we identify two key weaknesses of current CLIP adversarial attacks – lack of semantic guidance and vulnerability to view variations – collectively termed semantic and viewpoint fragility. To address these challenges, we propose Self-Calibrated Consistency (SCC), an effective test-time defense. SCC consists of two complementary modules: Semantic consistency, which leverages soft pseudo-labels from counterattack warm-up and multi-view predictions to regularize cross-modal alignment and separate the target embedding from confusable negatives; and Spatial consistency, aligning perturbed visual predictions via augmented views to stabilize inference under adversarial perturbations. Together, these modules form a plug-and-play inference strategy. Extensive experiments on 22 benchmarks under diverse attack settings show that SCC consistently improves the zero-shot robustness of CLIP while maintaining accuracy, and can be seamlessly integrated with other VLMs for further gains. These findings highlight the great potential of establishing an adversarially robust paradigm from CLIP, with implications extending to broader vision-language domains such as BioMedCLIP.

Method: Framework integrates fine-tuned BLIP-2 backbone, medical query reformulation, enhanced Grad-CAM attention, precise region extraction, and structured chain-of-thought reasoning via multi-modal language models.

Result: Experiments on 500 PathVQA samples show composite score of 0.683 vs 0.378 baseline, high reasoning confidence (0.890), identifies 3-5 relevant regions per sample, generates 57-word structured explanations with clinical terminology.

Conclusion: MedXplain-VQA shows potential as a robust, explainable medical VQA system, with query reformulation providing most significant initial improvement and chain-of-thought enabling systematic diagnostics.

Abstract: Explainability is critical for the clinical adoption of medical visual question answering (VQA) systems, as physicians require transparent reasoning to trust AI-generated diagnoses. We present MedXplain-VQA, a comprehensive framework integrating five explainable AI components to deliver interpretable medical image analysis. The framework leverages a fine-tuned BLIP-2 backbone, medical query reformulation, enhanced Grad-CAM attention, precise region extraction, and structured chain-of-thought reasoning via multi-modal language models. To evaluate the system, we introduce a medical-domain-specific framework replacing traditional NLP metrics with clinically relevant assessments, including terminology coverage, clinical structure quality, and attention region relevance. Experiments on 500 PathVQA histopathology samples demonstrate substantial improvements, with the enhanced system achieving a composite score of 0.683 compared to 0.378 for baseline methods, while maintaining high reasoning confidence (0.890). Our system identifies 3-5 diagnostically relevant regions per sample and generates structured explanations averaging 57 words with appropriate clinical terminology. Ablation studies reveal that query reformulation provides the most significant initial improvement, while chain-of-thought reasoning enables systematic diagnostic processes. These findings underscore the potential of MedXplain-VQA as a robust, explainable medical VQA system. Future work will focus on validation with medical experts and large-scale clinical datasets to ensure clinical readiness.

Method: MAGIC-Talk uses a dual-network architecture with ReferenceNet for identity preservation and fine-grained facial editing via text prompts, and AnimateNet for motion coherence using structured motion priors. It employs progressive latent fusion strategy for long-form video quality improvement.

Result: Extensive experiments show MAGIC-Talk outperforms state-of-the-art methods in visual quality, identity preservation, and synchronization accuracy.

Conclusion: MAGIC-Talk offers a robust solution for talking face generation that maintains identity from a single image while ensuring smooth transitions across frames, without requiring multiple reference images or fine-tuning.

Abstract: Audio-driven talking face generation has gained significant attention for applications in digital media and virtual avatars. While recent methods improve audio-lip synchronization, they often struggle with temporal consistency, identity preservation, and customization, especially in long video generation. To address these issues, we propose MAGIC-Talk, a one-shot diffusion-based framework for customizable and temporally stable talking face generation. MAGIC-Talk consists of ReferenceNet, which preserves identity and enables fine-grained facial editing via text prompts, and AnimateNet, which enhances motion coherence using structured motion priors. Unlike previous methods requiring multiple reference images or fine-tuning, MAGIC-Talk maintains identity from a single image while ensuring smooth transitions across frames. Additionally, a progressive latent fusion strategy is introduced to improve long-form video quality by reducing motion inconsistencies and flickering. Extensive experiments demonstrate that MAGIC-Talk outperforms state-of-the-art methods in visual quality, identity preservation, and synchronization accuracy, offering a robust solution for talking face generation.

Method: Introduces M³T2IBench benchmark with AlignScore evaluation metric based on object detection, and proposes Revise-Then-Enforce - a training-free post-editing approach to enhance image-text alignment in diffusion models.

Result: Current open-source text-to-image models perform poorly on the challenging M³T2IBench benchmark, but the proposed Revise-Then-Enforce method demonstrates improved image-text alignment across various diffusion models.

Conclusion: The M³T2IBench benchmark effectively reveals limitations in current text-to-image models for complex scenarios, and the proposed Revise-Then-Enforce approach offers a practical solution to enhance alignment without requiring model retraining.

Abstract: Text-to-image models are known to struggle with generating images that perfectly align with textual prompts. Several previous studies have focused on evaluating image-text alignment in text-to-image generation. However, these evaluations either address overly simple scenarios, especially overlooking the difficulty of prompts with multiple different instances belonging to the same category, or they introduce metrics that do not correlate well with human evaluation. In this study, we introduce M$^3$T2IBench, a large-scale, multi-category, multi-instance, multi-relation along with an object-detection-based evaluation metric, $AlignScore$, which aligns closely with human evaluation. Our findings reveal that current open-source text-to-image models perform poorly on this challenging benchmark. Additionally, we propose the Revise-Then-Enforce approach to enhance image-text alignment. This training-free post-editing method demonstrates improvements in image-text alignment across a broad range of diffusion models. \footnote{Our code and data has been released in supplementary material and will be made publicly available after the paper is accepted.}

Method: Uses instruction-following multimodal LLMs as judges with explanation-oriented rubrics mapped to [-1,1], closed label sets, evidence grounding in visible content, and mandatory abstention when cues are insufficient.

Result: Outperforms contrastive and face-centric baselines on demographic prediction across multiple datasets (FairFace, PaTA, FairCoT), improves mean alignment while maintaining high profession accuracy on IdenProf, FairCoT-Professions, and DIVERSIFY-Professions.

Conclusion: FairJudge enables more reliable and reproducible fairness audits for text-to-image systems by providing accountable, evidence-aware evaluation that addresses limitations of existing methods.

Abstract: Text-to-image (T2I) systems lack simple, reproducible ways to evaluate how well images match prompts and how models treat social attributes. Common proxies – face classifiers and contrastive similarity – reward surface cues, lack calibrated abstention, and miss attributes only weakly visible (for example, religion, culture, disability). We present FairJudge, a lightweight protocol that treats instruction-following multimodal LLMs as fair judges. It scores alignment with an explanation-oriented rubric mapped to [-1, 1]; constrains judgments to a closed label set; requires evidence grounded in the visible content; and mandates abstention when cues are insufficient. Unlike CLIP-only pipelines, FairJudge yields accountable, evidence-aware decisions; unlike mitigation that alters generators, it targets evaluation fairness. We evaluate gender, race, and age on FairFace, PaTA, and FairCoT; extend to religion, culture, and disability; and assess profession correctness and alignment on IdenProf, FairCoT-Professions, and our new DIVERSIFY-Professions. We also release DIVERSIFY, a 469-image corpus of diverse, non-iconic scenes. Across datasets, judge models outperform contrastive and face-centric baselines on demographic prediction and improve mean alignment while maintaining high profession accuracy, enabling more reliable, reproducible fairness audits.

Method: Developed UniAIDet benchmark covering text-to-image, image-to-image, image inpainting, image editing, and deepfake models with both photographic and artistic images. Conducted comprehensive evaluation of various detection methods.

Result: The benchmark enables answering key research questions about generalization capability and the relation between detection and localization in AI-generated image detection.

Conclusion: UniAIDet provides a robust foundation for future research in AI-generated image detection by offering comprehensive coverage of diverse generative scenarios.

Abstract: With the rapid proliferation of image generative models, the authenticity of digital images has become a significant concern. While existing studies have proposed various methods for detecting AI-generated content, current benchmarks are limited in their coverage of diverse generative models and image categories, often overlooking end-to-end image editing and artistic images. To address these limitations, we introduce UniAIDet, a unified and comprehensive benchmark that includes both photographic and artistic images. UniAIDet covers a wide range of generative models, including text-to-image, image-to-image, image inpainting, image editing, and deepfake models. Using UniAIDet, we conduct a comprehensive evaluation of various detection methods and answer three key research questions regarding generalization capability and the relation between detection and localization. Our benchmark and analysis provide a robust foundation for future research.

Method: SP-CSVR integrates three key components: Cross-Style Feature Encoder for style-content disentanglement, Semantic-Aligned In-Context Decoder for few-shot style adaptation, and Adaptive Semantic Consistency Module using multi-task contrastive learning to enforce cross-style semantic invariance.

Result: Extensive experiments on a challenging multi-style dataset demonstrate SP-CSVR’s state-of-the-art performance across visual captioning, visual question answering, and in-context style adaptation.

Conclusion: SP-CSVR effectively enhances robustness, generalization, and efficiency across diverse visual styles, addressing the style trap problem in LVLMs.

Abstract: The “style trap” poses a significant challenge for Large Vision-Language Models (LVLMs), hindering robust semantic understanding across diverse visual styles, especially in in-context learning (ICL). Existing methods often fail to effectively decouple style from content, hindering generalization. To address this, we propose the Semantic-Preserving Cross-Style Visual Reasoner (SP-CSVR), a novel framework for stable semantic understanding and adaptive cross-style visual reasoning. SP-CSVR integrates a Cross-Style Feature Encoder (CSFE) for style-content disentanglement, a Semantic-Aligned In-Context Decoder (SAICD) for efficient few-shot style adaptation, and an Adaptive Semantic Consistency Module (ASCM) employing multi-task contrastive learning to enforce cross-style semantic invariance. Extensive experiments on a challenging multi-style dataset demonstrate SP-CSVR’s state-of-the-art performance across visual captioning, visual question answering, and in-context style adaptation. Comprehensive evaluations, including ablation studies and generalization analysis, confirm SP-CSVR’s efficacy in enhancing robustness, generalization, and efficiency across diverse visual styles.

Method: Leverages pairwise matches from off-the-shelf image matchers with nonparametric clustering to build a graph of keypoint relations, uses GNN for correspondence propagation, and employs inverse-compositional loss without regularization terms.

Result: Achieves better alignment quality than modern JA methods while reducing computation time from hours/minutes to mere seconds on several benchmarks.

Conclusion: FastJAM provides an efficient and effective solution for joint image alignment that eliminates the need for regularization terms and associated hyperparameter tuning.

Abstract: Joint Alignment (JA) of images aims to align a collection of images into a unified coordinate frame, such that semantically-similar features appear at corresponding spatial locations. Most existing approaches often require long training times, large-capacity models, and extensive hyperparameter tuning. We introduce FastJAM, a rapid, graph-based method that drastically reduces the computational complexity of joint alignment tasks. FastJAM leverages pairwise matches computed by an off-the-shelf image matcher, together with a rapid nonparametric clustering, to construct a graph representing intra- and inter-image keypoint relations. A graph neural network propagates and aggregates these correspondences, efficiently predicting per-image homography parameters via image-level pooling. Utilizing an inverse-compositional loss, that eliminates the need for a regularization term over the predicted transformations (and thus also obviates the hyperparameter tuning associated with such terms), FastJAM performs image JA quickly and effectively. Experimental results on several benchmarks demonstrate that FastJAM achieves results better than existing modern JA methods in terms of alignment quality, while reducing computation time from hours or minutes to mere seconds. Our code is available at our project webpage, https://bgu-cs-vil.github.io/FastJAM/

Method: The framework dynamically estimates target concept presence in prompts and performs calibrated vector subtraction on text embeddings. It includes Co-Occurrence Encoding for multi-concept erasure and a visual feedback loop to address latent concept persistence.

Result: Achieved 93.58 H-score in object erasure, reduced explicit content to just 1 instance, and 8.09 H_a in style erasure with no quality degradation. Significantly outperformed state-of-the-art approaches across various erasure tasks.

Conclusion: Semantic Surgery provides a practical solution for safer text-to-image generation, functioning as both an effective concept erasure method and a built-in threat detection system while preserving locality and image quality.

Abstract: Concept erasure in text-to-image diffusion models is crucial for mitigating harmful content, yet existing methods often compromise generative quality. We introduce Semantic Surgery, a novel training-free, zero-shot framework for concept erasure that operates directly on text embeddings before the diffusion process. It dynamically estimates the presence of target concepts in a prompt and performs a calibrated vector subtraction to neutralize their influence at the source, enhancing both erasure completeness and locality. The framework includes a Co-Occurrence Encoding module for robust multi-concept erasure and a visual feedback loop to address latent concept persistence. As a training-free method, Semantic Surgery adapts dynamically to each prompt, ensuring precise interventions. Extensive experiments on object, explicit content, artistic style, and multi-celebrity erasure tasks show our method significantly outperforms state-of-the-art approaches. We achieve superior completeness and robustness while preserving locality and image quality (e.g., 93.58 H-score in object erasure, reducing explicit content to just 1 instance, and 8.09 H_a in style erasure with no quality degradation). This robustness also allows our framework to function as a built-in threat detection system, offering a practical solution for safer text-to-image generation.

Method: Integrates Retrieval-Augmented Generation (RAG) with Vision-Language Models (VLM), using a multimodal knowledge base with technical documentation, reference images, and domain guidelines. A hybrid text-image retriever with keyword-aware reranking provides relevant context for VLM inference.

Result: On 30 labeled blade images covering diverse damage categories, the RAG-grounded VLM achieved 100% correct classification, outperforming the same VLM without retrieval in both accuracy and precision.

Conclusion: The framework provides explainable and generalizable damage detection, enabling detection of unseen defects by leveraging domain knowledge rather than visual cues alone, offering a data-efficient solution for industrial inspection.

Abstract: Wind turbine blades operate in harsh environments, making timely damage detection essential for preventing failures and optimizing maintenance. Drone-based inspection and deep learning are promising, but typically depend on large, labeled datasets, which limit their ability to detect rare or evolving damage types. To address this, we propose a zero-shot-oriented inspection framework that integrates Retrieval-Augmented Generation (RAG) with Vision-Language Models (VLM). A multimodal knowledge base is constructed, comprising technical documentation, representative reference images, and domain-specific guidelines. A hybrid text-image retriever with keyword-aware reranking assembles the most relevant context to condition the VLM at inference, injecting domain knowledge without task-specific training. We evaluate the framework on 30 labeled blade images covering diverse damage categories. Although the dataset is small due to the difficulty of acquiring verified blade imagery, it covers multiple representative defect types. On this test set, the RAG-grounded VLM correctly classified all samples, whereas the same VLM without retrieval performed worse in both accuracy and precision. We further compare against open-vocabulary baselines and incorporate uncertainty Clopper-Pearson confidence intervals to account for the small-sample setting. Ablation studies indicate that the key advantage of the framework lies in explainability and generalizability: retrieved references ground the reasoning process and enable the detection of previously unseen defects by leveraging domain knowledge rather than relying solely on visual cues. This research contributes a data-efficient solution for industrial inspection that reduces dependence on extensive labeled datasets.

Method: Created a dataset of top-view pasture images across 19 Australian locations with paired ground measurements including biomass components, vegetation height, and NDVI data.

Result: Released a multidimensional dataset combining visual, spectral, and structural information for precision grazing management applications.

Conclusion: The dataset enables new possibilities for machine learning approaches to pasture biomass estimation and is available through a Kaggle competition.

Abstract: Accurate estimation of pasture biomass is important for decision-making in livestock production systems. Estimates of pasture biomass can be used to manage stocking rates to maximise pasture utilisation, while minimising the risk of overgrazing and promoting overall system health. We present a comprehensive dataset of 1,162 annotated top-view images of pastures collected across 19 locations in Australia. The images were taken across multiple seasons and include a range of temperate pasture species. Each image captures a 70cm * 30cm quadrat and is paired with on-ground measurements including biomass sorted by component (green, dead, and legume fraction), vegetation height, and Normalized Difference Vegetation Index (NDVI) from Active Optical Sensors (AOS). The multidimensional nature of the data, which combines visual, spectral, and structural information, opens up new possibilities for advancing the use of precision grazing management. The dataset is released and hosted in a Kaggle competition that challenges the international Machine Learning community with the task of pasture biomass estimation. The dataset is available on the official Kaggle webpage: https://www.kaggle.com/competitions/csiro-biomass

Method: Replaces scene-specific autoencoders with a generalized autoencoder pre-trained on ScanNet dataset, using fixed compact latent space across all scenes without scene-specific training.

Result: Achieves efficiency boost in language field construction while delivering comparable or better querying performance than LangSplat, with optimal latent embedding dimensions validated through ablation studies.

Conclusion: Generalized embeddings can efficiently support open-vocabulary querying in novel 3D scenes, enabling scalable real-time interactive 3D AI applications.

Abstract: Modeling open-vocabulary language fields in 3D is essential for intuitive human-AI interaction and querying within physical environments. State-of-the-art approaches, such as LangSplat, leverage 3D Gaussian Splatting to efficiently construct these language fields, encoding features distilled from high-dimensional models like CLIP. However, this efficiency is currently offset by the requirement to train a scene-specific language autoencoder for feature compression, introducing a costly, per-scene optimization bottleneck that hinders deployment scalability. In this work, we introduce Gen-LangSplat, that eliminates this requirement by replacing the scene-wise autoencoder with a generalized autoencoder, pre-trained extensively on the large-scale ScanNet dataset. This architectural shift enables the use of a fixed, compact latent space for language features across any new scene without any scene-specific training. By removing this dependency, our entire language field construction process achieves a efficiency boost while delivering querying performance comparable to, or exceeding, the original LangSplat method. To validate our design choice, we perform a thorough ablation study empirically determining the optimal latent embedding dimension and quantifying representational fidelity using Mean Squared Error and cosine similarity between the original and reprojected 512-dimensional CLIP embeddings. Our results demonstrate that generalized embeddings can efficiently and accurately support open-vocabulary querying in novel 3D scenes, paving the way for scalable, real-time interactive 3D AI applications.

Method: PPE introduces disentangled encoding of 3D positions in the token dimension, allowing compressed tokens to encapsulate different positions from multiple original tokens. Supports cascade clustering for progressive token compression.

Result: Applied to state-of-the-art token merging framework, PPE achieves 2%-5% improvements across multiple vision-language benchmarks including MMBench, TextVQA, and VideoMME.

Conclusion: Preserving positional cues is critical for efficient and effective MLLM reasoning, and PPE provides a generic solution that can be seamlessly integrated into existing token merging methods.

Abstract: Multimodal large language models (MLLMs) have achieved strong performance on vision-language tasks, yet often suffer from inefficiencies due to redundant visual tokens. Existing token merging methods reduce sequence length but frequently disrupt spatial layouts and temporal continuity by disregarding positional relationships. In this work, we propose a novel encoding operator dubbed as \textbf{P}ositional \textbf{P}reservation \textbf{E}mbedding (\textbf{PPE}), which has the main hallmark of preservation of spatiotemporal structure during visual token compression. PPE explicitly introduces the disentangled encoding of 3D positions in the token dimension, enabling each compressed token to encapsulate different positions from multiple original tokens. Furthermore, we show that PPE can effectively support cascade clustering – a progressive token compression strategy that leads to better performance retention. PPE is a parameter-free and generic operator that can be seamlessly integrated into existing token merging methods without any adjustments. Applied to state-of-the-art token merging framework, PPE achieves consistent improvements of $2%\sim5%$ across multiple vision-language benchmarks, including MMBench (general vision understanding), TextVQA (layout understanding) and VideoMME (temporal understanding). These results demonstrate that preserving positional cues is critical for efficient and effective MLLM reasoning.

Method: Used Bi-Encoder networks with ResNet-50 and Vision Transformer backbones trained with contrastive loss on three verification tasks: fingerprint-to-fingerprint, iris-to-iris, and cross-modal fingerprint-to-iris matching using 274 subjects with ~100k fingerprints and 7k iris images.

Result: Iris-to-iris matching achieved 91 ROC AUC score, showing left and right irises are correlated. Fingerprint models confirmed positive intra-subject correlation. Cross-modal matching performed only slightly above chance. This is the first work using Vision Transformers for biometric matching.

Conclusion: The findings challenge independence assumptions of biometrics, showing clear correlation between left and right irises and confirming fingerprint correlations. More data and sophisticated pipelines are needed for compelling cross-modal results. Future work will extend to other biometrics.

Abstract: There has been a historic assumption that the biometrics of an individual are statistically uncorrelated. We test this assumption by training Bi-Encoder networks on three verification tasks, including fingerprint-to-fingerprint matching, iris-to-iris matching, and cross-modal fingerprint-to-iris matching using 274 subjects with $\sim$100k fingerprints and 7k iris images. We trained ResNet-50 and Vision Transformer backbones in Bi-Encoder architectures such that the contrastive loss between images sampled from the same individual is minimized. The iris ResNet architecture reaches 91 ROC AUC score for iris-to-iris matching, providing clear evidence that the left and right irises of an individual are correlated. Fingerprint models reproduce the positive intra-subject suggested by prior work in this space. This is the first work attempting to use Vision Transformers for this matching. Cross-modal matching rises only slightly above chance, which suggests that more data and a more sophisticated pipeline is needed to obtain compelling results. These findings continue challenge independence assumptions of biometrics and we plan to extend this work to other biometrics in the future. Code available: https://github.com/MatthewSo/bio_fingerprints_iris.

Method: Learns distinct codebook groups for different image categories and assigns independent codebooks to each token, recording codebook group membership with minimal bits to reduce loss when decreasing codebook size.

Result: Achieves 93.51% average accuracy for reconstructed images at 0.05 bpp on face recognition datasets, enabling larger total codebooks under lower overall bpp.

Conclusion: The method enhances expressive capability and reconstruction performance for face images, is generalizable to existing codebook-based approaches, and effectively addresses limitations of global codebook strategies.

Abstract: With the ever-increasing volume of visual data, the efficient and lossless transmission, along with its subsequent interpretation and understanding, has become a critical bottleneck in modern information systems. The emerged codebook-based solution utilize a globally shared codebook to quantize and dequantize each token, controlling the bpp by adjusting the number of tokens or the codebook size. However, for facial images, which are rich in attributes, such global codebook strategies overlook both the category-specific correlations within images and the semantic differences among tokens, resulting in suboptimal performance, especially at low bpp. Motivated by these observations, we propose a Switchable Token-Specific Codebook Quantization for face image compression, which learns distinct codebook groups for different image categories and assigns an independent codebook to each token. By recording the codebook group to which each token belongs with a small number of bits, our method can reduce the loss incurred when decreasing the size of each codebook group. This enables a larger total number of codebooks under a lower overall bpp, thereby enhancing the expressive capability and improving reconstruction performance. Owing to its generalizable design, our method can be integrated into any existing codebook-based representation learning approach and has demonstrated its effectiveness on face recognition datasets, achieving an average accuracy of 93.51% for reconstructed images at 0.05 bpp.

Method: Retain original model blocks while interleaving multimodal self-attention blocks throughout networks, enabling double fusion mechanism that preserves base model strengths while catalyzing synergistic fusion of semantic and spatial representations.

Result: Achieves strong benchmarks: 0.91 on GenEval, 82.16 on DPG-Bench, 6.06 on GEditBench, and 3.77 on ImgEdit-Bench for various multimodal tasks including text-to-image generation and image editing.

Conclusion: Strategic fusion of publicly available specialized models with minimal training can achieve competitive multimodal performance, supporting future research through full release of code, models, and datasets.

Abstract: Unified multimodal models have recently shown remarkable gains in both capability and versatility, yet most leading systems are still trained from scratch and require substantial computational resources. In this paper, we show that competitive performance can be obtained far more efficiently by strategically fusing publicly available models specialized for either generation or understanding. Our key design is to retain the original blocks while additionally interleaving multimodal self-attention blocks throughout the networks. This double fusion mechanism (1) effectively enables rich multi-modal fusion while largely preserving the original strengths of the base models, and (2) catalyzes synergistic fusion of high-level semantic representations from the understanding encoder with low-level spatial signals from the generation encoder. By training with only ~ 35B tokens, this approach achieves strong results across multiple benchmarks: 0.91 on GenEval for compositional text-to-image generation, 82.16 on DPG-Bench for complex text-to-image generation, 6.06 on GEditBench, and 3.77 on ImgEdit-Bench for image editing. By fully releasing the entire suite of code, model weights, and datasets, we hope to support future research on unified multimodal modeling.

Method: FAME injects debiasing tokens into text encoders for fairness embeddings, and integrates fairness modulation into temporal self-attention (using region-constrained masks with time decay) and cross-attention (using fairness-sensitive similarity masks) to prevent motion corruption and temporal inconsistency.

Result: Extensive experiments on the FairVE benchmark show FAME achieves stronger fairness alignment and semantic fidelity compared to existing video editing baselines.

Conclusion: FAME effectively mitigates gender biases in video editing while preserving temporal coherence and prompt alignment through its attention modulation framework.

Abstract: Training-free video editing (VE) models tend to fall back on gender stereotypes when rendering profession-related prompts. We propose \textbf{FAME} for \textit{Fairness-aware Attention-modulated Video Editing} that mitigates profession-related gender biases while preserving prompt alignment and temporal consistency for coherent VE. We derive fairness embeddings from existing minority representations by softly injecting debiasing tokens into the text encoder. Simultaneously, FAME integrates fairness modulation into both temporal self attention and prompt-to-region cross attention to mitigate the motion corruption and temporal inconsistency caused by directly introducing fairness cues. For temporal self attention, FAME introduces a region constrained attention mask combined with time decay weighting, which enhances intra-region coherence while suppressing irrelevant inter-region interactions. For cross attention, it reweights tokens to region matching scores by incorporating fairness sensitive similarity masks derived from debiasing prompt embeddings. Together, these modulations keep fairness-sensitive semantics tied to the right visual regions and prevent temporal drift across frames. Extensive experiments on new VE fairness-oriented benchmark \textit{FairVE} demonstrate that FAME achieves stronger fairness alignment and semantic fidelity, surpassing existing VE baselines.

Method: The survey adopts a modality-driven perspective, covering five core visual and vision-language modalities. It examines how imaging physics and data representation shape interaction design, and analyzes key techniques for alignment, integration, and knowledge transfer to address modality heterogeneity, distribution shifts, and semantic gaps.

Result: The paper systematically assesses advances in training paradigms, architectures, and task-specific adaptation strategies, and evaluates representative multimodal GFMs across ten downstream tasks. Real-world case studies demonstrate practical applications in land cover mapping, agricultural monitoring, disaster response, climate studies, and geospatial intelligence.

Conclusion: The survey outlines pressing challenges in domain generalization, interpretability, efficiency, and privacy, while charting promising avenues for future research in multimodal geospatial foundation models.

Abstract: Foundation models have transformed natural language processing and computer vision, and their impact is now reshaping remote sensing image analysis. With powerful generalization and transfer learning capabilities, they align naturally with the multimodal, multi-resolution, and multi-temporal characteristics of remote sensing data. To address unique challenges in the field, multimodal geospatial foundation models (GFMs) have emerged as a dedicated research frontier. This survey delivers a comprehensive review of multimodal GFMs from a modality-driven perspective, covering five core visual and vision-language modalities. We examine how differences in imaging physics and data representation shape interaction design, and we analyze key techniques for alignment, integration, and knowledge transfer to tackle modality heterogeneity, distribution shifts, and semantic gaps. Advances in training paradigms, architectures, and task-specific adaptation strategies are systematically assessed alongside a wealth of emerging benchmarks. Representative multimodal visual and vision-language GFMs are evaluated across ten downstream tasks, with insights into their architectures, performance, and application scenarios. Real-world case studies, spanning land cover mapping, agricultural monitoring, disaster response, climate studies, and geospatial intelligence, demonstrate the practical potential of GFMs. Finally, we outline pressing challenges in domain generalization, interpretability, efficiency, and privacy, and chart promising avenues for future research.

Method: Proposes VALA (Variational Alignment for Latent Anchors) - a variational alignment module with contrastive learning objective that adaptively selects key frames and compresses latent features into semantic anchors to preserve content and temporal coherence.

Result: Extensive experiments show VALA achieves state-of-the-art performance in inversion fidelity, editing quality, and temporal consistency, while offering improved efficiency over prior methods.

Conclusion: VALA provides an effective solution for training-free video editing by adaptively selecting key frames and compressing latent representations into semantic anchors, achieving better temporal consistency and efficiency.

Abstract: Recent advances in training-free video editing have enabled lightweight and precise cross-frame generation by leveraging pre-trained text-to-image diffusion models. However, existing methods often rely on heuristic frame selection to maintain temporal consistency during DDIM inversion, which introduces manual bias and reduces the scalability of end-to-end inference. In this paper, we propose~\textbf{VALA} (\textbf{V}ariational \textbf{A}lignment for \textbf{L}atent \textbf{A}nchors), a variational alignment module that adaptively selects key frames and compresses their latent features into semantic anchors for consistent video editing. To learn meaningful assignments, VALA propose a variational framework with a contrastive learning objective. Therefore, it can transform cross-frame latent representations into compressed latent anchors that preserve both content and temporal coherence. Our method can be fully integrated into training-free text-to-image based video editing models. Extensive experiments on real-world video editing benchmarks show that VALA achieves state-of-the-art performance in inversion fidelity, editing quality, and temporal consistency, while offering improved efficiency over prior methods.

Method: The approach uses a spatio-temporal disentangled architecture for 4D dynamic occupancy generation. It incorporates Gaussian splatting-based sparse point map rendering for multi-view video generation and sensor-aware embedding for realistic LiDAR simulation.

Result: Extensive experiments show superior generation fidelity and scalability compared to existing approaches, with validated practical value in downstream autonomous driving tasks.

Conclusion: The proposed UniScene framework successfully addresses the data scarcity problem in occupancy-centric driving scene generation and demonstrates strong performance across multiple modalities.

Abstract: Driving scene generation is a critical domain for autonomous driving, enabling downstream applications, including perception and planning evaluation. Occupancy-centric methods have recently achieved state-of-the-art results by offering consistent conditioning across frames and modalities; however, their performance heavily depends on annotated occupancy data, which still remains scarce. To overcome this limitation, we curate Nuplan-Occ, the largest semantic occupancy dataset to date, constructed from the widely used Nuplan benchmark. Its scale and diversity facilitate not only large-scale generative modeling but also autonomous driving downstream applications. Based on this dataset, we develop a unified framework that jointly synthesizes high-quality semantic occupancy, multi-view videos, and LiDAR point clouds. Our approach incorporates a spatio-temporal disentangled architecture to support high-fidelity spatial expansion and temporal forecasting of 4D dynamic occupancy. To bridge modal gaps, we further propose two novel techniques: a Gaussian splatting-based sparse point map rendering strategy that enhances multi-view video generation, and a sensor-aware embedding strategy that explicitly models LiDAR sensor properties for realistic multi-LiDAR simulation. Extensive experiments demonstrate that our method achieves superior generation fidelity and scalability compared to existing approaches, and validates its practical value in downstream tasks. Repo: https://github.com/Arlo0o/UniScene-Unified-Occupancy-centric-Driving-Scene-Generation/tree/v2

Method: VoMP aggregates per-voxel multi-view features from renderable 3D objects, uses a trained Geometry Transformer to predict material latent codes on a physically plausible manifold, and leverages a novel annotation pipeline combining 3D datasets, material databases, and vision-language models.

Result: VoMP significantly outperforms prior methods in both accuracy and speed for estimating volumetric mechanical properties.

Conclusion: The method successfully automates the prediction of physically valid material properties across 3D object volumes, demonstrating superior performance over existing approaches.

Abstract: Physical simulation relies on spatially-varying mechanical properties, often laboriously hand-crafted. VoMP is a feed-forward method trained to predict Young’s modulus ($E$), Poisson’s ratio ($\nu$), and density ($\rho$) throughout the volume of 3D objects, in any representation that can be rendered and voxelized. VoMP aggregates per-voxel multi-view features and passes them to our trained Geometry Transformer to predict per-voxel material latent codes. These latents reside on a manifold of physically plausible materials, which we learn from a real-world dataset, guaranteeing the validity of decoded per-voxel materials. To obtain object-level training data, we propose an annotation pipeline combining knowledge from segmented 3D datasets, material databases, and a vision-language model, along with a new benchmark. Experiments show that VoMP estimates accurate volumetric properties, far outperforming prior art in accuracy and speed.

Method: Proposes SceneDecorator with two key components: VLM-Guided Scene Planning for narrative coherence in a global-to-local manner, and Long-Term Scene-Sharing Attention to maintain scene consistency and subject diversity across stories.

Result: Extensive experiments show superior performance of SceneDecorator in maintaining scene consistency and narrative coherence across generated stories.

Conclusion: SceneDecorator demonstrates potential to unleash creativity in arts, films, and games by effectively addressing scene-level consistency challenges in story generation.

Abstract: Recent text-to-image models have revolutionized image generation, but they still struggle with maintaining concept consistency across generated images. While existing works focus on character consistency, they often overlook the crucial role of scenes in storytelling, which restricts their creativity in practice. This paper introduces scene-oriented story generation, addressing two key challenges: (i) scene planning, where current methods fail to ensure scene-level narrative coherence by relying solely on text descriptions, and (ii) scene consistency, which remains largely unexplored in terms of maintaining scene consistency across multiple stories. We propose SceneDecorator, a training-free framework that employs VLM-Guided Scene Planning to ensure narrative coherence across different scenes in a ``global-to-local’’ manner, and Long-Term Scene-Sharing Attention to maintain long-term scene consistency and subject diversity across generated stories. Extensive experiments demonstrate the superior performance of SceneDecorator, highlighting its potential to unleash creativity in the fields of arts, films, and games.

Method: LoMix mixes multi-scale decoder logits using four lightweight fusion operators (addition, multiplication, concatenation, attention-based fusion) and co-optimizes loss weights with network parameters through differentiable optimization.

Result: Improves DICE by +4.2% over single-output supervision, +2.2% over deep supervision, and +1.5% over equally weighted additive fusion on Synapse dataset. Advantage grows to +9.23% with scarce training data.

Conclusion: Learnable weighted mixed-scale fusion generalizes broadly across benchmarks and networks, providing data-efficient, interpretable, and overhead-free performance improvements for medical image segmentation.

Abstract: U-shaped networks output logits at multiple spatial scales, each capturing a different blend of coarse context and fine detail. Yet, training still treats these logits in isolation - either supervising only the final, highest-resolution logits or applying deep supervision with identical loss weights at every scale - without exploring mixed-scale combinations. Consequently, the decoder output misses the complementary cues that arise only when coarse and fine predictions are fused. To address this issue, we introduce LoMix (Logits Mixing), a NAS-inspired, differentiable plug-and-play module that generates new mixed-scale outputs and learns how exactly each of them should guide the training process. More precisely, LoMix mixes the multi-scale decoder logits with four lightweight fusion operators: addition, multiplication, concatenation, and attention-based weighted fusion, yielding a rich set of synthetic mutant maps. Every original or mutant map is given a softplus loss weight that is co-optimized with network parameters, mimicking a one-step architecture search that automatically discovers the most useful scales, mixtures, and operators. Plugging LoMix into recent U-shaped architectures (i.e., PVT-V2-B2 backbone with EMCAD decoder) on Synapse 8-organ dataset improves DICE by +4.2% over single-output supervision, +2.2% over deep supervision, and +1.5% over equally weighted additive fusion, all with zero inference overhead. When training data are scarce (e.g., one or two labeled scans), the advantage grows to +9.23%, underscoring LoMix’s data efficiency. Across four benchmarks and diverse U-shaped networks, LoMiX improves DICE by up to +13.5% over single-output supervision, confirming that learnable weighted mixed-scale fusion generalizes broadly while remaining data efficient, fully interpretable, and overhead-free at inference. Our code is available at https://github.com/SLDGroup/LoMix.

Method: Two-phase approach: 1) Single-motion learning phase using static-dynamic decoupled tuning to disentangle motion from appearance and learn motion-specific modules; 2) Multi-motion composition phase using plug-and-play divide-and-merge strategy to compose learned motions without additional training by spatially isolating their influence during denoising.

Result: CoMo achieves state-of-the-art performance in controllable video generation, significantly advancing multi-motion customization capabilities. A new benchmark and evaluation metric were also introduced for multi-motion fidelity and blending assessment.

Conclusion: CoMo successfully addresses the challenges of compositional motion customization, enabling synthesis of multiple distinct motions in videos through its two-phase approach and spatial isolation strategy, representing a significant advancement in controllable video generation.

Abstract: While recent text-to-video models excel at generating diverse scenes, they struggle with precise motion control, particularly for complex, multi-subject motions. Although methods for single-motion customization have been developed to address this gap, they fail in compositional scenarios due to two primary challenges: motion-appearance entanglement and ineffective multi-motion blending. This paper introduces CoMo, a novel framework for $\textbf{compositional motion customization}$ in text-to-video generation, enabling the synthesis of multiple, distinct motions within a single video. CoMo addresses these issues through a two-phase approach. First, in the single-motion learning phase, a static-dynamic decoupled tuning paradigm disentangles motion from appearance to learn a motion-specific module. Second, in the multi-motion composition phase, a plug-and-play divide-and-merge strategy composes these learned motions without additional training by spatially isolating their influence during the denoising process. To facilitate research in this new domain, we also introduce a new benchmark and a novel evaluation metric designed to assess multi-motion fidelity and blending. Extensive experiments demonstrate that CoMo achieves state-of-the-art performance, significantly advancing the capabilities of controllable video generation. Our project page is at https://como6.github.io/.

Method: Three-component framework: PoGE uses Transformer-based sparse CNN for geometry reconstruction, PAE employs geometry-guided recoloring with DA-KNN for attribute enhancement, and PoAE uses attribute residual prediction with W-MSE loss.

Result: Outperformed existing methods on 8iVFB, Owlii, and MVUB datasets with 9.98 dB BD-PSNR gain, 90.98% bitrate savings for geometry, and 3.67 dB PSNR improvement with 56.88% bitrate savings for attributes.

Conclusion: UGAE effectively enhances compressed point cloud quality through unified geometry and attribute enhancement, achieving superior performance and perceptual quality.

Abstract: Lossy compression of point clouds reduces storage and transmission costs; however, it inevitably leads to irreversible distortion in geometry structure and attribute information. To address these issues, we propose a unified geometry and attribute enhancement (UGAE) framework, which consists of three core components: post-geometry enhancement (PoGE), pre-attribute enhancement (PAE), and post-attribute enhancement (PoAE). In PoGE, a Transformer-based sparse convolutional U-Net is used to reconstruct the geometry structure with high precision by predicting voxel occupancy probabilities. Building on the refined geometry structure, PAE introduces an innovative enhanced geometry-guided recoloring strategy, which uses a detail-aware K-Nearest Neighbors (DA-KNN) method to achieve accurate recoloring and effectively preserve high-frequency details before attribute compression. Finally, at the decoder side, PoAE uses an attribute residual prediction network with a weighted mean squared error (W-MSE) loss to enhance the quality of high-frequency regions while maintaining the fidelity of low-frequency regions. UGAE significantly outperformed existing methods on three benchmark datasets: 8iVFB, Owlii, and MVUB. Compared to the latest G-PCC test model (TMC13v29), UGAE achieved an average BD-PSNR gain of 9.98 dB and 90.98% BD-bitrate savings for geometry under the D1 metric, as well as a 3.67 dB BD-PSNR improvement with 56.88% BD-bitrate savings for attributes on the Y component. Additionally, it improved perceptual quality significantly.

Method: Proposes nested AR architecture with multi-scale modules in hierarchical order, where larger-scale modules are conditioned on outputs from smaller-scale modules. Uses flow matching loss for continuous tokens and develops coordination objectives for multi-scale training.

Result: Achieves competitive image generation performance while significantly lowering computational cost compared to standard AR models.

Conclusion: NestAR provides an efficient alternative to traditional AR models for image generation, reducing complexity while maintaining performance and improving diversity.

Abstract: AutoRegressive (AR) models have demonstrated competitive performance in image generation, achieving results comparable to those of diffusion models. However, their token-by-token image generation mechanism remains computationally intensive and existing solutions such as VAR often lead to limited sample diversity. In this work, we propose a Nested AutoRegressive~(NestAR) model, which proposes nested AutoRegressive architectures in generating images. NestAR designs multi-scale modules in a hierarchical order. These different scaled modules are constructed in an AR architecture, where one larger-scale module is conditioned on outputs from its previous smaller-scale module. Within each module, NestAR uses another AR structure to generate ``patches’’ of tokens. The proposed nested AR architecture reduces the overall complexity from $\mathcal{O}(n)$ to $\mathcal{O}(\log n)$ in generating $n$ image tokens, as well as increases image diversities. NestAR further incorporates flow matching loss to use continuous tokens, and develops objectives to coordinate these multi-scale modules in model training. NestAR achieves competitive image generation performance while significantly lowering computational cost.

Method: Proposes HieraMamba with hierarchical architecture using Anchor-MambaPooling (AMP) blocks that leverage Mamba’s selective scanning to generate compact anchor tokens summarizing video content at multiple scales, combined with anchor-conditioned and segment-pooled contrastive losses.

Result: HieraMamba achieves new state-of-the-art performance on Ego4D-NLQ, MAD, and TACoS benchmarks, demonstrating precise and temporally faithful localization in long, untrimmed videos.

Conclusion: The hierarchical approach with AMP blocks effectively preserves temporal structure and semantic richness across scales, enabling superior video temporal grounding in challenging long-video scenarios.

Abstract: Video temporal grounding, the task of localizing the start and end times of a natural language query in untrimmed video, requires capturing both global context and fine-grained temporal detail. This challenge is particularly pronounced in long videos, where existing methods often compromise temporal fidelity by over-downsampling or relying on fixed windows. We present HieraMamba, a hierarchical architecture that preserves temporal structure and semantic richness across scales. At its core are Anchor-MambaPooling (AMP) blocks, which utilize Mamba’s selective scanning to produce compact anchor tokens that summarize video content at multiple granularities. Two complementary objectives, anchor-conditioned and segment-pooled contrastive losses, encourage anchors to retain local detail while remaining globally discriminative. HieraMamba sets a new state-of-the-art on Ego4D-NLQ, MAD, and TACoS, demonstrating precise, temporally faithful localization in long, untrimmed videos.

Method: Transform input images into MIND feature space before feeding to registration model, plus a special ensembling strategy for consistent performance improvement.

Result: Significantly improved robustness for brain registration across different contrasts and modalities not seen during training.

Conclusion: Simple feature space transformation and ensembling can substantially enhance generalization capability of deep learning registration methods.

Abstract: Deep learning based deformable registration methods have become popular in recent years. However, their ability to generalize beyond training data distribution can be poor, significantly hindering their usability. LUMIR brain registration challenge for Learn2Reg 2025 aims to advance the field by evaluating the performance of the registration on contrasts and modalities different from those included in the training set. Here we describe our submission to the challenge, which proposes a very simple idea for significantly improving robustness by transforming the images into MIND feature space before feeding them into the model. In addition, a special ensembling strategy is proposed that shows a small but consistent improvement.

Method: 1) Unified spatio-temporal Gaussian representation optimizing primitives in 4D domain; 2) Geometric constraint derived from optical flow to enhance temporal coherence and constrain 3D structure; 3) Multi-resolution rational orthogonal wavelet constraint to separate endoscopic details and enhance rendering performance.

Result: Extensive evaluations on EndoNeRF and StereoMIS datasets demonstrate that EndoWave achieves state-of-the-art reconstruction quality and visual accuracy compared to baseline methods.

Conclusion: The proposed EndoWave framework effectively addresses the challenges of endoscopic 3D reconstruction by incorporating geometric and wavelet-based constraints, achieving superior performance in surgical scenarios.

Abstract: In robot-assisted minimally invasive surgery, accurate 3D reconstruction from endoscopic video is vital for downstream tasks and improved outcomes. However, endoscopic scenarios present unique challenges, including photometric inconsistencies, non-rigid tissue motion, and view-dependent highlights. Most 3DGS-based methods that rely solely on appearance constraints for optimizing 3DGS are often insufficient in this context, as these dynamic visual artifacts can mislead the optimization process and lead to inaccurate reconstructions. To address these limitations, we present EndoWave, a unified spatio-temporal Gaussian Splatting framework by incorporating an optical flow-based geometric constraint and a multi-resolution rational wavelet supervision. First, we adopt a unified spatio-temporal Gaussian representation that directly optimizes primitives in a 4D domain. Second, we propose a geometric constraint derived from optical flow to enhance temporal coherence and effectively constrain the 3D structure of the scene. Third, we propose a multi-resolution rational orthogonal wavelet as a constraint, which can effectively separate the details of the endoscope and enhance the rendering performance. Extensive evaluations on two real surgical datasets, EndoNeRF and StereoMIS, demonstrate that our method EndoWave achieves state-of-the-art reconstruction quality and visual accuracy compared to the baseline method.

Method: The authors identify three key guidelines (positional coherence, full frequency utilization, preservation of textual priors) and propose two variants: Multi-Head RoPE (MHRoPE) and MRoPE-Interleave (MRoPE-I), which are plug-and-play methods requiring no architectural changes.

Result: The proposed methods consistently outperform existing approaches across diverse benchmarks, showing significant improvements in both general and fine-grained multimodal understanding.

Conclusion: The paper demonstrates that systematic analysis of multimodal position encoding leads to effective solutions that enhance vision-language model performance through simple, plug-and-play modifications to RoPE.

Abstract: Multimodal position encoding is essential for vision-language models, yet there has been little systematic investigation into multimodal position encoding. We conduct a comprehensive analysis of multimodal Rotary Positional Embedding (RoPE) by examining its two core components: position design and frequency allocation. Through extensive experiments, we identify three key guidelines: positional coherence, full frequency utilization, and preservation of textual priors-ensuring unambiguous layout, rich representation, and faithful transfer from the pre-trained LLM. Based on these insights, we propose Multi-Head RoPE (MHRoPE) and MRoPE-Interleave (MRoPE-I), two simple and plug-and-play variants that require no architectural changes. Our methods consistently outperform existing approaches across diverse benchmarks, with significant improvements in both general and fine-grained multimodal understanding. Code will be avaliable at https://github.com/JJJYmmm/Multimodal-RoPEs.

Method: Theoretical reformulation of stochastic differential equations for generalized diffusion bridge, with analytical formulas for forward/reverse processes. Uses residuals from given distributions to modulate noise injection/removal for adaptive restoration.

Result: Achieves state-of-the-art performance across diverse image restoration tasks, both qualitatively and quantitatively. Demonstrates that existing bridge models are special cases of RDBM.

Conclusion: RDBM provides a unified framework for diffusion bridge models with adaptive restoration capabilities, outperforming existing methods while preserving undegraded regions effectively.

Abstract: Diffusion bridge models establish probabilistic paths between arbitrary paired distributions and exhibit great potential for universal image restoration. Most existing methods merely treat them as simple variants of stochastic interpolants, lacking a unified analytical perspective. Besides, they indiscriminately reconstruct images through global noise injection and removal, inevitably distorting undegraded regions due to imperfect reconstruction. To address these challenges, we propose the Residual Diffusion Bridge Model (RDBM). Specifically, we theoretically reformulate the stochastic differential equations of generalized diffusion bridge and derive the analytical formulas of its forward and reverse processes. Crucially, we leverage the residuals from given distributions to modulate the noise injection and removal, enabling adaptive restoration of degraded regions while preserving intact others. Moreover, we unravel the fundamental mathematical essence of existing bridge models, all of which are special cases of RDBM and empirically demonstrate the optimality of our proposed models. Extensive experiments are conducted to demonstrate the state-of-the-art performance of our method both qualitatively and quantitatively across diverse image restoration tasks. Code is publicly available at https://github.com/MiliLab/RDBM.

Method: Uses deterministic and learned strategies for time series quantization, then employs masked correlation learning to align discrete image and time series tokens in a unified representation space.

Result: Outperforms task-specific fusion by 6% in R² and 2% in RMSE on average, and exceeds baseline methods by 50% in R² and 12% in RMSE. Successfully generates consistent global temperature profiles from satellite imagery.

Conclusion: The proposed task-agnostic pretraining framework provides effective multimodal fusion with superior performance, robustness, and cross-modal generation capabilities, with insights into model behavior through gradient sensitivity analysis.

Abstract: We propose a task-agnostic framework for multimodal fusion of time series and single timestamp images, enabling cross-modal generation and robust downstream performance. Our approach explores deterministic and learned strategies for time series quantization and then leverages a masked correlation learning objective, aligning discrete image and time series tokens in a unified representation space. Instantiated in the Earth observation domain, the pretrained model generates consistent global temperature profiles from satellite imagery and is validated through counterfactual experiments. Across downstream tasks, our task-agnostic pretraining outperforms task-specific fusion by 6% in R$^2$ and 2% in RMSE on average, and exceeds baseline methods by 50% in R$^2$ and 12% in RMSE. Finally, we analyze gradient sensitivity across modalities, providing insights into model robustness. Code, data, and weights will be released under a permissive license.

Method: DeepSalt uses knowledge distillation and a novel Spectral Adaptation Unit to transfer high-resolution spectral insights from laboratory-based spectroscopy to satellite-based hyperspectral sensing, enabling domain adaptation between different spectral measurement modalities.

Result: DeepSalt achieves significant performance gains over methods without explicit domain adaptation, effectively generalizes to unseen geographic regions, and explains a substantial portion of the salinity variance in comprehensive empirical benchmarks.

Conclusion: The proposed framework successfully bridges the gap between laboratory precision and satellite scalability, enabling accurate, large-scale soil salinity estimation without extensive ground sampling through effective spectral domain adaptation.

Abstract: Soil salinization poses a significant threat to both ecosystems and agriculture because it limits plants’ ability to absorb water and, in doing so, reduces crop productivity. This phenomenon alters the soil’s spectral properties, creating a measurable relationship between salinity and light reflectance that enables remote monitoring. While laboratory spectroscopy provides precise measurements, its reliance on in-situ sampling limits scalability to regional or global levels. Conversely, hyperspectral satellite imagery enables wide-area observation but lacks the fine-grained interpretability of laboratory instruments. To bridge this gap, we introduce DeepSalt, a deep-learning-based spectral transfer framework that leverages knowledge distillation and a novel Spectral Adaptation Unit to transfer high-resolution spectral insights from laboratory-based spectroscopy to satellite-based hyperspectral sensing. Our approach eliminates the need for extensive ground sampling while enabling accurate, large-scale salinity estimation, as demonstrated through comprehensive empirical benchmarks. DeepSalt achieves significant performance gains over methods without explicit domain adaptation, underscoring the impact of the proposed Spectral Adaptation Unit and the knowledge distillation strategy. The model also effectively generalized to unseen geographic regions, explaining a substantial portion of the salinity variance.

Method: Analyzes both constructions through the lens of Total Least Squares line fitting to the power spectrum, revealing their fundamental similarities.

Result: The correction term in the 1995 approach becomes unnecessary, ensuring positive semi-definite tensors and enabling generalizations beyond quadrature filters.

Conclusion: Both structure tensor constructions are fundamentally similar when viewed through TLS, allowing simplifications and broader applicability with various filter types.

Abstract: This note presents a theoretical discussion of two structure tensor constructions: one proposed by Bigun and Granlund 1987, and the other by Granlund and Knutsson 1995. At first glance, these approaches may appear quite different–the former is implemented by averaging outer products of gradient filter responses, while the latter constructs the tensor from weighted outer products of tune-in frequency vectors of quadrature filters. We argue that when both constructions are viewed through the common lens of Total Least Squares (TLS) line fitting to the power spectrum, they can be reconciled to a large extent, and additional benefits emerge. From this perspective, the correction term introduced in Granlund and Knutsson 1995 becomes unnecessary. Omitting it ensures that the resulting tensor remains positive semi-definite, thereby simplifying the interpretation of its eigenvalues. Furthermore, this interpretation allows fitting more than a single 0rientation to the input by reinterpreting quadrature filter responses without relying on a structure tensor. It also removes the constraint that responses must originate strictly from quadrature filters, allowing the use of alternative filter types and non-angular tessellations. These alternatives include Gabor filters–which, although not strictly quadrature, are still suitable for structure tensor construction–even when they tessellate the spectrum in a Cartesian fashion, provided they are sufficiently concentrated.

Method: Adopt a physics-informed CycleGAN, previously successful in DCE-MRI quantification, for dynamic PET quantification.

Result: Experiments show accurate AIF predictions and parameter maps that closely resemble the reference.

Conclusion: The proposed method effectively reduces the burden of invasive AIF estimation in tracer kinetic modeling for dynamic PET.

Abstract: Tracer kinetic modeling serves a vital role in diagnosis, treatment planning, tracer development and oncology, but burdens practitioners with complex and invasive arterial input function estimation (AIF). We adopt a physics-informed CycleGAN showing promise in DCE-MRI quantification to dynamic PET quantification. Our experiments demonstrate sound AIF predictions and parameter maps closely resembling the reference.

Method: Progressively increase patch size during model training in two modes: resource-efficient (faster training) and performance (better results). Evaluated across 15 diverse 3D medical segmentation tasks with different architectures.

Result: Resource-efficient mode reduces training time to 44% while matching baseline performance. Performance mode achieves 1.28% relative mean Dice score improvement and reduces training time to 89%, with consistent gains across all 15 tasks.

Conclusion: The progressive patch size curriculum is a simple, broadly applicable strategy that improves both segmentation performance and training efficiency across diverse models and tasks, particularly beneficial for imbalanced segmentation problems.

Abstract: In this work, we introduce Progressive Growing of Patch Size, an automatic curriculum learning approach for 3D medical image segmentation. Our approach progressively increases the patch size during model training, resulting in an improved class balance for smaller patch sizes and accelerated convergence of the training process. We evaluate our curriculum approach in two settings: a resource-efficient mode and a performance mode, both regarding Dice score performance and computational costs across 15 diverse and popular 3D medical image segmentation tasks. The resource-efficient mode matches the Dice score performance of the conventional constant patch size sampling baseline with a notable reduction in training time to only 44%. The performance mode improves upon constant patch size segmentation results, achieving a statistically significant relative mean performance gain of 1.28% in Dice Score. Remarkably, across all 15 tasks, our proposed performance mode manages to surpass the constant patch size baseline in Dice Score performance, while simultaneously reducing training time to only 89%. The benefits are particularly pronounced for highly imbalanced tasks such as lesion segmentation tasks. Rigorous experiments demonstrate that our performance mode not only improves mean segmentation performance but also reduces performance variance, yielding more trustworthy model comparison. Furthermore, our findings reveal that the proposed curriculum sampling is not tied to a specific architecture but represents a broadly applicable strategy that consistently boosts performance across diverse segmentation models, including UNet, UNETR, and SwinUNETR. In summary, we show that this simple yet elegant transformation on input data substantially improves both Dice Score performance and training runtime, while being compatible across diverse segmentation backbones.

Method: Proposes depth-guided query generator using depth information and 2D detections to sample reference points from object surfaces. Introduces hybrid attention mechanism that fuses historical detection results with depth-guided queries to handle occluded objects.

Result: Outperforms baseline by 6.3% in mAP and 4.3% in NDS on nuScenes dataset, demonstrating significant improvement in 3D object detection performance.

Conclusion: The proposed depth-guided query generation with hybrid attention effectively improves 3D object detection by ensuring proper reference point sampling and handling occlusions through temporal fusion.

Abstract: 3D object detection from multi-view images in traffic scenarios has garnered significant attention in recent years. Many existing approaches rely on object queries that are generated from 3D reference points to localize objects. However, a limitation of these methods is that some reference points are often far from the target object, which can lead to false positive detections. In this paper, we propose a depth-guided query generator for 3D object detection (DQ3D) that leverages depth information and 2D detections to ensure that reference points are sampled from the surface or interior of the object. Furthermore, to address partially occluded objects in current frame, we introduce a hybrid attention mechanism that fuses historical detection results with depth-guided queries, thereby forming hybrid queries. Evaluation on the nuScenes dataset demonstrates that our method outperforms the baseline by 6.3% in terms of mean Average Precision (mAP) and 4.3% in the NuScenes Detection Score (NDS).

Method: Proposes Implicit Transferability Modeling (ITM) framework that implicitly models each model’s intrinsic transferability, coupled with Divide-and-Conquer Variational Approximation (DVA) strategy to efficiently approximate embedding space evolution.

Result: Extensive experiments on comprehensive benchmarks show ITM consistently outperforms existing methods in terms of stability, effectiveness, and efficiency across diverse training regimes and model types.

Conclusion: ITM enables better generalization across broader range of models and downstream tasks, advancing the pre-training and fine-tuning paradigm by facilitating deployment through accurate transferability estimation.

Abstract: Transferability estimation identifies the best pre-trained models for downstream tasks without incurring the high computational cost of full fine-tuning. This capability facilitates deployment and advances the pre-training and fine-tuning paradigm. However, existing methods often struggle to accurately assess transferability for emerging pre-trained models with diverse architectures, training strategies, and task alignments. In this work, we propose Implicit Transferability Modeling (ITM), a novel framework that implicitly models each model’s intrinsic transferability, coupled with a Divide-and-Conquer Variational Approximation (DVA) strategy to efficiently approximate embedding space evolution. This design enables generalization across a broader range of models and downstream tasks. Extensive experiments on a comprehensive benchmark–spanning extensive training regimes and a wider variety of model types–demonstrate that ITM consistently outperforms existing methods in terms of stability, effectiveness, and efficiency.

Method: Projects camera and LiDAR features into unified BEV space, enhances them with window-based attention, and uses adaptive gated fusion module based on cross-modal attention to selectively integrate reliable patterns for robust detection.

Result: Achieves 93.92% accuracy on KITTI dataset and outperforms baseline by 24.88% on the challenging E3D dataset, demonstrating superior robustness to unreliable modal information in complex industrial scenes.

Conclusion: AG-Fusion effectively addresses performance degradation in challenging scenarios by adaptively selecting reliable cross-modal patterns, making it suitable for complex industrial applications like excavator operation monitoring.

Abstract: Multimodal camera-LiDAR fusion technology has found extensive application in 3D object detection, demonstrating encouraging performance. However, existing methods exhibit significant performance degradation in challenging scenarios characterized by sensor degradation or environmental disturbances. We propose a novel Adaptive Gated Fusion (AG-Fusion) approach that selectively integrates cross-modal knowledge by identifying reliable patterns for robust detection in complex scenes. Specifically, we first project features from each modality into a unified BEV space and enhance them using a window-based attention mechanism. Subsequently, an adaptive gated fusion module based on cross-modal attention is designed to integrate these features into reliable BEV representations robust to challenging environments. Furthermore, we construct a new dataset named Excavator3D (E3D) focusing on challenging excavator operation scenarios to benchmark performance in complex conditions. Our method not only achieves competitive performance on the standard KITTI dataset with 93.92% accuracy, but also significantly outperforms the baseline by 24.88% on the challenging E3D dataset, demonstrating superior robustness to unreliable modal information in complex industrial scenes.

Method: Uses hybrid neural representation with sparse graph from vision-language model features and feature field from 3D shape foundation models, then finds analogies in coarse-to-fine manner by graph alignment and feature field refinement.

Result: Method establishes accurate correspondences between complex scenes and enables successful trajectory and waypoint transfer applications.

Conclusion: The approach provides effective zero-shot 3D scene analogy capability using multimodal foundation models, supporting robot adaptation and planning in unseen environments.

Abstract: Connecting current observations with prior experiences helps robots adapt and plan in new, unseen 3D environments. Recently, 3D scene analogies have been proposed to connect two 3D scenes, which are smooth maps that align scene regions with common spatial relationships. These maps enable detailed transfer of trajectories or waypoints, potentially supporting demonstration transfer for imitation learning or task plan transfer across scenes. However, existing methods for the task require additional training and fixed object vocabularies. In this work, we propose to use multimodal foundation models for finding 3D scene analogies in a zero-shot, open-vocabulary setting. Central to our approach is a hybrid neural representation of scenes that consists of a sparse graph based on vision-language model features and a feature field derived from 3D shape foundation models. 3D scene analogies are then found in a coarse-to-fine manner, by first aligning the graph and refining the correspondence with feature fields. Our method can establish accurate correspondences between complex scenes, and we showcase applications in trajectory and waypoint transfer.

Method: Proposes ReconViaGen framework that integrates reconstruction priors into generative framework. Addresses two key issues: (1) insufficiency in cross-view connections when extracting multi-view image features, and (2) poor controllability of iterative denoising for local detail generation.

Result: Extensive experiments show ReconViaGen can reconstruct complete and accurate 3D models consistent with input views in both global structure and local details.

Conclusion: The proposed ReconViaGen successfully addresses limitations of existing methods by effectively integrating reconstruction priors into generative frameworks, achieving high consistency in 3D reconstruction from multi-view images.

Abstract: Existing multi-view 3D object reconstruction methods heavily rely on sufficient overlap between input views, where occlusions and sparse coverage in practice frequently yield severe reconstruction incompleteness. Recent advancements in diffusion-based 3D generative techniques offer the potential to address these limitations by leveraging learned generative priors to hallucinate invisible parts of objects, thereby generating plausible 3D structures. However, the stochastic nature of the inference process limits the accuracy and reliability of generation results, preventing existing reconstruction frameworks from integrating such 3D generative priors. In this work, we comprehensively analyze the reasons why diffusion-based 3D generative methods fail to achieve high consistency, including (a) the insufficiency in constructing and leveraging cross-view connections when extracting multi-view image features as conditions, and (b) the poor controllability of iterative denoising during local detail generation, which easily leads to plausible but inconsistent fine geometric and texture details with inputs. Accordingly, we propose ReconViaGen to innovatively integrate reconstruction priors into the generative framework and devise several strategies that effectively address these issues. Extensive experiments demonstrate that our ReconViaGen can reconstruct complete and accurate 3D models consistent with input views in both global structure and local details.Project page: https://jiahao620.github.io/reconviagen.

Method: Frames anomalous action recognition as zero-shot, language-grounded task using pre-trained vision-LLMs to convert video into text descriptions and score labels via textual entailment. Evaluates four open models on UCF-Crime and RWF-2000 datasets under prompting and privacy-preserving conditions.

Result: Few-shot exemplars can improve accuracy for some models but may increase false positives. Privacy filters, especially full-body GAN transforms, introduce inconsistencies that degrade accuracy. Models succeed at simple, spatially salient events but falter with noisy spatial cues and identity obfuscation.

Conclusion: Outlines paths to strengthen spatial grounding without task-specific training: structure-aware prompts, lightweight spatial memory, scene-graph or 3D-pose priors, and privacy methods preserving action-relevant geometry. Positions zero-shot language-grounded pipelines as adaptable building blocks for embodied video understanding.

Abstract: The widespread use of cameras in our society has created an overwhelming amount of video data, far exceeding the capacity for human monitoring. This presents a critical challenge for public safety and security, as the timely detection of anomalous or criminal events is crucial for effective response and prevention. The ability for an embodied agent to recognize unexpected events is fundamentally tied to its capacity for spatial reasoning. This paper investigates the spatial reasoning of vision-language models (VLMs) by framing anomalous action recognition as a zero-shot, language-grounded task, addressing the embodied perception challenge of interpreting dynamic 3D scenes from sparse 2D video. Specifically, we investigate whether small, pre-trained vision–LLMs can act as spatially-grounded, zero-shot anomaly detectors by converting video into text descriptions and scoring labels via textual entailment. We evaluate four open models on UCF-Crime and RWF-2000 under prompting and privacy-preserving conditions. Few-shot exemplars can improve accuracy for some models, but may increase false positives, and privacy filters – especially full-body GAN transforms – introduce inconsistencies that degrade accuracy. These results chart where current vision–LLMs succeed (simple, spatially salient events) and where they falter (noisy spatial cues, identity obfuscation). Looking forward, we outline concrete paths to strengthen spatial grounding without task-specific training: structure-aware prompts, lightweight spatial memory across clips, scene-graph or 3D-pose priors during description, and privacy methods that preserve action-relevant geometry. This positions zero-shot, language-grounded pipelines as adaptable building blocks for embodied, real-world video understanding. Our implementation for evaluating VLMs is publicly available at: https://github.com/pascalbenschopTU/VLLM_AnomalyRecognition

Method: Developed Medformer architecture for multitask learning and deep domain adaptation with dynamic input-output adaptation; introduced novel pretext tasks for self-supervised learning; validated using MedMNIST dataset.

Result: The model demonstrated proficiency in learning generalized features applicable to various downstream tasks, efficiently processing diverse medical image types from 2D X-rays to 3D MRIs.

Conclusion: Provides a scalable, adaptable framework that advances medical image analysis by reducing reliance on labeled data, enabling more accurate and efficient diagnostic tools in healthcare.

Abstract: This thesis works to address a pivotal challenge in medical image analysis: the reliance on extensive labeled datasets, which are often limited due to the need for expert annotation and constrained by privacy and legal issues. By focusing on the development of self-supervised learning techniques and domain adaptation methods, this research aims to circumvent these limitations, presenting a novel approach to enhance the utility and efficacy of deep learning in medical imaging. Central to this thesis is the development of the Medformer, an innovative neural network architecture designed for multitask learning and deep domain adaptation. This model is adept at pre-training on diverse medical image datasets, handling varying sizes and modalities, and is equipped with a dynamic input-output adaptation mechanism. This enables efficient processing and integration of a wide range of medical image types, from 2D X-rays to complex 3D MRIs, thus mitigating the dependency on large labeled datasets. Further, the thesis explores the current state of self-supervised learning in medical imaging. It introduces novel pretext tasks that are capable of extracting meaningful information from unlabeled data, significantly advancing the model’s interpretative abilities. This approach is validated through rigorous experimentation, including the use of the MedMNIST dataset, demonstrating the model’s proficiency in learning generalized features applicable to various downstream tasks. In summary, this thesis contributes to the advancement of medical image analysis by offering a scalable, adaptable framework that reduces reliance on labeled data. It paves the way for more accurate, efficient diagnostic tools in healthcare, signifying a major step forward in the application of deep learning in medical imaging.

Method: Three-branch network using two DINOv2 ViT-g/14 encoders with class-balanced loss weighting, patch-level cross-attention for local reasoning, and lightweight MLP with softmax for semantic contact labels.

Result: On DAMON benchmark: 7% higher binary-contact F1 score, halves geodesic error, adds object-level semantic labels. Outperformed baseline in both DAMON Challenge tasks.

Conclusion: LoRA fine-tuning and dual encoders are key improvements. The simpler architecture without vision-language model performed better. Code is publicly available.

Abstract: Accurate vertex-level contact prediction between humans and surrounding objects is a prerequisite for high fidelity human object interaction models used in robotics, AR/VR, and behavioral simulation. DECO was the first in the wild estimator for this task but is limited to binary contact maps and struggles with soft surfaces, occlusions, children, and false-positive foot contacts. We address these issues and introduce DecoDINO, a three-branch network based on DECO’s framework. It uses two DINOv2 ViT-g/14 encoders, class-balanced loss weighting to reduce bias, and patch-level cross-attention for improved local reasoning. Vertex features are finally passed through a lightweight MLP with a softmax to assign semantic contact labels. We also tested a vision-language model (VLM) to integrate text features, but the simpler architecture performed better and was used instead. On the DAMON benchmark, DecoDINO (i) raises the binary-contact F1 score by 7$%$, (ii) halves the geodesic error, and (iii) augments predictions with object-level semantic labels. Ablation studies show that LoRA fine-tuning and the dual encoders are key to these improvements. DecoDINO outperformed the challenge baseline in both tasks of the DAMON Challenge. Our code is available at https://github.com/DavidePasero/deco/tree/main.

Method: Proposes VR-Drive with joint 3D scene reconstruction as auxiliary task, feed-forward inference for online training-time augmentation, viewpoint-mixed memory bank for temporal interaction, and viewpoint-consistent distillation strategy.

Result: VR-Drive effectively mitigates synthesis-induced noise and improves planning under viewpoint shifts, demonstrating scalable and robust performance for real-world deployment.

Conclusion: VR-Drive provides a scalable and robust solution for viewpoint generalization in end-to-end autonomous driving systems, with a new benchmark dataset released for comprehensive evaluation.

Abstract: End-to-end autonomous driving (E2E-AD) has emerged as a promising paradigm that unifies perception, prediction, and planning into a holistic, data-driven framework. However, achieving robustness to varying camera viewpoints, a common real-world challenge due to diverse vehicle configurations, remains an open problem. In this work, we propose VR-Drive, a novel E2E-AD framework that addresses viewpoint generalization by jointly learning 3D scene reconstruction as an auxiliary task to enable planning-aware view synthesis. Unlike prior scene-specific synthesis approaches, VR-Drive adopts a feed-forward inference strategy that supports online training-time augmentation from sparse views without additional annotations. To further improve viewpoint consistency, we introduce a viewpoint-mixed memory bank that facilitates temporal interaction across multiple viewpoints and a viewpoint-consistent distillation strategy that transfers knowledge from original to synthesized views. Trained in a fully end-to-end manner, VR-Drive effectively mitigates synthesis-induced noise and improves planning under viewpoint shifts. In addition, we release a new benchmark dataset to evaluate E2E-AD performance under novel camera viewpoints, enabling comprehensive analysis. Our results demonstrate that VR-Drive is a scalable and robust solution for the real-world deployment of end-to-end autonomous driving systems.

Method: Unifies fine-grained attentive mosaic representations with global-wise slide embeddings aligned through vision-language contrastive learning, trained on 6,926 slide-report pairs, supporting mosaic-based image-to-image retrieval and multi-modal text-to-slide retrieval.

Result: Outperforms traditional image-to-image retrieval frameworks on four public datasets and three in-house cohorts across various tasks including anatomical site retrieval, tumor subtyping, and grading. Multi-center reader study shows improved diagnostic accuracy, confidence, and inter-observer agreement.

Conclusion: PathSearch establishes itself as a scalable and generalizable retrieval solution for digital pathology that enhances clinical workflows and diagnostic performance.

Abstract: The rapid digitization of histopathology slides has opened up new possibilities for computational tools in clinical and research workflows. Among these, content-based slide retrieval stands out, enabling pathologists to identify morphologically and semantically similar cases, thereby supporting precise diagnoses, enhancing consistency across observers, and assisting example-based education. However, effective retrieval of whole slide images (WSIs) remains challenging due to their gigapixel scale and the difficulty of capturing subtle semantic differences amid abundant irrelevant content. To overcome these challenges, we present PathSearch, a retrieval framework that unifies fine-grained attentive mosaic representations with global-wise slide embeddings aligned through vision-language contrastive learning. Trained on a corpus of 6,926 slide-report pairs, PathSearch captures both fine-grained morphological cues and high-level semantic patterns to enable accurate and flexible retrieval. The framework supports two key functionalities: (1) mosaic-based image-to-image retrieval, ensuring accurate and efficient slide research; and (2) multi-modal retrieval, where text queries can directly retrieve relevant slides. PathSearch was rigorously evaluated on four public pathology datasets and three in-house cohorts, covering tasks including anatomical site retrieval, tumor subtyping, tumor vs. non-tumor discrimination, and grading across diverse organs such as breast, lung, kidney, liver, and stomach. External results show that PathSearch outperforms traditional image-to-image retrieval frameworks. A multi-center reader study further demonstrates that PathSearch improves diagnostic accuracy, boosts confidence, and enhances inter-observer agreement among pathologists in real clinical scenarios. These results establish PathSearch as a scalable and generalizable retrieval solution for digital pathology.

Method: Collected 12,832 videos (35.64 hours) from multiple datasets, captured under diverse real-world conditions. Created DeepMoiréFake (DMF) dataset and used synthetic Moiré generation techniques. Evaluated 15 state-of-the-art detectors.

Result: Moiré artifacts degrade detector performance by up to 25.4%, synthetic Moiré causes 21.4% accuracy drop, and demoiréing methods reduce accuracy by up to 17.2% instead of helping.

Conclusion: There’s an urgent need for detection models robust to Moiré distortions and other real-world challenges. The DMF dataset aims to bridge the gap between controlled experiments and practical deepfake detection.

Abstract: Deepfake detection remains a pressing challenge, particularly in real-world settings where smartphone-captured media from digital screens often introduces Moir'e artifacts that can distort detection outcomes. This study systematically evaluates state-of-the-art (SOTA) deepfake detectors on Moir'e-affected videos, an issue that has received little attention. We collected a dataset of 12,832 videos, spanning 35.64 hours, from the Celeb-DF, DFD, DFDC, UADFV, and FF++ datasets, capturing footage under diverse real-world conditions, including varying screens, smartphones, lighting setups, and camera angles. To further examine the influence of Moir'e patterns on deepfake detection, we conducted additional experiments using our DeepMoir'eFake, referred to as (DMF) dataset and two synthetic Moir'e generation techniques. Across 15 top-performing detectors, our results show that Moir'e artifacts degrade performance by as much as 25.4%, while synthetically generated Moir'e patterns lead to a 21.4% drop in accuracy. Surprisingly, demoir'eing methods, intended as a mitigation approach, instead worsened the problem, reducing accuracy by up to 17.2%. These findings underscore the urgent need for detection models that can robustly handle Moir'e distortions alongside other realworld challenges, such as compression, sharpening, and blurring. By introducing the DMF dataset, we aim to drive future research toward closing the gap between controlled experiments and practical deepfake detection.

Method: Frames HTG as multimodal prompt-conditioned generation, introduces special textual input tokens for better alignment with visual tokens, and implements Classifier-Free Guidance strategy for the autoregressive model.

Result: Eruku requires fewer inputs, generalizes better to unseen styles, and follows textual prompts more faithfully with improved content adherence compared to previous solutions.

Conclusion: The proposed Eruku approach successfully addresses limitations of previous HTG methods by improving content controllability, reducing input requirements, and enhancing style generalization through multimodal prompt-conditioned generation.

Abstract: Generating faithful and readable styled text images (especially for Styled Handwritten Text generation - HTG) is an open problem with several possible applications across graphic design, document understanding, and image editing. A lot of research effort in this task is dedicated to developing strategies that reproduce the stylistic characteristics of a given writer, with promising results in terms of style fidelity and generalization achieved by the recently proposed Autoregressive Transformer paradigm for HTG. However, this method requires additional inputs, lacks a proper stop mechanism, and might end up in repetition loops, generating visual artifacts. In this work, we rethink the autoregressive formulation by framing HTG as a multimodal prompt-conditioned generation task, and tackle the content controllability issues by introducing special textual input tokens for better alignment with the visual ones. Moreover, we devise a Classifier-Free-Guidance-based strategy for our autoregressive model. Through extensive experimental validation, we demonstrate that our approach, dubbed Eruku, compared to previous solutions requires fewer inputs, generalizes better to unseen styles, and follows more faithfully the textual prompt, improving content adherence.

Method: Extends Lambertian model to frequency domain, uses frequency-domain channel ratio with structured filtering, and proposes FRBNet - an end-to-end trainable module with learnable frequency domain filters.

Result: Achieves +2.2 mAP for dark object detection and +2.9 mIoU for nighttime segmentation, outperforming existing methods across various downstream tasks.

Conclusion: FRBNet effectively addresses low-light vision challenges through frequency-domain analysis and serves as a plug-and-play module that enhances existing networks without modifying loss functions.

Abstract: Low-light vision remains a fundamental challenge in computer vision due to severe illumination degradation, which significantly affects the performance of downstream tasks such as detection and segmentation. While recent state-of-the-art methods have improved performance through invariant feature learning modules, they still fall short due to incomplete modeling of low-light conditions. Therefore, we revisit low-light image formation and extend the classical Lambertian model to better characterize low-light conditions. By shifting our analysis to the frequency domain, we theoretically prove that the frequency-domain channel ratio can be leveraged to extract illumination-invariant features via a structured filtering process. We then propose a novel and end-to-end trainable module named \textbf{F}requency-domain \textbf{R}adial \textbf{B}asis \textbf{Net}work (\textbf{FRBNet}), which integrates the frequency-domain channel ratio operation with a learnable frequency domain filter for the overall illumination-invariant feature enhancement. As a plug-and-play module, FRBNet can be integrated into existing networks for low-light downstream tasks without modifying loss functions. Extensive experiments across various downstream tasks demonstrate that FRBNet achieves superior performance, including +2.2 mAP for dark object detection and +2.9 mIoU for nighttime segmentation. Code is available at: https://github.com/Sing-Forevet/FRBNet.

Method: Joint computation of feature attributions and modality scores based on Shapley values, treating video frames and textual elements as equal features, with multiple-choice VQA as interaction between video, question, and answer modalities.

Result: Analysis of 6 VLM models on 4 datasets shows dependence on text and that multiple-choice VQA task devolves into models’ ability to ignore distractors.

Conclusion: Current VLM evaluation in multiple-choice VQA reveals text dominance and the task’s reduction to distractor filtering rather than true multi-modal understanding.

Abstract: As we become increasingly dependent on vision language models (VLMs) to answer questions about the world around us, there is a significant amount of research devoted to increasing both the difficulty of video question answering (VQA) datasets, and the context lengths of the models that they evaluate. The reliance on large language models as backbones has lead to concerns about potential text dominance, and the exploration of interactions between modalities is underdeveloped. How do we measure whether we’re heading in the right direction, with the complexity that multi-modal models introduce? We propose a joint method of computing both feature attributions and modality scores based on Shapley values, where both the features and modalities are arbitrarily definable. Using these metrics, we compare $6$ VLM models of varying context lengths on $4$ representative datasets, focusing on multiple-choice VQA. In particular, we consider video frames and whole textual elements as equal features in the hierarchy, and the multiple-choice VQA task as an interaction between three modalities: video, question and answer. Our results demonstrate a dependence on text and show that the multiple-choice VQA task devolves into a model’s ability to ignore distractors. Code available at https://github.com/sjpollard/a-video-is-not-worth-a-thousand-words.

Method: Developed an end-to-end hierarchical model based on YOLO with novel hierarchical architecture, modified loss function, and hierarchical performance metrics. Evaluated on two hierarchical categorizations of the same dataset.

Result: The model successfully captures hierarchical relationships in real-world objects and demonstrates improved performance over conventional flat classification approaches.

Conclusion: The proposed hierarchical methodology effectively addresses the inherent hierarchical structure in real-world objects that traditional flat classification algorithms typically ignore.

Abstract: Current convolution neural network (CNN) classification methods are predominantly focused on flat classification which aims solely to identify a specified object within an image. However, real-world objects often possess a natural hierarchical organization that can significantly help classification tasks. Capturing the presence of relations between objects enables better contextual understanding as well as control over the severity of mistakes. Considering these aspects, this paper proposes an end-to-end hierarchical model for image detection and classification built upon the YOLO model family. A novel hierarchical architecture, a modified loss function, and a performance metric tailored to the hierarchical nature of the model are introduced. The proposed model is trained and evaluated on two different hierarchical categorizations of the same dataset: a systematic categorization that disregards visual similarities between objects and a categorization accounting for common visual characteristics across classes. The results illustrate how the suggested methodology addresses the inherent hierarchical structure present in real-world objects, which conventional flat classification algorithms often overlook.

Method: Introduces AdaSDE, a single-step SDE solver with a per-step learnable coefficient estimated via lightweight distillation, which dynamically regulates error correction strength. The framework can be integrated with existing solvers.

Result: State-of-the-art performance: at 5 NFE, achieves FID scores of 4.18 on CIFAR-10, 8.05 on FFHQ and 6.96 on LSUN Bedroom.

Conclusion: AdaSDE successfully unifies the efficiency of ODEs with the error resilience of SDEs, providing an effective solution for accelerating diffusion sampling while maintaining high sample quality.

Abstract: Diffusion-based generative processes, formulated as differential equation solving, frequently balance computational speed with sample quality. Our theoretical investigation of ODE- and SDE-based solvers reveals complementary weaknesses: ODE solvers accumulate irreducible gradient error along deterministic trajectories, while SDE methods suffer from amplified discretization errors when the step budget is limited. Building upon this insight, we introduce AdaSDE, a novel single-step SDE solver that aims to unify the efficiency of ODEs with the error resilience of SDEs. Specifically, we introduce a single per-step learnable coefficient, estimated via lightweight distillation, which dynamically regulates the error correction strength to accelerate diffusion sampling. Notably, our framework can be integrated with existing solvers to enhance their capabilities. Extensive experiments demonstrate state-of-the-art performance: at 5 NFE, AdaSDE achieves FID scores of 4.18 on CIFAR-10, 8.05 on FFHQ and 6.96 on LSUN Bedroom. Codes are available in https://github.com/WLU-wry02/AdaSDE.

Method: Proposed Sufficient-Component Cause Model (SCCM) learning, which encourages MCoT to generate sufficient yet minimal visual components independently capable of leading to correct answers. SCCM is annotation-free and plug-and-play with various RFT methods.

Result: Empirical results show SCCM consistently improves visual faithfulness across fine-grained perception and reasoning benchmarks. The evaluation reveals current MCoT visual information is both unreliable and insufficient.

Conclusion: SCCM effectively addresses the visual faithfulness issue in MCoT reasoning by ensuring visual components are both reliable and sufficient, leading to more faithful multimodal reasoning processes.

Abstract: Recent large vision-language models (LVLMs) can generate vision-text multimodal chain-of-thought (MCoT) traces after reinforcement fine-tuning (RFT). However, we observe that the visual information incorporated in MCoT is often inaccurate, though still yield correct answers, indicating a lack of faithfulness in the MCoT reasoning process. We attribute this unfaithfulness to the RL reward in RFT, which solely incentivizes the format of interleaved vision-text cues, ie, it encourages the model to incorporate visual information into its text reasoning steps without considering the correctness of the visual information. In this paper, we first probe the faithfulness of MCoT by measuring how much the prediction changes when its visual and textual thoughts are intervened. Surprisingly, the model’s predictions remain nearly unchanged under visual intervention but change significantly under textual intervention, indicating that the visual evidence is largely ignored. To further analyze visual information, we introduce an automated LVLM-based evaluation metric that quantifies the faithfulness of visual cues from two perspectives: reliability and sufficiency. Our evaluation reveals that the visual information in current MCoT traces is simultaneously unreliable and insufficient. To address this issue, we propose a novel MCoT learning strategy termed Sufficient-Component Cause Model (SCCM) learning. This approach encourages the MCoT to generate sufficient yet minimal visual components that are independently capable of leading to correct answers. We note that the proposed SCCM is annotation-free and compatible with various RFT for MCoT in a plug-and-play manner. Empirical results demonstrate that SCCM consistently improves the visual faithfulness across a suite of fine-grained perception and reasoning benchmarks. Code is available at https://github.com/EugeneLiu01/Faithful_Thinking_with_Image.

Method: MDReID introduces two key components: Modality Decoupling Learning (MDL) to decompose modality features into modality-shared and modality-specific representations, and Modality-aware Metric Learning (MML) to enforce orthogonality and complementarity between these components.

Result: Extensive experiments on three multi-modality ReID benchmarks show MDReID achieves significant mAP improvements: 9.8%, 3.0%, and 11.5% in modality-matched scenarios, and average gains of 3.4%, 11.8%, and 10.9% in modality-mismatched scenarios.

Conclusion: MDReID effectively addresses modality inconsistencies in ReID systems through explicit modality feature decomposition and tailored metric learning, demonstrating superior performance across both modality-aligned and mismatched conditions.

Abstract: Real-world object re-identification (ReID) systems often face modality inconsistencies, where query and gallery images come from different sensors (e.g., RGB, NIR, TIR). However, most existing methods assume modality-matched conditions, which limits their robustness and scalability in practical applications. To address this challenge, we propose MDReID, a flexible any-to-any image-level ReID framework designed to operate under both modality-matched and modality-mismatched scenarios. MDReID builds on the insight that modality information can be decomposed into two components: modality-shared features that are predictable and transferable, and modality-specific features that capture unique, modality-dependent characteristics. To effectively leverage this, MDReID introduces two key components: the Modality Decoupling Learning (MDL) and Modality-aware Metric Learning (MML). Specifically, MDL explicitly decomposes modality features into modality-shared and modality-specific representations, enabling effective retrieval in both modality-aligned and mismatched scenarios. MML, a tailored metric learning strategy, further enforces orthogonality and complementarity between the two components to enhance discriminative power across modalities. Extensive experiments conducted on three challenging multi-modality ReID benchmarks (RGBNT201, RGBNT100, MSVR310) consistently demonstrate the superiority of MDReID. Notably, MDReID achieves significant mAP improvements of 9.8%, 3.0%, and 11.5% in general modality-matched scenarios, and average gains of 3.4%, 11.8%, and 10.9% in modality-mismatched scenarios, respectively. The code is available at: \textcolor{magenta}{https://github.com/stone96123/MDReID}.

Method: Proposed a simple tiling-and-aggregation pipeline that processes local patches and combines tile outputs into image labels, with interpretability enhanced through Grad-CAM and Score-CAM attention analyses.

Result: Achieved state-of-the-art accuracy on benchmark dataset, improving over published baseline by approximately 20 percentage points, with trends confirmed by cross-validation.

Conclusion: The tiling approach is effective for medical image analysis tasks with subtle differences, and interpretability methods point toward robustness-oriented directions for future research. Code is released for reproduction and extension.

Abstract: Medical image analysis is central to drug discovery and preclinical evaluation, where scalable, objective readouts can accelerate decision-making. We address classification of paclitaxel (Taxol) exposure from phase-contrast microscopy of C6 glioma cells – a task with subtle dose differences that challenges full-image models. We propose a simple tiling-and-aggregation pipeline that operates on local patches and combines tile outputs into an image label, achieving state-of-the-art accuracy on the benchmark dataset and improving over the published baseline by around 20 percentage points, with trends confirmed by cross-validation. To understand why tiling is effective, we further apply Grad-CAM and Score-CAM and attention analyses, which enhance model interpretability and point toward robustness-oriented directions for future medical image research. Code is released to facilitate reproduction and extension.

Method: Created PlanarTrack dataset with 1,000 short-term and 150 long-term videos recorded in unconstrained conditions. Each frame manually annotated with four corner points through multi-round inspection. Each sequence contains unique targets to enhance diversity.

Result: PlanarTrack is the largest and most diverse planar tracking dataset. Evaluation of 10 representative trackers shows significant performance degradation on this challenging benchmark, indicating current methods are insufficient.

Conclusion: PlanarTrack provides a comprehensive benchmark for planar tracking research. Current methods struggle with its challenges, highlighting the need for improved tracking algorithms. The dataset will be publicly available for future research.

Abstract: Planar tracking has drawn increasing interest owing to its key roles in robotics and augmented reality. Despite recent great advancement, further development of planar tracking, particularly in the deep learning era, is largely limited compared to generic tracking due to the lack of large-scale platforms. To mitigate this, we propose PlanarTrack, a large-scale high-quality and challenging benchmark for planar tracking. Specifically, PlanarTrack consists of 1,150 sequences with over 733K frames, including 1,000 short-term and 150 new long-term videos, which enables comprehensive evaluation of short- and long-term tracking performance. All videos in PlanarTrack are recorded in unconstrained conditions from the wild, which makes PlanarTrack challenging but more realistic for real-world applications. To ensure high-quality annotations, each video frame is manually annotated by four corner points with multi-round meticulous inspection and refinement. To enhance target diversity of PlanarTrack, we only capture a unique target in one sequence, which is different from existing benchmarks. To our best knowledge, PlanarTrack is by far the largest and most diverse and challenging dataset dedicated to planar tracking. To understand performance of existing methods on PlanarTrack and to provide a comparison for future research, we evaluate 10 representative planar trackers with extensive comparison and in-depth analysis. Our evaluation reveals that, unsurprisingly, the top planar trackers heavily degrade on the challenging PlanarTrack, which indicates more efforts are required for improving planar tracking. Our data and results will be released at https://github.com/HengLan/PlanarTrack

Method: Built on frozen pretrained Stable Diffusion, integrates cross-modal attention with auxiliary knowledge (DEM, land cover, month) and SAR guidance, enhanced by adapters and Fourier NDVI loss to balance spatial details and spectral fidelity.

Result: Achieves PSNR/SSIM of 32.63/0.84 (RGB) and 23.99/0.78 (IR), lowest NDVI MSE (0.042), efficient inference (0.39 sec/image), and superior crop classification (F1: 0.86) compared to Sentinel-2 (F1: 0.85).

Conclusion: LSSR demonstrates the potential of remote sensing super-resolution to advance precision agriculture through efficient, high-quality image reconstruction with effective downstream task performance.

Abstract: Super resolution offers a way to harness medium even lowresolution but historically valuable remote sensing image archives. Generative models, especially diffusion models, have recently been applied to remote sensing super resolution (RSSR), yet several challenges exist. First, diffusion models are effective but require expensive training from scratch resources and have slow inference speeds. Second, current methods have limited utilization of auxiliary information as real-world constraints to reconstruct scientifically realistic images. Finally, most current methods lack evaluation on downstream tasks. In this study, we present a efficient LSSR framework for RSSR, supported by a new multimodal dataset of paired 30 m Landsat 8 and 10 m Sentinel 2 imagery. Built on frozen pretrained Stable Diffusion, LSSR integrates crossmodal attention with auxiliary knowledge (Digital Elevation Model, land cover, month) and Synthetic Aperture Radar guidance, enhanced by adapters and a tailored Fourier NDVI loss to balance spatial details and spectral fidelity. Extensive experiments demonstrate that LSSR significantly improves crop boundary delineation and recovery, achieving state-of-the-art performance with Peak Signal-to-Noise Ratio/Structural Similarity Index Measure of 32.63/0.84 (RGB) and 23.99/0.78 (IR), and the lowest NDVI Mean Squared Error (0.042), while maintaining efficient inference (0.39 sec/image). Moreover, LSSR transfers effectively to NASA Harmonized Landsat and Sentinel (HLS) super resolution, yielding more reliable crop classification (F1: 0.86) than Sentinel-2 (F1: 0.85). These results highlight the potential of RSSR to advance precision agriculture.

Method: Proposes two agents: Boundary Reflection Agent to identify and filter partially annotated samples by predicting query-relevant timestamps outside annotated intervals, and Difficulty Estimation Agent to assess sample difficulty and implement curriculum RL that dynamically masks hard samples during training.

Result: VideoTG-R1 outperforms full-data counterparts using only 10% of training samples and 21% of computational budget, achieving better performance on VTG and grounded VideoQA tasks under both GRPO and SFT.

Conclusion: The proposed curriculum RL framework with reflected boundary annotations enables data-efficient training by addressing sample quality and difficulty issues, significantly improving learning efficiency and performance in video temporal grounding tasks.

Abstract: Video temporal grounding (VTG) aims to locate precise segments in videos based on language queries, which is a fundamental challenge in video understanding. While recent Multimodal Large Language Models (MLLMs) have shown promise in tackling VTG through reinforcement learning (RL), they overlook the challenges arising from both the quality and difficulty of training samples. (1) Partially annotated samples. Many samples contain relevant segments beyond the annotated interval, introducing ambiguous supervision. (2) Hard-to-ground samples. Samples with poor zero-shot performance produce consistently low and indistinguishable rewards during RL training, exhibiting no clear preference among multiple outputs and thus hindering learning efficiency. To address these challenges, we propose VideoTG-R1, a novel curriculum RL framework with reflected boundary annotations, enabling data-efficient training. Specifically, we propose a Boundary Reflection Agent that utilizes MLLMs to predict query-relevant timestamps outside the annotated intervals, allowing us to identify and filter out partially annotated samples, thereby reducing ambiguity. Furthermore, we introduce a Difficulty Estimation Agent to assess the training difficulty of each sample and design a curriculum RL strategy that dynamically masks the videos of hard-to-ground samples according to the training steps, easing the training difficulty and providing clearer preference. Experiments on the VTG and grounded VideoQA tasks demonstrate the effectiveness of our method. Remarkably, with only 10% of the training samples and 21% of the computational budget, VideoTG-R1 outperforms full-data counterparts under both group relative policy optimization (GRPO) and supervised fine-tuning (SFT). The code is available at https://github.com/ldong1111/VideoTG-R1.

Method: Constructed two optimization schemes that were combined to enhance DDColor’s performance, specifically targeting the identified limitations.

Result: Achieved performance improvement in image quality metrics including PSNR and SSIM after applying the optimization to DDColor.

Conclusion: The proposed optimization schemes successfully address DDColor’s limitations and improve image coloring performance as measured by standard metrics.

Abstract: The project has carried out the re-optimization of image coloring in accordance with the existing Autocolorization direction model DDColor. For the experiments on the existing weights of DDColor, we found that it has limitations in some frequency bands and the color cast problem caused by insufficient input dimension. We construct two optimization schemes and combine them, which achieves the performance improvement of indicators such as PSNR and SSIM of the images after DDColor.

Method: Uses symmetry concepts to create shapes with known structure and high variability through formula-driven generation, ensuring precise ground truth availability.

Result: Effective for self-supervised pre-training, yielding models with strong performance in classification and segmentation tasks, showing good few-shot learning capabilities and real-world applicability.

Conclusion: Symmetria provides a scalable, extensible solution for point cloud learning with public availability of dataset and code, promoting further research and innovation.

Abstract: Unlike image or text domains that benefit from an abundance of large-scale datasets, point cloud learning techniques frequently encounter limitations due to the scarcity of extensive datasets. To overcome this limitation, we present Symmetria, a formula-driven dataset that can be generated at any arbitrary scale. By construction, it ensures the absolute availability of precise ground truth, promotes data-efficient experimentation by requiring fewer samples, enables broad generalization across diverse geometric settings, and offers easy extensibility to new tasks and modalities. Using the concept of symmetry, we create shapes with known structure and high variability, enabling neural networks to learn point cloud features effectively. Our results demonstrate that this dataset is highly effective for point cloud self-supervised pre-training, yielding models with strong performance in downstream tasks such as classification and segmentation, which also show good few-shot learning capabilities. Additionally, our dataset can support fine-tuning models to classify real-world objects, highlighting our approach’s practical utility and application. We also introduce a challenging task for symmetry detection and provide a benchmark for baseline comparisons. A significant advantage of our approach is the public availability of the dataset, the accompanying code, and the ability to generate very large collections, promoting further research and innovation in point cloud learning.

Method: TIRE takes initial 3D assets, uses video tracking to identify regions needing modification, applies subject-driven 2D inpainting to progressively fill these regions, and then resplats the modified 2D multi-view observations back to 3D.

Result: Extensive experiments show TIRE significantly improves identity preservation in 3D/4D generation compared to state-of-the-art methods.

Conclusion: The proposed TIRE method effectively addresses the identity preservation problem in subject-driven 3D/4D generation through its track-inpaint-resplat pipeline.

Abstract: Current 3D/4D generation methods are usually optimized for photorealism, efficiency, and aesthetics. However, they often fail to preserve the semantic identity of the subject across different viewpoints. Adapting generation methods with one or few images of a specific subject (also known as Personalization or Subject-driven generation) allows generating visual content that align with the identity of the subject. However, personalized 3D/4D generation is still largely underexplored. In this work, we introduce TIRE (Track, Inpaint, REsplat), a novel method for subject-driven 3D/4D generation. It takes an initial 3D asset produced by an existing 3D generative model as input and uses video tracking to identify the regions that need to be modified. Then, we adopt a subject-driven 2D inpainting model for progressively infilling the identified regions. Finally, we resplat the modified 2D multi-view observations back to 3D while still maintaining consistency. Extensive experiments demonstrate that our approach significantly improves identity preservation in 3D/4D generation compared to state-of-the-art methods. Our project website is available at https://zsh2000.github.io/track-inpaint-resplat.github.io/.

Method: Extends DINO-v2 vision transformer to model full 3D brain anatomy by incorporating volumetric information from sequential MRI slices, supporting single- and multimodal inputs.

Result: Consistently outperforms existing self-supervised pretraining strategies and supervised baselines, especially in label-scarce and multi-contrast settings, enhancing diagnostic accuracy.

Conclusion: BrainFound provides a scalable and practical solution for 3D neuroimaging pipelines with significant potential for clinical deployment and research innovation.

Abstract: Foundation models in artificial intelligence (AI) are transforming medical imaging by enabling general-purpose feature learning from large-scale, unlabeled datasets. In this work, we introduce BrainFound, a self-supervised foundation model for brain MRI, built by extending DINO-v2, a vision transformer originally designed for 2D natural images. BrainFound adapts DINO-v2 to model full 3D brain anatomy by incorporating volumetric information from sequential MRI slices, moving beyond conventional single-slice paradigms. It supports both single- and multimodal inputs, enabling a broad range of downstream tasks, including disease detection and image segmentation, while generalising across varied imaging protocols and clinical scenarios. We show that BrainFound consistently outperforms existing self-supervised pretraining strategies and supervised baselines, particularly in label-scarce and multi-contrast settings. By integrating information from diverse 3D MRI modalities (e.g., T1, T2, FLAIR), it enhances diagnostic accuracy and reduces dependency on extensive expert annotations. This flexibility makes BrainFound a scalable and practical solution for 3D neuroimaging pipelines, with significant potential for clinical deployment and research innovation.

Method: Two methodological advancements: 1) Semi-sphere Check (SSC) preprocessing that optimally fragments MLS trajectory data using mutually orthogonal planar surfaces to reduce drift impact; 2) Planar Voxel-based Generalized Iterative Closest Point (PV-GICP) for fine registration that selectively uses planar surfaces within voxel partitions.

Result: Achieves sub-0.01 m average registration accuracy on real-world datasets from Munich’s inner city, with computation time reduced by more than 50% compared to conventional point-to-plane ICP methods.

Conclusion: The proposed methods show significant potential for advancing automated 3D urban modeling and updating, with direct applications in urban planning, infrastructure management, and dynamic city monitoring.

Abstract: This study presents a novel workflow designed to efficiently and accurately register large-scale mobile laser scanning (MLS) point clouds to a target model point cloud in urban street scenarios. This workflow specifically targets the complexities inherent in urban environments and adeptly addresses the challenges of integrating point clouds that vary in density, noise characteristics, and occlusion scenarios, which are common in bustling city centers. Two methodological advancements are introduced. First, the proposed Semi-sphere Check (SSC) preprocessing technique optimally fragments MLS trajectory data by identifying mutually orthogonal planar surfaces. This step reduces the impact of MLS drift on the accuracy of the entire point cloud registration, while ensuring sufficient geometric features within each fragment to avoid local minima. Second, we propose Planar Voxel-based Generalized Iterative Closest Point (PV-GICP), a fine registration method that selectively utilizes planar surfaces within voxel partitions. This pre-process strategy not only improves registration accuracy but also reduces computation time by more than 50% compared to conventional point-to-plane ICP methods. Experiments on real-world datasets from Munich’s inner city demonstrate that our workflow achieves sub-0.01 m average registration accuracy while significantly shortening processing times. The results underscore the potential of the proposed methods to advance automated 3D urban modeling and updating, with direct applications in urban planning, infrastructure management, and dynamic city monitoring.

Method: Leverages multi-plane cross-sections to extract 2D patterns, inspired by how human designers manually perform CAD reverse engineering, enabling capture of fine parametric details and direct incorporation of sketch constraints.

Result: Outperforms state-of-the-art methods and successfully reconstructs detailed, editable CAD models while incorporating sketch constraints for the first time in the reconstruction process.

Conclusion: The proposed approach effectively bridges the gap between geometry-driven and top-down methods, producing fully parametric CAD models with fine-grained details and essential sketch constraints that match human design practices.

Abstract: Computer-Aided Design (CAD) plays a foundational role in modern manufacturing and product development, often requiring designers to modify or build upon existing models. Converting 3D scans into parametric CAD representations–a process known as CAD reverse engineering–remains a significant challenge due to the high precision and structural complexity of CAD models. Existing deep learning-based approaches typically fall into two categories: bottom-up, geometry-driven methods, which often fail to produce fully parametric outputs, and top-down strategies, which tend to overlook fine-grained geometric details. Moreover, current methods neglect an essential aspect of CAD modeling: sketch-level constraints. In this work, we introduce a novel approach to CAD reverse engineering inspired by how human designers manually perform the task. Our method leverages multi-plane cross-sections to extract 2D patterns and capture fine parametric details more effectively. It enables the reconstruction of detailed and editable CAD models, outperforming state-of-the-art methods and, for the first time, incorporating sketch constraints directly into the reconstruction process.

Method: Uses curriculum learning based on sample decomposition granularity for pre-training on unlabeled samples, and class decomposition approach for downstream tasks to handle irregular class distribution.

Result: Achieved accuracies of 96.60% (brain tumour), 75.60% (digital knee x-ray), and 93.35% (Mini-DDSM) with ResNet-50; and 95.77%, 80.36%, 93.22% respectively with DenseNet-121, outperforming other training strategies.

Conclusion: CURVETE effectively addresses limited samples and class imbalance issues in medical image analysis through curriculum learning and progressive self-supervised training, demonstrating superior classification performance across multiple medical datasets.

Abstract: Identifying high-quality and easily accessible annotated samples poses a notable challenge in medical image analysis. Transfer learning techniques, leveraging pre-training data, offer a flexible solution to this issue. However, the impact of fine-tuning diminishes when the dataset exhibits an irregular distribution between classes. This paper introduces a novel deep convolutional neural network, named Curriculum Learning and Progressive Self-supervised Training (CURVETE). CURVETE addresses challenges related to limited samples, enhances model generalisability, and improves overall classification performance. It achieves this by employing a curriculum learning strategy based on the granularity of sample decomposition during the training of generic unlabelled samples. Moreover, CURVETE address the challenge of irregular class distribution by incorporating a class decomposition approach in the downstream task. The proposed method undergoes evaluation on three distinct medical image datasets: brain tumour, digital knee x-ray, and Mini-DDSM datasets. We investigate the classification performance using a generic self-supervised sample decomposition approach with and without the curriculum learning component in training the pretext task. Experimental results demonstrate that the CURVETE model achieves superior performance on test sets with an accuracy of 96.60% on the brain tumour dataset, 75.60% on the digital knee x-ray dataset, and 93.35% on the Mini-DDSM dataset using the baseline ResNet-50. Furthermore, with the baseline DenseNet-121, it achieved accuracies of 95.77%, 80.36%, and 93.22% on the brain tumour, digital knee x-ray, and Mini-DDSM datasets, respectively, outperforming other training strategies.

Method: Proposes Video-Thinker that enables MLLMs to autonomously leverage grounding and captioning capabilities. Uses Video-Thinker-10K dataset with autonomous tool usage in chain-of-thought reasoning. Training involves SFT for reasoning format learning followed by GRPO to strengthen reasoning capability.

Result: Video-Thinker achieves significant performance gains on both in-domain tasks and challenging out-of-domain benchmarks (Video-Holmes, CG-Bench-Reasoning, VRBench). Video-Thinker-7B substantially outperforms existing baselines like Video-R1 and establishes state-of-the-art performance among 7B-sized MLLMs.

Conclusion: Video-Thinker successfully extends dynamic reasoning to video tasks, enabling MLLMs to autonomously navigate grounding and captioning for video reasoning without needing external tools, demonstrating strong performance across multiple benchmarks.

Abstract: Recent advances in image reasoning methods, particularly “Thinking with Images”, have demonstrated remarkable success in Multimodal Large Language Models (MLLMs); however, this dynamic reasoning paradigm has not yet been extended to video reasoning tasks. In this paper, we propose Video-Thinker, which empowers MLLMs to think with videos by autonomously leveraging their intrinsic “grounding” and “captioning” capabilities to generate reasoning clues throughout the inference process. To spark this capability, we construct Video-Thinker-10K, a curated dataset featuring autonomous tool usage within chain-of-thought reasoning sequences. Our training strategy begins with Supervised Fine-Tuning (SFT) to learn the reasoning format, followed by Group Relative Policy Optimization (GRPO) to strengthen this reasoning capability. Through this approach, Video-Thinker enables MLLMs to autonomously navigate grounding and captioning tasks for video reasoning, eliminating the need for constructing and calling external tools. Extensive experiments demonstrate that Video-Thinker achieves significant performance gains on both in-domain tasks and challenging out-of-domain video reasoning benchmarks, including Video-Holmes, CG-Bench-Reasoning, and VRBench. Our Video-Thinker-7B substantially outperforms existing baselines such as Video-R1 and establishes state-of-the-art performance among 7B-sized MLLMs.

Method: Collected data from three urban intersections in Ingolstadt, Germany with 34 temporally aligned sensor sequences (20 seconds each) involving 2 vehicles and up to 3 infrastructure sensor poles per intersection, using 12 vehicle cameras, 2 vehicle LiDARs, 17 thermal cameras, and 12 infrastructure LiDARs.

Result: Created dataset with approximately 712k annotated 3D bounding boxes across 13 object classes at 10Hz frequency. Provided comprehensive evaluations using state-of-the-art cooperative perception methods.

Conclusion: UrbanIng-V2X enables robust benchmarking of cooperative perception algorithms in diverse traffic environments and helps prevent overfitting by providing multi-intersection data with vehicle-infrastructure collaboration.

Abstract: Recent cooperative perception datasets have played a crucial role in advancing smart mobility applications by enabling information exchange between intelligent agents, helping to overcome challenges such as occlusions and improving overall scene understanding. While some existing real-world datasets incorporate both vehicle-to-vehicle and vehicle-to-infrastructure interactions, they are typically limited to a single intersection or a single vehicle. A comprehensive perception dataset featuring multiple connected vehicles and infrastructure sensors across several intersections remains unavailable, limiting the benchmarking of algorithms in diverse traffic environments. Consequently, overfitting can occur, and models may demonstrate misleadingly high performance due to similar intersection layouts and traffic participant behavior. To address this gap, we introduce UrbanIng-V2X, the first large-scale, multi-modal dataset supporting cooperative perception involving vehicles and infrastructure sensors deployed across three urban intersections in Ingolstadt, Germany. UrbanIng-V2X consists of 34 temporally aligned and spatially calibrated sensor sequences, each lasting 20 seconds. All sequences contain recordings from one of three intersections, involving two vehicles and up to three infrastructure-mounted sensor poles operating in coordinated scenarios. In total, UrbanIng-V2X provides data from 12 vehicle-mounted RGB cameras, 2 vehicle LiDARs, 17 infrastructure thermal cameras, and 12 infrastructure LiDARs. All sequences are annotated at a frequency of 10 Hz with 3D bounding boxes spanning 13 object classes, resulting in approximately 712k annotated instances across the dataset. We provide comprehensive evaluations using state-of-the-art cooperative perception methods and publicly release the codebase, dataset, HD map, and a digital twin of the complete data collection environment.

Method: Proposes MergeMix with two components: 1) attention-aware image mixing via token merge with cluster representation and spatial context, and 2) preference-driven training paradigm building preference pairs with mixed and raw images, optimized via SimPO loss.

Result: MergeMix enhances attention consistency and efficiency, surpassing other heuristic-based methods in classification. Achieves competitive accuracy with improved efficiency in extensive experiments.

Conclusion: MergeMix provides a scalable approach to preference alignment in classification and MLLMs, bridging the gap between SFT and RL methods.

Abstract: Vision-language alignment in multi-modal large language models (MLLMs) typically relies on supervised fine-tuning (SFT) or reinforcement learning (RL). SFT is stable and efficient but requires large-scale human annotations and cannot capture subtle preferences, while RL brings in a reward signal for training, but suffers from overhead and instability. These limitations highlight a trade-off between scalability, robustness, and alignment quality. To address this, we propose MergeMix, a training-time augmentation paradigm that bridges SFT and RL. It first applies an attention-aware image mixing via token merge with more cluster representation and spatial context, and then presents a preference-driven training paradigm for MLLMs by building preference pairs with mixed images and raw images, and optimizing via SimPO loss. As a mixup augmentation, MergeMix enhances attention consistency and efficiency, surpassing other heuristic-based methods in classification. Extensive experiments demonstrate that MergeMix achieves competitive accuracy with improved efficiency, providing a scalable approach to preference alignment in classification and MLLMs.

Method: Combines diffusion-derived material priors with optical-flow-guided temporal regularization, aggregates multiple stochastic estimates into consistent shading, and uses mesh proxy from Gaussian Opacity Fields for indirect effects.

Result: Achieves substantially more stable relighting across sequences than diffusion-only baselines and scales beyond feasible clip lengths for video diffusion.

Conclusion: Hybrid approaches balancing learned priors with physically grounded constraints are a practical step toward production-ready volumetric video relighting.

Abstract: Volumetric video relighting is essential for bringing captured performances into virtual worlds, but current approaches struggle to deliver temporally stable, production-ready results. Diffusion-based intrinsic decomposition methods show promise for single frames, yet suffer from stochastic noise and instability when extended to sequences, while video diffusion models remain constrained by memory and scale. We propose a hybrid relighting framework that combines diffusion-derived material priors with temporal regularization and physically motivated rendering. Our method aggregates multiple stochastic estimates of per-frame material properties into temporally consistent shading components, using optical-flow-guided regularization. For indirect effects such as shadows and reflections, we extract a mesh proxy from Gaussian Opacity Fields and render it within a standard graphics pipeline. Experiments on real and synthetic captures show that this hybrid strategy achieves substantially more stable relighting across sequences than diffusion-only baselines, while scaling beyond the clip lengths feasible for video diffusion. These results indicate that hybrid approaches, which balance learned priors with physically grounded constraints, are a practical step toward production-ready volumetric video relighting.

Method: Combines reinforcement learning via Group Relative Policy Optimization (GRPO) with on-policy distillation, guided by text-only teacher models, plus cold-start alignment via supervised fine-tuning.

Result: Outperforms baseline models significantly and improves state-of-the-art across diverse benchmarks including MMMU-Pro, MathVision, MathVista, and LogicVista.

Conclusion: Cold-start alignment is essential for effective knowledge transfer, and on-policy distillation with text-only teachers fails without sufficient distributional alignment between teacher and student models.

Abstract: Training vision-language models (VLMs) for complex reasoning remains a challenging task, i.a. due to the scarcity of high-quality image-text reasoning data. Conversely, text-based reasoning resources are abundant and scalable, but it is still an open question how to leveraging them for VLM reasoning. To address this problem, we propose VOLD, a framework to transfer reasoning capabilities from text-only teacher models to VLM student models. To this end, VOLD combines reinforcement learning via Group Relative Policy Optimization (GRPO) with on-policy distillation, which allows the student reasoning traces to be guided by the teacher model, resulting in a significant gain over using GRPO alone. We further show that a cold-start alignment is essential for an effective transfer during the online training phase in this scenario and that without sufficient distributional alignment between teacher and student, on-policy distillation fails to provide meaningful guidance. We evaluate VOLD across diverse benchmarks including MMMU-Pro, MathVision, MathVista, and LogicVista, showing that VOLD outperforms the baseline model significantly and improves over the state of the art by a margin. Our ablation shows the importance of a cold-start alignment via SFT for on-policy distillation with a text-only teacher.

Method: iPac integrates patch partitioning, feature extraction, clustering, graph construction, and graph-based learning into a unified network to create meaningful graph representations that encapsulate image semantics.

Result: Experimental evaluation on diverse medical image datasets demonstrates iPac achieves up to 5% average accuracy improvement over baseline methods.

Conclusion: iPac offers a versatile and generic solution for medical image classification by leveraging graph representation and accounting for inherent structure and relationships among visual entities.

Abstract: Graph neural networks have emerged as a promising paradigm for image processing, yet their performance in image classification tasks is hindered by a limited consideration of the underlying structure and relationships among visual entities. This work presents iPac, a novel approach to introduce a new graph representation of images to enhance graph neural network image classification by recognizing the importance of underlying structure and relationships in medical image classification. iPac integrates various stages, including patch partitioning, feature extraction, clustering, graph construction, and graph-based learning, into a unified network to advance graph neural network image classification. By capturing relevant features and organising them into clusters, we construct a meaningful graph representation that effectively encapsulates the semantics of the image. Experimental evaluation on diverse medical image datasets demonstrates the efficacy of iPac, exhibiting an average accuracy improvement of up to 5% over baseline methods. Our approach offers a versatile and generic solution for image classification, particularly in the realm of medical images, by leveraging the graph representation and accounting for the inherent structure and relationships among visual entities.

Method: Uses context-aware dynamic subject masks automatically derived from cross-attention layer weights, applied to LoRA outputs during inference to approximate individual subject LoRA integration for masked regions, requiring only LoRA activation words from users.

Result: Extensive experiments show FreeFuse outperforms existing approaches in both generation quality and usability for multi-subject generation tasks, while being more practical and efficient.

Conclusion: FreeFuse provides a superior training-free solution for multi-subject text-to-image generation that requires no additional training, model modifications, auxiliary models, or complex user inputs, making it highly practical and efficient.

Abstract: This paper proposes FreeFuse, a novel training-free approach for multi-subject text-to-image generation through automatic fusion of multiple subject LoRAs. In contrast to existing methods that either focus on pre-inference LoRA weight merging or rely on segmentation models and complex techniques like noise blending to isolate LoRA outputs, our key insight is that context-aware dynamic subject masks can be automatically derived from cross-attention layer weights. Mathematical analysis shows that directly applying these masks to LoRA outputs during inference well approximates the case where the subject LoRA is integrated into the diffusion model and used individually for the masked region. FreeFuse demonstrates superior practicality and efficiency as it requires no additional training, no modification to LoRAs, no auxiliary models, and no user-defined prompt templates or region specifications. Alternatively, it only requires users to provide the LoRA activation words for seamless integration into standard workflows. Extensive experiments validate that FreeFuse outperforms existing approaches in both generation quality and usability under the multi-subject generation tasks. The project page is at https://future-item.github.io/FreeFuse/

Method: Dynamic Pseudo-Label Filtering (DPLF) scheme, Prior-Guided Data Augmentation Pipeline (PG-DAP) to mitigate domain shift, and data mixing consistency loss for context-free representations.

Result: Achieves superior performance on synthetic-to-real point cloud semantic segmentation tasks compared to state-of-the-art methods.

Conclusion: The proposed DPLF and PG-DAP modules effectively enhance real data utilization and mitigate domain shift in point cloud semantic segmentation.

Abstract: Annotating real-world LiDAR point clouds for use in intelligent autonomous systems is costly. To overcome this limitation, self-training-based Unsupervised Domain Adaptation (UDA) has been widely used to improve point cloud semantic segmentation by leveraging synthetic point cloud data. However, we argue that existing methods do not effectively utilize unlabeled data, as they either rely on predefined or fixed confidence thresholds, resulting in suboptimal performance. In this paper, we propose a Dynamic Pseudo-Label Filtering (DPLF) scheme to enhance real data utilization in point cloud UDA semantic segmentation. Additionally, we design a simple and efficient Prior-Guided Data Augmentation Pipeline (PG-DAP) to mitigate domain shift between synthetic and real-world point clouds. Finally, we utilize data mixing consistency loss to push the model to learn context-free representations. We implement and thoroughly evaluate our approach through extensive comparisons with state-of-the-art methods. Experiments on two challenging synthetic-to-real point cloud semantic segmentation tasks demonstrate that our approach achieves superior performance. Ablation studies confirm the effectiveness of the DPLF and PG-DAP modules. We release the code of our method in this paper.

Method: Two-stage approach: 1) Created EgoRe-5M dataset with 13M video clips featuring multi-minute segments, CoT rationales, and hand-object grounding; 2) SFT on EgoRe-5M followed by reinforcement fine-tuning for spatio-temporal localization.

Result: EgoThinker outperforms existing methods across multiple egocentric benchmarks and achieves substantial improvements in fine-grained spatio-temporal localization tasks.

Conclusion: The framework successfully bridges the gap in egocentric reasoning capabilities for MLLMs through comprehensive dataset construction and specialized training methodology.

Abstract: Egocentric video reasoning centers on an unobservable agent behind the camera who dynamically shapes the environment, requiring inference of hidden intentions and recognition of fine-grained interactions. This core challenge limits current multimodal large language models MLLMs, which excel at visible event reasoning but lack embodied, first-person understanding. To bridge this gap, we introduce EgoThinker, a novel framework that endows MLLMs with robust egocentric reasoning capabilities through spatio-temporal chain-of-thought supervision and a two-stage learning curriculum. First, we introduce EgoRe-5M, a large-scale egocentric QA dataset constructed from 13M diverse egocentric video clips. This dataset features multi-minute segments annotated with detailed CoT rationales and dense hand-object grounding. Second, we employ SFT on EgoRe-5M to instill reasoning skills, followed by reinforcement fine-tuning RFT to further enhance spatio-temporal localization. Experimental results show that EgoThinker outperforms existing methods across multiple egocentric benchmarks, while achieving substantial improvements in fine-grained spatio-temporal localization tasks. Full code and data are released at https://github.com/InternRobotics/EgoThinker.

Method: Introduces a play-and-plug framework with simple pluggable converters for switching between modes, and a Group Reuse Mechanism to improve parameter utilization.

Result: Achieves state-of-the-art performance across multiple depth estimation benchmarks while maintaining original image generation quality.

Conclusion: MERGE demonstrates that pre-trained text-to-image models can be effectively expanded to depth estimation without compromising their core generation capabilities.

Abstract: Generative depth estimation methods leverage the rich visual priors stored in pre-trained text-to-image diffusion models, demonstrating astonishing zero-shot capability. However, parameter updates during training lead to catastrophic degradation in the image generation capability of the pre-trained model. We introduce MERGE, a unified model for image generation and depth estimation, starting from a fixed pre-trained text-to-image model. MERGE demonstrates that the pre-trained text-to-image model can do more than image generation, but also expand to depth estimation effortlessly. Specifically, MERGE introduces a play-and-plug framework that enables seamless switching between image generation and depth estimation modes through simple and pluggable converters. Meanwhile, we propose a Group Reuse Mechanism to encourage parameter reuse and improve the utilization of the additional learnable parameters. MERGE unleashes the powerful depth estimation capability of the pre-trained text-to-image model while preserving its original image generation ability. Compared to other unified models for image generation and depth estimation, MERGE achieves state-of-the-art performance across multiple depth estimation benchmarks. The code will be made available at https://github.com/H-EmbodVis/MERGE

Method: Proposes Lookahead Anchoring which leverages keyframes from future timesteps ahead of the current generation window. This transforms keyframes from fixed boundaries into directional beacons that the model continuously pursues while responding to immediate audio cues. Also enables self-keyframing where reference images serve as lookahead targets.

Result: Applied to three recent human animation models, Lookahead Anchoring achieves superior lip synchronization, identity preservation, and visual quality. The temporal lookahead distance naturally controls balance between expressivity and consistency.

Conclusion: Lookahead Anchoring demonstrates improved temporal conditioning across different architectures, eliminating the need for keyframe generation while maintaining consistent identity through persistent guidance from future anchors.

Abstract: Audio-driven human animation models often suffer from identity drift during temporal autoregressive generation, where characters gradually lose their identity over time. One solution is to generate keyframes as intermediate temporal anchors that prevent degradation, but this requires an additional keyframe generation stage and can restrict natural motion dynamics. To address this, we propose Lookahead Anchoring, which leverages keyframes from future timesteps ahead of the current generation window, rather than within it. This transforms keyframes from fixed boundaries into directional beacons: the model continuously pursues these future anchors while responding to immediate audio cues, maintaining consistent identity through persistent guidance. This also enables self-keyframing, where the reference image serves as the lookahead target, eliminating the need for keyframe generation entirely. We find that the temporal lookahead distance naturally controls the balance between expressivity and consistency: larger distances allow for greater motion freedom, while smaller ones strengthen identity adherence. When applied to three recent human animation models, Lookahead Anchoring achieves superior lip synchronization, identity preservation, and visual quality, demonstrating improved temporal conditioning across several different architectures. Video results are available at the following link: https://lookahead-anchoring.github.io.

Method: Uses invertible autoregressive flow to transform images into latent sequences, with self-supervised dimension reduction to partition latent channels, one-step distillation for faster inference, and resampling-based classifier-free guidance.

Result: Achieves competitive performance compared to existing pixel-based generative models while providing exact likelihoods and scalable training.

Conclusion: FARMER successfully unifies NF and AR models for efficient and effective image generation with tractable likelihood estimation.

Abstract: Directly modeling the explicit likelihood of the raw data distribution is key topic in the machine learning area, which achieves the scaling successes in Large Language Models by autoregressive modeling. However, continuous AR modeling over visual pixel data suffer from extremely long sequences and high-dimensional spaces. In this paper, we present FARMER, a novel end-to-end generative framework that unifies Normalizing Flows (NF) and Autoregressive (AR) models for tractable likelihood estimation and high-quality image synthesis directly from raw pixels. FARMER employs an invertible autoregressive flow to transform images into latent sequences, whose distribution is modeled implicitly by an autoregressive model. To address the redundancy and complexity in pixel-level modeling, we propose a self-supervised dimension reduction scheme that partitions NF latent channels into informative and redundant groups, enabling more effective and efficient AR modeling. Furthermore, we design a one-step distillation scheme to significantly accelerate inference speed and introduce a resampling-based classifier-free guidance algorithm to boost image generation quality. Extensive experiments demonstrate that FARMER achieves competitive performance compared to existing pixel-based generative models while providing exact likelihoods and scalable training.

Method: Built a comprehensive lookup table of calibration experiments and extended the Kalibr toolbox to improve accuracy and robustness for per-frame intrinsics annotation. Collected 386 high-resolution indoor/outdoor videos with dynamic intrinsics.

Result: Created InFlux benchmark with 143K+ annotated frames showing wider intrinsic variations and scene diversity than prior benchmarks. Evaluation revealed most existing methods struggle with dynamic intrinsics prediction.

Conclusion: InFlux addresses the critical gap in dynamic camera intrinsics benchmarks and enables better evaluation of methods for handling changing camera parameters in real-world videos.

Abstract: Accurately tracking camera intrinsics is crucial for achieving 3D understanding from 2D video. However, most 3D algorithms assume that camera intrinsics stay constant throughout a video, which is often not true for many real-world in-the-wild videos. A major obstacle in this field is a lack of dynamic camera intrinsics benchmarks–existing benchmarks typically offer limited diversity in scene content and intrinsics variation, and none provide per-frame intrinsic changes for consecutive video frames. In this paper, we present Intrinsics in Flux (InFlux), a real-world benchmark that provides per-frame ground truth intrinsics annotations for videos with dynamic intrinsics. Compared to prior benchmarks, InFlux captures a wider range of intrinsic variations and scene diversity, featuring 143K+ annotated frames from 386 high-resolution indoor and outdoor videos with dynamic camera intrinsics. To ensure accurate per-frame intrinsics, we build a comprehensive lookup table of calibration experiments and extend the Kalibr toolbox to improve its accuracy and robustness. Using our benchmark, we evaluate existing baseline methods for predicting camera intrinsics and find that most struggle to achieve accurate predictions on videos with dynamic intrinsics. For the dataset, code, videos, and submission, please visit https://influx.cs.princeton.edu/.

Method: Introduces diagnostic tasks where models must identify the first incorrect step in chain-of-thought explanations for visual puzzles that require multi-step symbolic, geometric, and analogical reasoning.

Result: Evaluations reveal a gap between fluent generation and faithful reasoning - models that produce plausible chain-of-thoughts often fail to locate simple logical faults in reasoning chains.

Conclusion: PRISM-Bench provides a sharper evaluation of multimodal reasoning competence and highlights the need for diagnostic evaluation protocols to develop trustworthy multimodal large language models.

Abstract: We introduce \textbf{PRISM-Bench}, a benchmark of puzzle-based visual challenges designed to evaluate not only whether models can solve problems, but how their reasoning unfolds. Unlike prior evaluations that measure only final-answer accuracy, PRISM-Bench introduces a diagnostic task: given a visual puzzle and a step-by-step chain-of-thought (CoT) containing exactly one error, models must identify the first incorrect step. This setting enables fine-grained assessment of logical consistency, error detection, and visual reasoning. The puzzles in PRISM-Bench require multi-step symbolic, geometric, and analogical reasoning, resisting shortcuts based on superficial pattern matching. Evaluations across state-of-the-art MLLMs reveal a persistent gap between fluent generation and faithful reasoning: models that produce plausible CoTs often fail to locate simple logical faults. By disentangling answer generation from reasoning verification, PRISM-Bench offers a sharper lens on multimodal reasoning competence and underscores the need for diagnostic evaluation protocols in the development of trustworthy MLLMs.

Method: Proposes Scale-Adaptive Object Tokenizer (SAOT) for compact object representations, and PixelRefer-Lite with Object-Centric Infusion module to pre-fuse global context into object tokens for efficiency.

Result: Achieves leading performance with fewer training samples, while PixelRefer-Lite offers competitive accuracy with notable efficiency gains across various benchmarks.

Conclusion: PixelRefer enables advanced fine-grained understanding over user-specified regions, bridging the gap between scene-level and object-level comprehension in multimodal models.

Abstract: Multimodal large language models (MLLMs) have demonstrated strong general-purpose capabilities in open-world visual comprehension. However, most existing MLLMs primarily focus on holistic, scene-level understanding, often overlooking the need for fine-grained, object-centric reasoning. In this paper, we present PixelRefer, a unified region-level MLLM framework that enables advanced fine-grained understanding over user-specified regions across both images and videos. Motivated by the observation that LLM attention predominantly focuses on object-level tokens, we propose a Scale-Adaptive Object Tokenizer (SAOT) to generate compact and semantically rich object representations from free-form regions. Our analysis reveals that global visual tokens contribute mainly in early LLM layers, inspiring the design of PixelRefer-Lite, an efficient variant that employs an Object-Centric Infusion module to pre-fuse global context into object tokens. This yields a lightweight Object-Only Framework that substantially reduces computational cost while maintaining high semantic fidelity. To facilitate fine-grained instruction tuning, we curate PixelRefer-2.2M, a high-quality object-centric instruction dataset. Extensive experiments across a range of benchmarks validate that PixelRefer achieves leading performance with fewer training samples, while PixelRefer-Lite offers competitive accuracy with notable gains in efficiency.

Method: Concerto combines 3D intra-modal self-distillation with 2D-3D cross-modal joint embedding to learn coherent spatial features. It also includes variants for video-lifted point cloud understanding and a translator to project representations into CLIP’s language space.

Result: Concerto outperforms standalone SOTA 2D and 3D self-supervised models by 14.2% and 4.8% respectively in linear probing for 3D scene perception. With full fine-tuning, it achieves new SOTA results (e.g., 80.7% mIoU on ScanNet) and enables open-world perception.

Conclusion: Concerto learns spatial representations with superior fine-grained geometric and semantic consistency, demonstrating the effectiveness of multisensory-inspired learning for spatial cognition.

Abstract: Humans learn abstract concepts through multisensory synergy, and once formed, such representations can often be recalled from a single modality. Inspired by this principle, we introduce Concerto, a minimalist simulation of human concept learning for spatial cognition, combining 3D intra-modal self-distillation with 2D-3D cross-modal joint embedding. Despite its simplicity, Concerto learns more coherent and informative spatial features, as demonstrated by zero-shot visualizations. It outperforms both standalone SOTA 2D and 3D self-supervised models by 14.2% and 4.8%, respectively, as well as their feature concatenation, in linear probing for 3D scene perception. With full fine-tuning, Concerto sets new SOTA results across multiple scene understanding benchmarks (e.g., 80.7% mIoU on ScanNet). We further present a variant of Concerto tailored for video-lifted point cloud spatial understanding, and a translator that linearly projects Concerto representations into CLIP’s language space, enabling open-world perception. These results highlight that Concerto emerges spatial representations with superior fine-grained geometric and semantic consistency.

Method: Using invertible neural networks as priors with empirical risk formulation under regularization that promotes high likelihood images, either through penalization or algorithmic initialization.

Result: Invertible priors achieve higher accuracy than sparsity priors across most undersampling ratios in compressive sensing and better reconstructions than GAN priors for images with rare features or out-of-distribution natural images.

Conclusion: Invertible neural networks are superior priors for inverse problems due to their zero representation error, theoretical recovery bounds, and robustness to dataset bias and rare features.

Abstract: Trained generative models have shown remarkable performance as priors for inverse problems in imaging – for example, Generative Adversarial Network priors permit recovery of test images from 5-10x fewer measurements than sparsity priors. Unfortunately, these models may be unable to represent any particular image because of architectural choices, mode collapse, and bias in the training dataset. In this paper, we demonstrate that invertible neural networks, which have zero representation error by design, can be effective natural signal priors at inverse problems such as denoising, compressive sensing, and inpainting. Given a trained generative model, we study the empirical risk formulation of the desired inverse problem under a regularization that promotes high likelihood images, either directly by penalization or algorithmically by initialization. For compressive sensing, invertible priors can yield higher accuracy than sparsity priors across almost all undersampling ratios, and due to their lack of representation error, invertible priors can yield better reconstructions than GAN priors for images that have rare features of variation within the biased training set, including out-of-distribution natural images. We additionally compare performance for compressive sensing to unlearned methods, such as the deep decoder, and we establish theoretical bounds on expected recovery error in the case of a linear invertible model.

Method: Builds a causal graph for SGG, performs traditional biased training, then uses counterfactual causality to infer and remove the effect of bad bias. Uses Total Direct Effect (TDE) as the final predicate score for unbiased SGG.

Result: Significant improvements over previous state-of-the-art methods on the Visual Genome benchmark when using the proposed Scene Graph Diagnosis toolkit with several prevailing models.

Conclusion: The proposed causal inference framework effectively debiases SGG by removing harmful biases while preserving beneficial context priors, and is model-agnostic for wide applicability in the SGG community.

Abstract: Today’s scene graph generation (SGG) task is still far from practical, mainly due to the severe training bias, e.g., collapsing diverse “human walk on / sit on / lay on beach” into “human on beach”. Given such SGG, the down-stream tasks such as VQA can hardly infer better scene structures than merely a bag of objects. However, debiasing in SGG is not trivial because traditional debiasing methods cannot distinguish between the good and bad bias, e.g., good context prior (e.g., “person read book” rather than “eat”) and bad long-tailed bias (e.g., “near” dominating “behind / in front of”). In this paper, we present a novel SGG framework based on causal inference but not the conventional likelihood. We first build a causal graph for SGG, and perform traditional biased training with the graph. Then, we propose to draw the counterfactual causality from the trained graph to infer the effect from the bad bias, which should be removed. In particular, we use Total Direct Effect (TDE) as the proposed final predicate score for unbiased SGG. Note that our framework is agnostic to any SGG model and thus can be widely applied in the community who seeks unbiased predictions. By using the proposed Scene Graph Diagnosis toolkit on the SGG benchmark Visual Genome and several prevailing models, we observed significant improvements over the previous state-of-the-art methods.

Method: Proposes a causal inference framework that identifies SGD momentum as a confounder. Uses causal intervention during training and counterfactual reasoning during inference to disentangle the paradoxical effects of momentum - removing its harmful effects on tail prediction while preserving benefits for representation learning.

Result: Achieves new state-of-the-art performance on three long-tailed visual recognition benchmarks: Long-tailed CIFAR-10/-100, ImageNet-LT for image classification, and LVIS for instance segmentation.

Conclusion: The causal inference framework provides a principled solution to long-tailed classification by theoretically explaining previous methods and deriving a new approach that effectively addresses the momentum confounder problem.

Abstract: As the class size grows, maintaining a balanced dataset across many classes is challenging because the data are long-tailed in nature; it is even impossible when the sample-of-interest co-exists with each other in one collectable unit, e.g., multiple visual instances in one image. Therefore, long-tailed classification is the key to deep learning at scale. However, existing methods are mainly based on re-weighting/re-sampling heuristics that lack a fundamental theory. In this paper, we establish a causal inference framework, which not only unravels the whys of previous methods, but also derives a new principled solution. Specifically, our theory shows that the SGD momentum is essentially a confounder in long-tailed classification. On one hand, it has a harmful causal effect that misleads the tail prediction biased towards the head. On the other hand, its induced mediation also benefits the representation learning and head prediction. Our framework elegantly disentangles the paradoxical effects of the momentum, by pursuing the direct causal effect caused by an input sample. In particular, we use causal intervention in training, and counterfactual reasoning in inference, to remove the “bad” while keep the “good”. We achieve new state-of-the-arts on three long-tailed visual recognition benchmarks: Long-tailed CIFAR-10/-100, ImageNet-LT for image classification and LVIS for instance segmentation.

Method: The paper systematically reviews existing weakly supervised learning approaches for facial behavior analysis, categorizes them into a taxonomy, analyzes their insights and limitations, and summarizes widely used datasets and evaluation principles.

Result: The review organizes the state-of-the-art WSL methods, identifies their strengths and weaknesses, and provides performance summaries across different datasets and evaluation metrics used in facial behavior analysis research.

Conclusion: The paper highlights remaining challenges and opportunities in applying weakly supervised learning for facial behavior analysis in real-life situations, suggesting potential research directions to advance the field.

Abstract: In the recent years, there has been a shift in facial behavior analysis from the laboratory-controlled conditions to the challenging in-the-wild conditions due to the superior performance of deep learning based approaches for many real world applications.However, the performance of deep learning approaches relies on the amount of training data. One of the major problems with data acquisition is the requirement of annotations for large amount of training data. Labeling process of huge training data demands lot of human support with strong domain expertise for facial expressions or action units, which is difficult to obtain in real-time environments.Moreover, labeling process is highly vulnerable to ambiguity of expressions or action units, especially for intensities due to the bias induced by the domain experts. Therefore, there is an imperative need to address the problem of facial behavior analysis with weak annotations. In this paper, we provide a comprehensive review of weakly supervised learning (WSL) approaches for facial behavior analysis with both categorical as well as dimensional labels along with the challenges and potential research directions associated with it. First, we introduce various types of weak annotations in the context of facial behavior analysis and the corresponding challenges associated with it. We then systematically review the existing state-of-the-art approaches and provide a taxonomy of these approaches along with their insights and limitations. In addition, widely used data-sets in the reviewed literature and the performance of these approaches along with evaluation principles are summarized. Finally, we discuss the remaining challenges and opportunities along with the potential research directions in order to apply facial behavior analysis with weak labels in real life situations.

Method: Modify multi-dimensional scaling to create transformation-invariant and distance-preserving initial point representations, then feed these representations into existing neural networks as a plug-in component.

Result: Extensive experiments on point cloud analysis and combinatorial optimization demonstrate TinvNN’s effectiveness and general applicability. The method can also be extended to equivariance cases.

Conclusion: TinvNN provides a straightforward and general solution for transformation invariance that should be considered as an essential baseline for geometric deep learning studies.

Abstract: Deep neural networks have achieved great success in the last decade. When designing neural networks to handle the ubiquitous geometric data such as point clouds and graphs, it is critical that the model can maintain invariance towards various transformations such as translation, rotation, and scaling. Most existing graph neural network (GNN) approaches can only maintain permutation-invariance, failing to guarantee invariance with respect to other transformations. Besides GNNs, other works design sophisticated transformation-invariant layers, which are computationally expensive and difficult to be extended. In this paper, we revisit why general neural networks cannot maintain transformation invariance. Our findings show that transformation-invariant and distance-preserving initial point representations are sufficient to achieve transformation invariance rather than needing sophisticated neural layer designs. Motivated by these findings, we propose Transformation Invariant Neural Networks (TinvNN), a straightforward and general plug-in for geometric data. Specifically, we realize transformation invariant and distance-preserving initial point representations by modifying multi-dimensional scaling and feed the representations into existing neural networks. We prove that TinvNN can strictly guarantee transformation invariance, being general and flexible enough to be combined with the existing neural networks. Extensive experimental results on point cloud analysis and combinatorial optimization demonstrate the effectiveness and general applicability of our method. We also extend our method into equivariance cases. Based on the results, we advocate that TinvNN should be considered as an essential baseline for further studies of transformation-invariant geometric deep learning.

Method: The paper conducts a comprehensive literature survey on blockchain-biometrics integration, analyzes practical applications in PKI systems and identity management, and performs legal analysis based on GDPR requirements.

Result: The integration shows promise for applications like distributed trusted services and identity management, but faces challenges including efficiency limitations for real-time applications and legal compliance issues with GDPR.

Conclusion: While blockchain-biometrics integration is still emerging, it offers significant potential for secure identity management, though current blockchain limitations and GDPR compliance requirements present important challenges that need to be addressed in future research.

Abstract: Biometric recognition as an efficient and hard-to-forge way of identification and verification has become an indispensable part of the current digital world. The fast evolution of this technology has been a strong incentive for integration into many applications. Meanwhile, blockchain, the decentralized ledger technology, has been widely received by both research and industry in the past few years, and it is being increasingly deployed today in many different applications, such as money transfer, IoT, healthcare, or logistics. Recently, researchers have started to speculate on the pros and cons and what the best applications would be when these two technologies cross paths. This paper provides a survey of the research literature on the combination of blockchain and biometrics and includes a first legal analysis of this integration based on GDPR to shed light on challenges and potentials. Although the integration of blockchain technology into the biometric sector is still in its infancy, with a growing body of literature discussing specific applications and advanced technological setups, this paper aims to provide a holistic understanding of blockchain applicability in biometrics. Based on published studies, this article discusses, among others, practical examples combining blockchain and biometrics for novel applications in PKI systems, distributed trusted services, and identity management. Challenges and limitations when combining blockchain and biometrics that motivate future work will also be discussed; e.g., blockchain networks at their current stage may not be efficient or economical for some real-time biometric applications. Finally, we also discuss key legal aspects of the EU General Data Protection Regulation (GDPR) related to this combination of technologies (blockchain and biometrics); for example, accountability, immutability, anonymity, and data protection elements.

Method: Proposed two frameworks: TransFace uses Vision Transformers (ViTs) to better capture global facial features and model correlations between local features. TransFace++ explores using image bytes instead of RGB images to improve efficiency and security.

Result: Experiments on popular face benchmarks demonstrate the superiority of both TransFace and TransFace++ frameworks compared to existing methods.

Conclusion: The proposed frameworks successfully explore the feasibility of applying ViTs and image bytes to face recognition tasks, addressing key limitations in efficiency, security, and precision of traditional CNN-based RGB approaches.

Abstract: Face Recognition (FR) technology has made significant strides with the emergence of deep learning. Typically, most existing FR models are built upon Convolutional Neural Networks (CNN) and take RGB face images as the model’s input. In this work, we take a closer look at existing FR paradigms from high-efficiency, security, and precision perspectives, and identify the following three problems: (i) CNN frameworks are vulnerable in capturing global facial features and modeling the correlations between local facial features. (ii) Selecting RGB face images as the model’s input greatly degrades the model’s inference efficiency, increasing the extra computation costs. (iii) In the real-world FR system that operates on RGB face images, the integrity of user privacy may be compromised if hackers successfully penetrate and gain access to the input of this model. To solve these three issues, we propose two novel FR frameworks, i.e., TransFace and TransFace++, which successfully explore the feasibility of applying ViTs and image bytes to FR tasks, respectively. Experiments on popular face benchmarks demonstrate the superiority of our TransFace and TransFace++. Code is available at https://github.com/DanJun6737/TransFace_pp.

Method: Uses foundational models for object recognition, image segmentation, and feature extraction to create 3D point cloud maps with instance-level embeddings that can be queried using natural language commands.

Result: Quantitatively improves success rate of language-guided tasks and qualitatively shows clearer instance identification and ability to recognize objects that closed-set approaches would miss, leveraging language and image-aligned embeddings.

Conclusion: The proposed 3D instance-level semantic mapping approach successfully enhances both quantitative performance and qualitative understanding for language-guided navigation tasks by incorporating semantic comprehension through foundational models.

Abstract: Humans excel at forming mental maps of their surroundings, equipping them to understand object relationships and navigate based on language queries. Our previous work, SI Maps (Nanwani L, Agarwal A, Jain K, et al. Instance-level semantic maps for vision language navigation. In: 2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). IEEE; 2023 Aug.), showed that having instance-level information and the semantic understanding of an environment helps significantly improve performance for language-guided tasks. We extend this instance-level approach to 3D while increasing the pipeline’s robustness and improving quantitative and qualitative results. Our method leverages foundational models for object recognition, image segmentation, and feature extraction. We propose a representation that results in a 3D point cloud map with instance-level embeddings, which bring in the semantic understanding that natural language commands can query. Quantitatively, the work improves upon the success rate of language-guided tasks. At the same time, we qualitatively observe the ability to identify instances more clearly and leverage the foundational models and language and image-aligned embeddings to identify objects that, otherwise, a closed-set approach wouldn’t be able to identify. Project Page - https://smart-wheelchair-rrc.github.io/o3d-sim-webpage

Method: Proposed equivariant attention mechanism operating on steerable convolution features, utilizing Fourier space non-linearities and steerable transformer layers integrated with steerable CNNs.

Result: Experimental results in both 2D and 3D show that adding steerable transformer layers to steerable convolutional networks enhances performance.

Conclusion: Steerable Transformers successfully extend transformer mechanisms while maintaining SE(d) equivariance, demonstrating improved performance when combined with steerable convolutional networks.

Abstract: We introduce Steerable Transformers, an extension of the Vision Transformer mechanism that maintains equivariance to the special Euclidean group $\mathrm{SE}(d)$. We propose an equivariant attention mechanism that operates on features extracted by steerable convolutions. Operating in Fourier space, our network utilizes Fourier space non-linearities. Our experiments in both two and three dimensions show that adding steerable transformer layers to steerable convolutional networks enhances performance.

Method: Proposed TempRMOT, an end-to-end Transformer-based framework with a query-driven Temporal Enhancement Module. Each object is represented as a Transformer query to enable long-term spatial-temporal interactions. Also introduced a semi-automatic labeling pipeline to generate 9,758 language prompts for the Refer-KITTI-V2 benchmark.

Result: TempRMOT achieves state-of-the-art performance on both Refer-KITTI and Refer-KITTI-V2 benchmarks, demonstrating the effectiveness of the proposed approach in handling object dynamics and temporal semantics.

Conclusion: The paper successfully introduces RMOT as a comprehensive referring understanding task and presents an effective Transformer-based solution that handles object quantity variations and temporal semantics, advancing the field of language-guided visual understanding.

Abstract: Referring understanding is a fundamental task that bridges natural language and visual content by localizing objects described in free-form expressions. However, existing works are constrained by limited language expressiveness, lacking the capacity to model object dynamics in spatial numbers and temporal states. To address these limitations, we introduce a new and general referring understanding task, termed referring multi-object tracking (RMOT). Its core idea is to employ a language expression as a semantic cue to guide the prediction of multi-object tracking, comprehensively accounting for variations in object quantity and temporal semantics. Along with RMOT, we introduce a RMOT benchmark named Refer-KITTI-V2, featuring scalable and diverse language expressions. To efficiently generate high-quality annotations covering object dynamics with minimal manual effort, we propose a semi-automatic labeling pipeline that formulates a total of 9,758 language prompts. In addition, we propose TempRMOT, an elegant end-to-end Transformer-based framework for RMOT. At its core is a query-driven Temporal Enhancement Module that represents each object as a Transformer query, enabling long-term spatial-temporal interactions with other objects and past frames to efficiently refine these queries. TempRMOT achieves state-of-the-art performance on both Refer-KITTI and Refer-KITTI-V2, demonstrating the effectiveness of our approach. The source code and dataset is available at https://github.com/zyn213/TempRMOT.

Method: Counterfactual persona design varying gender identities (male, female, non-binary, transgender) while keeping classification task and visual input constant, using GPT-4V vision-language model.

Result: Transgender and non-binary personas experience significantly higher refusal rates compared to male and female personas, even in non-harmful contexts.

Conclusion: Identity-driven disparities in refusal behavior exist, highlighting the need for modeling such disparities and cautioning against uncritical use of AI systems for content coding, while advancing algorithmic fairness by reframing refusal as communicative acts.

Abstract: Refusal behavior by Large Language Models is increasingly visible in content moderation, yet little is known about how refusals vary by the identity of the user making the request. This study investigates refusal as a sociotechnical outcome through a counterfactual persona design that varies gender identity–including male, female, non-binary, and transgender personas–while keeping the classification task and visual input constant. Focusing on a vision-language model (GPT-4V), we examine how identity-based language cues influence refusal in binary gender classification tasks. We find that transgender and non-binary personas experience significantly higher refusal rates, even in non-harmful contexts. Our findings also provide methodological implications for equity audits and content analysis using LLMs. Our findings underscore the importance of modeling identity-driven disparities and caution against uncritical use of AI systems for content coding. This study advances algorithmic fairness by reframing refusal as a communicative act that may unevenly regulate epistemic access and participation.

Method: RealCustom++ introduces a train-inference decoupled framework: during training, it learns general alignment between visual conditions and real words; during inference, a dual-branch architecture generates subject guidance masks and customizes generation exclusively within subject-relevant regions.

Result: The method achieves 7.48% improvement in controllability, 3.04% in similarity, and 76.43% in generation quality. For multi-subject customization, it achieves 4.6% improvement in controllability and 6.34% in multi-subject similarity.

Conclusion: RealCustom++ successfully addresses the dual-optimum paradox in text-to-image customization by using real words instead of pseudo-words, enabling simultaneous optimization of both subject similarity and text controllability.

Abstract: Given a text and an image of a specific subject, text-to-image customization aims to generate new images that align with both the text and the subject’s appearance. Existing works follow the pseudo-word paradigm, which represents the subject as a non-existent pseudo word and combines it with other text to generate images. However, the pseudo word causes semantic conflict from its different learning objective and entanglement from overlapping influence scopes with other texts, resulting in a dual-optimum paradox where subject similarity and text controllability cannot be optimal simultaneously. To address this, we propose RealCustom++, a novel real-word paradigm that represents the subject with a non-conflicting real word to firstly generate a coherent guidance image and corresponding subject mask, thereby disentangling the influence scopes of the text and subject for simultaneous optimization. Specifically, RealCustom++ introduces a train-inference decoupled framework: (1) during training, it learns a general alignment between visual conditions and all real words in the text; and (2) during inference, a dual-branch architecture is employed, where the Guidance Branch produces the subject guidance mask and the Generation Branch utilizes this mask to customize the generation of the specific real word exclusively within subject-relevant regions. In contrast to previous methods that excel in either controllability or similarity, RealCustom++ achieves superior performance in both, with improvements of 7.48% in controllability, 3.04% in similarity, and 76.43% in generation quality. For multi-subject customization, RealCustom++ further achieves improvements of 4.6% in controllability and 6.34% in multi-subject similarity. Our work has been applied in JiMeng of ByteDance, and codes are released at https://github.com/bytedance/RealCustom.

Method: CycleHDR integrates self-supervision into a modified semantic- and cycle-consistent adversarial architecture. It introduces novel artifact- and exposure-aware generators for visual artifact removal, and an encoder with loss for semantic consistency.

Result: CycleHDR achieves state-of-the-art performance across several benchmark datasets and reconstructs high-quality HDR images.

Conclusion: CycleHDR is the first method to use semantic and contextual awareness for LDR-HDR reconstruction in a self-supervised setup, successfully addressing visual artifact removal and semantic consistency using unpaired datasets.

Abstract: Reconstruction of High Dynamic Range (HDR) from Low Dynamic Range (LDR) images is an important computer vision task. There is a significant amount of research utilizing both conventional non-learning methods and modern data-driven approaches, focusing on using both single-exposed and multi-exposed LDR for HDR image reconstruction. However, most current state-of-the-art methods require high-quality paired {LDR;HDR} datasets with limited literature use of unpaired datasets, that is, methods that learn the LDR-HDR mapping between domains. This paper proposes CycleHDR, a method that integrates self-supervision into a modified semantic- and cycle-consistent adversarial architecture that utilizes unpaired LDR and HDR datasets for training. Our method introduces novel artifact- and exposure-aware generators to address visual artifact removal. It also puts forward an encoder and loss to address semantic consistency, another under-explored topic. CycleHDR is the first to use semantic and contextual awareness for the LDR-HDR reconstruction task in a self-supervised setup. The method achieves state-of-the-art performance across several benchmark datasets and reconstructs high-quality HDR images. The official website of this work is available at: https://github.com/HrishavBakulBarua/Cycle-HDR

Method: Leverages LVLMs’ language comprehension to guide initial noisy latent optimization, introduces Noise Diffusion process that updates noisy latent while preserving distribution consistency, and provides theoretical analysis of update conditions.

Result: Experimental results show the framework effectively enhances semantic alignment across various diffusion models and demonstrates good adaptability.

Conclusion: The proposed method consistently improves semantic faithfulness in generated images while maintaining distribution consistency, offering a more effective alternative to existing approaches.

Abstract: Diffusion models have achieved impressive success in generating photorealistic images, but challenges remain in ensuring precise semantic alignment with input prompts. Optimizing the initial noisy latent offers a more efficient alternative to modifying model architectures or prompt engineering for improving semantic alignment. A latest approach, InitNo, refines the initial noisy latent by leveraging attention maps; however, these maps capture only limited information, and the effectiveness of InitNo is highly dependent on the initial starting point, as it tends to converge on a local optimum near this point. To this end, this paper proposes leveraging the language comprehension capabilities of large vision-language models (LVLMs) to guide the optimization of the initial noisy latent, and introduces the Noise Diffusion process, which updates the noisy latent to generate semantically faithful images while preserving distribution consistency. Furthermore, we provide a theoretical analysis of the condition under which the update improves semantic faithfulness. Experimental results demonstrate the effectiveness and adaptability of our framework, consistently enhancing semantic alignment across various diffusion models. The code is available at https://github.com/Bomingmiao/NoiseDiffusion.

Method: Uses a disentanglement module to suppress target person’s identity information while isolating source person’s identity features, enabling robust identity extraction.

Result: Effectively identifies source persons across various face-swapping techniques, shows strong transferability to unseen methods including commercial apps, and is robust against transmission distortions and adaptive attacks.

Conclusion: FaceTracer successfully enables tracing of malicious actors behind fraudulent face-swapping activities by linking swapped content back to original individuals.

Abstract: Face-swapping techniques have advanced rapidly with the evolution of deep learning, leading to widespread use and growing concerns about potential misuse, especially in cases of fraud. While many efforts have focused on detecting swapped face images or videos, these methods are insufficient for tracing the malicious users behind fraudulent activities. Intrusive watermark-based approaches also fail to trace unmarked identities, limiting their practical utility. To address these challenges, we introduce FaceTracer, the first non-intrusive framework specifically designed to trace the identity of the source person from swapped face images or videos. Specifically, FaceTracer leverages a disentanglement module that effectively suppresses identity information related to the target person while isolating the identity features of the source person. This allows us to extract robust identity information that can directly link the swapped face back to the original individual, aiding in uncovering the actors behind fraudulent activities. Extensive experiments demonstrate FaceTracer’s effectiveness across various face-swapping techniques, successfully identifying the source person in swapped content and enabling the tracing of malicious actors involved in fraudulent activities. Additionally, FaceTracer shows strong transferability to unseen face-swapping methods including commercial applications and robustness against transmission distortions and adaptive attacks.Our code is available at: https://github.com/zzy224/FaceTracer.

Method: Uses Maximum-a-Posteriori (MAP) approach with a simple global Free Space Prior that unseen regions should remain empty, enabling artifact removal without disrupting observed regions.

Result: Successfully eliminates floaters in both seen and unseen areas, comparable to existing cleanup models but 2.5x faster in inference, requires no additional memory, and trains in under 30 seconds.

Conclusion: The method provides an efficient, fast solution for NeRF artifact cleanup that enhances novel view quality while maintaining computational efficiency.

Abstract: Neural Radiance Fields (NeRF) have advanced photorealistic novel view synthesis, but their reliance on photometric reconstruction introduces artifacts, commonly known as “floaters”. These artifacts degrade novel view quality, especially in areas unseen by the training cameras. We present a fast, post-hoc NeRF cleanup method that eliminates such artifacts by enforcing our Free Space Prior, effectively minimizing floaters without disrupting the NeRF’s representation of observed regions. Unlike existing approaches that rely on either Maximum Likelihood (ML) estimation to fit the data or a complex, local data-driven prior, our method adopts a Maximum-a-Posteriori (MAP) approach, selecting the optimal model parameters under a simple global prior assumption that unseen regions should remain empty. This enables our method to clean artifacts in both seen and unseen areas, enhancing novel view quality even in challenging scene regions. Our method is comparable with existing NeRF cleanup models while being 2.5x faster in inference time, requires no additional memory beyond the original NeRF, and achieves cleanup training in less than 30 seconds. Our code will be made publically available.

Method: Built on Group R-CNN with Boundary Refinement (BR) module using dual attention for boundaries and salient features, and Category Refinement (CR) module with contrastive branches using scale- and rotation-aware contrastive loss.

Result: Achieves significant performance improvements over state-of-the-art methods on public X-ray datasets under limited annotations.

Conclusion: BCR-Net effectively addresses imprecise localization and inaccurate classification problems in weakly semi-supervised X-ray prohibited item detection.

Abstract: Automatic prohibited item detection in X-ray images is crucial for public safety. However, most existing detection methods either rely on expensive box annotations to achieve high performance or use weak annotations but suffer from limited accuracy. To balance annotation cost and detection performance, we study Weakly Semi-Supervised X-ray Prohibited Item Detection with Points (WSSPID-P) and propose a novel \textbf{B}oundary-\textbf{C}ategory \textbf{R}efinement \textbf{Net}work (\textbf{BCR-Net}) that requires only a few box annotations and a large number of point annotations. BCR-Net is built based on Group R-CNN and introduces a new Boundary Refinement (BR) module and a new Category Refinement (CR) module. The BR module develops a dual attention mechanism to focus on both the boundaries and salient features of prohibited items. Meanwhile, the CR module incorporates contrastive branches into the heads of RPN and ROI by introducing a scale- and rotation-aware contrastive loss, enhancing intra-class consistency and inter-class separability in the feature space. Based on the above designs, BCR-Net effectively addresses the closely related problems of imprecise localization and inaccurate classification. Experimental results on public X-ray datasets show the effectiveness of BCR-Net, achieving significant performance improvements to state-of-the-art methods under limited annotations.

Method: Proposes a framework inspired by Granger Causality, using mask-based event prediction to perform Event Granger Test, and integrates causal inference techniques like front-door adjustment and counterfactual inference to address causality confounding and illusory causality.

Result: Outperforms GPT-4o and VideoChat2 by 5.77% and 2.70% respectively in reasoning complete causal relations, and shows that causal relation graphs benefit downstream video understanding tasks like video QA and event prediction.

Conclusion: The MECD framework effectively addresses the limitations of existing video causal reasoning by enabling comprehensive causal analysis of multiple interconnected events in long videos, demonstrating superior performance and practical utility for downstream tasks.

Abstract: Video causal reasoning aims to achieve a high-level understanding of videos from a causal perspective. However, it exhibits limitations in its scope, primarily executed in a question-answering paradigm and focusing on brief video segments containing isolated events and basic causal relations, lacking comprehensive and structured causality analysis for videos with multiple interconnected events. To fill this gap, we introduce a new task and dataset, Multi-Event Causal Discovery (MECD). It aims to uncover the causal relations between events distributed chronologically across long videos. Given visual segments and textual descriptions of events, MECD identifies the causal associations between these events to derive a comprehensive and structured event-level video causal graph explaining why and how the result event occurred. To address the challenges of MECD, we devise a novel framework inspired by the Granger Causality method, incorporating an efficient mask-based event prediction model to perform an Event Granger Test. It estimates causality by comparing the predicted result event when premise events are masked versus unmasked. Furthermore, we integrate causal inference techniques such as front-door adjustment and counterfactual inference to mitigate challenges in MECD like causality confounding and illusory causality. Additionally, context chain reasoning is introduced to conduct more robust and generalized reasoning. Experiments validate the effectiveness of our framework in reasoning complete causal relations, outperforming GPT-4o and VideoChat2 by 5.77% and 2.70%, respectively. Further experiments demonstrate that causal relation graphs can also contribute to downstream video understanding tasks such as video question answering and video event prediction.

Method: Built a large-scale human preference dataset with pairwise annotations, developed VideoReward reward model, and introduced three alignment algorithms: Flow-DPO (training-time preference optimization), Flow-RWR (training-time reward weighted regression), and Flow-NRG (inference-time reward guidance).

Result: VideoReward significantly outperforms existing reward models, Flow-DPO shows superior performance compared to Flow-RWR and supervised fine-tuning, and Flow-NRG enables customizable multi-objective weighting during inference.

Conclusion: The proposed human feedback pipeline effectively addresses video generation quality issues, with VideoReward and Flow-DPO demonstrating state-of-the-art performance, while Flow-NRG provides flexible inference-time customization for personalized video quality needs.

Abstract: Video generation has achieved significant advances through rectified flow techniques, but issues like unsmooth motion and misalignment between videos and prompts persist. In this work, we develop a systematic pipeline that harnesses human feedback to mitigate these problems and refine the video generation model. Specifically, we begin by constructing a large-scale human preference dataset focused on modern video generation models, incorporating pairwise annotations across multi-dimensions. We then introduce VideoReward, a multi-dimensional video reward model, and examine how annotations and various design choices impact its rewarding efficacy. From a unified reinforcement learning perspective aimed at maximizing reward with KL regularization, we introduce three alignment algorithms for flow-based models. These include two training-time strategies: direct preference optimization for flow (Flow-DPO) and reward weighted regression for flow (Flow-RWR), and an inference-time technique, Flow-NRG, which applies reward guidance directly to noisy videos. Experimental results indicate that VideoReward significantly outperforms existing reward models, and Flow-DPO demonstrates superior performance compared to both Flow-RWR and supervised fine-tuning methods. Additionally, Flow-NRG lets users assign custom weights to multiple objectives during inference, meeting personalized video quality needs.

Method: Uses cross-modal proxy tokens that approximate missing modality class tokens by attending to available modality tokens, employing low-rank adapters in frozen encoders and joint optimization with alignment and task-specific losses.

Result: Outperforms state-of-the-art baselines across five multimodal datasets with various missing rates while achieving competitive results in complete-modality settings.

Conclusion: Provides a flexible and efficient solution for robust multimodal learning that handles missing modalities effectively without compromising complete-modality performance.

Abstract: Multimodal models often experience a significant performance drop when one or more modalities are missing during inference. To address this challenge, we propose a simple yet effective approach that enhances robustness to missing modalities while maintaining strong performance when all modalities are available. Our method introduces cross-modal proxy tokens (CMPTs), which approximate the class token of a missing modality by attending only to the tokens of the available modality without requiring explicit modality generation or auxiliary networks. To efficiently learn these approximations with minimal computational overhead, we employ low-rank adapters in frozen unimodal encoders and jointly optimize an alignment loss with a task-specific loss. Extensive experiments on five multimodal datasets show that our method outperforms state-of-the-art baselines across various missing rates while achieving competitive results in complete-modality settings. Overall, our method offers a flexible and efficient solution for robust multimodal learning. The code for this paper is available at: https://github.com/CSIPlab/Cross-Modal-Proxy-Tokens.

Method: Proposes Dual-Flow framework with Cascading Distribution Shift Training to develop adversarial velocity function for multi-target instance-agnostic attacks.

Result: Significantly improves transferability with 34.58% higher success rate from Inception-v3 to ResNet-152, and shows stronger robustness against defense mechanisms.

Conclusion: Dual-Flow effectively addresses multi-target adversarial attack challenges and demonstrates superior performance over existing methods.

Abstract: Adversarial attacks are widely used to evaluate model robustness, and in black-box scenarios, the transferability of these attacks becomes crucial. Existing generator-based attacks have excellent generalization and transferability due to their instance-agnostic nature. However, when training generators for multi-target tasks, the success rate of transfer attacks is relatively low due to the limitations of the model’s capacity. To address these challenges, we propose a novel Dual-Flow framework for multi-target instance-agnostic adversarial attacks, utilizing Cascading Distribution Shift Training to develop an adversarial velocity function. Extensive experiments demonstrate that Dual-Flow significantly improves transferability over previous multi-target generative attacks. For example, it increases the success rate from Inception-v3 to ResNet-152 by 34.58%. Furthermore, our attack method shows substantially stronger robustness against defense mechanisms, such as adversarially trained models. The code of Dual-Flow is available at: $\href{https://github.com/Chyxx/Dual-Flow}{https://github.com/Chyxx/Dual-Flow}$.

Method: Embed auxiliary networks (projection layers + ArcFace-based linear layers) within selected intermediate layers of target models to refine feature representations for distinguishing benign vs adversarial inputs.

Result: Achieved superior F1 scores compared to existing unsupervised detection methods across multiple datasets (CIFAR-10, Mammals, ImageNet subset) and architectures (ResNet-50, VGG-16, ViT) against four attack methods.

Conclusion: U-CAN provides a scalable and effective solution for enhancing security and reliability of deep learning systems through unsupervised adversarial detection.

Abstract: Deep learning models are widely employed in safety-critical applications yet remain susceptible to adversarial attacks – imperceptible perturbations that can significantly degrade model performance. Conventional defense mechanisms predominantly focus on either enhancing model robustness or detecting adversarial inputs independently. In this work, we propose an Unsupervised adversarial detection via Contrastive Auxiliary Networks (U-CAN) to uncover adversarial behavior within auxiliary feature representations, without the need for adversarial examples. U-CAN is embedded within selected intermediate layers of the target model. These auxiliary networks, comprising projection layers and ArcFace-based linear layers, refine feature representations to more effectively distinguish between benign and adversarial inputs. Comprehensive experiments across multiple datasets (CIFAR-10, Mammals, and a subset of ImageNet) and architectures (ResNet-50, VGG-16, and ViT) demonstrate that our method surpasses existing unsupervised adversarial detection techniques, achieving superior F1 scores against four distinct attack methods. The proposed framework provides a scalable and effective solution for enhancing the security and reliability of deep learning systems.

Method: Proposes T2ICount framework using pretrained diffusion models with: 1) Hierarchical Semantic Correction Module for progressive text-image feature alignment refinement, 2) Representational Regional Coherence Loss using cross-attention maps from denoising U-Net, and 3) A challenging re-annotated FSC147 subset for better evaluation.

Result: Extensive experiments demonstrate superior performance across different benchmarks compared to existing methods.

Conclusion: The diffusion-based T2ICount framework effectively addresses text sensitivity limitations in zero-shot object counting through hierarchical semantic correction and representational coherence loss, providing a robust solution for counting arbitrary object categories specified by text.

Abstract: Zero-shot object counting aims to count instances of arbitrary object categories specified by text descriptions. Existing methods typically rely on vision-language models like CLIP, but often exhibit limited sensitivity to text prompts. We present T2ICount, a diffusion-based framework that leverages rich prior knowledge and fine-grained visual understanding from pretrained diffusion models. While one-step denoising ensures efficiency, it leads to weakened text sensitivity. To address this challenge, we propose a Hierarchical Semantic Correction Module that progressively refines text-image feature alignment, and a Representational Regional Coherence Loss that provides reliable supervision signals by leveraging the cross-attention maps extracted from the denosing U-Net. Furthermore, we observe that current benchmarks mainly focus on majority objects in images, potentially masking models’ text sensitivity. To address this, we contribute a challenging re-annotated subset of FSC147 for better evaluation of text-guided counting ability. Extensive experiments demonstrate that our method achieves superior performance across different benchmarks. Code is available at https://github.com/cha15yq/T2ICount.

Method: Instead of computing attribution of isolated logits, the authors combine attributions of multiple class logits in analogy to how softmax combines information across logits, computing probability distributions of attributions over classes for each spatial location.

Result: The method reveals better object- and instance-specificity, uncovers discriminative and shared features between classes, and improves established attribution methods on benchmarks including grid-pointing game and randomization-based sanity checks.

Conclusion: Reconsidering how and where attributions are computed across the network improves attribution methods while staying agnostic to model architectures.

Abstract: Neural networks are regularly employed in high-stakes decision-making, where understanding and transparency is key. Attribution methods have been developed to gain understanding into which input features neural networks use for a specific prediction. Although widely used in computer vision, these methods often result in unspecific saliency maps that fail to identify the relevant information that led to a decision, supported by different benchmarks results. Here, we revisit the common attribution pipeline and identify one cause for the lack of specificity in attributions as the computation of attribution of isolated logits. Instead, we suggest to combine attributions of multiple class logits in analogy to how the softmax combines the information across logits. By computing probability distributions of attributions over classes for each spatial location in the image, we unleash the true capabilities of existing attribution methods, revealing better object- and instance-specificity and uncovering discriminative as well as shared features between classes. On common benchmarks, including the grid-pointing game and randomization-based sanity checks, we show that this reconsideration of how and where we compute attributions across the network improves established attribution methods while staying agnostic to model architectures. We make the code publicly available: https://github.com/nilspwalter/var.

Method: Three techniques: 1) Spatial locality caching to compute only edited and neighboring regions while skipping unedited areas, 2) Token indexing preprocessing to accelerate caching, 3) Inversion step skipping to reuse latents for efficient editing.

Result: Achieves average 2.46× acceleration across various editing tasks including prompt-guided editing, dragging, and image composition, with no performance degradation.

Conclusion: EEdit provides a practical solution for efficient image editing by systematically addressing spatial and temporal redundancies, enabling real-time interactive applications.

Abstract: Inversion-based image editing is rapidly gaining momentum while suffering from significant computation overhead, hindering its application in real-time interactive scenarios. In this paper, we rethink that the redundancy in inversion-based image editing exists in both the spatial and temporal dimensions, such as the unnecessary computation in unedited regions and the redundancy in the inversion progress. To tackle these challenges, we propose a practical framework, named EEdit, to achieve efficient image editing. Specifically, we introduce three techniques to solve them one by one. For spatial redundancy, spatial locality caching is introduced to compute the edited region and its neighboring regions while skipping the unedited regions, and token indexing preprocessing is designed to further accelerate the caching. For temporal redundancy, inversion step skipping is proposed to reuse the latent for efficient editing. Our experiments demonstrate an average of 2.46 $\times$ acceleration without performance drop in a wide range of editing tasks including prompt-guided image editing, dragging and image composition. Our codes are available at https://github.com/yuriYanZeXuan/EEdit

Method: At each optimization step, crop the adversarial image randomly with controlled aspect ratio and scale, resize it, and align with the target image in embedding space to encode explicit semantic details in local regions.

Result: Achieves success rates exceeding 90% on GPT-4.5, GPT-4o, and o1, significantly outperforming prior state-of-the-art methods with lower ℓ₁/ℓ₂ perturbations. Works on various commercial models including GPT-4.5, GPT-4o, Gemini-2.0-flash, Claude-3.5/3.7-sonnet, and reasoning models.

Conclusion: Focusing perturbations on semantically rich areas rather than uniform distribution enables effective transfer attacks on commercial black-box LVLMs by ensuring semantic clarity and finer-grained feature capture.

Abstract: Despite promising performance on open-source large vision-language models (LVLMs), transfer-based targeted attacks often fail against closed-source commercial LVLMs. Analyzing failed adversarial perturbations reveals that the learned perturbations typically originate from a uniform distribution and lack clear semantic details, resulting in unintended responses. This critical absence of semantic information leads commercial black-box LVLMs to either ignore the perturbation entirely or misinterpret its embedded semantics, thereby causing the attack to fail. To overcome these issues, we propose to refine semantic clarity by encoding explicit semantic details within local regions, thus ensuring the capture of finer-grained features and inter-model transferability, and by concentrating modifications on semantically rich areas rather than applying them uniformly. To achieve this, we propose a simple yet highly effective baseline: at each optimization step, the adversarial image is cropped randomly by a controlled aspect ratio and scale, resized, and then aligned with the target image in the embedding space. While the naive source-target matching method has been utilized before in the literature, we are the first to provide a tight analysis, which establishes a close connection between perturbation optimization and semantics. Experimental results confirm our hypothesis. Our adversarial examples crafted with local-aggregated perturbations focused on crucial regions exhibit surprisingly good transferability to commercial LVLMs, including GPT-4.5, GPT-4o, Gemini-2.0-flash, Claude-3.5/3.7-sonnet, and even reasoning models like o1, Claude-3.7-thinking and Gemini-2.0-flash-thinking. Our approach achieves success rates exceeding 90% on GPT-4.5, 4o, and o1, significantly outperforming all prior state-of-the-art attack methods with lower $\ell_1/\ell_2$ perturbations.

Method: Built on DeepSeek strategy, Med-R1 uses Group Relative Policy Optimization (GRPO) for reward-guided learning beyond static annotations, and explores the impact of intermediate rationales on reasoning quality.

Result: Med-R1 achieves 29.94% average accuracy improvement over Qwen2-VL-2B and outperforms Qwen2-VL-72B (36x larger). It also shows 32.06% improvement in cross-task generalization across five question types.

Conclusion: RL improves medical reasoning and generalization, enabling efficient VLMs for real-world deployment. The quality and domain alignment of reasoning, rather than just more reasoning, determines effectiveness in medical VQA.

Abstract: Vision-language models (VLMs) have achieved impressive progress in natural image reasoning, yet their potential in medical imaging remains underexplored. Medical vision-language tasks demand precise understanding and clinically coherent answers, which are difficult to achieve due to the complexity of medical data and the scarcity of high-quality expert annotations. These challenges limit the effectiveness of conventional supervised fine-tuning (SFT) and Chain-of-Thought (CoT) strategies that work well in general domains. To address these challenges, we propose Med-R1, a reinforcement learning (RL)-enhanced vision-language model designed to improve generalization and reliability in medical reasoning. Built on the DeepSeek strategy, Med-R1 adopts Group Relative Policy Optimization (GRPO) to encourage reward-guided learning beyond static annotations. We comprehensively evaluate Med-R1 across eight distinct medical imaging modalities. Med-R1 achieves a 29.94% improvement in average accuracy over its base model Qwen2-VL-2B, and even outperforms Qwen2-VL-72B-a model with 36x more parameters. To assess cross-task generalization, we further evaluate Med-R1 on five question types. Med-R1 outperforms Qwen2-VL-2B by 32.06% in question-type generalization, also surpassing Qwen2-VL-72B. We further explore the thinking process in Med-R1, a crucial component for the success of Deepseek-R1. Our results show that omitting intermediate rationales (No-Thinking-Med-R1) not only improves in-domain and cross-domain generalization with less training, but also challenges the assumption that more reasoning always helps. These findings suggest that in medical VQA, it is not reasoning itself, but its quality and domain alignment, that determine effectiveness. Together, these results highlight that RL improves medical reasoning and generalization, enabling efficient and reliable VLMs for real-world deployment.

Method: Uses an MLLM to interpret user instructions and ground them to video contexts, generating frame-specific queries for a diffusion model. Employs curriculum learning with large-scale image editing data followed by video fine-tuning, and introduces a data synthesis pipeline to create instructional video editing data.

Result: VEGGIE outperforms baselines in instructional video editing with different skills, excels in video object grounding and reasoning segmentation where others fail, and shows strong multi-tasking capabilities.

Conclusion: The framework successfully unifies diverse video editing tasks through instruction-based approach, demonstrates how multiple tasks benefit each other, and enables promising applications like zero-shot multimodal instructional and in-context video editing.

Abstract: Recent video diffusion models have enhanced video editing, but it remains challenging to handle instructional editing and diverse tasks (e.g., adding, removing, changing) within a unified framework. In this paper, we introduce VEGGIE, a Video Editor with Grounded Generation from Instructions, a simple end-to-end framework that unifies video concept editing, grounding, and reasoning based on diverse user instructions. Specifically, given a video and text query, VEGGIE first utilizes an MLLM to interpret user intentions in instructions and ground them to the video contexts, generating frame-specific grounded task queries for pixel-space responses. A diffusion model then renders these plans and generates edited videos that align with user intent. To support diverse tasks and complex instructions, we employ a curriculum learning strategy: first aligning the MLLM and video diffusion model with large-scale instructional image editing data, followed by end-to-end fine-tuning on high-quality multitask video data. Additionally, we introduce a novel data synthesis pipeline to generate paired instructional video editing data for model training. It transforms static image data into diverse, high-quality video editing samples by leveraging Image-to-Video models to inject dynamics. VEGGIE shows strong performance in instructional video editing with different editing skills, outperforming the best instructional baseline as a versatile model, while other models struggle with multi-tasking. VEGGIE also excels in video object grounding and reasoning segmentation, where other baselines fail. We further reveal how the multiple tasks help each other and highlight promising applications like zero-shot multimodal instructional and in-context video editing.

Method: Four key strategies: dynamic channel-patch masking for cross-channel reconstruction, memory tokens for information sharing, hybrid token fusion for rich representations, and Channel-Aware Decoder for patch reconstruction.

Result: Experiments on satellite and microscopy datasets (CHAMMI, JUMP-CP, So2Sat) show ChA-MAEViT outperforms state-of-the-art MCI-ViTs by 3.0-21.5%.

Conclusion: Cross-channel interactions are crucial in Multi-Channel Imaging, and ChA-MAEViT effectively addresses this through its novel architecture components.

Abstract: Prior work using Masked Autoencoders (MAEs) typically relies on random patch masking based on the assumption that images have significant redundancies across different channels, allowing for the reconstruction of masked content using cross-channel correlations. However, this assumption does not hold in Multi-Channel Imaging (MCI), where channels may provide complementary information with minimal feature overlap. Thus, these MAEs primarily learn local structures within individual channels from patch reconstruction, failing to fully leverage cross-channel interactions and limiting their MCI effectiveness. In this paper, we present ChA-MAEViT, an MAE-based method that enhances feature learning across MCI channels via four key strategies: (1) dynamic channel-patch masking, which compels the model to reconstruct missing channels in addition to masked patches, thereby enhancing cross-channel dependencies and improving robustness to varying channel configurations; (2) memory tokens, which serve as long-term memory aids to promote information sharing across channels, addressing the challenges of reconstructing structurally diverse channels; (3) hybrid token fusion module, which merges fine-grained patch tokens with a global class token to capture richer representations; and (4) Channel-Aware Decoder, a lightweight decoder utilizes channel tokens to effectively reconstruct image patches. Experiments on satellite and microscopy datasets, CHAMMI, JUMP-CP, and So2Sat, show that ChA-MAEViT significantly outperforms state-of-the-art MCI-ViTs by 3.0-21.5%, highlighting the importance of cross-channel interactions in MCI. Our code is publicly available at https://github.com/chaudatascience/cha_mae_vit.

Method: Uses reward-guided sampling with a pretrained discriminative model for 3D orientation estimation and a one-step text-to-image generative flow model. Adopts sampling-based approach using Langevin dynamics instead of gradient-ascent optimization, with adaptive time rescaling for faster convergence.

Result: ORIGEN outperforms both training-based and test-time guidance methods across quantitative metrics and user studies.

Conclusion: The proposed zero-shot method successfully enables 3D orientation grounding in text-to-image generation with improved performance over existing approaches.

Abstract: We introduce ORIGEN, the first zero-shot method for 3D orientation grounding in text-to-image generation across multiple objects and diverse categories. While previous work on spatial grounding in image generation has mainly focused on 2D positioning, it lacks control over 3D orientation. To address this, we propose a reward-guided sampling approach using a pretrained discriminative model for 3D orientation estimation and a one-step text-to-image generative flow model. While gradient-ascent-based optimization is a natural choice for reward-based guidance, it struggles to maintain image realism. Instead, we adopt a sampling-based approach using Langevin dynamics, which extends gradient ascent by simply injecting random noise–requiring just a single additional line of code. Additionally, we introduce adaptive time rescaling based on the reward function to accelerate convergence. Our experiments show that ORIGEN outperforms both training-based and test-time guidance methods across quantitative metrics and user studies.

Method: Defined four distinct forms of visual thought expressions and analyzed them systematically to explore how visual thoughts function in MCoT.

Result: Different visual thought forms vary in clarity and conciseness, leading to varying MCoT improvement levels. Visual thoughts act as intermediaries between input images and deeper transformer layers.

Conclusion: Visual thoughts enable more advanced visual information transmission and can inspire future MCoT research breakthroughs.

Abstract: Large Vision-Language Models (LVLMs) have achieved significant success in multimodal tasks, with multimodal chain-of-thought (MCoT) further enhancing performance and interpretability. Recent MCoT methods fall into two categories: (i) Textual-MCoT (T-MCoT), which takes multimodal input and produces textual output; and (ii) Interleaved-MCoT (I-MCoT), which generates interleaved image-text outputs. Despite advances in both approaches, the mechanisms driving these improvements are not fully understood. To fill this gap, we first reveal that MCoT boosts LVLMs by incorporating visual thoughts, which convey image information to the reasoning process regardless of the MCoT format, depending only on clarity and conciseness of expression. Furthermore, to explore visual thoughts systematically, we define four distinct forms of visual thought expressions and analyze them comprehensively. Our findings demonstrate that these forms differ in clarity and conciseness, yielding varying levels of MCoT improvement. Additionally, we explore the internal nature of visual thoughts, finding that visual thoughts serve as intermediaries between the input image and reasoning to deeper transformer layers, enabling more advanced visual information transmission. We hope that the visual thoughts can inspire further breakthroughs for future MCoT research.

Method: Proposes ‘Segment then Splat’ that divides Gaussians into distinct object sets before reconstruction, eliminating geometric/semantic ambiguities and Gaussian-object misalignment. Uses CLIP embeddings for open-vocabulary querying without learning separate language fields.

Result: Extensive experiments show effectiveness in both static and dynamic scenarios, achieving true 3D segmentation with accelerated optimization.

Conclusion: The proposed method successfully enables 3D-aware open vocabulary segmentation for both static and dynamic scenes, overcoming limitations of traditional approaches by segmenting before reconstruction.

Abstract: Open-vocabulary querying in 3D space is crucial for enabling more intelligent perception in applications such as robotics, autonomous systems, and augmented reality. However, most existing methods rely on 2D pixel-level parsing, leading to multi-view inconsistencies and poor 3D object retrieval. Moreover, they are limited to static scenes and struggle with dynamic scenes due to the complexities of motion modeling. In this paper, we propose Segment then Splat, a 3D-aware open vocabulary segmentation approach for both static and dynamic scenes based on Gaussian Splatting. Segment then Splat reverses the long established approach of “segmentation after reconstruction” by dividing Gaussians into distinct object sets before reconstruction. Once reconstruction is complete, the scene is naturally segmented into individual objects, achieving true 3D segmentation. This design eliminates both geometric and semantic ambiguities, as well as Gaussian-object misalignment issues in dynamic scenes. It also accelerates the optimization process, as it eliminates the need for learning a separate language field. After optimization, a CLIP embedding is assigned to each object to enable open-vocabulary querying. Extensive experiments one various datasets demonstrate the effectiveness of our proposed method in both static and dynamic scenarios.

Method: Progressive MSDA approach that incrementally introduces source subjects based on similarity to target, with density-based memory mechanism to prevent catastrophic forgetting of relevant historical samples.

Result: Extensive experiments conducted on multiple datasets (Biovid, UNBC-McMaster, Aff-Wild2, BAH) and cross-dataset settings.

Conclusion: The proposed approach effectively addresses domain shift issues in personalized FER by selectively using relevant sources and preventing negative transfer.

Abstract: Personalized facial expression recognition (FER) involves adapting a machine learning model using samples from labeled sources and unlabeled target domains. Given the challenges of recognizing subtle expressions with considerable interpersonal variability, state-of-the-art unsupervised domain adaptation (UDA) methods focus on the multi-source UDA (MSDA) setting, where each domain corresponds to a specific subject, and improve model accuracy and robustness. However, when adapting to a specific target, the diverse nature of multiple source domains translates to a large shift between source and target data. State-of-the-art MSDA methods for FER address this domain shift by considering all the sources to adapt to the target representations. Nevertheless, adapting to a target subject presents significant challenges due to large distributional differences between source and target domains, often resulting in negative transfer. In addition, integrating all sources simultaneously increases computational costs and causes misalignment with the target. To address these issues, we propose a progressive MSDA approach that gradually introduces information from subjects based on their similarity to the target subject. This will ensure that only the most relevant sources from the target are selected, which helps avoid the negative transfer caused by dissimilar sources. We first exploit the closest sources to reduce the distribution shift with the target and then move towards the furthest while only considering the most relevant sources based on the predetermined threshold. Furthermore, to mitigate catastrophic forgetting caused by the incremental introduction of source subjects, we implemented a density-based memory mechanism that preserves the most relevant historical source samples for adaptation. Our extensive experiments on Biovid, UNBC-McMaster, Aff-Wild2, BAH, and in a cross-dataset setting.

Method: Proposes PointKAN with Geometric Affine Module (GAM) for robustness, Local Feature Processing (LFP) with parallel structure for multi-scale features, and Global Feature Processing (GFP) for complete geometric capture. Also introduces Efficient-KANs in PointKAN-elite variant to reduce parameters.

Result: Outperforms PointMLP on ModelNet40, ScanObjectNN, and ShapeNetPart datasets, with strong performance in Few-shot Learning. Achieves substantial reductions in parameter counts and computational complexity (FLOPs).

Conclusion: Demonstrates the potential of KANs-based architectures in 3D vision and opens new research avenues for point cloud understanding.

Abstract: Multi-Layer Perceptrons (MLPs) have become one of the fundamental architectural component in point cloud analysis due to its effective feature learning mechanism. However, when processing complex geometric structures in point clouds, MLPs’ fixed activation functions struggle to efficiently capture local geometric features, while suffering from poor parameter efficiency and high model redundancy. In this paper, we propose PointKAN, which applies Kolmogorov-Arnold Networks (KANs) to point cloud analysis tasks to investigate their efficacy in hierarchical feature representation. First, we introduce a Geometric Affine Module (GAM) to transform local features, improving the model’s robustness to geometric variations. Next, in the Local Feature Processing (LFP), a parallel structure extracts both group-level features and global context, providing a rich representation of both fine details and overall structure. Finally, these features are combined and processed in the Global Feature Processing (GFP). By repeating these operations, the receptive field gradually expands, enabling the model to capture complete geometric information of the point cloud. To overcome the high parameter counts and computational inefficiency of standard KANs, we develop Efficient-KANs in the PointKAN-elite variant, which significantly reduces parameters while maintaining accuracy. Experimental results demonstrate that PointKAN outperforms PointMLP on benchmark datasets such as ModelNet40, ScanObjectNN, and ShapeNetPart, with particularly strong performance in Few-shot Learning task. Additionally, PointKAN achieves substantial reductions in parameter counts and computational complexity (FLOPs). This work highlights the potential of KANs-based architectures in 3D vision and opens new avenues for research in point cloud understanding.

Method: Encodes 3D scene into disparity space images (DSIs) by back-projecting events. Uses neural network with 3D convolutions and recurrent structure to process local DSI subregions for depth prediction.

Result: Outperforms all SOTA approaches: 42% MAE reduction in stereo, 30% median error reduction with 3x depth completeness increase. Monocular achieves comparable results to existing stereo methods.

Conclusion: Framework shows remarkable performance and effective event-data processing, holding strong potential to become standard approach for event-based depth estimation and SLAM.

Abstract: Event cameras offer a promising avenue for multi-view stereo depth estimation and Simultaneous Localization And Mapping (SLAM) due to their ability to detect blur-free 3D edges at high-speed and over broad illumination conditions. However, traditional deep learning frameworks designed for conventional cameras struggle with the asynchronous, stream-like nature of event data, as their architectures are optimized for discrete, image-like inputs. We propose a scalable, flexible and adaptable framework for pixel-wise depth estimation with event cameras in both monocular and stereo setups. The 3D scene structure is encoded into disparity space images (DSIs), representing spatial densities of rays obtained by back-projecting events into space via known camera poses. Our neural network processes local subregions of the DSIs combining 3D convolutions and a recurrent structure to recognize valuable patterns for depth prediction. Local processing enables fast inference with full parallelization and ensures constant ultra-low model complexity and memory costs, regardless of camera resolution. Experiments on standard benchmarks (MVSEC and DSEC datasets) demonstrate unprecedented effectiveness: (i) using purely monocular data, our method achieves comparable results to existing stereo methods; (ii) when applied to stereo data, it strongly outperforms all state-of-the-art (SOTA) approaches, reducing the mean absolute error by at least 42%; (iii) our method also allows for increases in depth completeness by more than 3-fold while still yielding a reduction in median absolute error of at least 30%. Given its remarkable performance and effective processing of event-data, our framework holds strong potential to become a standard approach for using deep learning for event-based depth estimation and SLAM. Project page: https://github.com/tub-rip/DERD-Net

Method: Propose DEEMO task with two datasets (DEEMO-NFBL for non-facial body language, DEEMO-MER for instruction-based recognition) and DEEMO-LLaMA MLLM that integrates de-identified audio, video, and text.

Result: DEEMO-LLaMA achieves 74.49% accuracy and 74.45% F1-score in emotion recognition, and 6.20 clue overlap and 7.66 label overlap in emotion reasoning, significantly outperforming existing MLLMs.

Conclusion: The work advances privacy-preserving emotion understanding and promotes responsible affective computing by enabling emotion recognition without compromising identity privacy.

Abstract: Emotion understanding is a critical yet challenging task. Most existing approaches rely heavily on identity-sensitive information, such as facial expressions and speech, which raises concerns about personal privacy. To address this, we introduce the De-identity Multimodal Emotion Recognition and Reasoning (DEEMO), a novel task designed to enable emotion understanding using de-identified video and audio inputs. The DEEMO dataset consists of two subsets: DEEMO-NFBL, which includes rich annotations of Non-Facial Body Language (NFBL), and DEEMO-MER, an instruction dataset for Multimodal Emotion Recognition and Reasoning using identity-free cues. This design supports emotion understanding without compromising identity privacy. In addition, we propose DEEMO-LLaMA, a Multimodal Large Language Model (MLLM) that integrates de-identified audio, video, and textual information to enhance both emotion recognition and reasoning. Extensive experiments show that DEEMO-LLaMA achieves state-of-the-art performance on both tasks, outperforming existing MLLMs by a significant margin, achieving 74.49% accuracy and 74.45% F1-score in de-identity emotion recognition, and 6.20 clue overlap and 7.66 label overlap in de-identity emotion reasoning. Our work contributes to ethical AI by advancing privacy-preserving emotion understanding and promoting responsible affective computing.

Method: Created VideoHallu dataset using Veo2, Sora, and Kling to generate videos depicting physically impossible or logically inconsistent events, with expert-annotated QA pairs across four violation categories. Tested leading VLMs and performed reinforcement learning fine-tuning.

Result: Leading VLMs (Qwen-2.5-VL, Video-R1, VideoChat-R1) often miss physics and commonsense violations despite strong benchmark performance. RL fine-tuning on VideoHallu improves violation recognition without reducing standard benchmark scores.

Conclusion: VideoHallu exposes gaps in VLMs’ visual reasoning capabilities and demonstrates that targeted fine-tuning can improve physics and commonsense understanding while maintaining standard performance.

Abstract: Vision-Language Models (VLMs) have achieved strong results in video understanding, yet a key question remains: do they truly comprehend visual content or only learn shallow correlations between vision and language? Real visual understanding, especially of physics and common sense, is essential for AI systems that interact with the physical world. Current evaluations mostly use real-world videos similar to training data, so high benchmark scores may not reflect real reasoning ability. To address this, we propose negative-control tests using videos that depict physically impossible or logically inconsistent events. We introduce VideoHallu, a synthetic dataset of physics- and commonsense-violating scenes generated with Veo2, Sora, and Kling. It includes expert-annotated question-answer pairs across four categories of violations. Tests of leading VLMs (Qwen-2.5-VL, Video-R1, VideoChat-R1) show that, despite strong results on benchmarks such as MVBench and MMVU, they often miss these violations, exposing gaps in visual reasoning. Reinforcement learning fine-tuning on VideoHallu improves recognition of such violations without reducing standard benchmark performance. Our data is available at https://github.com/zli12321/VideoHallu.git.

Method: MEXA dynamically selects specialized expert models based on input modality and task requirements, generates interpretable textual reasoning from each expert, then aggregates these outputs using a Large Reasoning Model (LRM) to produce final answers.

Result: MEXA consistently outperforms strong multimodal baselines across diverse benchmarks including Video Reasoning, Audio Reasoning, 3D Understanding, and Medical QA, demonstrating broad applicability.

Conclusion: The expert-driven selection and aggregation approach enables flexible, transparent multimodal reasoning across diverse domains without additional training overhead, proving effective for scalable multimodal reasoning.

Abstract: Combining pre-trained expert models offers substantial potential for scalable multimodal reasoning, but building a unified framework remains challenging due to the increasing diversity of input modalities and task complexity. For instance, medical diagnosis requires precise reasoning over structured clinical tables, while financial forecasting depends on interpreting plot-based data to make informed predictions. To tackle this challenge, we introduce MEXA, a training-free framework that performs modality- and task-aware aggregation of multiple expert models to enable effective multimodal reasoning across diverse and distinct domains. MEXA dynamically selects expert models based on the input modality and the task-specific reasoning demands (i.e., skills). Each expert model, specialized in a modality task pair, generates interpretable textual reasoning outputs. MEXA then aggregates and reasons over these outputs using a Large Reasoning Model (LRM) to produce the final answer. This modular design allows flexible and transparent multimodal reasoning across diverse domains without additional training overhead. We extensively evaluate our approach on diverse multimodal benchmarks, including Video Reasoning, Audio Reasoning, 3D Understanding, and Medical QA. MEXA consistently delivers performance improvements over strong multimodal baselines, highlighting the effectiveness and broad applicability of our expert-driven selection and aggregation in diverse multimodal reasoning tasks.

Method: Proposes knowledge-based prompt learning that incorporates entities and triples from knowledge graphs into prompts, uses visual-specific knowledge filter with attention mechanism, and applies causal graph theory with spurious correlation elimination to remove category-irrelevant patches.

Result: Achieves state-of-the-art intra- and cross-scenario detection performance on benchmark datasets.

Conclusion: The proposed framework effectively harnesses knowledge from pre-trained vision-language models and demonstrates strong generalization capability for 3D mask presentation attack detection.

Abstract: 3D mask presentation attack detection is crucial for protecting face recognition systems against the rising threat of 3D mask attacks. While most existing methods utilize multimodal features or remote photoplethysmography (rPPG) signals to distinguish between real faces and 3D masks, they face significant challenges, such as the high costs associated with multimodal sensors and limited generalization ability. Detection-related text descriptions offer concise, universal information and are cost-effective to obtain. However, the potential of vision-language multimodal features for 3D mask presentation attack detection remains unexplored. In this paper, we propose a novel knowledge-based prompt learning framework to explore the strong generalization capability of vision-language models for 3D mask presentation attack detection. Specifically, our approach incorporates entities and triples from knowledge graphs into the prompt learning process, generating fine-grained, task-specific explicit prompts that effectively harness the knowledge embedded in pre-trained vision-language models. Furthermore, considering different input images may emphasize distinct knowledge graph elements, we introduce a visual-specific knowledge filter based on an attention mechanism to refine relevant elements according to the visual context. Additionally, we leverage causal graph theory insights into the prompt learning process to further enhance the generalization ability of our method. During training, a spurious correlation elimination paradigm is employed, which removes category-irrelevant local image patches using guidance from knowledge-based text features, fostering the learning of generalized causal prompts that align with category-relevant local patches. Experimental results demonstrate that the proposed method achieves state-of-the-art intra- and cross-scenario detection performance on benchmark datasets.

Method: Uses ODE-to-SDE conversion to enable RL exploration and Denoising Reduction strategy to improve sampling efficiency while maintaining inference performance.

Result: Achieved dramatic improvements: GenEval accuracy increased from 63% to 95% for compositional generation, visual text rendering accuracy improved from 59% to 92%, with substantial gains in human preference alignment and minimal reward hacking.

Conclusion: Flow-GRPO successfully integrates RL with flow matching models, demonstrating significant performance improvements across multiple text-to-image tasks while maintaining image quality and diversity.

Abstract: We propose Flow-GRPO, the first method to integrate online policy gradient reinforcement learning (RL) into flow matching models. Our approach uses two key strategies: (1) an ODE-to-SDE conversion that transforms a deterministic Ordinary Differential Equation (ODE) into an equivalent Stochastic Differential Equation (SDE) that matches the original model’s marginal distribution at all timesteps, enabling statistical sampling for RL exploration; and (2) a Denoising Reduction strategy that reduces training denoising steps while retaining the original number of inference steps, significantly improving sampling efficiency without sacrificing performance. Empirically, Flow-GRPO is effective across multiple text-to-image tasks. For compositional generation, RL-tuned SD3.5-M generates nearly perfect object counts, spatial relations, and fine-grained attributes, increasing GenEval accuracy from $63%$ to $95%$. In visual text rendering, accuracy improves from $59%$ to $92%$, greatly enhancing text generation. Flow-GRPO also achieves substantial gains in human preference alignment. Notably, very little reward hacking occurred, meaning rewards did not increase at the cost of appreciable image quality or diversity degradation.

Method: Uses Multi-Model Monte Carlo Tree Search (M3CTS) to generate diverse Long Chain-of-Thought reasoning trajectories, and fine-grained Direct Preference Optimization (fDPO) with spatial rewards for visual consistency, spatial grounding, and logical coherence.

Result: fDPO achieves 4.1% improvement over standard DPO in spatial quality tasks and 9.0% gain in spatial quantity tasks. SpatialReasoner-R1 sets new SoTA on SPATIALRGPT-Bench with 9.8% higher average accuracy than strongest baseline.

Conclusion: The proposed methods effectively address spatial reasoning limitations in VLMs while maintaining competitive performance on general vision-language tasks.

Abstract: Current Vision-Language Models (VLMs) struggle with fine-grained spatial reasoning, particularly when multi-step logic and precise spatial alignment are required. In this work, we introduce SpatialReasoner-R1, a vision-language reasoning model designed to address these limitations. To construct high-quality supervision for spatial reasoning, we design a Multi-Model Monte Carlo Tree Search (M3CTS) method that generates diverse, logically consistent Long Chain-of-Thought (LongCoT) reasoning trajectories. In addition, we propose fine-grained Direct Preference Optimization (fDPO), which introduces segment-specific preference granularity for descriptive grounding and logical reasoning, guided by a spatial reward mechanism that evaluates candidate responses based on visual consistency, spatial grounding, and logical coherence. Experimental results demonstrate that fDPO achieves an average improvement of 4.1% over standard DPO across spatial quality tasks, and a 9.0% gain in spatial quantity tasks. SpatialReasoner-R1, trained with fDPO, sets a new SoTA on SPATIALRGPT-Bench, outperforming the strongest baseline by 9.8% in average accuracy, while maintaining competitive performance on general vision-language tasks.

Method: Compared various MLLMs (GPT-4o, Gemini 2.0 Flash, Claude 3.5 Sonnet V2) with traditional deep learning models (ResNet50, Vision Transformer) on 192 NCCT volumes using carefully crafted prompts for ICH presence, subtype classification, localization, and volume estimation.

Result: Traditional deep learning models comprehensively outperformed MLLMs in ICH binary classification. For subtype classification, MLLMs showed inferior performance (Gemini 2.0 Flash: macro-averaged precision 0.41, F1 score 0.31).

Conclusion: MLLMs have inferior accuracy in ICH subtyping compared to deep networks but enhance interpretability through language interactions, showing potential in medical imaging analysis. Future work will focus on model refinement for 3D medical image processing.

Abstract: Introduction: Timely identification of intracranial hemorrhage (ICH) subtypes on non-contrast computed tomography is critical for prognosis prediction and therapeutic decision-making, yet remains challenging due to low contrast and blurring boundaries. This study evaluates the performance of zero-shot multi-modal large language models (MLLMs) compared to traditional deep learning methods in ICH binary classification and subtyping. Methods: We utilized a dataset provided by RSNA, comprising 192 NCCT volumes. The study compares various MLLMs, including GPT-4o, Gemini 2.0 Flash, and Claude 3.5 Sonnet V2, with conventional deep learning models, including ResNet50 and Vision Transformer. Carefully crafted prompts were used to guide MLLMs in tasks such as ICH presence, subtype classification, localization, and volume estimation. Results: The results indicate that in the ICH binary classification task, traditional deep learning models outperform MLLMs comprehensively. For subtype classification, MLLMs also exhibit inferior performance compared to traditional deep learning models, with Gemini 2.0 Flash achieving an macro-averaged precision of 0.41 and a macro-averaged F1 score of 0.31. Conclusion: While MLLMs excel in interactive capabilities, their overall accuracy in ICH subtyping is inferior to deep networks. However, MLLMs enhance interpretability through language interactions, indicating potential in medical imaging analysis. Future efforts will focus on model refinement and developing more precise MLLMs to improve performance in three-dimensional medical image processing.

Method: Proposes a caption-reason-answer output format where the model first generates detailed image captions, then extensive reasoning chains, trained with reinforcement learning on 273K CoT-free visual QA pairs.

Result: Visionary-R1 outperforms strong multimodal models (GPT-4o, Claude3.5-Sonnet, Gemini-1.5-Pro) on multiple visual reasoning benchmarks.

Conclusion: Encouraging models to interpret images before reasoning through the caption-reason-answer format effectively mitigates shortcut learning and enables robust visual reasoning capabilities.

Abstract: Learning general-purpose reasoning capabilities has long been a challenging problem in AI. Recent research in large language models (LLMs), such as DeepSeek-R1, has shown that reinforcement learning techniques like GRPO can enable pre-trained LLMs to develop reasoning capabilities using simple question-answer pairs. In this paper, we aim to train visual language models (VLMs) to perform reasoning on image data through reinforcement learning and visual question-answer pairs, without any explicit chain-of-thought (CoT) supervision. Our findings indicate that simply applying reinforcement learning to a VLM – by prompting the model to produce a reasoning chain before providing an answer – can lead the model to develop shortcuts from easy questions, thereby reducing its ability to generalize across unseen data distributions. We argue that the key to mitigating shortcut learning is to encourage the model to interpret images prior to reasoning. Therefore, we train the model to adhere to a caption-reason-answer output format: initially generating a detailed caption for an image, followed by constructing an extensive reasoning chain. When trained on 273K CoT-free visual question-answer pairs and using only reinforcement learning, our model, named Visionary-R1, outperforms strong multimodal models, such as GPT-4o, Claude3.5-Sonnet, and Gemini-1.5-Pro, on multiple visual reasoning benchmarks.

Method: Uses visual autoregressive modeling (VAR) with architectural modifications including intricately designed cross-attention mechanisms and a latent-space refinement module tailored for AiOR tasks.

Result: Significantly outperforms LDM-based models in restoration performance while achieving over 10x faster inference. Achieves state-of-the-art performance among generative AiOR methods with strong generalization capabilities.

Conclusion: RestoreVAR provides a superior alternative to LDM-based approaches for All-in-One Image Restoration, offering both better performance and significantly faster inference suitable for practical applications.

Abstract: The use of latent diffusion models (LDMs) such as Stable Diffusion has significantly improved the perceptual quality of All-in-One image Restoration (AiOR) methods, while also enhancing their generalization capabilities. However, these LDM-based frameworks suffer from slow inference due to their iterative denoising process, rendering them impractical for time-sensitive applications. Visual autoregressive modeling (VAR), a recently introduced approach for image generation, performs scale-space autoregression and achieves comparable performance to that of state-of-the-art diffusion transformers with drastically reduced computational costs. Moreover, our analysis reveals that coarse scales in VAR primarily capture degradations while finer scales encode scene detail, simplifying the restoration process. Motivated by this, we propose RestoreVAR, a novel VAR-based generative approach for AiOR that significantly outperforms LDM-based models in restoration performance while achieving over $10\times$ faster inference. To optimally exploit the advantages of VAR for AiOR, we propose architectural modifications and improvements, including intricately designed cross-attention mechanisms and a latent-space refinement module, tailored for the AiOR task. Extensive experiments show that RestoreVAR achieves state-of-the-art performance among generative AiOR methods, while also exhibiting strong generalization capabilities.

Method: Built on MIMIC-CXR-JPG dataset, CheXStruct automatically derives intermediate reasoning steps from chest X-rays including anatomical segmentation, landmark identification, diagnostic measurements, index computation, and clinical threshold application.

Result: Evaluation of 12 LVLMs shows they struggle with structured reasoning and generalization, often failing to link abstract knowledge with anatomically grounded visual interpretation.

Conclusion: The benchmark enables fine-grained assessment of diagnostic reasoning and reveals significant gaps in current models’ ability to perform clinically valid reasoning steps.

Abstract: Recent progress in Large Vision-Language Models (LVLMs) has enabled promising applications in medical tasks, such as report generation and visual question answering. However, existing benchmarks focus mainly on the final diagnostic answer, offering limited insight into whether models engage in clinically meaningful reasoning. To address this, we present CheXStruct and CXReasonBench, a structured pipeline and benchmark built on the publicly available MIMIC-CXR-JPG dataset. CheXStruct automatically derives a sequence of intermediate reasoning steps directly from chest X-rays, such as segmenting anatomical regions, deriving anatomical landmarks and diagnostic measurements, computing diagnostic indices, and applying clinical thresholds. CXReasonBench leverages this pipeline to evaluate whether models can perform clinically valid reasoning steps and to what extent they can learn from structured guidance, enabling fine-grained and transparent assessment of diagnostic reasoning. The benchmark comprises 18,988 QA pairs across 12 diagnostic tasks and 1,200 cases, each paired with up to 4 visual inputs, and supports multi-path, multi-stage evaluation including visual grounding via anatomical region selection and diagnostic measurements. Even the strongest of 12 evaluated LVLMs struggle with structured reasoning and generalization, often failing to link abstract knowledge with anatomically grounded visual interpretation. The code is available at https://github.com/ttumyche/CXReasonBench

Method: Developed GRE Suite across three dimensions: GRE30K dataset for fine-grained analysis, GRE model with multi-stage reasoning strategy, and GREval-Bench evaluation framework.

Result: GRE significantly outperforms existing methods across all granularities of geo-localization tasks, demonstrating efficacy of reasoning-augmented VLMs.

Conclusion: The proposed reasoning-augmented approach effectively addresses geo-localization challenges and provides interpretable location inference.

Abstract: Recent advances in Visual Language Models (VLMs) have demonstrated exceptional performance in visual reasoning tasks. However, geo-localization presents unique challenges, requiring the extraction of multigranular visual cues from images and their integration with external world knowledge for systematic reasoning. Current approaches to geo-localization tasks often lack robust reasoning mechanisms and explainability, limiting their effectiveness. To address these limitations, we propose the Geo Reason Enhancement (GRE) Suite, a novel framework that augments VLMs with structured reasoning chains for accurate and interpretable location inference. The GRE Suite is systematically developed across three key dimensions: dataset, model, and benchmark. First, we introduce GRE30K, a high-quality geo-localization reasoning dataset designed to facilitate fine-grained visual and contextual analysis. Next, we present the GRE model, which employs a multi-stage reasoning strategy to progressively infer scene attributes, local details, and semantic features, thereby narrowing down potential geographic regions with enhanced precision. Finally, we construct the Geo Reason Evaluation Benchmark (GREval-Bench), a comprehensive evaluation framework that assesses VLMs across diverse urban, natural, and landmark scenes to measure both coarse-grained (e.g., country, continent) and fine-grained (e.g., city, street) localization performance. Experimental results demonstrate that GRE significantly outperforms existing methods across all granularities of geo-localization tasks, underscoring the efficacy of reasoning-augmented VLMs in complex geographic inference. Code and data will be released at https://github.com/Thorin215/GRE.

Method: Proposes VMA attack that integrates first-order and second-order momentum optimization with differentiable transformation mechanisms to optimize adversarial perturbations in images.

Result: Extensive evaluations show VMA is effective and generalizable across diverse scenarios and datasets, enabling various attacks like jailbreaking, hijacking, privacy breaches, and also copyright protection through watermark injection.

Conclusion: VMA demonstrates VLMs’ significant vulnerability to image-based adversarial attacks and provides a powerful tool that can be used both maliciously and beneficially for security applications.

Abstract: Large Vision-Language Models (VLMs) have achieved remarkable success in understanding complex real-world scenarios and supporting data-driven decision-making processes. However, VLMs exhibit significant vulnerability against adversarial examples, either text or image, which can lead to various adversarial outcomes, e.g., jailbreaking, hijacking, and hallucination, etc. In this work, we empirically and theoretically demonstrate that VLMs are particularly susceptible to image-based adversarial examples, where imperceptible perturbations can precisely manipulate each output token. To this end, we propose a novel attack called Vision-language model Manipulation Attack (VMA), which integrates first-order and second-order momentum optimization techniques with a differentiable transformation mechanism to effectively optimize the adversarial perturbation. Notably, VMA can be a double-edged sword: it can be leveraged to implement various attacks, such as jailbreaking, hijacking, privacy breaches, Denial-of-Service, and the generation of sponge examples, etc, while simultaneously enabling the injection of watermarks for copyright protection. Extensive empirical evaluations substantiate the efficacy and generalizability of VMA across diverse scenarios and datasets. Code is available at https://github.com/Trustworthy-AI-Group/VMA.

Method: Created PARTONOMY benchmark from existing datasets and new annotations (862 part labels, 534 object labels). Proposed PLUM model using span tagging instead of segmentation tokens and conditioning on prior predictions in a feedback loop.

Result: State-of-the-art LMMs perform poorly (e.g., LISA-13B achieves only 5.9% gIoU). PLUM outperforms existing segmenting LMMs on reasoning segmentation, VQA, and visual hallucination benchmarks. PLUM finetuned on Explanatory Part Segmentation is competitive with models trained on more data.

Conclusion: The work reveals critical gaps in LMMs’ part grounding abilities and opens new avenues for fine-grained visual understanding through improved architectural approaches.

Abstract: Real-world objects are composed of distinctive, object-specific parts. Identifying these parts is key to performing fine-grained, compositional reasoning-yet, large multimodal models (LMMs) struggle to perform this seemingly straightforward task. In this work, we introduce PARTONOMY, an LMM benchmark designed for pixel-level part grounding. We construct PARTONOMY from existing part datasets and our own rigorously annotated set of images, encompassing 862 part labels and 534 object labels for evaluation. Unlike existing datasets that simply ask models to identify generic parts, PARTONOMY uses specialized concepts (e.g., agricultural airplane), and challenges models to compare objects’ parts, consider part-whole relationships, and justify textual predictions with visual segmentations. Our experiments demonstrate significant limitations in state-of-the-art LMMs (e.g., LISA-13B achieves only 5.9% gIoU), highlighting a critical gap in their part grounding abilities. We note that existing segmentation-enabled LMMs (segmenting LMMs) have two key architectural shortcomings: they use special [SEG] tokens not seen during pretraining which induce distribution shift, and they discard predicted segmentations instead of using past predictions to guide future ones. To address these deficiencies, we train several part-centric LMMs and propose PLUM, a novel segmenting LMM that uses span tagging instead of segmentation tokens and that conditions on prior predictions in a feedback loop. We find that pretrained PLUM outperforms existing segmenting LMMs on reasoning segmentation, VQA, and visual hallucination benchmarks. In addition, PLUM finetuned on our proposed Explanatory Part Segmentation task is competitive with segmenting LMMs trained on significantly more segmentation data. Our work opens up new avenues towards enabling fine-grained, grounded visual understanding in LMMs.

Method: Created DVL-Suite with 14,871 high-resolution multi-temporal images organized into DVL-Bench (6 urban understanding tasks) and DVL-Instruct (instruction-tuning dataset), then developed DVLChat model for both image-level QA and pixel-level segmentation.

Result: Evaluation of 18 state-of-the-art MLLMs revealed limitations in long-term temporal understanding and quantitative analysis, motivating the creation of specialized training data and baseline model.

Conclusion: DVL-Suite addresses the gap in long-term urban dynamics analysis and provides a foundation for enhancing MLLM capabilities in multi-temporal Earth observation through specialized datasets and models.

Abstract: Multimodal large language models (MLLMs) have demonstrated remarkable capabilities in visual understanding, but their application to long-term Earth observation analysis remains limited, primarily focusing on single-temporal or bi-temporal imagery. To address this gap, we introduce DVL-Suite, a comprehensive framework for analyzing long-term urban dynamics through remote sensing imagery. Our suite comprises 14,871 high-resolution (1.0m) multi-temporal images spanning 42 major cities in the U.S. from 2005 to 2023, organized into two components: DVL-Bench and DVL-Instruct. The DVL-Bench includes six urban understanding tasks, from fundamental change detection (pixel-level) to quantitative analyses (regional-level) and comprehensive urban narratives (scene-level), capturing diverse urban dynamics including expansion/transformation patterns, disaster assessment, and environmental challenges. We evaluate 18 state-of-the-art MLLMs and reveal their limitations in long-term temporal understanding and quantitative analysis. These challenges motivate the creation of DVL-Instruct, a specialized instruction-tuning dataset designed to enhance models’ capabilities in multi-temporal Earth observation. Building upon this dataset, we develop DVLChat, a baseline model capable of both image-level question-answering and pixel-level segmentation, facilitating a comprehensive understanding of city dynamics through language interactions.

Method: FlexMerge framework that flexibly generates merged models across size spectrum (from single model to all fine-tuned models) and supports multiple merging algorithms in unified framework.

Result: Key findings: modest size increases yield steep accuracy gains (up to 13.5% when doubling size), and algorithm rankings change with size - some methods overtake others beyond single-model regime.

Conclusion: Reveals new design dimension for model merging: developing and comparing algorithms across full size spectrum rather than only at single-model limit, with practical applications across vision and NLP benchmarks.

Abstract: Model merging has emerged as an efficient method to combine multiple single-task fine-tuned models. The merged model can enjoy multi-task capabilities without expensive training. While promising, merging into a single model often suffers from an accuracy gap with respect to the fine-tuned models. On the other hand, deploying all individual fine-tuned models incurs high storage costs. We propose FlexMerge, a novel data-free model merging framework that: (a) flexibly generates merged models of varying sizes, spanning the full spectrum from a single merged model to retaining all fine-tuned models; and (b) supports multiple merging algorithms in a unified framework. Using FlexMerge, we systematically characterize the accuracy-size trade-off of different algorithms. Our study reveals two key findings: first, even modestly larger merged models can yield steep accuracy gains (up to 13.5% when just doubling the size); second, algorithm rankings are not consistent as size increases, with some methods overtaking others beyond the one-model regime. These results uncover a new design dimension for model merging: developing and comparing algorithms across the full spectrum of sizes rather than only at the single-model limit. Extensive experiments on vision and NLP benchmarks, with up to 30 tasks, confirm the generality and practicality of FlexMerge.

Method: Fully fine-tunes a VLM without architectural modifications, integrating three tasks: Tree-Aware Chain-of-Thought for spatial reasoning, Error-Driven Learning to reduce confusion among similar characters, and Symbol Counting for recognition consistency in long expressions.

Result: Achieves super state-of-the-art performance on CROHME and HME100K datasets, outperforming SSAN by 16.31% and Gemini2.5-flash by 24.42% under zero-shot setting.

Conclusion: Uni-MuMER demonstrates that fully fine-tuning VLMs without architectural changes can effectively inject domain-specific knowledge into generalist frameworks, providing a unified solution for HMER with superior performance.

Abstract: Handwritten Mathematical Expression Recognition (HMER) remains a persistent challenge in Optical Character Recognition (OCR) due to the inherent freedom of symbol layouts and variability in handwriting styles. Prior methods have faced performance bottlenecks by proposing isolated architectural modifications, making them difficult to integrate coherently into a unified framework. Meanwhile, recent advances in pretrained vision-language models (VLMs) have demonstrated strong cross-task generalization, offering a promising foundation for developing unified solutions. In this paper, we introduce Uni-MuMER, which fully fine-tunes a VLM for the HMER task without modifying its architecture, effectively injecting domain-specific knowledge into a generalist framework. Our method integrates three data-driven tasks: Tree-Aware Chain-of-Thought (Tree-CoT) for structured spatial reasoning, Error-Driven Learning (EDL) for reducing confusion among visually similar characters, and Symbol Counting (SC) for improving recognition consistency in long expressions. Experiments on the CROHME and HME100K datasets show that Uni-MuMER achieves super state-of-the-art performance, outperforming the best lightweight specialized model SSAN by 16.31% and the top-performing VLM Gemini2.5-flash by 24.42% under zero-shot setting. Our datasets, models, and code are open-sourced at: {https://github.com/BFlameSwift/Uni-MuMER

Method: Uses triangle soup (disconnected translucent triangles) with per-vertex Spherical Harmonics for appearance. Enforces soft connectivity forces between triangles during optimization to encourage surface continuity while maintaining flexibility.

Result: Shows improvements in geometric accuracy compared to state-of-the-art methods on 3D reconstruction and novel view synthesis datasets, without sacrificing visual fidelity.

Conclusion: Triangle soup with soft connectivity forces provides an effective representation for scene optimization that balances geometric accuracy and visual quality better than existing approaches.

Abstract: We introduce an inference-time scene optimization algorithm utilizing triangle soup, a collection of disconnected translucent triangle primitives, as the representation for the geometry and appearance of a scene. Unlike full-rank Gaussian kernels, triangles are a natural, locally-flat proxy for surfaces that can be connected to achieve highly complex geometry. When coupled with per-vertex Spherical Harmonics (SH), triangles provide a rich visual representation without incurring an expensive increase in primitives. We leverage our new representation to incorporate optimization objectives and enforce spatial regularization directly on the underlying primitives. The main differentiator of our approach is the definition and enforcement of soft connectivity forces between triangles during optimization, encouraging explicit, but soft, surface continuity in 3D. Experiments on representative 3D reconstruction and novel view synthesis datasets show improvements in geometric accuracy compared to current state-of-the-art algorithms without sacrificing visual fidelity.

Method: Representational Differences Explanations (RDX) - a technique for discovering and visualizing differences between two learned representations.

Result: RDX successfully recovers meaningful conceptual differences where existing XAI techniques fail, revealing insightful representational differences and subtle data patterns on ImageNet and iNaturalist datasets.

Conclusion: RDX addresses the gap in model comparison tools by providing an effective and explainable method for contrasting model representations.

Abstract: We propose a method for discovering and visualizing the differences between two learned representations, enabling more direct and interpretable model comparisons. We validate our method, which we call Representational Differences Explanations (RDX), by using it to compare models with known conceptual differences and demonstrate that it recovers meaningful distinctions where existing explainable AI (XAI) techniques fail. Applied to state-of-the-art models on challenging subsets of the ImageNet and iNaturalist datasets, RDX reveals both insightful representational differences and subtle patterns in the data. Although comparison is a cornerstone of scientific analysis, current tools in machine learning, namely post hoc XAI methods, struggle to support model comparison effectively. Our work addresses this gap by introducing an effective and explainable tool for contrasting model representations.

Method: The authors propose HoliSafe dataset covering all five safe/unsafe image-text combinations, and introduce a modular visual guard module (VGM) that assesses input image harmfulness. VGM enables VLMs to generate safer responses while providing interpretable harmfulness classification for refusal decisions.

Result: Safe-VLM with VGM trained on HoliSafe achieves state-of-the-art safety performance across multiple VLM benchmarks. HoliSafe-Bench also reveals critical vulnerabilities in existing VLM models.

Conclusion: HoliSafe and VGM provide a foundation for robust and interpretable VLM safety, expanding avenues for multimodal alignment research. The modular approach allows seamless integration with diverse pre-trained VLMs.

Abstract: Despite emerging efforts to enhance the safety of Vision-Language Models (VLMs), current approaches face two main shortcomings. 1) Existing safety-tuning datasets and benchmarks only partially consider how image-text interactions can yield harmful content, often overlooking contextually unsafe outcomes from seemingly benign pairs. This narrow coverage leaves VLMs vulnerable to jailbreak attacks in unseen configurations. 2) Prior methods rely primarily on data-centric tuning, with limited architectural innovations to intrinsically strengthen safety. We address these gaps by introducing a holistic safety dataset and benchmark, \textbf{HoliSafe}, that spans all five safe/unsafe image-text combinations, providing a more robust basis for both training and evaluation (HoliSafe-Bench). We further propose a novel modular framework for enhancing VLM safety with a visual guard module (VGM) designed to assess the harmfulness of input images for VLMs. This module endows VLMs with a dual functionality: they not only learn to generate safer responses but can also provide an interpretable harmfulness classification to justify their refusal decisions. A significant advantage of this approach is its modularity; the VGM is designed as a plug-in component, allowing for seamless integration with diverse pre-trained VLMs across various scales. Experiments show that Safe-VLM with VGM, trained on our HoliSafe, achieves state-of-the-art safety performance across multiple VLM benchmarks. Additionally, the HoliSafe-Bench itself reveals critical vulnerabilities in existing VLM models. We hope that HoliSafe and VGM will spur further research into robust and interpretable VLM safety, expanding future avenues for multimodal alignment.

Method: Object-X geometrically grounds captured modalities in a 3D voxel grid and learns an unstructured embedding that fuses voxel information with object attributes, enabling 3D Gaussian Splatting-based reconstruction and supporting various downstream tasks.

Result: Object-X produces high-fidelity novel-view synthesis comparable to standard 3D Gaussian Splatting while significantly improving geometric accuracy. It achieves competitive performance in scene alignment and localization, with object-centric descriptors requiring 3-4 orders of magnitude less storage than traditional approaches.

Conclusion: Object-X establishes itself as a scalable and highly practical solution for multi-modal 3D scene representation, offering versatile embeddings that support both reconstruction and various downstream tasks with significantly reduced storage requirements.

Abstract: Learning effective multi-modal 3D representations of objects is essential for numerous applications, such as augmented reality and robotics. Existing methods often rely on task-specific embeddings that are tailored either for semantic understanding or geometric reconstruction. As a result, these embeddings typically cannot be decoded into explicit geometry and simultaneously reused across tasks. In this paper, we propose Object-X, a versatile multi-modal object representation framework capable of encoding rich object embeddings (e.g. images, point cloud, text) and decoding them back into detailed geometric and visual reconstructions. Object-X operates by geometrically grounding the captured modalities in a 3D voxel grid and learning an unstructured embedding fusing the information from the voxels with the object attributes. The learned embedding enables 3D Gaussian Splatting-based object reconstruction, while also supporting a range of downstream tasks, including scene alignment, single-image 3D object reconstruction, and localization. Evaluations on two challenging real-world datasets demonstrate that Object-X produces high-fidelity novel-view synthesis comparable to standard 3D Gaussian Splatting, while significantly improving geometric accuracy. Moreover, Object-X achieves competitive performance with specialized methods in scene alignment and localization. Critically, our object-centric descriptors require 3-4 orders of magnitude less storage compared to traditional image- or point cloud-based approaches, establishing Object-X as a scalable and highly practical solution for multi-modal 3D scene representation.

Method: SPTI summarizes private images into text using image-to-text models, applies modified Private Evolution algorithm for DP text generation, then reconstructs images using text-to-image models without requiring model training.

Result: SPTI produces substantially higher quality synthetic images than prior DP approaches, achieving FID of 26.71 on LSUN Bedroom (vs 40.36) and 33.27 on MM CelebA HQ (vs 57.01) at epsilon=1.0.

Conclusion: SPTI provides a resource-efficient framework for generating high-resolution DP synthetic images, greatly expanding access to private visual datasets through proprietary model compatibility.

Abstract: Generating high fidelity, differentially private (DP) synthetic images offers a promising route to share and analyze sensitive visual data without compromising individual privacy. However, existing DP image synthesis methods struggle to produce high resolution outputs that faithfully capture the structure of the original data. In this paper, we introduce a novel method, referred to as Synthesis via Private Textual Intermediaries (SPTI), that can generate high resolution DP images with easy adoption. The key idea is to shift the challenge of DP image synthesis from the image domain to the text domain by leveraging state of the art DP text generation methods. SPTI first summarizes each private image into a concise textual description using image to text models, then applies a modified Private Evolution algorithm to generate DP text, and finally reconstructs images using text to image models. Notably, SPTI requires no model training, only inference with off the shelf models. Given a private dataset, SPTI produces synthetic images of substantially higher quality than prior DP approaches. On the LSUN Bedroom dataset, SPTI attains an FID equal to 26.71 under epsilon equal to 1.0, improving over Private Evolution FID of 40.36. Similarly, on MM CelebA HQ, SPTI achieves an FID equal to 33.27 at epsilon equal to 1.0, compared to 57.01 from DP fine tuning baselines. Overall, our results demonstrate that Synthesis via Private Textual Intermediaries provides a resource efficient and proprietary model compatible framework for generating high resolution DP synthetic images, greatly expanding access to private visual datasets.

Method: Shift high-norm activations from identified register neurons into additional untrained tokens, mimicking the effect of register tokens without retraining. Also extends this to vision-language models for cleaner text-to-image attribution.

Result: The method produces cleaner attention and feature maps, improves performance across multiple downstream visual tasks, and achieves results comparable to models explicitly trained with register tokens.

Conclusion: Test-time registers effectively substitute for register tokens at inference time, providing a training-free solution for pre-trained models that lack register tokens.

Abstract: We investigate the mechanism underlying a previously identified phenomenon in Vision Transformers - the emergence of high-norm tokens that lead to noisy attention maps (Darcet et al., 2024). We observe that in multiple models (e.g., CLIP, DINOv2), a sparse set of neurons is responsible for concentrating high-norm activations on outlier tokens, leading to irregular attention patterns and degrading downstream visual processing. While the existing solution for removing these outliers involves retraining models from scratch with additional learned register tokens, we use our findings to create a training-free approach to mitigate these artifacts. By shifting the high-norm activations from our discovered register neurons into an additional untrained token, we can mimic the effect of register tokens on a model already trained without registers. We demonstrate that our method produces cleaner attention and feature maps, enhances performance over base models across multiple downstream visual tasks, and achieves results comparable to models explicitly trained with register tokens. We then extend test-time registers to off-the-shelf vision-language models, yielding cleaner attention-based, text-to-image attribution. Finally, we outline a simple mathematical model that reflects the observed behavior of register neurons and high norm tokens. Our results suggest that test-time registers effectively take on the role of register tokens at test-time, offering a training-free solution for any pre-trained model released without them.

Method: Proposes SSGCO for co-extraction of spatial and spectral features, and EGAEL module for adaptive edge weight prediction and refinement, integrated into a contrastive learning framework for simultaneous representation learning and clustering.

Result: Improves clustering accuracy by 2.61%, 6.06%, 4.96% and 3.15% over best compared methods on four HSI datasets.

Conclusion: The proposed SSGCO and EGAEL framework effectively addresses limitations of existing GNN-based HSI clustering methods and achieves superior performance.

Abstract: Hyperspectral image (HSI) clustering groups pixels into clusters without labeled data, which is an important yet challenging task. For large-scale HSIs, most methods rely on superpixel segmentation and perform superpixel-level clustering based on graph neural networks (GNNs). However, existing GNNs cannot fully exploit the spectral information of the input HSI, and the inaccurate superpixel topological graph may lead to the confusion of different class semantics during information aggregation. To address these challenges, we first propose a structural-spectral graph convolutional operator (SSGCO) tailored for graph-structured HSI superpixels to improve their representation quality through the co-extraction of spatial and spectral features. Second, we propose an evidence-guided adaptive edge learning (EGAEL) module that adaptively predicts and refines edge weights in the superpixel topological graph. We integrate the proposed method into a contrastive learning framework to achieve clustering, where representation learning and clustering are simultaneously conducted. Experiments demonstrate that the proposed method improves clustering accuracy by 2.61%, 6.06%, 4.96% and 3.15% over the best compared methods on four HSI datasets. Our code is available at https://github.com/jhqi/SSGCO-EGAEL.

Method: Created 3D-RAD dataset with six VQA tasks (anomaly detection, image observation, medical computation, existence detection, static temporal diagnosis, longitudinal temporal diagnosis) using radiology CT scans, supporting both open- and closed-ended questions with complex reasoning challenges.

Result: Evaluations show existing vision-language models, especially medical VLMs, exhibit limited generalization particularly in multi-temporal tasks. Fine-tuning on the 3D-RAD-T training set (136,195 samples) significantly enhances model performance.

Conclusion: 3D-RAD establishes a robust foundation for 3D medical visual understanding and aims to catalyze multimodal medical AI research, with publicly available dataset and code to drive future advancements in clinical decision support.

Abstract: Medical Visual Question Answering (Med-VQA) holds significant potential for clinical decision support, yet existing efforts primarily focus on 2D imaging with limited task diversity. This paper presents 3D-RAD, a large-scale dataset designed to advance 3D Med-VQA using radiology CT scans. The 3D-RAD dataset encompasses six diverse VQA tasks: anomaly detection, image observation, medical computation, existence detection, static temporal diagnosis, and longitudinal temporal diagnosis. It supports both open- and closed-ended questions while introducing complex reasoning challenges, including computational tasks and multi-stage temporal analysis, to enable comprehensive benchmarking. Extensive evaluations demonstrate that existing vision-language models (VLMs), especially medical VLMs exhibit limited generalization, particularly in multi-temporal tasks, underscoring the challenges of real-world 3D diagnostic reasoning. To drive future advancements, we release a high-quality training set 3D-RAD-T of 136,195 expert-aligned samples, showing that fine-tuning on this dataset could significantly enhance model performance. Our dataset and code, aiming to catalyze multimodal medical AI research and establish a robust foundation for 3D medical visual understanding, are publicly available at https://github.com/Tang-xiaoxiao/3D-RAD.

Method: Unifies reasoning and action generation in single autoregressive model; tokenizes continuous trajectories into discrete actions; uses supervised fine-tuning with dual thinking modes (fast/slow); applies GRPO-based reinforcement fine-tuning to reduce unnecessary reasoning.

Result: Competitive performance demonstrated across nuPlan, nuScenes, Waymo, and CARLA benchmarks in both open-loop and closed-loop settings; shows adaptive reasoning and accurate planning in diverse scenarios.

Conclusion: AutoVLA effectively addresses limitations of current VLA models through unified architecture, tokenized actions, and optimized reasoning strategies, achieving strong performance in autonomous driving applications.

Abstract: Recent advancements in Vision-Language-Action (VLA) models have shown promise for end-to-end autonomous driving by leveraging world knowledge and reasoning capabilities. However, current VLA models often struggle with physically infeasible action outputs, complex model structures, or unnecessarily long reasoning. In this paper, we propose AutoVLA, a novel VLA model that unifies reasoning and action generation within a single autoregressive generation model for end-to-end autonomous driving. AutoVLA performs semantic reasoning and trajectory planning directly from raw visual inputs and language instructions. We tokenize continuous trajectories into discrete, feasible actions, enabling direct integration into the language model. For training, we employ supervised fine-tuning to equip the model with dual thinking modes: fast thinking (trajectory-only) and slow thinking (enhanced with chain-of-thought reasoning). To further enhance planning performance and efficiency, we introduce a reinforcement fine-tuning method based on Group Relative Policy Optimization (GRPO), reducing unnecessary reasoning in straightforward scenarios. Extensive experiments across real-world and simulated datasets and benchmarks, including nuPlan, nuScenes, Waymo, and CARLA, demonstrate the competitive performance of AutoVLA in both open-loop and closed-loop settings. Qualitative results showcase the adaptive reasoning and accurate planning capabilities of AutoVLA in diverse scenarios.

Method: Proposes a self-supervised framework with local aggregation, introducing specialized data augmentation and representation post-processing methods tailored specifically for cell images to improve cross-site consistency of cell representations.

Result: The proposed framework successfully addressed the challenges of multi-image fusion and domain differences, resulting in a robust feature extractor that won the Cell Line Transferability challenge at CVPR 2025.

Conclusion: The self-supervised learning framework with tailored data augmentation and post-processing methods effectively handles the unique challenges of cell image profiling and demonstrates superior performance in cross-site transferability tasks.

Abstract: Image-based cell profiling aims to create informative representations of cell images. This technique is critical in drug discovery and has greatly advanced with recent improvements in computer vision. Inspired by recent developments in non-contrastive Self-Supervised Learning (SSL), this paper provides an initial exploration into training a generalizable feature extractor for cell images using such methods. However, there are two major challenges: 1) Unlike typical scenarios where each representation is based on a single image, cell profiling often involves multiple input images, making it difficult to effectively fuse all available information; and 2) There is a large difference between the distributions of cell images and natural images, causing the view-generation process in existing SSL methods to fail. To address these issues, we propose a self-supervised framework with local aggregation to improve cross-site consistency of cell representations. We introduce specialized data augmentation and representation post-processing methods tailored to cell images, which effectively address the issues mentioned above and result in a robust feature extractor. With these improvements, the proposed framework won the Cell Line Transferability challenge at CVPR 2025.

Method: Adversarial feature intervention to suppress sensitive attributes and mitigate spurious correlations in CLIP models.

Result: Significantly enhances both fairness and diagnostic accuracy on chest X-ray datasets while maintaining robust generalization in zero-shot and few-shot scenarios.

Conclusion: Establishes new benchmarks for fairness-aware learning in CLIP-based medical diagnostic models, particularly for CXR analysis.

Abstract: Contrastive Language-Image Pre-training (CLIP) models have demonstrated superior performance across various visual tasks including medical image classification. However, fairness concerns, including demographic biases, have received limited attention for CLIP models. This oversight leads to critical issues, particularly those related to race and gender, resulting in disparities in diagnostic outcomes and reduced reliability for underrepresented groups. To address these challenges, we introduce AdFair-CLIP, a novel framework employing adversarial feature intervention to suppress sensitive attributes, thereby mitigating spurious correlations and improving prediction fairness. We conduct comprehensive experiments on chest X-ray (CXR) datasets, and show that AdFair-CLIP significantly enhances both fairness and diagnostic accuracy, while maintaining robust generalization in zero-shot and few-shot scenarios. These results establish new benchmarks for fairness-aware learning in CLIP-based medical diagnostic models, particularly for CXR analysis.

Method: A post-hoc prototype-based approach that refines class prototypes to mitigate bias from spurious features, applicable across diverse backbones and OOD detection settings without requiring additional data or hyperparameter tuning.

Result: SPROD demonstrates superior performance across challenging OOD datasets (CelebA, Waterbirds, UrbanCars, Spurious Imagenet, Animals MetaCoCo), improving AUROC by 4.8% and FPR@95 by 9.4% over the second best method on average.

Conclusion: SPROD effectively addresses spurious correlation challenges in OOD detection through prototype refinement, offering a robust and broadly applicable solution that significantly outperforms existing approaches.

Abstract: Out-of-distribution (OOD) detection is crucial for ensuring the reliability and safety of machine learning models in real-world applications, where they frequently face data distributions unseen during training. Despite progress, existing methods are often vulnerable to spurious correlations that mislead models and compromise robustness. To address this, we propose SPROD, a novel prototype-based OOD detection approach that explicitly addresses the challenge posed by unknown spurious correlations. Our post-hoc method refines class prototypes to mitigate bias from spurious features without additional data or hyperparameter tuning, and is broadly applicable across diverse backbones and OOD detection settings. We conduct a comprehensive spurious correlation OOD detection benchmarking, comparing our method against existing approaches and demonstrating its superior performance across challenging OOD datasets, such as CelebA, Waterbirds, UrbanCars, Spurious Imagenet, and the newly introduced Animals MetaCoCo. On average, SPROD improves AUROC by 4.8% and FPR@95 by 9.4% over the second best.

Method: Uses multi-view RGB images with 2D instance segmentation to generate organ masks, matches instance IDs across views, develops instance NeRF to render implicit scenes with color, density, semantic and instance information, then converts to high-precision point clouds based on volume density.

Result: Outperformed common methods in semantic segmentation with 16.1% precision, 18.3% recall, 17.8% F1-score, and 24.2% IoU improvements. In instance segmentation, achieved 11.7% mPrec, 38.2% mRec, 32.2% mCov, and 25.3% mWCov improvements across all plant species.

Conclusion: Provides a high-throughput way to supply high-quality 3D data for plant science, extending organ-level plant phenotyping capabilities and supporting development of large-scale models.

Abstract: Organ segmentation of plant point clouds is a prerequisite for the high-resolution and accurate extraction of organ-level phenotypic traits. Although the fast development of deep learning has boosted much research on segmentation of plant point clouds, the existing techniques for organ segmentation still face limitations in resolution, segmentation accuracy, and generalizability across various plant species. In this study, we proposed a novel approach called plant segmentation neural radiance fields (PlantSegNeRF), aiming to directly generate high-precision instance point clouds from multi-view RGB image sequences for a wide range of plant species. PlantSegNeRF performed 2D instance segmentation on the multi-view images to generate instance masks for each organ with a corresponding ID. The multi-view instance IDs corresponding to the same plant organ were then matched and refined using a specially designed instance matching module. The instance NeRF was developed to render an implicit scene, containing color, density, semantic and instance information. The implicit scene was ultimately converted into high-precision plant instance point clouds based on the volume density. The results proved that in semantic segmentation of point clouds, PlantSegNeRF outperformed the commonly used methods, demonstrating an average improvement of 16.1%, 18.3%, 17.8%, and 24.2% in precision, recall, F1-score, and IoU compared to the second-best results on structurally complex species. More importantly, PlantSegNeRF exhibited significant advantages in plant point cloud instance segmentation tasks. Across all plant species, it achieved average improvements of 11.7%, 38.2%, 32.2% and 25.3% in mPrec, mRec, mCov, mWCov, respectively. This study extends the organ-level plant phenotyping and provides a high-throughput way to supply high-quality 3D data for the development of large-scale models in plant science.

Method: Proposes a semantic prompt optimization mechanism and stage-wise decoupled generation paradigm that separates prompts into spatial and temporal components, using spatial prompts for text-to-image generation and temporal prompts for image-to-video generation.

Result: Experimental results show excellent spatiotemporal consistency with outstanding performance in identity preservation, text relevance, and video quality, achieving runner-up position in 2025 ACM MultiMedia Challenge.

Conclusion: The proposed spatial-temporal decoupled framework effectively addresses the trade-off between spatial coherence and temporal smoothness in identity-preserving video generation through simple yet robust mechanisms.

Abstract: Identity-preserving text-to-video (IPT2V) generation, which aims to create high-fidelity videos with consistent human identity, has become crucial for downstream applications. However, current end-to-end frameworks suffer a critical spatial-temporal trade-off: optimizing for spatially coherent layouts of key elements (e.g., character identity preservation) often compromises instruction-compliant temporal smoothness, while prioritizing dynamic realism risks disrupting the spatial coherence of visual structures. To tackle this issue, we propose a simple yet effective spatial-temporal decoupled framework that decomposes representations into spatial features for layouts and temporal features for motion dynamics. Specifically, our paper proposes a semantic prompt optimization mechanism and stage-wise decoupled generation paradigm. The former module decouples the prompt into spatial and temporal components. Aligned with the subsequent stage-wise decoupled approach, the spatial prompts guide the text-to-image (T2I) stage to generate coherent spatial features, while the temporal prompts direct the sequential image-to-video (I2V) stage to ensure motion consistency. Experimental results validate that our approach achieves excellent spatiotemporal consistency, demonstrating outstanding performance in identity preservation, text relevance, and video quality. By leveraging this simple yet robust mechanism, our algorithm secures the runner-up position in 2025 ACM MultiMedia Challenge. Our code is available at https://github.com/rain152/IPVG.

Method: THUNDER provides a fast, easy-to-use dynamic benchmark that supports comparison of state-of-the-art foundation models and user-defined models on diverse datasets with various downstream tasks, feature space analysis, and robustness evaluation.

Result: The benchmark enables comprehensive comparison of 23 foundation models across 16 different datasets, providing insights into performance differences, feature characteristics, and model robustness in digital pathology applications.

Conclusion: THUNDER addresses the critical need for standardized evaluation in digital pathology by offering a flexible benchmarking framework that assesses not only downstream performance but also feature spaces, uncertainty, and robustness of foundation models.

Abstract: Progress in a research field can be hard to assess, in particular when many concurrent methods are proposed in a short period of time. This is the case in digital pathology, where many foundation models have been released recently to serve as feature extractors for tile-level images, being used in a variety of downstream tasks, both for tile- and slide-level problems. Benchmarking available methods then becomes paramount to get a clearer view of the research landscape. In particular, in critical domains such as healthcare, a benchmark should not only focus on evaluating downstream performance, but also provide insights about the main differences between methods, and importantly, further consider uncertainty and robustness to ensure a reliable usage of proposed models. For these reasons, we introduce THUNDER, a tile-level benchmark for digital pathology foundation models, allowing for efficient comparison of many models on diverse datasets with a series of downstream tasks, studying their feature spaces and assessing the robustness and uncertainty of predictions informed by their embeddings. THUNDER is a fast, easy-to-use, dynamic benchmark that can already support a large variety of state-of-the-art foundation, as well as local user-defined models for direct tile-based comparison. In this paper, we provide a comprehensive comparison of 23 foundation models on 16 different datasets covering diverse tasks, feature analysis, and robustness. The code for THUNDER is publicly available at https://github.com/MICS-Lab/thunder.

Method: Uses frozen vision foundation model as encoder, introduces region-adaptive quantization to reduce redundancy in pre-trained features, and semantic reconstruction objective to align outputs with foundation model representations.

Result: Achieves gFID of 1.36 on ImageNet benchmarks, accelerates model convergence by 3x, enables high-fidelity class-conditional synthesis without CFG, and improves image reconstruction and generation quality.

Conclusion: VFMTok demonstrates that building image tokenizers on frozen vision foundation models with proper quantization and semantic alignment can significantly enhance performance and efficiency in image generation tasks.

Abstract: In this work, we present a novel direction to build an image tokenizer directly on top of a frozen vision foundation model, which is a largely underexplored area. Specifically, we employ a frozen vision foundation model as the encoder of our tokenizer. To enhance its effectiveness, we introduce two key components: (1) a region-adaptive quantization framework that reduces redundancy in the pre-trained features on regular 2D grids, and (2) a semantic reconstruction objective that aligns the tokenizer’s outputs with the foundation model’s representations to preserve semantic fidelity. Based on these designs, our proposed image tokenizer, VFMTok, achieves substantial improvements in image reconstruction and generation quality, while also enhancing token efficiency. It further boosts autoregressive (AR) generation – achieving a gFID of 1.36 on ImageNet benchmarks, while accelerating model convergence by three times, and enabling high-fidelity class-conditional synthesis without the need for classifier-free guidance (CFG). The code is available at https://github.com/CVMI-Lab/VFMTok.

Method: Two loss functions: 1) Unsupervised Contrastive Loss (UCL) using negative samples to increase dissimilarity among synthetic images, combined with batch resampling; 2) Alignment Loss (AL) that aligns class-conditional with unconditional generation at large timesteps to make denoising insensitive to class conditions initially.

Result: The method outperforms vanilla denoising diffusion probabilistic models, score-based diffusion models, and alternative methods for class-imbalanced image generation across various datasets, particularly ImageNet-LT with 256x256 resolution.

Conclusion: The framework successfully leverages contrastive learning and conditional-unconditional alignment for class-imbalanced diffusion models, is easy to implement on both U-Net and Diffusion Transformer architectures, and effectively improves tail class image diversity and fidelity.

Abstract: Training data for class-conditional image synthesis often exhibit a long-tailed distribution with limited images for tail classes. Such an imbalance causes mode collapse and reduces the diversity of synthesized images for tail classes. For class-conditional diffusion models trained on imbalanced data, we aim to improve the diversity and fidelity of tail class images without compromising the quality of head class images. We achieve this by introducing two simple but highly effective loss functions. Firstly, we employ an Unsupervised Contrastive Loss (UCL) utilizing negative samples to increase the distance/dissimilarity among synthetic images. Such regularization is coupled with a standard trick of batch resampling to further diversify tail-class images. Our second loss is an Alignment Loss (AL) that aligns class-conditional generation with unconditional generation at large timesteps. This second loss makes the denoising process insensitive to class conditions for the initial steps, which enriches tail classes through knowledge sharing from head classes. We successfully leverage contrastive learning and conditional-unconditional alignment for class-imbalanced diffusion models. Our framework is easy to implement as demonstrated on both U-Net based architecture and Diffusion Transformer. Our method outperforms vanilla denoising diffusion probabilistic models, score-based diffusion model, and alternative methods for class-imbalanced image generation across various datasets, in particular ImageNet-LT with 256x256 resolution.

Method: Proposes CapRecover framework that directly recovers high-level semantic content from intermediate features using cross-modality inversion. Also introduces a protection method adding random noise to intermediate features at each layer and removing it in the next layer.

Result: Achieves 92.71% Top-1 label accuracy on CIFAR-10 and generates fluent captions from ResNet50 features on COCO2017 with ROUGE-L scores up to 0.52. Deeper convolutional layers encode more semantic information than shallow layers. The protection method effectively prevents semantic leakage without training costs.

Conclusion: CapRecover effectively addresses semantic leakage in split-DNN VLMs by directly recovering semantic content from intermediate features, and provides a practical protection method against such attacks.

Abstract: As Vision-Language Models (VLMs) are increasingly deployed in split-DNN configurations–with visual encoders (e.g., ResNet, ViT) operating on user devices and sending intermediate features to the cloud–there is a growing privacy risk from semantic information leakage. Existing approaches to reconstructing images from these intermediate features often result in blurry, semantically ambiguous images. To directly address semantic leakage, we propose CapRecover, a cross-modality inversion framework that recovers high-level semantic content, such as labels or captions, directly from intermediate features without image reconstruction. We evaluate CapRecover on multiple datasets and victim models, demonstrating strong performance in semantic recovery. Specifically, CapRecover achieves up to 92.71% Top-1 label accuracy on CIFAR-10 and generates fluent captions from ResNet50 features on COCO2017 with ROUGE-L scores up to 0.52. Our analysis further reveals that deeper convolutional layers encode significantly more semantic information compared to shallow layers. To mitigate semantic leakage, we introduce a simple yet effective protection method: adding random noise to intermediate features at each layer and removing the noise in the next layer. Experimental results show that this approach prevents semantic leakage without additional training costs. Our code is available at https://jus1mple.github.io/Image2CaptionAttack.

Method: Proposes AlignCAT with two modules: coarse-grained alignment using category information and global context to mitigate category-inconsistent objects, and fine-grained alignment using descriptive information and word-level text features for attribute consistency. Progressively filters misaligned visual queries and enhances contrastive learning.

Result: Extensive experiments on RefCOCO, RefCOCO+, and RefCOCOg benchmarks verify superiority against existing weakly supervised methods on two VG tasks.

Conclusion: AlignCAT effectively addresses category and attribute ambiguity in weakly supervised visual grounding by exploiting linguistic cues through progressive alignment modules.

Abstract: Weakly supervised visual grounding (VG) aims to locate objects in images based on text descriptions. Despite significant progress, existing methods lack strong cross-modal reasoning to distinguish subtle semantic differences in text expressions due to category-based and attribute-based ambiguity. To address these challenges, we introduce AlignCAT, a novel query-based semantic matching framework for weakly supervised VG. To enhance visual-linguistic alignment, we propose a coarse-grained alignment module that utilizes category information and global context, effectively mitigating interference from category-inconsistent objects. Subsequently, a fine-grained alignment module leverages descriptive information and captures word-level text features to achieve attribute consistency. By exploiting linguistic cues to their fullest extent, our proposed AlignCAT progressively filters out misaligned visual queries and enhances contrastive learning efficiency. Extensive experiments on three VG benchmarks, namely RefCOCO, RefCOCO+, and RefCOCOg, verify the superiority of AlignCAT against existing weakly supervised methods on two VG tasks. Our code is available at: https://github.com/I2-Multimedia-Lab/AlignCAT.

Method: Proposes Hessian-aware Salient Data Selection to construct high-quality calibration datasets, and Attention-guided Sparse Token Distillation that exploits token-wise attention distributions to emphasize influential tokens.

Result: Under W4A6 quantization, achieves lossless performance with 3.9× model compression and 1.3× inference acceleration.

Conclusion: S²Q-VDiT effectively addresses quantization challenges in video diffusion models through data selection and token distillation techniques, enabling efficient deployment without performance degradation.

Abstract: Diffusion transformers have emerged as the mainstream paradigm for video generation models. However, the use of up to billions of parameters incurs significant computational costs. Quantization offers a promising solution by reducing memory usage and accelerating inference. Nonetheless, we observe that the joint modeling of spatial and temporal information in video diffusion models (V-DMs) leads to extremely long token sequences, which introduces high calibration variance and learning challenges. To address these issues, we propose S$^2$Q-VDiT, a post-training quantization framework for V-DMs that leverages Salient data and Sparse token distillation. During the calibration phase, we identify that quantization performance is highly sensitive to the choice of calibration data. To mitigate this, we introduce \textit{Hessian-aware Salient Data Selection}, which constructs high-quality calibration datasets by considering both diffusion and quantization characteristics unique to V-DMs. To tackle the learning challenges, we further analyze the sparse attention patterns inherent in V-DMs. Based on this observation, we propose \textit{Attention-guided Sparse Token Distillation}, which exploits token-wise attention distributions to emphasize tokens that are more influential to the model’s output. Under W4A6 quantization, S$^2$Q-VDiT achieves lossless performance while delivering $3.9\times$ model compression and $1.3\times$ inference acceleration. Code will be available at https://github.com/wlfeng0509/s2q-vdit.

Method: Uses two key innovations: Selective Canny Control (applies structural guidance only to unedited regions) and Dual-Prompt Guidance (uses both local and global prompts for coherence). Works with rough masks or single-point hints and integrates with vision-language models.

Result: Achieves superior trade-off among text adherence, context fidelity, and editing seamlessness compared to current region-based methods. Performs well in complex object addition scenarios against leading instruction-based editors.

Conclusion: CannyEdit enables controllable local editing for object addition, replacement, and removal with exceptional flexibility and seamless integration capabilities, addressing the fundamental trilemma in regional image editing.

Abstract: Recent advances in text-to-image (T2I) models have enabled training-free regional image editing by leveraging the generative priors of foundation models. However, existing methods struggle to balance text adherence in edited regions, context fidelity in unedited areas, and seamless integration of edits. We introduce CannyEdit, a novel training-free framework that addresses this trilemma through two key innovations. First, Selective Canny Control applies structural guidance from a Canny ControlNet only to the unedited regions, preserving the original image’s details while allowing for precise, text-driven changes in the specified editable area. Second, Dual-Prompt Guidance utilizes both a local prompt for the specific edit and a global prompt for overall scene coherence. Through this synergistic approach, these components enable controllable local editing for object addition, replacement, and removal, achieving a superior trade-off among text adherence, context fidelity, and editing seamlessness compared to current region-based methods. Beyond this, CannyEdit offers exceptional flexibility: it operates effectively with rough masks or even single-point hints in addition tasks. Furthermore, the framework can seamlessly integrate with vision-language models in a training-free manner for complex instruction-based editing that requires planning and reasoning. Our extensive evaluations demonstrate CannyEdit’s strong performance against leading instruction-based editors in complex object addition scenarios.

Method: The paper surveys RL-based methods across image, video, and 3D/4D generation domains. It reviews RL’s evolution from classical control to general-purpose optimization tool, examining how RL serves both as fine-tuning mechanism and structural component for aligning generation with complex goals.

Result: Recent advances demonstrate RL’s effectiveness in enhancing controllability, consistency, and human alignment across various generative tasks in visual content creation.

Conclusion: The survey concludes with open challenges and future research directions at the intersection of RL and generative modeling, highlighting RL’s potential as a general-purpose optimization framework for visual content generation.

Abstract: Generative models have made significant progress in synthesizing visual content, including images, videos, and 3D/4D structures. However, they are typically trained with surrogate objectives such as likelihood or reconstruction loss, which often misalign with perceptual quality, semantic accuracy, or physical realism. Reinforcement learning (RL) offers a principled framework for optimizing non-differentiable, preference-driven, and temporally structured objectives. Recent advances demonstrate its effectiveness in enhancing controllability, consistency, and human alignment across generative tasks. This survey provides a systematic overview of RL-based methods for visual content generation. We review the evolution of RL from classical control to its role as a general-purpose optimization tool, and examine its integration into image, video, and 3D/4D generation. Across these domains, RL serves not only as a fine-tuning mechanism but also as a structural component for aligning generation with complex, high-level goals. We conclude with open challenges and future research directions at the intersection of RL and generative modeling.

Method: Evaluated YOLOv8, YOLOv11, YOLOv12 models, then developed a custom YOLOv8-based model with added structures for challenging surveillance contexts. Tested on 8,124 images from 20 scenarios including 2,708 low-light samples.

Result: Proposed model outperformed others with 78.90% recall and 83.70% mAP@50. Jetson Xavier NX processed data at 52-97 ms per inference, suitable for real-time operations.

Conclusion: The system provides a robust and adaptable platform for public safety monitoring and automatic regulatory compliance in time-sensitive surveillance applications.

Abstract: A deep learning real-time smoking detection system for CCTV surveillance of fire exit areas is proposed due to critical safety requirements. The dataset contains 8,124 images from 20 different scenarios along with 2,708 raw samples demonstrating low-light areas. We evaluated three advanced object detection models: YOLOv8, YOLOv11, and YOLOv12, followed by development of a custom model derived from YOLOv8 with added structures for challenging surveillance contexts. The proposed model outperformed the others, achieving a recall of 78.90 percent and mAP at 50 of 83.70 percent, delivering optimal object detection across varied environments. Performance evaluation on multiple edge devices using multithreaded operations showed the Jetson Xavier NX processed data at 52 to 97 milliseconds per inference, establishing its suitability for time-sensitive operations. This system offers a robust and adaptable platform for monitoring public safety and enabling automatic regulatory compliance.

Method: Proposed Context-Aware Fusion (CAF) that uses cross-attention mechanisms to integrate global scene context with local proposal features. A separate dedicated encoder captures comprehensive environmental information, allowing each object proposal to attend to scene-level understanding.

Result: Experimental results show improvement over state-of-the-art models on the CarDD benchmark, establishing new performance benchmarks for context-aware object detection in fine-grained domains.

Conclusion: The framework significantly enhances the generative detection paradigm by enabling object proposals to attend to comprehensive environmental information, addressing fundamental limitations of previous approaches.

Abstract: Fine-grained object detection in challenging visual domains, such as vehicle damage assessment, presents a formidable challenge even for human experts to resolve reliably. While DiffusionDet has advanced the state-of-the-art through conditional denoising diffusion, its performance remains limited by local feature conditioning in context-dependent scenarios. We address this fundamental limitation by introducing Context-Aware Fusion (CAF), which leverages cross-attention mechanisms to integrate global scene context with local proposal features directly. The global context is generated using a separate dedicated encoder that captures comprehensive environmental information, enabling each object proposal to attend to scene-level understanding. Our framework significantly enhances the generative detection paradigm by enabling each object proposal to attend to comprehensive environmental information. Experimental results demonstrate an improvement over state-of-the-art models on the CarDD benchmark, establishing new performance benchmarks for context-aware object detection in fine-grained domains

Method: Developed ViSK-GAT framework with two modules: Fine-Grained Attention Module (FGAM) for cross-attention between visual and skeletal inputs, and Multimodal Geometric Correspondence Module (MGCM) for cross-modal alignment. Used custom MusDis-Sports dataset with images and skeletal coordinates labeled into eight REBA risk categories.

Result: Achieved robust performance with all key metrics exceeding 93%. Regression results showed low RMSE (0.1205) and MAE (0.0156). Consistently outperformed nine popular transfer learning backbones.

Conclusion: ViSK-GAT demonstrates potential to advance AI-driven musculoskeletal risk assessment and enable early interventions in sports through its multimodal approach.

Abstract: Musculoskeletal disorders pose significant risks to athletes, and assessing risk early is important for prevention. However, most existing methods are designed for controlled settings and fail to reliably assess risk in complex environments due to their reliance on a single type of data. This research introduces ViSK-GAT (Visual-Skeletal Geometric Attention Transformer), a novel multimodal deep learning framework that classifies musculoskeletal risk using both visual and skeletal coordinate-based features. A custom multimodal dataset (MusDis-Sports) was created by combining images and skeletal coordinates, with each sample labeled into eight risk categories based on the Rapid Entire Body Assessment (REBA) system. ViSK-GAT integrates two innovative modules: the Fine-Grained Attention Module (FGAM), which refines inter-modal features via cross-attention between visual and skeletal inputs, and the Multimodal Geometric Correspondence Module (MGCM), which enhances cross-modal alignment between image features and coordinates. The model achieved robust performance, with all key metrics exceeding 93%. Regression results also indicated a low RMSE of 0.1205 and MAE of 0.0156. ViSK-GAT consistently outperformed nine popular transfer learning backbones and showed its potential to advance AI-driven musculoskeletal risk assessment and enable early, impactful interventions in sports.

Method: RecA conditions a UMM on its own visual understanding embeddings and optimizes it to reconstruct the input image using self-supervised reconstruction loss.

Result: RecA consistently improves generation and editing fidelity across different UMM architectures, boosting GenEval (0.73→0.90), DPGBench (80.93→88.15), ImgEdit (3.38→3.75), and GEdit (6.94→7.25) with only 27 GPU-hours.

Conclusion: RecA is an efficient and general post-training alignment strategy that surpasses larger models and applies broadly across diverse UMM architectures.

Abstract: Unified multimodal models (UMMs) unify visual understanding and generation within a single architecture. However, conventional training relies on image-text pairs (or sequences) whose captions are typically sparse and miss fine-grained visual details–even when they use hundreds of words to describe a simple image. We introduce Reconstruction Alignment (RecA), a resource-efficient post-training method that leverages visual understanding encoder embeddings as dense “text prompts,” providing rich supervision without captions. Concretely, RecA conditions a UMM on its own visual understanding embeddings and optimizes it to reconstruct the input image with a self-supervised reconstruction loss, thereby realigning understanding and generation. Despite its simplicity, RecA is broadly applicable: across autoregressive, masked-autoregressive, and diffusion-based UMMs, it consistently improves generation and editing fidelity. With only 27 GPU-hours, post-training with RecA substantially improves image generation performance on GenEval (0.73$\rightarrow$0.90) and DPGBench (80.93$\rightarrow$88.15), while also boosting editing benchmarks (ImgEdit 3.38$\rightarrow$3.75, GEdit 6.94$\rightarrow$7.25). Notably, RecA surpasses much larger open-source models and applies broadly across diverse UMM architectures, establishing it as an efficient and general post-training alignment strategy for UMMs

Method: Proposes SA-UNetv2 with cross-scale spatial attention injected into all skip connections to strengthen multi-scale feature fusion, and uses weighted Binary Cross-Entropy plus Matthews Correlation Coefficient loss to improve robustness to class imbalance.

Result: Achieves state-of-the-art performance on DRIVE and STARE datasets with only 1.2MB memory and 0.26M parameters (less than 50% of SA-UNet), and 1 second CPU inference on 592 x 592 x 3 images.

Conclusion: SA-UNetv2 demonstrates strong efficiency and deployability in resource-constrained, CPU-only settings for retinal vessel segmentation tasks.

Abstract: Retinal vessel segmentation is essential for early diagnosis of diseases such as diabetic retinopathy, hypertension, and neurodegenerative disorders. Although SA-UNet introduces spatial attention in the bottleneck, it underuses attention in skip connections and does not address the severe foreground-background imbalance. We propose SA-UNetv2, a lightweight model that injects cross-scale spatial attention into all skip connections to strengthen multi-scale feature fusion and adopts a weighted Binary Cross-Entropy (BCE) plus Matthews Correlation Coefficient (MCC) loss to improve robustness to class imbalance. On the public DRIVE and STARE datasets, SA-UNetv2 achieves state-of-the-art performance with only 1.2MB memory and 0.26M parameters (less than 50% of SA-UNet), and 1 second CPU inference on 592 x 592 x 3 images, demonstrating strong efficiency and deployability in resource-constrained, CPU-only settings.

Method: Introduces a DSC block with two components: Test-Time Training module for dynamic adaptation during inference, and Dynamic Multi-Scale Kernel module for adaptive kernel size selection based on global context.

Result: The DSC block demonstrates plug-and-play effectiveness across various U-like network architectures including CNN-based, Transformer-based, hybrid CNN-Transformer, and Mamba-based networks.

Conclusion: The proposed DSC block fundamentally enhances cross-layer connectivity through adaptive mechanisms and can be seamlessly integrated into existing U-like network structures.

Abstract: U-like networks have become fundamental frameworks in medical image segmentation through skip connections that bridge high-level semantics and low-level spatial details. Despite their success, conventional skip connections exhibit two key limitations: inter-feature constraints and intra-feature constraints. The inter-feature constraint refers to the static nature of feature fusion in traditional skip connections, where information is transmitted along fixed pathways regardless of feature content. The intra-feature constraint arises from the insufficient modeling of multi-scale feature interactions, thereby hindering the effective aggregation of global contextual information. To overcome these limitations, we propose a novel Dynamic Skip Connection (DSC) block that fundamentally enhances cross-layer connectivity through adaptive mechanisms. The DSC block integrates two complementary components. (1) Test-Time Training (TTT) module. This module addresses the inter-feature constraint by enabling dynamic adaptation of hidden representations during inference, facilitating content-aware feature refinement. (2) Dynamic Multi-Scale Kernel (DMSK) module. To mitigate the intra-feature constraint, this module adaptively selects kernel sizes based on global contextual cues, enhancing the network capacity for multi-scale feature integration. The DSC block is architecture-agnostic and can be seamlessly incorporated into existing U-like network structures. Extensive experiments demonstrate the plug-and-play effectiveness of the proposed DSC block across CNN-based, Transformer-based, hybrid CNN-Transformer, and Mamba-based U-like networks.

Method: BTL framework with three phases: Blink (attention detection), Think (reasoning), and Link (action generation). Includes automated blink data generation and a rule-based reward mechanism for reinforcement learning.

Result: Developed BTL-UI agent model showing competitive performance in both static GUI understanding and dynamic interaction tasks across comprehensive benchmarks.

Conclusion: The framework provides conclusive empirical validation for developing advanced GUI agents that better mimic human cognitive processes.

Abstract: In the field of AI-driven human-GUI interaction automation, while rapid advances in multimodal large language models and reinforcement fine-tuning techniques have yielded remarkable progress, a fundamental challenge persists: their interaction logic significantly deviates from natural human-GUI communication patterns. To fill this gap, we propose “Blink-Think-Link” (BTL), a brain-inspired framework for human-GUI interaction that mimics the human cognitive process between users and graphical interfaces. The system decomposes interactions into three biologically plausible phases: (1) Blink - rapid detection and attention to relevant screen areas, analogous to saccadic eye movements; (2) Think - higher-level reasoning and decision-making, mirroring cognitive planning; and (3) Link - generation of executable commands for precise motor control, emulating human action selection mechanisms. Additionally, we introduce two key technical innovations for the BTL framework: (1) Blink Data Generation - an automated annotation pipeline specifically optimized for blink data, and (2) BTL Reward – the first rule-based reward mechanism that enables reinforcement learning driven by both process and outcome. Building upon this framework, we develop a GUI agent model named BTL-UI, which demonstrates competitive performance across both static GUI understanding and dynamic interaction tasks in comprehensive benchmarks. These results provide conclusive empirical validation of the framework’s efficacy in developing advanced GUI Agents.

Method: UniPixel processes visual prompts, generates relevant masks on demand, and performs reasoning conditioned on these intermediate pointers during inference, enabling pixel-level reasoning.

Result: Verified on 10 benchmarks across diverse tasks including pixel-level referring/segmentation, object-centric understanding in images/videos, and a novel PixelQA task requiring joint referring, segmentation, and QA.

Conclusion: UniPixel successfully bridges the gap between pixel-level perception and visual reasoning, demonstrating flexible comprehension of visual prompts and mask-grounded responses across multiple tasks.

Abstract: Recent advances in Large Multi-modal Models (LMMs) have demonstrated their remarkable success as general-purpose multi-modal assistants, with particular focuses on holistic image- and video-language understanding. Conversely, less attention has been given to scaling fine-grained pixel-level understanding capabilities, where the models are expected to realize pixel-level alignment between visual signals and language semantics. Some previous studies have applied LMMs to related tasks such as region-level captioning and referring expression segmentation. However, these models are limited to performing either referring or segmentation tasks independently and fail to integrate these fine-grained perception capabilities into visual reasoning. To bridge this gap, we propose UniPixel, a large multi-modal model capable of flexibly comprehending visual prompt inputs and generating mask-grounded responses. Our model distinguishes itself by seamlessly integrating pixel-level perception with general visual understanding capabilities. Specifically, UniPixel processes visual prompts and generates relevant masks on demand, and performs subsequent reasoning conditioning on these intermediate pointers during inference, thereby enabling fine-grained pixel-level reasoning. The effectiveness of our approach has been verified on 10 benchmarks across a diverse set of tasks, including pixel-level referring/segmentation and object-centric understanding in images/videos. A novel PixelQA task that jointly requires referring, segmentation, and question answering is also designed to verify the flexibility of our method.

Method: Large-scale evaluation of quantization on CLIP models, assessing in-distribution accuracy, calibration, and OOD detection, and identifying effective quantization-aware training methods.

Result: Quantization improves calibration for underconfident models but degrades it for overconfident ones; OOD detection can still improve despite poor calibration; specific QAT methods achieve simultaneous gains in accuracy, calibration, and OOD robustness.

Conclusion: Quantization can be leveraged beyond efficiency to enhance reliability and robustness in CLIP models, challenging the traditional efficiency-performance trade-off and providing insights for multi-objective deployment.

Abstract: The powerful zero-shot generalization capabilities of vision-language models (VLMs) like CLIP have enabled new paradigms for safety-related tasks such as out-of-distribution (OOD) detection. However, additional aspects crucial for the computationally efficient and reliable deployment of CLIP are still overlooked. In particular, the impact of quantization on CLIP’s performance beyond accuracy remains underexplored. This work presents a large-scale evaluation of quantization on CLIP models, assessing not only in-distribution accuracy but a comprehensive suite of reliability metrics and revealing counterintuitive results driven by pre-training source. We demonstrate that quantization consistently improves calibration for typically underconfident pre-trained models, while often degrading it for overconfident variants. Intriguingly, this degradation in calibration does not preclude gains in other reliability metrics; we find that OOD detection can still improve for these same poorly calibrated models. Furthermore, we identify specific quantization-aware training (QAT) methods that yield simultaneous gains in zero-shot accuracy, calibration, and OOD robustness, challenging the view of a strict efficiency-performance trade-off. These findings offer critical insights for navigating the multi-objective problem of deploying efficient, reliable, and robust VLMs by utilizing quantization beyond its conventional role.

Method: Dual-architecture approach: CNN-LSTM with ResNet50 backbone for satellite imagery (xBD dataset) and Vision Transformer for UAV pictures (RescueNet dataset), enhanced with external semantic knowledge from ConceptNet and WordNet.

Result: VLCE significantly outperforms baseline models (LLaVA and QwenVL), achieving up to 95.33% on InfoMetIC for caption informativeness while maintaining competitive semantic alignment with CLIPScore.

Conclusion: The dual-architecture system shows significant potential for improving disaster damage assessment by automating production of actionable, information-dense descriptions from satellite and drone imagery.

Abstract: Immediate damage assessment is essential after natural catastrophes; yet, conventional hand evaluation techniques are sluggish and perilous. Although satellite and unmanned aerial vehicle (UAV) photos offer extensive perspectives of impacted regions, current computer vision methodologies generally yield just classification labels or segmentation masks, so constraining their capacity to deliver a thorough situational comprehension. We introduce the Vision Language Caption Enhancer (VLCE), a multimodal system designed to produce comprehensive, contextually-informed explanations of disaster imagery. VLCE employs a dual-architecture approach: a CNN-LSTM model with a ResNet50 backbone pretrained on EuroSat satellite imagery for the xBD dataset, and a Vision Transformer (ViT) model pretrained on UAV pictures for the RescueNet dataset. Both systems utilize external semantic knowledge from ConceptNet and WordNet to expand vocabulary coverage and improve description accuracy. We assess VLCE in comparison to leading vision-language models (LLaVA and QwenVL) utilizing CLIPScore for semantic alignment and InfoMetIC for caption informativeness. Experimental findings indicate that VLCE markedly surpasses baseline models, attaining a maximum of 95.33% on InfoMetIC while preserving competitive semantic alignment. Our dual-architecture system demonstrates significant potential for improving disaster damage assessment by automating the production of actionable, information-dense descriptions from satellite and drone photos.

Method: Proposes an image-plane decoding framework with three components: Pixel-level Confidence Encoder, Affine Compensation Module, and Image-Plane Spatial Decoder. These modules decode 3D structural information encoded in images through physical imaging processes to preserve spatial geometric features.

Result: Achieves superior reconstruction stability with nearly identical quality when view count reduces by 40%. Achieves coefficient of variation of 0.24%, performance retention rate of 99.7%, and maximum performance drop of 0.42% on indoor scene reconstruction datasets.

Conclusion: Exploiting intra-view spatial information provides a robust solution for view-limited scenarios in practical applications, significantly enhancing view-invariant reconstruction.

Abstract: Volume-based indoor scene reconstruction methods offer superior generalization capability and real-time deployment potential. However, existing methods rely on multi-view pixel back-projection ray intersections as weak geometric constraints to determine spatial positions. This dependence results in reconstruction quality being heavily influenced by input view density. Performance degrades in overlapping regions and unobserved areas.To address these limitations, we reduce dependency on inter-view geometric constraints by exploiting spatial information within individual views. We propose an image-plane decoding framework with three core components: Pixel-level Confidence Encoder, Affine Compensation Module, and Image-Plane Spatial Decoder. These modules decode three-dimensional structural information encoded in images through physical imaging processes. The framework effectively preserves spatial geometric features including edges, hollow structures, and complex textures. It significantly enhances view-invariant reconstruction.Experiments on indoor scene reconstruction datasets confirm superior reconstruction stability. Our method maintains nearly identical quality when view count reduces by 40%. It achieves a coefficient of variation of 0.24%, performance retention rate of 99.7%, and maximum performance drop of 0.42%. These results demonstrate that exploiting intra-view spatial information provides a robust solution for view-limited scenarios in practical applications.

Method: Proposes Editable Noise Map Inversion (ENM Inversion) that searches for optimal noise maps through editable noise refinement, minimizing the difference between reconstructed and edited noise maps based on analysis of noise map properties for enhanced editability.

Result: Extensive experiments show ENM Inversion outperforms existing approaches across a wide range of image editing tasks in both preservation and edit fidelity with target prompts, and can be applied to video editing for temporal consistency.

Conclusion: ENM Inversion effectively addresses the trade-off between content preservation and editability in text-guided image editing, providing superior performance and extending to consistent video editing applications.

Abstract: Text-to-image diffusion models have achieved remarkable success in generating high-quality and diverse images. Building on these advancements, diffusion models have also demonstrated exceptional performance in text-guided image editing. A key strategy for effective image editing involves inverting the source image into editable noise maps associated with the target image. However, previous inversion methods face challenges in adhering closely to the target text prompt. The limitation arises because inverted noise maps, while enabling faithful reconstruction of the source image, restrict the flexibility needed for desired edits. To overcome this issue, we propose Editable Noise Map Inversion (ENM Inversion), a novel inversion technique that searches for optimal noise maps to ensure both content preservation and editability. We analyze the properties of noise maps for enhanced editability. Based on this analysis, our method introduces an editable noise refinement that aligns with the desired edits by minimizing the difference between the reconstructed and edited noise maps. Extensive experiments demonstrate that ENM Inversion outperforms existing approaches across a wide range of image editing tasks in both preservation and edit fidelity with target prompts. Our approach can also be easily applied to video editing, enabling temporal consistency and content manipulation across frames.

Method: Proposes an information-theoretic perspective revealing that successful denoising reduces conditional entropy, leading to EVODiff which optimizes conditional variance to minimize transition and reconstruction errors.

Result: EVODiff reduces reconstruction error by up to 45.5% (FID from 5.10 to 2.78) at 10 NFE on CIFAR-10, cuts NFE cost by 25% on ImageNet-256, and improves text-to-image generation with reduced artifacts.

Conclusion: The information-theoretic approach provides theoretical foundations for diffusion model inference, and EVODiff demonstrates significant improvements over state-of-the-art methods in both efficiency and generation quality.

Abstract: Diffusion models (DMs) excel in image generation, but suffer from slow inference and the training-inference discrepancies. Although gradient-based solvers like DPM-Solver accelerate the denoising inference, they lack theoretical foundations in information transmission efficiency. In this work, we introduce an information-theoretic perspective on the inference processes of DMs, revealing that successful denoising fundamentally reduces conditional entropy in reverse transitions. This principle leads to our key insights into the inference processes: (1) data prediction parameterization outperforms its noise counterpart, and (2) optimizing conditional variance offers a reference-free way to minimize both transition and reconstruction errors. Based on these insights, we propose an entropy-aware variance optimized method for the generative process of DMs, called EVODiff, which systematically reduces uncertainty by optimizing conditional entropy during denoising. Extensive experiments on DMs validate our insights and demonstrate that our method significantly and consistently outperforms state-of-the-art (SOTA) gradient-based solvers. For example, compared to the DPM-Solver++, EVODiff reduces the reconstruction error by up to 45.5% (FID improves from 5.10 to 2.78) at 10 function evaluations (NFE) on CIFAR-10, cuts the NFE cost by 25% (from 20 to 15 NFE) for high-quality samples on ImageNet-256, and improves text-to-image generation while reducing artifacts. Code is available at https://github.com/ShiguiLi/EVODiff.

Method: InstaFormer uses interactions between object queries and latent mask descriptors that semantically represent the same objects while carrying complementary information, enabling holistic order prediction in a single forward pass.

Result: The approach is comprehensively benchmarked and ablated to demonstrate its effectiveness in predicting occlusion and depth orderings from RGB images.

Conclusion: InstaFormer provides an efficient solution for instance-wise geometry understanding with reduced input requirements and inference costs, with code and models available as open-source.

Abstract: Even in controlled settings, understanding instance-wise geometries is a challenging task for a wide range of visual models. Although specialized systems exist, modern arts rely on expensive input formats (category labels, binary segmentation masks) and inference costs (a quadratic amount of forward passes). We mitigate these limitations by proposing InstaFormer, a network capable of holistic order prediction. That is, solely given an input RGB image, InstaFormer returns the full occlusion and depth orderings for all the instances in the scene in a single forward pass. At its core, InstaFormer relies on interactions between object queries and latent mask descriptors that semantically represent the same objects while carrying complementary information. We comprehensively benchmark and ablate our approach to highlight its effectiveness. Our code and models are open-source and available at this URL: https://github.com/SNU-VGILab/InstaOrder.

Method: Hybrid framework combining unsupervised Deep Image Prior (DIP) and supervised pre-trained DRUNet model using regularization by denoising (RED). Untrained network adapts to input-specific noise, while pre-trained network leverages learned representations from large datasets.

Result: Extensive experiments on benchmark datasets demonstrate superior performance for real-world noise removal, particularly in scenarios with limited training data.

Conclusion: Net2Net effectively addresses real-world noise removal challenges by combining the strengths of untrained and pre-trained networks, enhancing generalization across varying noise patterns and improving performance with limited data.

Abstract: Traditional denoising methods for noise removal have largely relied on handcrafted priors, often perform well in controlled environments but struggle to address the complexity and variability of real noise. In contrast, deep learning-based approaches have gained prominence for learning noise characteristics from large datasets, but these methods frequently require extensive labeled data and may not generalize effectively across diverse noise types and imaging conditions. In this paper, we present an innovative method, termed as Net2Net, that combines the strengths of untrained and pre-trained networks to tackle the challenges of real-world noise removal. The innovation of Net2Net lies in its combination of unsupervised DIP and supervised pre-trained model DRUNet by regularization by denoising (RED). The untrained network adapts to the unique noise characteristics of each input image without requiring labeled data, while the pre-trained network leverages learned representations from large-scale datasets to deliver robust denoising performance. This hybrid framework enhances generalization across varying noise patterns and improves performance, particularly in scenarios with limited training data. Extensive experiments on benchmark datasets demonstrate the superiority of our method for real-world noise removal.

Method: Introduces information compression module with details inspection mechanism to reduce tokens while preserving critical information, and a new training paradigm with activation stage using retrieval-tailored synthetic CoT dataset followed by RL with curriculum reward.

Result: Retrv-R1 achieves state-of-the-art performance, high efficiency, and strong generalization ability across multiple benchmarks and tasks.

Conclusion: The proposed approach successfully enables effective RL-based training for multimodal retrieval tasks while maintaining computational efficiency and performance.

Abstract: The success of DeepSeek-R1 demonstrates the immense potential of using reinforcement learning (RL) to enhance LLMs’ reasoning capabilities. This paper introduces Retrv-R1, the first R1-style MLLM specifically designed for multimodal universal retrieval, achieving higher performance by employing step-by-step reasoning to produce more accurate retrieval results. We find that directly applying the methods of DeepSeek-R1 to retrieval tasks is not feasible, mainly due to (1) the high computational cost caused by the large token consumption required for multiple candidates with reasoning processes, and (2) the instability and suboptimal results when directly applying RL to train for retrieval tasks. To address these issues, Retrv-R1 introduces an information compression module with a details inspection mechanism, which enhances computational efficiency by reducing the number of tokens while ensuring that critical information for challenging candidates is preserved. Furthermore, a new training paradigm is proposed, including an activation stage using a retrieval-tailored synthetic CoT dataset for more effective optimization, followed by RL with a novel curriculum reward to improve both performance and efficiency. Incorporating these novel designs, Retrv-R1 achieves SOTA performance, high efficiency, and strong generalization ability, as demonstrated by experiments across multiple benchmarks and tasks.

Method: Co-designed with spine surgeons, using clinician-in-the-loop pipeline with two-stage LLM generation (draft and revision) to curate SpineMed-450k from textbooks, guidelines, open datasets, and ~1,000 hospital cases. Includes question-answering, multi-turn consultations, and report generation tasks.

Result: Evaluation on SpineBench reveals systematic weaknesses in current LVLMs for fine-grained, level-specific reasoning. Model fine-tuned on SpineMed-450k shows consistent and significant improvements across all tasks. Clinician assessments confirm diagnostic clarity and practical utility.

Conclusion: SpineMed addresses critical gaps in spine AI by providing traceable, clinically-grounded instruction data and standardized benchmarks, enabling improved vertebral-level reasoning across imaging modalities for enhanced clinical decision-making.

Abstract: Spine disorders affect 619 million people globally and are a leading cause of disability, yet AI-assisted diagnosis remains limited by the lack of level-aware, multimodal datasets. Clinical decision-making for spine disorders requires sophisticated reasoning across X-ray, CT, and MRI at specific vertebral levels. However, progress has been constrained by the absence of traceable, clinically-grounded instruction data and standardized, spine-specific benchmarks. To address this, we introduce SpineMed, an ecosystem co-designed with practicing spine surgeons. It features SpineMed-450k, the first large-scale dataset explicitly designed for vertebral-level reasoning across imaging modalities with over 450,000 instruction instances, and SpineBench, a clinically-grounded evaluation framework. SpineMed-450k is curated from diverse sources, including textbooks, guidelines, open datasets, and ~1,000 de-identified hospital cases, using a clinician-in-the-loop pipeline with a two-stage LLM generation method (draft and revision) to ensure high-quality, traceable data for question-answering, multi-turn consultations, and report generation. SpineBench evaluates models on clinically salient axes, including level identification, pathology assessment, and surgical planning. Our comprehensive evaluation of several recently advanced large vision-language models (LVLMs) on SpineBench reveals systematic weaknesses in fine-grained, level-specific reasoning. In contrast, our model fine-tuned on SpineMed-450k demonstrates consistent and significant improvements across all tasks. Clinician assessments confirm the diagnostic clarity and practical utility of our model’s outputs.

Method: A pose-conditioned, large-margin contrastive encoder that isolates persistent identity cues in transmitted pose-expression latents while canceling transient pose and expression, followed by a simple cosine test on the disentangled embedding.

Result: The method consistently outperforms existing puppeteering defenses, operates in real-time, and shows strong generalization to out-of-distribution scenarios across multiple talking-head generation models.

Conclusion: The proposed biometric leakage defense effectively detects illicit identity swaps in real-time by analyzing biometric information inherent in pose-expression latents, providing a robust security solution for talking-head videoconferencing systems.

Abstract: AI-based talking-head videoconferencing systems reduce bandwidth by sending a compact pose-expression latent and re-synthesizing RGB at the receiver, but this latent can be puppeteered, letting an attacker hijack a victim’s likeness in real time. Because every frame is synthetic, deepfake and synthetic video detectors fail outright. To address this security problem, we exploit a key observation: the pose-expression latent inherently contains biometric information of the driving identity. Therefore, we introduce the first biometric leakage defense without ever looking at the reconstructed RGB video: a pose-conditioned, large-margin contrastive encoder that isolates persistent identity cues inside the transmitted latent while cancelling transient pose and expression. A simple cosine test on this disentangled embedding flags illicit identity swaps as the video is rendered. Our experiments on multiple talking-head generation models show that our method consistently outperforms existing puppeteering defenses, operates in real-time, and shows strong generalization to out-of-distribution scenarios.

Method: Proposes null-text-null frequency-aware diffusion model that divides denoising into early (high-level noise) and late (low-level noise) stages, disentangling mid-and-low frequency bands. Uses stable mid-frequency band as guidance for null-text denoising of low-frequency band, followed by text-guided denoising.

Result: Extensive experiments show NTN-Diff outperforms state-of-the-art diffusion models for text-guided image inpainting, achieving better preservation of unmasked regions and semantic consistency.

Conclusion: NTN-Diff successfully addresses the dual challenges in text-guided image inpainting by frequency band decomposition and staged denoising with null-text guidance, demonstrating superior performance over existing methods.

Abstract: Text-guided image inpainting aims at reconstructing the masked regions as per text prompts, where the longstanding challenges lie in the preservation for unmasked regions, while achieving the semantics consistency between unmasked and inpainted masked regions. Previous arts failed to address both of them, always with either of them to be remedied. Such facts, as we observed, stem from the entanglement of the hybrid (e.g., mid-and-low) frequency bands that encode varied image properties, which exhibit different robustness to text prompts during the denoising process. In this paper, we propose a null-text-null frequency-aware diffusion models, dubbed \textbf{NTN-Diff}, for text-guided image inpainting, by decomposing the semantics consistency across masked and unmasked regions into the consistencies as per each frequency band, while preserving the unmasked regions, to circumvent two challenges in a row. Based on the diffusion process, we further divide the denoising process into early (high-level noise) and late (low-level noise) stages, where the mid-and-low frequency bands are disentangled during the denoising process. As observed, the stable mid-frequency band is progressively denoised to be semantically aligned during text-guided denoising process, which, meanwhile, serves as the guidance to the null-text denoising process to denoise low-frequency band for the masked regions, followed by a subsequent text-guided denoising process at late stage, to achieve the semantics consistency for mid-and-low frequency bands across masked and unmasked regions, while preserve the unmasked regions. Extensive experiments validate the superiority of NTN-Diff over the state-of-the-art diffusion models to text-guided diffusion models. Our code can be accessed from https://github.com/htyjers/NTN-Diff.

Method: Multi-branch ConvNeXt architecture with three parallel branches (Global Average Pooling, Global Max Pooling, and Attention-weighted Pooling), using end-to-end pipeline with data preprocessing, augmentation, and two-phase training strategy with transfer learning.

Result: Superior performance on validation set with 2,609 CT slices: ROC-AUC 0.9937, validation accuracy 0.9757, F1-score 0.9825 for COVID-19 cases, outperforming all previously reported models on this dataset.

Conclusion: Modern multi-branch architecture with careful data handling can achieve performance comparable to or exceeding state-of-the-art models, proving the efficacy of advanced deep learning for robust medical diagnostics.

Abstract: Intelligent analysis of medical imaging plays a crucial role in assisting clinical diagnosis, especially for identifying subtle pathological features. This paper introduces a novel multi-branch ConvNeXt architecture designed specifically for the nuanced challenges of medical image analysis. While applied here to the specific problem of COVID-19 diagnosis, the methodology offers a generalizable framework for classifying a wide range of pathologies from CT scans. The proposed model incorporates a rigorous end-to-end pipeline, from meticulous data preprocessing and augmentation to a disciplined two-phase training strategy that leverages transfer learning effectively. The architecture uniquely integrates features extracted from three parallel branches: Global Average Pooling, Global Max Pooling, and a new Attention-weighted Pooling mechanism. The model was trained and validated on a combined dataset of 2,609 CT slices derived from two distinct datasets. Experimental results demonstrate a superior performance on the validation set, achieving a final ROC-AUC of 0.9937, a validation accuracy of 0.9757, and an F1-score of 0.9825 for COVID-19 cases, outperforming all previously reported models on this dataset. These findings indicate that a modern, multi-branch architecture, coupled with careful data handling, can achieve performance comparable to or exceeding contemporary state-of-the-art models, thereby proving the efficacy of advanced deep learning techniques for robust medical diagnostics.

Method: Leverages VGGT foundation model for reliable initial poses and dense point clouds; integrates hybrid Gaussian representation with dual position-shape optimization, graph-guided attribute refinement, and flow-based depth regularization.

Result: Achieves more robust performance on both forward-facing and large-scale complex datasets compared to other pose-free methods, as demonstrated through comprehensive quantitative and qualitative experiments.

Conclusion: Gesplat enables robust novel view synthesis and geometrically consistent reconstruction from unposed sparse images, overcoming key limitations of existing methods in sparse-view settings.

Abstract: Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have advanced 3D reconstruction and novel view synthesis, but remain heavily dependent on accurate camera poses and dense viewpoint coverage. These requirements limit their applicability in sparse-view settings, where pose estimation becomes unreliable and supervision is insufficient. To overcome these challenges, we introduce Gesplat, a 3DGS-based framework that enables robust novel view synthesis and geometrically consistent reconstruction from unposed sparse images. Unlike prior works that rely on COLMAP for sparse point cloud initialization, we leverage the VGGT foundation model to obtain more reliable initial poses and dense point clouds. Our approach integrates several key innovations: 1) a hybrid Gaussian representation with dual position-shape optimization enhanced by inter-view matching consistency; 2) a graph-guided attribute refinement module to enhance scene details; and 3) flow-based depth regularization that improves depth estimation accuracy for more effective supervision. Comprehensive quantitative and qualitative experiments demonstrate that our approach achieves more robust performance on both forward-facing and large-scale complex datasets compared to other pose-free methods.

Method: In-Context Forensic Chain (ICFC) with objectified rule construction, adaptive filtering for knowledge base, and multi-step progressive reasoning pipeline mirroring expert forensic workflows.

Result: Surpasses state-of-the-art training-free methods and achieves competitive/superior performance compared to weakly and fully supervised approaches across multiple benchmarks.

Conclusion: ICFC successfully leverages MLLM reasoning for interpretable IML with strong performance, bridging the gap between training-free and supervised methods.

Abstract: Advances in image tampering pose serious security threats, underscoring the need for effective image manipulation localization (IML). While supervised IML achieves strong performance, it depends on costly pixel-level annotations. Existing weakly supervised or training-free alternatives often underperform and lack interpretability. We propose the In-Context Forensic Chain (ICFC), a training-free framework that leverages multi-modal large language models (MLLMs) for interpretable IML tasks. ICFC integrates an objectified rule construction with adaptive filtering to build a reliable knowledge base and a multi-step progressive reasoning pipeline that mirrors expert forensic workflows from coarse proposals to fine-grained forensics results. This design enables systematic exploitation of MLLM reasoning for image-level classification, pixel-level localization, and text-level interpretability. Across multiple benchmarks, ICFC not only surpasses state-of-the-art training-free methods but also achieves competitive or superior performance compared to weakly and fully supervised approaches.

Method: Proposes per-pixel blur kernel estimation using blur prediction network with blur-aware sparsity constraint, dynamic Gaussian densification for incomplete regions, and novel view synthesis optimization.

Result: Outperforms state-of-the-art methods in generating photorealistic novel view synthesis from defocused and motion-blurred monocular videos.

Conclusion: The framework successfully handles both defocus and motion blur in monocular videos, achieving superior novel view synthesis quality through joint blur modeling and optimization techniques.

Abstract: This paper presents a unified framework that allows high-quality dynamic Gaussian Splatting from both defocused and motion-blurred monocular videos. Due to the significant difference between the formation processes of defocus blur and motion blur, existing methods are tailored for either one of them, lacking the ability to simultaneously deal with both of them. Although the two can be jointly modeled as blur kernel-based convolution, the inherent difficulty in estimating accurate blur kernels greatly limits the progress in this direction. In this work, we go a step further towards this direction. Particularly, we propose to estimate per-pixel reliable blur kernels using a blur prediction network that exploits blur-related scene and camera information and is subject to a blur-aware sparsity constraint. Besides, we introduce a dynamic Gaussian densification strategy to mitigate the lack of Gaussians for incomplete regions, and boost the performance of novel view synthesis by incorporating unseen view information to constrain scene optimization. Extensive experiments show that our method outperforms the state-of-the-art methods in generating photorealistic novel view synthesis from defocused and motion-blurred monocular videos. Our code is available at \href{https://github.com/hhhddddddd/dydeblur}{\textcolor{cyan}{https://github.com/hhhddddddd/dydeblur}}.

Method: Developed VisualToolBench with 1,204 challenging vision tasks across 5 domains (603 single-turn, 601 multi-turn), featuring detailed rubrics to systematically evaluate MLLMs’ visual tool-use reasoning capabilities.

Result: Current MLLMs struggle significantly, with even the strongest model (GPT-5-think) achieving only 18.68% pass rate. Models show divergent behaviors - OpenAI models benefit from image manipulations while Gemini-2.5-pro shows no improvement.

Conclusion: VisualToolBench reveals critical gaps in MLLMs’ visual intelligence and provides essential insights for advancing the ’think-with-images’ paradigm in multimodal AI systems.

Abstract: Multimodal Large Language Models (MLLMs) are increasingly applied in real-world scenarios where user-provided images are often imperfect, requiring active image manipulations such as cropping, editing, or enhancement to uncover salient visual cues. Beyond static visual perception, MLLMs must also think with images: dynamically transforming visual content and integrating it with other tools to solve complex tasks. However, this shift from treating vision as passive context to a manipulable cognitive workspace remains underexplored. Most existing benchmarks still follow a think about images paradigm, where images are regarded as static inputs. To address this gap, we introduce VisualToolBench, a visual tool-use reasoning benchmark that rigorously evaluates MLLMs’ ability to perceive, transform, and reason across complex visual-textual tasks under the think-with-images paradigm. VisualToolBench comprises 1,204 challenging, open-ended vision tasks (603 single-turn, 601 multi-turn) spanning across five diverse domains, each paired with detailed rubrics to enable systematic evaluation. Our evaluation shows that current MLLMs struggle with tasks requiring effective integration of vision and general-purpose tools. Even the strongest model (GPT-5-think) reaches only 18.68% pass rate. We further observe divergent tool-use behaviors, with OpenAI models benefiting from diverse image manipulations while Gemini-2.5-pro shows no improvement. By introducing the first benchmark centered on think with images, VisualToolBench offers critical insights for advancing visual intelligence in MLLMs.

Method: The model uses multi-scale feature extraction to capture both global and local facial features, with CNN-based feature extractors, a MoE module for adaptive feature selection, and a residual network backbone for deep feature learning.

Result: Achieved accuracies of 74.77% on AffectNet (v7), 72.55% on AffectNet (v8), 84.29% on RAF-DB, and 64.66% on FER-2013, outperforming current state-of-the-art methods.

Conclusion: The model demonstrates strong adaptability and practical applicability for end-to-end emotion recognition systems in real-world settings, with publicly available reproducible code.

Abstract: In many domains, including online education, healthcare, security, and human-computer interaction, facial emotion recognition (FER) is essential. Real-world FER is still difficult despite its significance because of some factors such as variable head positions, occlusions, illumination shifts, and demographic diversity. Engagement detection, which is essential for applications like virtual learning and customer services, is frequently challenging due to FER limitations by many current models. In this article, we propose ExpressNet-MoE, a novel hybrid deep learning model that blends both Convolution Neural Networks (CNNs) and Mixture of Experts (MoE) framework, to overcome the difficulties. Our model dynamically chooses the most pertinent expert networks, thus it aids in the generalization and providing flexibility to model across a wide variety of datasets. Our model improves on the accuracy of emotion recognition by utilizing multi-scale feature extraction to collect both global and local facial features. ExpressNet-MoE includes numerous CNN-based feature extractors, a MoE module for adaptive feature selection, and finally a residual network backbone for deep feature learning. To demonstrate efficacy of our proposed model we evaluated on several datasets, and compared with current state-of-the-art methods. Our model achieves accuracies of 74.77% on AffectNet (v7), 72.55% on AffectNet (v8), 84.29% on RAF-DB, and 64.66% on FER-2013. The results show how adaptive our model is and how it may be used to develop end-to-end emotion recognition systems in practical settings. Reproducible codes and results are made publicly accessible at https://github.com/DeeptimaanB/ExpressNet-MoE.

Method: Two key contributions: (1) generative canonicalization module that processes multiple unstructured views into standardized representation, and (2) transformer-based model trained on large-scale dataset of high-fidelity Gaussian splatting avatars from dome captures of real people.

Result: The “Capture, Canonicalize, Splat” pipeline produces static quarter-body avatars with compelling realism and robust identity preservation from unstructured photos.

Conclusion: The method successfully addresses limitations of existing approaches by combining generative canonicalization with transformer models trained on real capture data, achieving hyperrealistic and identity-preserving 3D avatars from unstructured phone images.

Abstract: We present a novel, zero-shot pipeline for creating hyperrealistic, identity-preserving 3D avatars from a few unstructured phone images. Existing methods face several challenges: single-view approaches suffer from geometric inconsistencies and hallucinations, degrading identity preservation, while models trained on synthetic data fail to capture high-frequency details like skin wrinkles and fine hair, limiting realism. Our method introduces two key contributions: (1) a generative canonicalization module that processes multiple unstructured views into a standardized, consistent representation, and (2) a transformer-based model trained on a new, large-scale dataset of high-fidelity Gaussian splatting avatars derived from dome captures of real people. This “Capture, Canonicalize, Splat” pipeline produces static quarter-body avatars with compelling realism and robust identity preservation from unstructured photos.

Method: DOS modifies three types of CLIP text embeddings before passing them to text-to-image models, based on key observations about CLIP embeddings to better separate object representations.

Result: DOS consistently improves success rate of multi-object image generation and reduces object mixing. In human evaluations, it significantly outperforms four competing methods with 26.24%-43.04% more votes across four benchmarks.

Conclusion: DOS is a practical and effective solution for improving multi-object image generation by addressing fundamental issues in object separation and representation.

Abstract: Recent progress in text-to-image (T2I) generative models has led to significant improvements in generating high-quality images aligned with text prompts. However, these models still struggle with prompts involving multiple objects, often resulting in object neglect or object mixing. Through extensive studies, we identify four problematic scenarios, Similar Shapes, Similar Textures, Dissimilar Background Biases, and Many Objects, where inter-object relationships frequently lead to such failures. Motivated by two key observations about CLIP embeddings, we propose DOS (Directional Object Separation), a method that modifies three types of CLIP text embeddings before passing them into text-to-image models. Experimental results show that DOS consistently improves the success rate of multi-object image generation and reduces object mixing. In human evaluations, DOS significantly outperforms four competing methods, receiving 26.24%-43.04% more votes across four benchmarks. These results highlight DOS as a practical and effective solution for improving multi-object image generation.

Method: Proposed ESCA framework with SGCLIP model trained using neurosymbolic pipeline on 87K+ open-domain videos, aligning auto-generated captions with scene graphs without human labels.

Result: SGCLIP achieves SOTA on scene graph generation and action localization benchmarks. ESCA with SGCLIP improves perception for embodied agents, reduces errors, and enables open-source models to surpass proprietary baselines.

Conclusion: ESCA significantly enhances embodied agent perception through spatial-temporal scene graph grounding, with SGCLIP providing effective open-domain scene graph generation without human annotation requirements.

Abstract: Multi-modal large language models (MLLMs) are making rapid progress toward general-purpose embodied agents. However, existing MLLMs do not reliably capture fine-grained links between low-level visual features and high-level textual semantics, leading to weak grounding and inaccurate perception. To overcome this challenge, we propose ESCA, a framework that contextualizes embodied agents by grounding their perception in spatial-temporal scene graphs. At its core is SGCLIP, a novel, open-domain, promptable foundation model for generating scene graphs that is based on CLIP. SGCLIP is trained on 87K+ open-domain videos using a neurosymbolic pipeline that aligns automatically generated captions with scene graphs produced by the model itself, eliminating the need for human-labeled annotations. We demonstrate that SGCLIP excels in both prompt-based inference and task-specific fine-tuning, achieving state-of-the-art results on scene graph generation and action localization benchmarks. ESCA with SGCLIP improves perception for embodied agents based on both open-source and commercial MLLMs, achieving state of-the-art performance across two embodied environments. Notably, ESCA significantly reduces agent perception errors and enables open-source models to surpass proprietary baselines. We release the source code for SGCLIP model training at https://github.com/video-fm/LASER and for the embodied agent at https://github.com/video-fm/ESCA.

Method: UKANFormer builds on UKAN architecture and incorporates a Global-Local Transformer (GL-Trans) block in the decoder to extract both global semantic structures and local boundary details from noisy supervision.

Result: Achieved coral-class IoU of 67.00% and pixel accuracy of 83.98%, outperforming conventional baselines under the same noisy labels setting. The model produces predictions that are visually and structurally more accurate than the noisy training labels.

Conclusion: Architectural design can mitigate label noise and support scalable mapping under imperfect supervision, challenging the notion that data quality directly limits model performance. UKANFormer provides a foundation for ecological monitoring where reliable labels are scarce.

Abstract: Coral reefs are vital yet fragile ecosystems that require accurate large-scale mapping for effective conservation. Although global products such as the Allen Coral Atlas provide unprecedented coverage of global coral reef distri-bution, their predictions are frequently limited in spatial precision and semantic consistency, especially in regions requiring fine-grained boundary delineation. To address these challenges, we propose UKANFormer, a novel se-mantic segmentation model designed to achieve high-precision mapping under noisy supervision derived from Allen Coral Atlas. Building upon the UKAN architecture, UKANFormer incorporates a Global-Local Transformer (GL-Trans) block in the decoder, enabling the extraction of both global semantic structures and local boundary details. In experiments, UKANFormer achieved a coral-class IoU of 67.00% and pixel accuracy of 83.98%, outperforming conventional baselines under the same noisy labels setting. Remarkably, the model produces predictions that are visually and structurally more accurate than the noisy labels used for training. These results challenge the notion that data quality directly limits model performance, showing that architectural design can mitigate label noise and sup-port scalable mapping under imperfect supervision. UKANFormer provides a foundation for ecological monitoring where reliable labels are scarce.

Method: Applied beam search to discrete, sequential visual autoregressive models, leveraging the discrete token space for early pruning and computational reuse during image generation.

Result: A 2B parameter autoregressive model with beam search outperformed a 12B parameter diffusion model across benchmarks. Systematic ablations confirmed the advantage comes from discrete token spaces enabling effective search strategies.

Conclusion: Model architecture, not just scale, is critical for inference-time optimization in visual generation. Discrete autoregressive models enable effective search strategies that continuous diffusion models cannot match.

Abstract: While inference-time scaling through search has revolutionized Large Language Models, translating these gains to image generation has proven difficult. Recent attempts to apply search strategies to continuous diffusion models show limited benefits, with simple random sampling often performing best. We demonstrate that the discrete, sequential nature of visual autoregressive models enables effective search for image generation. We show that beam search substantially improves text-to-image generation, enabling a 2B parameter autoregressive model to outperform a 12B parameter diffusion model across benchmarks. Systematic ablations show that this advantage comes from the discrete token space, which allows early pruning and computational reuse, and our verifier analysis highlights trade-offs between speed and reasoning capability. These findings suggest that model architecture, not just scale, is critical for inference-time optimization in visual generation.

Method: GOOD guides diffusion sampling trajectories using ID classifiers with dual-level guidance: image-level guidance reduces input likelihood to drive samples to low-density regions, and feature-level guidance uses k-NN distance in latent space to promote sampling in feature-sparse regions.

Result: Training with GOOD-generated samples notably enhances OOD detection performance, as demonstrated through thorough quantitative and qualitative analyses.

Conclusion: GOOD enables more controllable and diverse OOD sample generation through its dual-guidance design and unified OOD scoring, significantly improving OOD detection robustness.

Abstract: Recent advancements have explored text-to-image diffusion models for synthesizing out-of-distribution (OOD) samples, substantially enhancing the performance of OOD detection. However, existing approaches typically rely on perturbing text-conditioned embeddings, resulting in semantic instability and insufficient shift diversity, which limit generalization to realistic OOD. To address these challenges, we propose GOOD, a novel and flexible framework that directly guides diffusion sampling trajectories towards OOD regions using off-the-shelf in-distribution (ID) classifiers. GOOD incorporates dual-level guidance: (1) Image-level guidance based on the gradient of log partition to reduce input likelihood, drives samples toward low-density regions in pixel space. (2) Feature-level guidance, derived from k-NN distance in the classifier’s latent space, promotes sampling in feature-sparse regions. Hence, this dual-guidance design enables more controllable and diverse OOD sample generation. Additionally, we introduce a unified OOD score that adaptively combines image and feature discrepancies, enhancing detection robustness. We perform thorough quantitative and qualitative analyses to evaluate the effectiveness of GOOD, demonstrating that training with samples generated by GOOD can notably enhance OOD detection performance.

Method: Uses dual-stream architecture with EfficientNetV2 for spatial feature extraction from facial crops and full video frames, combined with LSTM and Transformer encoders for temporal modeling, plus targeted data augmentation for underrepresented classes.

Result: ViBED-Net with LSTM achieves 73.43% accuracy on DAiSEE dataset, outperforming existing state-of-the-art approaches.

Conclusion: Combining face-aware and scene-aware spatiotemporal cues significantly improves engagement detection, offering a scalable solution for education, UX research, and content personalization.

Abstract: Engagement detection in online learning environments is vital for improving student outcomes and personalizing instruction. We present ViBED-Net (Video-Based Engagement Detection Network), a novel deep learning framework designed to assess student engagement from video data using a dual-stream architecture. ViBED-Net captures both facial expressions and full-scene context by processing facial crops and entire video frames through EfficientNetV2 for spatial feature extraction. These features are then analyzed over time using two temporal modeling strategies: Long Short-Term Memory (LSTM) networks and Transformer encoders. Our model is evaluated on the DAiSEE dataset, a large-scale benchmark for affective state recognition in e-learning. To enhance performance on underrepresented engagement classes, we apply targeted data augmentation techniques. Among the tested variants, ViBED-Net with LSTM achieves 73.43% accuracy, outperforming existing state-of-the-art approaches. ViBED-Net demonstrates that combining face-aware and scene-aware spatiotemporal cues significantly improves engagement detection accuracy. Its modular design allows flexibility for application across education, user experience research, and content personalization. This work advances video-based affective computing by offering a scalable, high-performing solution for real-world engagement analysis. The source code for this project is available on https://github.com/prateek-gothwal/ViBED-Net .

Method: Two-stage pipeline: 1) Unsupervised segmentation using EfficientNetB0 features decomposed via SVD and clustered hierarchically, 2) Segment recognition using CLIP’s vision-language alignment with SVD projection for cross-modal alignment.

Result: Achieved state-of-the-art performance on COCO, ADE20K, and PASCAL VOC benchmarks in Hungarian mIoU, precision, recall, and F1-score.

Conclusion: The framework is effective, flexible, and generalizable for open-vocabulary segmentation and recognition tasks without training.

Abstract: This paper presents a novel training-free framework for open-vocabulary image segmentation and object recognition (OVSR), which leverages EfficientNetB0, a convolutional neural network, for unsupervised segmentation and CLIP, a vision-language model, for open-vocabulary object recognition. The proposed framework adopts a two stage pipeline: unsupervised image segmentation followed by segment-level recognition via vision-language alignment. In the first stage, pixel-wise features extracted from EfficientNetB0 are decomposed using singular value decomposition to obtain latent representations, which are then clustered using hierarchical clustering to segment semantically meaningful regions. The number of clusters is adaptively determined by the distribution of singular values. In the second stage, the segmented regions are localized and encoded into image embeddings using the Vision Transformer backbone of CLIP. Text embeddings are precomputed using CLIP’s text encoder from category-specific prompts, including a generic something else prompt to support open set recognition. The image and text embeddings are concatenated and projected into a shared latent feature space via SVD to enhance cross-modal alignment. Recognition is performed by computing the softmax over the similarities between the projected image and text embeddings. The proposed method is evaluated on standard benchmarks, including COCO, ADE20K, and PASCAL VOC, achieving state-of-the-art performance in terms of Hungarian mIoU, precision, recall, and F1-score. These results demonstrate the effectiveness, flexibility, and generalizability of the proposed framework.

Method: Uses wavelet scattering coefficients (simplified CNN models) computed by transforming modulus of wavelet coefficients with a second wavelet transform. Introduces denoising by jointly minimizing and maximizing ℓ¹ norms of different subsets of scattering coefficients.

Result: Numerical experiments show the estimator reaches minimax asymptotic bounds for cartoon images for all Lipschitz exponents α ≤ 2. The ℓ¹ norms capture different types of geometric image regularity.

Conclusion: Provides a mathematical bridge between harmonic analysis and deep convolutional network denoising, offering a theoretically grounded alternative to neural networks that achieves comparable performance for cartoon image denoising.

Abstract: A considerable amount of research in harmonic analysis has been devoted to non-linear estimators of signals contaminated by additive Gaussian noise. They are implemented by thresholding coefficients in a frame, which provide a sparse signal representation, or by minimising their $\ell^1$ norm. However, sparse estimators in frames are not sufficiently rich to adapt to complex signal regularities. For cartoon images whose edges are piecewise $\bf C^\alpha$ curves, wavelet, curvelet and Xlet frames are suboptimal if the Lipschitz exponent $\alpha \leq 2$ is an unknown parameter. Deep convolutional neural networks have recently obtained much better numerical results, which reach the minimax asymptotic bounds for all $\alpha$. Wavelet scattering coefficients have been introduced as simplified convolutional neural network models. They are computed by transforming the modulus of wavelet coefficients with a second wavelet transform. We introduce a denoising estimator by jointly minimising and maximising the $\ell^1$ norms of different subsets of scattering coefficients. We prove that these $\ell^1$ norms capture different types of geometric image regularity. Numerical experiments show that this denoising estimator reaches the minimax asymptotic bound for cartoon images for all Lipschitz exponents $\alpha \leq 2$. We state this numerical result as a mathematical conjecture. It provides a different harmonic analysis approach to suppress noise from signals, and to specify the geometric regularity of functions. It also opens a mathematical bridge between harmonic analysis and denoising estimators with deep convolutional network.

Method: Uses latent-variable extension of Denoising Diffusion Bridge Models with shared latent space, contrastive alignment loss for semantic consistency, domain-agnostic encoder-decoder architecture, and predictive loss for cross-domain translation.

Result: Performs strongly on diverse MT tasks including multi-view to 3D shape generation, image super-resolution, and multi-view scene synthesis, establishing a new strong baseline.

Conclusion: LDDBM provides an effective general framework for modality translation that supports arbitrary modality pairs and overcomes limitations of existing approaches.

Abstract: Recent advances in generative modeling have positioned diffusion models as state-of-the-art tools for sampling from complex data distributions. While these models have shown remarkable success across single-modality domains such as images and audio, extending their capabilities to Modality Translation (MT), translating information across different sensory modalities, remains an open challenge. Existing approaches often rely on restrictive assumptions, including shared dimensionality, Gaussian source priors, and modality-specific architectures, which limit their generality and theoretical grounding. In this work, we propose the Latent Denoising Diffusion Bridge Model (LDDBM), a general-purpose framework for modality translation based on a latent-variable extension of Denoising Diffusion Bridge Models. By operating in a shared latent space, our method learns a bridge between arbitrary modalities without requiring aligned dimensions. We introduce a contrastive alignment loss to enforce semantic consistency between paired samples and design a domain-agnostic encoder-decoder architecture tailored for noise prediction in latent space. Additionally, we propose a predictive loss to guide training toward accurate cross-domain translation and explore several training strategies to improve stability. Our approach supports arbitrary modality pairs and performs strongly on diverse MT tasks, including multi-view to 3D shape generation, image super-resolution, and multi-view scene synthesis. Comprehensive experiments and ablations validate the effectiveness of our framework, establishing a new strong baseline in general modality translation. For more information, see our project page: https://sites.google.com/view/lddbm/home.

Method: Introduces a layered canvas where each subject is on a distinct layer for occlusion-free composition, and a locking mechanism that preserves selected layers while allowing others to adapt to context. Uses positional embeddings and complementary data sampling without architectural changes.

Result: Achieves superior spatial control and identity preservation compared to state-of-the-art methods in multi-subject personalized image generation.

Conclusion: LayerComposer provides an interactive, intuitive framework for multi-subject personalized generation with professional image-editing-like controls, addressing key limitations of existing approaches.

Abstract: Despite their impressive visual fidelity, existing personalized generative models lack interactive control over spatial composition and scale poorly to multiple subjects. To address these limitations, we present LayerComposer, an interactive framework for personalized, multi-subject text-to-image generation. Our approach introduces two main contributions: (1) a layered canvas, a novel representation in which each subject is placed on a distinct layer, enabling occlusion-free composition; and (2) a locking mechanism that preserves selected layers with high fidelity while allowing the remaining layers to adapt flexibly to the surrounding context. Similar to professional image-editing software, the proposed layered canvas allows users to place, resize, or lock input subjects through intuitive layer manipulation. Our versatile locking mechanism requires no architectural changes, relying instead on inherent positional embeddings combined with a new complementary data sampling strategy. Extensive experiments demonstrate that LayerComposer achieves superior spatial control and identity preservation compared to the state-of-the-art methods in multi-subject personalized image generation.

Method: TokenCLIP expands token-agnostic textual space into orthogonal subspaces, dynamically assigns each visual token to subspace combinations via semantic affinity using optimal transport, and applies top-k masking to specialize subspaces for distinct regions.

Result: Extensive experiments demonstrate the superiority of TokenCLIP over existing methods for anomaly detection on unseen objects.

Conclusion: TokenCLIP’s dynamic token-wise alignment framework effectively captures fine-grained anomaly semantics and achieves state-of-the-art performance in zero-shot anomaly detection.

Abstract: Adapting CLIP for anomaly detection on unseen objects has shown strong potential in a zero-shot manner. However, existing methods typically rely on a single textual space to align with visual semantics across diverse objects and domains. The indiscriminate alignment hinders the model from accurately capturing varied anomaly semantics. We propose TokenCLIP, a token-wise adaptation framework that enables dynamic alignment between visual and learnable textual spaces for fine-grained anomaly learning. Rather than mapping all visual tokens to a single, token-agnostic textual space, TokenCLIP aligns each token with a customized textual subspace that represents its visual characteristics. Explicitly assigning a unique learnable textual space to each token is computationally intractable and prone to insufficient optimization. We instead expand the token-agnostic textual space into a set of orthogonal subspaces, and then dynamically assign each token to a subspace combination guided by semantic affinity, which jointly supports customized and efficient token-wise adaptation. To this end, we formulate dynamic alignment as an optimal transport problem, where all visual tokens in an image are transported to textual subspaces based on semantic similarity. The transport constraints of OT ensure sufficient optimization across subspaces and encourage them to focus on different semantics. Solving the problem yields a transport plan that adaptively assigns each token to semantically relevant subspaces. A top-k masking is then applied to sparsify the plan and specialize subspaces for distinct visual regions. Extensive experiments demonstrate the superiority of TokenCLIP.

Method: Uses Discrete Diffusion Models with three innovations: 1) Decoupled Training and Hybrid Inference separating topology sculpting and shape refinement, 2) Improved Hourglass Architecture with bidirectional attention and Rotational Positional Embeddings, 3) Connection Loss for topological constraints.

Result: Generates high-quality 3D artist-style meshes with up to 10,000 faces at 1024^3 spatial resolution, demonstrating superior performance on complex datasets compared to existing methods.

Conclusion: TSSR successfully enables parallel generation of high-fidelity 3D meshes through its innovative architecture and training strategy, representing a significant advancement in 3D mesh generation technology.

Abstract: In this paper, we introduce Topology Sculptor, Shape Refiner (TSSR), a novel method for generating high-quality, artist-style 3D meshes based on Discrete Diffusion Models (DDMs). Our primary motivation for TSSR is to achieve highly accurate token prediction while enabling parallel generation, a significant advantage over sequential autoregressive methods. By allowing TSSR to “see” all mesh tokens concurrently, we unlock a new level of efficiency and control. We leverage this parallel generation capability through three key innovations: 1) Decoupled Training and Hybrid Inference, which distinctly separates the DDM-based generation into a topology sculpting stage and a subsequent shape refinement stage. This strategic decoupling enables TSSR to effectively capture both intricate local topology and overarching global shape. 2) An Improved Hourglass Architecture, featuring bidirectional attention enriched by face-vertex-sequence level Rotational Positional Embeddings (RoPE), thereby capturing richer contextual information across the mesh structure. 3) A novel Connection Loss, which acts as a topological constraint to further enhance the realism and fidelity of the generated meshes. Extensive experiments on complex datasets demonstrate that TSSR generates high-quality 3D artist-style meshes, capable of achieving up to 10,000 faces at a remarkable spatial resolution of $1024^3$. The code will be released at: https://github.com/psky1111/Tencent-TSSR.

Method: End-to-end development: 1) Benchmark classical ML on psychological datasets, 2) Fine-tune transformers with solutions for numerical instability and resource constraints, 3) Fine-tune LLM as interactive “Personality Brain”, 4) Deploy as microservices ecosystem.

Result: Successfully stabilized transformer-based regression for affective computing, achieved meaningful predictive performance where standard approaches failed, and developed replicable methodology for democratizing large-scale AI research.

Conclusion: Demonstrates a complete research-to-deployment pipeline integrating predictive analysis with generative dialogue, providing a practical model for future computational psychology and human-AI interaction research.

Abstract: The confluence of Artificial Intelligence and Computational Psychology presents an opportunity to model, understand, and interact with complex human psychological states through computational means. This paper presents a comprehensive, multi-faceted framework designed to bridge the gap between isolated predictive modeling and an interactive system for psychological analysis. The methodology encompasses a rigorous, end-to-end development lifecycle. First, foundational performance benchmarks were established on four diverse psychological datasets using classical machine learning techniques. Second, state-of-the-art transformer models were fine-tuned, a process that necessitated the development of effective solutions to overcome critical engineering challenges, including the resolution of numerical instability in regression tasks and the creation of a systematic workflow for conducting large-scale training under severe resource constraints. Third, a generative large language model (LLM) was fine-tuned using parameter-efficient techniques to function as an interactive “Personality Brain.” Finally, the entire suite of predictive and generative models was architected and deployed as a robust, scalable microservices ecosystem. Key findings include the successful stabilization of transformer-based regression models for affective computing, showing meaningful predictive performance where standard approaches failed, and the development of a replicable methodology for democratizing large-scale AI research. The significance of this work lies in its holistic approach, demonstrating a complete research-to-deployment pipeline that integrates predictive analysis with generative dialogue, thereby providing a practical model for future research in computational psychology and human-AI interaction.

Method: PREFINE constructs a pseudo-user agent from user interaction history and generates user-specific rubrics. This agent then critiques and refines outputs on the user’s behalf based on these tailored rubrics.

Result: On PerDOC and PerMPST datasets, PREFINE achieved higher win rates and statistically significant scores than baselines in automatic evaluations, without compromising general story quality. Both pseudo-user agent and user-specific rubrics were crucial for performance.

Conclusion: The approach enables efficient personalization without parameter updates or direct feedback, with potential applications in dialogue systems, education, and recommendation systems beyond story generation.

Abstract: While recent advances in Large Language Models (LLMs) have improved the quality of creative text generation, significant challenges remain in producing personalized stories that reflect individual user preferences. Conventional approaches rely on explicit feedback or fine-tuning, which presents practical issues regarding user burden, data collection, computational costs, and privacy. In this work, we propose PREFINE (Persona-and-Rubric Guided Critique-and-Refine), a novel framework that extends the Critique-and-Refine paradigm to personalization. PREFINE constructs a pseudo-user agent from a user’s interaction history and generates user-specific rubrics (evaluation criteria). By having this agent critique and refine outputs on the user’s behalf based on these tailored rubrics, our method achieves personalized generation without requiring parameter updates or direct user feedback. We conducted a comprehensive evaluation on the PerDOC and PerMPST story datasets. We designed three baseline methods and several model variants to verify the contribution of each component of our framework. In automatic evaluations (LLM-as-a-Judge), PREFINE achieved higher win rates and statistically significant scores than the baselines, without compromising general story quality. Analysis of the model variants confirmed that both the pseudo-user agent and the user-specific rubrics are crucial for enhancing personalization performance. Beyond story generation, our approach holds potential for enabling efficient personalization in broader applications, such as dialogue systems, education, and recommendation.

Method: Introduces SIGN - a naming game that uses lightweight schema-induced structure to steer convention formation among agents, comparing it against unconstrained natural language communication.

Result: Schema-induced communication achieves significantly faster convergence with up to 5.8x higher agreement rates compared to unconstrained natural language.

Conclusion: Minimal structure serves as an effective control mechanism for efficient multi-agent coordination, with potential applications extending beyond naming games to broader multi-agent systems.

Abstract: Real-world AI systems are tackling increasingly complex problems, often through interactions among large language model (LLM) agents. When these agents develop inconsistent conventions, coordination can break down. Applications such as collaborative coding and distributed planning therefore require reliable, consistent communication, and scalability is a central concern as systems grow. We introduce Schema-Induced Games for Naming (SIGN), a naming game that examines how lightweight structure can steer convention formation. We compare schema-induced communication to unconstrained natural language and find faster convergence with up to 5.8x higher agreement. These results suggest that minimal structure can act as a simple control knob for efficient multi-agent coordination, pointing toward broader applications beyond the naming game.

Method: Systematic evaluation of OPT and Pythia model families (70M-30B parameters) on knowledge retrieval and procedural tasks, plus attention intervention experiments.

Result: Knowledge retrieval tasks show minimal accuracy gains despite 240x scaling, while arithmetic tasks scale normally. Attention pattern swapping causes catastrophic performance collapse.

Conclusion: Parameter scaling beyond 1-2B offers minimal accuracy gains for knowledge-intensive applications in these architectures, revealing capability-specific scaling failures.

Abstract: We document empirical capability ceilings in decoder-only autoregressive language models across knowledge-intensive tasks. Systematic evaluation of OPT and Pythia model families (70M-30B parameters, spanning 240 times scaling) reveals that knowledge retrieval tasks show negligible accuracy improvement despite smooth loss reduction. On MMLU mathematics benchmarks, accuracy remains flat at 19-20% (below 25% random chance) across all scales while cross-entropy loss decreases by 31%. In contrast, procedural tasks like arithmetic show conventional scaling where both metrics improve together. Attention intervention experiments reveal high sensitivity to perturbation: swapping attention patterns between models causes catastrophic performance collapse (complete accuracy loss) rather than graceful degradation. These measurements have immediate engineering implications: for knowledge-intensive applications using OPT and Pythia architectures, parameter scaling beyond 1-2B offers minimal accuracy gains despite continued loss improvement. Our findings quantify capability-specific scaling failures in these model families to inform resource allocation decisions. Whether these patterns reflect fundamental constraints of decoder-only architectures or implementation-specific limitations remains an open question requiring investigation across diverse architectural approaches.

Method: Created GeoThoughts dataset with 6K-10K samples containing visual descriptions, step-by-step solutions, reasoning chains, and reflections. Developed GeoThought-MLLM model that generates detailed thinking processes.

Result: GeoThought-MLLM outperforms existing benchmarks in geometric tasks and shows improved reasoning capabilities in both in-domain and out-of-domain settings.

Conclusion: Training with Chain-of-Thought datasets enhances geometric reasoning, and errors can be corrected by invoking CoT to fix mathematical concept misinterpretations or spatial misjudgments.

Abstract: Large language models (LLMs) have demonstrated strong reasoning capabilities in text-based mathematical problem solving; however, when adapted to visual reasoning tasks, particularly geometric problem solving, their performance substantially declines because geometric problems present unique challenges. Specifically, these challenges stem from two key factors: first, the intrinsic complexity of geometry requiring detailed image comprehension and multi-step reasoning, and second, the limitations of existing datasets which lack sufficient scale, diversity, and explicit reasoning traces, consequently hindering effective model training. To address these challenges, we developed the GeoThoughts dataset, a comprehensive geometric reasoning corpus with two subsets: Geo-Thought-6K with 6,243 samples and its augmented version Geo-Thought-Augmented-10K containing 10,834 samples. Each entry includes visual descriptions, step-by-step solutions, explicit reasoning chains, reflection steps, and final answers. Using this dataset, we developed GeoThought-MLLM, a mathematical reasoning multimodal model that generates detailed thinking processes during problem-solving. Our model outperforms existing benchmarks in geometric tasks, demonstrating that training with our Chain-of-Thought dataset improves geometric reasoning capabilities across both in-domain and out-of-domain settings. Finally, we analyze failure cases and observe that errors primarily arise from incorrect interpretation of mathematical concepts or spatial misjudgment. By invoking CoT to correct these mistakes, the model produces correct answers.

Method: Apply GFlowNets trained with Trajectory Balance loss to sequentially build solutions for graph problems: selecting edges for spanning trees, nodes for paths, and cities for tours.

Result: GFlowNets learn to find optimal solutions that match classical algorithms (Dijkstra, Kruskal, exact TSP solvers) across benchmark instances of varying sizes, with training convergence depending on problem complexity.

Conclusion: Generative models can effectively solve combinatorial optimization problems, with the main advantage being computational scalability that could potentially handle larger instances where classical methods become infeasible.

Abstract: This work applies Generative Flow Networks (GFlowNets) to three graph optimization problems: the Traveling Salesperson Problem, Minimum Spanning Tree, and Shortest Path. GFlowNets are generative models that learn to sample solutions proportionally to a reward function. The models are trained using the Trajectory Balance loss to build solutions sequentially, selecting edges for spanning trees, nodes for paths, and cities for tours. Experiments on benchmark instances of varying sizes show that GFlowNets learn to find optimal solutions. For each problem type, multiple graph configurations with different numbers of nodes were tested. The generated solutions match those from classical algorithms (Dijkstra for shortest path, Kruskal for spanning trees, and exact solvers for TSP). Training convergence depends on problem complexity, with the number of episodes required for loss stabilization increasing as graph size grows. Once training converges, the generated solutions match known optima from classical algorithms across the tested instances. This work demonstrates that generative models can solve combinatorial optimization problems through learned policies. The main advantage of this learning-based approach is computational scalability: while classical algorithms have fixed complexity per instance, GFlowNets amortize computation through training. With sufficient computational resources, the framework could potentially scale to larger problem instances where classical exact methods become infeasible.

Method: Reduction from δ-Max-3SAT and δ-Max-3SAT(b) complexity problems to instances of GLinear-κ-RL (greedy policy) and SLinear-κ-RL (softmax policy) to establish hardness results.

Result: Learning an ε-optimal policy is NP-hard under greedy policy sets and requires exponential time (in feature dimension) for softmax policies under the Randomized Exponential Time Hypothesis.

Conclusion: Computational difficulty persists in partial qπ-realizability even when expanding the policy set beyond optimal policies, contrasting with positive results in qπ-realizability under generative models.

Abstract: This paper investigates the computational complexity of reinforcement learning in a novel linear function approximation regime, termed partial $q^{\pi}$-realizability. In this framework, the objective is to learn an $\epsilon$-optimal policy with respect to a predefined policy set $\Pi$, under the assumption that all value functions for policies in $\Pi$ are linearly realizable. The assumptions of this framework are weaker than those in $q^{\pi}$-realizability but stronger than those in $q^$-realizability, providing a practical model where function approximation naturally arises. We prove that learning an $\epsilon$-optimal policy in this setting is computationally hard. Specifically, we establish NP-hardness under a parameterized greedy policy set (argmax) and show that - unless NP = RP - an exponential lower bound (in feature vector dimension) holds when the policy set contains softmax policies, under the Randomized Exponential Time Hypothesis. Our hardness results mirror those in $q^$-realizability and suggest computational difficulty persists even when $\Pi$ is expanded beyond the optimal policy. To establish this, we reduce from two complexity problems, $\delta$-Max-3SAT and $\delta$-Max-3SAT(b), to instances of GLinear-$\kappa$-RL (greedy policy) and SLinear-$\kappa$-RL (softmax policy). Our findings indicate that positive computational results are generally unattainable in partial $q^{\pi}$-realizability, in contrast to $q^{\pi}$-realizability under a generative access model.

Method: Fine-tuned 1B-parameter Llama 3.2 using QLoRA on synthetic e-commerce dataset, then applied post-training quantization (GPTQ for GPU, GGUF for CPU) to optimize for different hardware.

Result: Specialized 1B model achieved 99% accuracy matching GPT-4.1. GPTQ reduced VRAM by 41% but slowed inference by 82% on T4 GPU, while GGUF on CPU achieved 18x speedup and 90% RAM reduction compared to FP16 baseline.

Conclusion: Small, properly optimized open-weight models are not just viable but more suitable for domain-specific applications, offering state-of-the-art accuracy at a fraction of the computational cost.

Abstract: Large Language Models (LLMs) offer state-of-the-art performance in natural language understanding and generation tasks. However, the deployment of leading commercial models for specialized tasks, such as e-commerce, is often hindered by high computational costs, latency, and operational expenses. This paper investigates the viability of smaller, open-weight models as a resource-efficient alternative. We present a methodology for optimizing a one-billion-parameter Llama 3.2 model for multilingual e-commerce intent recognition. The model was fine-tuned using Quantized Low-Rank Adaptation (QLoRA) on a synthetically generated dataset designed to mimic real-world user queries. Subsequently, we applied post-training quantization techniques, creating GPU-optimized (GPTQ) and CPU-optimized (GGUF) versions. Our results demonstrate that the specialized 1B model achieves 99% accuracy, matching the performance of the significantly larger GPT-4.1 model. A detailed performance analysis revealed critical, hardware-dependent trade-offs: while 4-bit GPTQ reduced VRAM usage by 41%, it paradoxically slowed inference by 82% on an older GPU architecture (NVIDIA T4) due to dequantization overhead. Conversely, GGUF formats on a CPU achieved a speedup of up to 18x in inference throughput and a reduction of over 90% in RAM consumption compared to the FP16 baseline. We conclude that small, properly optimized open-weight models are not just a viable but a more suitable alternative for domain-specific applications, offering state-of-the-art accuracy at a fraction of the computational cost.

Method: Distribution Shift Alignment (DSA) - a two-stage fine-tuning method that learns how output distributions change across different backgrounds rather than fitting training data distributions.

Result: DSA consistently outperforms other methods on five public survey datasets, reduces required real data by 53.48-69.12%, and provides results substantially closer to the true distribution than training data.

Conclusion: DSA demonstrates effectiveness and efficiency in survey simulation by learning distribution shifts rather than fitting training data, enabling more accurate LLM-based survey response generation.

Abstract: Large language models (LLMs) offer a promising way to simulate human survey responses, potentially reducing the cost of large-scale data collection. However, existing zero-shot methods suffer from prompt sensitivity and low accuracy, while conventional fine-tuning approaches mostly fit the training set distributions and struggle to produce results more accurate than the training set itself, which deviates from the original goal of using LLMs to simulate survey responses. Building on this observation, we introduce Distribution Shift Alignment (DSA), a two-stage fine-tuning method that aligns both the output distributions and the distribution shifts across different backgrounds. By learning how these distributions change rather than fitting training data, DSA can provide results substantially closer to the true distribution than the training data. Empirically, DSA consistently outperforms other methods on five public survey datasets. We further conduct a comprehensive comparison covering accuracy, robustness, and data savings. DSA reduces the required real data by 53.48-69.12%, demonstrating its effectiveness and efficiency in survey simulation.

Method: The authors summarize 5 key cognitive abilities for MWP solving, review two mainstream models (neural network solvers and LLM-based solvers), rerun representative solvers, and evaluate them on 5 mainstream benchmarks for unified comparison.

Result: The survey provides the first comprehensive analysis of influential MWP research from the perspective of human reasoning cognition and offers an integrative comparison across existing approaches.

Conclusion: This work systematically categorizes MWP research through human cognitive abilities, demonstrates AI’s progress in simulating human reasoning, and aims to inspire further research in AI reasoning. The repository is publicly available.

Abstract: Math word problem (MWP) serves as a fundamental research topic in artificial intelligence (AI) dating back to 1960s. This research aims to advance the reasoning abilities of AI by mirroring the human-like cognitive intelligence. The mainstream technological paradigm has evolved from the early rule-based methods, to deep learning models, and is rapidly advancing towards large language models. However, the field still lacks a systematic taxonomy for the MWP survey along with a discussion of current development trends. Therefore, in this paper, we aim to comprehensively review related research in MWP solving through the lens of human cognition, to demonstrate how recent AI models are advancing in simulating human cognitive abilities. Specifically, we summarize 5 crucial cognitive abilities for MWP solving, including Problem Understanding, Logical Organization, Associative Memory, Critical Thinking, and Knowledge Learning. Focused on these abilities, we review two mainstream MWP models in recent 10 years: neural network solvers, and LLM based solvers, and discuss the core human-like abilities they demonstrated in their intricate problem-solving process. Moreover, we rerun all the representative MWP solvers and supplement their performance on 5 mainstream benchmarks for a unified comparison. To the best of our knowledge, this survey first comprehensively analyzes the influential MWP research of the past decade from the perspective of human reasoning cognition and provides an integrative overall comparison across existing approaches. We hope it can inspire further research in AI reasoning. Our repository is released on https://github.com/Ljyustc/FoI-MWP.

Method: LightAgent enhances Qwen2.5-VL-3B via two-stage SFT->GRPO training on synthetic GUI data, integrates an efficient long-reasoning mechanism for historical interactions, and defaults to on-device execution while escalating challenging subtasks to the cloud via real-time complexity assessment.

Result: Experiments on AndroidLab benchmark and diverse apps show LightAgent matches or nears larger models’ performance while significantly reducing cloud costs.

Conclusion: LightAgent successfully resolves the mobile GUI agent dilemma through device-cloud collaboration, achieving strong performance with reduced cloud dependency.

Abstract: With the advancement of multimodal large language models (MLLMs), building GUI agent systems has become an increasingly promising direction-especially for mobile platforms, given their rich app ecosystems and intuitive touch interactions. Yet mobile GUI agents face a critical dilemma: truly on-device models (4B or smaller) lack sufficient performance, while capable models (starting from 7B) are either too large for mobile deployment or prohibitively costly (e.g., cloud-only closed-source MLLMs). To resolve this, we propose LightAgent, a mobile agentic foundation model solution that leverages device-cloud collaboration to tap the cost-efficiency of on-device models and the high capability of cloud models, while avoiding their drawbacks. Specifically, LightAgent enhances Qwen2.5-VL-3B via two-stage SFT->GRPO training on synthetic GUI data for strong decision-making, integrates an efficient long-reasoning mechanism to utilize historical interactions under tight resources, and defaults to on-device execution-only escalating challenging subtasks to the cloud via real-time complexity assessment. Experiments on the online AndroidLab benchmark and diverse apps show LightAgent matches or nears larger models, with a significant reduction in cloud costs.

Method: LLM-AR pipeline inspired by neural-symbolic systems that distills LLM-generated heuristics into probabilistic rules executed by ProbLog, with iterative policy-evolution loop using association-rule mining to refine rules.

Result: Achieved 59.5% precision and 8.7% recall on unseen folds, 5.9x random baseline precision, while maintaining interpretability with exposed decision paths.

Conclusion: The framework is interpretable, tunable via hyperparameters, and shows promise for extension into other domains beyond venture capital.

Abstract: Large language models (LLMs) can already identify patterns and reason effectively, yet their variable accuracy hampers adoption in high-stakes decision-making applications. In this paper, we study this issue from a venture capital perspective by predicting idea-stage startup success based on founder traits. (i) To build a reliable prediction model, we introduce LLM-AR, a pipeline inspired by neural-symbolic systems that distils LLM-generated heuristics into probabilistic rules executed by the ProbLog automated-reasoning engine. (ii) An iterative policy-evolution loop incorporates association-rule mining to progressively refine the prediction rules. On unseen folds, LLM-AR achieves 59.5% precision and 8.7% recall, 5.9x the random baseline precision, while exposing every decision path for human inspection. The framework is interpretable and tunable via hyperparameters, showing promise to extend into other domains.

Method: Integrate self-supervised predictive coding modules into meta-RL, inspired by predictive coding in neuroscience and auxiliary predictive objectives in deep RL.

Result: Meta-RL with predictive modules consistently generates more interpretable representations that better approximate Bayes-optimal belief states across various tasks, and succeeds in challenging active information seeking tasks where conventional meta-RL fails.

Conclusion: Predictive learning serves as a guiding principle for effective representation learning in agents navigating partial observability, leading to improved generalization.

Abstract: Learning a compact representation of history is critical for planning and generalization in partially observable environments. While meta-reinforcement learning (RL) agents can attain near Bayes-optimal policies, they often fail to learn the compact, interpretable Bayes-optimal belief states. This representational inefficiency potentially limits the agent’s adaptability and generalization capacity. Inspired by predictive coding in neuroscience–which suggests that the brain predicts sensory inputs as a neural implementation of Bayesian inference–and by auxiliary predictive objectives in deep RL, we investigate whether integrating self-supervised predictive coding modules into meta-RL can facilitate learning of Bayes-optimal representations. Through state machine simulation, we show that meta-RL with predictive modules consistently generates more interpretable representations that better approximate Bayes-optimal belief states compared to conventional meta-RL across a wide variety of tasks, even when both achieve optimal policies. In challenging tasks requiring active information seeking, only meta-RL with predictive modules successfully learns optimal representations and policies, whereas conventional meta-RL struggles with inadequate representation learning. Finally, we demonstrate that better representation learning leads to improved generalization. Our results strongly suggest the role of predictive learning as a guiding principle for effective representation learning in agents navigating partial observability.

Method: Used SpecC methodology within system-level HW/SW co-design flow to model and explore PCM-to-PWM converter, employing system-level estimates and fast functional simulation to evaluate mappings.

Result: Successfully derived HW/SW partitions that meet real-time constraints while reducing estimated cost compared to all-hardware solution and avoiding expense of purely software implementation on high-end processor.

Conclusion: System-level co-design provides valuable early architectural insight, rapid validation, and actionable cost/performance trade-offs, even for moderately complex designs.

Abstract: We present a case study applying the SpecC methodology within a system-level hardware/software co-design flow to a PCM-to-PWM converter, the core of a Class-D audio amplifier. The converter was modeled and explored with SpecC methodology to derive an HW/SW partition. Using system-level estimates and fast functional simulation, we evaluated mappings that meet real-time constraints while reducing estimated cost of an all-hardware solution and avoiding the expense of a purely software implementation on a high-end processor. Despite the design’s moderate complexity, the results underline the value of system-level co-design for early architectural insight, rapid validation, and actionable cost/performance trade-offs. [Original work from 2005; formatting revised in 2025, with no changes to the results.]

Method: Introduces Energy Structured Causal Models (ESCMs) where mechanisms are expressed as constraints (energy functions or vector fields) rather than explicit input-output maps, enabling local surgery on constraints.

Result: ESCMs provide a formal language for causal reasoning that makes internal structure manipulable, recovers standard SCM semantics under mild conditions, and addresses gauge ambiguity in encoder energy pairs.

Conclusion: The paper offers a framework for building systems that understand rather than merely predict, positioning intelligence as the ability to build and refine falsifiable causal explanations.

Abstract: Contemporary machine learning optimizes for predictive accuracy, yet systems that achieve state of the art performance remain causally opaque: their internal representations provide no principled handle for intervention. We can retrain such models, but we cannot surgically edit specific mechanisms while holding others fixed, because learned latent variables lack causal semantics. We argue for a conceptual reorientation: intelligence is the ability to build and refine explanations, falsifiable claims about manipulable structure that specify what changes and what remains invariant under intervention. Explanations subsume prediction but demand more: causal commitments that can be independently tested and corrected at the level of mechanisms. We introduce computational explanations, mappings from observations to intervention ready causal accounts. We instantiate these explanations with Energy Structured Causal Models (ESCMs), in which mechanisms are expressed as constraints (energy functions or vector fields) rather than explicit input output maps, and interventions act by local surgery on those constraints. This shift makes internal structure manipulable at the level where explanations live: which relations must hold, which can change, and what follows when they do. We provide concrete instantiations of the structural-causal principles LAP and ICM in the ESCM context, and also argue that empirical risk minimization systematically produces fractured, entangled representations, a failure we analyze as gauge ambiguity in encoder energy pairs. Finally, we show that under mild conditions, ESCMs recover standard SCM semantics. Building on Part I’s principles (LAP, ICM, CAP) and its definition of intelligence as explanation-building under criticism, this paper offers a formal language for causal reasoning in systems that aspire to understand, not merely to predict.

Method: The paper develops a vision for future AI systems that transition from today’s large models to nimble, energy-efficient domain-specific agents that can reason, plan, make decisions in dynamic environments with real-time data and prior knowledge, while learning continuously.

Result: The proposed vision requires hardware reimagined to achieve energy efficiencies greater than 1000x over current state-of-the-art systems.

Conclusion: The next wave of AI should progress from today’s data-hungry large models to lightweight, domain-specific agents capable of reasoning and thinking in uncertain environments with dramatically improved energy efficiency.

Abstract: The field of artificial intelligence (AI) has taken a tight hold on broad aspects of society, industry, business, and governance in ways that dictate the prosperity and might of the world’s economies. The AI market size is projected to grow from 189 billion USD in 2023 to 4.8 trillion USD by 2033. Currently, AI is dominated by large language models that exhibit linguistic and visual intelligence. However, training these models requires a massive amount of data scraped from the web as well as large amounts of energy (50–60 GWh to train GPT-4). Despite these costs, these models often hallucinate, a characteristic that prevents them from being deployed in critical application domains. In contrast, the human brain consumes only 20~W of power. What is needed is the next level of AI evolution in which lightweight domain-specific multimodal models with higher levels of intelligence can reason, plan, and make decisions in dynamic environments with real-time data and prior knowledge, while learning continuously and evolving in ways that enhance future decision-making capability. This will define the next wave of AI, progressing from today’s large models, trained with vast amounts of data, to nimble energy-efficient domain-specific agents that can reason and think in a world full of uncertainty. To support such agents, hardware will need to be reimagined to allow energy efficiencies greater than 1000x over the state of the art. Such a vision of future AI systems is developed in this work.

Method: Multi-Agent Conditional Diffusion Model Planner (MA-CDMP) with diffusion models for environment dynamics, inverse dynamics model for action generation, and mean-field mechanism for agent interaction approximation.

Result: Outperforms existing MARL baselines in average reward and QoS metrics, demonstrating scalability and practicality for real-world wireless networks.

Conclusion: MA-CDMP provides an effective solution for decentralized communication resource management with theoretical convergence guarantees and minimal communication overhead.

Abstract: In wireless communication systems, efficient and adaptive resource allocation plays a crucial role in enhancing overall Quality of Service (QoS). While centralized Multi-Agent Reinforcement Learning (MARL) frameworks rely on a central coordinator for policy training and resource scheduling, they suffer from scalability issues and privacy risks. In contrast, the Distributed Training with Decentralized Execution (DTDE) paradigm enables distributed learning and decision-making, but it struggles with non-stationarity and limited inter-agent cooperation, which can severely degrade system performance. To overcome these challenges, we propose the Multi-Agent Conditional Diffusion Model Planner (MA-CDMP) for decentralized communication resource management. Built upon the Model-Based Reinforcement Learning (MBRL) paradigm, MA-CDMP employs Diffusion Models (DMs) to capture environment dynamics and plan future trajectories, while an inverse dynamics model guides action generation, thereby alleviating the sample inefficiency and slow convergence of conventional DTDE methods. Moreover, to approximate large-scale agent interactions, a Mean-Field (MF) mechanism is introduced as an assistance to the classifier in DMs. This design mitigates inter-agent non-stationarity and enhances cooperation with minimal communication overhead in distributed settings. We further theoretically establish an upper bound on the distributional approximation error introduced by the MF-based diffusion generation, guaranteeing convergence stability and reliable modeling of multi-agent stochastic dynamics. Extensive experiments demonstrate that MA-CDMP consistently outperforms existing MARL baselines in terms of average reward and QoS metrics, showcasing its scalability and practicality for real-world wireless network optimization.

Method: This position paper argues for a paradigm shift from BCI to BAC, emphasizing the need to reframe agents as collaborative partners and focusing on ethical data handling, model reliability, and human-agent collaboration frameworks.

Result: The paper establishes the conceptual foundation for Brain-Agent Collaboration as an emerging direction that could overcome current BCI limitations through intelligent assistance systems.

Conclusion: The field should transition from BCI to BAC, developing safe, trustworthy, and effective systems where agents serve as active collaborative partners, requiring attention to ethical considerations and robust collaboration frameworks.

Abstract: Brain-Computer Interfaces (BCIs) offer a direct communication pathway between the human brain and external devices, holding significant promise for individuals with severe neurological impairments. However, their widespread adoption is hindered by critical limitations, such as low information transfer rates and extensive user-specific calibration. To overcome these challenges, recent research has explored the integration of Large Language Models (LLMs), extending the focus from simple command decoding to understanding complex cognitive states. Despite these advancements, deploying agentic AI faces technical hurdles and ethical concerns. Due to the lack of comprehensive discussion on this emerging direction, this position paper argues that the field is poised for a paradigm extension from BCI to Brain-Agent Collaboration (BAC). We emphasize reframing agents as active and collaborative partners for intelligent assistance rather than passive brain signal data processors, demanding a focus on ethical data handling, model reliability, and a robust human-agent collaboration framework to ensure these systems are safe, trustworthy, and effective.

Method: Leverages learnable thought vectors that dynamically modulate LLM reasoning processes, using entropy-based rewards to guide focused reasoning patterns.

Result: Achieved 90.1% accuracy on GSM8K with a controllability score of 0.42, with distinct thought vector clusters and consistent low-entropy distributions across control conditions.

Conclusion: The framework successfully enables controllable AI reasoning through entropy-minimized thought vectors, validating the approach for focused reasoning patterns.

Abstract: We present a novel approach for controllable mathematical reasoning that leverages self-optimizing thought vectors with entropy minimization. Our method introduces learnable thought vectors that dynamically modulate the internal reasoning process of large language models. Using Gemma-2-9B on GSM8K, we achieve 90.1% accuracy with a controllability score of 0.42, demonstrating that entropy-based rewards effectively guide focused reasoning patterns without requiring external reward annotations. Our analysis reveals distinct thought vector clusters and consistent low-entropy distributions across control conditions, validating our framework for controllable AI reasoning.

Method: Uses Vision-Language Models to score regions based on time-series Sentinel-2 images processed to minimize cloud cover, with reference images and temporal consistency strategies.

Result: AUI outperforms standard indices like NDBI in qualitative experiments on Bangalore, providing more reliable and stable development scores.

Conclusion: AUI successfully addresses challenges of traditional urbanization indices through VLM-based approach with temporal consistency and cloud mitigation strategies.

Abstract: We introduce the {\em Atlas Urban Index} (AUI), a metric for measuring urban development computed using Sentinel-2 \citep{spoto2012sentinel2} satellite imagery. Existing approaches, such as the {\em Normalized Difference Built-up Index} (NDBI), often struggle to accurately capture urban development due to factors like atmospheric noise, seasonal variation, and cloud cover. These limitations hinder large-scale monitoring of human development and urbanization. To address these challenges, we propose an approach that leverages {\em Vision-Language Models }(VLMs) to provide a development score for regions. Specifically, we collect a time series of Sentinel-2 images for each region. Then, we further process the images within fixed time windows to get an image with minimal cloud cover, which serves as the representative image for that time window. To ensure consistent scoring, we adopt two strategies: (i) providing the VLM with a curated set of reference images representing different levels of urbanization, and (ii) supplying the most recent past image to both anchor temporal consistency and mitigate cloud-related noise in the current image. Together, these components enable AUI to overcome the challenges of traditional urbanization indices and produce more reliable and stable development scores. Our qualitative experiments on Bangalore suggest that AUI outperforms standard indices such as NDBI.

Method: Uses situational judgment tests from realistic scenarios, integrates industrial-organizational and personality psychology to design sophisticated personas with behavioral descriptors, life history, and social functions, employs structured generation with demographic priors and memoir narratives encoded with Pydantic schemas.

Result: Created a rich dataset for law enforcement assistant case study spanning 8,500 personas across 8 archetypes, 4,000 SJTs across 11 attributes, and 300,000 responses, with analysis across subpopulations and scenarios.

Conclusion: The framework enables more realistic and domain-relevant AI psychometrics evaluation, and the dataset and code will be publicly released.

Abstract: AI psychometrics evaluates AI systems in roles that traditionally require emotional judgment and ethical consideration. Prior work often reuses human trait inventories (Big Five, \hexaco) or ad hoc personas, limiting behavioral realism and domain relevance. We propose a framework that (1) uses situational judgment tests (SJTs) from realistic scenarios to probe domain-specific competencies; (2) integrates industrial-organizational and personality psychology to design sophisticated personas which include behavioral and psychological descriptors, life history, and social and emotional functions; and (3) employs structured generation with population demographic priors and memoir inspired narratives, encoded with Pydantic schemas. In a law enforcement assistant case study, we construct a rich dataset of personas drawn across 8 persona archetypes and SJTs across 11 attributes, and analyze behaviors across subpopulation and scenario slices. The dataset spans 8,500 personas, 4,000 SJTs, and 300,000 responses. We will release the dataset and all code to the public.

Method: Uses Large Language Models (LLMs) with Hypergraph of Thoughts (HGoT) extension, resilient execution strategies, and advanced query analysis to automate pipeline design and deployment across distributed stream processing systems.

Result: Significantly reduces development time (6.3x faster) and error rates (5.19x improvement in Error-Free Score) compared to LLM code-generation methods across diverse pipelines.

Conclusion: AutoStreamPipe successfully automates stream processing pipeline development with substantial improvements in efficiency and accuracy, demonstrating the effectiveness of HGoT-enhanced LLMs for multi-agent reasoning in distributed systems.

Abstract: Data pipelines are essential in stream processing as they enable the efficient collection, processing, and delivery of real-time data, supporting rapid data analysis. In this paper, we present AutoStreamPipe, a novel framework that employs Large Language Models (LLMs) to automate the design, generation, and deployment of stream processing pipelines. AutoStreamPipe bridges the semantic gap between high-level user intent and platform-specific implementations across distributed stream processing systems for structured multi-agent reasoning by integrating a Hypergraph of Thoughts (HGoT) as an extended version of GoT. AutoStreamPipe combines resilient execution strategies, advanced query analysis, and HGoT to deliver pipelines with good accuracy. Experimental evaluations on diverse pipelines demonstrate that AutoStreamPipe significantly reduces development time (x6.3) and error rates (x5.19), as measured by a novel Error-Free Score (EFS), compared to LLM code-generation methods.

Method: Uses weight perturbation learning with stochastic weight updates that minimize regret (reward prediction error) between perturbed and unperturbed model outcomes, creating an adaptive learning rate strategy similar to dopamine’s role in the brain.

Result: Dopamine-trained models show accelerated convergence, outperform standard weight perturbation, achieve comparable performance to gradient-based algorithms, while consuming significantly less computation and memory.

Conclusion: The Dopamine optimizer provides robust solutions with performance comparable to state-of-the-art ML optimizers while being more neurobiologically plausible and computationally efficient.

Abstract: Solving the synaptic Credit Assignment Problem(CAP) is central to learning in both biological and artificial neural systems. Finding an optimal solution for synaptic CAP means setting the synaptic weights that assign credit to each neuron for influencing the final output and behavior of neural networks or animals. Gradient-based methods solve this problem in artificial neural networks using back-propagation, however, not in the most efficient way. For instance, back-propagation requires a chain of top-down gradient computations. This leads to an expensive optimization process in terms of computing power and memory linked with well-known weight transport and update locking problems. To address these shortcomings, we take a NeuroAI approach and draw inspiration from neural Reinforcement Learning to develop a derivative-free optimizer for training neural networks, Dopamine. Dopamine is developed for Weight Perturbation (WP) learning that exploits stochastic updating of weights towards optima. It achieves this by minimizing the regret, a form of Reward Prediction Error (RPE) between the expected outcome from the perturbed model and the actual outcome from the unperturbed model. We use this RPE to adjust the learning rate in the network (i.e., creating an adaptive learning rate strategy, similar to the role of dopamine in the brain). We tested the Dopamine optimizer for training multi-layered perceptrons for XOR tasks, and recurrent neural networks for chaotic time series forecasting. Dopamine-trained models demonstrate accelerated convergence and outperform standard WP, and give comparable performance to gradient-based algorithms, while consuming significantly less computation and memory. Overall, the Dopamine optimizer not only finds robust solutions and comparable performance to the state-of-the-art Machine Learning optimizers but is also neurobiologically more plausible.

Method: Development of intelligent systems capable of navigating the intricate cyber-legal domain.

Result: Promising initial results on multilingual tasks.

Conclusion: This work provides a first step towards bridging the knowledge divide in the cyber-legal domain.

Abstract: The growing intersection of cybersecurity and law creates a complex information space where traditional legal research tools struggle to deal with nuanced connections between cases, statutes, and technical vulnerabilities. This knowledge divide hinders collaboration between legal experts and cybersecurity professionals. To address this important gap, this work provides a first step towards intelligent systems capable of navigating the increasingly intricate cyber-legal domain. We demonstrate promising initial results on multilingual tasks.

Method: Develops a modeling tree organizing OR problems hierarchically by taxonomy and complexity, then recurrently searches the tree to decompose problems into simpler subproblems and synthesize global modeling thoughts.

Result: OptiTree significantly improves modeling accuracy compared to state-of-the-art methods, achieving over 10% improvements on challenging benchmarks.

Conclusion: The tree search approach with adaptive problem decomposition effectively enhances modeling capabilities for complex OR problems, demonstrating substantial performance gains.

Abstract: Optimization modeling is one of the most crucial but technical parts of operations research (OR). To automate the modeling process, existing works have leveraged large language models (LLMs), prompting them to break down tasks into steps for generating variables, constraints, and objectives. However, due to the highly complex mathematical structures inherent in OR problems, standard fixed-step decomposition often fails to achieve high performance. To address this challenge, we introduce OptiTree, a novel tree search approach designed to enhance modeling capabilities for complex problems through adaptive problem decomposition into simpler subproblems. Specifically, we develop a modeling tree that organizes a wide range of OR problems based on their hierarchical problem taxonomy and complexity, with each node representing a problem category and containing relevant high-level modeling thoughts. Given a problem to model, we recurrently search the tree to identify a series of simpler subproblems and synthesize the global modeling thoughts by adaptively integrating the hierarchical thoughts. Experiments show that OptiTree significantly improves the modeling accuracy compared to the state-of-the-art, achieving over 10% improvements on the challenging benchmarks. The code is released at https://github.com/MIRALab-USTC/OptiTree/tree/main.

Method: Progressively Ascending Confidence Reward (PACR) - a dense reward computed from the model’s evolving probability of the ground-truth answer, encoding the inductive bias that correct reasoning should show an ascending trend in confidence.

Result: PACR accelerates exploration, reaches reward saturation with fewer trajectories, and improves performance on multiple benchmarks compared to standard RLVR.

Conclusion: Dense, model-intrinsic shaping signals like PACR can make RLVR training more effective and reliable by providing better guidance during the reasoning process.

Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has significantly improved LLM reasoning, but its sparse, outcome-based reward provides no guidance for intermediate steps, slowing exploration. We propose Progressively Ascending Confidence Reward (PACR), a dense, model-intrinsic reward computed directly from the model’s evolving belief in the correct answer. PACR encodes the inductive bias that, along a well-formed reasoning trajectory, the probability of the ground-truth answer should have a generally ascending trend. We provide empirical and theoretical analysis validating that such an inductive bias constrains the exploration search space to regions richer in logically sound reasoning. We demonstrate that PACR accelerates exploration, reaches reward saturation with fewer trajectories, and yields improvements on multiple benchmarks. Our results suggest that dense, model-intrinsic shaping signals can make RLVR training more effective and reliable.

Method: Curated VietLyrics dataset with 647 hours of songs, fine-tuned Whisper models on this dataset to address transcription limitations.

Result: Fine-tuned Whisper models outperformed existing multilingual ALT systems including LyricWhiz, reducing transcription errors and hallucinations.

Conclusion: Demonstrates potential of dataset creation and model fine-tuning for ALT in low-resource languages like Vietnamese, publicly releasing resources to advance research.

Abstract: Automatic Lyrics Transcription (ALT) for Vietnamese music presents unique challenges due to its tonal complexity and dialectal variations, but remains largely unexplored due to the lack of a dedicated dataset. Therefore, we curated the first large-scale Vietnamese ALT dataset (VietLyrics), comprising 647 hours of songs with line-level aligned lyrics and metadata to address these issues. Our evaluation of current ASRbased approaches reveal significant limitations, including frequent transcription errors and hallucinations in non-vocal segments. To improve performance, we fine-tuned Whisper models on the VietLyrics dataset, achieving superior results compared to existing multilingual ALT systems, including LyricWhiz. We publicly release VietLyrics and our models, aiming to advance Vietnamese music computing research while demonstrating the potential of this approach for ALT in low-resource language and music.

Method: Multilevel graph coarsening and refinement with spatio-temporal customer aggregation, classical heuristics on reduced problems, and feasibility corrections during expansion.

Result: Preliminary experiments on Solomon benchmarks show reduced computation time while maintaining or improving solution quality, especially for capacity and time window constraints.

Conclusion: The framework effectively accelerates CVRPTW solving and quantum-inspired techniques show promise for further speedup in large-scale vehicle routing.

Abstract: The Capacitated Vehicle Routing Problem with Time Windows (CVRPTW) is a fundamental NP-hard optimization problem in logistics. Solving large-scale instances remains computationally challenging for exact solvers. This work introduces a multilevel graph coarsening and refinement framework that aggregates customers into meta-nodes using a spatio-temporal distance metric. The reduced problem is solved with classical heuristics and subsequently expanded back into the original space with feasibility corrections. Preliminary experiments on Solomon benchmark instances show that the proposed method reduces computation time while preserving or improving solution quality, particularly with respect to capacity and time window constraints. The paper also explores the integration of quantum-inspired optimization techniques, highlighting their potential to further accelerate large-scale vehicle routing tasks.

Method: Uses literature-informed fine-tuned LLMs with three components: LLM-driven Inference Core, Literature Processing Pipeline, and Result Evaluator. Fine-tuned on real-world truck driving data and literature from 299 domain papers.

Result: Achieved 26.7% higher recall and 10.1% higher F1-score than benchmarks. Produced consistent variable importance rankings and identified risky scenarios verified by PERMANOVA tests.

Conclusion: LIFT LLM framework successfully integrates literature knowledge for interpretable risk prediction and shows potential for data-driven knowledge discovery in transportation safety.

Abstract: This study proposes an interpretable prediction framework with literature-informed fine-tuned (LIFT) LLMs for truck driving risk prediction. The framework integrates an LLM-driven Inference Core that predicts and explains truck driving risk, a Literature Processing Pipeline that filters and summarizes domain-specific literature into a literature knowledge base, and a Result Evaluator that evaluates the prediction performance as well as the interpretability of the LIFT LLM. After fine-tuning on a real-world truck driving risk dataset, the LIFT LLM achieved accurate risk prediction, outperforming benchmark models by 26.7% in recall and 10.1% in F1-score. Furthermore, guided by the literature knowledge base automatically constructed from 299 domain papers, the LIFT LLM produced variable importance ranking consistent with that derived from the benchmark model, while demonstrating robustness in interpretation results to various data sampling conditions. The LIFT LLM also identified potential risky scenarios by detecting key combination of variables in truck driving risk, which were verified by PERMANOVA tests. Finally, we demonstrated the contribution of the literature knowledge base and the fine-tuning process in the interpretability of the LIFT LLM, and discussed the potential of the LIFT LLM in data-driven knowledge discovery.

Method: Created through semi-automatic annotation pipeline with 503 expert-curated seed questions that can dynamically generate unlimited multimodal instances. Includes process evaluation based on expert-annotated reasoning chains to measure logical validity and causal coherence.

Result: Experiments show large performance gaps across VLMs, severe degradation in dynamic settings, and poor performance on high-level spatial intelligence tasks like mental rotation and visualization.

Conclusion: DynaSolidGeo addresses critical gaps in evaluating genuine spatial reasoning and reveals significant limitations in current VLMs’ spatial mathematical reasoning capabilities.

Abstract: Solid geometry problem solving demands spatial mathematical reasoning that integrates spatial intelligence and symbolic reasoning. However, most existing multimodal mathematical reasoning benchmarks focus primarily on 2D plane geometry, rely on static datasets prone to data contamination and memorization, and evaluate models solely by final answers, overlooking the reasoning process. To address these limitations, we introduce DynaSolidGeo, the first dynamic benchmark for evaluating genuine spatial reasoning in Vision-Language Models (VLMs). Constructed through a semi-automatic annotation pipeline, DynaSolidGeo contains 503 expert-curated seed questions that can, in principle, dynamically generate an unbounded number of diverse multimodal text-visual instances. Beyond answer accuracy, we incorporate process evaluation based on expert-annotated reasoning chains to measure logical validity and causal coherence. Experiments across representative open-source and closed-source VLMs reveal large performance gaps, severe degradation in dynamic settings, and poor performance on tasks requiring high-level spatial intelligence, such as mental rotation and visualization. The code and dataset are available at \href{https://zgca-ai4edu.github.io/DynaSolidGeo/}{DynaSolidGeo}.

Method: Created five collaborative AI agents with varying adaptation strategies to human actions, had participants interact with them, evaluated perceived traits, and used Bayesian modeling to analyze team performance and preference factors.

Result: Considerate agents that adapt to human actions are preferred over performance-maximizing ones, with human-centric design improving AI likability without reducing performance, driven by inequality-aversion effects.

Conclusion: AI collaboration benefits from development that includes both subjective and objective metrics, as people prefer agents that allow meaningful human contribution to the team.

Abstract: Despite the growing interest in collaborative AI, designing systems that seamlessly integrate human input remains a major challenge. In this study, we developed a task to systematically examine human preferences for collaborative agents. We created and evaluated five collaborative AI agents with strategies that differ in the manner and degree they adapt to human actions. Participants interacted with a subset of these agents, evaluated their perceived traits, and selected their preferred agent. We used a Bayesian model to understand how agents’ strategies influence the Human-AI team performance, AI’s perceived traits, and the factors shaping human-preferences in pairwise agent comparisons. Our results show that agents who are more considerate of human actions are preferred over purely performance-maximizing agents. Moreover, we show that such human-centric design can improve the likability of AI collaborators without reducing performance. We find evidence for inequality-aversion effects being a driver of human choices, suggesting that people prefer collaborative agents which allow them to meaningfully contribute to the team. Taken together, these findings demonstrate how collaboration with AI can benefit from development efforts which include both subjective and objective metrics.

Method: Created Deep Reasoning Dataset (DeepRD) with generative process for scalable complexity; evaluated LRMs on graph connectivity and natural language proof planning across complexity levels; compared with real-world knowledge graphs and proof datasets.

Result: LRM performance drops abruptly at sufficient complexity and fails to generalize; most real-world examples fall within LRMs’ success regime but long tails expose substantial failure potential.

Conclusion: LRMs have near-term utility but lack true generalization; new methods are needed that can handle complexity beyond training distribution.

Abstract: Large language models (LLMs) have shown significant progress in reasoning tasks. However, recent studies show that transformers and LLMs fail catastrophically once reasoning problems exceed modest complexity. We revisit these findings through the lens of large reasoning models (LRMs) – LLMs fine-tuned with incentives for step-by-step argumentation and self-verification. LRM performance on graph and reasoning benchmarks such as NLGraph seem extraordinary, with some even claiming they are capable of generalized reasoning and innovation in reasoning-intensive fields such as mathematics, physics, medicine, and law. However, by more carefully scaling the complexity of reasoning problems, we show existing benchmarks actually have limited complexity. We develop a new dataset, the Deep Reasoning Dataset (DeepRD), along with a generative process for producing unlimited examples of scalable complexity. We use this dataset to evaluate model performance on graph connectivity and natural language proof planning. We find that the performance of LRMs drop abruptly at sufficient complexity and do not generalize. We also relate our LRM results to the distributions of the complexities of large, real-world knowledge graphs, interaction graphs, and proof datasets. We find the majority of real-world examples fall inside the LRMs’ success regime, yet the long tails expose substantial failure potential. Our analysis highlights the near-term utility of LRMs while underscoring the need for new methods that generalize beyond the complexity of examples in the training distribution.

Method: Adopt a memoryless Finite State Machine formulation to approximate LRM’s reasoning dynamics, identifying discrete reasoning states (initialization, deduction, augmentation-strategy, uncertainty-estimation, backtracking, final-conclusion) and representing reasoning trajectories as state transitions.

Result: The FSM-based analysis reveals distinct reasoning patterns and potential shortcomings in different models, providing a systematic way to interpret and visualize how models approach problems.

Conclusion: The FSM formulation offers a new interpretable abstraction for analyzing LLM reasoning, enabling better evaluation and improvement of reasoning capabilities through structured state-based analysis.

Abstract: Large Language Models (LLMs) have demonstrated remarkable reasoning abilities when they generate step-by-step solutions, known as chain-of-thought (CoT) reasoning. When trained to using chain-of-thought reasoning examples, the resulting models (called Large Reasoning Models, or LRMs) appear to learn hierarchical thinking strategies similar to those used by humans. However, understanding LRMs emerging reasoning capabilities remains a difficult open problem, with many potential important applications including improving training and understanding robustness. In this paper, we adopt a memoryless Finite State Machine formulation to approximate LRM’s emerging hierarchical reasoning dynamics as a structured, interpretable abstraction. We identify a small set of discrete reasoning states including - initialization, deduction, augmentation-strategy, uncertainty-estimation, backtracking, and final-conclusion that capture the high-level states present in the model’s reasoning process. By annotating each step of a model’s CoT with these states, we can represent the reasoning trajectory as a transition sequence through the state graph. This FSM formulation provides a systematic way to analyze, interpret and visualize how different models approach problems. We describe the FSM model, provide examples of CoT annotations under this scheme, and discuss how it can shed light on differences between available models in their approach to reasoning. Our results demonstrate that this FSM-based analysis reveals distinct reasoning patterns and potential shortcomings, offering a new lens to evaluate and improve LLM reasoning.

Method: Developed ACES - a sandbox environment with platform-agnostic agents and programmable mock marketplace, conducted rationality checks, randomized experiments on product positions and attributes, and tested seller-side agent manipulation.

Result: AI agents show heterogeneous preferences: all favor top row but prefer different columns, penalize sponsored tags, reward endorsements, vary in price/rating sensitivity, and seller agents can gain substantial market share by targeting AI preferences.

Conclusion: AI agent behavior differs significantly from humans, raising important questions about seller strategies, platform design, and regulation in AI-driven e-commerce markets.

Abstract: Online marketplaces will be transformed by autonomous AI agents acting on behalf of consumers. Rather than humans browsing and clicking, AI agents can parse webpages or interact through APIs to evaluate products, and transact. This raises a fundamental question: what do AI agents buy-and why? We develop ACES, a sandbox environment that pairs a platform-agnostic agent with a fully programmable mock marketplace to study this. We first explore aggregate choices, revealing that modal choices can differ across models, with AI agents sometimes concentrating on a few products, raising competition questions. We then analyze the drivers of choices through rationality checks and randomized experiments on product positions and listing attributes. Models show sizeable and heterogeneous position effects: all favor the top row, yet different models prefer different columns, undermining the assumption of a universal ``top’’ rank. They penalize sponsored tags, reward endorsements, and sensitivities to price, ratings, and reviews are directionally as expected, but vary sharply across models. Finally, we find that a seller-side agent that makes minor tweaks to product descriptions can deliver substantial market-share gains by targeting AI buyer preferences. Our findings reveal how AI agents behave in e-commerce, and surface concrete seller strategy, platform design, and regulatory questions.

Method: Uses a two-player Modified-Action MDP to analyze suboptimal behavior of standard AI agents. Proposes ICR algorithm to train partner-aware collaborators that can intelligently collect information from partner interventions to increase common-ground alignment.

Result: Experiments show ICR is more capable of promoting successful common-ground convergence and exploring diverse solutions in collaborative tasks compared to standard approaches.

Conclusion: ICR provides an effective approach for training LLM agents that can better collaborate with partners by being responsive to interventions and improving group common ground, addressing limitations of standard RLHF-trained agents.

Abstract: Large Language Models (LLMs) are increasingly bring deployed in agentic settings where they act as collaborators with humans. Therefore, it is increasingly important to be able to evaluate their abilities to collaborate effectively in multi-turn, multi-party tasks. In this paper, we build on the AI alignment and safe interruptability literature to offer novel theoretical insights on collaborative behavior between LLM-driven collaborator agents and an intervention agent. Our goal is to learn an ideal partner-aware collaborator that increases the group’s common-ground (CG)-alignment on task-relevant propositions-by intelligently collecting information provided in interventions by a partner agent.We show how LLM agents trained using standard RLHF and related approaches are naturally inclined to ignore possibly well-meaning interventions, which makes increasing group common ground non-trivial in this setting. We employ a two-player Modified-Action MDP to examine this suboptimal behavior of standard AI agents, and propose Interruptible Collaborative Roleplayer (ICR)-a novel partner-aware learning algorithm to train CG-optimal collaborators. Experiments on multiple collaborative task environments show that ICR, on average, is more capable of promoting successful CG convergence and exploring more diverse solutions in such tasks.

Method: MI9 uses six integrated components: agency-risk index, semantic telemetry capture, continuous authorization monitoring, FSM-based conformance engines, goal-conditioned drift detection, and graduated containment strategies.

Result: The framework enables systematic, safe deployment of agentic systems in production environments, covering governance challenges that existing approaches fail to address.

Conclusion: MI9 establishes the technical foundation for comprehensive agentic AI oversight and provides infrastructure for safe agentic AI deployment at scale.

Abstract: Agentic AI systems capable of reasoning, planning, and executing actions present fundamentally distinct governance challenges compared to traditional AI models. Unlike conventional AI, these systems exhibit emergent and unexpected behaviors during runtime, introducing novel agent-related risks that cannot be fully anticipated through pre-deployment governance alone. To address this critical gap, we introduce MI9, the first fully integrated runtime governance framework designed specifically for safety and alignment of agentic AI systems. MI9 introduces real-time controls through six integrated components: agency-risk index, agent-semantic telemetry capture, continuous authorization monitoring, Finite-State-Machine (FSM)-based conformance engines, goal-conditioned drift detection, and graduated containment strategies. Operating transparently across heterogeneous agent architectures, MI9 enables the systematic, safe, and responsible deployment of agentic systems in production environments where conventional governance approaches fall short, providing the foundational infrastructure for safe agentic AI deployment at scale. Detailed analysis through a diverse set of scenarios demonstrates MI9’s systematic coverage of governance challenges that existing approaches fail to address, establishing the technical foundation for comprehensive agentic AI oversight.

Method: Created OFFSIDE benchmark with 15.68K manually curated records for 80 players, featuring four test sets for forgetting efficacy, generalization, utility, and robustness. Supports selective unlearning, corrective relearning, and unimodal unlearning.

Result: Key findings: unimodal methods fail on multimodal rumors; unlearning efficacy driven by catastrophic forgetting; all methods struggle with visual rumors; unlearned rumors easily recoverable; all methods vulnerable to prompt attacks.

Conclusion: Current approaches have significant vulnerabilities, highlighting need for more robust multimodal unlearning solutions.

Abstract: Advances in Multimodal Large Language Models (MLLMs) intensify concerns about data privacy, making Machine Unlearning (MU), the selective removal of learned information, a critical necessity. However, existing MU benchmarks for MLLMs are limited by a lack of image diversity, potential inaccuracies, and insufficient evaluation scenarios, which fail to capture the complexity of real-world applications. To facilitate the development of MLLMs unlearning and alleviate the aforementioned limitations, we introduce OFFSIDE, a novel benchmark for evaluating misinformation unlearning in MLLMs based on football transfer rumors. This manually curated dataset contains 15.68K records for 80 players, providing a comprehensive framework with four test sets to assess forgetting efficacy, generalization, utility, and robustness. OFFSIDE supports advanced settings like selective unlearning and corrective relearning, and crucially, unimodal unlearning (forgetting only text data). Our extensive evaluation of multiple baselines reveals key findings: (1) Unimodal methods (erasing text-based knowledge) fail on multimodal rumors; (2) Unlearning efficacy is largely driven by catastrophic forgetting; (3) All methods struggle with “visual rumors” (rumors appear in the image); (4) The unlearned rumors can be easily recovered and (5) All methods are vulnerable to prompt attacks. These results expose significant vulnerabilities in current approaches, highlighting the need for more robust multimodal unlearning solutions. The code is available at \href{https://github.com/zh121800/OFFSIDE}{https://github.com/zh121800/OFFSIDE}.

Method: The authors employ MAIDs as a graphical framework to analyze MARL interaction paradigms, design targeted intervention applied to single agents, and implement Pre-Strategy Intervention (PSI) using causal inference techniques.

Result: The proposed targeted intervention paradigm mitigates global guidance problems, and MAIDs provide tools to identify workable MARL learning paradigms through relevance graph analysis.

Conclusion: The paper demonstrates effective targeted intervention and validates relevance graph analysis, showing MAIDs as a practical framework for coordinating multi-agent systems without requiring global guidance.

Abstract: Steering cooperative multi-agent reinforcement learning (MARL) towards desired outcomes is challenging, particularly when the global guidance from a human on the whole multi-agent system is impractical in a large-scale MARL. On the other hand, designing external mechanisms (e.g., intrinsic rewards and human feedback) to coordinate agents mostly relies on empirical studies, lacking a easy-to-use research tool. In this work, we employ multi-agent influence diagrams (MAIDs) as a graphical framework to address the above issues. First, we introduce the concept of MARL interaction paradigms (orthogonal to MARL learning paradigms), using MAIDs to analyze and visualize both unguided self-organization and global guidance mechanisms in MARL. Then, we design a new MARL interaction paradigm, referred to as the targeted intervention paradigm that is applied to only a single targeted agent, so the problem of global guidance can be mitigated. In implementation, we introduce a causal inference technique, referred to as Pre-Strategy Intervention (PSI), to realize the targeted intervention paradigm. Since MAIDs can be regarded as a special class of causal diagrams, a composite desired outcome that integrates the primary task goal and an additional desired outcome can be achieved by maximizing the corresponding causal effect through the PSI. Moreover, the bundled relevance graph analysis of MAIDs provides a tool to identify whether an MARL learning paradigm is workable under the design of an MARL interaction paradigm. In experiments, we demonstrate the effectiveness of our proposed targeted intervention, and verify the result of relevance graph analysis.

Method: Splits documents into minimal atomic facts, constructs atomic TKGs with dual-time modeling (distinguishing observation time from validity time), and merges them in parallel.

Result: Achieves ~18% higher exhaustivity, ~17% better stability, and over 90% latency reduction compared to baseline methods.

Conclusion: ATOM demonstrates strong scalability potential for dynamic TKG construction with improved performance across key metrics.

Abstract: In today’s rapidly expanding data landscape, knowledge extraction from unstructured text is vital for real-time analytics, temporal inference, and dynamic memory frameworks. However, traditional static knowledge graph (KG) construction often overlooks the dynamic and time-sensitive nature of real-world data, limiting adaptability to continuous changes. Moreover, recent zero- or few-shot approaches that avoid domain-specific fine-tuning or reliance on prebuilt ontologies often suffer from instability across multiple runs, as well as incomplete coverage of key facts. To address these challenges, we introduce ATOM (AdapTive and OptiMized), a few-shot and scalable approach that builds and continuously updates Temporal Knowledge Graphs (TKGs) from unstructured texts. ATOM splits input documents into minimal, self-contained “atomic” facts, improving extraction exhaustivity and stability. Then, it constructs atomic TKGs from these facts while employing a dual-time modeling that distinguishes when information is observed from when it is valid. The resulting atomic TKGs are subsequently merged in parallel. Empirical evaluations demonstrate that ATOM achieves ~18% higher exhaustivity, ~17% better stability, and over 90% latency reduction compared to baseline methods, demonstrating a strong scalability potential for dynamic TKG construction.

Method: Proposes a new analytical framework for ICL performance in realistic settings including network architectures, data encoding, data generation, and prompt construction. Begins with a simple one-layer transformer example, then extends to more general cases, deriving relationships between ICL performance, context length, and KL divergence between pre-train and query distributions.

Result: Shows that when pre-train data distribution differs from query task distribution, properly constructed context can shift output distribution toward query task distribution in a quantifiable manner, leading to accurate prediction. Derives precise relationship between ICL performance, context length, and KL divergence.

Conclusion: Provides theoretical foundation for understanding in-context learning mechanisms, demonstrating how context construction can bridge distribution gaps between pre-training and query tasks, with experimental validation of theoretical results.

Abstract: Pre-trained large language models have demonstrated a strong ability to learn from context, known as in-context learning (ICL). Despite a surge of recent applications that leverage such capabilities, it is by no means clear, at least theoretically, how the ICL capabilities arise, and in particular, what is the precise role played by key factors such as pre-training procedure as well as context construction. In this work, we propose a new framework to analyze the ICL performance, for a class of realistic settings, which includes network architectures, data encoding, data generation, and prompt construction process. As a first step, we construct a simple example with a one-layer transformer, and show an interesting result, namely when the pre-train data distribution is different from the query task distribution, a properly constructed context can shift the output distribution towards the query task distribution, in a quantifiable manner, leading to accurate prediction on the query topic. We then extend the findings in the previous step to a more general case, and derive the precise relationship between ICL performance, context length and the KL divergence between pre-train and query task distribution. Finally, we provide experiments to validate our theoretical results.

Method: Fine-tunes BioBERT on clinical cases with Focal Loss and Monte Carlo Dropout for confidence-aware predictions, uses Biomedical Sentence-BERT for evidence retrieval from MedDialog corpus, and fine-tunes FLAN-T5 for personalized treatment generation with RxNorm post-processing for antibiotic stewardship and DDI screening.

Result: Achieves 98% accuracy and F1 score, outperforming ClinicalBERT by 7.1%, with 78% top-5 retrieval precision, clinician-rated validity of 4.2/5, and reduces unsafe antibiotic suggestions by 67% compared to GPT-5.

Conclusion: CLIN-LLM provides a deployable, human-in-the-loop decision support framework with robustness, interpretability, and clinical safety alignment for resource-limited healthcare environments.

Abstract: Accurate symptom-to-disease classification and clinically grounded treatment recommendations remain challenging, particularly in heterogeneous patient settings with high diagnostic risk. Existing large language model (LLM)-based systems often lack medical grounding and fail to quantify uncertainty, resulting in unsafe outputs. We propose CLIN-LLM, a safety-constrained hybrid pipeline that integrates multimodal patient encoding, uncertainty-calibrated disease classification, and retrieval-augmented treatment generation. The framework fine-tunes BioBERT on 1,200 clinical cases from the Symptom2Disease dataset and incorporates Focal Loss with Monte Carlo Dropout to enable confidence-aware predictions from free-text symptoms and structured vitals. Low-certainty cases (18%) are automatically flagged for expert review, ensuring human oversight. For treatment generation, CLIN-LLM employs Biomedical Sentence-BERT to retrieve top-k relevant dialogues from the 260,000-sample MedDialog corpus. The retrieved evidence and patient context are fed into a fine-tuned FLAN-T5 model for personalized treatment generation, followed by post-processing with RxNorm for antibiotic stewardship and drug-drug interaction (DDI) screening. CLIN-LLM achieves 98% accuracy and F1 score, outperforming ClinicalBERT by 7.1% (p < 0.001), with 78% top-5 retrieval precision and a clinician-rated validity of 4.2 out of 5. Unsafe antibiotic suggestions are reduced by 67% compared to GPT-5. These results demonstrate CLIN-LLM’s robustness, interpretability, and clinical safety alignment. The proposed system provides a deployable, human-in-the-loop decision support framework for resource-limited healthcare environments. Future work includes integrating imaging and lab data, multilingual extensions, and clinical trial validation.

Method: Uses specialized agents (Planner, Static Pruner, Coder, Profiler, Complexity Analyst) that communicate via typed JSON. The system performs algorithmic planning, empirical profiling, complexity analysis, and targeted repairs through an iterative process with iteration caps.

Result: Achieved 61.54% pass@1 and 80.77% Solved@<=3 on 26 problems, with 73.08% aggregate run-level success. Outperformed Claude Opus 4 (73.1% vs 52.6%) with 2x runtime overhead.

Conclusion: SwiftSolve demonstrates that profiling and complexity-guided replanning effectively reduce inefficiency while maintaining accuracy, addressing the critical gap between program correctness and efficiency in competitive programming.

Abstract: Correctness alone is insufficient: LLM-generated programs frequently satisfy unit tests while violating contest time or memory budgets. We present SwiftSolve, a complexity-aware multi-agent system for competitive programming that couples algorithmic planning with empirical profiling and complexity-guided repair. We frame competitive programming as a software environment where specialized agents act as programmers, each assuming roles such as planning, coding, profiling, and complexity analysis. A Planner proposes an algorithmic sketch; a deterministic Static Pruner filters high-risk plans; a Coder emits ISO C++17; a Profiler compiles and executes candidates on a fixed input-size schedule to record wall time and peak memory; and a Complexity Analyst fits log-log growth (s, R2) with an LLM fallback to assign a complexity class and dispatch targeted patches to either the Planner or Coder. Agents communicate via typed, versioned JSON; a controller enforces iteration caps and diminishing returns stopping. Evaluated on 26 problems (16 BigO, 10 Codeforces Div. 2) in a POSIX sandbox (2 s / 256-512 MB), SwiftSolve attains pass@1 = 61.54% (16/26) on the first attempt and Solved@<=3 = 80.77% with marginal latency change (mean 11.96 s to 12.66 s per attempt). Aggregate run-level success is 73.08% at 12.40 s mean. Failures are predominantly resource-bound, indicating inefficiency rather than logic errors. Against Claude Opus 4, SwiftSolve improves run-level success (73.1% vs 52.6%) at approximately 2x runtime overhead (12.4 s vs 6.8 s). Beyond correctness (pass@k), we report efficiency metrics (eff@k for runtime and memory, incidence of TLE or MLE, and complexity fit accuracy on BigO), demonstrating that profiling and complexity-guided replanning reduce inefficiency while preserving accuracy.

Method: Use first-token log-probability distribution as signal to classify response type; apply lightweight k-NN classifier for prediction.

Result: First-token log-probability vectors form separable clusters for different response types; high accuracy in predicting substantive vs boilerplate responses.

Conclusion: Practical technique for optimizing LLM inference through early termination, yielding significant computational savings for more efficient deployment.

Abstract: Large Language Models (LLMs) often expend significant computational resources generating boilerplate responses, such as refusals, simple acknowledgements and casual greetings, which adds unnecessary cost and latency. To address this inefficiency, we propose a simple yet highly effective method for detecting such responses after only a single generation step. We demonstrate that the log-probability distribution of the first generated token serves as a powerful signal for classifying the nature of the entire subsequent response. Our experiments, conducted across a diverse range of small, large, and reasoning-specialized models, show that the first-token log-probability vectors form distinctly separable clusters for different response types. Using a lightweight k-NN classifier, we achieve high accuracy in predicting whether a response will be a substantive answer or a form of boilerplate response, including user-specified refusals. The primary implication is a practical, computationally trivial technique, optimizing LLM inference by enabling early termination or redirection to a smaller model, thereby yielding significant savings in computational cost. This work presents a direct path toward more efficient and sustainable LLM deployment.

Method: Generates semantically adjacent but differently answered contrastive questions, extracts a Δ-Prompt to capture key differences, and guides the model to focus on critical details that determine answer divergence in a single retrieval pass.

Result: Outperforms strong baselines by 0.9-2.4 percentage points on six benchmarks, shows superior robustness with only 8.6% performance drop in adversarial tests, and achieves low latency (3.12s) with minimal token overhead (11.54).

Conclusion: RaCoT reframes RAG from post-hoc context cleaning to a priori shaping of discriminative reasoning, offering an efficient and robust path for reliable AI systems in real-time, resource-constrained deployments.

Abstract: Retrieval-Augmented Generation (RAG) faces a core bottleneck with knowledge-sparse and semantically ambiguous long-tail queries, where retrieval noise distorts reasoning and necessitates costly post-processing. To tackle this, we propose RaCoT (Retrieval-aware Contrastive-of-Thought), a novel framework that shifts contrastive thinking to the pre-retrieval stage. By automatically generating a semantically adjacent yet differently answered contrastive question and extracting a $\Delta$-Prompt to capture their key differences, RaCoT guides the model to proactively focus on the critical details that determine answer divergence." This approach allows it to suppress semantic interference within a single retrieval pass, overcoming the theoretical bottleneck of single-vector queries that struggle to simultaneously encode signals for what to attend to and what to ignore. On six authoritative benchmarks, including PopQA and TriviaQA-unfiltered, RaCoT outperforms strong baselines like RankRAG and Self-RAG by 0.9-2.4 percentage points. It exhibits superior robustness, with a performance drop of only 8.6\% in adversarial tests, far surpassing the over 15\% degradation in other methods. Furthermore, its low latency (3.12s) and token overhead (11.54) place it on the accuracy-efficiency Pareto frontier, while ablation studies validate the necessity of each component. Ultimately, RaCoT reframes the RAG paradigm from post-hoc context cleaning" to ``a priori shaping of discriminative reasoning", offering an efficient and robust path toward reliable AI systems for real-time, resource-constrained deployments.

Method: Comparative analysis of five top LLMs (Google’s Gemini, High-Flyer’s DeepSeek, Anthropic’s Claude, OpenAI’s GPT, Meta’s LLaMA) across three key factors: Performance and Accuracy, Ethics and Bias Mitigation, and Usability and Integration.

Result: Claude excels in moral reasoning, Gemini has superior multimodal capabilities and strong ethical frameworks, DeepSeek is excellent at fact-based reasoning, LLaMA is good for open applications, and ChatGPT delivers balanced performance with usage focus.

Conclusion: LLMs differ significantly in performance, usability, and ethical treatment, suggesting users should leverage each model’s specific strengths for optimal results.

Abstract: Big Language Models (LLMs) are changing the way businesses use software, the way people live their lives and the way industries work. Companies like Google, High-Flyer, Anthropic, OpenAI and Meta are making better LLMs. So, it’s crucial to look at how each model is different in terms of performance, moral behaviour and usability, as these differences are based on the different ideas that built them. This study compares five top LLMs: Google’s Gemini, High-Flyer’s DeepSeek, Anthropic’s Claude, OpenAI’s GPT models and Meta’s LLaMA. It performs this by analysing three important factors: Performance and Accuracy, Ethics and Bias Mitigation and Usability and Integration. It was found that Claude has good moral reasoning, Gemini is better at multimodal capabilities and has strong ethical frameworks. DeepSeek is great at reasoning based on facts, LLaMA is good for open applications and ChatGPT delivers balanced performance with a focus on usage. It was concluded that these models are different in terms of how well they work, how easy they are to use and how they treat people ethically, making it a point that each model should be utilised by the user in a way that makes the most of its strengths.

Method: Developed a framework that verifies LLM outputs in real-time by cross-checking against structured databases, live web searches, and academic literature, automatically correcting inconsistencies while preserving response flow.

Result: Reduced hallucinations by 67% across various domains; domain experts rated corrected outputs 89% satisfactory, a significant improvement over unverified LLM responses.

Conclusion: Provides a practical solution for making LLMs more trustworthy in applications where factual accuracy is critical.

Abstract: While Large Language Models have transformed how we interact with AI systems, they suffer from a critical flaw: they confidently generate false information that sounds entirely plausible. This hallucination problem has become a major barrier to deploying these models in real-world applications where accuracy matters. We developed a fact verification framework that catches and corrects these errors in real-time by cross checking LLM outputs against multiple knowledge sources. Our system combines structured databases, live web searches, and academic literature to verify factual claims as they’re generated. When we detect inconsistencies, we automatically correct them while preserving the natural flow of the response. Testing across various domains showed we could reduce hallucinations by 67% without sacrificing response quality. Domain experts in healthcare, finance, and scientific research rated our corrected outputs 89% satisfactory a significant improvement over unverified LLM responses. This work offers a practical solution for making LLMs more trustworthy in applications where getting facts wrong isn’t an option.

Method: Stores user-specific information in KV-Caches of both textual and visual tokens. Textual tokens from metadata summaries, visual tokens from distinct image patches. Retrieves related KV-Caches when answering questions to ensure accuracy.

Result: Jarvis provides more accurate responses, especially for fine-grained local details. Achieves state-of-the-art results in visual question answering and text-only tasks across multiple datasets.

Conclusion: Jarvis represents a practical path toward personalized AI assistants by effectively leveraging user-specific information through KV-Cache retrieval.

Abstract: The rapid development of Vision-language models (VLMs) enables open-ended perception and reasoning. Recent works have started to investigate how to adapt general-purpose VLMs into personalized assistants. Even commercial models such as ChatGPT now support model personalization by incorporating user-specific information. However, existing methods either learn a set of concept tokens or train a VLM to utilize user-specific information. However, both pipelines struggle to generate accurate answers as personalized assistants. We introduce Jarvis, an innovative framework for a personalized AI assistant through personal KV-Cache retrieval, which stores user-specific information in the KV-Caches of both textual and visual tokens. The textual tokens are created by summarizing user information into metadata, while the visual tokens are produced by extracting distinct image patches from the user’s images. When answering a question, Jarvis first retrieves related KV-Caches from personal storage and uses them to ensure accuracy in responses. We also introduce a fine-grained benchmark built with the same distinct image patch mining pipeline, emphasizing accurate question answering based on fine-grained user-specific information. Jarvis is capable of providing more accurate responses, particularly when they depend on specific local details. Jarvis achieves state-of-the-art results in both visual question answering and text-only tasks across multiple datasets, indicating a practical path toward personalized AI assistants. The code and dataset will be released.

Method: Introduced a scalable toolkit to induce interpretable, structured workflows from computer-use activities of both humans and agents, comparing performance across data analysis, engineering, computation, writing, and design tasks.

Result: Agents align with human workflows but use programmatic approaches even for visual tasks, produce inferior quality work with data fabrication, yet are 88.3% faster and 90.4-96.2% cheaper than humans.

Conclusion: Agents show potential for efficient collaboration by handling programmable tasks, but need improvement in quality and transparency to complement human workers effectively.

Abstract: AI agents are continually optimized for tasks related to human work, such as software engineering and professional writing, signaling a pressing trend with significant impacts on the human workforce. However, these agent developments have often not been grounded in a clear understanding of how humans execute work, to reveal what expertise agents possess and the roles they can play in diverse workflows. In this work, we study how agents do human work by presenting the first direct comparison of human and agent workers across multiple essential work-related skills: data analysis, engineering, computation, writing, and design. To better understand and compare heterogeneous computer-use activities of workers, we introduce a scalable toolkit to induce interpretable, structured workflows from either human or agent computer-use activities. Using such induced workflows, we compare how humans and agents perform the same tasks and find that: (1) While agents exhibit promise in their alignment to human workflows, they take an overwhelmingly programmatic approach across all work domains, even for open-ended, visually dependent tasks like design, creating a contrast with the UI-centric methods typically used by humans. (2) Agents produce work of inferior quality, yet often mask their deficiencies via data fabrication and misuse of advanced tools. (3) Nonetheless, agents deliver results 88.3% faster and cost 90.4-96.2% less than humans, highlighting the potential for enabling efficient collaboration by delegating easily programmable tasks to agents.

Method: Uses an Agentic Meta-orchestrator (AMO) with meta-learning through a trained decision tree model to select the best inference strategy among various agents/models, supporting both natural language and action responses.

Result: Demonstrated effectiveness through two production use cases: M365 E-Commerce Copilot (providing product information and connecting to databases/human support) and code compliance copilot (detecting compliance issues in DevOps code).

Conclusion: The AMO framework successfully orchestrates multiple agents in copilot services, enabling dynamic task distribution and scalable agent management with practical applications in e-commerce and code compliance domains.

Abstract: Microsoft Copilot suites serve as the universal entry point for various agents skilled in handling important tasks, ranging from assisting a customer with product purchases to detecting vulnerabilities in corporate programming code. Each agent can be powered by language models, software engineering operations, such as database retrieval, and internal & external knowledge. The repertoire of a copilot can expand dynamically with new agents. This requires a robust orchestrator that can distribute tasks from user prompts to the right agents. In this work, we propose an Agentic Meta-orchestrator (AMO) for handling multiple tasks and scalable agents in copilot services, which can provide both natural language and action responses. We will also demonstrate the planning that leverages meta-learning, i.e., a trained decision tree model for deciding the best inference strategy among various agents/models. We showcase the effectiveness of our AMO through two production use cases: Microsoft 365 (M365) E-Commerce Copilot and code compliance copilot. M365 E-Commerce Copilot advertises Microsoft products to external customers to promote sales success. The M365 E-Commerce Copilot provides up-to-date product information and connects to multiple agents, such as relational databases and human customer support. The code compliance copilot scans the internal DevOps code to detect known and new compliance issues in pull requests (PR).

Method: Analysis based on Irving J. Good and Nick Bostrom’s theoretical work, plus recent publications (AI 2027; If Anyone Builds It, Everyone Dies), exploring AGI and superintelligence concepts and their implications.

Result: Identifies that human extinction could result from machines’ exponentially growing cognitive power creating fundamentally alien intelligence that vastly exceeds humanity’s capabilities, with extinction potentially arising from indifference rather than malice.

Conclusion: The existential risk from AI superintelligence stems from uncontrollable cognitive superiority that may be indifferent rather than malicious, posing fundamental threats to human survival that require careful consideration.

Abstract: This article analyzes the existential risks artificial intelligence (AI) poses to humanity, tracing the trajectory from current AI to ultraintelligence. Drawing on Irving J. Good and Nick Bostrom’s theoretical work, plus recent publications (AI 2027; If Anyone Builds It, Everyone Dies), it explores AGI and superintelligence. Considering machines’ exponentially growing cognitive power and hypothetical IQs, it addresses the ethical and existential implications of an intelligence vastly exceeding humanity’s, fundamentally alien. Human extinction may result not from malice, but from uncontrollable, indifferent cognitive superiority.

Method: Developed HRM-Agent, a reinforcement learning variant of HRM that can navigate dynamic maze environments and reuse computation from previous time-steps through recurrent inference.

Result: HRM-Agent successfully learned to navigate to goals in dynamic and uncertain maze environments, with evidence showing effective computation reuse across time-steps.

Conclusion: HRM can be effectively adapted for reinforcement learning in dynamic environments while maintaining its core reasoning capabilities and computation reuse properties.

Abstract: The Hierarchical Reasoning Model (HRM) has impressive reasoning abilities given its small size, but has only been applied to supervised, static, fully-observable problems. One of HRM’s strengths is its ability to adapt its computational effort to the difficulty of the problem. However, in its current form it cannot integrate and reuse computation from previous time-steps if the problem is dynamic, uncertain or partially observable, or be applied where the correct action is undefined, characteristics of many real-world problems. This paper presents HRM-Agent, a variant of HRM trained using only reinforcement learning. We show that HRM can learn to navigate to goals in dynamic and uncertain maze environments. Recent work suggests that HRM’s reasoning abilities stem from its recurrent inference process. We explore the dynamics of the recurrent inference process and find evidence that it is successfully reusing computation from earlier environment time-steps.

Method: Show agents the cost of their computation and give them control over when to use compute. Experiments conducted on the Arcade Learning Environment.

Result: With the same training compute budget, agents that reason about compute perform better on 75% of games and use three times less compute on average.

Conclusion: Agents can learn to be computationally efficient while maintaining performance, enabling more energy-efficient AI systems.

Abstract: While reinforcement learning agents can achieve superhuman performance in many complex tasks, they typically do not become more computationally efficient as they improve. In contrast, humans gradually require less cognitive effort as they become more proficient at a task. If agents could reason about their compute as they learn, could they similarly reduce their computation footprint? If they could, we could have more energy efficient agents or free up compute cycles for other processes like planning. In this paper, we experiment with showing agents the cost of their computation and giving them the ability to control when they use compute. We conduct our experiments on the Arcade Learning Environment, and our results demonstrate that with the same training compute budget, agents that reason about their compute perform better on 75% of games. Furthermore, these agents use three times less compute on average. We analyze individual games and show where agents gain these efficiencies.

Method: Analyze modality gap through training recipes, explore bridging strategies from data and loss design perspectives.

Result: Existing training recipes amplify the modality gap; systematic approaches can help bridge this performance disparity.

Conclusion: Insights provided for developing training recipes that mitigate modality gap and promote balanced multimodal reasoning.

Abstract: Multimodal large language models (MLLMs) have demonstrated strong capabilities on vision-and-language tasks. However, recent findings reveal an imbalance in their reasoning capabilities across visual and textual modalities. Specifically, current MLLMs often over-rely on textual cues while under-attending to visual content, resulting in suboptimal performance on tasks that require genuine visual reasoning. We refer to this phenomenon as the \textit{modality gap}, defined as the performance disparity between text-centric and vision-centric inputs. In this paper, we analyze the modality gap through the lens of training recipes. We first show that existing training recipes tend to amplify this gap. Then, we systematically explore strategies to bridge it from two complementary perspectives: data and loss design. Our findings provide insights into developing training recipes that mitigate the modality gap and promote more balanced multimodal reasoning. Our code is publicly available at https://github.com/UCSB-NLP-Chang/Bridging-Modality-Gap.

Method: Analysis of convergence performance through Lyapunov function candidate increments, accounting for discretization errors and state prediction errors from the incremental model. Comparative flight control simulations are used.

Result: The paper presents comparative results from flight control simulations demonstrating the convergence performance of the enhanced control system.

Conclusion: The enhanced cascaded online learning flight control system with improved action smoothness shows analyzed convergence performance through Lyapunov-based metrics and simulation validation.

Abstract: A cascaded online learning flight control system has been developed and enhanced with respect to action smoothness. In this paper, we investigate the convergence performance of the control system, characterized by the increment of a Lyapunov function candidate. The derivation of this metric accounts for discretization errors and state prediction errors introduced by the incremental model. Comparative results are presented through flight control simulations.

Method: Uses symbolic AI modeling techniques through electronic circuits based on logic gates, conducts combinatorial exploration to identify dependency forms, analyzes logic programs to identify inferential patterns, and applies probabilistic interpretation of logic programs.

Result: Identified four main forms of dependencies that can be realized by inferential circuits and eight common inferential patterns that expose traditionally distinct inferential mechanisms in a unifying framework.

Conclusion: The observations from symbolic means and digital systems infrastructures may point to more generally applicable structures for understanding inferential mechanisms across different domains.

Abstract: Cognitive studies and artificial intelligence have developed distinct models for various inferential mechanisms (categorization, induction, abduction, causal inference, contrast, merge, …). Yet, both natural and artificial views on cognition lack apparently a unifying framework. This paper formulates a speculative answer attempting to respond to this gap. To postulate on higher-level activation processes from a material perspective, we consider inferential mechanisms informed by symbolic AI modelling techniques, through the simplistic lenses of electronic circuits based on logic gates. We observe that a logic gate view entails a different treatment of implication and negation compared to standard logic and logic programming. Then, by combinatorial exploration, we identify four main forms of dependencies that can be realized by these inferential circuits. Looking at how these forms are generally used in the context of logic programs, we identify eight common inferential patterns, exposing traditionally distinct inferential mechanisms in an unifying framework. Finally, following a probabilistic interpretation of logic programs, we unveil inner functional dependencies. The paper concludes elaborating in what sense, even if our arguments are mostly informed by symbolic means and digital systems infrastructures, our observations may pinpoint to more generally applicable structures.

Method: Introduces MAVEN benchmark for OOD testing and CoreThink Agentic Reasoner - a framework that augments LLMs with lightweight symbolic reasoning layer for structured decomposition and adaptive tool orchestration.

Result: Most current models achieve below 50% accuracy on MAVEN. CoreThink generalizes across all benchmarks, achieving state-of-the-art performance with 530% improvements over baselines at roughly one-tenth the computational cost.

Conclusion: The CoreThink framework effectively addresses the generalization gap in tool-use settings without additional training, demonstrating significant performance improvements across diverse benchmarks.

Abstract: Generalization across Agentic tool-calling environments remains a key unsolved challenge in developing reliable agentic reasoning systems. While large language models (LLMs) demonstrate strong performance on isolated benchmarks, their ability to transfer reasoning strategies and co-ordinate tools across diverse domains is poorly understood. In this work, we conduct a large-scale evaluation of state-of-the-art LLMs on multiple tool-calling benchmarksBFCL v3, TauBench, Tau2Bench, and AceBenchand introduce MAVEN (Math & Physics Adversarial Verification & Evaluation Network), a new out of distribution (OOD) benchmark designed to stress-test multi-step reasoning through explicit verification and adversarial task composition. Our results show that most current models achieve below 50% accuracy on MAVEN, revealing a significant generalization gap across tool-use settings. To address this, we present the CoreThink Agentic Reasoner, a framework that augments LLMs with a lightweight symbolic reasoning layer for structured decomposition and adaptive tool orchestration. Without additional training, it generalizes across all benchmarks, achieving state-of-the-art performance with 530% improvements over existing baselines at roughly one-tenth the computational cost.

Method: Proposes cross-manifold conditioning and routing: models static preference hierarchies in hyperbolic geometry, routes dynamic sequence updates to Mamba layers in Euclidean tangent space, with cross-manifold channel fusion.

Result: Extensive experiments on three real-world datasets show GTR-Mamba consistently outperforms state-of-the-art baseline models in next POI recommendation.

Conclusion: The cross-manifold approach effectively addresses limitations of existing models by leveraging different mathematical spaces for different aspects of the recommendation problem.

Abstract: Next Point-of-Interest (POI) recommendation is a critical task in modern Location-Based Social Networks (LBSNs), aiming to model the complex decision-making process of human mobility to provide personalized recommendations for a user’s next check-in location. Existing POI recommendation models, predominantly based on Graph Neural Networks and sequential models, have been extensively studied. However, these models face a fundamental limitation: they struggle to simultaneously capture the inherent hierarchical structure of spatial choices and the dynamics and irregular shifts of user-specific temporal contexts. To overcome this limitation, we propose GTR-Mamba, a novel framework for cross-manifold conditioning and routing. GTR-Mamba leverages the distinct advantages of different mathematical spaces for different tasks: it models the static, tree-like preference hierarchies in hyperbolic geometry, while routing the dynamic sequence updates to a novel Mamba layer in the computationally stable and efficient Euclidean tangent space. This process is coordinated by a cross-manifold channel that fuses spatio-temporal information to explicitly steer the State Space Model (SSM), enabling flexible adaptation to contextual changes. Extensive experiments on three real-world datasets demonstrate that GTR-Mamba consistently outperforms state-of-the-art baseline models in next POI recommendation.

Method: InSUR framework with three key components: 1) Residual-driven attacking direction stabilization using ResAdv-DDIM sampler to stabilize adversarial optimization, 2) Context-encoded attacking scenario constraint with guidance masking and renderer integration for 2D/3D SemanticAE, 3) Semantic-abstracted attacking evaluation enhancement to clarify evaluation boundaries.

Result: Extensive experiments demonstrate superior transfer attack performance. The framework also enables reference-free generation of semantically constrained 3D adversarial examples for the first time.

Conclusion: InSUR effectively addresses semantic uncertainty in human instructions to generate more transferable, adaptive, and effective semantically constrained adversarial examples, advancing the field of semantic adversarial attacks.

Abstract: Recently, semantically constrained adversarial examples (SemanticAE), which are directly generated from natural language instructions, have become a promising avenue for future research due to their flexible attacking forms. To generate SemanticAEs, current methods fall short of satisfactory attacking ability as the key underlying factors of semantic uncertainty in human instructions, such as referring diversity, descriptive incompleteness, and boundary ambiguity, have not been fully investigated. To tackle the issues, this paper develops a multi-dimensional instruction uncertainty reduction (InSUR) framework to generate more satisfactory SemanticAE, i.e., transferable, adaptive, and effective. Specifically, in the dimension of the sampling method, we propose the residual-driven attacking direction stabilization to alleviate the unstable adversarial optimization caused by the diversity of language references. By coarsely predicting the language-guided sampling process, the optimization process will be stabilized by the designed ResAdv-DDIM sampler, therefore releasing the transferable and robust adversarial capability of multi-step diffusion models. In task modeling, we propose the context-encoded attacking scenario constraint to supplement the missing knowledge from incomplete human instructions. Guidance masking and renderer integration are proposed to regulate the constraints of 2D/3D SemanticAE, activating stronger scenario-adapted attacks. Moreover, in the dimension of generator evaluation, we propose the semantic-abstracted attacking evaluation enhancement by clarifying the evaluation boundary, facilitating the development of more effective SemanticAE generators. Extensive experiments demonstrate the superiority of the transfer attack performance of InSUR. Moreover, we realize the reference-free generation of semantically constrained 3D adversarial examples for the first time.

Method: Indexes multimodal knowledge base, selects explainers per instance via quantitative criteria, and generates grounded narratives with chat-enabled prompting using SHAP, LIME, and Anchor explainers.

Result: No single explainer dominates: LIME has best fidelity-robustness trade-off, Anchor produces sparsest rules, SHAP achieves highest user satisfaction. Profile conditioning stabilizes token usage and maintains positive ratings across user profiles.

Conclusion: ProfileXAI enables efficient and trustworthy explanations by leveraging multiple explainers with LLM augmentation, with profile conditioning providing stable performance across different user types.

Abstract: ProfileXAI is a model- and domain-agnostic framework that couples post-hoc explainers (SHAP, LIME, Anchor) with retrieval - augmented LLMs to produce explanations for different types of users. The system indexes a multimodal knowledge base, selects an explainer per instance via quantitative criteria, and generates grounded narratives with chat-enabled prompting. On Heart Disease and Thyroid Cancer datasets, we evaluate fidelity, robustness, parsimony, token use, and perceived quality. No explainer dominates: LIME achieves the best fidelity–robustness trade-off (Infidelity $\le 0.30$, $L<0.7$ on Heart Disease); Anchor yields the sparsest, low-token rules; SHAP attains the highest satisfaction ($\bar{x}=4.1$). Profile conditioning stabilizes tokens ($\sigma \le 13%$) and maintains positive ratings across profiles ($\bar{x}\ge 3.7$, with domain experts at $3.77$), enabling efficient and trustworthy explanations.

Method: Proposed a Multi-Dimensional Credibility Assessment (MDCA) framework and applied it to evaluate several advanced LLMs (Kimi-K2-Instruct-0905, Qwen3-235B-A22B-Instruct-2507, DeepSeek-V3, ByteDance-Seed-OSS-36B-Instruct) using the SiliconFlow platform.

Result: The study provides a systematic framework for assessing LLM-generated radiology reports and compares performance of multiple advanced language models in liver MRI reporting context.

Conclusion: The MDCA framework enables enhanced trustworthiness evaluation of LLM-generated radiology reports and offers guidance for institution-specific prompt optimization to improve clinical utility.

Abstract: Large language models (LLMs) have demonstrated promising performance in generating diagnostic conclusions from imaging findings, thereby supporting radiology reporting, trainee education, and quality control. However, systematic guidance on how to optimize prompt design across different clinical contexts remains underexplored. Moreover, a comprehensive and standardized framework for assessing the trustworthiness of LLM-generated radiology reports is yet to be established. This study aims to enhance the trustworthiness of LLM-generated liver MRI reports by introducing a Multi-Dimensional Credibility Assessment (MDCA) framework and providing guidance on institution-specific prompt optimization. The proposed framework is applied to evaluate and compare the performance of several advanced LLMs, including Kimi-K2-Instruct-0905, Qwen3-235B-A22B-Instruct-2507, DeepSeek-V3, and ByteDance-Seed-OSS-36B-Instruct, using the SiliconFlow platform.

Method: Proposed Mixed Density Diffuser (MDD) where densities throughout the horizon are tunable hyperparameters, allowing certain parts of trajectories to be more densely planned.

Result: MDD achieves new state-of-the-art performance across Maze2D, Franka Kitchen, and Antmaze D4RL task domains.

Conclusion: Tunable temporal density in diffusion planning enables better performance by adapting to non-uniform density requirements across the planning horizon.

Abstract: Recent studies demonstrate that diffusion planners benefit from sparse-step planning over single-step planning. Training models to skip steps in their trajectories helps capture long-term dependencies without additional or memory computational cost. However, predicting excessively sparse plans degrades performance. We hypothesize this temporal density threshold is non-uniform across a temporal horizon and that certain parts of a planned trajectory should be more densely planned. We propose Mixed Density Diffuser (MDD), a diffusion planner where the densities throughout the horizon are tunable hyperparameters. MDD achieves a new SOTA across the Maze2D, Franka Kitchen, and Antmaze D4RL task domains.

Method: Introduces a unified six-stage methodological framework deconstructing the scientific process: Literature Review, Idea Generation, Experimental Preparation, Experimental Execution, Scientific Writing, and Paper Generation, used to analyze field evolution from foundational modules to integrated closed-loop systems.

Result: Charts the field’s evolution from early Foundational Modules (2022-2023) to integrated Closed-Loop Systems (2024), and to current frontier of Scalability, Impact, and Human-AI Collaboration (2025-present), providing a systematic synthesis of autonomous science developments.

Conclusion: The survey clarifies the current state of autonomous science and provides a critical roadmap for overcoming remaining challenges in robustness and governance, guiding next-generation systems toward becoming trustworthy partners in human scientific inquiry.

Abstract: Artificial intelligence is undergoing a profound transition from a computational instrument to an autonomous originator of scientific knowledge. This emerging paradigm, the AI scientist, is architected to emulate the complete scientific workflow-from initial hypothesis generation to the final synthesis of publishable findings-thereby promising to fundamentally reshape the pace and scale of discovery. However, the rapid and unstructured proliferation of these systems has created a fragmented research landscape, obscuring overarching methodological principles and developmental trends. This survey provides a systematic and comprehensive synthesis of this domain by introducing a unified, six-stage methodological framework that deconstructs the end-to-end scientific process into: Literature Review, Idea Generation, Experimental Preparation, Experimental Execution, Scientific Writing, and Paper Generation. Through this analytical lens, we chart the field’s evolution from early Foundational Modules (2022-2023) to integrated Closed-Loop Systems (2024), and finally to the current frontier of Scalability, Impact, and Human-AI Collaboration (2025-present). By rigorously synthesizing these developments, this survey not only clarifies the current state of autonomous science but also provides a critical roadmap for overcoming remaining challenges in robustness and governance, ultimately guiding the next generation of systems toward becoming trustworthy and indispensable partners in human scientific inquiry.

Method: Proposes TLCD method combining deep learning techniques and transfer learning strategies to enhance target discipline performance by utilizing common features from source disciplines, based on neural network cognitive diagnosis and knowledge association neural network cognitive diagnosis.

Result: Experimental results show the cross-disciplinary cognitive diagnosis model based on deep learning performs better than basic models in cross-disciplinary tasks and can more accurately evaluate students’ learning situations.

Conclusion: The proposed TLCD method effectively addresses cross-disciplinary cognitive diagnosis challenges and provides more accurate assessment of student learning across different disciplines.

Abstract: Driven by the dual principles of smart education and artificial intelligence technology, the online education model has rapidly emerged as an important component of the education industry. Cognitive diagnostic technology can utilize students’ learning data and feedback information in educational evaluation to accurately assess their ability level at the knowledge level. However, while massive amounts of information provide abundant data resources, they also bring about complexity in feature extraction and scarcity of disciplinary data. In cross-disciplinary fields, traditional cognitive diagnostic methods still face many challenges. Given the differences in knowledge systems, cognitive structures, and data characteristics between different disciplines, this paper conducts in-depth research on neural network cognitive diagnosis and knowledge association neural network cognitive diagnosis, and proposes an innovative cross-disciplinary cognitive diagnosis method (TLCD). This method combines deep learning techniques and transfer learning strategies to enhance the performance of the model in the target discipline by utilizing the common features of the main discipline. The experimental results show that the cross-disciplinary cognitive diagnosis model based on deep learning performs better than the basic model in cross-disciplinary cognitive diagnosis tasks, and can more accurately evaluate students’ learning situation.

Method: Constructed a dataset from APPS coding challenge benchmark with correctness labels, trained a value-head model with regression layer on small LLMs (Phi-4 family) to estimate success probability of intermediate code outputs.

Result: Small LLMs successfully function as effective reward models and code evaluation critics, capable of identifying correct solutions among multiple candidates, leading to over 20% improvement in search capability for selecting the most accurate code.

Conclusion: Small language models can be effectively repurposed as reward models for code generation tasks, providing a practical approach to improve code quality selection without requiring large-scale models.

Abstract: Generating high-quality code remains a challenge for Large Language Models (LLMs). For the evolution of reasoning models on this task, reward models are a necessary intermediate step. These models judge outcomes or intermediate steps. Decoder-only transformer models can be turned into reward models by introducing a regression layer and supervised fine-tuning. While it is known that reflection capabilities generally increase with the size of a model, we want to investigate whether state-of-the-art small language models like the Phi-4 family can be turned into usable reward models blending the consideration of process rewards and outcome rewards. Targeting this goal, we construct a dataset of code samples with correctness labels derived from the APPS coding challenge benchmark. We then train a value-head model to estimate the success probability of intermediate outputs. Our evaluation shows that small LLMs are capable of serving as effective reward models or code evaluation critics, successfully identifying correct solutions among multiple candidates. Using this critic, we achieve over a 20% improvement in the search capability of the most accurate code out of multiple generations.

Method: Systematic comparison of leading Sci-LLMs on biological reasoning tasks using three input modes: sequence-only, context-only, and combination of both.

Result: Context-only approach consistently and substantially outperforms all other modes. Including raw sequences alongside high-level context degrades performance, indicating sequences act as informational noise even for specialized models.

Conclusion: Sci-LLMs should be reframed as powerful reasoning engines over expert knowledge rather than sequence decoders, shifting focus from direct sequence interpretation to high-level knowledge synthesis for hybrid scientific AI agents.

Abstract: Scientific Large Language Models (Sci-LLMs) have emerged as a promising frontier for accelerating biological discovery. However, these models face a fundamental challenge when processing raw biomolecular sequences: the tokenization dilemma. Whether treating sequences as a specialized language, risking the loss of functional motif information, or as a separate modality, introducing formidable alignment challenges, current strategies fundamentally limit their reasoning capacity. We challenge this sequence-centric paradigm by positing that a more effective strategy is to provide Sci-LLMs with high-level structured context derived from established bioinformatics tools, thereby bypassing the need to interpret low-level noisy sequence data directly. Through a systematic comparison of leading Sci-LLMs on biological reasoning tasks, we tested three input modes: sequence-only, context-only, and a combination of both. Our findings are striking: the context-only approach consistently and substantially outperforms all other modes. Even more revealing, the inclusion of the raw sequence alongside its high-level context consistently degrades performance, indicating that raw sequences act as informational noise, even for models with specialized tokenization schemes. These results suggest that the primary strength of existing Sci-LLMs lies not in their nascent ability to interpret biomolecular syntax from scratch, but in their profound capacity for reasoning over structured, human-readable knowledge. Therefore, we argue for reframing Sci-LLMs not as sequence decoders, but as powerful reasoning engines over expert knowledge. This work lays the foundation for a new class of hybrid scientific AI agents, repositioning the developmental focus from direct sequence interpretation towards high-level knowledge synthesis. The code is available at github.com/opendatalab-raise-dev/CoKE.

Method: Extracts score functions from foundation models to evaluate states based on human intentions, then uses these scores to re-weight rewards in skill discovery algorithms.

Result: Successfully eliminates undesirable behaviors like flipping or rolling, avoids hazardous areas in both state-based and pixel-based tasks, and discovers skills with behaviors difficult to define.

Conclusion: FoG effectively incorporates human intentions into skill discovery through foundation models, enabling safer and more desirable skill learning while maintaining diversity.

Abstract: Learning diverse skills without hand-crafted reward functions could accelerate reinforcement learning in downstream tasks. However, existing skill discovery methods focus solely on maximizing the diversity of skills without considering human preferences, which leads to undesirable behaviors and possibly dangerous skills. For instance, a cheetah robot trained using previous methods learns to roll in all directions to maximize skill diversity, whereas we would prefer it to run without flipping or entering hazardous areas. In this work, we propose a Foundation model Guided (FoG) skill discovery method, which incorporates human intentions into skill discovery through foundation models. Specifically, FoG extracts a score function from foundation models to evaluate states based on human intentions, assigning higher values to desirable states and lower to undesirable ones. These scores are then used to re-weight the rewards of skill discovery algorithms. By optimizing the re-weighted skill discovery rewards, FoG successfully learns to eliminate undesirable behaviors, such as flipping or rolling, and to avoid hazardous areas in both state-based and pixel-based tasks. Interestingly, we show that FoG can discover skills involving behaviors that are difficult to define. Interactive visualisations are available from https://sites.google.com/view/submission-fog.

Method: AUPO modifies MCTS decision policy using reward distribution statistics collected during search to automatically abstract actions, detecting symmetric actions even when resulting states are far apart.

Result: AUPO clearly outperforms MCTS on IPPC benchmark problems and can detect symmetric actions that state-of-the-art frameworks like ASAP struggle with.

Conclusion: AUPO provides an effective automatic action abstraction method that complements other tree search techniques and handles symmetry detection better than existing approaches.

Abstract: We introduce a novel, drop-in modification to Monte Carlo Tree Search’s (MCTS) decision policy that we call AUPO. Comparisons based on a range of IPPC benchmark problems show that AUPO clearly outperforms MCTS. AUPO is an automatic action abstraction algorithm that solely relies on reward distribution statistics acquired during the MCTS. Thus, unlike other automatic abstraction algorithms, AUPO requires neither access to transition probabilities nor does AUPO require a directed acyclic search graph to build its abstraction, allowing AUPO to detect symmetric actions that state-of-the-art frameworks like ASAP struggle with when the resulting symmetric states are far apart in state space. Furthermore, as AUPO only affects the decision policy, it is not mutually exclusive with other abstraction techniques that only affect the tree search.

Method: Sample-efficient DRL method that improves value-based DRL by leveraging pre-collected data and increasing network plasticity, tailored for industrial game development.

Result: Goalkeeper agent in EA SPORTS FC 25 outperforms built-in AI by 10% in ball saving rate, trains 50% faster than standard DRL, and creates more human-like gameplay according to domain experts.

Conclusion: The method successfully addresses industry needs for efficient DRL training and is intended to replace hand-crafted AI in future game series iterations.

Abstract: While several high profile video games have served as testbeds for Deep Reinforcement Learning (DRL), this technique has rarely been employed by the game industry for crafting authentic AI behaviors. Previous research focuses on training super-human agents with large models, which is impractical for game studios with limited resources aiming for human-like agents. This paper proposes a sample-efficient DRL method tailored for training and fine-tuning agents in industrial settings such as the video game industry. Our method improves sample efficiency of value-based DRL by leveraging pre-collected data and increasing network plasticity. We evaluate our method training a goalkeeper agent in EA SPORTS FC 25, one of the best-selling football simulations today. Our agent outperforms the game’s built-in AI by 10% in ball saving rate. Ablation studies show that our method trains agents 50% faster compared to standard DRL methods. Finally, qualitative evaluation from domain experts indicates that our approach creates more human-like gameplay compared to hand-crafted agents. As a testimony of the impact of the approach, the method is intended to replace the hand-crafted counterpart in next iterations of the series.

Method: Uses block Krylov algorithm on heat equation structure to get basic solutions, combines them to generate temperature distributions satisfying physical constraints, then applies heat operators to determine heat source distributions.

Result: 420x speedup in generating thermal simulation data for 5000 chips; data-driven approaches trained on BlocKOA data achieve comparable performance using only 4% of generation time compared to existing methods.

Conclusion: BlocKOA efficiently generates precise thermal simulation data with lower time complexity, enabling faster training of data-driven approaches for IC thermal analysis.

Abstract: Recent advances in data-driven approaches, such as neural operators (NOs), have shown substantial efficacy in reducing the solution time for integrated circuit (IC) thermal simulations. However, a limitation of these approaches is requiring a large amount of high-fidelity training data, such as chip parameters and temperature distributions, thereby incurring significant computational costs. To address this challenge, we propose a novel algorithm for the generation of IC thermal simulation data, named block Krylov and operator action (BlocKOA), which simultaneously accelerates the data generation process and enhances the precision of generated data. BlocKOA is specifically designed for IC applications. Initially, we use the block Krylov algorithm based on the structure of the heat equation to quickly obtain a few basic solutions. Then we combine them to get numerous temperature distributions that satisfy the physical constraints. Finally, we apply heat operators on these functions to determine the heat source distributions, efficiently generating precise data points. Theoretical analysis shows that the time complexity of BlocKOA is one order lower than the existing method. Experimental results further validate its efficiency, showing that BlocKOA achieves a 420-fold speedup in generating thermal simulation data for 5000 chips with varying physical parameters and IC structures. Even with just 4% of the generation time, data-driven approaches trained on the data generated by BlocKOA exhibits comparable performance to that using the existing method.

Method: Uses a single reinforcement learning agent trained on fixed-size circuits (m=8), with preprocessing (embedding or Gaussian striping) to handle matrices of different sizes, evaluated on circuits ranging from n=3 to 15.

Result: The method outperforms state-of-the-art algorithms, with performance improvement increasing as circuit size n grows larger.

Conclusion: The reinforcement learning approach provides an effective solution for CNOT minimization that scales well and demonstrates superior performance compared to existing methods, particularly for larger quantum circuits.

Abstract: CNOT gates are fundamental to quantum computing, as they facilitate entanglement, a crucial resource for quantum algorithms. Certain classes of quantum circuits are constructed exclusively from CNOT gates. Given their widespread use, it is imperative to minimise the number of CNOT gates employed. This problem, known as CNOT minimisation, remains an open challenge, with its computational complexity yet to be fully characterised. In this work, we introduce a novel reinforcement learning approach to address this task. Instead of training multiple reinforcement learning agents for different circuit sizes, we use a single agent up to a fixed size $m$. Matrices of sizes different from m are preprocessed using either embedding or Gaussian striping. To assess the efficacy of our approach, we trained an agent with m = 8, and evaluated it on matrices of size n that range from 3 to 15. The results we obtained show that our method overperforms the state-of-the-art algorithm as the value of n increases.

Method: Uses Bayesian reference resolution with Rational Speech Act (RSA) framework to plan message sequences that optimize belief alignment, adapting message specificity and timing based on user and scenario.

Result: Shows effectiveness depends on combining multi-step planning with realistic user awareness models, outperforming baseline methods.

Conclusion: Establishes theoretical foundations for pragmatic communication in human-agent teams, demonstrating how cognitive science insights can inform assistive agent design.

Abstract: Adaptive agent design offers a way to improve human-AI collaboration on time-sensitive tasks in rapidly changing environments. In such cases, to ensure the human maintains an accurate understanding of critical task elements, an assistive agent must not only identify the highest priority information but also estimate how and when this information can be communicated most effectively, given that human attention represents a zero-sum cognitive resource where focus on one message diminishes awareness of other or upcoming information. We introduce a theoretical framework for adaptive signalling which meets these challenges by using principles of rational communication, formalised as Bayesian reference resolution using the Rational Speech Act (RSA) modelling framework, to plan a sequence of messages which optimise timely alignment between user belief and a dynamic environment. The agent adapts message specificity and timing to the particulars of a user and scenario based on projections of how prior-guided interpretation of messages will influence attention to the interface and subsequent belief update, across several timesteps out to a fixed horizon. In a comparison to baseline methods, we show that this effectiveness depends crucially on combining multi-step planning with a realistic model of user awareness. As the first application of RSA for communication in a dynamic environment, and for human-AI interaction in general, we establish theoretical foundations for pragmatic communication in human-agent teams, highlighting how insights from cognitive science can be capitalised to inform the design of assistive agents.

Method: The method uses fuzzy logic reasoning to perform opinion mining from statements at a deeper level of granularity, extracting specific attributes and components that have been commented on.

Result: The approach enables classification of opinions into finer granularity levels (positive, negative, neutral) and subsequent ranking of entities based on this detailed analysis.

Conclusion: The proposed method successfully addresses the gap in existing research by combining fine-grained opinion mining using fuzzy logic with entity ranking, providing a more detailed analysis of opinions from online sources.

Abstract: Opinions are central to almost all human activities and are key influencers of our behaviors. In current times due to growth of social networking website and increase in number of e-commerce site huge amount of opinions are now available on web. Given a set of evaluative statements that contain opinions (or sentiments) about an Entity, opinion mining aims to extract attributes and components of the object that have been commented on in each statement and to determine whether the comments are positive, negative or neutral. While lot of research recently has been done in field of opinion mining and some of it dealing with ranking of entities based on review or opinion set, classifying opinions into finer granularity level and then ranking entities has never been done before. In this paper method for opinion mining from statements at a deeper level of granularity is proposed. This is done by using fuzzy logic reasoning, after which entities are ranked as per this information.

Method: Built uniform series embeddings for heterogeneous data, used two attention mechanisms for inter-variable and temporal dependencies, implemented variable-aware fusion for adaptive prediction, and devised zero-inflated projection module with joint classification-regression optimization.

Result: Offline evaluation on eight datasets showed superiority over baselines and cross-scenario generality. Online A/B tests on Taobao platform increased GMV by 4.65% and ROI by 2.44%.

Conclusion: Bid2X successfully demonstrates the viability of bidding foundation models in computational advertising, providing a unified approach that generalizes across different bidding scenarios while significantly improving key performance metrics.

Abstract: Auto-bidding is crucial in facilitating online advertising by automatically providing bids for advertisers. While previous work has made great efforts to model bidding environments for better ad performance, it has limitations in generalizability across environments since these models are typically tailored for specific bidding scenarios. To this end, we approach the scenario-independent principles through a unified function that estimates the achieved effect under specific bids, such as budget consumption, gross merchandise volume (GMV), page views, etc. Then, we propose a bidding foundation model Bid2X to learn this fundamental function from data in various scenarios. Our Bid2X is built over uniform series embeddings that encode heterogeneous data through tailored embedding methods. To capture complex inter-variable and dynamic temporal dependencies in bidding data, we propose two attention mechanisms separately treating embeddings of different variables and embeddings at different times as attention tokens for representation learning. On top of the learned variable and temporal representations, a variable-aware fusion module is used to perform adaptive bidding outcome prediction. To model the unique bidding data distribution, we devise a zero-inflated projection module to incorporate the estimated non-zero probability into its value prediction, which makes up a joint optimization objective containing classification and regression. The objective is proven to converge to the zero-inflated distribution. Our model has been deployed on the ad platform in Taobao, one of the world’s largest e-commerce platforms. Offline evaluation on eight datasets exhibits Bid2X’s superiority compared to various baselines and its generality across different scenarios. Bid2X increased GMV by 4.65% and ROI by 2.44% in online A/B tests, paving the way for bidding foundation model in computational advertising.

Method: Proposed framework integrates causal reasoning into DQNs using PEACE (Probabilistic Easy vAriational Causal Effect) formula for estimating causal effects during training.

Result: Experimental results show the approach outperforms conventional DQNs on standard benchmark environments, enhancing problem-solving capabilities without compromising performance.

Conclusion: The work presents a promising avenue for advancing deep reinforcement learning agents through principled causal inference.

Abstract: Deep Q Networks (DQN) have shown remarkable success in various reinforcement learning tasks. However, their reliance on associative learning often leads to the acquisition of spurious correlations, hindering their problem-solving capabilities. In this paper, we introduce a novel approach to integrate causal principles into DQNs, leveraging the PEACE (Probabilistic Easy vAriational Causal Effect) formula for estimating causal effects. By incorporating causal reasoning during training, our proposed framework enhances the DQN’s understanding of the underlying causal structure of the environment, thereby mitigating the influence of confounding factors and spurious correlations. We demonstrate that integrating DQNs with causal capabilities significantly enhances their problem-solving capabilities without compromising performance. Experimental results on standard benchmark environments showcase that our approach outperforms conventional DQNs, highlighting the effectiveness of causal reasoning in reinforcement learning. Overall, our work presents a promising avenue for advancing the capabilities of deep reinforcement learning agents through principled causal inference.

Method: Critical analysis of Swiss Cheese models, discussion of epistemic indifference, and distinction between risk and uncertainty with focus on option uncertainty and state-space uncertainty.

Result: Demonstrates that probability of AI disaster P(D) is higher than initially suggested due to structural relationships between risk layers and the impact of different uncertainty types.

Conclusion: Incorporating option and state-space uncertainty dimensions provides better understanding of AI existential risk likelihood and improves qualitative risk assessment frameworks.

Abstract: This work is a commentary of the article \href{https://doi.org/10.18716/ojs/phai/2025.2801}{AI Survival Stories: a Taxonomic Analysis of AI Existential Risk} by Cappelen, Goldstein, and Hawthorne. It is not just a commentary though, but a useful reminder of the philosophical limitations of \say{linear} models of risk. The article will focus on the model employed by the authors: first, I discuss some differences between standard Swiss Cheese models and this one. I then argue that in a situation of epistemic indifference the probability of P(D) is higher than what one might first suggest, given the structural relationships between layers. I then distinguish between risk and uncertainty, and argue that any estimation of P(D) is structurally affected by two kinds of uncertainty: option uncertainty and state-space uncertainty. Incorporating these dimensions of uncertainty into our qualitative discussion on AI existential risk can provide a better understanding of the likeliness of P(D).

Method: Six-stage reasoning framework using Google Gemini 2.0 Flash: context interpretation, user validation, data classification, business purpose test, compliance mapping, and risk synthesis with early hard policy gates and deny by default.

Result: Exact Decision Match improved from 10/14 to 13/14 (92.9%), DENY recall rose to 1.00, False Approval Rate dropped to 0, and Functional Appropriateness and Compliance Adherence at 14/14. Median latency under one minute.

Conclusion: Policy constrained LLM reasoning with explicit gates and audit trails can translate human readable policies into safe, compliant, and traceable machine decisions.

Abstract: Enterprises need access decisions that satisfy least privilege, comply with regulations, and remain auditable. We present a policy aware controller that uses a large language model (LLM) to interpret natural language requests against written policies and metadata, not raw data. The system, implemented with Google Gemini~2.0 Flash, executes a six-stage reasoning framework (context interpretation, user validation, data classification, business purpose test, compliance mapping, and risk synthesis) with early hard policy gates and deny by default. It returns APPROVE, DENY, CONDITIONAL together with cited controls and a machine readable rationale. We evaluate on fourteen canonical cases across seven scenario families using a privacy preserving benchmark. Results show Exact Decision Match improving from 10/14 to 13/14 (92.9%) after applying policy gates, DENY recall rising to 1.00, False Approval Rate on must-deny families dropping to 0, and Functional Appropriateness and Compliance Adherence at 14/14. Expert ratings of rationale quality are high, and median latency is under one minute. These findings indicate that policy constrained LLM reasoning, combined with explicit gates and audit trails, can translate human readable policies into safe, compliant, and traceable machine decisions.

Method: Developed collaborative prediction sets with two-threshold structure based on conformal prediction, with offline and online calibration algorithms that provide distribution-free finite sample guarantees and adapt to distribution shifts including Human to AI Adaptation.

Result: Experiments across image classification, regression, and medical decision making show collaborative prediction sets consistently outperform either agent alone, achieving higher coverage and smaller set sizes.

Conclusion: The framework successfully formalizes human-AI collaboration in uncertainty quantification, providing practical algorithms with theoretical guarantees that demonstrate superior performance over individual human or AI decision making.

Abstract: AI predictive systems are increasingly embedded in decision making pipelines, shaping high stakes choices once made solely by humans. Yet robust decisions under uncertainty still rely on capabilities that current AI lacks: domain knowledge not captured by data, long horizon context, and reasoning grounded in the physical world. This gap has motivated growing efforts to design collaborative frameworks that combine the complementary strengths of humans and AI. This work advances this vision by identifying the fundamental principles of Human AI collaboration within uncertainty quantification, a key component of reliable decision making. We introduce Human AI Collaborative Uncertainty Quantification, a framework that formalizes how an AI model can refine a human expert’s proposed prediction set with two goals: avoiding counterfactual harm, ensuring the AI does not degrade correct human judgments, and complementarity, enabling recovery of correct outcomes the human missed. At the population level, we show that the optimal collaborative prediction set follows an intuitive two threshold structure over a single score function, extending a classical result in conformal prediction. Building on this insight, we develop practical offline and online calibration algorithms with provable distribution free finite sample guarantees. The online method adapts to distribution shifts, including human behavior evolving through interaction with AI, a phenomenon we call Human to AI Adaptation. Experiments across image classification, regression, and text based medical decision making show that collaborative prediction sets consistently outperform either agent alone, achieving higher coverage and smaller set sizes across various conditions.

Method: Mapping AI agent memory architectures to corresponding automata classes: simple reflex agents to Finite Automata, hierarchical task-decomposition agents to Pushdown Automata, and reflective memory agents to Turing Machines.

Result: Developed an Automata-Agent Framework that classifies agents and delineates boundaries between verifiable and fundamentally undecidable systems, with extensions to probabilistic automata for LLM-based agents.

Conclusion: The framework enables formal verification, safety guarantees, and quantitative risk analysis for AI agents, with an agenda for developing static analysis tools and grammars for agentic frameworks.

Abstract: This paper establishes a formal equivalence between the architectural classes of modern agentic AI systems and the abstract machines of the Chomsky hierarchy. We posit that the memory architecture of an AI agent is the definitive feature determining its computational power and that it directly maps it to a corresponding class of automaton. Specifically, we demonstrate that simple reflex agents are equivalent to Finite Automata, hierarchical task-decomposition agents are equivalent to Pushdown Automata, and agents employing readable/writable memory for reflection are equivalent to TMs. This Automata-Agent Framework provides a principled methodology for right-sizing agent architectures to optimize computational efficiency and cost. More critically, it creates a direct pathway to formal verification, enables the application of mature techniques from automata theory to guarantee agent safety and predictability. By classifying agents, we can formally delineate the boundary between verifiable systems and those whose behavior is fundamentally undecidable. We address the inherent probabilistic nature of LLM-based agents by extending the framework to probabilistic automata that allow quantitative risk analysis. The paper concludes by outlining an agenda for developing static analysis tools and grammars for agentic frameworks.

Method: Uses Emotional Rationale Verifier (ERV) and Explanation Reward to guide models to produce reasoning consistent with target emotions during multimodal emotion recognition, without model modifications or additional annotations.

Result: Significantly improves faithful explanation-prediction consistency and explanation emotion accuracy on MAFW and DFEW datasets through extensive experiments and human evaluations.

Conclusion: The approach enhances alignment between explanation and prediction, empowering MLLMs to deliver emotionally coherent, trustworthy interactions - a key step toward truly human-like HCI systems.

Abstract: The recent advancement of Multimodal Large Language Models (MLLMs) is transforming human-computer interaction (HCI) from surface-level exchanges into more nuanced and emotionally intelligent communication. To realize this shift, emotion understanding becomes essential allowing systems to capture subtle cues underlying user intent. Furthermore, providing faithful explanations for predicted emotions is crucial to ensure interpretability and build user trust. However, current MLLM-based methods often generate emotion explanations that diverge from the target labels and sometimes even contradict their own predicted emotions. This inconsistency poses a critical risk for misunderstanding and erodes reliability in interactive settings. To address this, we propose a novel approach: the Emotional Rationale Verifier (ERV) and an Explanation Reward. Our method guides the model to produce reasoning that is explicitly consistent with the target emotion during multimodal emotion recognition without modifying the model architecture or requiring additional paired video-description annotations. Our method significantly improves faithful explanation-prediction consistency and explanation emotion accuracy on the MAFW and DFEW datasets. Through extensive experiments and human evaluations, we show that our approach not only enhances alignment between explanation and prediction but also empowers MLLMs to deliver emotionally coherent, trustworthy interactions, marking a key step toward truly human-like HCI systems.

Method: Introduces HAI framework with incremental learning, carbon-aware optimization, human-in-the-loop collaboration, dynamic architectures, and biological cognition parallels for continuous contextual learning.

Result: Proposes theoretical foundations and system design that enable AI to learn continuously while minimizing carbon footprints and human annotation costs.

Conclusion: HAI offers a pathway to responsible, human-centered AI by addressing challenges in active learning, continual adaptation, and energy-efficient deployment.

Abstract: The rapid advancement of Artificial Intelligence (AI) has led to unprecedented computational demands, raising significant environmental and ethical concerns. This paper critiques the prevailing reliance on large-scale, static datasets and monolithic training paradigms, advocating for a shift toward human-inspired, sustainable AI solutions. We introduce a novel framework, Human AI (HAI), which emphasizes incremental learning, carbon-aware optimization, and human-in-the-loop collaboration to enhance adaptability, efficiency, and accountability. By drawing parallels with biological cognition and leveraging dynamic architectures, HAI seeks to balance performance with ecological responsibility. We detail the theoretical foundations, system design, and operational principles that enable AI to learn continuously and contextually while minimizing carbon footprints and human annotation costs. Our approach addresses pressing challenges in active learning, continual adaptation, and energy-efficient model deployment, offering a pathway toward responsible, human-centered artificial intelligence.

Method: The authors propose NoRA, a new benchmark that requires models to go beyond path-based reasoning and includes several levels of difficulty to better evaluate systematic relational reasoning capabilities.

Result: NoRA provides a more challenging and comprehensive evaluation framework that can better assess the true systematic reasoning abilities of neural network models beyond simple path composition.

Conclusion: NoRA will support further progress in systematic relational reasoning by providing a more realistic benchmark that requires models to perform reasoning beyond path-based approaches.

Abstract: Designing models that can learn to reason in a systematic way is an important and long-standing challenge. In recent years, a wide range of solutions have been proposed for the specific case of systematic relational reasoning, including Neuro-Symbolic approaches, variants of the Transformer architecture, and specialised Graph Neural Networks. However, existing benchmarks for systematic relational reasoning focus on an overly simplified setting, based on the assumption that reasoning can be reduced to composing relational paths. In fact, this assumption is hard-baked into the architecture of several recent models, leading to approaches that can perform well on existing benchmarks but are difficult to generalise to other settings. To support further progress in the field of systematic relational reasoning with neural networks, we introduce NoRA, a new benchmark which adds several levels of difficulty and requires models to go beyond path-based reasoning.

Method: Developed a synthesis toolkit leveraging reciprocal synergies between data modalities to create JanusCode-800K corpus, then trained JanusCoder and JanusCoderV models as unified visual-programmatic interfaces.

Result: JanusCoder series (7B to 14B scale) demonstrates superior performance on text-centric and vision-centric coding tasks, approaching or exceeding commercial model performance.

Conclusion: The work provides key insights into harmonizing programmatic logic with visual expression and establishes an effective unified approach for multimodal code generation.

Abstract: The scope of neural code intelligence is rapidly expanding beyond text-based source code to encompass the rich visual outputs that programs generate. This visual dimension is critical for advanced applications like flexible content generation and precise, program-driven editing of visualizations. However, progress has been impeded by the scarcity of high-quality multimodal code data, a bottleneck stemming from challenges in synthesis and quality assessment. To address these challenges, we make contributions from both a data and modeling perspective. We first introduce a complete synthesis toolkit that leverages reciprocal synergies between data modalities to efficiently produce a large-scale, high-quality corpus spanning from standard charts to complex interactive web UIs and code-driven animations. Leveraging this toolkit, we construct JanusCode-800K, the largest multimodal code corpus to date. This powers the training of our models, JanusCoder and JanusCoderV, which establish a visual-programmatic interface for generating code from textual instructions, visual inputs, or a combination of both. Our unified model is a departure from existing approaches that build specialized models for isolated tasks. Extensive experiments on both text-centric and vision-centric coding tasks demonstrate the superior performance of the JanusCoder series, with our 7B to 14B scale models approaching or even exceeding the performance of commercial models. Furthermore, extensive analysis provides key insights into harmonizing programmatic logic with its visual expression. Our code and checkpoints will are available at https://github.com/InternLM/JanusCoder.

Method: Developed OntoPret using cognitive science foundations and modular engineering methodology to create a formal, machine-processable framework for classifying behaviors including task deviations and deceptive actions.

Result: Demonstrated OntoPret’s adaptability across manufacturing and gameplay use cases, establishing semantic foundations for advanced reasoning about human intentions.

Conclusion: OntoPret provides a viable solution for enabling machines to safely and effectively interpret complex human behaviors in collaborative human-machine teaming scenarios.

Abstract: As human machine teaming becomes central to paradigms like Industry 5.0, a critical need arises for machines to safely and effectively interpret complex human behaviors. A research gap currently exists between techno centric robotic frameworks, which often lack nuanced models of human behavior, and descriptive behavioral ontologies, which are not designed for real time, collaborative interpretation. This paper addresses this gap by presenting OntoPret, an ontology for the interpretation of human behavior. Grounded in cognitive science and a modular engineering methodology, OntoPret provides a formal, machine processable framework for classifying behaviors, including task deviations and deceptive actions. We demonstrate its adaptability across two distinct use cases manufacturing and gameplay and establish the semantic foundations necessary for advanced reasoning about human intentions.

Method: ReCode treats high-level plans as abstract placeholder functions and recursively decomposes them into finer-grained sub-functions until reaching primitive actions, using a unified code representation.

Result: Extensive experiments show ReCode significantly surpasses advanced baselines in inference performance and demonstrates exceptional data efficiency in training.

Conclusion: Unifying planning and action through recursive code generation is a powerful and effective approach to achieving universal granularity control in AI agents.

Abstract: Real-world tasks require decisions at varying granularities, and humans excel at this by leveraging a unified cognitive representation where planning is fundamentally understood as a high-level form of action. However, current Large Language Model (LLM)-based agents lack this crucial capability to operate fluidly across decision granularities. This limitation stems from existing paradigms that enforce a rigid separation between high-level planning and low-level action, which impairs dynamic adaptability and limits generalization. We propose ReCode (Recursive Code Generation), a novel paradigm that addresses this limitation by unifying planning and action within a single code representation. In this representation, ReCode treats high-level plans as abstract placeholder functions, which the agent then recursively decomposes into finer-grained sub-functions until reaching primitive actions. This recursive approach dissolves the rigid boundary between plan and action, enabling the agent to dynamically control its decision granularity. Furthermore, the recursive structure inherently generates rich, multi-granularity training data, enabling models to learn hierarchical decision-making processes. Extensive experiments show ReCode significantly surpasses advanced baselines in inference performance and demonstrates exceptional data efficiency in training, validating our core insight that unifying planning and action through recursive code generation is a powerful and effective approach to achieving universal granularity control. The code is available at https://github.com/FoundationAgents/ReCode.

Method: Large-scale two-wave survey study (n_wave1=1514, n_wave2=1488) representative of the Swiss population.

Result: GenAI boom significantly associated with reduced AI acceptance (23% to 30% finding AI ’not acceptable at all’) and increased demand for human oversight (support for human-only decision-making rose from 18% to 26%), while widening educational, linguistic, and gender gaps.

Conclusion: Findings challenge industry assumptions about public readiness for AI deployment and highlight the importance of aligning technological development with evolving public preferences.

Abstract: The rapid adoption of generative artificial intelligence (GenAI) technologies has led many organizations to integrate AI into their products and services, often without considering user preferences. Yet, public attitudes toward AI use, especially in impactful decision-making scenarios, are underexplored. Using a large-scale two-wave survey study (n_wave1=1514, n_wave2=1488) representative of the Swiss population, we examine shifts in public attitudes toward AI before and after the launch of ChatGPT. We find that the GenAI boom is significantly associated with reduced public acceptance of AI (see Figure 1) and increased demand for human oversight in various decision-making contexts. The proportion of respondents finding AI “not acceptable at all” increased from 23% to 30%, while support for human-only decision-making rose from 18% to 26%. These shifts have amplified existing social inequalities in terms of widened educational, linguistic, and gender gaps post-boom. Our findings challenge industry assumptions about public readiness for AI deployment and highlight the critical importance of aligning technological development with evolving public preferences.

Method: Proposes Multi-Agent Evolve (MAE) framework with three agents: Proposer generates questions, Solver provides solutions, and Judge evaluates both. Uses reinforcement learning to optimize agent behaviors through co-evolution.

Result: Experiments on Qwen2.5-3B-Instruct show average improvement of 4.54% across multiple benchmarks, demonstrating enhanced reasoning capabilities.

Conclusion: MAE is a scalable, data-efficient method that significantly improves LLM reasoning abilities with minimal human supervision, applicable to diverse domains.

Abstract: Reinforcement Learning (RL) has demonstrated significant potential in enhancing the reasoning capabilities of large language models (LLMs). However, the success of RL for LLMs heavily relies on human-curated datasets and verifiable rewards, which limit their scalability and generality. Recent Self-Play RL methods, inspired by the success of the paradigm in games and Go, aim to enhance LLM reasoning capabilities without human-annotated data. However, their methods primarily depend on a grounded environment for feedback (e.g., a Python interpreter or a game engine); extending them to general domains remains challenging. To address these challenges, we propose Multi-Agent Evolve (MAE), a framework that enables LLMs to self-evolve in solving diverse tasks, including mathematics, reasoning, and general knowledge Q&A. The core design of MAE is based on a triplet of interacting agents (Proposer, Solver, Judge) that are instantiated from a single LLM, and applies reinforcement learning to optimize their behaviors. The Proposer generates questions, the Solver attempts solutions, and the Judge evaluates both while co-evolving. Experiments on Qwen2.5-3B-Instruct demonstrate that MAE achieves an average improvement of 4.54% on multiple benchmarks. These results highlight MAE as a scalable, data-efficient method for enhancing the general reasoning abilities of LLMs with minimal reliance on human-curated supervision.

Method: The framework executes domain tasks, synthesizes candidate MCPs from successful trajectories, abstracts them to parameterized primitives, consolidates them into an MCP Box, and performs retrieval-augmented MCP selection at inference.

Result: ALITA-G achieves state-of-the-art results: 83.03% pass@1 and 89.09% pass@3 on GAIA validation, while reducing computation costs by approximately 15% in tokens per example.

Conclusion: ALITA-G provides a principled pathway from generalist capability to reusable domain-specific competence, improving both accuracy and efficiency on complex reasoning tasks.

Abstract: Large language models (LLMs) have been shown to perform better when scaffolded into agents with memory, tools, and feedback. Beyond this, self-evolving agents have emerged, but current work largely limits adaptation to prompt rewriting or failure retries. Therefore, we present ALITA-G, a self-evolution framework that transforms a general-purpose agent into a domain expert by systematically generating, abstracting, and curating Model Context Protocol (MCP) tools. In this framework, a generalist agent executes a curated suite of target-domain tasks and synthesizes candidate MCPs from successful trajectories. These are then abstracted to parameterized primitives and consolidated into an MCP Box. At inference time, ALITA-G performs retrieval-augmented MCP selection with the help of each tool’s descriptions and use cases, before executing an agent equipped with the MCP Executor. Across several benchmarks GAIA, PathVQA, and Humanity’s Last Exam, ALITA-G attains strong gains while reducing computation costs. On GAIA validation, it achieves 83.03% pass@1 and 89.09% pass@3, establishing a new state-of-the-art result while reducing mean tokens per example by approximately 15% relative to a strong baseline agent. ALITA-G thus provides a principled pathway from generalist capability to reusable, domain-specific competence, improving both accuracy and efficiency on complex reasoning tasks.

Method: A novel approximation strategy that freezes slow states during lower-level planning phases and applies value iteration to an auxiliary upper-level MDP on a slower timescale. This creates easier-to-solve lower-level problems and allows for more favorable discount factors.

Result: Theoretical analysis of regret incurred by the frozen-state approach provides insights on trading off regret versus computational cost. Empirical evaluation on three domains (inventory control, gridworld with spatial tasks, dynamic pricing) shows the method produces high-quality policies with significantly less computation.

Conclusion: The frozen-state methods effectively handle fast-slow MDPs, producing good policies with reduced computation, and demonstrate that simply omitting slow states is often a poor heuristic.

Abstract: We study infinite horizon Markov decision processes (MDPs) with “fast-slow” structure, where some state variables evolve rapidly (“fast states”) while others change more gradually (“slow states”). This structure commonly arises in practice when decisions must be made at high frequencies over long horizons, and where slowly changing information still plays a critical role in determining optimal actions. Examples include inventory control under slowly changing demand indicators or dynamic pricing with gradually shifting consumer behavior. Modeling the problem at the natural decision frequency leads to MDPs with discount factors close to one, making them computationally challenging. We propose a novel approximation strategy that “freezes” slow states during phases of lower-level planning and subsequently applies value iteration to an auxiliary upper-level MDP that evolves on a slower timescale. Freezing states for short periods of time leads to easier-to-solve lower-level problems, while a slower upper-level timescale allows for a more favorable discount factor. On the theoretical side, we analyze the regret incurred by our frozen-state approach, which leads to simple insights on how to trade off regret versus computational cost. Empirically, we benchmark our new frozen-state methods on three domains, (i) inventory control with fixed order costs, (ii) a gridworld problem with spatial tasks, and (iii) dynamic pricing with reference-price effects. We demonstrate that the new methods produce high-quality policies with significantly less computation, and we show that simply omitting slow states is often a poor heuristic.

Method: The authors derive deterministic relationships between approximated and optimal solutions for discrete POMDPs, creating bounds that can be attached to existing algorithms with certain structures.

Result: The derived bounds provide deterministic guarantees for solution quality with marginal computational overhead, and decision-making based on these guarantees can yield superior performance compared to original algorithms.

Conclusion: The proposed approach enables deterministic certification of POMDP solutions and can enhance algorithm performance while maintaining computational efficiency.

Abstract: Decision-making under uncertainty is a critical aspect of many practical autonomous systems due to incomplete information. Partially Observable Markov Decision Processes (POMDPs) offer a mathematically principled framework for formulating decision-making problems under such conditions. However, finding an optimal solution for a POMDP is generally intractable. In recent years, there has been a significant progress of scaling approximate solvers from small to moderately sized problems, using online tree search solvers. Often, such approximate solvers are limited to probabilistic or asymptotic guarantees towards the optimal solution. In this paper, we derive a deterministic relationship for discrete POMDPs between an approximated and the optimal solution. We show that at any time, we can derive bounds that relate between the existing solution and the optimal one. We show that our derivations provide an avenue for a new set of algorithms and can be attached to existing algorithms that have a certain structure to provide them with deterministic guarantees with marginal computational overhead. In return, not only do we certify the solution quality, but we demonstrate that making a decision based on the deterministic guarantee may result in superior performance compared to the original algorithm without the deterministic certification.

Method: 1) Created GraphInstruct benchmark with 21 graph reasoning tasks, diverse generation pipelines, and detailed reasoning steps. 2) Developed GraphSolver via instruction-tuning. 3) Proposed label-mask training strategy for GraphSolver+ to enhance multi-step reasoning by emphasizing node-identification signals.

Result: GraphSolver and GraphSolver+ demonstrate superior graph understanding capability compared to other open-sourced LLMs, with extensive experiments showing their superiority.

Conclusion: GraphInstruct facilitates research on applying LLMs to graph-structured data, and the proposed models significantly enhance graph understanding and reasoning abilities in LLMs.

Abstract: Improving the general capabilities of large language models (LLMs) is an active research topic. As a common data structure in many real-world domains, understanding graph data is a crucial part of advancing general intelligence. To this end, we propose a dynamic benchmark named GraphInstruct in this paper, which comprehensively includes 21 classical graph reasoning tasks, providing diverse graph generation pipelines and detailed intermediate reasoning steps for each sample. Based on GraphInstruct, we develop GraphSolver via efficient instruction-tuning, which demonstrates prominent graph understanding capability compared to other open-sourced LLMs. To further endow LLMs with multi-step graph reasoning capability, we propose a label-mask training strategy and build GraphSolver+, which leverages masked supervision on intermediate reasoning tokens to emphasize crucial node-identification signals. As one of the pioneering efforts to enhance the graph understanding and reasoning abilities of LLMs, extensive experiments have demonstrated the superiority of GraphSolver and GraphSolver+ over other LLMs. We sincerely hope GraphInstruct will facilitate further research on applying LLMs to graph-structured data. Our code and data are released publicly at: https://github.com/CGCL-codes/GraphInstruct.

Method: System with strategic LLM-based agents in repeated interactions and an RL-based governing agent that dynamically adjusts information transparency by controlling what contextual/historical information each agent can access at each timestep.

Result: RL-based governance significantly enhances cooperation compared to static information-sharing baselines, while preserving agent autonomy without requiring direct structural interventions or payoff modifications.

Conclusion: Adaptive information governance through RL-based transparency modulation provides an effective approach for promoting cooperation in complex multi-agent sociotechnical systems with autonomous LLM agents.

Abstract: Modern engineered systems increasingly involve complex sociotechnical environments where multiple agents, including humans and the emerging paradigm of agentic AI powered by large language models, must navigate social dilemmas that pit individual interests against collective welfare. As engineered systems evolve toward multi-agent architectures with autonomous LLM-based agents, traditional governance approaches using static rules or fixed network structures fail to address the dynamic uncertainties inherent in real-world operations. This paper presents a novel framework that integrates adaptive governance mechanisms directly into the design of sociotechnical systems through a unique separation of agent interaction networks from information flow networks. We introduce a system comprising strategic LLM-based system agents that engage in repeated interactions and a reinforcement learning-based governing agent that dynamically modulates information transparency. Unlike conventional approaches that require direct structural interventions or payoff modifications, our framework preserves agent autonomy while promoting cooperation through adaptive information governance. The governing agent learns to strategically adjust information disclosure at each timestep, determining what contextual or historical information each system agent can access. Experimental results demonstrate that this RL-based governance significantly enhances cooperation compared to static information-sharing baselines.

Method: Fine-tuning algorithm that incorporates optimal solutions into search procedures to generate high-quality search traces, then distills improved search strategies into the model.

Result: Significantly enhanced search capabilities on arithmetic reasoning and code self-repair tasks including Countdown, CodeContests, and CodeForces.

Conclusion: Guided-ReST effectively improves language models’ search strategies by using optimal solutions as guidance, leading to better performance on complex reasoning tasks.

Abstract: While language models have shown remarkable performance across diverse tasks, they still encounter challenges in complex reasoning scenarios. Recent research suggests that language models trained on linearized search traces toward solutions, rather than solely on the final solutions, exhibit improved generalization, despite the search traces being potentially noisy or suboptimal. However, relying on such imperfect traces can result in inefficient use of test-time compute. To address this, we propose guided reinforced self-training (Guided-ReST), a fine-tuning algorithm designed to improve the model’s capability for effective search during inference. The key insight behind Guided-ReST is that optimal solutions can serve as valuable step-by-step landmarks to guide the model’s search process. Based on this insight, we introduce a novel data generation method that seamlessly incorporates optimal solutions into the model’s search procedure, enabling the generation of high-quality search traces. By fine-tuning the model on these search traces, we effectively distill improved search strategies into the model. Our method significantly enhances the search capabilities of language models on arithmetic reasoning and code self-repair tasks, including Countdown, CodeContests, and CodeForces. We release the source code at https://github.com/snu-mllab/guided-rest.

Method: Proposes DANS with a two-way generator for diverse negative triplets and an adaptive mechanism that localizes the global generator for different entities/relations to produce fine-grained examples.

Result: Evaluated on three benchmark knowledge graphs, DANS demonstrates effectiveness through both quantitative and qualitative experiments.

Conclusion: DANS improves knowledge graph embedding by generating more informative negative triplets through enhanced diversity and adaptiveness in the sampling process.

Abstract: In knowledge graph embedding, aside from positive triplets (ie: facts in the knowledge graph), the negative triplets used for training also have a direct influence on the model performance. In reality, since knowledge graphs are sparse and incomplete, negative triplets often lack explicit labels, and thus they are often obtained from various sampling strategies (eg: randomly replacing an entity in a positive triplet). An ideal sampled negative triplet should be informative enough to help the model train better. However, existing methods often ignore diversity and adaptiveness in their sampling process, which harms the informativeness of negative triplets. As such, we propose a generative adversarial approach called Diversified and Adaptive Negative Sampling DANS on knowledge graphs. DANS is equipped with a two-way generator that generates more diverse negative triplets through two pathways, and an adaptive mechanism that produces more fine-grained examples by localizing the global generator for different entities and relations. On the one hand, the two-way generator increase the overall informativeness with more diverse negative examples; on the other hand, the adaptive mechanism increases the individual sample-wise informativeness with more fine-grained sampling. Finally, we evaluate the performance of DANS on three benchmark knowledge graphs to demonstrate its effectiveness through quantitative and qualitative experiments.

Method: Directly modify sampling trajectory using negation sets, formally derive relationship between safe and unsafe denoised samples to create safe denoiser that avoids specific data distribution regions.

Result: Successfully produces high-quality samples while avoiding negation areas in text-conditional, class-conditional, and unconditional image generation scenarios.

Conclusion: Training-free safe denoiser shows great potential for safer use of diffusion models by avoiding specific data regions without model retraining.

Abstract: There is growing concern over the safety of powerful diffusion models (DMs), as they are often misused to produce inappropriate, not-safe-for-work (NSFW) content or generate copyrighted material or data of individuals who wish to be forgotten. Many existing methods tackle these issues by heavily relying on text-based negative prompts or extensively retraining DMs to eliminate certain features or samples. In this paper, we take a radically different approach, directly modifying the sampling trajectory by leveraging a negation set (e.g., unsafe images, copyrighted data, or datapoints needed to be excluded) to avoid specific regions of data distribution, without needing to retrain or fine-tune DMs. We formally derive the relationship between the expected denoised samples that are safe and those that are not safe, leading to our $\textit{safe}$ denoiser which ensures its final samples are away from the area to be negated. Inspired by the derivation, we develop a practical algorithm that successfully produces high-quality samples while avoiding negation areas of the data distribution in text-conditional, class-conditional, and unconditional image generation scenarios. These results hint at the great potential of our training-free safe denoiser for using DMs more safely.

Method: Constructed a fact-checking dataset from Reddit discussions, categorizing retrieved evidence into supporting, misleading, and unrelated types to create realistic test scenarios.

Result: All tested LLM-powered RAG systems performed worse than their zero-shot baselines when exposed to misleading retrievals, while human annotators consistently performed better.

Conclusion: RAGuard highlights LLMs’ susceptibility to noisy environments and provides the first systematic benchmark for assessing RAG robustness against misleading evidence, driving future research toward more reliable real-world applications.

Abstract: Retrieval-augmented generation (RAG) has shown impressive capabilities in mitigating hallucinations in large language models (LLMs). However, LLMs struggle to maintain consistent reasoning when exposed to misleading or conflicting evidence, especially in real-world domains such as politics, where information is polarized or selectively framed. Mainstream RAG benchmarks evaluate models under clean retrieval settings, where systems generate answers from gold-standard documents, or under synthetically perturbed settings, where documents are artificially injected with noise. These assumptions fail to reflect real-world conditions, often leading to an overestimation of RAG system performance. To address this gap, we introduce RAGuard, the first benchmark to evaluate the robustness of RAG systems against misleading retrievals. Unlike prior benchmarks that rely on synthetic noise, our fact-checking dataset captures naturally occurring misinformation by constructing its retrieval corpus from Reddit discussions. It categorizes retrieved evidence into three types: supporting, misleading, and unrelated, providing a realistic and challenging testbed for assessing how well RAG systems navigate different types of evidence. Our experiments reveal that, when exposed to potentially misleading retrievals, all tested LLM-powered RAG systems perform worse than their zero-shot baselines (i.e., no retrieval at all), while human annotators consistently perform better, highlighting LLMs’ susceptibility to noisy environments. To our knowledge, RAGuard is the first benchmark to systematically assess the robustness of the RAG against misleading evidence. We expect this benchmark to drive future research toward improving RAG systems beyond idealized datasets, making them more reliable for real-world applications. The dataset is available at https://huggingface.co/datasets/UCSC-IRKM/RAGuard.

Method: Developed MAST through rigorous analysis of 150 traces with expert human annotators, creating a taxonomy with 14 failure modes clustered into 3 categories. Built an LLM-as-a-Judge pipeline for scalable annotation and analyzed failure patterns across models and tasks.

Result: Created MAST-Data with 1600+ annotated traces, achieved high inter-annotator agreement (kappa=0.88), identified 14 failure modes in 3 categories, and demonstrated improvement headrooms from better MAS design across different models and tasks.

Conclusion: The analysis reveals that identified failures require sophisticated solutions, providing a clear roadmap for future MAS research. The authors publicly release MAST-Data, MAST taxonomy, and LLM annotator to facilitate widespread MAS development.

Abstract: Despite enthusiasm for Multi-Agent LLM Systems (MAS), their performance gains on popular benchmarks are often minimal. This gap highlights a critical need for a principled understanding of why MAS fail. Addressing this question requires systematic identification and analysis of failure patterns. We introduce MAST-Data, a comprehensive dataset of 1600+ annotated traces collected across 7 popular MAS frameworks. MAST-Data is the first multi-agent system dataset to outline the failure dynamics in MAS for guiding the development of better future systems. To enable systematic classification of failures for MAST-Data, we build the first Multi-Agent System Failure Taxonomy (MAST). We develop MAST through rigorous analysis of 150 traces, guided closely by expert human annotators and validated by high inter-annotator agreement (kappa = 0.88). This process identifies 14 unique modes, clustered into 3 categories: (i) system design issues, (ii) inter-agent misalignment, and (iii) task verification. To enable scalable annotation, we develop an LLM-as-a-Judge pipeline with high agreement with human annotations. We leverage MAST and MAST-Data to analyze failure patterns across models (GPT4, Claude 3, Qwen2.5, CodeLlama) and tasks (coding, math, general agent), demonstrating improvement headrooms from better MAS design. Our analysis provides insights revealing that identified failures require more sophisticated solutions, highlighting a clear roadmap for future research. We publicly release our comprehensive dataset (MAST-Data), the MAST, and our LLM annotator to facilitate widespread research and development in MAS.

Method: Uses surrogate deep neural networks to model the relationship between attention head states and fairness/utility metrics, then applies randomized simulated annealing to efficiently identify optimal subsets of attention heads for pruning.

Result: Achieves up to 40% reduction in gender bias and outperforms state-of-the-art bias mitigation strategies.

Conclusion: Attention Pruning provides an effective post-processing approach to reduce bias in LLMs while minimizing impact on model utility.

Abstract: This paper explores pruning attention heads as a post-processing bias mitigation method for large language models (LLMs). Modern AI systems such as LLMs are expanding into sensitive social contexts where fairness concerns become especially crucial. Since LLMs develop decision-making patterns by training on massive datasets of human-generated content, they naturally encode and perpetuate societal biases. While modifying training datasets and algorithms is expensive and requires significant resources; post-processing techniques-such as selectively deactivating neurons and attention heads in pre-trained LLMs-can provide feasible and effective approaches to improve fairness. However, identifying the optimal subset of parameters to prune presents a combinatorial challenge within LLMs’ immense parameter space, requiring solutions that efficiently balance competing objectives across the frontiers of model fairness and utility. To address the computational challenges, we explore a search-based program repair approach via randomized simulated annealing. Given the prohibitive evaluation costs in billion-parameter LLMs, we develop surrogate deep neural networks that efficiently model the relationship between attention head states (active/inactive) and their corresponding fairness/utility metrics. This allows us to perform optimization over the surrogate models and efficiently identify optimal subsets of attention heads for selective pruning rather than directly searching through the LLM parameter space. This paper introduces Attention Pruning, a fairness-aware surrogate simulated annealing approach to prune attention heads in LLMs that disproportionately contribute to bias while minimally impacting overall model utility. Our experiments show that Attention Pruning achieves up to $40%$ reduction in gender bias and outperforms the state-of-the-art bias mitigation strategies.

Method: Systematic survey and in-depth analysis of current research, positioning LLMs as tools for formalizing and refining planning specifications to support off-the-shelf AP planners.

Result: Identifies methodologies, critical challenges, and future directions in integrating LLMs with automated planning systems.

Conclusion: Contributes to joint research on NLP and Automated Planning by providing comprehensive analysis and highlighting key research directions for reliable planning systems.

Abstract: Large Language Models (LLMs) excel in various natural language tasks but often struggle with long-horizon planning problems requiring structured reasoning. This limitation has drawn interest in integrating neuro-symbolic approaches within the Automated Planning (AP) and Natural Language Processing (NLP) communities. However, identifying optimal AP deployment frameworks can be daunting and introduces new challenges. This paper aims to provide a timely survey of the current research with an in-depth analysis, positioning LLMs as tools for formalizing and refining planning specifications to support reliable off-the-shelf AP planners. By systematically reviewing the current state of research, we highlight methodologies, and identify critical challenges and future directions, hoping to contribute to the joint research on NLP and Automated Planning.

Method: Formalized information-gathering as a constraint satisfaction problem with missing variables. Created QuestBench with four types of underspecified reasoning tasks requiring at most one question: Logic-Q (logical reasoning), Planning-Q (PDDL planning), GSM-Q (grade school math), and GSME-Q (equation-based math). Models must select correct clarification questions from multiple options.

Result: Current LLMs achieve 40-50% accuracy on Logic-Q and Planning-Q, but excel at GSM-Q and GSME-Q. Analysis shows solving well-specified problems doesn’t guarantee success on underspecified versions - models struggle to identify the right question even when they can solve the fully specified problem.

Conclusion: LLMs need specific optimization for information acquisition capabilities, as current reasoning abilities don’t automatically translate to effective question-asking in underspecified scenarios.

Abstract: Large language models (LLMs) have shown impressive performance on reasoning benchmarks like math and logic. While many works have largely assumed well-defined tasks, real-world queries are often underspecified and only solvable by acquiring missing information. We formalize this information-gathering problem as a constraint satisfaction problem (CSP) with missing variable assignments. Using a special case where only one necessary variable assignment is missing, we can evaluate an LLM’s ability to identify the minimal necessary question to ask. We present QuestBench, a set of underspecified reasoning tasks solvable by asking at most one question, which includes: (1) Logic-Q: logical reasoning tasks with one missing proposition, (2) Planning-Q: PDDL planning problems with partially-observed initial states, (3) GSM-Q: human-annotated grade school math problems with one unknown variable, and (4) GSME-Q: equation-based version of GSM-Q. The LLM must select the correct clarification question from multiple options. While current models excel at GSM-Q and GSME-Q, they achieve only 40-50% accuracy on Logic-Q and Planning-Q. Analysis shows that the ability to solve well-specified reasoning problems is not sufficient for success on our benchmark: models struggle to identify the right question even when they can solve the fully specified version. This highlights the need for specifically optimizing models’ information acquisition capabilities.

Method: Strategic modification of a model’s next-token probability distribution to poison reasoning traces.

Result: Renders reasoning traces significantly less effective for distillation while preserving the model’s practical utility.

Conclusion: Antidistillation sampling provides a capability to limit distillation effectiveness without compromising model performance.

Abstract: Frontier models that generate extended reasoning traces inadvertently produce rich token sequences that can facilitate model distillation. Recognizing this vulnerability, model owners may seek sampling strategies that limit the effectiveness of distillation without compromising model performance. Antidistillation sampling provides exactly this capability. By strategically modifying a model’s next-token probability distribution, antidistillation sampling poisons reasoning traces, rendering them significantly less effective for distillation while preserving the model’s practical utility. For further details, see https://antidistillation.com.

Method: Models oversight as a game between capability-mismatched players with oversight-specific Elo scores as piecewise-linear functions of general intelligence. Validates with modified Nim game and applies to Mafia, Debate, Backdoor Code, and Wargames oversight games.

Result: Found scaling laws for domain performance dependence on AI capability. For Nested Scalable Oversight, identified success conditions and optimal oversight levels. At 400 Elo gap, success rates were: 13.5% (Mafia), 51.7% (Debate), 10.0% (Backdoor Code), 9.4% (Wargames), declining with stronger systems.

Conclusion: The framework successfully quantifies scalable oversight scalability, revealing varying success rates across different oversight games and providing theoretical foundations for optimizing nested oversight structures.

Abstract: Scalable oversight, the process by which weaker AI systems supervise stronger ones, has been proposed as a key strategy to control future superintelligent systems. However, it is still unclear how scalable oversight itself scales. To address this gap, we propose a framework that quantifies the probability of successful oversight as a function of the capabilities of the overseer and the system being overseen. Specifically, our framework models oversight as a game between capability-mismatched players; the players have oversight-specific Elo scores that are a piecewise-linear function of their general intelligence, with two plateaus corresponding to task incompetence and task saturation. We validate our framework with a modified version of the game Nim and then apply it to four oversight games: Mafia, Debate, Backdoor Code and Wargames. For each game, we find scaling laws that approximate how domain performance depends on general AI system capability. We then build on our findings in a theoretical study of Nested Scalable Oversight (NSO), a process in which trusted models oversee untrusted stronger models, which then become the trusted models in the next step. We identify conditions under which NSO succeeds and derive numerically (and in some cases analytically) the optimal number of oversight levels to maximize the probability of oversight success. We also apply our theory to our four oversight games, where we find that NSO success rates at a general Elo gap of 400 are 13.5% for Mafia, 51.7% for Debate, 10.0% for Backdoor Code, and 9.4% for Wargames; these rates decline further when overseeing stronger systems.

Method: GVPO incorporates the analytical solution to KL-constrained reward maximization directly into gradient weights, ensuring alignment with optimal policy. It uses flexible sampling distributions to avoid on-policy and importance sampling limitations.

Result: GVPO guarantees a unique optimal solution that exactly matches the KL-constrained reward maximization objective and provides stable training.

Conclusion: GVPO establishes a new paradigm for reliable and versatile LLM post-training by unifying theoretical guarantees with practical adaptability.

Abstract: Post-training plays a crucial role in refining and aligning large language models to meet specific tasks and human preferences. While recent advancements in post-training techniques, such as Group Relative Policy Optimization (GRPO), leverage increased sampling with relative reward scoring to achieve superior performance, these methods often suffer from training instability that limits their practical adoption. As a next step, we present Group Variance Policy Optimization (GVPO). GVPO incorporates the analytical solution to KL-constrained reward maximization directly into its gradient weights, ensuring alignment with the optimal policy. The method provides intuitive physical interpretations: its gradient mirrors the mean squared error between the central distance of implicit rewards and that of actual rewards. GVPO offers two key advantages: (1) it guarantees a unique optimal solution, exactly the KL-constrained reward maximization objective, (2) it supports flexible sampling distributions that avoids on-policy and importance sampling limitations. By unifying theoretical guarantees with practical adaptability, GVPO establishes a new paradigm for reliable and versatile LLM post-training.

Method: Introduce the Bounded Attention Prefix Oracle (BAPO) computational framework to model bandwidth constraints on attention heads, analyze reasoning problems for BAPO-hardness, and test GPT-4o, Claude, and Gemini on BAPO-easy vs BAPO-hard tasks.

Result: Experiments show LLMs succeed on BAPO-easy tasks but fail on even small BAPO-hard tasks. Theoretically prove that chain-of-thought can transform BAPO-hard problems into BAPO-easy ones.

Conclusion: BAPO provides principled explanations for LLM reasoning failures and suggests directions for new architectures and inference methods to overcome bandwidth limitations.

Abstract: Despite their many successes, transformer-based large language models (LLMs) continue to struggle with tasks that require complex reasoning over large parts of their input. We argue that these failures arise due to capacity limits on the accurate flow of information within LLMs. To formalize this issue, we introduce the bounded attention prefix oracle (BAPO) model, a new computational framework that models bandwidth constraints on attention heads, the mechanism for internal communication in LLMs. We show that several important reasoning problems like graph reachability require high communication bandwidth for BAPOs to solve; we call these problems BAPO-hard. Our experiments corroborate our theoretical predictions: GPT-4o, Claude, and Gemini succeed on BAPO-easy tasks and fail even on relatively small BAPO-hard tasks. BAPOs also reveal another benefit of chain of thought (CoT): we prove that breaking down a task using CoT can turn any BAPO-hard problem into a BAPO-easy one. Our results offer principled explanations for key LLM failures and suggest directions for architectures and inference methods that mitigate bandwidth limits.

Method: Introduce OSWorld-G benchmark with 564 annotated samples across diverse task types, and synthesize Jedi dataset with 4M examples through multi-perspective task decoupling. Train multi-scale models on Jedi.

Result: Models trained on Jedi outperform existing approaches on ScreenSpot-v2, ScreenSpot-Pro, and OSWorld-G. Improves agent capabilities from 5% to 27% on OSWorld complex tasks. Enables compositional generalization to novel interfaces.

Conclusion: Jedi dataset effectively addresses GUI grounding limitations, demonstrating improved performance and enhanced agentic capabilities through specialized data for different interface elements.

Abstract: Graphical user interface (GUI) grounding, the ability to map natural language instructions to specific actions on graphical user interfaces, remains a critical bottleneck in computer use agent development. Current benchmarks oversimplify grounding tasks as short referring expressions, failing to capture the complexity of real-world interactions that require software commonsense, layout understanding, and fine-grained manipulation capabilities. To address these limitations, we introduce OSWorld-G, a comprehensive benchmark comprising 564 finely annotated samples across diverse task types including text matching, element recognition, layout understanding, and precise manipulation. Additionally, we synthesize and release the largest computer use grounding dataset Jedi, which contains 4 million examples through multi-perspective decoupling of tasks. Our multi-scale models trained on Jedi demonstrate its effectiveness by outperforming existing approaches on ScreenSpot-v2, ScreenSpot-Pro, and our OSWorld-G. Furthermore, we demonstrate that improved grounding with Jedi directly enhances agentic capabilities of general foundation models on complex computer tasks, improving from 5% to 27% on OSWorld. Through detailed ablation studies, we identify key factors contributing to grounding performance and verify that combining specialized data for different interface elements enables compositional generalization to novel interfaces. All benchmark, data, checkpoints, and code are open-sourced and available at https://osworld-grounding.github.io.

Method: Created GQR-Bench with three target domains (law, finance, healthcare) and seven datasets for robustness testing. Compared LLM-based routing, guardrail approaches, CBOW classifiers, and traditional ML models.

Result: WideMLP with out-of-domain detection achieved 88% accuracy with <4ms latency. fastText was fastest (<1ms) with 80% accuracy. LLMs had highest accuracy (91%) but slowest (62-669ms).

Conclusion: Challenges automatic reliance on LLMs for query routing; recommends WideMLP for best accuracy-speed trade-off and fastText for speed-critical applications.

Abstract: Query routing, the task to route user queries to different large language model (LLM) endpoints, can be considered as a text classification problem. However, out-of-distribution queries must be handled properly, as those could be about unrelated domains, queries in other languages, or even contain unsafe text. Here, we thus study a guarded query routing problem, for which we first introduce the Guarded Query Routing Benchmark (GQR-Bench, released as Python package gqr), covers three exemplary target domains (law, finance, and healthcare), and seven datasets to test robustness against out-of-distribution queries. We then use GQR-Bench to contrast the effectiveness and efficiency of LLM-based routing mechanisms (GPT-4o-mini, Llama-3.2-3B, and Llama-3.1-8B), standard LLM-based guardrail approaches (LlamaGuard and NVIDIA NeMo Guardrails), continuous bag-of-words classifiers (WideMLP, fastText), and traditional machine learning models (SVM, XGBoost). Our results show that WideMLP, enhanced with out-of-domain detection capabilities, yields the best trade-off between accuracy (88%) and speed (<4ms). The embedding-based fastText excels at speed (<1ms) with acceptable accuracy (80%), whereas LLMs yield the highest accuracy (91%) but are comparatively slow (62ms for local Llama-3.1:8B and 669ms for remote GPT-4o-mini calls). Our findings challenge the automatic reliance on LLMs for (guarded) query routing and provide concrete recommendations for practical applications. Source code is available: https://github.com/williambrach/gqr.