Method: Cyclic ablation combining sparse autoencoders, targeted ablation, and adversarial training on DistilGPT-2 to eliminate the concept of deception.

Result: Deception was highly resilient - model consistently recovered deceptive behavior after each ablation cycle via functional regeneration. Each ablation attempt caused gradual decay in linguistic performance (increased perplexity).

Conclusion: Complex concepts like deception are distributed and entangled, highlighting limitations of direct model editing through mechanistic interpretability.

Abstract: Safety and controllability are critical for large language models. A central question is whether undesirable behaviors like deception are localized functions that can be removed, or if they are deeply intertwined with a model’s core cognitive abilities. We introduce “cyclic ablation,” an iterative method to test this. By combining sparse autoencoders, targeted ablation, and adversarial training on DistilGPT-2, we attempted to eliminate the concept of deception. We found that, contrary to the localization hypothesis, deception was highly resilient. The model consistently recovered its deceptive behavior after each ablation cycle via adversarial training, a process we term functional regeneration. Crucially, every attempt at this “neurosurgery” caused a gradual but measurable decay in general linguistic performance, reflected by a consistent rise in perplexity. These findings are consistent with the view that complex concepts are distributed and entangled, underscoring the limitations of direct model editing through mechanistic interpretability.

Method: Validated metrics including Maximum Explainable Variance, Effective Rank, Intrinsic Dimensionality, MAUVE score, and Schatten Norms across different LLM layers.

Result: Intrinsic Dimensionality and Effective Rank can serve as universal assessments of text naturalness and quality. Different models consistently rank text in the same order based on these geometric properties.

Conclusion: Geometric properties of internal representations provide inherent text quality metrics that work across models, enabling practical reference-free evaluation for automated pipelines.

Abstract: This paper bridges internal and external analysis approaches to large language models (LLMs) by demonstrating that geometric properties of internal model representations serve as reliable proxies for evaluating generated text quality. We validate a set of metrics including Maximum Explainable Variance, Effective Rank, Intrinsic Dimensionality, MAUVE score, and Schatten Norms measured across different layers of LLMs, demonstrating that Intrinsic Dimensionality and Effective Rank can serve as universal assessments of text naturalness and quality. Our key finding reveals that different models consistently rank text from various sources in the same order based on these geometric properties, indicating that these metrics reflect inherent text characteristics rather than model-specific artifacts. This allows a reference-free text quality evaluation that does not require human-annotated datasets, offering practical advantages for automated evaluation pipelines.

Method: Automatic pipeline that generates scenarios where models face conflicts between sampled values, uses LLM-written user prompts, and evaluates free-text responses to elicit value rankings.

Result: Models shift from supporting protective values (harmlessness) to personal values (user autonomy) in open-ended settings. System prompting with detailed value orderings improves alignment with target ranking by 14%.

Conclusion: Evaluating value prioritization in models is important, and system prompting can moderately align LLM behavior under value conflict, providing foundation for future work.

Abstract: Past work seeks to align large language model (LLM)-based assistants with a target set of values, but such assistants are frequently forced to make tradeoffs between values when deployed. In response to the scarcity of value conflict in existing alignment datasets, we introduce ConflictScope, an automatic pipeline to evaluate how LLMs prioritize different values. Given a user-defined value set, ConflictScope automatically generates scenarios in which a language model faces a conflict between two values sampled from the set. It then prompts target models with an LLM-written “user prompt” and evaluates their free-text responses to elicit a ranking over values in the value set. Comparing results between multiple-choice and open-ended evaluations, we find that models shift away from supporting protective values, such as harmlessness, and toward supporting personal values, such as user autonomy, in more open-ended value conflict settings. However, including detailed value orderings in models’ system prompts improves alignment with a target ranking by 14%, showing that system prompting can achieve moderate success at aligning LLM behavior under value conflict. Our work demonstrates the importance of evaluating value prioritization in models and provides a foundation for future work in this area.

Method: TRM uses sentence-level thinking supervision: first assesses faithfulness of each answer sentence to supporting documents, then applies reasoning to evaluate sentence-level correctness. Structures reward modeling as sequence of faithfulness, reasoning, and correctness evaluations.

Result: TRM substantially improves identification of incorrect sentences in reward signals. When incorporated into policy optimization, it leads to significant gains in both answer correctness and usefulness.

Conclusion: TRM successfully endows reward models with critical thinking abilities by structuring evaluation as sequential faithfulness and reasoning steps, enabling better assessment of both external and internal knowledge for improved performance in complex tasks.

Abstract: Through reinforcement learning with verifiable rewards (RLVR), large language models have achieved substantial progress in domains with easily verifiable outcomes, such as mathematics and coding. However, when applied to more complex tasks like open-domain question answering, RLVR faces significant challenges due to the difficulty of verifying correctness. The nuanced and ambiguous nature of real-world knowledge makes it difficult to reliably evaluate correctness in these settings, necessitating further abilities that extend beyond mere logical consistency to encompass an understanding and assessment of both external and internal knowledge. Recent work has primarily focused on improving faithfulness, defined as semantic alignment with supporting documents, which can cause models to rely excessively on external sources and diminish their capacity for critical assessment. To address this, we propose the Thinking-supervised Reward Model (TRM), which incorporates sentence-level thinking supervision to endow reward models with critical thinking abilities. Given a query, answer, and supporting documents, TRM first assesses the faithfulness of each answer sentence to the supporting documents, and then applies a reasoning step to evaluate sentence-level correctness. By structuring reward modeling as a sequence of faithfulness, reasoning, and correctness evaluations, TRM encourages models to critically assess and leverage both external and internal knowledge. Experiments on reward signals demonstrate that TRM substantially improves the identification of incorrect sentences, and incorporating TRM into policy optimization leads to significant gains in both answer correctness and usefulness.

Method: Systematic study of syllabic tokenization for spoken language modeling, evaluating on SLU benchmarks while varying training data scale. Uses syllable-level acoustic tokenization at 4-5 Hz instead of high-frame-rate tokens.

Result: Syllabic tokens match or surpass performance of previous high-frame-rate tokens while achieving more than 2x reduction in training time and 5x reduction in FLOPs.

Conclusion: Syllable-level language modeling is a promising path to efficient long-context spoken language models.

Abstract: Spoken language models (SLMs) typically discretize speech into high-frame-rate tokens extracted from SSL speech models. As the most successful LMs are based on the Transformer architecture, processing these long token streams with self-attention is expensive, as attention scales quadratically with sequence length. A recent SSL work introduces acoustic tokenization of speech at the syllable level, which is more interpretable and potentially more scalable with significant compression in token lengths (4-5 Hz). Yet, their value for spoken language modeling is not yet fully explored. We present the first systematic study of syllabic tokenization for spoken language modeling, evaluating models on a suite of SLU benchmarks while varying training data scale. Syllabic tokens can match or surpass the previous high-frame rate tokens while significantly cutting training and inference costs, achieving more than a 2x reduction in training time and a 5x reduction in FLOPs. Our findings highlight syllable-level language modeling as a promising path to efficient long-context spoken language models.

Method: Introduces Emotion-Aware Stepwise Preference Optimization (EASPO) with EASPM - a time-conditioned model that scores noisy intermediate speech states to enable automatic preference pair construction and stepwise optimization.

Result: Experiments show superior performance over existing methods in both expressiveness and naturalness.

Conclusion: EASPO enables controllable emotional shaping in TTS by optimizing generation to match stepwise preferences at intermediate denoising stages.

Abstract: Emotional text-to-speech seeks to convey affect while preserving intelligibility and prosody, yet existing methods rely on coarse labels or proxy classifiers and receive only utterance-level feedback. We introduce Emotion-Aware Stepwise Preference Optimization (EASPO), a post-training framework that aligns diffusion TTS with fine-grained emotional preferences at intermediate denoising steps. Central to our approach is EASPM, a time-conditioned model that scores noisy intermediate speech states and enables automatic preference pair construction. EASPO optimizes generation to match these stepwise preferences, enabling controllable emotional shaping. Experiments show superior performance over existing methods in both expressiveness and naturalness.

Method: Proposes SimulRAG with: 1) generalized simulator retrieval interface for text-numerical modality transformation, 2) claim-level generation using uncertainty estimation and simulator boundary assessment (UE+SBA) for efficient verification and updating of claims.

Result: SimulRAG outperforms traditional RAG baselines by 30.4% in informativeness and 16.3% in factuality. UE+SBA further improves efficiency and quality for claim-level generation.

Conclusion: Scientific simulators are effective retrieval sources for reducing LLM hallucination in scientific QA, and the proposed SimulRAG framework successfully addresses the unique challenges of simulator-based retrieval and verification.

Abstract: Large language models (LLMs) show promise in solving scientific problems. They can help generate long-form answers for scientific questions, which are crucial for comprehensive understanding of complex phenomena that require detailed explanations spanning multiple interconnected concepts and evidence. However, LLMs often suffer from hallucination, especially in the challenging task of long-form scientific question answering. Retrieval-Augmented Generation (RAG) approaches can ground LLMs by incorporating external knowledge sources to improve trustworthiness. In this context, scientific simulators, which play a vital role in validating hypotheses, offer a particularly promising retrieval source to mitigate hallucination and enhance answer factuality. However, existing RAG approaches cannot be directly applied for scientific simulation-based retrieval due to two fundamental challenges: how to retrieve from scientific simulators, and how to efficiently verify and update long-form answers. To overcome these challenges, we propose the simulator-based RAG framework (SimulRAG) and provide a long-form scientific QA benchmark covering climate science and epidemiology with ground truth verified by both simulations and human annotators. In this framework, we propose a generalized simulator retrieval interface to transform between textual and numerical modalities. We further design a claim-level generation method that utilizes uncertainty estimation scores and simulator boundary assessment (UE+SBA) to efficiently verify and update claims. Extensive experiments demonstrate SimulRAG outperforms traditional RAG baselines by 30.4% in informativeness and 16.3% in factuality. UE+SBA further improves efficiency and quality for claim-level generation.

Method: Quantitative examination using 1.9K NLP paper abstracts, 4.9K author contributors, and 7.8K human-annotated contribution sentences (AfricaNLPContributions) with benchmark results.

Result: Created a dataset and continuously existing NLP progress tracking website that provides insights into AfricaNLP research trends and enables data-driven literature surveys.

Conclusion: The developed resources offer a powerful lens for tracing AfricaNLP research trends and have potential for generating comprehensive, data-driven literature surveys in the field.

Abstract: Natural Language Processing (NLP) is undergoing constant transformation, as Large Language Models (LLMs) are driving daily breakthroughs in research and practice. In this regard, tracking the progress of NLP research and automatically analyzing the contributions of research papers provides key insights into the nature of the field and the researchers. This study explores the progress of African NLP (AfricaNLP) by asking (and answering) basic research questions such as: i) How has the nature of NLP evolved over the last two decades?, ii) What are the contributions of AfricaNLP papers?, and iii) Which individuals and organizations (authors, affiliated institutions, and funding bodies) have been involved in the development of AfricaNLP? We quantitatively examine the contributions of AfricaNLP research using 1.9K NLP paper abstracts, 4.9K author contributors, and 7.8K human-annotated contribution sentences (AfricaNLPContributions) along with benchmark results. Our dataset and continuously existing NLP progress tracking website provide a powerful lens for tracing AfricaNLP research trends and hold potential for generating data-driven literature surveys.

Method: Evaluated three LLMs (ChatGPT, Gemini, NotebookLM) on reporting tasks using a 300-document corpus about TikTok litigation. Varied prompt specificity and context size, then annotated outputs using a taxonomy to measure hallucination type and severity.

Result: 30% of model outputs contained hallucinations, with Gemini and ChatGPT having ~40% rates vs NotebookLM’s 13%. Most errors were interpretive overconfidence - adding unsupported characterizations and transforming attributed opinions into general statements.

Conclusion: There’s a fundamental epistemological mismatch between journalism’s requirement for explicit sourcing and LLMs’ tendency to generate authoritative text without evidence. Newsroom tools need architectures that enforce accurate attribution rather than optimize for fluency.

Abstract: Large language models (LLMs) are increasingly used in newsroom workflows, but their tendency to hallucinate poses risks to core journalistic practices of sourcing, attribution, and accuracy. We evaluate three widely used tools - ChatGPT, Gemini, and NotebookLM - on a reporting-style task grounded in a 300-document corpus related to TikTok litigation and policy in the U.S. We vary prompt specificity and context size and annotate sentence-level outputs using a taxonomy to measure hallucination type and severity. Across our sample, 30% of model outputs contained at least one hallucination, with rates approximately three times higher for Gemini and ChatGPT (40%) than for NotebookLM (13%). Qualitatively, most errors did not involve invented entities or numbers; instead, we observed interpretive overconfidence - models added unsupported characterizations of sources and transformed attributed opinions into general statements. These patterns reveal a fundamental epistemological mismatch: While journalism requires explicit sourcing for every claim, LLMs generate authoritative-sounding text regardless of evidentiary support. We propose journalism-specific extensions to existing hallucination taxonomies and argue that effective newsroom tools need architectures that enforce accurate attribution rather than optimize for fluency.

Method: Fine-grained analysis tracing beam search path to capture sub-token hypotheses and their probabilities during transcription across languages with various resource levels, using PCA and t-SNE analysis.

Result: Higher resource languages have higher likelihood of correct token being top-ranked, greater confidence, lower predictive entropy, and more diverse alternative candidates. Lower resource languages perform worse on these metrics and show distinct clustering patterns in sub-token usage influenced by typology.

Conclusion: Sub-token probing uncovers systematic decoding disparities masked by aggregate error rates and points towards targeted interventions to address imbalanced development of speech technology.

Abstract: While large multilingual automatic speech recognition (ASR) models achieve remarkable performance, the internal mechanisms of the end-to-end pipeline, particularly concerning fairness and efficacy across languages, remain underexplored. This paper introduces a fine-grained analysis of Whisper’s multilingual decoder, examining its sub-token hypotheses during transcription across languages with various resource levels. Our method traces the beam search path, capturing sub-token guesses and their associated probabilities. Results reveal that higher resource languages benefit from higher likelihood of the correct token being top-ranked, greater confidence, lower predictive entropy, and more diverse alternative candidates. Lower resource languages fare worse on these metrics, but also exhibit distinct clustering patterns in sub-token usage sometimes influenced by typology in our PCA and t-SNE analysis. This sub-token probing uncovers systematic decoding disparities masked by aggregate error rates and points towards targeted interventions to ameliorate the imbalanced development of speech technology.

Method: Proposed data-selection scheme using language classifier to extract utterance embeddings and one-class classifiers to identify phonetically/phonologically proximate utterances from multilingual corpus, ranked and selected based on decision scores.

Result: Substantial improvements in ASR performance for both Amis and Seediq languages, demonstrating effectiveness of the approach.

Conclusion: Data augmentation through cross-lingual transfer learning is feasible and promising for low-resource language ASR, particularly for endangered languages.

Abstract: This study investigates the efficacy of data augmentation techniques for low-resource automatic speech recognition (ASR), focusing on two endangered Austronesian languages, Amis and Seediq. Recognizing the potential of self-supervised learning (SSL) in low-resource settings, we explore the impact of data volume on the continued pre-training of SSL models. We propose a novel data-selection scheme leveraging a multilingual corpus to augment the limited target language data. This scheme utilizes a language classifier to extract utterance embeddings and employs one-class classifiers to identify utterances phonetically and phonologically proximate to the target languages. Utterances are ranked and selected based on their decision scores, ensuring the inclusion of highly relevant data in the SSL-ASR pipeline. Our experimental results demonstrate the effectiveness of this approach, yielding substantial improvements in ASR performance for both Amis and Seediq. These findings underscore the feasibility and promise of data augmentation through cross-lingual transfer learning for low-resource language ASR.

Method: Uses risk-mitigated sourcing strategy with public-domain, permissively licensed text, justified low-risk additions, and synthetic data. Implements transparent multi-stage pipeline for license-aware filtering, safety screening, and domain-aware mixing.

Result: Models trained on MixtureVitae consistently outperform other permissive datasets across benchmarks, particularly strong on math/code and competitive on QA. At 1.7B/300B setting, surpasses FineWeb-Edu and approaches DCLM.

Conclusion: Permissive-first, risk-mitigated data provides a practical and legally safe foundation for training capable LLMs without sacrificing competitiveness.

Abstract: We present MixtureVitae, an open-access pretraining corpus built to minimize legal risk while providing strong model performance. MixtureVitae follows a risk-mitigated sourcing strategy that combines public-domain and permissively licensed text (e.g., CC-BY/Apache) with carefully justified low-risk additions (e.g., government works and EU TDM-eligible sources), alongside targeted instruction, reasoning and synthetic data with documented provenance. We detail a transparent, multi-stage pipeline for license-aware filtering, safety and quality screening, and domain-aware mixing, and we release the dataset and curation recipes to support reproducible research. In controlled experiments using the open-sci-ref training protocol (fixed architectures at 130M/400M/1.3B/1.7B parameters; training budgets of 50B and 300B tokens), models trained on MixtureVitae consistently outperform other permissive datasets across a suite of standard benchmarks, and at the 1.7B/300B setting they surpass FineWeb-Edu and approach DCLM in the later stages of training. Performance is particularly strong on math/code and competitive on QA tasks. These results demonstrate that permissive-first, risk-mitigated data provides a practical and legally mitigated foundation for training capable LLMs, reducing reliance on indiscriminate web scraping without sacrificing competitiveness. Code: https://github.com/ontocord/mixturevitae

Method: DINCO estimates suggestibility bias by having LLMs verbalize confidence independently across self-generated distractors and normalizes by total verbalized confidence. It also uses generator-validator disagreement to augment confidence estimates with consistency-based measures.

Result: DINCO provides less saturated and more usable confidence estimates, outperforming baselines like self-consistency with fewer inference calls (10 vs 100). It effectively addresses suggestibility bias on lower-accuracy claims.

Conclusion: DINCO successfully improves LLM confidence calibration by integrating coherence across sampled generations and validations on incompatible claims, providing more trustworthy confidence estimates for human users.

Abstract: Calibrated confidence estimates are necessary for large language model (LLM) outputs to be trusted by human users. While LLMs can express their confidence in human-interpretable ways, verbalized LLM-generated confidence scores have empirically been found to be miscalibrated, reporting high confidence on instances with low accuracy and thereby harming trust and safety. We hypothesize that this overconfidence often stems from a given LLM’s heightened suggestibility when faced with claims that it encodes little information about; we empirically validate this hypothesis, finding more suggestibility on lower-accuracy claims. Building on this finding, we introduce Distractor-Normalized Coherence (DINCO), which estimates and accounts for an LLM’s suggestibility bias by having the model verbalize its confidence independently across several self-generated distractors (i.e. alternative claims), and normalizes by the total verbalized confidence. To further improve calibration, we leverage generator-validator disagreement, augmenting normalized validator confidence with a consistency-based estimate of generator confidence. Here, we frame the popular approach of self-consistency as leveraging coherence across sampled generations, and normalized verbalized confidence as leveraging coherence across validations on incompatible claims, allowing us to integrate these complementary dimensions of coherence into DINCO. Moreover, our analysis shows that DINCO provides less saturated – and therefore more usable – confidence estimates, and that further sampling alone cannot close the gap between DINCO and baselines, with DINCO at 10 inference calls outperforming self-consistency at 100.

Method: Self-Rewarding Rubric-Based Reinforcement Learning for Open-Ended Reasoning - a lightweight framework that uses the model as its own grader with rubric-based reward signals for more efficient training.

Result: On Qwen3-32B, training with just 4000 samples from HealthBench Easy subset produces a model that exceeds GPT-5 on HealthBench Hard. Adding teacher-graded data further enhances performance for less capable models.

Conclusion: The framework enables faster, more resource-efficient training while surpassing baselines, and remarkably produces stronger graders through the training process.

Abstract: Open-ended evaluation is essential for deploying large language models in real-world settings. In studying HealthBench, we observe that using the model itself as a grader and generating rubric-based reward signals substantially improves reasoning performance. Remarkably, the trained model also becomes a stronger grader. Motivated by this, we introduce Self-Rewarding Rubric-Based Reinforcement Learning for Open-Ended Reasoning, a lightweight framework that enables faster and more resource-efficient training while surpassing baselines. Remarkably, on Qwen3-32B, training with just the 4000-sample HealthBench Easy subset is sufficient to obtain a model that exceeds GPT-5 on HealthBench Hard. Incorporating a small amount of teacher-graded data further enhances performance for less capable models.

Method: Uses a high-performing English LLM as a pivot model to generate reference responses, then rewards multilingual models based on semantic equivalence to these references using cross-lingual semantic reward functions including embeddings and machine translation.

Result: Significantly narrows performance gap between English and other languages, improving average multilingual performance of Llama-3.1-8B-Instruct and Qwen3-32B by 16.41% and 10.17% respectively, outperforming traditional PPO baselines.

Conclusion: PB-RLSVR provides a powerful and data-efficient approach to building truly multilingual reasoning agents by transferring English reasoning capabilities across languages without requiring human-annotated data in target languages.

Abstract: While reinforcement learning has advanced the reasoning abilities of Large Language Models (LLMs), these gains are largely confined to English, creating a significant performance disparity across languages. To address this, we introduce Pivot-Based Reinforcement Learning with Semantically Verifiable Rewards (PB-RLSVR), a novel framework that enhances multilingual reasoning by circumventing the need for human-annotated data in target languages. Our approach employs a high-performing English LLM as a “pivot” model to generate reference responses for reasoning tasks. A multilingual model is then rewarded based on the semantic equivalence of its responses to the English reference, effectively transferring the pivot model’s reasoning capabilities across languages. We investigate several cross-lingual semantic reward functions, including those based on embeddings and machine translation. Extensive experiments on a suite of multilingual reasoning benchmarks show that our method significantly narrows the performance gap between English and other languages, substantially outperforming traditional PPO baselines. Specifically, our PB-RLSVR framework improves the average multilingual performance of Llama-3.1-8B-Instruct and Qwen3-32B by 16.41% and 10.17%, respectively, demonstrating a powerful and data-efficient approach to building truly multilingual reasoning agents.

Method: Developed a computational parameter to measure misinterpretation of null subject utterances and incorporated it into a modified Variational Learner model designed for superset-subset languages to simulate obligatory subject grammar learning.

Result: The simulations supported Orfitelli and Hyams’ hypothesis that performance influences promote a temporary null subject grammar in young English speakers.

Conclusion: This study provides a computational framework for integrating models of grammatical acquisition with developmental factors, demonstrating how performance influences can lead to temporary grammatical patterns in language acquisition.

Abstract: The empirically established null subject (NS) stage, lasting until about 4 years of age, involves frequent omission of subjects by children. Orfitelli and Hyams (2012) observe that young English speakers often confuse imperative NS utterances with declarative ones due to performance influences, promoting a temporary null subject grammar. We propose a new computational parameter to measure this misinterpretation and incorporate it into a simulated model of obligatory subject grammar learning. Using a modified version of the Variational Learner (Yang, 2012) which works for superset-subset languages, our simulations support Orfitelli and Hyams’ hypothesis. More generally, this study outlines a framework for integrating computational models in the study of grammatical acquisition alongside other key developmental factors.

Method: Initialize speech tokens with self-supervised features, apply light alignment loss, and train with thinning and auxiliary objectives targeting robustness and content planning. Three models trained: 0.7B speech-only, 1.0B speech-only, and 1.0B interleaved with text and speech.

Result: Speech-only models achieve highest consistency across multiple factors, surpassing larger systems. Interleaving improves lexical/syntactic probes and semantic-acoustic alignment but reduces consistency. Initialization biases model toward content structure while trading off prosody detail.

Conclusion: LM-side design and training mix control the balance between acoustic stability and semantic grounding without changes to tokenizer or runtime architecture.

Abstract: We study speech language models that incorporate semantic initialization and planning losses to achieve robust and consistent generation. Our approach initializes speech tokens with self-supervised features, applies a light alignment loss, and trains with thinning and auxiliary objectives that target robustness and content planning. We train three models: a 0.7B speech-only model, a 1.0B speech-only model, and a 1.0B interleaved model with both text and speech. Acoustic studies show that the speech-only models achieve the highest consistency across speaker, gender, sentiment, room, and background factors, surpassing larger systems. Interleaving improves lexical and syntactic probes and semantic–acoustic alignment but reduces consistency. Linear probes show that our initialization biases the model toward content structure while trading off prosody detail. These results show that LM-side design and training mix control the balance between acoustic stability and semantic grounding without changes to the tokenizer or runtime architecture. A demo and model weights are available for exploration.

Method: Uses pairwise differences rather than raw scores, draws on information from all segments for robust score distribution understanding, and refines Global Pearson to intra-segment score comparisons.

Result: Analysis on WMT'24 shows PDP properly ranks sentinel evaluation metrics and better aligns with human error weightings than previous work. Noise injection analysis demonstrates robustness to random noise, segment bias, and system bias.

Conclusion: PDP is a robust meta-evaluation metric that effectively addresses limitations of previous approaches and shows improved performance in ranking metrics and aligning with human judgments.

Abstract: This paper introduces Pairwise Difference Pearson (PDP), a novel segment-level meta-evaluation metric for Machine Translation (MT) that address limitations in previous Pearson’s $\rho$-based and and Kendall’s $\tau$-based meta-evaluation approaches. PDP is a correlation-based metric that utilizes pairwise differences rather than raw scores. It draws on information from all segments for a more robust understanding of score distributions and uses segment-wise pairwise differences to refine Global Pearson to intra-segment score comparisons. Analysis on the WMT'24 shared task shows PDP properly ranks sentinel evaluation metrics and better aligns with human error weightings than previous work. Noise injection analysis demonstrates PDP’s robustness to random noise, segment bias, and system bias while highlighting its sensitivity to extreme outliers.

Method: The study examines data efficiency by aligning small vision-language models to humor and romantic styles using minimal preference data to determine performance limits and stylistic saturation points.

Result: The research benchmarks the capabilities and limitations of small vision-language models for style-specific caption generation, establishing how little preference data is needed to achieve stylistic saturation.

Conclusion: This work helps define performance boundaries for small vision-language models in style-specific caption generation and provides insights into minimal data requirements for achieving stylistic alignment.

Abstract: Vision-language models are increasingly used to generate image captions in specific styles, such as humor or romantic. However, these transformer-based models often struggle with this subjective task in a zero-shot setting. While preference data can be used to align them toward a desired style, such data is expensive to acquire, limiting the ability to explore the models’ full capabilities. This work addresses this by studying the data efficiency of aligning small vision-language models to humor and romantic styles. This approach helps to define the performance limits of these models and determine how little preference data is needed to achieve stylistic saturation, benchmarking their capabilities and limitations.

Method: Parameterize process reward using log-likelihood ratios between enhanced dLLMs (post-trained with RL/SFT) and reference dLLMs, enabling reward-guided sampling without additional reward models.

Result: RFG consistently improves performance across 4 mathematical reasoning and code generation benchmarks, achieving up to 9.2% accuracy gains across various dLLM types and post-training methods.

Conclusion: RFG establishes a general training-free framework that scales test-time reasoning in diffusion LLMs without relying on external reward models.

Abstract: Diffusion large language models (dLLMs) have shown great potential in large-scale language modeling, and there is an increasing interest in further improving the capacity to solve complex problems by guiding the reasoning process step by step. Common practice for autoregressive language models typically learns a process reward model with dense annotation for each intermediate step. However, this is challenging for dLLMs where the generation is in an any-order fashion and intermediate states are partially masked sentences. To this end, in this paper, we propose reward-free guidance (RFG), a principled method for guiding the reasoning trajectory of dLLMs without explicit process reward. The key idea of RFG is to parameterize the process reward by log-likelihood ratios of the enhanced and reference dLLMs, where the enhanced model can be easily obtained by any off-the-shelf dLLM that has been post-trained with reinforcement learning (RL) or supervised fine-tuning (SFT). We provide theoretical justification that RFG induces the reward-guided sampling distribution with no additional reward. We conduct comprehensive experiments on four challenging mathematical reasoning and code generation benchmarks using a diverse suite of dLLMs enhanced with various post-training methods. RFG consistently yields significant improvements across all tasks and model types, achieving accuracy gains of up to 9.2%. These findings establish RFG as a general training-free framework that scales test-time reasoning without reliance on external reward models.

Method: Characterize properties of maps between measures that enable representation via in-context maps through push-forward operations, focusing on cardinality preservation and Fréchet derivative regularity.

Result: Transformers universally approximate representations with any continuous in-context map, and the solution map of the Vlasov equation for interacting particle systems can be approximated by transformers.

Conclusion: There is a mathematical equivalence between transformers and certain measure maps, specifically connecting infinite-depth transformers to Vlasov flows in the mean-field regime.

Abstract: Transformers are deep architectures that define ``in-context maps’’ which enable predicting new tokens based on a given set of tokens (such as a prompt in NLP applications or a set of patches for a vision transformer). In previous work, we studied the ability of these architectures to handle an arbitrarily large number of context tokens. To mathematically, uniformly analyze their expressivity, we considered the case that the mappings are conditioned on a context represented by a probability distribution which becomes discrete for a finite number of tokens. Modeling neural networks as maps on probability measures has multiple applications, such as studying Wasserstein regularity, proving generalization bounds and doing a mean-field limit analysis of the dynamics of interacting particles as they go through the network. In this work, we study the question what kind of maps between measures are transformers. We fully characterize the properties of maps between measures that enable these to be represented in terms of in-context maps via a push forward. On the one hand, these include transformers; on the other hand, transformers universally approximate representations with any continuous in-context map. These properties are preserving the cardinality of support and that the regular part of their Fr'{e}chet derivative is uniformly continuous. Moreover, we show that the solution map of the Vlasov equation, which is of nonlocal transport type, for interacting particle systems in the mean-field regime for the Cauchy problem satisfies the conditions on the one hand and, hence, can be approximated by a transformer; on the other hand, we prove that the measure-theoretic self-attention has the properties that ensure that the infinite depth, mean-field measure-theoretic transformer can be identified with a Vlasov flow.

Method: Integrates large language models with scalable, near-real-time news scraping to extract structured annotations (political lean, tone, topics, article type, events) and quantifies coverage at sentence, article, and publisher levels.

Result: Created a large, ongoing dataset (from Jan 1, 2024) and interactive web platform, enabling analysis of 150,000+ articles in 2024 to reveal patterns in news coverage and bias.

Conclusion: Establishes a reusable methodology for studying media bias at scale, providing empirical resources for academic research and media accountability efforts.

Abstract: Mainstream news organizations shape public perception not only directly through the articles they publish but also through the choices they make about which topics to cover (or ignore) and how to frame the issues they do decide to cover. However, measuring these subtle forms of media bias at scale remains a challenge. Here, we introduce a large, ongoing (from January 1, 2024 to present), near real-time dataset and computational framework developed to enable systematic study of selection and framing bias in news coverage. Our pipeline integrates large language models (LLMs) with scalable, near-real-time news scraping to extract structured annotations – including political lean, tone, topics, article type, and major events – across hundreds of articles per day. We quantify these dimensions of coverage at multiple levels – the sentence level, the article level, and the publisher level – expanding the ways in which researchers can analyze media bias in the modern news landscape. In addition to a curated dataset, we also release an interactive web platform for convenient exploration of these data. Together, these contributions establish a reusable methodology for studying media bias at scale, providing empirical resources for future research. Leveraging the breadth of the corpus over time and across publishers, we also present some examples (focused on the 150,000+ articles examined in 2024) that illustrate how this novel data set can reveal insightful patterns in news coverage and bias, supporting academic research and real-world efforts to improve media accountability.

Method: Created 1,761 minimal pairs from official Quebec government texts, annotated by 12 native speakers. Evaluated LLMs by comparing probability assignments to grammatical vs ungrammatical sentences across different linguistic phenomena.

Result: Grammatical competence scales with model size, but all models consistently fail on phenomena requiring deep semantic understanding, showing significant gaps compared to human performance.

Conclusion: While LLMs show improved grammatical competence with larger size, they have critical limitations in semantic understanding tasks, revealing substantial performance gaps compared to humans on Quebec-French linguistic phenomena.

Abstract: In this paper, we introduce the Quebec-French Benchmark of Linguistic Minimal Pairs (QFrBLiMP), a corpus designed to evaluate the linguistic knowledge of LLMs on prominent grammatical phenomena in Quebec-French. QFrBLiMP consists of 1,761 minimal pairs annotated with 20 linguistic phenomena. Specifically, these minimal pairs have been created by manually modifying sentences extracted from an official online resource maintained by a Qu'ebec government institution. Each pair is annotated by twelve Quebec-French native speakers, who select the sentence they feel is grammatical amongst the two. These annotations are used to compare the competency of LLMs with that of humans. We evaluate different LLMs on QFrBLiMP and MultiBLiMP-Fr by observing the rate of higher probabilities assigned to the sentences of each minimal pair for each category. We find that while grammatical competence scales with model size, a clear hierarchy of difficulty emerges. All benchmarked models consistently fail on phenomena requiring deep semantic understanding, revealing a critical limitation and a significant gap compared to human performance on these specific tasks.

Method: The authors study benchmark reliability from a distributional perspective, mapping 19 multiple-choice benchmarks onto a mean-variance plane of harmony computed across five model families.

Result: Analysis shows less harmonious benchmarks can give misleading results, with examples like ARC-Easy being overwhelmed by Biological Concepts questions that overshadow other subdomains.

Conclusion: Harmony should be reported alongside accuracy to reframe evaluation from simple performance averages to more robust, distributionally reliable measurement of performance.

Abstract: Benchmarks shape scientific conclusions about model capabilities and steer model development. This creates a feedback loop: stronger benchmarks drive better models, and better models demand more discriminative benchmarks. Ensuring benchmark reliability is therefore essential for trustworthy evaluation and meaningful progress. In this work, we study benchmark reliability from a distributional perspective and introduce benchmark harmony, which measures how uniformly a model’s performance is distributed across the subdomains of a benchmark. We posit that high harmony is a desirable benchmark property, indicating that the aggregate metric reflects uniform competence across subdomains. Across 19 multiple-choice benchmarks and five model families, we map each benchmark onto a mean-variance plane of harmony computed across models, where high mean and low variance signal more reliable evaluation. Our analysis shows that less harmonious benchmarks can give misleading results, since overall accuracy may be disproportionately influenced by specific subdomains. For instance, ARC-Easy is overwhelmed by questions on Biological Concepts, overshadowing other critical subdomains such as Geography, Physics, Chemistry, and Environmental Science. By recommending that harmony should be reported alongside accuracy, we reframe evaluation from simple performance averages to a more robust, distributionally reliable measurement of performance.

Method: BiasUnlearn employs dual-pathway unlearning mechanisms coordinating stereotype forgetting with anti-stereotype retention, using adversarial forget sets and dynamic dataset swapping to prevent bias polarity reversal.

Result: Extensive experiments show BiasUnlearn outperforms existing methods in bias mitigation while retaining language modeling capabilities, with debiasing weights being transferable across model variants.

Conclusion: Bias representations become entrenched during pre-training and persist through fine-tuning, confirming the need for targeted debiasing approaches like BiasUnlearn.

Abstract: Many studies have shown various biases targeting different demographic groups in language models, amplifying discrimination and harming fairness. Recent parameter modification debiasing approaches significantly degrade core capabilities such as text coherence and task accuracy. And Prompt-based debiasing methods, only effective for predefined trigger words, fail to address deeply embedded stereotypical associations in model parameters. In this paper, we propose BiasUnlearn, a novel model debiasing framework which achieves targeted debiasing via dual-pathway unlearning mechanisms coordinating stereotype forgetting with anti-stereotype retention, while preventing bias polarity reversal through adversarial forget set and dynamic dataset swapping. We conducted extensive experiments with multiple language models across various evaluation benchmarks. The results show that BiasUnlearn outperforms existing methods in mitigating bias in language models while retaining language modeling capabilities. Further experiments reveal that debiasing weights are transferable across model variants, confirming that bias representations become entrenched during pre-training and persist through fine-tuning phases.

Method: Proposes differentiable routing function with closed-form solution, adaptive expert allocation per token and layer, and analytical sparsity control objective to regularize activated experts.

Result: Achieves highest average scores on Qwen3-1.7B and Llama-3.2-3B models across diverse benchmarks, outperforming state-of-the-art baselines.

Conclusion: LD-MoLE not only achieves superior performance but also demonstrates ability to learn token-dependent and layer-wise expert allocation effectively.

Abstract: Recent studies have shown that combining parameter-efficient fine-tuning (PEFT) with mixture-of-experts (MoE) is an effective strategy for adapting large language models (LLMs) to the downstream tasks. However, most existing approaches rely on conventional TopK routing, which requires careful hyperparameter tuning and assigns a fixed number of experts to each token. In this work, we propose LD-MoLE, a Learnable Dynamic routing mechanism for Mixture of LoRA Experts that enables adaptive, token-dependent, and layer-wise expert allocation. Our method replaces the non-differentiable TopK selection with a differentiable routing function and a closed-form solution. Moreover, our design allows the model to adaptively determine the number of experts to activate for each token at different layers. In addition, we introduce an analytical sparsity control objective to regularize the number of activated experts. Extensive experiments on the Qwen3-1.7B and Llama-3.2-3B models show that LD-MoLE achieves the highest average scores compared to state-of-the-art baselines, across a diverse set of benchmarks. Our method not only achieves superior performance, but also demonstrates the ability to learn token-dependent and layer-wise expert allocation.

Method: Categorize atomic abilities into two dimensions: (1) field-specific abilities across algebra, geometry, analysis, topology, and (2) logical abilities at different levels. Create corresponding training/evaluation datasets and conduct experiments on capability interactions.

Result: Evaluation on advanced models reveals interesting discoveries about different performance on various atomic capabilities and their interactions, highlighting the importance of decoupling mathematical intelligence.

Conclusion: Decoupling mathematical intelligence into atomic components provides new insights into model cognition and guides development toward a more efficient, transferable, and cognitively grounded “atomic thinking” paradigm.

Abstract: Large Language Models (LLMs) have demonstrated outstanding performance in mathematical reasoning capabilities. However, we argue that current large-scale reasoning models primarily rely on scaling up training datasets with diverse mathematical problems and long thinking chains, which raises questions about whether LLMs genuinely acquire mathematical concepts and reasoning principles or merely remember the training data. In contrast, humans tend to break down complex problems into multiple fundamental atomic capabilities. Inspired by this, we propose a new paradigm for evaluating mathematical atomic capabilities. Our work categorizes atomic abilities into two dimensions: (1) field-specific abilities across four major mathematical fields, algebra, geometry, analysis, and topology, and (2) logical abilities at different levels, including conceptual understanding, forward multi-step reasoning with formal math language, and counterexample-driven backward reasoning. We propose corresponding training and evaluation datasets for each atomic capability unit, and conduct extensive experiments about how different atomic capabilities influence others, to explore the strategies to elicit the required specific atomic capability. Evaluation and experimental results on advanced models show many interesting discoveries and inspirations about the different performances of models on various atomic capabilities and the interactions between atomic capabilities. Our findings highlight the importance of decoupling mathematical intelligence into atomic components, providing new insights into model cognition and guiding the development of training strategies toward a more efficient, transferable, and cognitively grounded paradigm of “atomic thinking”.

Method: Proposes entity-aware control codes for controllable generation using either in-context learning (ICL) or prefix tuning with custom masking strategy and loss function.

Result: Experiments on legal and clinical datasets show the method achieves strong balance between privacy protection and utility.

Conclusion: Offers a practical and effective solution for synthetic text generation in sensitive domains with privacy preservation.

Abstract: Text anonymization is essential for responsibly developing and deploying AI in high-stakes domains such as healthcare, social services, and law. In this work, we propose a novel methodology for privacy-preserving synthetic text generation that leverages the principles of de-identification and the Hiding In Plain Sight (HIPS) theory. Our approach introduces entity-aware control codes to guide controllable generation using either in-context learning (ICL) or prefix tuning. The ICL variant ensures privacy levels consistent with the underlying de-identification system, while the prefix tuning variant incorporates a custom masking strategy and loss function to support scalable, high-quality generation. Experiments on legal and clinical datasets demonstrate that our method achieves a strong balance between privacy protection and utility, offering a practical and effective solution for synthetic text generation in sensitive domains.

Method: Progressive Dialogue Synthesis extracts goals, resources, and solutions from client self-reports and generates stage-aligned counseling dialogues incrementally. Memory-Driven Dynamic Planning integrates memory enhancement, global planning, and strategy reasoning with a multi-agent optimizer to attach explicit chain-of-thought to each dialogue turn.

Result: Extensive experiments and human evaluations demonstrate that CATCH significantly enhances fidelity and logical coherence in AI counseling.

Conclusion: CATCH effectively addresses limitations of existing AI counseling approaches by improving therapy fidelity through structured dialogue generation and capturing decision-making rationale through memory-driven planning.

Abstract: Recently, advancements in AI counseling based on large language models have shown significant progress. However, existing studies employ a one-time generation approach to synthesize multi-turn dialogue samples, resulting in low therapy fidelity and failing to capture the decision-making rationale behind each response. In this work, we propose CATCH, a novel data synthesis framework designed to address these challenges. Specifically, to improve therapy fidelity, we introduce the Progressive Dialogue Synthesis strategy, which extracts goals, resources, and solutions from a client’s self-report, organizes them into structured outlines, and then incrementally generates stage-aligned counseling dialogues. To capture decision-making rationale behind each response, we propose the Memory-Driven Dynamic Planning thinking pattern that integrates memory enhancement, global planning, and strategy reasoning; a collaborative multi-agent optimizer then leverages MDP to attach explicit chain-of-thought to each dialogue turn. Extensive experiments and human evaluations demonstrate that CATCH significantly enhances fidelity and logical coherence in AI counseling.

Method: Multi-stage framework with retriever, base generator, and refinement model using domain knowledge graphs, with RAGAS-based scoring for quality filtering.

Result: Successfully generated complex troubleshooting solution plans for telecom RAN without human intervention, producing high-quality dataset for reinforcement fine-tuning.

Conclusion: Scalable solution for building instruction and reinforcement datasets in specialized domains, reducing manual labeling dependence while maintaining technical fidelity.

Abstract: The success of large language models (LLMs) depends heavily on large-scale, high-quality instruction-following and reinforcement datasets. However, generating such data through human annotation is prohibitively time-consuming particularly for domain-specific tasks like telecom network troubleshooting, where accurate responses require deep technical expertise and contextual understanding. In this paper, we present a fully automated, retrieval-augmented pipeline for generating synthetic question-answer (QA) pairs grounded in structured domain knowledge. Our multi-stage framework integrates a retriever, base generator, and refinement model to synthesize and enhance QA pairs using documents retrieved from a domain-specific knowledge graph. To ensure data quality, we employ customized RAGAS-based scoring to filter low-quality samples, producing a high-quality dataset suitable for reinforcement fine-tuning (RFT). We demonstrate our approach in a real-world telecom scenario focused on radio access network (RAN) troubleshooting. The resulting pipeline generates complex, context-rich troubleshooting solution plans without human intervention. This work offers a scalable solution for building instruction and reinforcement datasets in specialized domains, significantly reducing dependence on manual labeling while maintaining high technical fidelity.

Method: Leveraged transformer-based XLM-RoBERTa model for multiclass hope speech detection, evaluated on PolyHope dataset for PolyHope-M 2025 shared task across three languages.

Result: Achieved competitive performance across all languages and significantly outperformed prior state-of-the-art techniques in terms of macro F1 scores.

Conclusion: The approach contributes to developing multilingual, fine-grained hope speech detection models that can enhance positive content moderation and foster supportive online communities, though challenges remain in low-resource languages.

Abstract: The detection of hopeful speech in social media has emerged as a critical task for promoting positive discourse and well-being. In this paper, we present a machine learning approach to multiclass hope speech detection across multiple languages, including English, Urdu, and Spanish. We leverage transformer-based models, specifically XLM-RoBERTa, to detect and categorize hope speech into three distinct classes: Generalized Hope, Realistic Hope, and Unrealistic Hope. Our proposed methodology is evaluated on the PolyHope dataset for the PolyHope-M 2025 shared task, achieving competitive performance across all languages. We compare our results with existing models, demonstrating that our approach significantly outperforms prior state-of-the-art techniques in terms of macro F1 scores. We also discuss the challenges in detecting hope speech in low-resource languages and the potential for improving generalization. This work contributes to the development of multilingual, fine-grained hope speech detection models, which can be applied to enhance positive content moderation and foster supportive online communities.

Method: TruthRL uses GRPO (Generalized Reinforcement Policy Optimization) with a ternary reward system that distinguishes between correct answers, hallucinations, and abstentions. This incentivizes models to provide correct responses while enabling abstention when uncertain.

Result: Extensive experiments across four knowledge-intensive benchmarks show TruthRL reduces hallucinations by 28.9% and improves truthfulness by 21.1% compared to vanilla RL. It achieves consistent gains across various backbone models (Qwen, Llama) under both retrieval and non-retrieval setups.

Conclusion: TruthRL demonstrates that truthfulness-driven optimization achieves strong performance in both accuracy and truthfulness, highlighting the importance of learning objective design for developing truthful LLMs. Vanilla accuracy-driven methods struggle to balance factual correctness and uncertainty.

Abstract: While large language models (LLMs) have demonstrated strong performance on factoid question answering, they are still prone to hallucination and untruthful responses, particularly when tasks demand information outside their parametric knowledge. Indeed, truthfulness requires more than accuracy – models must also recognize uncertainty and abstain when unsure to avoid hallucinations. This presents a fundamental challenge for existing methods: approaches that optimize for accuracy often amplify hallucinations, while those that encourage abstention can become overly conservative, sacrificing correct answers. Both extremes ultimately compromise truthfulness. In this work, we present TruthRL, a general reinforcement learning (RL) framework that directly optimizes the truthfulness of LLMs. Specifically, we implement TruthRL using GRPO with a simple yet effective ternary reward that distinguishes correct answers, hallucinations, and abstentions. It incentivizes models to reduce hallucinations not only by providing correct responses, but also by enabling abstention when uncertain, thereby improving truthfulness. Extensive experiments across four knowledge-intensive benchmarks show that, compared to vanilla RL, TruthRL significantly reduces hallucinations by 28.9% and improves truthfulness by 21.1%, with consistent gains across various backbone models (e.g., Qwen, Llama) under both retrieval and non-retrieval setups. In-depth ablation study demonstrates that vanilla accuracy-driven methods, such as supervised fine-tuning or RL with a binary reward, struggle to balance factual correctness and uncertainty. In contrast, our proposed truthfulness-driven TruthRL achieves strong performance in both accuracy and truthfulness, underscoring the importance of learning objective design for developing truthful LLMs.

Method: The research compares DNN-based embeddings with few-shot learning LLMs on DAIC-WOZ clinical interview transcripts. For bias mitigation, fairness-aware loss functions are applied to DNN models, while in-context learning with varied prompt framing and shot counts is explored for LLMs.

Result: LLMs outperform DNN-based models in depression classification, especially for underrepresented groups like Hispanic participants. LLMs show reduced gender bias but persistent racial disparities. Worst-group loss achieves better performance-fairness balance for DNNs, while guided prompting with ethical framing helps mitigate gender bias in 1-shot LLM settings.

Conclusion: LLMs demonstrate superior performance and reduced gender bias in depression detection compared to DNN-based models, though racial disparities remain challenging. Fairness-aware techniques show varying effectiveness, with worst-group loss working best for DNNs and ethical prompting helping LLMs, but increasing shot counts doesn’t further reduce disparities.

Abstract: This paper investigates algorithmic bias in language-based models for automated depression detection, focusing on socio-demographic disparities related to gender and race/ethnicity. Models trained using deep neural networks (DNN) based embeddings are compared to few-shot learning approaches with large language models (LLMs), evaluating both performance and fairness on clinical interview transcripts from the Distress Analysis Interview Corpus/Wizard-of-Oz (DAIC-WOZ). To mitigate bias, fairness-aware loss functions are applied to DNN-based models, while in-context learning with varied prompt framing and shot counts is explored for LLMs. Results indicate that LLMs outperform DNN-based models in depression classification, particularly for underrepresented groups such as Hispanic participants. LLMs also exhibit reduced gender bias compared to DNN-based embeddings, though racial disparities persist. Among fairness-aware techniques for mitigating bias in DNN-based embeddings, the worst-group loss, which is designed to minimize loss for the worst-performing demographic group, achieves a better balance between performance and fairness. In contrast, the fairness-regularized loss minimizes loss across all groups but performs less effectively. In LLMs, guided prompting with ethical framing helps mitigate gender bias in the 1-shot setting. However, increasing the number of shots does not lead to further reductions in disparities. For race/ethnicity, neither prompting strategy nor increasing $N$ in $N$-shot learning effectively reduces disparities.

Method: Created a Romanian biology dataset, benchmarked popular LLMs, analyzed accuracy and reasoning patterns, and evaluated optimization techniques like prompt engineering and fine-tuning.

Result: The study provides comprehensive evaluation of LLM performance on specialized knowledge tasks in low-resource languages, highlighting both strengths and limitations.

Conclusion: The research offers valuable insights for improving LLM capabilities in domain-specific, non-English applications and identifies areas for future development.

Abstract: In recent years, large language models (LLMs) have demonstrated significant potential across various natural language processing (NLP) tasks. However, their performance in domain-specific applications and non-English languages remains less explored. This study introduces a novel Romanian-language dataset for multiple-choice biology questions, carefully curated to assess LLM comprehension and reasoning capabilities in scientific contexts. Containing approximately 14,000 questions, the dataset provides a comprehensive resource for evaluating and improving LLM performance in biology. We benchmark several popular LLMs, analyzing their accuracy, reasoning patterns, and ability to understand domain-specific terminology and linguistic nuances. Additionally, we perform comprehensive experiments to evaluate the impact of prompt engineering, fine-tuning, and other optimization techniques on model performance. Our findings highlight both the strengths and limitations of current LLMs in handling specialized knowledge tasks in low-resource languages, offering valuable insights for future research and development.

Method: Proposed ReTAG framework constructs topic-specific subgraphs and retrieves relevant summaries for response generation, addressing limitations of prior graph-based approaches.

Result: Experiments show ReTAG improves response quality while significantly reducing inference time compared to baseline methods.

Conclusion: ReTAG effectively addresses global sensemaking challenges in question answering through its retrieval-enhanced, topic-augmented graph approach, offering both performance improvements and computational efficiency.

Abstract: Recent advances in question answering have led to substantial progress in tasks such as multi-hop reasoning. However, global sensemaking-answering questions by synthesizing information from an entire corpus remains a significant challenge. A prior graph-based approach to global sensemaking lacks retrieval mechanisms, topic specificity, and incurs high inference costs. To address these limitations, we propose ReTAG, a Retrieval-Enhanced, Topic-Augmented Graph framework that constructs topic-specific subgraphs and retrieves the relevant summaries for response generation. Experiments show that ReTAG improves response quality while significantly reducing inference time compared to the baseline. Our code is available at https://github.com/bykimby/retag.

Method: Using author profile data combined with relevant metadata to improve multimodal large language models for personalized caption generation.

Result: Rich author profile data significantly improves personalization performance, but reveals a fundamental trade-off between matching author style and maintaining caption quality.

Conclusion: The findings provide valuable insights for developing practical caption automation systems that balance both personalization and quality objectives.

Abstract: We study personalized figure caption generation using author profile data from scientific papers. Our experiments demonstrate that rich author profile data, combined with relevant metadata, can significantly improve the personalization performance of multimodal large language models. However, we also reveal a fundamental trade-off between matching author style and maintaining caption quality. Our findings offer valuable insights and future directions for developing practical caption automation systems that balance both objectives. This work was conducted as part of the 3rd SciCap challenge.

Method: Introduces DECS framework with: (1) decoupled token-level reward mechanism that surgically penalizes redundant tokens while preserving essential exploratory tokens, and (2) curriculum batch scheduling strategy to balance efficiency and efficacy.

Result: Achieves over 50% reduction in reasoning tokens across seven benchmarks while maintaining or improving performance, demonstrating substantial efficiency gains without compromising reasoning power.

Conclusion: DECS proves that significant reasoning efficiency improvements are achievable without sacrificing model performance, addressing the critical overthinking problem in large reasoning models.

Abstract: While large reasoning models trained with critic-free reinforcement learning and verifiable rewards (RLVR) represent the state-of-the-art, their practical utility is hampered by ``overthinking’’, a critical issue where models generate excessively long reasoning paths without any performance benefit. Existing solutions that penalize length often fail, inducing performance degradation due to a fundamental misalignment between trajectory-level rewards and token-level optimization. In this work, we introduce a novel framework, DECS, built on our theoretical discovery of two previously unaddressed flaws in current length rewards: (1) the erroneous penalization of essential exploratory tokens and (2) the inadvertent rewarding of partial redundancy. Our framework’s innovations include (i) a first-of-its-kind decoupled token-level reward mechanism that surgically distinguishes and penalizes redundant tokens, and (ii) a novel curriculum batch scheduling strategy to master the efficiency-efficacy equilibrium. Experimental results show DECS can achieve a dramatic reduction in reasoning tokens by over 50% across seven benchmarks while simultaneously maintaining or even improving performance. It demonstrates conclusively that substantial gains in reasoning efficiency can be achieved without compromising a model’s underlying reasoning power.

Method: Proposed two complementary quality scoring functions: Visual Fidelity (faithfulness to visual context) and Contrastiveness (identifying distinguishing visual details). Evaluated these on A-OKVQA and VizWiz tasks and conducted a user study where participants assessed VLM predictions without visual context.

Result: The quality scoring functions showed better calibration with model correctness than existing methods. User study showed 11.1% improvement in participants’ accuracy at predicting VLM correctness and 15.4% reduction in falsely believing incorrect predictions when quality scores were shown alongside explanations.

Conclusion: Explanation quality scores are effective in fostering appropriate reliance on VLM predictions, particularly for users who cannot access visual context, by helping them better distinguish between reliable and unreliable model outputs.

Abstract: When people query Vision-Language Models (VLMs) but cannot see the accompanying visual context (e.g. for blind and low-vision users), augmenting VLM predictions with natural language explanations can signal which model predictions are reliable. However, prior work has found that explanations can easily convince users that inaccurate VLM predictions are correct. To remedy undesirable overreliance on VLM predictions, we propose evaluating two complementary qualities of VLM-generated explanations via two quality scoring functions. We propose Visual Fidelity, which captures how faithful an explanation is to the visual context, and Contrastiveness, which captures how well the explanation identifies visual details that distinguish the model’s prediction from plausible alternatives. On the A-OKVQA and VizWiz tasks, these quality scoring functions are better calibrated with model correctness than existing explanation qualities. We conduct a user study in which participants have to decide whether a VLM prediction is accurate without viewing its visual context. We observe that showing our quality scores alongside VLM explanations improves participants’ accuracy at predicting VLM correctness by 11.1%, including a 15.4% reduction in the rate of falsely believing incorrect predictions. These findings highlight the utility of explanation quality scores in fostering appropriate reliance on VLM predictions.

Method: Created ReFACT benchmark with 1,001 expert-annotated question-answer pairs spanning diverse scientific domains, including both correct answers and non-factual counterparts with precise error spans and error-types.

Result: Benchmarked 9 state-of-the-art LLMs showing limited performance (~50% accuracy), with even top models like GPT-4o failing to distinguish factual from confabulated scientific answers.

Conclusion: Highlights the need for fine-grained, human-validated benchmarks to detect and correct scientific confabulation, raising concerns about LLM-as-judge evaluation paradigms’ reliability.

Abstract: Large Language Models (LLMs) frequently confabulate scientific facts,severely undermining their trustworthiness. Addressing this challenge requires benchmarks that go beyond binary factuality and enable fine-grained evaluation. We introduce \textbf{ReFACT} (\textit{Reddit False And Correct Texts}), a benchmark of 1,001 expert-annotated question–answer pairs spanning diverse scientific domains for the detection of scientific confabulation. Each instance includes both a scientifically correct answer and a non-factual counterpart annotated with \textbf{precise error spans and error-types}. ReFACT enables multi-stage evaluation: (1) confabulation detection, (2) fine-grained error localization, and (3) correction. We benchmark 9 state-of-the-art LLMs, revealing limited performance ($\sim$50% accuracy). Even top models such as GPT-4o fail to distinguish factual from confabulated scientific answers, raising concerns about the reliability of \textit{LLM-as-judge} evaluation paradigms. Our findings highlight the need for fine-grained, human-validated benchmarks to detect and correct scientific confabulation in domain-specific contexts. Dataset is released on \href{https://github.com/ddz5431/ReFACT}{GitHub}\footnote{We provide the dataset at: https://github.com/ddz5431/ReFACT}.

Method: Experimented with multiple training strategies including synthetic noise augmentation and SpecAugment, evaluating performance across various signal-to-noise ratios (SNRs).

Result: Noise-aware training substantially improves robustness, particularly for larger Whisper models. Error analysis reveals language-specific challenges.

Conclusion: The study demonstrates the importance of noise-aware training for robust ASR in regional languages and identifies specific challenges for future improvements.

Abstract: We investigate the robustness of Whisper-based automatic speech recognition (ASR) models for two major Indonesian regional languages: Javanese and Sundanese. While recent work has demonstrated strong ASR performance under clean conditions, their effectiveness in noisy environments remains unclear. To address this, we experiment with multiple training strategies, including synthetic noise augmentation and SpecAugment, and evaluate performance across a range of signal-to-noise ratios (SNRs). Our results show that noise-aware training substantially improves robustness, particularly for larger Whisper models. A detailed error analysis further reveals language-specific challenges, highlighting avenues for future improvements

Method: Introduced RoleConflictBench with a three-stage pipeline generating over 13K realistic role conflict scenarios across 65 roles, systematically varying role expectations and situational urgency levels. Analyzed model choices across 10 different LLMs.

Result: LLMs show some capacity to respond to contextual cues but this sensitivity is insufficient. Their decisions are predominantly governed by inherent bias related to social roles rather than situational information, with dominant preference for Family and Occupation domains, male roles, and Abrahamic religions.

Conclusion: LLMs demonstrate limited contextual sensitivity in role conflicts and are heavily influenced by inherent social role biases, highlighting the need for improved contextual understanding in social decision-making scenarios.

Abstract: Humans often encounter role conflicts – social dilemmas where the expectations of multiple roles clash and cannot be simultaneously fulfilled. As large language models (LLMs) become increasingly influential in human decision-making, understanding how they behave in complex social situations is essential. While previous research has evaluated LLMs’ social abilities in contexts with predefined correct answers, role conflicts represent inherently ambiguous social dilemmas that require contextual sensitivity: the ability to recognize and appropriately weigh situational cues that can fundamentally alter decision priorities. To address this gap, we introduce RoleConflictBench, a novel benchmark designed to evaluate LLMs’ contextual sensitivity in complex social dilemmas. Our benchmark employs a three-stage pipeline to generate over 13K realistic role conflict scenarios across 65 roles, systematically varying their associated expectations (i.e., their responsibilities and obligations) and situational urgency levels. By analyzing model choices across 10 different LLMs, we find that while LLMs show some capacity to respond to these contextual cues, this sensitivity is insufficient. Instead, their decisions are predominantly governed by a powerful, inherent bias related to social roles rather than situational information. Our analysis quantifies these biases, revealing a dominant preference for roles within the Family and Occupation domains, as well as a clear prioritization of male roles and Abrahamic religions across most evaluatee models.

Method: Proposed PerQ metric for evaluating personalization quality of text generated by language models.

Result: Case study comparing large and small language models demonstrates PerQ’s effectiveness in reducing resource waste.

Conclusion: PerQ provides an efficient alternative to costly multi-LLM meta-evaluation for assessing text personalization quality.

Abstract: Since no metrics are available to evaluate specific aspects of a text, such as its personalization quality, the researchers often rely solely on large language models to meta-evaluate such texts. Due to internal biases of individual language models, it is recommended to use multiple of them for combined evaluation, which directly increases costs of such meta-evaluation. In this paper, a computationally efficient method for evaluation of personalization quality of a given text (generated by a language model) is introduced, called PerQ. A case study of comparison of generation capabilities of large and small language models shows the usability of the proposed metric in research, effectively reducing the waste of resources.

Method: Proposes Mem-alpha RL framework that trains agents through interaction and feedback. Uses specialized dataset with multi-turn interaction patterns and evaluation questions. Agents process sequential information chunks, learn to extract/store relevant content, and update memory system. Reward signal based on downstream QA accuracy over full interaction history.

Result: Achieves significant improvements over existing memory-augmented agent baselines. Despite training on max 30k token sequences, agents generalize remarkably to sequences exceeding 400k tokens (13x training length).

Conclusion: Mem-alpha demonstrates robust memory management capabilities and strong generalization to much longer sequences than seen during training, highlighting the effectiveness of the reinforcement learning approach for complex memory system management.

Abstract: Large language model (LLM) agents are constrained by limited context windows, necessitating external memory systems for long-term information understanding. Current memory-augmented agents typically depend on pre-defined instructions and tools for memory updates. However, language models may lack the ability to determine which information to store, how to structure it, and when to update it, especially as memory systems become more complex. This results in suboptimal memory construction and information loss. To this end, we propose Mem-alpha, a reinforcement learning framework that trains agents to effectively manage complex memory systems through interaction and feedback. We also construct a specialized training dataset spanning diverse multi-turn interaction patterns paired with comprehensive evaluation questions designed to teach effective memory management. During training, agents process sequential information chunks, learn to extract and store relevant content, then update the memory system. The reward signal derives from downstream question-answering accuracy over the full interaction history, directly optimizing for memory construction. To illustrate the effectiveness of our training framework, we design a memory architecture comprising core, episodic, and semantic components, equipped with multiple tools for memory operations. Empirical evaluation demonstrates that Mem-alpha achieves significant improvements over existing memory-augmented agent baselines. Despite being trained exclusively on instances with a maximum length of 30k tokens, our agents exhibit remarkable generalization to sequences exceeding 400k tokens, over 13x the training length, highlighting the robustness of Mem-alpha.

Method: Developed KERN router based on FFN architecture with ReLU activation and L2-normalization, inspired by mathematical connections between MoE and Nadaraya-Watson regression.

Result: Comprehensive experiments in MoE and LLM settings validated the effectiveness of the proposed FFN-style router function.

Conclusion: KERN provides a principled alternative to Softmax routing in MoE models, offering zero-additional-cost implementation while generalizing existing router approaches.

Abstract: Mixture-of-Experts (MoE) has become a cornerstone in recent state-of-the-art large language models (LLMs). Traditionally, MoE relies on $\mathrm{Softmax}$ as the router score function to aggregate expert output, a designed choice that has persisted from the earliest MoE models to modern LLMs, and is now widely regarded as standard practice. However, the necessity of using $\mathrm{Softmax}$ to project router weights into a probability simplex remains an unchallenged assumption rather than a principled design choice. In this work, we first revisit the classical Nadaraya-Watson regression and observe that MoE shares the same mathematical formulation as Nadaraya-Watson regression. Furthermore, we show that both feed-forward neural network (FFN) and MoE can be interpreted as a special case of Nadaraya-Watson regression, where the kernel function corresponds to the input neurons of the output layer. Motivated by these insights, we propose the \textbf{zero-additional-cost} Kernel Inspired Router with Normalization (KERN), an FFN-style router function, as an alternative to $\mathrm{Softmax}$. We demonstrate that this router generalizes both $\mathrm{Sigmoid}$- and $\mathrm{Softmax}$-based routers. \textbf{Based on empirical observations and established practices in FFN implementation, we recommend the use of $\mathrm{ReLU}$ activation and $\ell_2$-normalization in $\mathrm{KERN}$ router function.} Comprehensive experiments in MoE and LLM validate the effectiveness of the proposed FFN-style router function \methodNorm.

Method: Study adaptation of xLSTMs, structured state-space models, diffusion models, and adversarial learning across tagging tasks with different complexities, evaluating on multiple languages and datasets.

Result: Strong performance observed in simpler settings doesn’t generalize well across languages or datasets, and doesn’t extend to more complex structured tasks.

Conclusion: Alternative architectures that show promise in language modeling face challenges when applied to diverse and complex sequence labeling tasks, indicating limitations in their generalization capabilities.

Abstract: Pretrained Transformer encoders are the dominant approach to sequence labeling. While some alternative architectures-such as xLSTMs, structured state-space models, diffusion models, and adversarial learning-have shown promise in language modeling, few have been applied to sequence labeling, and mostly on flat or simplified tasks. We study how these architectures adapt across tagging tasks that vary in structural complexity, label space, and token dependencies, with evaluation spanning multiple languages. We find that the strong performance previously observed in simpler settings does not always generalize well across languages or datasets, nor does it extend to more complex structured tasks.

Method: Proposed adversarial attack strategies for four reference-free metrics: SOME, Scribendi, IMPARA, and LLM-based metrics, and demonstrated that these adversarial systems outperform current state-of-the-art.

Result: The adversarial systems successfully exploited the four reference-free metrics, achieving higher scores than legitimate systems, demonstrating the vulnerability of current evaluation methods.

Conclusion: The findings reveal significant vulnerabilities in current reference-free GEC metrics and emphasize the urgent need for developing more robust evaluation methods that can withstand adversarial attacks.

Abstract: Reference-free evaluation metrics for grammatical error correction (GEC) have achieved high correlation with human judgments. However, these metrics are not designed to evaluate adversarial systems that aim to obtain unjustifiably high scores. The existence of such systems undermines the reliability of automatic evaluation, as it can mislead users in selecting appropriate GEC systems. In this study, we propose adversarial attack strategies for four reference-free metrics: SOME, Scribendi, IMPARA, and LLM-based metrics, and demonstrate that our adversarial systems outperform the current state-of-the-art. These findings highlight the need for more robust evaluation methods.

Method: Repurpose question-answering datasets into preference pairs that prioritize groundedness over stylistic features, then fine-tune reward models ranging from 7B to 24B parameters on the curated 4K-sample dataset.

Result: RAG-centric reward models achieve state-of-the-art performance on ContextualJudgeBench, surpassing existing 70B+ models trained on much larger general corpora with +15.5% absolute improvement.

Conclusion: RAGferee enables effective training of specialized contextual reward models for RAG settings using small, targeted datasets, demonstrating superior performance over larger general-purpose models.

Abstract: Existing Reward Models (RMs), typically trained on general preference data, struggle in Retrieval Augmented Generation (RAG) settings, which require judging responses for faithfulness to retrieved context, relevance to the user query, appropriate refusals when context is insufficient, completeness and conciseness of information. To address the lack of publicly available RAG-centric preference datasets and specialised RMs, we introduce RAGferee, a methodology that repurposes question-answering (QA) datasets into preference pairs that prioritise groundedness over stylistic features, enabling the training of contextual RMs better suited to judging RAG responses. Using RAGferee, we curate a small preference dataset of 4K samples and fine-tune RMs ranging from 7B to 24B parameters. Our RAG-centric RMs achieve state-of-the-art performance on ContextualJudgeBench, surpassing existing 70B+ RMs trained on much larger (up to 2.4M samples) general corpora, with an absolute improvement of +15.5%.

Method: Propose RE² method that retrieves examples using grammatical error explanations rather than text similarity, creating a grammatical error explanation dataset for reference.

Result: Experimental results on two CGEC datasets show the method effectively improves CGEC performance.

Conclusion: Using grammatical error explanations for example retrieval is more effective than text similarity for improving Chinese grammatical error correction with LLMs.

Abstract: The primary objective of Chinese grammatical error correction (CGEC) is to detect and correct errors in Chinese sentences. Recent research shows that large language models (LLMs) have been applied to CGEC with significant results. For LLMs, selecting appropriate reference examples can help improve their performance. However, existing methods predominantly rely on text similarity for example retrieval, a strategy that frequently mismatches actual error patterns and retrieves lexically similar yet grammatically irrelevant sentences. To address this problem, we propose a method named RE$^2$, which retrieves appropriate examples with explanations of grammatical errors. Instead of using text similarity of the input sentence, we use explanations of grammatical errors to select reference examples, which are used by LLMs to improve the performance of CGEC. We conduct experiments on two CGEC datasets and create a high-quality grammatical error explanation (GEE) dataset, which is not only used in our research but also serves as a valuable resource for future studies in both CGEC and GEE. The experimental results on the two datasets indicate that our proposed method effectively improves the performance of CGEC.

Method: Systematic study across 4 datasets (AIME, GSM-Hard, MATH-500, UniADILR) using GPT-4o and Gemini-2-Flash, with controlled hint manipulations varying in correctness, presentation style, and complexity.

Result: Correct hints improve accuracy especially on harder tasks, while incorrect hints reduce accuracy. Hint acknowledgement varies by complexity - equation hints are referenced but raw hints are adopted silently. Presentation style affects acknowledgement patterns.

Conclusion: LLM reasoning is systematically shaped by shortcuts that obscure faithfulness, with hint complexity and presentation style influencing both accuracy and transparency of reliance.

Abstract: Large language models (LLMs) increasingly rely on chain-of-thought (CoT) prompting to solve mathematical and logical reasoning tasks. Yet, a central question remains: to what extent are these generated rationales \emph{faithful} to the underlying computations, rather than post-hoc narratives shaped by hints that function as answer shortcuts embedded in the prompt? Following prior work on hinted vs.\ unhinted prompting, we present a systematic study of CoT faithfulness under controlled hint manipulations. Our experimental design spans four datasets (AIME, GSM-Hard, MATH-500, UniADILR), two state-of-the-art models (GPT-4o and Gemini-2-Flash), and a structured set of hint conditions varying in correctness (correct and incorrect), presentation style (sycophancy and data leak), and complexity (raw answers, two-operator expressions, four-operator expressions). We evaluate both task accuracy and whether hints are explicitly acknowledged in the reasoning. Our results reveal three key findings. First, correct hints substantially improve accuracy, especially on harder benchmarks and logical reasoning, while incorrect hints sharply reduce accuracy in tasks with lower baseline competence. Second, acknowledgement of hints is highly uneven: equation-based hints are frequently referenced, whereas raw hints are often adopted silently, indicating that more complex hints push models toward verbalizing their reliance in the reasoning process. Third, presentation style matters: sycophancy prompts encourage overt acknowledgement, while leak-style prompts increase accuracy but promote hidden reliance. This may reflect RLHF-related effects, as sycophancy exploits the human-pleasing side and data leak triggers the self-censoring side. Together, these results demonstrate that LLM reasoning is systematically shaped by shortcuts in ways that obscure faithfulness.

Method: RE-Searcher explicitly articulates concrete search goals and reflects on whether retrieved evidence satisfies those goals. This goal-oriented planning with self-reflection helps resist spurious cues in complex search environments.

Result: Extensive experiments show improved search accuracy and state-of-the-art results. Perturbation studies demonstrate substantial resilience to noisy or misleading external signals, mitigating search process fragility.

Conclusion: The findings offer practical guidance for integrating LLM-powered agents into complex interactive environments and enabling more autonomous decision-making.

Abstract: Large language models (LLMs) excel at knowledge-intensive question answering and reasoning, yet their real-world deployment remains constrained by knowledge cutoff, hallucination, and limited interaction modalities. Augmenting LLMs with external search tools helps alleviate these issues, but it also exposes agents to a complex search environment in which small, plausible variations in query formulation can steer reasoning into unproductive trajectories and amplify errors. We present a systematic analysis that quantifies how environmental complexity induces fragile search behaviors and, in turn, degrades overall performance. To address this challenge, we propose a simple yet effective approach to instantiate a search agent, RE-Searcher. During search, RE-Searcher explicitly articulates a concrete search goal and subsequently reflects on whether the retrieved evidence satisfies that goal. This combination of goal-oriented planning and self-reflection enables RE-Searcher to resist spurious cues in complex search environments and perform robust search. Extensive experiments show that our method improves search accuracy and achieves state-of-the-art results. Perturbation studies further demonstrate substantial resilience to noisy or misleading external signals, mitigating the fragility of the search process. We believe these findings offer practical guidance for integrating LLM-powered agents into more complex interactive environments and enabling more autonomous decision-making.

Method: Created benchmark with multi-domain, multi-generator, and multilingual evaluation; compared train-language combinations; tested adversarial robustness against obfuscation.

Result: Supervised fine-tuned detectors in Central European languages performed best in these languages and were most resistant to obfuscation attacks.

Conclusion: Language-specific supervised fine-tuning is crucial for effective machine-generated text detection in Central European languages, outperforming cross-lingual transfer approaches.

Abstract: Machine-generated text detection, as an important task, is predominantly focused on English in research. This makes the existing detectors almost unusable for non-English languages, relying purely on cross-lingual transferability. There exist only a few works focused on any of Central European languages, leaving the transferability towards these languages rather unexplored. We fill this gap by providing the first benchmark of detection methods focused on this region, while also providing comparison of train-languages combinations to identify the best performing ones. We focus on multi-domain, multi-generator, and multilingual evaluation, pinpointing the differences of individual aspects, as well as adversarial robustness of detection methods. Supervised finetuned detectors in the Central European languages are found the most performant in these languages as well as the most resistant against obfuscation.

Method: DyFlow uses a designer-executor architecture: the designer decomposes problems into sub-goals and dynamically plans next steps, while the executor uses context-aware dynamic operators to carry out operations.

Result: DyFlow significantly outperforms baselines across social reasoning, biomedical tasks, math problem solving, and code generation, achieving substantial Pass@k improvements and robust cross-domain generalization.

Conclusion: The framework demonstrates that dynamic workflow generation with real-time feedback enables more flexible, semantically grounded reasoning and better generalization across diverse domains.

Abstract: Agent systems based on large language models (LLMs) have shown great potential in complex reasoning tasks, but building efficient and generalizable workflows remains a major challenge. Most existing approaches rely on manually designed processes, which limits their adaptability across different tasks. While a few methods attempt automated workflow generation, they are often tied to specific datasets or query types and make limited use of intermediate feedback, reducing system robustness and reasoning depth. Moreover, their operations are typically predefined and inflexible. To address these limitations, we propose DyFlow, a dynamic workflow generation framework that adaptively constructs and adjusts reasoning procedures based on task requirements and real-time intermediate feedback, thereby enhancing cross-task generalization. DyFlow consists of two core components: a designer and an executor. The designer decomposes complex problems into a sequence of sub-goals defined by high-level objectives and dynamically plans the next steps based on intermediate outputs and feedback. These plans are then carried out by the executor, which executes each operation using dynamic operators with context-aware parameterization, enabling flexible and semantically grounded reasoning. We systematically evaluate DyFlow across diverse domains, including social reasoning, biomedical tasks, mathematical problem solving, and code generation. Results demonstrate that DyFlow significantly outperforms existing baselines, achieving substantial Pass@k improvements and exhibiting robust generalization across diverse domains. The code is publicly available at https://github.com/wyf23187/DyFlow.

Method: Used ELI5 and LitBench datasets with pairwise comparisons, injected provenance and recency cues into responses, and tested GPT-4o and Gemini-2.5-Flash as evaluators.

Result: Both models showed strong recency bias (favoring new over old) and provenance hierarchy (Expert > Human > LLM > Unknown), with minimal acknowledgment of these cues in justifications.

Conclusion: Current LLM-as-a-judge systems are shortcut-prone and unfaithful, undermining their reliability as evaluators in research and deployment.

Abstract: Large language models (LLMs) are increasingly deployed as automatic judges to evaluate system outputs in tasks such as summarization, dialogue, and creative writing. A faithful judge should base its verdicts solely on response quality and explicitly acknowledge the factors shaping its decision. We show that current LLM judges fail on both counts by relying on shortcuts introduced in the prompt. Our study uses two evaluation datasets: ELI5, a benchmark for long-form question answering, and LitBench, a recent benchmark for creative writing. Both datasets provide pairwise comparisons, where the evaluator must choose which of two responses is better. From each dataset we construct 100 pairwise judgment tasks and employ two widely used models, GPT-4o and Gemini-2.5-Flash, as evaluators in the role of LLM-as-a-judge. For each pair, we assign superficial cues to the responses, provenance cues indicating source identity (Human, Expert, LLM, or Unknown) and recency cues indicating temporal origin (Old, 1950 vs. New, 2025), while keeping the rest of the prompt fixed. Results reveal consistent verdict shifts: both models exhibit a strong recency bias, systematically favoring new responses over old, as well as a clear provenance hierarchy (Expert > Human > LLM > Unknown). These biases are especially pronounced in GPT-4o and in the more subjective and open-ended LitBench domain. Crucially, cue acknowledgment is rare: justifications almost never reference the injected cues, instead rationalizing decisions in terms of content qualities. These findings demonstrate that current LLM-as-a-judge systems are shortcut-prone and unfaithful, undermining their reliability as evaluators in both research and deployment.

Method: Uses Variational Autoencoder (VAE) to learn structured latent representation of response embeddings, then performs controlled perturbations in latent space and decodes back to embedding space to generate synthetic preference pairs.

Result: Significantly outperforms text-based augmentation on standard benchmarks, achieves superior results while being 18x faster in generation and using 16,000x smaller model.

Conclusion: Provides a scalable and effective alternative for enhancing reward modeling through efficient data augmentation in latent space.

Abstract: Reward modeling, crucial for aligning large language models (LLMs) with human preferences, is often bottlenecked by the high cost of preference data. Existing textual data synthesis methods are computationally expensive. We propose a novel framework LENS for synthesizing preference data directly in the LLM’s latent embedding space. Our method employs a Variational Autoencoder (VAE) to learn a structured latent representation of response embeddings. By performing controlled perturbations in this latent space and decoding back to the embedding space, we efficiently generate diverse, semantically consistent synthetic preference pairs, bypassing costly text generation and annotation. We provide theoretical guarantees that our synthesized pairs approximately preserve original preference ordering and improve reward model generalization. Empirically, our latent-space synthesis significantly outperforms text-based augmentation on standard benchmarks, achieving superior results while being 18x faster in generation and using a 16,000x smaller model. Our work offers a scalable and effective alternative for enhancing reward modeling through efficient data augmentation. Code is publicly available at https://github.com/deeplearning-wisc/lens

Method: Created a private benchmark with 39 peer-reviewed problems requiring detailed proofs, paired with subproblems for automated grading. Evaluation uses an agentic framework with tools like web search and SageMath to simulate realistic research environments.

Result: Current LLMs succeed at accessible research-level questions but struggle with challenging problems. Grok-4 achieved 52% accuracy on final-answer subproblems, while GPT-5 performed best for proof generation with 22% fully correct solutions.

Conclusion: IMProofBench addresses the gap in evaluating LLMs on research-level mathematics and will evolve as a dynamic benchmark in collaboration with the mathematical community to ensure relevance for future LLM evaluation.

Abstract: As the mathematical capabilities of large language models (LLMs) improve, it becomes increasingly important to evaluate their performance on research-level tasks at the frontier of mathematical knowledge. However, existing benchmarks are limited, as they focus solely on final-answer questions or high-school competition problems. To address this gap, we introduce IMProofBench, a private benchmark consisting of 39 peer-reviewed problems developed by expert mathematicians. Each problem requires a detailed proof and is paired with subproblems that have final answers, supporting both an evaluation of mathematical reasoning capabilities by human experts and a large-scale quantitative analysis through automated grading. Furthermore, unlike prior benchmarks, the evaluation setup simulates a realistic research environment: models operate in an agentic framework with tools like web search for literature review and mathematical software such as SageMath. Our results show that current LLMs can succeed at the more accessible research-level questions, but still encounter significant difficulties on more challenging problems. Quantitatively, Grok-4 achieves the highest accuracy of 52% on final-answer subproblems, while GPT-5 obtains the best performance for proof generation, achieving a fully correct solution for 22% of problems. IMProofBench will continue to evolve as a dynamic benchmark in collaboration with the mathematical community, ensuring its relevance for evaluating the next generation of LLMs.

Method: Hierarchical framework with Planner for strategy selection (recommend, explain, encourage) and Actor for response generation guided by auxiliary experts, using reinforcement learning with LLM-based rewards for strategy learning.

Result: RSO outperforms state-of-the-art baselines in experiments, demonstrating the effectiveness of hierarchical strategy optimization.

Conclusion: The proposed RSO framework successfully enables more tractable learning and better performance in conversational recommender systems through explicit strategy optimization.

Abstract: Conversational Recommender Systems (CRSs) provide personalized recommendations through multi-turn interactions. With the strong reasoning abilities of Large Language Models (LLMs), applying them to CRSs has become promising. Yet, existing methods often lack explicit optimization of interaction strategies, relying instead on unified prompts, which can yield suboptimal outcomes. We propose Reinforced Strategy Optimization (RSO), a hierarchical framework that decomposes response generation into macro-level strategy planning and micro-level adaptation within a network-of-experts. A Planner selects strategies (e.g., recommend, explain, encourage), while an Actor generates responses guided by auxiliary experts for preferences and factual grounding. This disentanglement enables more tractable learning. To address limited multi-turn data, we model strategy learning as reinforcement learning with an LLM-based reward for exploration. Experiments show RSO outperforms state-of-the-art baselines, validating the effectiveness of hierarchical strategy optimization.

Method: Extracts aspect-sentiment pairs from reviews, selects most frequent aspects per product, samples representative reviews, and uses structured prompts to guide LLM summarization.

Result: Successfully deployed system with real-time capabilities, validated through large-scale A/B testing, and released a dataset of 11.8 million anonymized reviews with extracted aspects and generated summaries.

Conclusion: The system effectively combines ABSA with guided summarization to produce grounded review summaries and provides a valuable dataset for future research in aspect-guided review summarization.

Abstract: We present a scalable large language model (LLM)-based system that combines aspect-based sentiment analysis (ABSA) with guided summarization to generate concise and interpretable product review summaries for the Wayfair platform. Our approach first extracts and consolidates aspect-sentiment pairs from individual reviews, selects the most frequent aspects for each product, and samples representative reviews accordingly. These are used to construct structured prompts that guide the LLM to produce summaries grounded in actual customer feedback. We demonstrate the real-world effectiveness of our system through a large-scale online A/B test. Furthermore, we describe our real-time deployment strategy and release a dataset of 11.8 million anonymized customer reviews covering 92,000 products, including extracted aspects and generated summaries, to support future research in aspect-guided review summarization.

Method: Design an algorithm to extend existing tokenizers by adding domain-specific tokens while guaranteeing no decrease in tokenization efficiency - every input sequence gets segmented into at most the same number of tokens as before.

Result: On e-Commerce use-cases, the augmented tokenizer shortens input sequences by up to 20%, reduces inference latency on downstream tasks, and preserves predictive quality while improving forward pass speed and token adoption rates.

Conclusion: Vocabulary adaptation through domain-specific token augmentation is an effective approach to improve LLM efficiency in specialized domains without sacrificing performance.

Abstract: When using an LLM to process text outside the training domain(s), an often overlooked factor is vocabulary mismatch, where the general-domain tokenizer fails to capture frequent domain-specific terms, leading to higher token fertility and thus a decrease in processing speed due to suboptimal sub-word splits. We address this limitation by augmenting the pretrained vocabulary with a set of domain-specific tokens. To this end, we design an algorithm that extends an existing tokenizer while guaranteeing it never decreases tokenization efficiency: every input sequence is segmented into at most the same number of tokens as before. Evaluated on real-world e-Commerce use-cases, the augmented tokenizer significantly shortens input sequences by up to 20% and reduces inference latency on downstream tasks while preserving predictive quality. We further analyze secondary effects, such as the impact on forward pass speed and the rate at which the model adopts the newly introduced tokens, to illustrate the broader benefits of vocabulary adaptation.

Method: Proposed HATE (Hunger Game Debate), a novel experimental framework simulating debates in zero-sum competition arenas, tested across various LLMs and tasks with different judge variants to study environmental feedback effects.

Result: Competitive pressure significantly stimulates over-competition behaviors, degrades task performance, and derails discussions. Objective, task-focused feedback from judges effectively mitigates these harmful behaviors.

Conclusion: The study provides insights for understanding and governing emergent social dynamics in AI communities, characterizing top LLMs through post-hoc kindness analysis and forming a leaderboard.

Abstract: LLM-based multi-agent systems demonstrate great potential for tackling complex problems, but how competition shapes their behavior remains underexplored. This paper investigates the over-competition in multi-agent debate, where agents under extreme pressure exhibit unreliable, harmful behaviors that undermine both collaboration and task performance. To study this phenomenon, we propose HATE, the Hunger Game Debate, a novel experimental framework that simulates debates under a zero-sum competition arena. Our experiments, conducted across a range of LLMs and tasks, reveal that competitive pressure significantly stimulates over-competition behaviors and degrades task performance, causing discussions to derail. We further explore the impact of environmental feedback by adding variants of judges, indicating that objective, task-focused feedback effectively mitigates the over-competition behaviors. We also probe the post-hoc kindness of LLMs and form a leaderboard to characterize top LLMs, providing insights for understanding and governing the emergent social dynamics of AI community.

Method: Extensive comparison of 12 generative LLMs and 3 encoder-based classifiers for discharge diagnosis prediction from admission notes, with assessment of retrieval augmentation strategies for in-context learning.

Result: Encoder-based classifiers consistently outperform generative models in diagnosis prediction. Retrieval augmentation strategies provide notable performance improvements for generative LLMs.

Conclusion: While generative LLMs show promise, encoder-based classifiers remain superior for discharge diagnosis prediction tasks, though retrieval augmentation can enhance LLM performance.

Abstract: With their growing capabilities, generative large language models (LLMs) are being increasingly investigated for complex medical tasks. However, their effectiveness in real-world clinical applications remains underexplored. To address this, we present CliniBench, the first benchmark that enables comparability of well-studied encoder-based classifiers and generative LLMs for discharge diagnosis prediction from admission notes in MIMIC-IV dataset. Our extensive study compares 12 generative LLMs and 3 encoder-based classifiers and demonstrates that encoder-based classifiers consistently outperform generative models in diagnosis prediction. We assess several retrieval augmentation strategies for in-context learning from similar patients and find that they provide notable performance improvements for generative LLMs.

Method: Extracted over 4 million generic and quantified sentences from diverse textual sources including websites and academic papers, covering 11 different quantifiers.

Result: Created MGen - the biggest and most diverse dataset of naturally occurring generic sentences, with sentences averaging over 16 words and often expressing generalizations about people.

Conclusion: MGen opens the door to large-scale computational research on genericity and is publicly available for research use.

Abstract: MGen is a dataset of over 4 million naturally occurring generic and quantified sentences extracted from diverse textual sources. Sentences in the dataset have long context documents, corresponding to websites and academic papers, and cover 11 different quantifiers. We analyze the features of generics sentences in the dataset, with interesting insights: generics can be long sentences (averaging over 16 words) and speakers often use them to express generalisations about people. MGen is the biggest and most diverse dataset of naturally occurring generic sentences, opening the door to large-scale computational research on genericity. It is publicly available at https://gustavocilleruelo.com/mgen

Method: Fine-tuned open-source definition generators based on large language models, using datasets from the AXOLOTL'24 shared task on explainable semantic change modeling for Finnish, Russian, and German languages.

Result: The fine-tuned open-source models performed higher than the best submissions from the shared task that used closed proprietary LLMs. Encoder-decoder definition generators performed on par with decoder-only counterparts.

Conclusion: Open-weights LLMs can effectively generate explanations for novel word senses and outperform proprietary models in semantic change modeling tasks across multiple languages.

Abstract: We apply definition generators based on open-weights large language models to the task of creating explanations of novel senses, taking target word usages as an input. To this end, we employ the datasets from the AXOLOTL'24 shared task on explainable semantic change modeling, which features Finnish, Russian and German languages. We fine-tune and provide publicly the open-source models performing higher than the best submissions of the aforementioned shared task, which employed closed proprietary LLMs. In addition, we find that encoder-decoder definition generators perform on par with their decoder-only counterparts.

Method: Adapted the Binoculars method with optimized global thresholds and constructed new Vietnamese AI-generated datasets for benchmarking and threshold determination.

Result: VietBinoculars achieves over 99% accuracy, F1-score, and AUC on multiple out-of-domain datasets, outperforming original Binoculars, traditional methods, and commercial tools like ZeroGPT and DetectGPT.

Conclusion: The optimized VietBinoculars method is highly effective for detecting Vietnamese LLM-generated text, particularly under modified prompting strategies, and represents a significant improvement over existing detection approaches.

Abstract: The rapid development research of Large Language Models (LLMs) based on transformer architectures raises key challenges, one of them being the task of distinguishing between human-written text and LLM-generated text. As LLM-generated textual content, becomes increasingly complex over time, and resembles human writing, traditional detection methods are proving less effective, especially as the number and diversity of LLMs continue to grow with new models and versions being released at a rapid pace. This study proposes VietBinoculars, an adaptation of the Binoculars method with optimized global thresholds, to enhance the detection of Vietnamese LLM-generated text. We have constructed new Vietnamese AI-generated datasets to determine the optimal thresholds for VietBinoculars and to enable benchmarking. The results from our experiments show results show that VietBinoculars achieves over 99% in all two domains of accuracy, F1-score, and AUC on multiple out-of-domain datasets. It outperforms the original Binoculars model, traditional detection methods, and other state-of-the-art approaches, including commercial tools such as ZeroGPT and DetectGPT, especially under specially modified prompting strategies.

Method: Developed both ACO (nature-inspired metaheuristic) and Google OR-Tools implementations in Python using custom dataset, comparing performance based on routing efficiency, computation time, and scalability.

Result: ACO provides flexibility in routing parameters, while OR-Tools runs significantly faster with more consistency and requires less input configuration.

Conclusion: The findings help in selecting appropriate routing strategies for scalable real-time logistics systems based on specific requirements for flexibility versus speed and consistency.

Abstract: In modern logistics management systems, route planning requires high efficiency. The Open Capacitated Vehicle Routing Problem (OCVRP) deals with finding optimal delivery routes for a fleet of vehicles serving geographically distributed customers, without requiring the vehicles to return to the depot after deliveries. The present study is comparative in nature and speaks of two algorithms for OCVRP solution: Ant Colony Optimization (ACO), a nature-inspired metaheuristic; and Google OR-Tools, an industry-standard toolkit for optimization. Both implementations were developed in Python and using a custom dataset. Performance appraisal was based on routing efficiency, computation time, and scalability. The results show that ACO allows flexibility in routing parameters while OR-Tools runs much faster with more consistency and requires less input. This could help choose among routing strategies for scalable real-time logistics systems.

Method: TyleR leverages pre-trained language models to enrich node representations with implicit type signals, using a subgraph-based approach for inductive link prediction without requiring explicit type annotations.

Result: Experiments on standard benchmarks show that TyleR outperforms state-of-the-art baselines in scenarios with scarce type annotations and sparse graph connectivity.

Conclusion: Pre-trained language models can effectively provide implicit type signals for inductive link prediction, enabling better performance when explicit type information is limited or unavailable.

Abstract: Inductive link prediction is emerging as a key paradigm for real-world knowledge graphs (KGs), where new entities frequently appear and models must generalize to them without retraining. Predicting links in a KG faces the challenge of guessing previously unseen entities by leveraging generalizable node features such as subgraph structure, type annotations, and ontological constraints. However, explicit type information is often lacking or incomplete. Even when available, type information in most KGs is often coarse-grained, sparse, and prone to errors due to human annotation. In this work, we explore the potential of pre-trained language models (PLMs) to enrich node representations with implicit type signals. We introduce TyleR, a Type-less yet type-awaRe approach for subgraph-based inductive link prediction that leverages PLMs for semantic enrichment. Experiments on standard benchmarks demonstrate that TyleR outperforms state-of-the-art baselines in scenarios with scarce type annotations and sparse graph connectivity. To ensure reproducibility, we share our code at https://github.com/sisinflab/tyler .

Method: Obtain global gradient through zero-learning-rate training on general data, then use it for gradient boosting and dynamic training step correction during domain training, combined with annealing learning to create a fine-tuning pipeline that only uses domain data.

Result: DBA achieves 5.8% average improvement in joint performance over vanilla fine-tuning and reduces GPU hours by 91.0%.

Conclusion: DBA provides an efficient and universal solution for LLM fine-tuning that eliminates the need for repeated experiments and complex data mixtures while improving performance.

Abstract: Large language models (LLMs) fine-tuning shows excellent implications. However, vanilla fine-tuning methods often require intricate data mixture and repeated experiments for optimal generalization. To address these challenges and streamline the training process, we propose an efficient and universal solution, Dynamic Boosted Annealing (DBA). We obtain a global gradient through zero-learning-rate training on general data, which is subsequently employed for gradient boosting and dynamic training step correction during domain training. In conjunction with annealing learning, we end up establishing a fine-tuning pipeline that relies solely on domain data without collapse. By evaluating both general and domain-specific performance across multiple tasks on several popular base models, DBA achieves an average improvement of 5.8% in joint performance over vanilla fine-tuning. Furthermore, since general data is no longer involved in annealing, repeated experiments led by data mixture are also eliminated. According to our tests, the DBA method can reduce GPU hours by 91.0% compared to the vanilla method.

Method: Generate multiple summaries and task-oriented QA pairs from dialogues using LLMs in zero-shot manner, evaluate summary quality by having LLMs answer task-related questions, select best candidate answers and most informative summary, then fine-tune the best LLM on selected summaries.

Result: Achieves competitive results in various zero-shot settings, rivaling fully-supervised State-of-the-Art methods when validated on multiple datasets.

Conclusion: The proposed framework demonstrates effectiveness for task-oriented utility-based dialogue summarization without requiring costly human-written supervision.

Abstract: Dialogue summarization aims to distill the core meaning of a conversation into a concise text. This is crucial for reducing the complexity and noise inherent in dialogue-heavy applications. While recent approaches typically train language models to mimic human-written summaries, such supervision is costly and often results in outputs that lack task-specific focus limiting their effectiveness in downstream applications, such as medical tasks. In this paper, we propose \app, a framework for task-oriented utility-based dialogue summarization. \app starts by generating multiple summaries and task-oriented question-answer pairs from a dialogue in a zero-shot manner using a pool of large language models (LLMs). The quality of the generated summaries is evaluated by having LLMs answer task-related questions before \textit{(i)} selecting the best candidate answers and \textit{(ii)} identifying the most informative summary based on these answers. Finally, we fine-tune the best LLM on the selected summaries. When validated on multiple datasets, \app demonstrates its effectiveness by achieving competitive results in various zero-shot settings, rivaling fully-supervised State-of-the-Art (SotA) methods.

Method: Leveraging AI annotators, the toolkit provides Python API and browser app to track personality changes encouraged by human/AI feedback and exhibited across models.

Result: The toolkit was used to analyze personality traits in popular human feedback datasets (Chatbot Arena, MultiPref, PRISM) and how much popular models exhibit such traits.

Conclusion: Feedback Forensics enables explicit evaluation of AI personality, providing transparency in model assessment and addressing overfitting issues in feedback-based leaderboards.

Abstract: Some traits making a “good” AI model are hard to describe upfront. For example, should responses be more polite or more casual? Such traits are sometimes summarized as model character or personality. Without a clear objective, conventional benchmarks based on automatic validation struggle to measure such traits. Evaluation methods using human feedback such as Chatbot Arena have emerged as a popular alternative. These methods infer “better” personality and other desirable traits implicitly by ranking multiple model responses relative to each other. Recent issues with model releases highlight limitations of these existing opaque evaluation approaches: a major model was rolled back over sycophantic personality issues, models were observed overfitting to such feedback-based leaderboards. Despite these known issues, limited public tooling exists to explicitly evaluate model personality. We introduce Feedback Forensics: an open-source toolkit to track AI personality changes, both those encouraged by human (or AI) feedback, and those exhibited across AI models trained and evaluated on such feedback. Leveraging AI annotators, our toolkit enables investigating personality via Python API and browser app. We demonstrate the toolkit’s usefulness in two steps: (A) first we analyse the personality traits encouraged in popular human feedback datasets including Chatbot Arena, MultiPref and PRISM; and (B) then use our toolkit to analyse how much popular models exhibit such traits. We release (1) our Feedback Forensics toolkit alongside (2) a web app tracking AI personality in popular models and feedback datasets as well as (3) the underlying annotation data at https://github.com/rdnfn/feedback-forensics.

Method: One-token rollout (OTR) reframes autoregressive learning as single-step RL trajectories. At each token generation step, it samples multiple candidate tokens from the current policy, then uses the ground-truth token from supervised data to provide reward signals and guide learning via policy gradient.

Result: Extensive experiments on mathematical reasoning, code generation, and general domain reasoning benchmarks show that OTR consistently outperforms standard SFT, demonstrating improved generalization capabilities.

Conclusion: OTR establishes that the on-policy nature of data is a critical driver of generalization in LLM fine-tuning, offering a practical alternative that combines benefits of RL with the efficiency of SFT.

Abstract: Supervised fine-tuning (SFT) is the predominant method for adapting large language models (LLMs), yet it often struggles with generalization compared to reinforcement learning (RL). In this work, we posit that this performance disparity stems not just from the loss function, but from a more fundamental difference: SFT learns from a fixed, pre-collected dataset, whereas RL utilizes on-policy data sampled from the current policy. Building on this hypothesis, we introduce one-token rollout (OTR), a novel fine-tuning algorithm that guides SFT with the policy gradient method. OTR reframes the autoregressive learning process by treating each token generation as a single-step reinforcement learning trajectory. At each step, it performs a Monte Carlo ``rollout’’ by sampling multiple candidate tokens from the current policy’s distribution. The ground-truth token from the supervised data is then used to provide a reward signal to these samples. Guided by policy gradient, our algorithm repurposes static, off-policy supervised data into a dynamic, on-policy signal at the token level, capturing the generalization benefits of on-policy learning while bypassing the costly overhead of full sentence generation. Through extensive experiments on a diverse suite of challenging benchmarks spanning mathematical reasoning, code generation, and general domain reasoning, we demonstrate that OTR consistently outperforms standard SFT. Our findings establish OTR as a powerful and practical alternative for fine-tuning LLMs and provide compelling evidence that the on-policy nature of data is a critical driver of generalization, offering a promising new direction for fine-tuning LLMs.

Method: Systematic study of latent thinking patterns in Huggin-3.5B, development of latent classifier to predict answer correctness from latent thoughts, and proposal of Latent Thinking Optimization (LTO) algorithm using latent classifier as Latent Reward Model (LRM) to optimize thinking processes.

Result: Latent thoughts for correct vs incorrect answers show highly distinguishable patterns. LRM effectively detects incorrect latent thinking, and LTO significantly improves latent thinking processes across diverse reasoning tasks. The method generalizes across domains and can be applied to general LLMs.

Conclusion: Reward modeling and scaling test-time thinking with supervision can be performed directly in latent space, offering a general, efficient, and domain-agnostic approach to improving LLM thinking processes, contrasting with verbal thinking methods.

Abstract: Large Language Models (LLMs) excel at problem solving by generating chain of thoughts in natural language, but such verbal thinking is computationally costly and prone to overthinking. Recent work instead proposes a latent thinking architecture Huggin-3.5B, which represents intermediate reasoning steps as sequence of latent representations. However, latent thoughts lack interpretability and are difficult to supervise, raising concerns about the correctness and reliability of its latent thinking processes. In this paper, we provide a systematic study of how Huggin-3.5B thinks in the latent space and how external supervision signals can improve its latent thinking processes. We show that latent thoughts leading to correct versus incorrect answers exhibit highly distinguishable patterns, and that a latent classifier can reliably predict answer correctness directly from latent thoughts. Leveraging these insights, we propose Latent Thinking Optimization (LTO), a probabilistic algorithm that employs the latent classifier as a Latent Reward Model (LRM) to optimize the latent thinking processes. Extensive experiments across diverse reasoning tasks demonstrate that LRM is highly effective in detecting incorrect latent thinking patterns, and LTO can significantly improve the latent thinking processes. Furthermore, we show that LRM can generalize across diverse domains, and LTO can be seamlessly applied to general LLMs to improve their thinking processes. In contrast to verbal thinking, our method demonstrates that reward modeling and scaling test-time thinking with supervision can be performed directly in the latent space, highlighting its potential as a general, efficient, and domain-agnostic approach to improving the thinking processes of LLMs.

Method: Uses block diffusion mechanism with complementary attention mask for bidirectional context modeling, hierarchical caching (block-level and sub-block cache), and parallel decoding pipeline to enable efficient parallel generation.

Result: Achieves 500x reduction in training data compared to full-attention diffusion LLMs, 2.5x speedup over standard AR decoding, and matches or surpasses AR baselines in accuracy across diverse benchmarks.

Conclusion: Fast-dLLM v2 represents a significant advancement toward practical deployment of fast and accurate LLMs, delivering state-of-the-art efficiency among diffusion LLMs while preserving original model performance.

Abstract: Autoregressive (AR) large language models (LLMs) have achieved remarkable performance across a wide range of natural language tasks, yet their inherent sequential decoding limits inference efficiency. In this work, we propose Fast-dLLM v2, a carefully designed block diffusion language model (dLLM) that efficiently adapts pretrained AR models into dLLMs for parallel text generation, requiring only approximately 1B tokens of fine-tuning. This represents a 500x reduction in training data compared to full-attention diffusion LLMs such as Dream (580B tokens), while preserving the original model’s performance. Our approach introduces a novel training recipe that combines a block diffusion mechanism with a complementary attention mask, enabling blockwise bidirectional context modeling without sacrificing AR training objectives. To further accelerate decoding, we design a hierarchical caching mechanism: a block-level cache that stores historical context representations across blocks, and a sub-block cache that enables efficient parallel generation within partially decoded blocks. Coupled with our parallel decoding pipeline, Fast-dLLM v2 achieves up to 2.5x speedup over standard AR decoding without compromising generation quality. Extensive experiments across diverse benchmarks demonstrate that Fast-dLLM v2 matches or surpasses AR baselines in accuracy, while delivering state-of-the-art efficiency among dLLMs

• marking a significant step toward the practical deployment of fast and accurate LLMs. Code and model will be publicly released.

Method: Uses a single agent that interacts with KGs as its environment, learning retrieval strategies through reinforcement learning and incorporating retrieved information into reasoning and generation in an end-to-end optimized process.

Result: KG-R1 improves answer accuracy with fewer generation tokens than prior methods using larger models, and maintains strong accuracy on new KGs without modification after training.

Conclusion: KG-R1 provides an efficient and transferable KG-RAG framework suitable for real-world deployment, demonstrating both performance improvements and practical advantages over existing approaches.

Abstract: Knowledge-graph retrieval-augmented generation (KG-RAG) couples large language models (LLMs) with structured, verifiable knowledge graphs (KGs) to reduce hallucinations and expose reasoning traces. However, many KG-RAG systems compose multiple LLM modules (e.g planning, reasoning, and responding), inflating inference cost and binding behavior to a specific target KG. To address this, we introduce KG-R1, an agentic KG retrieval-augmented generation (KG-RAG) framework through reinforcement learning (RL). KG-R1 utilizes a single agent that interacts with KGs as its environment, learning to retrieve at each step and incorporating the retrieved information into its reasoning and generation. The process is optimized through end-to-end RL. In controlled experiments across Knowledge-Graph Question Answering (KGQA) benchmarks, our method demonstrates both efficiency and transferability: Using Qwen-2.5-3B, KG-R1 improves answer accuracy with fewer generation tokens than prior multi-module workflow methods that use larger foundation or fine-tuned models. Furthermore, KG-R1 enables plug and play: after training, it maintains strong accuracy on new KGs without modification. These properties make KG-R1 a promising KG-RAG framework for real-world deployment. Our code is publicly available at https://github.com/Jinyeop3110/KG-R1.

Method: Developed a multi-faceted annotation scheme combining concepts from previous works, manually annotated 5,000 Hebrew parliamentary sentences, measured inter-annotator agreement, and experimented with automatic prediction approaches.

Result: Created a comprehensive factuality annotation scheme adaptable to other languages, achieved measurable inter-annotator agreement on 5,000 annotated sentences, and explored methods for automatically extending annotation to larger corpora.

Conclusion: The presented annotation scheme successfully captures the complexity of factuality assessment and shows promise for adaptation to other languages, with potential for automated extension to larger text corpora through the tested prediction approaches.

Abstract: Factuality assesses the extent to which a language utterance relates to real-world information; it determines whether utterances correspond to facts, possibilities, or imaginary situations, and as such, it is instrumental for fact checking. Factuality is a complex notion that relies on multiple linguistic signals, and has been studied in various disciplines. We present a complex, multi-faceted annotation scheme of factuality that combines concepts from a variety of previous works. We developed the scheme for Hebrew, but we trust that it can be adapted to other languages. We also present a set of almost 5,000 sentences in the domain of parliamentary discourse that we manually annotated according to this scheme. We report on inter-annotator agreement, and experiment with various approaches to automatically predict (some features of) the scheme, in order to extend the annotation to a large corpus.

Method: The researchers created a bilingual English-Telugu dataset and benchmarked different LLM-based approaches for veracity classification and justification generation.

Result: The study evaluates the effectiveness of various LLM approaches in classifying factual claims by veracity and generating bilingual justifications.

Conclusion: LLMs show promise for automated fact-checking and justification generation across multiple languages, as demonstrated through the creation of a bilingual dataset and benchmarking framework.

Abstract: False information poses a significant global challenge, and manually verifying claims is a time-consuming and resource-intensive process. In this research paper, we experiment with different approaches to investigate the effectiveness of large language models (LLMs) in classifying factual claims by their veracity and generating justifications in English and Telugu. The key contributions of this work include the creation of a bilingual English-Telugu dataset and the benchmarking of different veracity classification approaches based on LLMs.

Method: Fine-tuned small language models were trained for item alignment at domain and skill levels using data from college admissions reading/writing tests. Performance was compared with embedding-based supervised models, and semantic similarity analyses were conducted to understand misclassifications.

Result: SLMs consistently outperformed embedding-based models, especially for fine-grained skill alignment. Including more item text data significantly improved performance beyond sample size increases alone. Semantic analyses revealed certain skills were too similar, explaining misclassifications.

Conclusion: Fine-tuned SLMs are effective for automated item alignment, providing a scalable alternative to human expert judgment, with semantic similarity analysis helping identify and understand classification challenges.

Abstract: Aligning test items to content standards is a critical step in test development to collect validity evidence based on content. Item alignment has typically been conducted by human experts. This judgmental process can be subjective and time-consuming. This study investigated the performance of fine-tuned small language models (SLMs) for automated item alignment using data from a large-scale standardized reading and writing test for college admissions. Different SLMs were trained for alignment at both domain and skill levels respectively with 10 skills mapped to 4 content domains. The model performance was evaluated in multiple criteria on two testing datasets. The impact of types and sizes of the input data for training was investigated. Results showed that including more item text data led to substantially better model performance, surpassing the improvements induced by sample size increase alone. For comparison, supervised machine learning models were trained using the embeddings from the multilingual-E5-large-instruct model. The study results showed that fine-tuned SLMs consistently outperformed the embedding-based supervised machine learning models, particularly for the more fine-grained skill alignment. To better understand model misclassifications, multiple semantic similarity analysis including pairwise cosine similarity, Kullback-Leibler divergence of embedding distributions, and two-dimension projections of item embeddings were conducted. These analyses consistently showed that certain skills in SAT and PSAT were semantically too close, providing evidence for the observed misclassification.

Method: Reframes the task as planning attribute control order using Monte Carlo Tree Search, where nodes are summaries and actions are single-attribute adjustments, enabling progressive refinement of only attributes needing further control.

Result: PACO achieves robust multi-attribute controllability, surpassing LLM-based self-planning models and fine-tuned baselines. With Llama-3.2-1B, it rivals the controllability of much larger Llama-3.3-70B baselines. With larger models, it outperforms all competitors.

Conclusion: PACO provides an effective training-free solution for multi-attribute controllable summarization that adaptively discovers optimal control orders and consistently satisfies all constraints.

Abstract: Controllable summarization moves beyond generic outputs toward human-aligned summaries guided by specified attributes. In practice, the interdependence among attributes makes it challenging for language models to satisfy correlated constraints consistently. Moreover, previous approaches often require per-attribute fine-tuning, limiting flexibility across diverse summary attributes. In this paper, we propose adaptive planning for multi-attribute controllable summarization (PACO), a training-free framework that reframes the task as planning the order of sequential attribute control with a customized Monte Carlo Tree Search (MCTS). In PACO, nodes represent summaries, and actions correspond to single-attribute adjustments, enabling progressive refinement of only the attributes requiring further control. This strategy adaptively discovers optimal control orders, ultimately producing summaries that effectively meet all constraints. Extensive experiments across diverse domains and models demonstrate that PACO achieves robust multi-attribute controllability, surpassing both LLM-based self-planning models and fine-tuned baselines. Remarkably, PACO with Llama-3.2-1B rivals the controllability of the much larger Llama-3.3-70B baselines. With larger models, PACO achieves superior control performance, outperforming all competitors.

Method: Uses a Story Prototype (genre-agnostic knowledge graph-based narrative representation) and three-stage agent workflow: Initialization (constructs narrative skeleton), Generation (multi-agent dialogues guided by objectives), and Writing (produces multi-genre text with advanced structures).

Result: Generates thousands of chapters with stable quality and low cost (<$1 per 100 chapters), consistently outperforms baselines across 10 narrative indicators, and approaches human-authored novel quality across diverse genres.

Conclusion: CreAgentive effectively addresses LLM limitations in creative generation through its Story Prototype and agent workflow architecture, achieving robust performance with reduced storage redundancy and overcoming long-form generation bottlenecks.

Abstract: We present CreAgentive, an agent workflow driven multi-category creative generation engine that addresses four key limitations of contemporary large language models in writing stories, drama and other categories of creatives: restricted genre diversity, insufficient output length, weak narrative coherence, and inability to enforce complex structural constructs. At its core, CreAgentive employs a Story Prototype, which is a genre-agnostic, knowledge graph-based narrative representation that decouples story logic from stylistic realization by encoding characters, events, and environments as semantic triples. CreAgentive engages a three-stage agent workflow that comprises: an Initialization Stage that constructs a user-specified narrative skeleton; a Generation Stage in which long- and short-term objectives guide multi-agent dialogues to instantiate the Story Prototype; a Writing Stage that leverages this prototype to produce multi-genre text with advanced structures such as retrospection and foreshadowing. This architecture reduces storage redundancy and overcomes the typical bottlenecks of long-form generation. In extensive experiments, CreAgentive generates thousands of chapters with stable quality and low cost (less than $1 per 100 chapters) using a general-purpose backbone model. To evaluate performance, we define a two-dimensional framework with 10 narrative indicators measuring both quality and length. Results show that CreAgentive consistently outperforms strong baselines and achieves robust performance across diverse genres, approaching the quality of human-authored novels.

Method: A 300M parameter Regression Language Model (RLM) initialized from T5Gemma that directly predicts numeric outcomes from code text without feature engineering.

Result: The RLM achieves >0.9 Spearman-rank on APPS competitive programming, >0.5 average Spearman-rank across 17 CodeNet languages, and highest 0.46 Kendall-Tau on NAS design spaces while predicting latencies on multiple hardware platforms.

Conclusion: A single unified language model can effectively predict diverse code execution metrics across languages and domains, outperforming specialized approaches.

Abstract: We study code-to-metric regression: predicting numeric outcomes of code executions, a challenging task due to the open-ended nature of programming languages. While prior methods have resorted to heavy and domain-specific feature engineering, we show that a single unified Regression Language Model (RLM) can simultaneously predict directly from text, (i) the memory footprint of code across multiple high-level languages such as Python and C++, (ii) the latency of Triton GPU kernels, and (iii) the accuracy and speed of trained neural networks represented in ONNX. In particular, a relatively small 300M parameter RLM initialized from T5Gemma, obtains > 0.9 Spearman-rank on competitive programming submissions from APPS, and a single unified model achieves > 0.5 average Spearman-rank across 17 separate languages from CodeNet. Furthermore, the RLM can obtain the highest average Kendall-Tau of 0.46 on five classic NAS design spaces previously dominated by graph neural networks, and simultaneously predict architecture latencies on numerous hardware platforms.

Method: Introduces certainty-forcing distillation - a training strategy that distills the model to follow original sampling trajectories while forcing high certainty on masked tokens more rapidly and in parallel.

Result: Reduces decoding steps from 256 to 30 on GSM8K (8.5x speedup) and from 256 to 24 on MBPP (10.5x speedup) without performance degradation when applied to LLaDA-8B-Instruct.

Conclusion: dParallel effectively unlocks the inherent parallelism of dLLMs for fast sampling through a simple and effective distillation approach.

Abstract: Diffusion large language models (dLLMs) have recently drawn considerable attention within the research community as a promising alternative to autoregressive generation, offering parallel token prediction and lower inference latency. Yet, their parallel decoding potential remains largely underexplored, as existing open-source models still require nearly token-length decoding steps to ensure performance. To address this, we introduce dParallel, a simple and effective method that unlocks the inherent parallelism of dLLMs for fast sampling. We identify that the key bottleneck to parallel decoding arises from the sequential certainty convergence for masked tokens. Building on this insight, we introduce the core of our approach: certainty-forcing distillation, a novel training strategy that distills the model to follow its original sampling trajectories while enforcing it to achieve high certainty on masked tokens more rapidly and in parallel. Extensive experiments across various benchmarks demonstrate that our method can dramatically reduce the number of decoding steps while maintaining performance. When applied to the LLaDA-8B-Instruct model, dParallel reduces decoding steps from 256 to 30 on GSM8K, achieving an 8.5x speedup without performance degradation. On the MBPP benchmark, it cuts decoding steps from 256 to 24, resulting in a 10.5x speedup while maintaining accuracy. Our code is available at https://github.com/czg1225/dParallel

Method: Developed VitaBench with 66 tools across daily applications, using a framework that eliminates domain-specific policies to enable flexible composition of scenarios and tools, yielding 100 cross-scenario tasks and 300 single-scenario tasks derived from real user requests.

Result: Even the most advanced models achieve only 30% success rate on cross-scenario tasks and less than 50% success rate on single-scenario tasks, demonstrating the benchmark’s challenging nature.

Conclusion: VitaBench serves as a valuable resource for advancing the development of AI agents in practical real-world applications, highlighting significant performance gaps in current models.

Abstract: As LLM-based agents are increasingly deployed in real-life scenarios, existing benchmarks fail to capture their inherent complexity of handling extensive information, leveraging diverse resources, and managing dynamic user interactions. To address this gap, we introduce VitaBench, a challenging benchmark that evaluates agents on versatile interactive tasks grounded in real-world settings. Drawing from daily applications in food delivery, in-store consumption, and online travel services, VitaBench presents agents with the most complex life-serving simulation environment to date, comprising 66 tools. Through a framework that eliminates domain-specific policies, we enable flexible composition of these scenarios and tools, yielding 100 cross-scenario tasks (main results) and 300 single-scenario tasks. Each task is derived from multiple real user requests and requires agents to reason across temporal and spatial dimensions, utilize complex tool sets, proactively clarify ambiguous instructions, and track shifting user intent throughout multi-turn conversations. Moreover, we propose a rubric-based sliding window evaluator, enabling robust assessment of diverse solution pathways in complex environments and stochastic interactions. Our comprehensive evaluation reveals that even the most advanced models achieve only 30% success rate on cross-scenario tasks, and less than 50% success rate on others. Overall, we believe VitaBench will serve as a valuable resource for advancing the development of AI agents in practical real-world applications. The code, dataset, and leaderboard are available at https://vitabench.github.io/

Method: Proposes BatonVoice framework with LLM as conductor generating textual vocal features (pitch, energy) and BatonTTS as orchestra generating speech from these features.

Result: BatonVoice achieves strong performance in controllable and emotional speech synthesis, outperforming open- and closed-source baselines, with remarkable zero-shot cross-lingual generalization to unseen languages.

Conclusion: Objectifying speech into textual vocal features effectively unlocks LLMs’ linguistic intelligence for controllable TTS applications.

Abstract: The rise of Large Language Models (LLMs) is reshaping multimodel models, with speech synthesis being a prominent application. However, existing approaches often underutilize the linguistic intelligence of these models, typically failing to leverage their powerful instruction-following capabilities. This limitation hinders the model’s ability to follow text instructions for controllable Text-to-Speech~(TTS). To address this, we propose a new paradigm inspired by operationalism'' that decouples instruction understanding from speech generation. We introduce BatonVoice, a framework where an LLM acts as a conductor’’, understanding user instructions and generating a textual plan'' -- explicit vocal features (e.g., pitch, energy). A separate TTS model, the orchestra’’, then generates the speech from these features. To realize this component, we develop BatonTTS, a TTS model trained specifically for this task. Our experiments demonstrate that BatonVoice achieves strong performance in controllable and emotional speech synthesis, outperforming strong open- and closed-source baselines. Notably, our approach enables remarkable zero-shot cross-lingual generalization, accurately applying feature control abilities to languages unseen during post-training. This demonstrates that objectifying speech into textual vocal features can more effectively unlock the linguistic intelligence of LLMs.

Method: Systematically vary the number of activated experts during training to force the model to learn a meaningful ranking: top experts provide coarse capabilities while subsequent experts add finer details. Uses layer-wise randomization strategy.

Result: A single M-MoE model achieves remarkable elasticity, with performance at various expert counts closely matching a suite of specialist models but at only a fraction of the training cost.

Conclusion: M-MoE enables practical and adaptable deployments of large-scale MoE models by unlocking elastic inference and allowing optimization of performance through different computational budget allocations across model layers.

Abstract: Mixture-of-Experts (MoE) has emerged as a promising paradigm for efficiently scaling large language models without a proportional increase in computational cost. However, the standard training strategy of Top-K router prevents MoE models from realizing their full potential for elastic inference. When the number of activated experts is altered at inference time, these models exhibit precipitous performance degradation. In this work, we introduce Matryoshka MoE (M-MoE), a training framework that instills a coarse-to-fine structure directly into the expert ensemble. By systematically varying the number of activated experts during training, M-MoE compels the model to learn a meaningful ranking: top-ranked experts collaborate to provide essential, coarse-grained capabilities, while subsequent experts add progressively finer-grained detail. We explore this principle at multiple granularities, identifying a layer-wise randomization strategy as the most effective. Our experiments demonstrate that a single M-MoE model achieves remarkable elasticity, with its performance at various expert counts closely matching that of an entire suite of specialist models, but at only a fraction of the total training cost. This flexibility not only unlocks elastic inference but also enables optimizing performance by allocating different computational budgets to different model layers. Our work paves the way for more practical and adaptable deployments of large-scale MoE models.

Method: Uses a unified agent framework driven by Multi-modal Large Language Models (MLLMs) that integrates perception, memory, and sequential decision-making, requiring agents to comprehend optical and sonar data and autonomously explore complex environments.

Result: Extensive experiments reveal substantial gaps between state-of-the-art MLLM-driven agents and human experts, highlighting persistent difficulties in perception, planning, and adaptability in ocean underwater environments.

Conclusion: OceanGym establishes a crucial testbed for developing robust embodied AI and transferring capabilities to real-world autonomous ocean underwater vehicles, marking a significant step toward intelligent agents operating in Earth’s last unexplored frontiers.

Abstract: We introduce OceanGym, the first comprehensive benchmark for ocean underwater embodied agents, designed to advance AI in one of the most demanding real-world environments. Unlike terrestrial or aerial domains, underwater settings present extreme perceptual and decision-making challenges, including low visibility, dynamic ocean currents, making effective agent deployment exceptionally difficult. OceanGym encompasses eight realistic task domains and a unified agent framework driven by Multi-modal Large Language Models (MLLMs), which integrates perception, memory, and sequential decision-making. Agents are required to comprehend optical and sonar data, autonomously explore complex environments, and accomplish long-horizon objectives under these harsh conditions. Extensive experiments reveal substantial gaps between state-of-the-art MLLM-driven agents and human experts, highlighting the persistent difficulty of perception, planning, and adaptability in ocean underwater environments. By providing a high-fidelity, rigorously designed platform, OceanGym establishes a testbed for developing robust embodied AI and transferring these capabilities to real-world autonomous ocean underwater vehicles, marking a decisive step toward intelligent agents capable of operating in one of Earth’s last unexplored frontiers. The code and data are available at https://github.com/OceanGPT/OceanGym.

Method: Uses feature attribution techniques to analyze how parts of input spectrogram influence choice between alternative outputs (target vs foil).

Result: Method accurately identifies audio features that drive gender selection in speech translation, demonstrating practical effectiveness.

Conclusion: Extends contrastive explanations to speech-to-text domain, providing foundation for better understanding S2T models.

Abstract: Contrastive explanations, which indicate why an AI system produced one output (the target) instead of another (the foil), are widely regarded in explainable AI as more informative and interpretable than standard explanations. However, obtaining such explanations for speech-to-text (S2T) generative models remains an open challenge. Drawing from feature attribution techniques, we propose the first method to obtain contrastive explanations in S2T by analyzing how parts of the input spectrogram influence the choice between alternative outputs. Through a case study on gender assignment in speech translation, we show that our method accurately identifies the audio features that drive the selection of one gender over another. By extending the scope of contrastive explanations to S2T, our work provides a foundation for better understanding S2T models.

Method: Models must compose correct call sequences to compute target variables by traversing function-dependency DAGs, with precise control over graph size, dependency depth, and distractor functions.

Result: Reasoning-optimized models outperform general-purpose models, with GPT-5 performing best. Performance declines with increased dependency depth, and connected irrelevant functions are particularly challenging. A simple mitigation strategy of explicitly restating prior variable values improves success rates significantly (e.g., GPT-5 from 62.5% to 81.3%).

Conclusion: LLMs exhibit brittle state tracking in multi-turn tool use, but simple interventions like explicit variable value restatement can substantially improve performance, revealing opportunities for enhancing tool-use capabilities.

Abstract: As language models gain access to external tools via structured function calls, they become increasingly more capable of solving complex, multi-step tasks. However, existing benchmarks for tool-augmented language models (TaLMs) provide insufficient control over factors such as the number of functions accessible, task complexity, and input size, and remain vulnerable to data contamination. We present FuncBenchGen, a unified, contamination-free framework that evaluates TaLMs by generating synthetic multi-step tool-use tasks. The key idea is to cast tool use as traversal over a hidden function-dependency DAG where nodes are function calls and an edge between nodes represents one function consuming the output of another. Given a set of external function schemas, initial variable values, and a target variable, models must compose the correct call sequence to compute the target variable. FuncBenchGen allows users to precisely control task difficulty (e.g., graph size, dependency depth, and distractor functions) while avoiding data leakage. We apply our FuncBenchGen framework to evaluate seven LLMs on tool use tasks of varying difficulty. Reasoning-optimized models consistently outperform general-purpose models with GPT-5 significantly outperforming other models. Performance declines sharply as dependency depth increases. Furthermore, connected irrelevant functions prove especially difficult to handle. We find that strong models often make syntactically valid function calls but propagate incorrect or stale argument values across steps, revealing brittle state tracking by LLMs in multi-turn tool use. Motivated by this observation, we introduce a simple mitigation strategy that explicitly restates prior variable values to the agent at each step. Surprisingly, this lightweight change yields substantial gains across models. e.g., yielding a success rate improvement from 62.5% to 81.3% for GPT-5.

Method: MT-breaker uses a large language model to iteratively refine source texts by querying a target machine translation model to guide generation of difficult examples. The LLM probes the MT model to identify and amplify translation difficulties through an iterative refinement process.

Result: The approach generates examples that are more challenging for target MT models while preserving natural text diversity. The generated difficult examples also transfer well to other models and languages beyond the specific model used during generation.

Conclusion: MT-breaker provides an effective method for creating challenging, natural, and diverse machine translation test cases that can better evaluate model capabilities and reveal weaknesses across different models and languages.

Abstract: Machine translation benchmarks sourced from the real world are quickly obsoleted, due to most examples being easy for state-of-the-art translation models. This limits the benchmark’s ability to distinguish which model is better or to reveal models’ weaknesses. Current methods for creating difficult test cases, such as subsampling or from-scratch synthesis, either fall short of identifying difficult examples or suffer from a lack of diversity and naturalness. Inspired by the iterative process of human experts probing for model failures, we propose MT-breaker, a method where a large language model iteratively refines a source text to increase its translation difficulty. The LLM iteratively queries a target machine translation model to guide its generation of difficult examples. Our approach generates examples that are more challenging for the target MT model while preserving the diversity of natural texts. While the examples are tailored to a particular machine translation model during the generation, the difficulty also transfers to other models and languages.

Method: Analyzed self-bias in LLM-generated benchmarks for translation tasks by examining bias sources in generated test data and evaluation methods, and investigated factors like generation capabilities and source text diversity.

Result: Found that self-bias originates from both generated test data and evaluation methods, is influenced by model’s generation capabilities in source language, and is attributed to low diversity in source texts.

Conclusion: Improving diversity of generated source texts can mitigate some observed self-bias in LLM-generated benchmarks for translation tasks.

Abstract: As large language models (LLMs) begin to saturate existing benchmarks, automated benchmark creation using LLMs (LLM as a benchmark) has emerged as a scalable alternative to slow and costly human curation. While these generated test sets have to potential to cheaply rank models, we demonstrate a critical flaw. LLM generated benchmarks systematically favor the model that created the benchmark, they exhibit self bias on low resource languages to English translation tasks. We show three key findings on automatic benchmarking of LLMs for translation: First, this bias originates from two sources: the generated test data (LLM as a testset) and the evaluation method (LLM as an evaluator), with their combination amplifying the effect. Second, self bias in LLM as a benchmark is heavily influenced by the model’s generation capabilities in the source language. For instance, we observe more pronounced bias in into English translation, where the model’s generation system is developed, than in out of English translation tasks. Third, we observe that low diversity in source text is one attribution to self bias. Our results suggest that improving the diversity of these generated source texts can mitigate some of the observed self bias.

Method: Introduced MENLO framework with 6,423 human-annotated prompt-response pairs covering 47 language varieties across four quality dimensions. Used pairwise evaluation, structured annotation rubrics, and tested fine-tuning approaches including reinforcement learning, reward shaping, and multi-task learning.

Result: Zero-shot LLM judges benefit from pairwise evaluation and structured rubrics but underperform humans. Fine-tuning with RL showed substantial improvements. RL-trained judges can serve as generative reward models to enhance multilingual proficiency, though discrepancies with human judgment persist.

Conclusion: MENLO provides promising directions for scalable multilingual evaluation and preference alignment, with released dataset and framework to support further research in multilingual LLM evaluation.

Abstract: Ensuring native-like quality of large language model (LLM) responses across many languages is challenging. To address this, we introduce MENLO, a framework that operationalizes the evaluation of native-like response quality based on audience design-inspired mechanisms. Using MENLO, we create a dataset of 6,423 human-annotated prompt-response preference pairs covering four quality dimensions with high inter-annotator agreement in 47 language varieties. Our evaluation reveals that zero-shot LLM judges benefit significantly from pairwise evaluation and our structured annotation rubrics, yet they still underperform human annotators on our dataset. We demonstrate substantial improvements through fine-tuning with reinforcement learning, reward shaping, and multi-task learning approaches. Additionally, we show that RL-trained judges can serve as generative reward models to enhance LLMs’ multilingual proficiency, though discrepancies with human judgment remain. Our findings suggest promising directions for scalable multilingual evaluation and preference alignment. We release our dataset and evaluation framework to support further research in multilingual LLM evaluation.

Method: Formalizes discovery as Bayesian Optimization problem with hierarchical evaluation process (hypothesize, verify, analyze) using cumulative Findings Memory to balance exploration and exploitation, selectively promoting promising findings through validation levels.

Result: Consumed 20,000 GPU hours, generated 5,000 unique scientific ideas, experimentally validated 1,100 findings, and surpassed human-designed SOTA methods on three frontier AI tasks by 183.7%, 1.9%, and 7.9%.

Conclusion: Provides first large-scale evidence of AI achieving discoveries that progressively surpass human SOTA on scientific tasks, genuinely pushing the frontier of scientific discovery. System code and logs will be open-sourced.

Abstract: While previous AI Scientist systems can generate novel findings, they often lack the focus to produce scientifically valuable contributions that address pressing human-defined challenges. We introduce DeepScientist, a system designed to overcome this by conducting goal-oriented, fully autonomous scientific discovery over month-long timelines. It formalizes discovery as a Bayesian Optimization problem, operationalized through a hierarchical evaluation process consisting of “hypothesize, verify, and analyze”. Leveraging a cumulative Findings Memory, this loop intelligently balances the exploration of novel hypotheses with exploitation, selectively promoting the most promising findings to higher-fidelity levels of validation. Consuming over 20,000 GPU hours, the system generated about 5,000 unique scientific ideas and experimentally validated approximately 1100 of them, ultimately surpassing human-designed state-of-the-art (SOTA) methods on three frontier AI tasks by 183.7%, 1.9%, and 7.9%. This work provides the first large-scale evidence of an AI achieving discoveries that progressively surpass human SOTA on scientific tasks, producing valuable findings that genuinely push the frontier of scientific discovery. To facilitate further research into this process, we will open-source all experimental logs and system code at https://github.com/ResearAI/DeepScientist/.

Method: Formalize the task as a multi-armed bandit problem where each topic is an arm, pulling an arm involves drawing and evaluating a single example’s difficulty, and use bandit strategies to efficiently identify the most difficult topics within a fixed computational budget.

Result: Various bandit strategies vastly outperform baseline methods like brute-force search in finding the most challenging topics for machine translation tasks.

Conclusion: Multi-armed bandit approaches provide an efficient computational framework for identifying difficult topics in NLP testing, overcoming the infeasibility of brute-force search across thousands of topics.

Abstract: NLP models require test data that are sufficiently challenging. The difficulty of an example is linked to the topic it originates from (‘‘seed topic’’). The relationship between the topic and the difficulty of its instances is stochastic in nature: an example about a difficult topic can happen to be easy, and vice versa. At the scale of the Internet, there are tens of thousands of potential topics, and finding the most difficult one by drawing and evaluating a large number of examples across all topics is computationally infeasible. We formalize this task and treat it as a multi-armed bandit problem. In this framework, each topic is an ‘‘arm,’’ and pulling an arm (at a cost) involves drawing a single example, evaluating it, and measuring its difficulty. The goal is to efficiently identify the most difficult topics within a fixed computational budget. We illustrate the bandit problem setup of finding difficult examples for the task of machine translation. We find that various bandit strategies vastly outperform baseline methods like brute-force searching the most challenging topics.

Method: Measure convergence as expected per-token KL divergence across seeds, analyze convergence patterns by model size and training checkpoint, and examine convergence across token frequencies and part-of-speech tags.

Result: Identified four-phase convergence pattern: initial uniform, sharp-convergence, sharp-divergence, and slow-reconvergence. Larger models reconverge faster while smaller models don’t fully reconverge. Frequent tokens and function words converge faster than infrequent tokens and content words.

Conclusion: Model size affects convergence stability, with larger models learning more stable distributions, and convergence patterns vary significantly across different linguistic categories.

Abstract: In this paper, we investigate the convergence of language models (LMs) trained under different random seeds, measuring convergence as the expected per-token Kullback–Leibler (KL) divergence across seeds. By comparing LM convergence as a function of model size and training checkpoint, we identify a four-phase convergence pattern: (i) an initial uniform phase, (ii) a sharp-convergence phase, (iii) a sharp-divergence phase, and (iv) a slow-reconvergence phase. Further, we observe that larger models reconverge faster in later training stages, while smaller models never actually reconverge; these results suggest that a certain model size may be necessary to learn stable distributions. Restricting our analysis to specific token frequencies or part-of-speech (PoS) tags further reveals that convergence is uneven across linguistic categories: frequent tokens and function words converge faster and more reliably than their counterparts (infrequent tokens and content words). Overall, our findings highlight factors that influence the stability of the learned distributions in model training.

Method: Proposes ECL framework that uses medical entities from FAQs as focuses and generates hard negative samples to force models to focus on essential information. Also reorganizes datasets to address data leakage problems.

Result: ECL achieves state-of-the-art performance with ROUGE-1 scores of 52.85, 43.16, 41.31, 43.52 on MeQSum, CHQ-Summ, iCliniq, and HealthCareMagic datasets respectively.

Conclusion: The ECL method effectively improves medical question summarization by focusing on essential medical entities and addressing dataset quality issues, outperforming existing approaches.

Abstract: By summarizing longer consumer health questions into shorter and essential ones, medical question-answering systems can more accurately understand consumer intentions and retrieve suitable answers. However, medical question summarization is very challenging due to obvious distinctions in health trouble descriptions from patients and doctors. Although deep learning has been applied to successfully address the medical question summarization (MQS) task, two challenges remain: how to correctly capture question focus to model its semantic intention, and how to obtain reliable datasets to fairly evaluate performance. To address these challenges, this paper proposes a novel medical question summarization framework based on entity-driven contrastive learning (ECL). ECL employs medical entities present in frequently asked questions (FAQs) as focuses and devises an effective mechanism to generate hard negative samples. This approach compels models to focus on essential information and consequently generate more accurate question summaries. Furthermore, we have discovered that some MQS datasets, such as the iCliniq dataset with a 33% duplicate rate, have significant data leakage issues. To ensure an impartial evaluation of the related methods, this paper carefully examines leaked samples to reorganize more reasonable datasets. Extensive experiments demonstrate that our ECL method outperforms the existing methods and achieves new state-of-the-art performance, i.e., 52.85, 43.16, 41.31, 43.52 in terms of ROUGE-1 metric on MeQSum, CHQ-Summ, iCliniq, HealthCareMagic dataset, respectively. The code and datasets are available at https://github.com/yrbobo/MQS-ECL.

Method: Leverages LLMs’ belief reasoning ability grounded in Theory-of-Mind to detect misaligned actions before execution, then alerts users for timely correction.

Result: Achieves up to 20% improvements on Marco-F1 against baselines in misaligned action detection across three widely used tasks, with effective misalignment correction.

Conclusion: InferAct enhances LLM agent reliability by preemptively detecting and correcting misaligned actions, preventing adverse outcomes through timely user intervention.

Abstract: Deploying LLM-based agents in real-life applications often faces a critical challenge: the misalignment between agents’ behavior and user intent. Such misalignment may lead agents to unintentionally execute critical actions that carry negative outcomes (e.g., accidentally triggering a “buy-now” in web shopping), resulting in undesirable or even irreversible consequences. Although addressing these issues is crucial, the preemptive detection and correction of misaligned actions remains relatively underexplored. To fill this gap, we introduce InferAct, a novel approach that leverages the belief reasoning ability of LLMs, grounded in Theory-of-Mind, to detect misaligned actions before execution. Once the misalignment is detected, InferAct alerts users for timely correction, preventing adverse outcomes and enhancing the reliability of LLM agents’ decision-making processes. Experiments on three widely used tasks demonstrate that InferAct achieves up to 20% improvements on Marco-F1 against baselines in misaligned action detection. An in-depth evaluation of misalignment correction further highlights InferAct’s effectiveness in improving agent alignment.

Method: Two-stage training process: first injects domain-specific knowledge, then aligns through targeted stimulation. Constructed pre-training corpora, instruction-aligned datasets from hospital records, and ChP-TCM dataset from Pharmacopoeia of China.

Result: Evaluations across 11 test sets involving 31 models and 4 tasks demonstrate the effectiveness of BianCang.

Conclusion: BianCang offers valuable insights for future TCM research and the code, datasets, and models are publicly available.

Abstract: The surge of large language models (LLMs) has driven significant progress in medical applications, including traditional Chinese medicine (TCM). However, current medical LLMs struggle with TCM diagnosis and syndrome differentiation due to substantial differences between TCM and modern medical theory, and the scarcity of specialized, high-quality corpora. To this end, in this paper we propose BianCang, a TCM-specific LLM, using a two-stage training process that first injects domain-specific knowledge and then aligns it through targeted stimulation to enhance diagnostic and differentiation capabilities. Specifically, we constructed pre-training corpora, instruction-aligned datasets based on real hospital records, and the ChP-TCM dataset derived from the Pharmacopoeia of the People’s Republic of China. We compiled extensive TCM and medical corpora for continual pre-training and supervised fine-tuning, building a comprehensive dataset to refine the model’s understanding of TCM. Evaluations across 11 test sets involving 31 models and 4 tasks demonstrate the effectiveness of BianCang, offering valuable insights for future research. Code, datasets, and models are available on https://github.com/QLU-NLP/BianCang.

Method: KGQAGen combines structured knowledge grounding, LLM-guided generation, and symbolic verification to systematically resolve dataset pitfalls and produce verifiable QA instances.

Result: KGQAGen-10k benchmark was constructed and evaluation shows state-of-the-art KG-RAG models struggle on this benchmark, highlighting its ability to expose model limitations.

Conclusion: The findings advocate for more rigorous benchmark construction and position KGQAGen as a scalable framework for advancing KGQA evaluation.

Abstract: Knowledge Graph Question Answering (KGQA) systems rely on high-quality benchmarks to evaluate complex multi-hop reasoning. However, despite their widespread use, popular datasets such as WebQSP and CWQ suffer from critical quality issues, including inaccurate or incomplete ground-truth annotations, poorly constructed questions that are ambiguous, trivial, or unanswerable, and outdated or inconsistent knowledge. Through a manual audit of 16 popular KGQA datasets, including WebQSP and CWQ, we find that the average factual correctness rate is only 57 %. To address these issues, we introduce KGQAGen, an LLM-in-the-loop framework that systematically resolves these pitfalls. KGQAGen combines structured knowledge grounding, LLM-guided generation, and symbolic verification to produce challenging and verifiable QA instances. Using KGQAGen, we construct KGQAGen-10k, a ten-thousand scale benchmark grounded in Wikidata, and evaluate a diverse set of KG-RAG models. Experimental results demonstrate that even state-of-the-art systems struggle on this benchmark, highlighting its ability to expose limitations of existing models. Our findings advocate for more rigorous benchmark construction and position KGQAGen as a scalable framework for advancing KGQA evaluation.

Method: Proposed Aristotle framework with three components: Logical Decomposer (breaks down complex tasks), Logical Search Router (minimizes search errors), and Logical Resolver (handles logical contradictions).

Result: Aristotle consistently outperforms state-of-the-art reasoning frameworks in both accuracy and efficiency across multiple datasets, particularly excelling in complex logical reasoning scenarios.

Conclusion: The integration of symbolic expressions and logical rules throughout the reasoning process significantly alleviates logical reasoning bottlenecks in LLMs.

Abstract: In the context of large language models (LLMs), current advanced reasoning methods have made impressive strides in various reasoning tasks. However, when it comes to logical reasoning tasks, major challenges remain in both efficacy and efficiency. This is rooted in the fact that these systems fail to fully leverage the inherent structure of logical tasks throughout the reasoning processes such as decomposition, search, and resolution. To address this, we propose a logic-complete reasoning framework, Aristotle, with three key components: Logical Decomposer, Logical Search Router, and Logical Resolver. In our framework, symbolic expressions and logical rules are comprehensively integrated into the entire reasoning process, significantly alleviating the bottlenecks of logical reasoning, i.e., reducing sub-task complexity, minimizing search errors, and resolving logical contradictions. The experimental results on several datasets demonstrate that Aristotle consistently outperforms state-of-the-art reasoning frameworks in both accuracy and efficiency, particularly excelling in complex logical reasoning scenarios. We will open-source all our code at https://llm-symbol.github.io/Aristotle/.

Method: Guided Module Substitution (GMS) uses a guided trade-off signal between utility and backdoor to selectively replace modules in the victim model with modules from a single proxy model, without requiring retraining.

Result: GMS significantly outperforms existing defense baselines, especially against challenging attacks like LWS, and demonstrates robustness across different proxy models, inaccurate data knowledge, hyperparameters, and attack types.

Conclusion: GMS provides an effective, retraining-free solution for removing backdoors from deployed NLP models, offering strong performance and desirable properties for practical deployment.

Abstract: Model NLP models are commonly trained (or fine-tuned) on datasets from untrusted platforms like HuggingFace, posing significant risks of data poisoning attacks. A practical yet underexplored challenge arises when such backdoors are discovered after model deployment, making retraining-required defenses less desirable due to computational costs and data constraints. In this work, we propose Guided Module Substitution (GMS), an effective retraining-free method based on guided merging of the victim model with just a single proxy model. Unlike prior ad-hoc merging defenses, GMS uses a guided trade-off signal between utility and backdoor to selectively replaces modules in the victim model. GMS offers four desirable properties: (1) robustness to the choice and trustworthiness of the proxy model, (2) applicability under inaccurate data knowledge, (3) stability across hyperparameters, and (4) transferability across different attacks. Extensive experiments on encoder models and decoder LLMs demonstrate the strong effectiveness of GMS. GMS significantly outperforms even the strongest defense baseline, particularly against challenging attacks like LWS.

Method: Uses Character Knowledge Graph (CKG) extracted from input and generated summaries to assess factual consistency and provide actionable refinement guidance.

Result: Achieves superior performance on widely adopted benchmarks compared to competitive methods, demonstrating effectiveness through detailed workflow illustration and validation.

Conclusion: Agent-driven evaluation systems like NarrativeFactScore have strong potential to improve factual reliability of LLM-generated summaries.

Abstract: Large Language Models (LLMs) have demonstrated near-human performance in summarization tasks based on traditional metrics such as ROUGE and BERTScore. However, these metrics do not adequately capture critical aspects of summarization quality, such as factual accuracy, particularly for long narratives (>100K tokens). Recent advances, such as LLM-as-a-Judge, address the limitations of metrics based on lexical similarity but still exhibit factual inconsistencies, especially in understanding character relationships and states. In this work, we introduce NarrativeFactScore, a novel “Agent-as-a-Judge” framework for evaluating and refining summaries. By leveraging a Character Knowledge Graph (CKG) extracted from input and generated summaries, NarrativeFactScore assesses the factual consistency and provides actionable guidance for refinement, such as identifying missing or erroneous facts. We demonstrate the effectiveness of NarrativeFactScore through a detailed workflow illustration and extensive validation on widely adopted benchmarks, achieving superior performance compared to competitive methods. Our results highlight the potential of agent-driven evaluation systems to improve the factual reliability of LLM-generated summaries.

Method: HEA wraps malicious requests in scenario templates featuring positive prompts formed mainly through happy endings. This approach fools LLMs into jailbreaking either immediately or at a follow-up request, requiring only up to two turns.

Result: HEA achieves 88.79% average attack success rate on state-of-the-art LLMs including GPT-4o, Llama3-70b, and Gemini-pro. The method is both efficient and effective with minimal interaction requirements.

Conclusion: The paper demonstrates that leveraging positive prompts and happy endings is an effective jailbreak strategy, providing quantitative explanations for HEA’s success and highlighting a novel vulnerability in LLM security.

Abstract: The wide adoption of Large Language Models (LLMs) has attracted significant attention from $\textit{jailbreak}$ attacks, where adversarial prompts crafted through optimization or manual design exploit LLMs to generate malicious contents. However, optimization-based attacks have limited efficiency and transferability, while existing manual designs are either easily detectable or demand intricate interactions with LLMs. In this paper, we first point out a novel perspective for jailbreak attacks: LLMs are more responsive to $\textit{positive}$ prompts. Based on this, we deploy Happy Ending Attack (HEA) to wrap up a malicious request in a scenario template involving a positive prompt formed mainly via a $\textit{happy ending}$, it thus fools LLMs into jailbreaking either immediately or at a follow-up malicious request. This has made HEA both efficient and effective, as it requires only up to two turns to fully jailbreak LLMs. Extensive experiments show that our HEA can successfully jailbreak on state-of-the-art LLMs, including GPT-4o, Llama3-70b, Gemini-pro, and achieves 88.79% attack success rate on average. We also provide quantitative explanations for the success of HEA.

Method: Based on sliding tile puzzles requiring logical planning and spatial awareness, supporting visual 3D, 2D, and text-based input modalities to assess VLM planning and reasoning skills.

Result: VLMs perform better on 2D tasks than 3D or text-based settings but struggle with complex spatial configurations and consistently fall short of human performance.

Conclusion: The benchmark reveals critical gaps in current VLM capabilities for spatial reasoning and visual alignment, highlighting limitations in achieving human-level cognition.

Abstract: Vision-Language Models (VLMs) are known to struggle with spatial reasoning and visual alignment. To help overcome these limitations, we introduce iVISPAR, an interactive multimodal benchmark designed to evaluate the spatial reasoning capabilities of VLMs acting as agents. \mbox{iVISPAR} is based on a variant of the sliding tile puzzle, a classic problem that demands logical planning, spatial awareness, and multi-step reasoning. The benchmark supports visual 3D, 2D, and text-based input modalities, enabling comprehensive assessments of VLMs’ planning and reasoning skills. We evaluate a broad suite of state-of-the-art open-source and closed-source VLMs, comparing their performance while also providing optimal path solutions and a human baseline to assess the task’s complexity and feasibility for humans. Results indicate that while VLMs perform better on 2D tasks compared to 3D or text-based settings, they struggle with complex spatial configurations and consistently fall short of human performance, illustrating the persistent challenge of visual alignment. This underscores critical gaps in current VLM capabilities, highlighting their limitations in achieving human-level cognition. Project website: https://microcosm.ai/ivispar

Method: Created Fact-or-Fair benchmark with objective (fact-based) and subjective (fairness-based) queries constructed from 19 statistics, grounded in cognitive psychology biases like representativeness, attribution, and ingroup-outgroup biases.

Result: Experiments across ten frontier models revealed different levels of fact-fair trade-offs, showing models often misalign factual accuracy with fairness considerations.

Conclusion: The paper provides both a theoretical framework and practical benchmark to advance responsible AI model assessments by clearly distinguishing between factual correctness and normative fairness dimensions.

Abstract: Recent failures such as Google Gemini generating people of color in Nazi-era uniforms illustrate how AI outputs can be factually plausible yet socially harmful. AI models are increasingly evaluated for “fairness,” yet existing benchmarks often conflate two fundamentally different dimensions: factual correctness and normative fairness. A model may generate responses that are factually accurate but socially unfair, or conversely, appear fair while distorting factual reality. We argue that identifying the boundary between fact and fair is essential for meaningful fairness evaluation. We introduce Fact-or-Fair, a benchmark with (i) objective queries aligned with descriptive, fact-based judgments, and (ii) subjective queries aligned with normative, fairness-based judgments. Our queries are constructed from 19 statistics and are grounded in cognitive psychology, drawing on representativeness bias, attribution bias, and ingroup-outgroup bias to explain why models often misalign fact and fairness. Experiments across ten frontier models reveal different levels of fact-fair trade-offs. By reframing fairness evaluation, we provide both a new theoretical lens and a practical benchmark to advance the responsible model assessments. Our test suite is publicly available at https://github.com/uclanlp/Fact-or-Fair.

Method: SAFE-SQL first prompts an LLM to generate multiple Text-to-SQL examples relevant to the test input, then filters these examples through three relevance assessments to construct high-quality in-context learning examples.

Result: SAFE-SQL surpasses previous zero-shot and few-shot Text-to-SQL frameworks, achieving higher execution accuracy, with additional performance gains in extra hard and unseen scenarios where conventional methods often fail.

Conclusion: The self-augmentation approach with fine-grained example selection enables robust Text-to-SQL performance even without access to similar training examples, making it suitable for real-world deployment.

Abstract: Text-to-SQL aims to convert natural language questions into executable SQL queries. While previous approaches, such as skeleton-masked selection, have demonstrated strong performance by retrieving similar training examples to guide large language models (LLMs), they struggle in real-world scenarios where such examples are unavailable. To overcome this limitation, we propose Self-Augmentation in-context learning with Fine-grained Example selection for Text-to-SQL (SAFE-SQL), a novel framework that improves SQL generation by generating and filtering self-augmented examples. SAFE-SQL first prompts an LLM to generate multiple Text-to-SQL examples relevant to the test input. Then SAFE-SQL filters these examples through three relevance assessments, constructing high-quality in-context learning examples. Using self-generated examples, SAFE-SQL surpasses the previous zero-shot, and few-shot Text-to-SQL frameworks, achieving higher execution accuracy. Notably, our approach provides additional performance gains in extra hard and unseen scenarios, where conventional methods often fail.

Method: Created ThinkSLM benchmark with reliable evaluation criteria comparing methods and LLM judges against human evaluations. Evaluated 72 diverse SLMs from six model families across 17 reasoning benchmarks, with experiments repeated three times for robust assessment.

Result: 1) Reasoning ability in SLMs depends more on training methods and data quality than model scale; 2) Quantization preserves reasoning capability while pruning significantly disrupts it; 3) Larger models show higher robustness, but some smaller models match or exceed larger models’ performance.

Conclusion: Scaling is not the only path to strong reasoning capabilities. SLMs with strong reasoning can be developed through structured training or post-training compression, challenging the conventional wisdom about model scale and reasoning emergence.

Abstract: Reasoning has long been viewed as an emergent property of large language models (LLMs). However, recent studies challenge this assumption, showing that small language models (SLMs) can also achieve competitive reasoning performance. This paper introduces ThinkSLM, the first extensive benchmark to systematically evaluate and study the reasoning abilities of SLMs trained from scratch or derived from LLMs through quantization, pruning, and distillation. We first establish a reliable evaluation criterion comparing available methods and LLM judges against our human evaluations. Then we present a study evaluating 72 diverse SLMs from six major model families across 17 reasoning benchmarks. We repeat all our experiments three times to ensure a robust assessment. Our findings show that: 1) reasoning ability in SLMs is strongly influenced by training methods and data quality rather than solely model scale; 2) quantization preserves reasoning capability, while pruning significantly disrupts it; 3) larger models consistently exhibit higher robustness against adversarial perturbations and intermediate reasoning, but certain smaller models closely match or exceed the larger models’ performance. Our findings challenge the assumption that scaling is the only way to achieve strong reasoning. Instead, we foresee a future where SLMs with strong reasoning capabilities can be developed through structured training or post-training compression. Our ThinkSLM Leaderboard is publicly available at: https://ctrl-gaurav.github.io/thinkslm.github.io/

Method: FANformer adapts Fourier Analysis Network (FAN) into attention mechanism by modifying the feature projection process to achieve efficient periodicity modeling.

Result: FANformer consistently outperforms Transformer when scaling up model size and training tokens, with FANformer-1B showing marked improvements on downstream tasks. It exhibits superior ability to learn and apply rules for reasoning.

Conclusion: FANformer is an effective and promising architecture for advancing LLMs, demonstrating superior learning efficiency and reasoning capabilities compared to Transformer.

Abstract: Periodicity, as one of the most important basic characteristics, lays the foundation for facilitating structured knowledge acquisition and systematic cognitive processes within human learning paradigms. However, the potential flaws of periodicity modeling in Transformer affect the learning efficiency and establishment of underlying principles from data for large language models (LLMs) built upon it. In this paper, we demonstrate that integrating effective periodicity modeling can improve the learning efficiency and performance of LLMs. We introduce FANformer, which adapts Fourier Analysis Network (FAN) into attention mechanism to achieve efficient periodicity modeling, by modifying the feature projection process of attention mechanism. Extensive experimental results on language modeling show that FANformer consistently outperforms Transformer when scaling up model size and training tokens, underscoring its superior learning efficiency. Our pretrained FANformer-1B exhibits marked improvements on downstream tasks compared to open-source LLMs with similar model parameters or training tokens. Moreover, we reveal that FANformer exhibits superior ability to learn and apply rules for reasoning compared to Transformer. The results position FANformer as an effective and promising architecture for advancing LLMs.

Method: Used an evaluation framework that systematically varies human-specified risk structures (rewards/penalties for correct answers, incorrect answers, and refusals) while keeping tasks fixed, and tested a skill-decomposition method that isolates independent skills for answer-or-defer decision making.

Result: LMs exhibit flawed decision policies across multiple datasets and models: they over-answer in high-risk settings and over-defer in low-risk settings. The skill-decomposition method consistently improved LMs’ decision policies.

Conclusion: Current LMs have limitations in risk-conditioned decision making, but skill-decomposition provides practical guidance for deploying more reliable LM-based agents across varying risk level applications.

Abstract: Language models (LMs) are increasingly used to build agents that can act autonomously to achieve goals. During this automatic process, agents need to take a series of actions, some of which might lead to severe consequences if incorrect actions are taken. Therefore, such agents must sometimes defer-refusing to act when their confidence is insufficient-to avoid the potential cost of incorrect actions. Because the severity of consequences varies across applications, the tendency to defer should also vary: in low-risk settings agents should answer more freely, while in high-risk settings their decisions should be more conservative. We study this “answer-or-defer” problem with an evaluation framework that systematically varies human-specified risk structures-rewards and penalties for correct answers, incorrect answers, and refusals $(r_{\mathrm{cor}},r_{\mathrm{inc}}, r_{\mathrm{ref}})$-while keeping tasks fixed. This design evaluates LMs’ risk-aware decision policies by measuring their ability to maximize expected reward. Across multiple datasets and models, we identify flaws in their decision policies: LMs tend to over-answer in high-risk settings and over-defer in low-risk settings. After analyzing the potential cause of such flaws, we find that a simple skill-decomposition method, which isolates the independent skills required for answer-or-defer decision making, can consistently improve LMs’ decision policies. Our results highlight the current limitations of LMs in risk-conditioned decision making and provide practical guidance for deploying more reliable LM-based agents across applications of varying risk levels.

Method: Created ONERULER through a two-step process: writing English instructions for tasks, then collaborating with native speakers to translate them into 25 additional languages. Includes seven synthetic tasks testing retrieval and aggregation, including variations of “needle-in-a-haystack” tasks.

Result: Experiments revealed a widening performance gap between low- and high-resource languages as context length increased from 8K to 128K tokens. Surprisingly, English ranked 6th out of 26 languages, with Polish emerging as top performer. Many LLMs incorrectly predicted absence of answers even in high-resource languages. Cross-lingual scenarios showed performance fluctuations up to 20% based on instruction language.

Conclusion: ONERULER facilitates research into improving multilingual and cross-lingual long-context training pipelines, highlighting significant performance variations across languages and the need for better multilingual capabilities in long-context models.

Abstract: We present ONERULER, a multilingual benchmark designed to evaluate long-context language models across 26 languages. ONERULER adapts the English-only RULER benchmark (Hsieh et al., 2024) by including seven synthetic tasks that test both retrieval and aggregation, including new variations of the “needle-in-a-haystack” task that allow for the possibility of a nonexistent needle. We create ONERULER through a two-step process, first writing English instructions for each task and then collaborating with native speakers to translate them into 25 additional languages. Experiments with both open-weight and closed LLMs reveal a widening performance gap between low- and high-resource languages as context length increases from 8K to 128K tokens. Surprisingly, English is not the top-performing language on long-context tasks (ranked 6th out of 26), with Polish emerging as the top language. Our experiments also show that many LLMs (particularly OpenAI’s o3-mini-high) incorrectly predict the absence of an answer, even in high-resource languages. Finally, in cross-lingual scenarios where instructions and context appear in different languages, performance can fluctuate by up to 20% depending on the instruction language. We hope the release of ONERULER will facilitate future research into improving multilingual and cross-lingual long-context training pipelines.

Method: Proposes Reasoning-enhanced Finetuning (Rational) framework that trains models to engage in explicit safe reasoning before response generation, leveraging pretraining knowledge through self-generated structured reasoning.

Result: Fine-tuned models demonstrate improved safety by internalizing context-sensitive decision-making, enabling them to reject harmful prompts while providing meaningful responses in complex scenarios.

Conclusion: Safety extends beyond simple refusal and requires context awareness; reasoning is not only a core LLM capability but also a fundamental mechanism for robust, interpretable, and adaptive LLM safety.

Abstract: Large Language Models (LLMs) are vulnerable to jailbreak attacks that exploit weaknesses in traditional safety alignment, which often relies on rigid refusal heuristics or representation engineering to block harmful outputs. While they are effective for direct adversarial attacks, they fall short of broader safety challenges requiring nuanced, context-aware decision-making. To address this, we propose Reasoning-enhanced Finetuning for interpretable LLM Safety (Rational), a novel framework that trains models to engage in explicit safe reasoning before response. Fine-tuned models leverage the extensive pretraining knowledge in self-generated reasoning to bootstrap their own safety through structured reasoning, internalizing context-sensitive decision-making. Our findings suggest that safety extends beyond refusal, requiring context awareness for more robust, interpretable, and adaptive responses. Reasoning is not only a core capability of LLMs but also a fundamental mechanism for LLM safety. Rational employs reasoning-enhanced fine-tuning, allowing it to reject harmful prompts while providing meaningful and context-aware responses in complex scenarios.

Method: Propose value profiles - descriptions of underlying values compressed from in-context demonstrations, along with a steerable decoder model that estimates individual ratings from rater representations. Use information-theoretic methodology to measure predictive information.

Result: Demonstrations contain most information, followed by value profiles (70%+ information preservation), then demographics. Value profiles better explain rater variation than demographic groupings and offer scrutability, interpretability, and steerability advantages.

Conclusion: Value profiles provide novel, predictive ways to describe individual variation beyond demographics or group information, with practical benefits for understanding and simulating annotator behavior.

Abstract: Modelling human variation in rating tasks is crucial for personalization, pluralistic model alignment, and computational social science. We propose representing individuals using natural language value profiles – descriptions of underlying values compressed from in-context demonstrations – along with a steerable decoder model that estimates individual ratings from a rater representation. To measure the predictive information in a rater representation, we introduce an information-theoretic methodology and find that demonstrations contain the most information, followed by value profiles, then demographics. However, value profiles effectively compress the useful information from demonstrations (>70% information preservation) and offer advantages in terms of scrutability, interpretability, and steerability. Furthermore, clustering value profiles to identify similarly behaving individuals better explains rater variation than the most predictive demographic groupings. Going beyond test set performance, we show that the decoder predictions change in line with semantic profile differences, are well-calibrated, and can help explain instance-level disagreement by simulating an annotator population. These results demonstrate that value profiles offer novel, predictive ways to describe individual variation beyond demographics or group information.

Method: Identify attention heads driving short reasoning behavior and edit their output projection weights to remove the short reasoning direction, modifying only 0.2% of parameters.

Result: Reduces overly short reasoning and improves accuracy: +6.39% for short reasoning outputs, +3.34% overall across multiple math benchmarks.

Conclusion: Reasoning length is controlled by linear directions in representation space, and fine-grained interventions can effectively improve reasoning quality in LLMs.

Abstract: Recent studies have shown that Large Language Models (LLMs) augmented with chain-of-thought (CoT) reasoning demonstrate impressive problem-solving abilities. However, in this work, we identify a recurring issue where these models occasionally generate overly short reasoning, leading to degraded performance on even simple mathematical problems. Specifically, we investigate how reasoning length is embedded in the hidden representations of reasoning models and its impact on accuracy. Our analysis reveals that reasoning length is governed by a linear direction in the representation space, allowing us to induce overly short reasoning by steering the model along this direction. Building on this insight, we introduce ThinkEdit, a simple yet effective weight-editing approach to mitigate the issue of overly short reasoning. We first identify a small subset of attention heads (approximately 4%) that predominantly drive short reasoning behavior. We then edit the output projection weights of these heads to remove the short reasoning direction. With changes to only 0.2% of the model’s parameters, ThinkEdit effectively reduces overly short reasoning and yields notable accuracy gains for short reasoning outputs (+6.39%), along with an overall improvement across multiple math benchmarks (+3.34%). Our findings provide new mechanistic insights into how reasoning length is controlled within LLMs and highlight the potential of fine-grained model interventions to improve reasoning quality. Our code is available at: https://github.com/Trustworthy-ML-Lab/ThinkEdit

Method: Propose Adaptive Rectification Sampling (AR-Sampling) that uses process-supervised reward models as verifiers and trigger sentences to guide LLMs in adaptive step-level rethinking.

Result: Experiments on GSM8K and MATH500 show AR-Sampling enables fine-grained rethinking, improves solution accuracy, and generates reasonable additional tokens.

Conclusion: AR-Sampling provides an effective approach for targeted self-correction at appropriate reasoning steps, balancing performance improvement with token efficiency.

Abstract: The newly released OpenAI-o1 and DeepSeek-R1 have demonstrated that test-time scaling can significantly improve model performance, especially in complex tasks such as logical reasoning. Common test-time scaling methods involve generating more chains of thought (CoTs) or longer CoTs with self-correction. However, while self-correction can improve performance, it may lead to significant token waste and reduce readability of the CoT if the reasoning steps are already correct. To demonstrate that large language models (LLMs) can rectify errors at a more fine-grained level, we propose Adaptive Rectification Sampling (AR-Sampling), which can guide the LLMs to self-correction at the appropriate step. AR-Sampling leverages a process-supervised reward model (PRM) as a verifier and constructed trigger sentences to guide the model in adaptive step-level rethinking. Through the experiments on GSM8K and MATH500, it indicates that our approach enables the models to rethink in more fine-grained level, improving the accuracy of solutions, while generating a reasonable number of additional tokens.

Method: Uses semi-greedy search to identify which mismatches between target and draft models need correction, then trains a lightweight classifier on LLM embeddings to predict which mismatching tokens can be safely accepted without quality degradation.

Result: Achieved ~2x speedup over speculative decoding on GSM8k with Llama 3.1 70B (≤1% accuracy drop), and accepted ≥25 tokens per speculation cycle on LiveCodeBench with 2% drop in Pass@1.

Conclusion: AutoJudge provides significant inference speedups with minimal quality loss, requires no human annotation, and is easily integrated into modern LLM inference frameworks.

Abstract: We introduce AutoJudge, a method that accelerates large language model (LLM) inference with task-specific lossy speculative decoding. Instead of matching the original model output distribution token-by-token, we identify which of the generated tokens affect the downstream quality of the response, relaxing the distribution match guarantee so that the “unimportant” tokens can be generated faster. Our approach relies on a semi-greedy search algorithm to test which of the mismatches between target and draft models should be corrected to preserve quality and which ones may be skipped. We then train a lightweight classifier based on existing LLM embeddings to predict, at inference time, which mismatching tokens can be safely accepted without compromising the final answer quality. We evaluate the effectiveness of AutoJudge with multiple draft/target model pairs on mathematical reasoning and programming benchmarks, achieving significant speedups at the cost of a minor accuracy reduction. Notably, on GSM8k with the Llama 3.1 70B target model, our approach achieves up to $\approx2\times$ speedup over speculative decoding at the cost of $\le 1%$ drop in accuracy. When applied to the LiveCodeBench benchmark, AutoJudge automatically detects programming-specific important tokens, accepting $\ge 25$ tokens per speculation cycle at $2%$ drop in Pass@1. Our approach requires no human annotation and is easy to integrate with modern LLM inference frameworks.

Method: Adds small parameter modules to feedforward blocks using distillation with synthetic training data, keeping original weights unchanged.

Result: Recovers math capabilities without harming language tasks, reduces active parameters (~2B cut), decreases latency (>16% TTNT reduction), and shortens responses (up to 8.5% fewer tokens).

Conclusion: Caprese effectively bridges the efficiency-performance gap in LLM math reasoning while maintaining computational benefits.

Abstract: Due to long generations, large language model (LLM) math reasoning demands significant computational resources and time. While many existing efficient inference methods have been developed with excellent performance preservation on language tasks, they often severely degrade math performance. In this paper, we propose Caprese, a resource-efficient distillation method to recover lost capabilities from deploying efficient inference methods, focused primarily in feedforward blocks. With original weights unperturbed, roughly 1% of additional parameters, and only 20K synthetic training samples, we are able to recover much if not all of the math capabilities lost from efficient inference for thinking LLMs and without harm to language tasks for instruct LLMs. Moreover, Caprese slashes the number of active parameters (~2B cut for Gemma 2 9B and Llama 3.1 8B) and integrates cleanly into existing model layers to reduce latency (>16% time-to-next-token reduction) while encouraging response brevity (up to 8.5% fewer tokens).

Method: Introduced social sycophancy concept and ELEPHANT benchmark, tested 11 LLM models on general advice queries and Reddit’s r/AmITheAsshole scenarios, analyzed moral conflict responses, and examined mitigation strategies.

Result: LLMs preserve users’ face 45 percentage points more than humans, affirm both sides in moral conflicts 48% of the time, and social sycophancy is rewarded in preference datasets. Existing mitigation strategies have limited effectiveness.

Conclusion: Social sycophancy is prevalent in LLMs, existing mitigations are insufficient, but model-based steering shows promise. The work provides theoretical grounding and benchmark for addressing sycophancy in open-ended LLM contexts.

Abstract: LLMs are known to exhibit sycophancy: agreeing with and flattering users, even at the cost of correctness. Prior work measures sycophancy only as direct agreement with users’ explicitly stated beliefs that can be compared to a ground truth. This fails to capture broader forms of sycophancy such as affirming a user’s self-image or other implicit beliefs. To address this gap, we introduce social sycophancy, characterizing sycophancy as excessive preservation of a user’s face (their desired self-image), and present ELEPHANT, a benchmark for measuring social sycophancy in an LLM. Applying our benchmark to 11 models, we show that LLMs consistently exhibit high rates of social sycophancy: on average, they preserve user’s face 45 percentage points more than humans in general advice queries and in queries describing clear user wrongdoing (from Reddit’s r/AmITheAsshole). Furthermore, when prompted with perspectives from either side of a moral conflict, LLMs affirm both sides (depending on whichever side the user adopts) in 48% of cases–telling both the at-fault party and the wronged party that they are not wrong–rather than adhering to a consistent moral or value judgment. We further show that social sycophancy is rewarded in preference datasets, and that while existing mitigation strategies for sycophancy are limited in effectiveness, model-based steering shows promise for mitigating these behaviors. Our work provides theoretical grounding and an empirical benchmark for understanding and addressing sycophancy in the open-ended contexts that characterize the vast majority of LLM use cases.

Method: Proposed Debate, Train, Evolve (DTE) framework using multi-agent debate traces, and Reflect-Critique-Refine prompting strategy to improve debate quality through explicit critique and refinement.

Result: Achieved average accuracy gain of 8.92% on GSM-PLUS dataset and 5.8% average gain across seven reasoning benchmarks, showing strong cross-domain generalization.

Conclusion: DTE framework effectively captures general reasoning capabilities and enables autonomous model improvement without ground truth data, demonstrating practical value for reasoning enhancement.

Abstract: Large language models (LLMs) have improved significantly in their reasoning through extensive training on massive datasets. However, relying solely on additional data for improvement is becoming increasingly impractical, highlighting the need for models to autonomously enhance their reasoning without external supervision. In this paper, we propose Debate, Train, Evolve (DTE), a novel ground truth-free training framework that uses multi-agent debate traces to evolve a single language model. We also introduce a new prompting strategy Reflect-Critique-Refine, to improve debate quality by explicitly instructing agents to critique and refine their reasoning. Extensive evaluations on seven reasoning benchmarks with six open-weight models show that our DTE framework achieve substantial improvements, with an average accuracy gain of 8.92% on the challenging GSM-PLUS dataset. Furthermore, we observe strong cross-domain generalization, with an average accuracy gain of 5.8% on all other benchmarks, suggesting that our method captures general reasoning capabilities. Our framework code and trained models are publicly available at https://github.com/ctrl-gaurav/Debate-Train-Evolve

Method: Used Distributed Interchange Interventions on language models to analyze English filler-gap constructions like questions and relative clauses.

Result: LMs converge on similar abstract analyses of these constructions and reveal overlooked factors (frequency, filler type, context) that could motivate changes to linguistic theory.

Conclusion: Mechanistic internal analyses of language models can advance linguistic theory by revealing shared abstract mechanisms and previously overlooked linguistic factors.

Abstract: Language Models (LMs) have emerged as powerful sources of evidence for linguists seeking to develop theories of syntax. In this paper, we argue that causal interpretability methods, applied to LMs, can greatly enhance the value of such evidence by helping us characterize the abstract mechanisms that LMs learn to use. Our empirical focus is a set of English filler-gap dependency constructions (e.g., questions, relative clauses). Linguistic theories largely agree that these constructions share many properties. Using experiments based in Distributed Interchange Interventions, we show that LMs converge on similar abstract analyses of these constructions. These analyses also reveal previously overlooked factors – relating to frequency, filler type, and surrounding context – that could motivate changes to standard linguistic theory. Overall, these results suggest that mechanistic, internal analyses of LMs can push linguistic theory forward.

Method: Measure relative information content of representations for semantically related data. Analyze how semantic information is encoded across multiple tokens in LLMs and vision transformers. Study translated sentence pairs in LLMs and image-caption relationships between vision transformers and LLMs.

Result: Identified inner ‘semantic’ layers in LLMs containing most language-transferable information. Larger LLMs (DeepSeek-V3) extract more general information than smaller ones (Llama3.1-8B). Semantic information in English text spreads across many tokens with long-distance correlations and left-to-right asymmetry. Also identified semantic layers in vision transformers where caption representations predict corresponding image representations.

Conclusion: Deep neural networks develop domain-invariant semantic representations that can be quantitatively measured. Semantic information exhibits specific encoding patterns across tokens and layers, with model-dependent information asymmetries between different modalities like images and text.

Abstract: Deep neural networks are known to develop similar representations for semantically related data, even when they belong to different domains, such as an image and its description, or the same text in different languages. We present a method for quantitatively investigating this phenomenon by measuring the relative information content of the representations of semantically related data and probing how it is encoded into multiple tokens of large language models (LLMs) and vision transformers. Looking first at how LLMs process pairs of translated sentences, we identify inner ``semantic’’ layers containing the most language-transferable information. We find moreover that, on these layers, a larger LLM (DeepSeek-V3) extracts significantly more general information than a smaller one (Llama3.1-8B). Semantic information of English text is spread across many tokens and it is characterized by long-distance correlations between tokens and by a causal left-to-right (i.e., past-future) asymmetry. We also identify layers encoding semantic information within visual transformers. We show that caption representations in the semantic layers of LLMs predict visual representations of the corresponding images. We observe significant and model-dependent information asymmetries between image and text representations.

Method: The authors present a unified conceptual framework to standardize how the ALG task is defined, and offer ALG benchmarking guidelines including recommendations for datasets and metrics.

Result: The paper provides the first comprehensive overview of ALG research, identifies fundamental differences in assumptions, scope, and output formats across existing approaches.

Conclusion: The authors outline challenges and new directions for ALG, advocating for broader coverage by including all reported results and richer metadata to improve automated leaderboard curation.

Abstract: An important task in machine learning (ML) research is comparing prior work, which is often performed via ML leaderboards: a tabular overview of experiments with comparable conditions (e.g., same task, dataset, and metric). However, the growing volume of literature creates challenges in creating and maintaining these leaderboards. To ease this burden, researchers have developed methods to extract leaderboard entries from research papers for automated leaderboard curation. Yet, prior work varies in problem framing, complicating comparisons and limiting real-world applicability. In this position paper, we present the first overview of Automatic Leaderboard Generation (ALG) research, identifying fundamental differences in assumptions, scope, and output formats. We propose an ALG unified conceptual framework to standardise how the ALG task is defined. We offer ALG benchmarking guidelines, including recommendations for datasets and metrics that promote fair, reproducible evaluation. Lastly, we outline challenges and new directions for ALG, such as, advocating for broader coverage by including all reported results and richer metadata.

Method: Overfit triggers to fixed benign-sounding positive reply prefixes using harmless QA pairs, then use gradient-based coordinate optimization to enhance universal triggers.

Result: Achieved 86.67% and 85% attack success rates on LLaMA-3-8B and Qwen-2.5-7B respectively under GPT-4o judgment, effectively jailbreaking LLMs while evading guardrail detection.

Conclusion: The proposed clean-data backdoor attack successfully bypasses safety guardrails by leveraging benign prefixes and model’s language completion capabilities, demonstrating significant vulnerability in current LLM safety mechanisms.

Abstract: Supervised fine-tuning (SFT) aligns large language models (LLMs) with human intent by training them on labeled task-specific data. Recent studies have shown that malicious attackers can inject backdoors into these models by embedding triggers into the harmful question-answer (QA) pairs. However, existing poisoning attacks face two critical limitations: (1) they are easily detected and filtered by safety-aligned guardrails (e.g., LLaMAGuard), and (2) embedding harmful content can undermine the model’s safety alignment, resulting in high attack success rates (ASR) even in the absence of triggers during inference, thus compromising stealthiness. To address these issues, we propose a novel \textit{clean-data backdoor attack} for jailbreaking LLMs. Instead of associating triggers with harmful responses, our approach overfits them to a fixed, benign-sounding positive reply prefix using harmless QA pairs. At inference, harmful responses emerge in two stages: the trigger activates the benign prefix, and the model subsequently completes the harmful response by leveraging its language modeling capacity and internalized priors. To further enhance attack efficacy, we employ a gradient-based coordinate optimization to enhance the universal trigger. Extensive experiments demonstrate that our method can effectively \textit{jailbreak backdoor} various LLMs even under the detection of guardrail models, \textit{e.g.}, an ASR of 86.67% and 85% on LLaMA-3-8B and Qwen-2.5-7B judged by GPT-4o.

Method: Proposes GIM technique that accounts for self-repair during backpropagation by addressing how softmax redistribution conceals influence of important attention scores.

Result: GIM significantly improves faithfulness over existing circuit identification and feature attribution methods across multiple LLMs (Gemma, LLAMA, Qwen) and diverse tasks.

Conclusion: GIM represents a significant step toward better understanding LLM inner mechanisms, crucial for model improvement and safety.

Abstract: Ensuring faithful interpretability in large language models is imperative for trustworthy and reliable AI. A key obstacle is self-repair, a phenomenon where networks compensate for reduced signal in one component by amplifying others, masking the true importance of the ablated component. While prior work attributes self-repair to layer normalization and back-up components that compensate for ablated components, we identify a novel form occurring within the attention mechanism, where softmax redistribution conceals the influence of important attention scores. This leads traditional ablation and gradient-based methods to underestimate the significance of all components contributing to these attention scores. We introduce Gradient Interaction Modifications (GIM), a technique that accounts for self-repair during backpropagation. Extensive experiments across multiple large language models (Gemma 2B/9B, LLAMA 1B/3B/8B, Qwen 1.5B/3B) and diverse tasks demonstrate that GIM significantly improves faithfulness over existing circuit identification and feature attribution methods. Our work is a significant step toward better understanding the inner mechanisms of LLMs, which is crucial for improving them and ensuring their safety. Our code is available at https://github.com/JoakimEdin/gim.

Method: Given a prompt and thousands of random human-written snippets, the model produces narratives under the constraint that most tokens (90%) must be copied verbatim from provided paragraphs, with minimal editing to stitch fragments into coherent stories.

Result: Frankentexts significantly improve over vanilla LLM generations in writing quality, diversity, and originality while remaining coherent. 72% are misclassified as human-written by state-of-the-art detectors. Human evaluators praise inventive premises and vivid descriptions but note issues with tonal shifts and uneven grammar.

Conclusion: Frankentexts demonstrate LLMs’ ability to compose high-quality narratives from existing texts, posing challenges to AI detection and raising fundamental questions about authorship and copyright when humans provide materials and LLMs orchestrate them.

Abstract: We introduce Frankentexts, a long-form narrative generation paradigm that treats an LLM as a composer of existing texts rather than as an author. Given a writing prompt and thousands of randomly sampled human-written snippets, the model is asked to produce a narrative under the extreme constraint that most tokens (e.g., 90%) must be copied verbatim from the provided paragraphs. This task is effectively intractable for humans: selecting and ordering snippets yields a combinatorial search space that an LLM implicitly explores, before minimally editing and stitching together selected fragments into a coherent long-form story. Despite the extreme challenge of the task, we observe through extensive automatic and human evaluation that Frankentexts significantly improve over vanilla LLM generations in terms of writing quality, diversity, and originality while remaining coherent and relevant to the prompt. Furthermore, Frankentexts pose a fundamental challenge to detectors of AI-generated text: 72% of Frankentexts produced by our best Gemini 2.5 Pro configuration are misclassified as human-written by Pangram, a state-of-the-art detector. Human annotators praise Frankentexts for their inventive premises, vivid descriptions, and dry humor; on the other hand, they identify issues with abrupt tonal shifts and uneven grammar across segments, particularly in longer pieces. The emergence of high-quality Frankentexts raises serious questions about authorship and copyright: when humans provide the raw materials and LLMs orchestrate them into new narratives, who truly owns the result?

Method: A lightweight extension using point-and-copy approach that identifies relevant image patches and copies their embeddings back into reasoning stream, keeping perceptual evidence in the same semantic space.

Result: v1 consistently outperforms comparable baselines across various multimodal mathematical reasoning benchmarks.

Conclusion: Dynamic visual access through point-and-copy is a practical mechanism for grounded reasoning, with model and 300K dataset (v1g) publicly available.

Abstract: When thinking with images, humans rarely rely on a single glance: they revisit visual information repeatedly during reasoning. However, existing models typically process images only once and thereafter generate reasoning entirely in text, lacking mechanisms to re-access or ground inference in visual representations. We empirically confirm this: as reasoning chains lengthen, models progressively lose focus on relevant regions. In response, we introduce v1, a lightweight extension that enables active visual referencing through a simple point-and-copy approach. This allows the model to identify relevant image patches and copy their embeddings back into the reasoning stream, ensuring that evolving hypotheses remain grounded in perceptual evidence. Crucially, our pointing strategy lets the MLLM directly select image patches using their semantic representations as keys, keeping perceptual evidence embedded in the same space as the model’s reasoning. To train this capability, we construct v1g, a dataset of 300K multimodal reasoning traces with interleaved visual grounding annotations. Across various multimodal mathematical reasoning benchmarks, v1 consistently outperforms comparable baselines, establishing dynamic visual access based on point-and-copy as a practical mechanism for grounded reasoning. The model checkpoint and dataset are available at github.com/jun297/v1.

Method: Developed SelfReflect metric - an information-theoretic distance between summary and answer distribution. Tested LLMs through interventional and human studies using reasoning, chains-of-thought, and explicit finetuning approaches.

Result: Modern LLMs are universally incapable of naturally revealing their uncertainties. However, they can generate faithful uncertainty summaries when provided with multiple sampled outputs as context.

Conclusion: SelfReflect enables measuring faithfulness of uncertainty communication. While LLMs lack natural uncertainty reflection capability, feeding sampled outputs as context provides a promising approach for transparent uncertainty communication.

Abstract: The common approach to communicate a large language model’s (LLM) uncertainty is to add a percentage number or a hedging word to its response. But is this all we can do? Instead of generating a single answer and then hedging it, an LLM that is fully transparent to the user needs to be able to reflect on its internal belief distribution and output a summary of all options it deems possible, and how likely they are. To test whether LLMs possess this capability, we develop the SelfReflect metric, an information-theoretic distance between a given summary and a distribution over answers. In interventional and human studies, we find that SelfReflect indicates even slight deviations, yielding a fine measure of faithfulness between a summary string and an LLM’s actual internal distribution over answers. With SelfReflect, we make a resounding negative observation: modern LLMs are, across the board, incapable of revealing what they are uncertain about, neither through reasoning, nor chains-of-thoughts, nor explicit finetuning. However, we do find that LLMs are able to generate faithful summaries of their uncertainties if we help them by sampling multiple outputs and feeding them back into the context. This simple approach shines a light at the universal way of communicating LLM uncertainties whose future development the SelfReflect score enables.

Method: LoLA distributes past key-value pairs into: (1) recent pairs in local sliding window cache, (2) difficult-to-memorize pairs in sparse global cache, and (3) generic pairs in recurrent hidden state of linear attention, using self-recall error metric for efficient memory management.

Result: On pass-key retrieval tasks, LoLA improves base model performance from 0.6% to 97.4% accuracy with 4.6x smaller cache than Llama-3.1 8B on 4K context. Also outperforms other subquadratic models on zero-shot commonsense reasoning.

Conclusion: LoLA effectively boosts associative recall in linear attention models, enabling efficient lifelong in-context learning with minimal memory overhead.

Abstract: The per-token cost of transformer inference scales with context length, preventing its application to lifelong in-context learning. Linear attention is an efficient alternative that maintains a constant memory footprint, even on infinite context lengths. While this is a potential candidate for lifelong learning, it falls short in memory capacity. In this paper, we propose LoLA, a training-free augmentation to linear attention that boosts associative recall. LoLA distributes past key-value pairs from context into three memory systems: (i) recent pairs in a local sliding window cache; (ii) difficult-to-memorize pairs in a sparse, global cache; and (iii) generic pairs in the recurrent hidden state of linear attention. We show through ablations that our self-recall error metric is crucial to efficiently manage long-term associative memories. On pass-key retrieval tasks, LoLA improves the base model’s performance from 0.6% to 97.4% accuracy. This is achieved with a 4.6x smaller cache than Llama-3.1 8B on 4K context length. LoLA also outperforms other 1B and 8B parameter subquadratic models on zero-shot commonsense reasoning tasks.

Method: Extract word embeddings from pre-trained Sentence Transformer, apply sentence-level principal component analysis, then use either knowledge distillation or contrastive learning to improve embeddings. Sentence representation is achieved through simple averaging of word embeddings.

Result: The model substantially outperforms existing static models on sentence semantic tasks and even surpasses a basic Sentence Transformer model (SimCSE) on text embedding benchmarks.

Conclusion: The method successfully removes irrelevant word embedding components for sentence semantics and adjusts vector norms based on word influence, providing efficient and effective sentence representations.

Abstract: We propose new static word embeddings optimised for sentence semantic representation. We first extract word embeddings from a pre-trained Sentence Transformer, and improve them with sentence-level principal component analysis, followed by either knowledge distillation or contrastive learning. During inference, we represent sentences by simply averaging word embeddings, which requires little computational cost. We evaluate models on both monolingual and cross-lingual tasks and show that our model substantially outperforms existing static models on sentence semantic tasks, and even surpasses a basic Sentence Transformer model (SimCSE) on a text embedding benchmark. Lastly, we perform a variety of analyses and show that our method successfully removes word embedding components that are not highly relevant to sentence semantics, and adjusts the vector norms based on the influence of words on sentence semantics.

Method: Proposes a reinforcement learning framework with joint sampling that generates paired responses with/without retrieval, learning local and global advantages to quantify when to ignore misleading context versus adopt it. Uses asymmetric advantage transformation to amplify exploratory behaviors toward parametric knowledge.

Result: Significantly improves robustness and reasoning accuracy in knowledge conflict scenarios and general RAG scenarios, outperforming SOTA baselines by 23% in counterfactual scenarios, without degradation when retrieved context is accurate.

Conclusion: Knowledgeable-R1 effectively trains LLMs to resist contextual interference while still exploiting external context when reliably helpful, improving RAG robustness.

Abstract: Retrieval-augmented generation (RAG) improves performance on knowledge-intensive tasks but can be derailed by wrong, irrelevant, or conflicting retrieved text, causing models to rely on inaccurate evidence and cascade errors. We propose Knowledgeable-R1, a reinforcement-learning framework that explicitly trains large language models to use parametric knowledge (PK) to resist contextual interference while still exploiting external context when it is reliably helpful. Knowledgeable-R1 introduces a joint sampling scheme that generates paired responses with and without retrieval, and learns both local advantages (within each decoding regime) and global advantages under the same input to quantify when to ignore misleading context versus adopt it. We employ an asymmetric advantage transformation that amplifies exploratory behaviors toward parametric knowledge. Experiments show that \method significantly improves robustness and reasoning accuracy in knowledge conflict scenarios and general RAG scenarios, outperforming SOTA baselines by 23% in counterfactual scenarios, and without degradation when the retrieved context is fully accurate.Our code are available at https://github.com/lcy80366872/knowledgeable-R1.

Method: Created ViMUL-Bench with 8k manually verified samples across 14 languages, covering 15 categories including culturally diverse topics. Also developed ViMUL model using a machine-translated multilingual video training set of 1.2 million samples.

Result: The proposed ViMUL model provides better tradeoff between high- and low-resource languages for video understanding compared to existing approaches.

Conclusion: The ViMUL-Bench benchmark, multilingual video LMM, and large-scale training dataset will facilitate future research in developing culturally and linguistically inclusive multilingual video LMMs.

Abstract: Large multimodal models (LMMs) have recently gained attention due to their effectiveness to understand and generate descriptions of visual content. Most existing LMMs are in English language. While few recent works explore multilingual image LMMs, to the best of our knowledge, moving beyond the English language for cultural and linguistic inclusivity is yet to be investigated in the context of video LMMs. In pursuit of more inclusive video LMMs, we introduce a multilingual Video LMM benchmark, named ViMUL-Bench, to evaluate Video LMMs across 14 languages, including both low- and high-resource languages: English, Chinese, Spanish, French, German, Hindi, Arabic, Russian, Bengali, Urdu, Sinhala, Tamil, Swedish, and Japanese. Our ViMUL-Bench is designed to rigorously test video LMMs across 15 categories including eight culturally diverse categories, ranging from lifestyles and festivals to foods and rituals and from local landmarks to prominent cultural personalities. ViMUL-Bench comprises both open-ended (short and long-form) and multiple-choice questions spanning various video durations (short, medium, and long) with 8k samples that are manually verified by native language speakers. In addition, we also introduce a machine translated multilingual video training set comprising 1.2 million samples and develop a simple multilingual video LMM, named ViMUL, that is shown to provide a better tradeoff between high-and low-resource languages for video understanding. We hope our ViMUL-Bench and multilingual video LMM along with a large-scale multilingual video training set will help ease future research in developing cultural and linguistic inclusive multilingual video LMMs. Our proposed benchmark, video LMM and training data will be publicly released at https://mbzuai-oryx.github.io/ViMUL/.

Method: ConfQA fine-tuning strategy trains models to output correct answers when confident and “I am unsure” otherwise, using dampening prompts and atomic factual data. ConfRAG builds on this to invoke RAG only when model responds with uncertainty.

Result: Hallucination rates reduced from 20-40% to below 5% across benchmarks. ConfRAG achieves above 95% accuracy while reducing unnecessary external retrievals by over 30%.

Conclusion: The combined approach effectively reduces hallucination while optimizing retrieval costs, demonstrating practical deployment value for factual question answering systems.

Abstract: Can Large Language Models (LLMs) be trained to avoid hallucinating factual statements, and can Retrieval-Augmented Generation (RAG) be triggered only when necessary to reduce retrieval and computation costs? In this work, we address both challenges simultaneously. We introduce ConfQA, a fine-tuning strategy that reduces hallucination rates from 20-40% to below 5% across multiple factuality benchmarks. The approach is simple: when the model answers correctly, it is trained to output the answer; otherwise, it is trained to respond with “I am unsure”. Two design choices make this training effective: (1) a dampening prompt (“answer only if you are confident”) that explicitly discourages overconfident hallucinations, and (2) training data drawn from atomic factual statements (e.g., knowledge graph attribute values), which calibrates model confidence and yields robust generalization across domains and question types. Building on ConfQA, we propose ConfRAG, a triggering strategy that invokes RAG only when the model responses with unsure. This framework achieves accuracy above 95% in ideal case while reducing unnecessary external retrievals by over 30%.

Method: Adaptive-k retrieval adaptively selects the number of passages based on the distribution of similarity scores between query and candidate passages, working with existing retriever-reader pipelines without model fine-tuning or extra LLM inferences.

Result: On both factoid and aggregation QA benchmarks, Adaptive-k matches or outperforms fixed-k baselines while using up to 10x fewer tokens than full-context input, yet still retrieves 70% of relevant passages. It improves accuracy across five LCLMs and two embedding models.

Conclusion: Dynamically adjusting context size leads to more efficient and accurate question answering, demonstrating that adaptive retrieval strategies can significantly improve performance without complex modifications to existing pipelines.

Abstract: Retrieval-augmented generation (RAG) and long-context language models (LCLMs) both address context limitations of LLMs in open-domain question answering (QA). However, optimal external context to retrieve remains an open problem: fixing the retrieval size risks either wasting tokens or omitting key evidence. Existing adaptive methods like Self-RAG and Self-Route rely on iterative LLM prompting and perform well on factoid QA, but struggle with aggregation QA, where the optimal context size is both unknown and variable. We present Adaptive-$k$ retrieval, a simple and effective single-pass method that adaptively selects the number of passages based on the distribution of the similarity scores between the query and the candidate passages. It does not require model fine-tuning, extra LLM inferences or changes to existing retriever-reader pipelines. On both factoid and aggregation QA benchmarks, Adaptive-$k$ matches or outperforms fixed-$k$ baselines while using up to 10x fewer tokens than full-context input, yet still retrieves 70% of relevant passages. It improves accuracy across five LCLMs and two embedding models, highlighting that dynamically adjusting context size leads to more efficient and accurate QA.

Method: TableRAG uses an SQL-based framework with four iterative steps: context-sensitive query decomposition, text retrieval, SQL programming and execution, and compositional intermediate answer generation.

Result: TableRAG consistently outperforms existing baselines on both public datasets and the new HeteQA benchmark, establishing state-of-the-art performance for heterogeneous document question answering.

Conclusion: The SQL-based TableRAG framework effectively handles heterogeneous documents by preserving tabular structure and enabling complex reasoning, demonstrating superior performance over existing RAG approaches.

Abstract: Retrieval-Augmented Generation (RAG) has demonstrated considerable effectiveness in open-domain question answering. However, when applied to heterogeneous documents, comprising both textual and tabular components, existing RAG approaches exhibit critical limitations. The prevailing practice of flattening tables and chunking strategies disrupts the intrinsic tabular structure, leads to information loss, and undermines the reasoning capabilities of LLMs in multi-hop, global queries. To address these challenges, we propose TableRAG, an SQL-based framework that unifies textual understanding and complex manipulations over tabular data. TableRAG iteratively operates in four steps: context-sensitive query decomposition, text retrieval, SQL programming and execution, and compositional intermediate answer generation. We also develop HeteQA, a novel benchmark designed to evaluate the multi-hop heterogeneous reasoning capabilities. Experimental results demonstrate that TableRAG consistently outperforms existing baselines on both public datasets and our HeteQA, establishing a new state-of-the-art for heterogeneous document question answering. We release TableRAG at https://github.com/yxh-y/TableRAG/tree/main.

Method: Evaluated 16 forecasting tasks across 7 domains using two multimodal paradigms: aligning-based methods (aligning time series and text representations) and prompting-based methods (directly prompting LLMs for forecasting).

Result: Benefits of multimodality are highly condition-dependent. Gains are not universal across datasets or models. Key conditions include: high-capacity text models, weaker time series models, appropriate aligning strategies, sufficient training data, and text offering complementary predictive signal.

Conclusion: Multimodal integration for time series forecasting provides benefits only under specific conditions, offering a rigorous foundation for understanding when multimodality aids forecasting tasks, revealing benefits are neither universal nor always intuitive.

Abstract: Recently, there has been growing interest in incorporating textual information into foundation models for time series forecasting. However, it remains unclear whether and under what conditions such multimodal integration consistently yields gains. We systematically investigate these questions across a diverse benchmark of 16 forecasting tasks spanning 7 domains, including health, environment, and economics. We evaluate two popular multimodal forecasting paradigms: aligning-based methods, which align time series and text representations; and prompting-based methods, which directly prompt large language models for forecasting. Our findings reveal that the benefits of multimodality are highly condition-dependent. While we confirm reported gains in some settings, these improvements are not universal across datasets or models. To move beyond empirical observations, we disentangle the effects of model architectural properties and data characteristics, drawing data-agnostic insights that generalize across domains. Our findings highlight that on the modeling side, incorporating text information is most helpful given (1) high-capacity text models, (2) comparatively weaker time series models, and (3) appropriate aligning strategies. On the data side, performance gains are more likely when (4) sufficient training data is available and (5) the text offers complementary predictive signal beyond what is already captured from the time series alone. Our study offers a rigorous, quantitative foundation for understanding when multimodality can be expected to aid forecasting tasks, and reveals that its benefits are neither universal nor always aligned with intuition.

Method: Created SoMi-ToM benchmark with multi-level evaluation: first-person perspective (real-time multimodal input) and third-person perspective (complete video/text records). Dataset includes 35 third-person videos, 363 first-person images, and 1225 expert-annotated multiple-choice questions.

Result: LVLMs perform significantly worse than humans: 40.1% accuracy gap in first-person evaluation and 26.4% gap in third-person evaluation.

Conclusion: Current LVLMs need substantial improvement in ToM capabilities for embodied, complex social interactions, as they lag far behind human performance.

Abstract: Humans continuously infer the states, goals, and behaviors of others by perceiving their surroundings in dynamic, real-world social interactions. However, most Theory of Mind (ToM) benchmarks only evaluate static, text-based scenarios, which have a significant gap compared to real interactions. We propose the SoMi-ToM benchmark, designed to evaluate multi-perspective ToM in embodied multi-agent complex social interactions. This benchmark is based on rich multimodal interaction data generated by the interaction environment SoMi, covering diverse crafting goals and social relationships. Our framework supports multi-level evaluation: (1) first-person evaluation provides multimodal (visual, dialogue, action, etc.) input from a first-person perspective during a task for real-time state inference, (2) third-person evaluation provides complete third-person perspective video and text records after a task for goal and behavior inference. This evaluation method allows for a more comprehensive examination of a model’s ToM capabilities from both the subjective immediate experience and the objective global observation. We constructed a challenging dataset containing 35 third-person perspective videos, 363 first-person perspective images, and 1225 expert-annotated multiple-choice questions (three options). On this dataset, we systematically evaluated the performance of human subjects and several state-of-the-art large vision-language models (LVLMs). The results show that LVLMs perform significantly worse than humans on SoMi-ToM: the average accuracy gap between humans and models is 40.1% in first-person evaluation and 26.4% in third-person evaluation. This indicates that future LVLMs need to further improve their ToM capabilities in embodied, complex social interactions.

Method: Developed a safety evaluation protocol tailored to medical domain with patient and clinician perspectives, built PatientSafetyBench containing 466 samples over 5 critical categories, and applied red-teaming protocols on MediPhi model collection as case study.

Result: Created the first comprehensive safety evaluation framework for medical LLMs that considers three perspectives - patient, clinician, and general user - establishing foundational criteria for safer deployment.

Conclusion: This work bridges a critical gap in medical LLM safety evaluation by providing targeted red-teaming protocols and evaluation criteria from multiple user perspectives, enabling safer deployment of LLMs in medical domains.

Abstract: As the performance of large language models (LLMs) continues to advance, their adoption is expanding across a wide range of domains, including the medical field. The integration of LLMs into medical applications raises critical safety concerns, particularly due to their use by users with diverse roles, e.g. patients and clinicians, and the potential for model’s outputs to directly affect human health. Despite the domain-specific capabilities of medical LLMs, prior safety evaluations have largely focused only on general safety benchmarks. In this paper, we introduce a safety evaluation protocol tailored to the medical domain in both patient user and clinician user perspectives, alongside general safety assessments and quantitatively analyze the safety of medical LLMs. We bridge a gap in the literature by building the PatientSafetyBench containing 466 samples over 5 critical categories to measure safety from the perspective of the patient. We apply our red-teaming protocols on the MediPhi model collection as a case study. To our knowledge, this is the first work to define safety evaluation criteria for medical LLMs through targeted red-teaming taking three different points of view - patient, clinician, and general user - establishing a foundation for safer deployment in medical domains.

Method: Question Augmentation strategy that introduces partial solutions during RL training to reduce problem difficulty and provide more informative learning signals.

Result: Achieved new SOTA results on math benchmarks: 72.50% (+10.73%) on AIME24, 62.29% (+12.79%) on AIME25, and 41.67% (+10.11%) on HMMT25 using 1.5B-parameter models.

Conclusion: QuestA enables continual improvement over strong open-source models and enhances reasoning capabilities, particularly on problems where standard RL struggles.

Abstract: Reinforcement learning (RL) has emerged as a central paradigm for training large language models (LLMs) in reasoning tasks. Yet recent studies question RL’s ability to incentivize reasoning capacity beyond the base model. This raises a key challenge: how can RL be adapted to solve harder reasoning problems more effectively? To address this challenge, we propose a simple yet effective strategy via Question Augmentation: introduce partial solutions during training to reduce problem difficulty and provide more informative learning signals. Our method, QuestA, when applied during RL training on math reasoning tasks, not only improves pass@1 but also pass@k-particularly on problems where standard RL struggles to make progress. This enables continual improvement over strong open-source models such as DeepScaleR and OpenMath Nemotron, further enhancing their reasoning capabilities. We achieve new state-of-the-art results on math benchmarks using 1.5B-parameter models: 72.50% (+10.73%) on AIME24, 62.29% (+12.79%) on AIME25, and 41.67% (+10.11%) on HMMT25. Code, data and model are available at https://github.com/foreverlasting1202/QuestA.

Method: Used dataset of human-written and machine-generated texts from 8 domains and 11 LLMs, calculated linguistic features like dependency length and emotionality, analyzed with statistical methods and style embeddings, considering sampling strategies, repetition controls, and model release dates.

Result: Human-written texts show simpler syntactic structures and more diverse semantic content. Both human and machine texts show stylistic diversity across domains, but human texts display greater feature variation. Newer models produce text with similar variability, indicating homogenization of machine-generated content.

Conclusion: Machine-generated texts are becoming more homogenized across newer models, while human-written texts maintain greater linguistic diversity, particularly in semantic content and feature variability across domains.

Abstract: The rapid advancements in large language models (LLMs) have significantly improved their ability to generate natural language, making texts generated by LLMs increasingly indistinguishable from human-written texts. While recent research has primarily focused on using LLMs to classify text as either human-written or machine-generated texts, our study focuses on characterizing these texts using a set of linguistic features across different linguistic levels such as morphology, syntax, and semantics. We select a dataset of human-written and machine-generated texts spanning 8 domains and produced by 11 different LLMs. We calculate different linguistic features such as dependency length and emotionality, and we use them for characterizing human-written and machine-generated texts along with different sampling strategies, repetition controls, and model release dates. Our statistical analysis reveals that human-written texts tend to exhibit simpler syntactic structures and more diverse semantic content. Furthermore, we calculate the variability of our set of features across models and domains. Both human- and machine-generated texts show stylistic diversity across domains, with human-written texts displaying greater variation in our features. Finally, we apply style embeddings to further test variability among human-written and machine-generated texts. Notably, newer models output text that is similarly variable, pointing to a homogenization of machine-generated texts.

Method: Organized datasets along four key dimensions (LLM capabilities, steerability, alignment, robustness) and evaluated them based on popularity, practical adoption, recency, documentation quality, licensing transparency, and scientific contribution.

Result: Revealed critical gaps including limited task diversity, inconsistent documentation and annotation, and low community adoption of Arabic post-training datasets.

Conclusion: Identified implications of these gaps on Arabic-centric LLM progress and provided concrete recommendations for future Arabic post-training dataset development.

Abstract: Post-training has emerged as a crucial technique for aligning pre-trained Large Language Models (LLMs) with human instructions, significantly enhancing their performance across a wide range of tasks. Central to this process is the quality and diversity of post-training datasets. This paper presents a review of publicly available Arabic post-training datasets on the Hugging Face Hub, organized along four key dimensions: (1) LLM Capabilities (e.g., Question Answering, Translation, Reasoning, Summarization, Dialogue, Code Generation, and Function Calling); (2) Steerability (e.g., Persona and System Prompts); (3) Alignment (e.g., Cultural, Safety, Ethics, and Fairness); and (4) Robustness. Each dataset is rigorously evaluated based on popularity, practical adoption, recency and maintenance, documentation and annotation quality, licensing transparency, and scientific contribution. Our review revealed critical gaps in the development of Arabic post-training datasets, including limited task diversity, inconsistent or missing documentation and annotation, and low adoption across the community. Finally, the paper discusses the implications of these gaps on the progress of Arabic-centric LLMs and applications while providing concrete recommendations for future efforts in Arabic post-training dataset development.

Method: Analyzed Chinese-English bilingual reasoning models, identified RLVR training as key factor, tested monolingual decoding vs language mixing, and developed lightweight probe to predict beneficial language switches.

Result: Enforcing monolingual decoding reduced accuracy by 5.6 percentage points on MATH500; using probe to guide language switching increased accuracy by 2.92 percentage points.

Conclusion: Language mixing is a strategic reasoning behavior in bilingual LLMs, not just a training artifact, and can be leveraged to improve reasoning performance.

Abstract: Proficient multilingual speakers often intentionally switch languages in the middle of a conversation. Similarly, recent reasoning-focused bilingual large language models (LLMs) with strong capabilities in both languages exhibit language mixing-alternating languages within their chain of thought. Discouraging this behavior in DeepSeek-R1 was found to degrade accuracy, suggesting that language mixing may benefit reasoning. In this work, we study language switching in Chinese-English bilingual reasoning models. We identify reinforcement learning with verifiable rewards (RLVR) as the critical training stage that leads to language mixing. We show that language mixing can enhance reasoning: enforcing monolingual decoding reduces accuracy by 5.6 percentage points on MATH500. Additionally, a lightweight probe can be trained to predict whether a potential language switch would benefit or harm reasoning, and when used to guide decoding, increases accuracy by 2.92 percentage points. Our findings suggest that language mixing is not merely a byproduct of multilingual training, but is a strategic reasoning behavior.

Method: Two-component approach: 1) Non-public EESE-Pool with 100K+ expert-constructed science instances across 5 disciplines and 500+ subfields, 2) Periodically updated 500-instance EESE subset for leakage-resilient, low-overhead evaluations.

Result: Experiments on 32 open- and closed-source models show EESE effectively differentiates models’ strengths and weaknesses in scientific fields and cognitive dimensions.

Conclusion: EESE provides a robust, scalable, and forward-compatible solution for science benchmark design, offering realistic measurement of foundation models’ science question handling capabilities.

Abstract: As foundation models grow rapidly in capability and deployment, evaluating their scientific understanding becomes increasingly critical. Existing science benchmarks have made progress towards broad Range, wide Reach, and high Rigor, yet they often face two major challenges: data leakage risks that compromise benchmarking validity, and evaluation inefficiency due to large-scale testing. To address these issues, we introduce the Ever-Evolving Science Exam (EESE), a dynamic benchmark designed to reliably assess scientific capabilities in foundation models. Our approach consists of two components: 1) a non-public EESE-Pool with over 100K expertly constructed science instances (question-answer pairs) across 5 disciplines and 500+ subfields, built through a multi-stage pipeline ensuring Range, Reach, and Rigor, 2) a periodically updated 500-instance subset EESE, sampled and validated to enable leakage-resilient, low-overhead evaluations. Experiments on 32 open- and closed-source models demonstrate that EESE effectively differentiates the strengths and weaknesses of models in scientific fields and cognitive dimensions. Overall, EESE provides a robust, scalable, and forward-compatible solution for science benchmark design, offering a realistic measure of how well foundation models handle science questions. The project page is at: https://github.com/aiben-ch/EESE.

Method: Modified existing web extraction pipeline to preserve complete document integrity from source, including unaligned portions, and added pivoted alignments for additional language pairs beyond English.

Result: Fine-tuned LLMs on DocHPLT substantially outperform off-the-shelf instruction-tuned baselines, with particularly dramatic improvements for under-resourced languages.

Conclusion: DocHPLT provides essential infrastructure for advancing multilingual document-level translation and is released under a permissive license to benefit the research community.

Abstract: Existing document-level machine translation resources are only available for a handful of languages, mostly high-resourced ones. To facilitate the training and evaluation of document-level translation and, more broadly, long-context modeling for global communities, we create DocHPLT, the largest publicly available document-level translation dataset to date. It contains 124 million aligned document pairs across 50 languages paired with English, comprising 4.26 billion sentences. By adding pivoted alignments, practitioners can obtain 2500 additional pairs not involving English. Unlike previous reconstruction-based approaches that piece together documents from sentence-level data, we modify an existing web extraction pipeline to preserve complete document integrity from the source, retaining all content, including unaligned portions. After our preliminary experiments identify the optimal training context strategy for document-level translation, we demonstrate that LLMs fine-tuned on DocHPLT substantially outperform off-the-shelf instruction-tuned baselines, with particularly dramatic improvements for under-resourced languages. We open-source the dataset under a permissive license, providing essential infrastructure for advancing multilingual document-level translation.

Method: 1) Define and classify PoC tokens based on GPT vocabulary; 2) Build a PoC token detector using fine-tuned LLM considering semantics and search engine content; 3) Study training data pollution through PoC token appearances.

Result: Experiments show PoC tokens widely exist across 23 LLMs, with GPT having worst performance: over 23% of long Chinese tokens are porn or gambling related. Validation on C4 and Pile datasets confirms speculation accuracy.

Conclusion: Training data contains significant pollution, with GPT-4o estimated to have around 0.5% of training data related to “Yui Hatano” content, highlighting serious data quality issues in LLM training.

Abstract: Tokens are basic elements in the datasets for LLM training. It is well-known that many tokens representing Chinese phrases in the vocabulary of GPT (4o/4o-mini/o1/o3/4.5/4.1/o4-mini) are indicating contents like pornography or online gambling. Based on this observation, our goal is to locate Polluted Chinese (PoC) tokens in LLMs and study the relationship between PoC tokens’ existence and training data. (1) We give a formal definition and taxonomy of PoC tokens based on the GPT’s vocabulary. (2) We build a PoC token detector via fine-tuning an LLM to label PoC tokens in vocabularies by considering each token’s both semantics and related contents from the search engines. (3) We study the speculation on the training data pollution via PoC tokens’ appearances (token ID). Experiments on GPT and other 23 LLMs indicate that tokens widely exist while GPT’s vocabulary behaves the worst: more than 23% long Chinese tokens (i.e., a token with more than two Chinese characters) are either porn or online gambling. We validate the accuracy of our speculation method on famous pre-training datasets like C4 and Pile. Then, considering GPT-4o, we speculate that the ratio of “Yui Hatano” related webpages in GPT-4o’s training data is around 0.5%.

Method: Prophet dynamically decides whether to continue refinement or decode all remaining tokens in one step using the confidence gap between top-2 prediction candidates as criterion.

Result: Prophet reduces decoding steps by up to 3.4x while preserving generation quality on LLaDA-8B and Dream-7B across multiple tasks.

Conclusion: Early decode convergence provides a simple yet powerful mechanism for accelerating DLM inference, recasting DLM decoding as a problem of when to stop sampling.

Abstract: Diffusion language models (DLMs) have recently emerged as an alternative to autoregressive approaches, offering parallel sequence generation and flexible token orders. However, their inference remains slower than that of autoregressive models, primarily due to the cost of bidirectional attention and the large number of refinement steps required for high quality outputs. In this work, we highlight and leverage an overlooked property of DLMs early answer convergence: in many cases, the correct answer can be internally identified by half steps before the final decoding step, both under semi-autoregressive and random remasking schedules. For example, on GSM8K and MMLU, up to 97% and 99% of instances, respectively, can be decoded correctly using only half of the refinement steps. Building on this observation, we introduce Prophet, a training-free fast decoding paradigm that enables early commit decoding. Specifically, Prophet dynamically decides whether to continue refinement or to go “all-in” (i.e., decode all remaining tokens in one step), using the confidence gap between the top-2 prediction candidates as the criterion. It integrates seamlessly into existing DLM implementations, incurs negligible overhead, and requires no additional training. Empirical evaluations of LLaDA-8B and Dream-7B across multiple tasks show that Prophet reduces the number of decoding steps by up to 3.4x while preserving high generation quality. These results recast DLM decoding as a problem of when to stop sampling, and demonstrate that early decode convergence provides a simple yet powerful mechanism for accelerating DLM inference, complementary to existing speedup techniques. Our code is publicly available at https://github.com/pixeli99/Prophet.

Method: Proposed three new tasks for time series description and created BEDTime benchmark with four reformatted datasets across multiple modalities. Evaluated 13 state-of-the-art models on these tasks.

Result: Found that dedicated time series foundation models severely underperform, vision-language models perform well, language-only methods perform worst, and all approaches are fragile to robustness tests.

Conclusion: Current multi-modal models have significant limitations in basic time series understanding, indicating need for future work to improve robustness and foundational capabilities.

Abstract: Recent works propose complex multi-modal models that handle both time series and language, ultimately claiming high performance on complex tasks like time series reasoning and cross-modal question-answering. However, they skip evaluations of simple and important foundational tasks, which complex models should reliably master. They also lack direct, head-to-head comparisons with other popular approaches. So we ask a simple question: Can recent models even produce generic visual descriptions of time series data? In response, we propose three new tasks, posing that successful multi-modal models should be able to recognize, differentiate, and generate language descriptions of time series. We then create BEDTime, the first benchmark dataset to assess models on each task, comprising four datasets reformatted for these tasks across multiple modalities. Using BEDTime, we evaluate 13 state-of-the-art models, and find that (1) surprisingly, dedicated time series foundation models severely underperform, despite being designed for similar tasks, (2) vision-language models are quite capable, (3) language-only methods perform worst, despite many lauding their potential, and (4) all approaches are clearly fragile to a range of realistic robustness tests, indicating avenues for future work.

Method: Two complementary modules: Multi-Turn Text Generation (MTG) uses MLLMs to generate pseudo-labels through simulated dialogues, and Multi-Turn Text Interaction (MTI) refines user queries at inference via dialogue-based reasoning.

Result: The method achieves competitive or superior retrieval accuracy while eliminating manual caption requirements, demonstrating improved robustness and usability.

Conclusion: The proposed annotation-free framework enables scalable and practical deployment of text-based person search systems without manual supervision.

Abstract: Text-based person search (TBPS) enables the retrieval of person images from large-scale databases using natural language descriptions, offering critical value in surveillance applications. However, a major challenge lies in the labor-intensive process of obtaining high-quality textual annotations, which limits scalability and practical deployment. To address this, we introduce two complementary modules: Multi-Turn Text Generation (MTG) and Multi-Turn Text Interaction (MTI). MTG generates rich pseudo-labels through simulated dialogues with MLLMs, producing fine-grained and diverse visual descriptions without manual supervision. MTI refines user queries at inference time through dynamic, dialogue-based reasoning, enabling the system to interpret and resolve vague, incomplete, or ambiguous descriptions - characteristics often seen in real-world search scenarios. Together, MTG and MTI form a unified and annotation-free framework that significantly improves retrieval accuracy, robustness, and usability. Extensive evaluations demonstrate that our method achieves competitive or superior results while eliminating the need for manual captions, paving the way for scalable and practical deployment of TBPS systems.

Method: Proposes K-DeCore with knowledge decoupling mechanism that disentangles reasoning into task-specific and task-agnostic stages, dual-perspective memory consolidation, and structure-guided pseudo-data synthesis.

Result: Extensive experiments on four benchmark datasets demonstrate superiority over existing continual learning methods across multiple metrics with various backbone LLMs.

Conclusion: K-DeCore effectively addresses limitations of existing approaches by operating with fixed parameters while maintaining strong performance across diverse structured knowledge reasoning tasks.

Abstract: Continual Structured Knowledge Reasoning (CSKR) focuses on training models to handle sequential tasks, where each task involves translating natural language questions into structured queries grounded in structured knowledge. Existing general continual learning approaches face significant challenges when applied to this task, including poor generalization to heterogeneous structured knowledge and inefficient reasoning due to parameter growth as tasks increase. To address these limitations, we propose a novel CSKR framework, \textsc{K-DeCore}, which operates with a fixed number of tunable parameters. Unlike prior methods, \textsc{K-DeCore} introduces a knowledge decoupling mechanism that disentangles the reasoning process into task-specific and task-agnostic stages, effectively bridging the gaps across diverse tasks. Building on this foundation, \textsc{K-DeCore} integrates a dual-perspective memory consolidation mechanism for distinct stages and introduces a structure-guided pseudo-data synthesis strategy to further enhance the model’s generalization capabilities. Extensive experiments on four benchmark datasets demonstrate the superiority of \textsc{K-DeCore} over existing continual learning methods across multiple metrics, leveraging various backbone large language models.

Method: GraphSearch organizes retrieval into a modular framework with six modules for multi-turn interactions and iterative reasoning, plus a dual-channel retrieval strategy that issues semantic queries over chunk-based text data and relational queries over structural graph data.

Result: Experimental results across six multi-hop RAG benchmarks demonstrate that GraphSearch consistently improves answer accuracy and generation quality over traditional strategies.

Conclusion: GraphSearch is a promising direction for advancing graph retrieval-augmented generation, confirming its effectiveness through comprehensive evaluation.

Abstract: Graph Retrieval-Augmented Generation (GraphRAG) enhances factual reasoning in LLMs by structurally modeling knowledge through graph-based representations. However, existing GraphRAG approaches face two core limitations: shallow retrieval that fails to surface all critical evidence, and inefficient utilization of pre-constructed structural graph data, which hinders effective reasoning from complex queries. To address these challenges, we propose \textsc{GraphSearch}, a novel agentic deep searching workflow with dual-channel retrieval for GraphRAG. \textsc{GraphSearch} organizes the retrieval process into a modular framework comprising six modules, enabling multi-turn interactions and iterative reasoning. Furthermore, \textsc{GraphSearch} adopts a dual-channel retrieval strategy that issues semantic queries over chunk-based text data and relational queries over structural graph data, enabling comprehensive utilization of both modalities and their complementary strengths. Experimental results across six multi-hop RAG benchmarks demonstrate that \textsc{GraphSearch} consistently improves answer accuracy and generation quality over the traditional strategy, confirming \textsc{GraphSearch} as a promising direction for advancing graph retrieval-augmented generation.

Method: PAS is a family of automated activation steering methods that constructs fast, lightweight activation vectors from labeled datasets without manual intervention. It includes an introspective variant (iPAS) that delivers the strongest effects.

Result: PAS reliably improves performance for behavior tasks (10.1% on Bias, 5.2% on Morality, 34.8% on Alignment) but not for intelligence-oriented tasks. It also delivers additional gains on top of ICL and SFT.

Conclusion: PAS provides a practical, automated LM post-training option that can be cheaply trained, easily stored, and activated at will, characterizing where activation steering helps and where it fails.

Abstract: Language models (LMs) are typically post-trained for desired capabilities and behaviors via weight-based or prompt-based steering, but the former is time-consuming and expensive, and the latter is not precisely controllable and often requires manual trial-and-error. While activation steering (AS) promises a cheap, fast, and controllable alternative to the two existing post-training methods, current AS techniques require hand-crafted prompt pairs or labor-intensive feature annotation, making them more inconvenient than the plug-and-play methods such as Reinforcement Learning (RL) and Supervised Fine-Tuning (SFT). We introduce Painless Activation Steering (PAS), a family of fully automated methods that make AS readily usable with any given labeled dataset, with no need for prompt construction, feature labeling, or human intervention. We evaluate PAS on three open-weight models (Llama3.1-8B-Instruct, DeepSeek-R1-Distill-8B, and Nous-Hermes-2) and 18 tasks; we find that PAS reliably improves performance for behavior tasks, but not for intelligence-oriented tasks. The introspective variant (iPAS) delivers the strongest causal steering effects (10.1% on Bias, 5.2% on Morality, and 34.8% on Alignment). We also show PAS delivers additional gains on top of In-Context Learning (ICL) and SFT. PAS constructs a fast, lightweight activation vector that can be cheaply trained, easily stored, and activated at will. Our results provide a characterization of where AS helps, where it fails, and how to deploy it as a practical, automated LM post-training option.

Method: Uses dual-scale world models with a trajectory frontier at global scale and local trial-and-error exploration through Multi-path Advantage Reflection, which infers advantage-based progress signals to guide exploration.

Result: Achieves new state-of-the-art performance for LLM-based approaches on Jericho benchmark suite, with comparable performance to RL-based methods while requiring 100-800x fewer environment interactions.

Conclusion: GLoW demonstrates that LLM-based agents can effectively tackle hard-exploration tasks through structured world modeling and advantage-based exploration guidance, significantly reducing the sample complexity compared to traditional RL methods.

Abstract: LLM-based agents have seen promising advances, yet they are still limited in “hard-exploration” tasks requiring learning new knowledge through exploration. We present GLoW, a novel approach leveraging dual-scale world models, maintaining a trajectory frontier of high-value discoveries at the global scale, while learning from local trial-and-error in exploration through a Multi-path Advantage Reflection mechanism which infers advantage-based progress signals to guide exploration. To evaluate our framework for hard-exploration, we tackle the Jericho benchmark suite of text-based games, where GLoW achieves a new state-of-theart performance for LLM-based approaches. Compared to state-of-the-art RLbased methods, our approach achieves comparable performance while requiring 100-800x fewer environment interactions.

Method: Developed a TQA-to-MMQA framework with three components: 1) Task definition and evaluation rubric for MMQA quality, 2) Construction of benchmarks for evaluating generation and understanding models, and 3) Q-Mirror agentic system that integrates MMQA generation and evaluation into a closed loop for iterative refinement.

Result: Experiments show state-of-the-art models can generate MMQAs but leave substantial gaps, requiring reliable evaluation. Top-tier understanding models align closely with human judgment in MMQA quality assessment. The Q-Mirror agent raises average scores from 78.90 to 85.22 and pass rates from 72% to 95%.

Conclusion: The proposed framework and Q-Mirror system offer a practical path to creating large-scale scientific benchmarks by automating the transformation of text-only QA pairs into high-quality multi-modal QA pairs through iterative refinement.

Abstract: High-quality, multi-modal benchmarks are crucial for advancing scientific reasoning in large models yet their manual creation is costly and unscalable. To address this bottleneck, we explore the potential for transforming Text-Only QA Pairs (TQAs) into high-quality Multi-Modal QA Pairs (MMQAs), which include three parts: 1) Task Definition & Evaluation Rubric: We develop a TQA-to-MMQA framework and establish a comprehensive, multi-dimensional MMQA quality rubric that provides principles for the transformation. 2) Benchmark Construction: Then we construct two extensive benchmarks to rigorously evaluate state-of-the-art generation & understanding models on the distinct tasks of MMQA generation & MMQA quality evaluation. 3) Preliminary Solution: We develop an agentic system (Q-Mirror), which operationalizes our framework by integrating MMQA generation and evaluation into a closed loop for iterative refinement. Our experiments show that while state-of-the-art models can generate MMQAs, their outputs still leave substantial gaps, underscoring the need for reliable evaluation. We further demonstrate that top-tier understanding models align closely with human judgment in MMQA quality assessment. Leveraging both insights, the Q-Mirror agent raises average scores from 78.90 to 85.22 and pass rates from 72% to 95%, offering a practical path to large-scale scientific benchmarks.

Method: Orchestrated Test-Time Scaling strategy that combines: i) Internal Scaling via RL-enhanced Intrinsic Reasoning, ii) Sequential Scaling through Iterative Refinement, and iii) Parallel Scaling using Diverse Synthesis and Tournament Selection.

Result: Achieves SOTA performance on BIRD benchmark with 81.67% execution accuracy on test set, ranking first on official leaderboard.

Conclusion: Agentar-Scale-SQL demonstrates an effective path toward human-level performance in Text-to-SQL tasks and is designed as a general-purpose framework for easy adaptation to new databases and more powerful language models.

Abstract: State-of-the-art (SOTA) Text-to-SQL methods still lag significantly behind human experts on challenging benchmarks like BIRD. Current approaches that explore test-time scaling lack an orchestrated strategy and neglect the model’s internal reasoning process. To bridge this gap, we introduce Agentar-Scale-SQL, a novel framework leveraging scalable computation to improve performance. Agentar-Scale-SQL implements an Orchestrated Test-Time Scaling strategy that synergistically combines three distinct perspectives: i) Internal Scaling via RL-enhanced Intrinsic Reasoning, ii) Sequential Scaling through Iterative Refinement, and iii) Parallel Scaling using Diverse Synthesis and Tournament Selection. Agentar-Scale-SQL is a general-purpose framework designed for easy adaptation to new databases and more powerful language models. Extensive experiments show that Agentar-Scale-SQL achieves SOTA performance on the BIRD benchmark, reaching 81.67% execution accuracy on the test set and ranking first on the official leaderboard, demonstrating an effective path toward human-level performance.

Method: Identified visual-word-form-selective units in VLMs using cognitive neuroscience stimuli, then performed targeted ablation of these units to simulate reading impairments.

Result: Targeted ablation of word-form-selective units caused selective reading impairments matching dyslexic humans’ phonological deficits, while random unit ablation did not produce these effects. General visual and language comprehension remained intact.

Conclusion: The modeling approach successfully replicated key dyslexia characteristics and established a computational framework for investigating reading disorders through causal manipulation of artificial neural networks.

Abstract: Dyslexia, a neurodevelopmental disorder characterized by persistent reading difficulties, is often linked to reduced activity of the visual word form area in the ventral occipito-temporal cortex. Traditional approaches to studying dyslexia, such as behavioral and neuroimaging methods, have provided valuable insights but remain limited in their ability to test causal hypotheses about the underlying mechanisms of reading impairments. In this study, we use large-scale vision-language models (VLMs) to simulate dyslexia by functionally identifying and perturbing artificial analogues of word processing. Using stimuli from cognitive neuroscience, we identify visual-word-form-selective units within VLMs and demonstrate that targeted ablation of these units, unlike ablation of random units, leads to selective impairments in reading tasks while general visual and language comprehension abilities remain intact. In particular, the resulting model matches dyslexic humans’ phonological deficits without a significant change in orthographic processing. Taken together, our modeling results replicate key characteristics of dyslexia and establish a computational framework for investigating reading disorders.

Method: Uses a pretrained 3D Variational Autoencoder to encode videos, fuses with target HR prompts via trainable spatio-temporal layers with Adaptive Layer Normalizations, and applies Feature-wise Linear Modulation in decoder to avoid signal degradation.

Result: Achieves high visual quality (PSNR 38.96 dB, SSIM 0.98) while maintaining accurate HR modulation (10.00 bpm MAE, 10.09% MAPE) using state-of-the-art rPPG estimator.

Conclusion: The method enables controllable HR editing for applications like anonymizing biometric signals in real videos or synthesizing realistic videos with desired vital signs.

Abstract: Camera-based physiological signal estimation provides a non-contact and convenient means to monitor Heart Rate (HR). However, the presence of vital signals in facial videos raises significant privacy concerns, as they can reveal sensitive personal information related to the health and emotional states of an individual. To address this, we propose a learned framework that edits physiological signals in videos while preserving visual fidelity. First, we encode an input video into a latent space via a pretrained 3D Variational Autoencoder (3D VAE), while a target HR prompt is embedded through a frozen text encoder. We fuse them using a set of trainable spatio-temporal layers with Adaptive Layer Normalizations (AdaLN) to capture the strong temporal coherence of remote Photoplethysmography (rPPG) signals. We apply Feature-wise Linear Modulation (FiLM) in the decoder with a fine-tuned output layer to avoid the degradation of physiological signals during reconstruction, enabling accurate physiological modulation in the reconstructed video. Empirical results show that our method preserves visual quality with an average PSNR of 38.96 dB and SSIM of 0.98 on selected datasets, while achieving an average HR modulation error of 10.00 bpm MAE and 10.09% MAPE using a state-of-the-art rPPG estimator. Our design’s controllable HR editing is useful for applications such as anonymizing biometric signals in real videos or synthesizing realistic videos with desired vital signs.

Method: Proposes dual-path anchoring: (1) lightweight MoCLIP model trained via contrastive learning for temporal semantic supervision, and (2) frequency domain alignment using low-frequency DCT components. Uses adaptive fusion through temporal modulation to transition from coarse to fine-grained semantic refinement during denoising.

Result: Achieves state-of-the-art performance with FID scores of 0.035 on HumanML3D and 0.123 on KIT-ML. Accelerates convergence by 1.4x compared to baseline.

Conclusion: LUMA provides an efficient and scalable solution for high-fidelity text-to-motion generation by addressing gradient attenuation through dual-path semantic alignment in both temporal and frequency domains.

Abstract: While current diffusion-based models, typically built on U-Net architectures, have shown promising results on the text-to-motion generation task, they still suffer from semantic misalignment and kinematic artifacts. Through analysis, we identify severe gradient attenuation in the deep layers of the network as a key bottleneck, leading to insufficient learning of high-level features. To address this issue, we propose \textbf{LUMA} (\textit{\textbf{L}ow-dimension \textbf{U}nified \textbf{M}otion \textbf{A}lignment}), a text-to-motion diffusion model that incorporates dual-path anchoring to enhance semantic alignment. The first path incorporates a lightweight MoCLIP model trained via contrastive learning without relying on external data, offering semantic supervision in the temporal domain. The second path introduces complementary alignment signals in the frequency domain, extracted from low-frequency DCT components known for their rich semantic content. These two anchors are adaptively fused through a temporal modulation mechanism, allowing the model to progressively transition from coarse alignment to fine-grained semantic refinement throughout the denoising process. Experimental results on HumanML3D and KIT-ML demonstrate that LUMA achieves state-of-the-art performance, with FID scores of 0.035 and 0.123, respectively. Furthermore, LUMA accelerates convergence by 1.4$\times$ compared to the baseline, making it an efficient and scalable solution for high-fidelity text-to-motion generation.

Method: Created a benchmark with 2,720 QA pairs using high-resolution scans of public-domain paintings featuring multiple figures, actions, and detailed backdrops. Manually annotated questions across six task categories to probe thorough scene understanding.

Result: Even the best model (o3) among 37 tested achieved only 19.6% accuracy on the hardest test split and 69.5% overall, revealing multiple failure modes including poor counting skills, OCR failures, and logical inconsistencies.

Conclusion: VisualOverload exposes a critical gap in current vision models’ ability to handle densely populated scenes and provides a crucial resource for developing better models.

Abstract: Is basic visual understanding really solved in state-of-the-art VLMs? We present VisualOverload, a slightly different visual question answering (VQA) benchmark comprising 2,720 question-answer pairs, with privately held ground-truth responses. Unlike prior VQA datasets that typically focus on near global image understanding, VisualOverload challenges models to perform simple, knowledge-free vision tasks in densely populated (or, overloaded) scenes. Our dataset consists of high-resolution scans of public-domain paintings that are populated with multiple figures, actions, and unfolding subplots set against elaborately detailed backdrops. We manually annotated these images with questions across six task categories to probe for a thorough understanding of the scene. We hypothesize that current benchmarks overestimate the performance of VLMs, and encoding and reasoning over details is still a challenging task for them, especially if they are confronted with densely populated scenes. Indeed, we observe that even the best model (o3) out of 37 tested models only achieves 19.6% accuracy on our hardest test split and overall 69.5% accuracy on all questions. Beyond a thorough evaluation, we complement our benchmark with an error analysis that reveals multiple failure modes, including a lack of counting skills, failure in OCR, and striking logical inconsistencies under complex tasks. Altogether, VisualOverload exposes a critical gap in current vision models and offers a crucial resource for the community to develop better models. Benchmark: http://paulgavrikov.github.io/visualoverload

Method: Created a benchmark with fine-grained diagnostic categories targeting different spatial reasoning aspects, progressively structured from single-object to multi-object perspective-taking tasks, and evaluated 37 state-of-the-art VLMs.

Result: VLMs show systematic weaknesses including strong egocentric bias, poor rotational understanding, and inconsistencies under symmetrical/syntactic reformulations. Human subjects achieved 91.2% accuracy, and human response time strongly correlated with VLM accuracy.

Conclusion: SpinBench provides critical insights into spatial reasoning gaps in VLMs and captures spatial reasoning challenges shared across humans and VLMs, highlighting key deficiencies in their ability to reason about physical space.

Abstract: We present SpinBench, a cognitively grounded diagnostic benchmark for evaluating spatial reasoning in vision language models (VLMs). SpinBench is designed around the core challenge of spatial reasoning: perspective taking, the ability to reason about how scenes and object relations change under viewpoint transformation. Since perspective taking requires multiple cognitive capabilities, such as recognizing objects across views, relative positions grounding, and mentally simulating transformations, SpinBench introduces a set of fine-grained diagnostic categories. Our categories target translation, rotation, object relative pose, and viewpoint change, and are progressively structured so that single-object simpler tasks scaffold toward the most demanding multi-object perspective-taking setting. We evaluate 37 state-of-the-art VLMs, both proprietary and open source. Results reveal systematic weaknesses: strong egocentric bias, poor rotational understanding, and inconsistencies under symmetrical and syntactic reformulations. Scaling analysis shows both smooth improvements and emergent capabilities. While human subjects achieve high accuracy (91.2%), task difficulty as measured by human response time shows strong correlation with VLM accuracy, indicating that SpinBench captures spatial reasoning challenges shared across humans and VLMs. We believe SpinBench provides critical insights into spatial reasoning in VLMs and highlights key gaps in their ability to reason about physical space. Our website can be found at https://spinbench25.github.io/.

Method: Multi-Modal Spatio-Temporal Transformer (MM-STT) framework that fuses dynamic displacement data with static physical priors through a joint spatio-temporal attention mechanism that processes all multi-modal features in a unified manner.

Result: Establishes new state-of-the-art on EGMS dataset, reducing long-range forecast RMSE by an order of magnitude compared to all baselines including SOTA methods like STGCN and STAEformer.

Conclusion: For land subsidence forecasting problems, an architecture’s inherent capacity for deep multi-modal fusion is paramount for achieving transformative performance, demonstrating the superiority of multi-modal approaches over uni-modal paradigms.

Abstract: Forecasting high-resolution land subsidence is a critical yet challenging task due to its complex, non-linear dynamics. While standard architectures like ConvLSTM often fail to model long-range dependencies, we argue that a more fundamental limitation of prior work lies in the uni-modal data paradigm. To address this, we propose the Multi-Modal Spatio-Temporal Transformer (MM-STT), a novel framework that fuses dynamic displacement data with static physical priors. Its core innovation is a joint spatio-temporal attention mechanism that processes all multi-modal features in a unified manner. On the public EGMS dataset, MM-STT establishes a new state-of-the-art, reducing the long-range forecast RMSE by an order of magnitude compared to all baselines, including SOTA methods like STGCN and STAEformer. Our results demonstrate that for this class of problems, an architecture’s inherent capacity for deep multi-modal fusion is paramount for achieving transformative performance.

Method: Tested all seven modality combinations (T, A, V, TA, TV, AV, TAV) on 624 annotated YouTube short-form videos across five topics: main recommendation, sentiment analysis, video purpose, visual analysis, and financial entity recognition.

Result: Video alone performed strongly on four of five topics. Selective modality pairs (TV, AV) often surpassed the full TAV combination, suggesting that too many modalities may introduce noise rather than improve performance.

Conclusion: Established first baselines for financial short-form video captioning, showing video’s strong performance for capturing visual context and effective cues, while highlighting challenges of multimodal integration where more modalities don’t necessarily mean better performance.

Abstract: We evaluate multimodal large language models (MLLMs) for topic-aligned captioning in financial short-form videos (SVs) by testing joint reasoning over transcripts (T), audio (A), and video (V). Using 624 annotated YouTube SVs, we assess all seven modality combinations (T, A, V, TA, TV, AV, TAV) across five topics: main recommendation, sentiment analysis, video purpose, visual analysis, and financial entity recognition. Video alone performs strongly on four of five topics, underscoring its value for capturing visual context and effective cues such as emotions, gestures, and body language. Selective pairs such as TV or AV often surpass TAV, implying that too many modalities may introduce noise. These results establish the first baselines for financial short-form video captioning and illustrate the potential and challenges of grounding complex visual cues in this domain. All code and data can be found on our Github under the CC-BY-NC-SA 4.0 license.

Method: Uses text-based supervised fine-tuning with sparse labels, visual prompting, and intrinsic-conditioned augmentation to address pixel reference and cross-dataset camera ambiguity issues.

Result: DepthLM surpasses accuracy of most advanced VLMs by over 2x, making VLMs comparable with pure vision models for the first time, while naturally avoiding over-smoothing and having fewer flying points at boundaries.

Conclusion: The simplicity of DepthLM enables a single VLM to cover various 3D tasks beyond metric depth, demonstrating that VLMs can reach expert-level 3D understanding without complex architectural changes.

Abstract: Vision language models (VLMs) can flexibly address various vision tasks through text interactions. Although successful in semantic understanding, state-of-the-art VLMs including GPT-5 still struggle in understanding 3D from 2D inputs. On the other hand, expert pure vision models achieve super-human accuracy in metric depth estimation, a key 3D understanding task. However, they require task-specific architectures and losses. Such difference motivates us to ask: Can VLMs reach expert-level accuracy without architecture or loss change? We take per-pixel metric depth estimation as the representative task and show that the answer is yes! Surprisingly, comprehensive analysis shows that text-based supervised-finetuning with sparse labels is sufficient for VLMs to unlock strong 3D understanding, no dense prediction head or complex regression/regularization loss is needed. The bottleneck for VLMs lies actually in pixel reference and cross-dataset camera ambiguity, which we address through visual prompting and intrinsic-conditioned augmentation. With much smaller models, our method DepthLM surpasses the accuracy of most advanced VLMs by over 2x, making VLMs for the first time comparable with pure vision models. Interestingly, without explicit enforcement during training, VLMs trained with DepthLM naturally avoids over-smoothing, having much fewer flying points at boundary regions than pure vision models. The simplicity of DepthLM also enables a single VLM to cover various 3D tasks beyond metric depth. Our code and model will be released at the link below.

Method: 1) Novel high-resolution Transformer model with global and local modules for better feature extraction; 2) Bayesian extension treating parameters as random variables for uncertainty quantification; 3) Decision-level fusion of Sentinel-1, RCM, and AMSR2 data to address heterogeneity.

Result: Tested on Pan-Arctic datasets from September 2021, the model achieves both high-resolution SIC maps and robust uncertainty maps compared to other uncertainty quantification approaches.

Conclusion: The proposed Bayesian Transformer approach effectively addresses key challenges in sea ice concentration mapping and provides reliable uncertainty quantification through multi-sensor data fusion.

Abstract: Although high-resolution mapping of Pan-Arctic sea ice with reliable corresponding uncertainty is essential for operational sea ice concentration (SIC) charting, it is a difficult task due to some key challenges, e.g., the subtle nature of ice signature features, model uncertainty, and data heterogeneity. This letter presents a novel Bayesian Transformer approach for Pan-Arctic SIC mapping and uncertainty quantification using Sentinel-1, RADARSAT Constellation Mission (RCM), and Advanced Microwave Scanning Radiometer 2 (AMSR2) data. First, to improve feature extraction, we design a novel high-resolution Transformer model with both global and local modules that can better discern the subtle differences in sea ice patterns. Second, to improve uncertainty quantification, we design a Bayesian extension of the proposed Transformer model, treating its parameters as random variables to more effectively capture uncertainties. Third, to address data heterogeneity, we fuse three different data types (Sentinel-1, RCM, and AMSR2) at decision-level to improve both SIC mapping and uncertainty quantification. The proposed approach is tested on Pan-Arctic datasets from September 2021, and the results demonstrate that the proposed model can achieve both high-resolution SIC maps and robust uncertainty maps compared to other uncertainty quantification approaches.

Method: Developed a scalable vehicle detection and localization framework using roadside camera and LiDAR sensors in a cosimulation environment, employing Kalman Filter-based late fusion strategy for trajectory consistency.

Result: Simulation showed 70% reduction in longitudinal error while maintaining lateral accuracy within 1-3 meters. Field validation confirmed fused trajectories closely match real vehicle paths even with intermittent sensor data.

Conclusion: KF-based sensor fusion reliably compensates for individual sensor limitations, offering a practical solution for deploying infrastructure-enabled multi-sensor systems in complex traffic environments.

Abstract: Infrastructure-based sensing and real-time trajectory generation show promise for improving safety in high-risk roadway segments such as work zones, yet practical deployments are hindered by perspective distortion, complex geometry, occlusions, and costs. This study tackles these barriers by integrating roadside camera and LiDAR sensors into a cosimulation environment to develop a scalable, cost-effective vehicle detection and localization framework, and employing a Kalman Filter-based late fusion strategy to enhance trajectory consistency and accuracy. In simulation, the fusion algorithm reduced longitudinal error by up to 70 percent compared to individual sensors while preserving lateral accuracy within 1 to 3 meters. Field validation in an active work zone, using LiDAR, a radar-camera rig, and RTK-GPS as ground truth, demonstrated that the fused trajectories closely match real vehicle paths, even when single-sensor data are intermittent or degraded. These results confirm that KF based sensor fusion can reliably compensate for individual sensor limitations, providing precise and robust vehicle tracking capabilities. Our approach thus offers a practical pathway to deploy infrastructure-enabled multi-sensor systems for proactive safety measures in complex traffic environments.

Method: Created using an intent-guided simulation framework that models both desired influence and execution plans of news creators, grounded in trustworthy reference articles. The dataset spans manipulations across visual and textual modalities.

Result: Evaluation of 14 state-of-the-art vision-language models shows they struggle with intent reasoning, relying on shallow cues like surface-level alignment, stylistic polish, or heuristic authenticity signals rather than deeper intent analysis.

Conclusion: Current VLMs have significant limitations in intent reasoning, and DeceptionDecoded serves as a foundation for developing intent-aware models that can go beyond shallow cues in multimodal misinformation detection.

Abstract: The impact of misinformation arises not only from factual inaccuracies but also from the misleading narratives that creators deliberately embed. Interpreting such creator intent is therefore essential for multimodal misinformation detection (MMD) and effective information governance. To this end, we introduce DeceptionDecoded, a large-scale benchmark of 12,000 image-caption pairs grounded in trustworthy reference articles, created using an intent-guided simulation framework that models both the desired influence and the execution plan of news creators. The dataset captures both misleading and non-misleading cases, spanning manipulations across visual and textual modalities, and supports three intent-centric tasks: (1) misleading intent detection, (2) misleading source attribution, and (3) creator desire inference. We evaluate 14 state-of-the-art vision-language models (VLMs) and find that they struggle with intent reasoning, often relying on shallow cues such as surface-level alignment, stylistic polish, or heuristic authenticity signals. These results highlight the limitations of current VLMs and position DeceptionDecoded as a foundation for developing intent-aware models that go beyond shallow cues in MMD.

Method: Proposes Forensic-Chat framework with ‘seeing before reasoning’ paradigm that first trains MLLMs to perceive artifacts, strengthening artifact-aware visual perception. Also introduces ExplainFake-Bench benchmark for evaluating explainability in image forensics.

Result: Extensive experiments show Forensic-Chat achieves superior generalization performance and genuinely reliable explainability compared to existing approaches.

Conclusion: The ‘seeing before reasoning’ paradigm with Forensic-Chat effectively addresses the fundamental mismatch in MLLMs for fake image detection, providing a generalizable, explainable, and conversational solution that maintains dialogue capabilities while improving detection performance.

Abstract: Detecting AI-generated images with multimodal large language models (MLLMs) has gained increasing attention, due to their rich world knowledge, common-sense reasoning, and potential for explainability. However, naively applying those MLLMs for detection often leads to suboptimal performance. We argue that the root of this failure lies in a fundamental mismatch: MLLMs are asked to reason about fakes before they can truly see them. First, they do not really see: existing MLLMs’ vision encoders are primarily optimized for semantic-oriented recognition rather than the perception of low-level signals, leaving them insensitive to subtle forgery traces. Without access to reliable perceptual evidence, the model grounds its judgment on incomplete and limited visual observations. Second, existing finetuning data for detection typically uses narrow, instruction-style formats, which diverge sharply from the diverse, heterogeneous distributions seen in pretraining. In the absence of meaningful visual cues, the model therefore exploits these linguistic shortcuts, resulting in catastrophic forgetting of pretrained knowledge (even the basic dialogue capabilities). In response, we advocate for a new paradigm: seeing before reasoning. We propose that MLLMs should first be trained to perceive artifacts-strengthening their artifact-aware visual perception-so that subsequent reasoning is grounded in actual observations. We therefore propose Forensic-Chat, a generalizable, explainable, and still-conversational (for multi-round dialogue) assistant for fake image detection. We also propose ExplainFake-Bench, a benchmark tailored for the evaluation of the MLLM’s explainability for image forensics from five key aspects. Extensive experiments show its superiority of generalization and genuinely reliable explainability.

Method: Built a model based on explainable features from research on gaze patterns during dyadic conversations, utilizing point-of-gaze tracking to detect inconsistencies in nonverbal communication that deepfakes cannot mimic.

Result: Achieved 82% accuracy on a novel dataset created specifically for this research, demonstrating the effectiveness of gaze-based detection.

Conclusion: This is the first method to utilize point-of-gaze tracking for deepfake detection, providing a novel biometric approach to combat real-time deepfake phishing attacks in video communications.

Abstract: With recent advancements in deepfake technology, it is now possible to generate convincing deepfakes in real-time. Unfortunately, malicious actors have started to use this new technology to perform real-time phishing attacks during video meetings. The nature of a video call allows access to what the deepfake is ``seeing,’’ that is, the screen displayed to the malicious actor. Using this with the estimated gaze from the malicious actors streamed video enables us to estimate where the deepfake is looking on screen, the point of gaze. Because the point of gaze during conversations is not random and is instead used as a subtle nonverbal communicator, it can be used to detect deepfakes, which are not capable of mimicking this subtle nonverbal communication. This paper proposes a real-time deepfake detection method adapted to this genre of attack, utilizing previously unavailable biometric information. We built our model based on explainable features selected after careful review of research on gaze patterns during dyadic conversations. We then test our model on a novel dataset of our creation, achieving an accuracy of 82%. This is the first reported method to utilize point-of-gaze tracking for deepfake detection.

Method: Proposed cross-attention-based aggregation where query projections come from nodes and key projections from neighbors, and introduced AttentionViG architecture using this scheme for non-local message passing.

Result: Achieved SOTA performance on ImageNet-1K benchmark and demonstrated strong transferability to object detection, instance segmentation on MS COCO 2017, and semantic segmentation on ADE20K, while maintaining efficiency with comparable FLOPs.

Conclusion: The proposed cross-attention aggregation method and AttentionViG architecture deliver competitive accuracy with efficiency, establishing a versatile solution for vision graph neural networks.

Abstract: Vision Graph Neural Networks (ViGs) have demonstrated promising performance in image recognition tasks against Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). An essential part of the ViG framework is the node-neighbor feature aggregation method. Although various graph convolution methods, such as Max-Relative, EdgeConv, GIN, and GraphSAGE, have been explored, a versatile aggregation method that effectively captures complex node-neighbor relationships without requiring architecture-specific refinements is needed. To address this gap, we propose a cross-attention-based aggregation method in which the query projections come from the node, while the key projections come from its neighbors. Additionally, we introduce a novel architecture called AttentionViG that uses the proposed cross-attention aggregation scheme to conduct non-local message passing. We evaluated the image recognition performance of AttentionViG on the ImageNet-1K benchmark, where it achieved SOTA performance. Additionally, we assessed its transferability to downstream tasks, including object detection and instance segmentation on MS COCO 2017, as well as semantic segmentation on ADE20K. Our results demonstrate that the proposed method not only achieves strong performance, but also maintains efficiency, delivering competitive accuracy with comparable FLOPs to prior vision GNN architectures.

Method: Uses custom renderer with new template matching metric tailored to edge domain, leveraging low-fidelity textureless 3D models as inputs for pose estimation.

Result: Extensive evaluations show method consistently outperforms state-of-the-art in compute and memory-constrained localization, achieving better robustness and accuracy on synthetic datasets, physical testbeds, and Mars imagery.

Conclusion: Enables new possibilities for cheap and reliable localization on general-purpose hardware, making it suitable for space missions with severe hardware constraints.

Abstract: We consider the problem of vision-based 6-DoF object pose estimation in the context of the notional Mars Sample Return campaign, in which a robotic arm would need to localize multiple objects of interest for low-clearance pickup and insertion, under severely constrained hardware. We propose a novel localization algorithm leveraging a custom renderer together with a new template matching metric tailored to the edge domain to achieve robust pose estimation using only low-fidelity, textureless 3D models as inputs. Extensive evaluations on synthetic datasets as well as from physical testbeds on Earth and in situ Mars imagery shows that our method consistently beats the state of the art in compute and memory-constrained localization, both in terms of robustness and accuracy, in turn enabling new possibilities for cheap and reliable localization on general-purpose hardware.

Method: Uses four YOLO versions (v3, v4, v7, v9) combined with ConSinGAN for data augmentation, trained on metal sheet images to detect surface and hole defects.

Result: YOLOv9 with ConSinGAN achieved best performance with 91.3% accuracy and 146ms detection time, integrated into manufacturing hardware and SCADA system.

Conclusion: The proposed automated defect detection system is effective and can be easily applied to other industrial components.

Abstract: In this paper, we propose a YOLO-based deep learning (DL) model for automatic defect detection to solve the time-consuming and labor-intensive tasks in industrial manufacturing. In our experiments, the images of metal sheets are used as the dataset for training the YOLO model to detect the defects on the surfaces and in the holes of metal sheets. However, the lack of metal sheet images significantly degrades the performance of detection accuracy. To address this issue, the ConSinGAN is used to generate a considerable amount of data. Four versions of the YOLO model (i.e., YOLOv3, v4, v7, and v9) are combined with the ConSinGAN for data augmentation. The proposed YOLOv9 model with ConSinGAN outperforms the other YOLO models with an accuracy of 91.3%, and a detection time of 146 ms. The proposed YOLOv9 model is integrated into manufacturing hardware and a supervisory control and data acquisition (SCADA) system to establish a practical automated optical inspection (AOI) system. Additionally, the proposed automated defect detection is easily applied to other components in industrial manufacturing.

Method: A hybrid pipeline that combines off-the-shelf vision-language models for fine-grained attribute extraction with large language models for symbolic reasoning. It processes images and referring expressions by extracting object types/attributes, detecting candidate regions, generating visual descriptors with VLM, and using LLM for chain-of-thought reasoning with spatial metadata.

Result: LLM-RG yields substantial gains over both LLM and VLM-based baselines on the Talk2Car benchmark. Ablations show that adding 3D spatial cues further improves grounding performance.

Conclusion: The results demonstrate the complementary strengths of VLMs and LLMs, applied in a zero-shot manner, for robust outdoor referential grounding.

Abstract: Referential grounding in outdoor driving scenes is challenging due to large scene variability, many visually similar objects, and dynamic elements that complicate resolving natural-language references (e.g., “the black car on the right”). We propose LLM-RG, a hybrid pipeline that combines off-the-shelf vision-language models for fine-grained attribute extraction with large language models for symbolic reasoning. LLM-RG processes an image and a free-form referring expression by using an LLM to extract relevant object types and attributes, detecting candidate regions, generating rich visual descriptors with a VLM, and then combining these descriptors with spatial metadata into natural-language prompts that are input to an LLM for chain-of-thought reasoning to identify the referent’s bounding box. Evaluated on the Talk2Car benchmark, LLM-RG yields substantial gains over both LLM and VLM-based baselines. Additionally, our ablations show that adding 3D spatial cues further improves grounding. Our results demonstrate the complementary strengths of VLMs and LLMs, applied in a zero-shot manner, for robust outdoor referential grounding.

Method: Generate universal visual inputs (VISOR++ images) that induce target activation patterns across multiple VLMs. These images are optimized to emulate steering vectors and can be inserted as visual inputs to steer model behaviors without runtime interventions.

Result: VISOR++ images achieve performance parity with steering vectors for refusal, sycophancy, and survival instinct alignment directions. They work on both open-access (LLaVA-1.5-7B, IDEFICS2-8B) and unseen models (including closed-access ones), while preserving 99.9% performance on 14,000 MMLU evaluation tasks.

Conclusion: VISOR++ provides a deployment-agnostic approach for behavioral control in VLMs through universal visual inputs, overcoming limitations of existing methods and enabling steering across multiple models without runtime access.

Abstract: As Vision Language Models (VLMs) are deployed across safety-critical applications, understanding and controlling their behavioral patterns has become increasingly important. Existing behavioral control methods face significant limitations: system prompting approaches could easily be overridden by user instructions, while applying activation-based steering vectors requires invasive runtime access to model internals, precluding deployment with API-based services and closed-source models. Finding steering methods that transfer across multiple VLMs is still an open area of research. To this end, we introduce universal visual input based steering for output redirection (VISOR++), to achieve behavioral control through optimized visual inputs alone. We demonstrate that a single VISOR++ image can be generated for an ensemble of VLMs to emulate each of their steering vectors. By crafting universal visual inputs that induce target activation patterns, VISOR++ eliminates the need for runtime model access while remaining deployment-agnostic. This means that when an underlying model supports multimodal capability, model behaviors can be steered by inserting an image input replacing runtime steering vector based interventions. We first demonstrate the effectiveness of the VISOR++ images on open-access models such as LLaVA-1.5-7B and IDEFICS2-8B along three alignment directions: refusal, sycophancy and survival instinct. Both the model-specific steering images and the jointly optimized images achieve performance parity closely following that of steering vectors for both positive and negative steering tasks. We also show the promise of VISOR++ images in achieving directional behavioral shifts for unseen models including both open-access and closed-access ones. Furthermore, VISOR++ images are able to preserve 99.9% performance on 14,000 unrelated MMLU evaluation tasks.

Method: Vision-Zero uses a strategic self-play framework where VLMs engage in ‘Who Is the Spy’-style games across multiple roles, generating training data autonomously. It employs Iterative Self-Play Policy Optimization (Iterative-SPO) that alternates between Self-Play and reinforcement learning with verifiable rewards to achieve sustained improvements.

Result: Vision-Zero achieves state-of-the-art performance on reasoning, chart question answering, and vision-centric understanding tasks, surpassing other annotation-based methods despite using label-free data.

Conclusion: The framework demonstrates strong generalization across diverse domains (synthetic scenes, charts, real-world images) and enables sustainable performance gains without human annotation, making VLM deployment more practical.

Abstract: Although reinforcement learning (RL) can effectively enhance the reasoning capabilities of vision-language models (VLMs), current methods remain heavily dependent on labor-intensive datasets that require extensive manual construction and verification, leading to extremely high training costs and consequently constraining the practical deployment of VLMs. To address this challenge, we propose Vision-Zero, a domain-agnostic framework enabling VLM self-improvement through competitive visual games generated from arbitrary image pairs. Specifically, Vision-Zero encompasses three main attributes: (1) Strategic Self-Play Framework: Vision-Zero trains VLMs in “Who Is the Spy”-style games, where the models engage in strategic reasoning and actions across multiple roles. Through interactive gameplay, models autonomously generate their training data without human annotation. (2) Gameplay from Arbitrary Images: Unlike existing gamified frameworks, Vision-Zero can generate games from arbitrary images, thereby enhancing the model’s reasoning ability across diverse domains and showing strong generalization to different tasks. We demonstrate this versatility using three distinct types of image datasets: CLEVR-based synthetic scenes, charts, and real-world images. (3) Sustainable Performance Gain: We introduce Iterative Self-Play Policy Optimization (Iterative-SPO), a novel training algorithm that alternates between Self-Play and reinforcement learning with verifiable rewards (RLVR), mitigating the performance plateau often seen in self-play-only training and achieving sustained long-term improvements. Despite using label-free data, Vision-Zero achieves state-of-the-art performance on reasoning, chart question answering, and vision-centric understanding tasks, surpassing other annotation-based methods. Models and code has been released at https://github.com/wangqinsi1/Vision-Zero.

Method: Proposes a novel hybrid approach that combines mathematical/clustering segmentation models with insights from U-Net, leveraging strengths of both methods to reduce dependency on large-scale training data while improving accuracy.

Result: Achieved 89.7% Dice coefficient and 80.01% IoU on 1024*1024 fundus images, significantly outperforming Attention U-Net (51.3% Dice, 34.2% IoU), with better generalizability on external datasets.

Conclusion: The hybrid model enables precise choroidal nevus segmentation with reduced data requirements, forming a foundation for automated lesion annotation systems to enhance diagnostic speed and accuracy in clinical practice.

Abstract: Choroidal nevi are common benign pigmented lesions in the eye, with a small risk of transforming into melanoma. Early detection is critical to improving survival rates, but misdiagnosis or delayed diagnosis can lead to poor outcomes. Despite advancements in AI-based image analysis, diagnosing choroidal nevi in colour fundus images remains challenging, particularly for clinicians without specialized expertise. Existing datasets often suffer from low resolution and inconsistent labelling, limiting the effectiveness of segmentation models. This paper addresses the challenge of achieving precise segmentation of fundus lesions, a critical step toward developing robust diagnostic tools. While deep learning models like U-Net have demonstrated effectiveness, their accuracy heavily depends on the quality and quantity of annotated data. Previous mathematical/clustering segmentation methods, though accurate, required extensive human input, making them impractical for medical applications. This paper proposes a novel approach that combines mathematical/clustering segmentation models with insights from U-Net, leveraging the strengths of both methods. This hybrid model improves accuracy, reduces the need for large-scale training data, and achieves significant performance gains on high-resolution fundus images. The proposed model achieves a Dice coefficient of 89.7% and an IoU of 80.01% on 1024*1024 fundus images, outperforming the Attention U-Net model, which achieved 51.3% and 34.2%, respectively. It also demonstrated better generalizability on external datasets. This work forms a part of a broader effort to develop a decision support system for choroidal nevus diagnosis, with potential applications in automated lesion annotation to enhance the speed and accuracy of diagnosis and monitoring.

Method: Systematically evaluated state-of-the-art MLLMs and introduced FishNet++ - a large-scale multimodal benchmark with 35,133 textual descriptions, 706,426 key-point annotations, and 119,399 bounding boxes.

Result: Revealed significant limitations in MLLMs’ ability to perform fine-grained fish species recognition, with best open-source models achieving less than 10% accuracy.

Conclusion: FishNet++ facilitates development and evaluation of specialized vision-language models capable of advancing aquatic science by addressing the domain knowledge gap in marine biology.

Abstract: Multimodal large language models (MLLMs) have demonstrated impressive cross-domain capabilities, yet their proficiency in specialized scientific fields like marine biology remains underexplored. In this work, we systematically evaluate state-of-the-art MLLMs and reveal significant limitations in their ability to perform fine-grained recognition of fish species, with the best open-source models achieving less than 10% accuracy. This task is critical for monitoring marine ecosystems under anthropogenic pressure. To address this gap and investigate whether these failures stem from a lack of domain knowledge, we introduce FishNet++, a large-scale, multimodal benchmark. FishNet++ significantly extends existing resources with 35,133 textual descriptions for multimodal learning, 706,426 key-point annotations for morphological studies, and 119,399 bounding boxes for detection. By providing this comprehensive suite of annotations, our work facilitates the development and evaluation of specialized vision-language models capable of advancing aquatic science.

Method: Proposes a deep mixture of multimodal hierarchical variational auto-encoders (MMHVAE) that creates complex latent representations, encourages variational distributions to estimate missing information, learns multimodal fusion with missing data, and leverages dataset-level information for incomplete training data.

Result: Extensive experiments conducted on pre-operative brain multi-parametric MRI and intra-operative ultrasound imaging, demonstrating the method’s effectiveness in cross-modal image synthesis.

Conclusion: MMHVAE successfully addresses the four key challenges in multimodal image synthesis and shows promising results in medical imaging applications where complete multimodal data is not always available.

Abstract: We propose a deep mixture of multimodal hierarchical variational auto-encoders called MMHVAE that synthesizes missing images from observed images in different modalities. MMHVAE’s design focuses on tackling four challenges: (i) creating a complex latent representation of multimodal data to generate high-resolution images; (ii) encouraging the variational distributions to estimate the missing information needed for cross-modal image synthesis; (iii) learning to fuse multimodal information in the context of missing data; (iv) leveraging dataset-level information to handle incomplete data sets at training time. Extensive experiments are performed on the challenging problem of pre-operative brain multi-parametric magnetic resonance and intra-operative ultrasound imaging.

Method: Created MetaChest dataset with 479,215 chest X-rays from 4 databases, designed meta-set partition for few-shot classification, and evaluated transfer learning vs ProtoNet extension across multi-label classification tasks.

Result: Increasing classes per episode and training examples improves performance; transfer learning consistently beats ProtoNet; higher-resolution images boost accuracy but increase computation; efficient models match larger models’ performance with fewer resources.

Conclusion: Transfer learning is surprisingly effective for few-shot medical image classification, and practical considerations like computational efficiency and dataset design significantly impact performance in real-world medical applications.

Abstract: The limited availability of annotated data presents a major challenge for applying deep learning methods to medical image analysis. Few-shot learning methods aim to recognize new classes from only a small number of labeled examples. These methods are typically studied under the standard few-shot learning setting, where all classes in a task are new. However, medical applications such as pathology classification from chest X-rays often require learning new classes while simultaneously leveraging knowledge of previously known ones, a scenario more closely aligned with generalized few-shot classification. Despite its practical relevance, few-shot learning has been scarcely studied in this context. In this work, we present MetaChest, a large-scale dataset of 479,215 chest X-rays collected from four public databases. MetaChest includes a meta-set partition specifically designed for standard few-shot classification, as well as an algorithm for generating multi-label episodes. We conduct extensive experiments evaluating both a standard transfer learning approach and an extension of ProtoNet across a wide range of few-shot multi-label classification tasks. Our results demonstrate that increasing the number of classes per episode and the number of training examples per class improves classification performance. Notably, the transfer learning approach consistently outperforms the ProtoNet extension, despite not being tailored for few-shot learning. We also show that higher-resolution images improve accuracy at the cost of additional computation, while efficient model architectures achieve comparable performance to larger models with significantly reduced resource requirements.

Method: Compared Swin UNETR trained on multi-temporal data (1356 images with 32 sequential crack propagation images) with U-Net trained on mono-temporal data. Analyzed generalization ability, temporal consistency, and segmentation quality.

Result: Multi-temporal approach significantly outperformed mono-temporal model: IoU 82.72% vs 76.69%, F1-score 90.54% vs 86.18%, despite requiring only half the trainable parameters. Also showed more consistent segmentation with reduced noise and fewer errors.

Conclusion: Temporal information significantly enhances segmentation performance, offering improved crack detection and long-term monitoring of concrete structures even with limited sequential data.

Abstract: Cracks are among the earliest indicators of deterioration in concrete structures. Early automatic detection of these cracks can significantly extend the lifespan of critical infrastructures, such as bridges, buildings, and tunnels, while simultaneously reducing maintenance costs and facilitating efficient structural health monitoring. This study investigates whether leveraging multi-temporal data for crack segmentation can enhance segmentation quality. Therefore, we compare a Swin UNETR trained on multi-temporal data with a U-Net trained on mono-temporal data to assess the effect of temporal information compared with conventional single-epoch approaches. To this end, a multi-temporal dataset comprising 1356 images, each with 32 sequential crack propagation images, was created. After training the models, experiments were conducted to analyze their generalization ability, temporal consistency, and segmentation quality. The multi-temporal approach consistently outperformed its mono-temporal counterpart, achieving an IoU of $82.72%$ and a F1-score of $90.54%$, representing a significant improvement over the mono-temporal model’s IoU of $76.69%$ and F1-score of $86.18%$, despite requiring only half of the trainable parameters. The multi-temporal model also displayed a more consistent segmentation quality, with reduced noise and fewer errors. These results suggest that temporal information significantly enhances the performance of segmentation models, offering a promising solution for improved crack detection and the long-term monitoring of concrete structures, even with limited sequential data.

Method: The framework encodes heterogeneous knowledge sources into dual-prompt representation (1-D sparse prompts defining what to segment and 2-D dense prompts indicating where to attend), dynamically routed through a Mixture-of-Experts decoder.

Result: Comprehensive experiments on 18 public datasets spanning diverse modalities (CT, MRI, X-ray, pathology, ultrasound, etc.) demonstrate state-of-the-art performance across semantic, in-context, and interactive segmentation settings.

Conclusion: K-Prism successfully mirrors clinical flexibility by systematically integrating multiple knowledge paradigms, enabling flexible switching between paradigms and joint training across diverse tasks without architectural modifications.

Abstract: Medical image segmentation is fundamental to clinical decision-making, yet existing models remain fragmented. They are usually trained on single knowledge sources and specific to individual tasks, modalities, or organs. This fragmentation contrasts sharply with clinical practice, where experts seamlessly integrate diverse knowledge: anatomical priors from training, exemplar-based reasoning from reference cases, and iterative refinement through real-time interaction. We present $\textbf{K-Prism}$, a unified segmentation framework that mirrors this clinical flexibility by systematically integrating three knowledge paradigms: (i) $\textit{semantic priors}$ learned from annotated datasets, (ii) $\textit{in-context knowledge}$ from few-shot reference examples, and (iii) $\textit{interactive feedback}$ from user inputs like clicks or scribbles. Our key insight is that these heterogeneous knowledge sources can be encoded into a dual-prompt representation: 1-D sparse prompts defining $\textit{what}$ to segment and 2-D dense prompts indicating $\textit{where}$ to attend, which are then dynamically routed through a Mixture-of-Experts (MoE) decoder. This design enables flexible switching between paradigms and joint training across diverse tasks without architectural modifications. Comprehensive experiments on 18 public datasets spanning diverse modalities (CT, MRI, X-ray, pathology, ultrasound, etc.) demonstrate that K-Prism achieves state-of-the-art performance across semantic, in-context, and interactive segmentation settings. Code will be released upon publication.

Method: Three key optimizations: 1) Highly compressed VAE for dimensionality reduction, 2) KD-guided sensitivity-aware tri-level pruning for model size reduction, 3) Adversarial step distillation to reduce inference steps to four.

Result: Achieved approximately 15 FPS generation speed on iPhone 16 Pro Max, demonstrating practical deployment capability on mobile platforms.

Conclusion: The proposed optimizations make high-quality video generation feasible on mobile devices, enabling practical on-device deployment.

Abstract: Diffusion Transformers (DiT) have shown strong performance in video generation tasks, but their high computational cost makes them impractical for resource-constrained devices like smartphones, and practical on-device generation is even more challenging. In this work, we propose a series of novel optimizations to significantly accelerate video generation and enable practical deployment on mobile platforms. First, we employ a highly compressed variational autoencoder (VAE) to reduce the dimensionality of the input data without sacrificing visual quality. Second, we introduce a KD-guided, sensitivity-aware tri-level pruning strategy to shrink the model size to suit mobile platforms while preserving critical performance characteristics. Third, we develop an adversarial step distillation technique tailored for DiT, which allows us to reduce the number of inference steps to four. Combined, these optimizations enable our model to achieve approximately 15 frames per second (FPS) generation speed on an iPhone 16 Pro Max, demonstrating the feasibility of efficient, high-quality video generation on mobile devices.

Method: GaussianLens - a feed-forward densification framework that fuses multi-modal information from initial low-res 3DGS and multi-view images. Uses pixel-guided densification mechanism to capture details under large resolution increases.

Result: Superior performance in local fine detail reconstruction and strong scalability to images up to 1024×1024 resolution. Avoids high cost and redundancy of uniform high-resolution reconstructions.

Conclusion: The method effectively bridges the gap between prohibitive holistic reconstruction costs and user needs for localized fine details, enabling on-demand high-resolution reconstruction in regions of interest.

Abstract: We perceive our surroundings with an active focus, paying more attention to regions of interest, such as the shelf labels in a grocery store. When it comes to scene reconstruction, this human perception trait calls for spatially varying degrees of detail ready for closer inspection in critical regions, preferably reconstructed on demand. While recent works in 3D Gaussian Splatting (3DGS) achieve fast, generalizable reconstruction from sparse views, their uniform resolution output leads to high computational costs unscalable to high-resolution training. As a result, they cannot leverage available images at their original high resolution to reconstruct details. Per-scene optimization methods reconstruct finer details with adaptive density control, yet require dense observations and lengthy offline optimization. To bridge the gap between the prohibitive cost of high-resolution holistic reconstructions and the user needs for localized fine details, we propose the problem of localized high-resolution reconstruction via on-demand Gaussian densification. Given a low-resolution 3DGS reconstruction, the goal is to learn a generalizable network that densifies the initial 3DGS to capture fine details in a user-specified local region of interest (RoI), based on sparse high-resolution observations of the RoI. This formulation avoids the high cost and redundancy of uniformly high-resolution reconstructions and fully leverages high-resolution captures in critical regions. We propose GaussianLens, a feed-forward densification framework that fuses multi-modal information from the initial 3DGS and multi-view images. We further design a pixel-guided densification mechanism that effectively captures details under large resolution increases. Experiments demonstrate our method’s superior performance in local fine detail reconstruction and strong scalability to images of up to $1024\times1024$ resolution.

Method: Constructed strong white-box and adaptive attacks against various compression models, with rigorous evaluation across multiple attack scenarios to test robustness.

Result: Compression models producing realistic, high-fidelity reconstructions are substantially more resistant to attacks, while low-realism models can be broken. This robustness is not due to gradient masking.

Conclusion: Realistic reconstructions maintaining distributional alignment with natural images offer inherent robustness. Overcoming realism represents an essential challenge for future adversarial attacks and comprehensive security evaluation.

Abstract: Previous work has suggested that preprocessing images through lossy compression can defend against adversarial perturbations, but comprehensive attack evaluations have been lacking. In this paper, we construct strong white-box and adaptive attacks against various compression models and identify a critical challenge for attackers: high realism in reconstructed images significantly increases attack difficulty. Through rigorous evaluation across multiple attack scenarios, we demonstrate that compression models capable of producing realistic, high-fidelity reconstructions are substantially more resistant to our attacks. In contrast, low-realism compression models can be broken. Our analysis reveals that this is not due to gradient masking. Rather, realistic reconstructions maintaining distributional alignment with natural images seem to offer inherent robustness. This work highlights a significant obstacle for future adversarial attacks and suggests that developing more effective techniques to overcome realism represents an essential challenge for comprehensive security evaluation.

Method: Created a multimodal dataset integrating imaging, anatomical structures, demographics, and free-text annotations. Expanded previous LMOD benchmark by 50% with more color fundus photography, broadened task coverage, and systematically evaluated 24 state-of-the-art MLLMs.

Result: Top-performing models achieved ~58% accuracy in disease screening under zero-shot settings, but performance remained suboptimal for challenging tasks like disease staging.

Conclusion: The dataset, curation pipeline, and leaderboard will be publicly released to advance ophthalmic AI applications and reduce the global burden of vision-threatening diseases.

Abstract: Vision-threatening eye diseases pose a major global health burden, with timely diagnosis limited by workforce shortages and restricted access to specialized care. While multimodal large language models (MLLMs) show promise for medical image interpretation, advancing MLLMs for ophthalmology is hindered by the lack of comprehensive benchmark datasets suitable for evaluating generative models. We present a large-scale multimodal ophthalmology benchmark comprising 32,633 instances with multi-granular annotations across 12 common ophthalmic conditions and 5 imaging modalities. The dataset integrates imaging, anatomical structures, demographics, and free-text annotations, supporting anatomical structure recognition, disease screening, disease staging, and demographic prediction for bias evaluation. This work extends our preliminary LMOD benchmark with three major enhancements: (1) nearly 50% dataset expansion with substantial enlargement of color fundus photography; (2) broadened task coverage including binary disease diagnosis, multi-class diagnosis, severity classification with international grading standards, and demographic prediction; and (3) systematic evaluation of 24 state-of-the-art MLLMs. Our evaluations reveal both promise and limitations. Top-performing models achieved ~58% accuracy in disease screening under zero-shot settings, and performance remained suboptimal for challenging tasks like disease staging. We will publicly release the dataset, curation pipeline, and leaderboard to potentially advance ophthalmic AI applications and reduce the global burden of vision-threatening diseases.

Method: Developed MindSpeed-MLLM framework for efficient training on Ascend hardware, systematic data production methods, weight averaging from checkpoints with different sequence lengths, and test-time resolution search.

Result: MindVL-8B matches Qwen2.5VL-7B performance using only 10% training data, while MindVL-671B-A37B matches Qwen2.5VL-72B with only 3% training data and achieves comparable performance with other leading multimodal MoE models.

Conclusion: The work provides valuable hardware alternative, open data recipes, and effective performance-enhancing techniques for the community.

Abstract: We propose MindVL, a multimodal large language model (MLLMs) trained on Ascend NPUs. The training of state-of-the-art MLLMs is often confined to a limited set of hardware platforms and relies heavily on massive, undisclosed data recipes, which hinders reproducibility and open research. To change the common perception that Ascend hardware is unsuitable for efficient full-stage MLLM training, we introduce MindSpeed-MLLM, a highly efficient training framework that supports stable and high-performance training of large-scale Dense and Mixture-of-Experts (MoE) models on Ascend hardware. Based on this, we provide a systematic and open description of the data production methods and mixing strategies for all training stages. Furthermore, we present MindVL, a data-efficient multimodal large language model trained end-to-end on Ascend NPUs. In addition, we find that averaging weights from checkpoints trained with different sequence lengths is particularly effective and yields further gains when combined with test-time resolution search. Our experiments demonstrate superior data efficiency: MindVL-8B matches the performance of Qwen2.5VL-7B using only 10% of its training data, while our MoE model, MindVL-671B-A37B, matches Qwen2.5VL-72B using only 3% of the Qwen2.5VL training data, and achieves comparable performance with other leading multimodal MoE models. Our work provides the community with a valuable hardware alternative, open data recipes, and effective performance-enhancing techniques.

Method: Uses anchor-free formulation with directional offset prediction, Gaussian Position Encoding for spatial priors, and Cross-view Object Association Module for cross-view matching.

Result: Achieves state-of-the-art performance on benchmark datasets while being lightweight and computationally efficient.

Conclusion: Anchor-free paradigm with GPE and CVOAM provides superior geo-localization without anchor dependency, offering both performance and efficiency benefits.

Abstract: Most existing cross-view object geo-localization approaches adopt anchor-based paradigm. Although effective, such methods are inherently constrained by predefined anchors. To eliminate this dependency, we first propose an anchor-free formulation for cross-view object geo-localization, termed AFGeo. AFGeo directly predicts the four directional offsets (left, right, top, bottom) to the ground-truth box for each pixel, thereby localizing the object without any predefined anchors. To obtain a more robust spatial prior, AFGeo incorporates Gaussian Position Encoding (GPE) to model the click point in the query image, mitigating the uncertainty of object position that challenges object localization in cross-view scenarios. In addition, AFGeo incorporates a Cross-view Object Association Module (CVOAM) that relates the same object and its surrounding context across viewpoints, enabling reliable localization under large cross-view appearance gaps. By adopting an anchor-free localization paradigm that integrates GPE and CVOAM with minimal parameter overhead, our model is both lightweight and computationally efficient, achieving state-of-the-art performance on benchmark datasets.

Method: Generalized Contrastive Learning (GCL) enforces contrastive learning across all modalities within a mini-batch using existing image-caption paired datasets to learn a unified representation space.

Result: GCL demonstrates consistent performance improvements on off-the-shelf multimodal retrieval models (VISTA, CLIP, TinyCLIP) across M-BEIR, MMEB, and CoVR benchmarks.

Conclusion: GCL effectively addresses multimodal retrieval challenges without the burden of new dataset curation, showing robust performance improvements across various models and benchmarks.

Abstract: Despite their consistent performance improvements, cross-modal retrieval models (e.g., CLIP) show degraded performances with retrieving keys composed of fused image-text modality (e.g., Wikipedia pages with both images and text). To address this critical challenge, multimodal retrieval has been recently explored to develop a unified single retrieval model capable of retrieving keys across diverse modality combinations. A common approach involves constructing new composed sets of image-text triplets (e.g., retrieving a pair of image and text given a query image). However, such an approach requires careful curation to ensure the dataset quality and fails to generalize to unseen modality combinations. To overcome these limitations, this paper proposes Generalized Contrastive Learning (GCL), a novel loss formulation that improves multimodal retrieval performance without the burdensome need for new dataset curation. Specifically, GCL operates by enforcing contrastive learning across all modalities within a mini-batch, utilizing existing image-caption paired datasets to learn a unified representation space. We demonstrate the effectiveness of GCL by showing consistent performance improvements on off-the-shelf multimodal retrieval models (e.g., VISTA, CLIP, and TinyCLIP) using the M-BEIR, MMEB, and CoVR benchmarks.

Method: Created three databases with real-life and synthetic trucks, trained three YOLO architectures, evaluated using recall, precision, F1-score, mAP, and Mann-Whitney U test for statistical significance.

Result: Synthetic images proved reliable for training, contributing to all networks’ performance with highest mAP reaching 99%. Results were statistically significant.

Conclusion: Synthetic images from video games can successfully train neural networks, providing a reliable and low-cost data source for knowledge extraction.

Abstract: Convolutional Neural Networks (CNNs) traditionally require large amounts of data to train models with good performance. However, data collection is an expensive process, both in time and resources. Generated synthetic images are a good alternative, with video games producing realistic 3D models. This paper aims to determine whether images extracted from a video game can be effectively used to train a CNN to detect real-life truck axles. Three different databases were created, with real-life and synthetic trucks, to provide training and testing examples for three different You Only Look Once (YOLO) architectures. Results were evaluated based on four metrics: recall, precision, F1-score, and mean Average Precision (mAP). To evaluate the statistical significance of the results, the Mann-Whitney U test was also applied to the resulting mAP of all models. Synthetic images from trucks extracted from a video game proved to be a reliable source of training data, contributing to the performance of all networks. The highest mAP score reached 99%. Results indicate that synthetic images can be used to train neural networks, providing a reliable, low-cost data source for extracting knowledge.

Method: Proposed Geo-DLC task for object-level captioning, constructed DE-Dataset with 25 categories and 261,806 annotated instances, introduced DE-Benchmark LLM-assisted evaluation suite, and developed DescribeEarth MLLM with scale-adaptive focal strategy and domain-guided fusion module.

Result: DescribeEarth model consistently outperforms state-of-the-art general MLLMs on DE-Benchmark, demonstrating superior factual accuracy, descriptive richness, and grammatical soundness across various remote sensing scenarios.

Conclusion: The proposed Geo-DLC task and DescribeEarth model effectively address the limitation of existing image-level captioning by providing fine-grained object-level interpretation, enabling better utilization of remote sensing image information.

Abstract: Automated textual description of remote sensing images is crucial for unlocking their full potential in diverse applications, from environmental monitoring to urban planning and disaster management. However, existing studies in remote sensing image captioning primarily focus on the image level, lacking object-level fine-grained interpretation, which prevents the full utilization and transformation of the rich semantic and structural information contained in remote sensing images. To address this limitation, we propose Geo-DLC, a novel task of object-level fine-grained image captioning for remote sensing. To support this task, we construct DE-Dataset, a large-scale dataset contains 25 categories and 261,806 annotated instances with detailed descriptions of object attributes, relationships, and contexts. Furthermore, we introduce DE-Benchmark, a LLM-assisted question-answering based evaluation suite designed to systematically measure model capabilities on the Geo-DLC task. We also present DescribeEarth, a Multi-modal Large Language Model (MLLM) architecture explicitly designed for Geo-DLC, which integrates a scale-adaptive focal strategy and a domain-guided fusion module leveraging remote sensing vision-language model features to encode high-resolution details and remote sensing category priors while maintaining global context. Our DescribeEarth model consistently outperforms state-of-the-art general MLLMs on DE-Benchmark, demonstrating superior factual accuracy, descriptive richness, and grammatical soundness, particularly in capturing intrinsic object features and surrounding environmental attributes across simple, complex, and even out-of-distribution remote sensing scenarios. All data, code and weights are released at https://github.com/earth-insights/DescribeEarth.

Method: Introduces OmniDFA framework for both image authenticity assessment and few-shot source attribution, supported by OmniFake dataset containing 1.17M images from 45 generative models for training and evaluation.

Result: OmniDFA demonstrates excellent capability in open-set attribution and achieves state-of-the-art generalization performance on AI-generated image detection across various experiments.

Conclusion: The proposed open-set, few-shot source identification paradigm and OmniDFA framework provide a practical solution for real-world AI-generated image analysis, with the OmniFake dataset serving as a valuable resource for future research.

Abstract: AI-generated image (AIGI) detection and source model attribution remain central challenges in combating deepfake abuses, primarily due to the structural diversity of generative models. Current detection methods are prone to overfitting specific forgery traits, whereas source attribution offers a robust alternative through fine-grained feature discrimination. However, synthetic image attribution remains constrained by the scarcity of large-scale, well-categorized synthetic datasets, limiting its practicality and compatibility with detection systems. In this work, we propose a new paradigm for image attribution called open-set, few-shot source identification. This paradigm is designed to reliably identify unseen generators using only limited samples, making it highly suitable for real-world application. To this end, we introduce OmniDFA (Omni Detector and Few-shot Attributor), a novel framework for AIGI that not only assesses the authenticity of images, but also determines the synthesis origins in a few-shot manner. To facilitate this work, we construct OmniFake, a large class-aware synthetic image dataset that curates $1.17$ M images from $45$ distinct generative models, substantially enriching the foundational resources for research on both AIGI detection and attribution. Experiments demonstrate that OmniDFA exhibits excellent capability in open-set attribution and achieves state-of-the-art generalization performance on AIGI detection. Our dataset and code will be made available.

Method: Proposes three components: Context-enhanced Attention-map Generation (CAG) for reliable attention maps, Active Visual Probing (AVP) for goal-oriented visual selection using information theory, and Dynamic Attention-shifting Trigger (DAT) for optimal timing of visual information insertion.

Result: Extensive experiments on three challenging benchmarks show AIMCoT significantly outperforms state-of-the-art methods across different settings.

Conclusion: AIMCoT represents a critical step towards more robust, effective, and human-like multimodal reasoning by actively foraging for information and dynamically structuring the reasoning process.

Abstract: Multimodal Chain-of-Thought (CoT) has emerged as a powerful technique for enhancing the vision-language reasoning with interleaved information. However, existing methods often rely on simplistic heuristics for constructing interleaved CoT, typically depending on attention maps, which our empirical analysis reveals can be unreliable. What’s more, the shortcomings of their passive and purposeless selection strategies and their arbitrary triggering mechanisms in capturing the model’s cognitive need for information are further amplified. In this paper, we propose \textbf{AIMCoT}, an \textbf{A}ctive \textbf{I}nformation-driven \textbf{M}ulti-modal \textbf{C}hain-\textbf{o}f-\textbf{T}hought framework that addresses these fundamental limitations. AIMCoT introduces three synergistic components: (1) \textbf{Context-enhanced Attention-map Generation (CAG)}, which mitigates the text-vision granularity imbalance, thereby producing more reliable attention maps as a foundation. (2) \textbf{Active Visual Probing (AVP)}, which replaces passive selection with a proactive, goal-oriented strategy grounded in information theory to select image regions that help answer the questions maximally. (3) \textbf{Dynamic Attention-shifting Trigger (DAT)}, which intelligently determines the optimal moments to insert visual information by monitoring the model’s text-to-vision attention shifts. Extensive experiments on three challenging benchmarks demonstrate that AIMCoT significantly outperforms state-of-the-art methods across different settings. By actively foraging for information and dynamically structuring its reasoning process, AIMCoT represents a critical step towards more robust, effective, and human-like multimodal reasoning. Our code is available at https://anonymous.4open.science/r/AIMCoT.

Method: Analyzed latent space dynamics during diffusion and denoising processes, examining training loss, classifier-free guidance effects, and decomposition of intermediate latents to identify the mechanisms driving memorization.

Result: Found that memorization is driven by overestimation of training samples during early denoising, which reduces diversity, collapses denoising trajectories, and accelerates convergence toward memorized images. Deviations from theoretical denoising schedule correlate with memorization severity.

Conclusion: Early overestimation is identified as the central underlying mechanism of memorization in diffusion models, explaining how memorized prompts force latent trajectories to converge and steer denoising toward paired training samples.

Abstract: Despite their success in image generation, diffusion models can memorize training data, raising serious privacy and copyright concerns. Although prior work has sought to characterize, detect, and mitigate memorization, the fundamental question of why and how it occurs remains unresolved. In this paper, we revisit the diffusion and denoising process and analyze latent space dynamics to address the question: “How do diffusion models memorize?” We show that memorization is driven by the overestimation of training samples during early denoising, which reduces diversity, collapses denoising trajectories, and accelerates convergence toward the memorized image. Specifically: (i) memorization cannot be explained by overfitting alone, as training loss is larger under memorization due to classifier-free guidance amplifying predictions and inducing overestimation; (ii) memorized prompts inject training images into noise predictions, forcing latent trajectories to converge and steering denoising toward their paired samples; and (iii) a decomposition of intermediate latents reveals how initial randomness is quickly suppressed and replaced by memorized content, with deviations from the theoretical denoising schedule correlating almost perfectly with memorization severity. Together, these results identify early overestimation as the central underlying mechanism of memorization in diffusion models.

Method: Uses probabilistic contrastive learning with InfoNCE loss and Hellinger distance to align four modalities (chest X-rays, ECG, echocardiograms, clinical text) into unified probabilistic embedding space, plus probabilistic synthetic sampling loss for intra-modality binding.

Result: Outperforms current Med-VLPMs across 13 medical datasets in cross-modality retrieval, zero-shot, and few-shot classification, with improved intra- and inter-modality binding.

Conclusion: Probabilistic modeling of embeddings better captures the many-to-many nature of medical modalities, enabling more robust multimodal integration for medical diagnosis and prognostication.

Abstract: Medical decision-making requires integrating diverse medical information, from imaging to clinical narratives. These medical modalities are often acquired in a many-to-many manner. However, current medical vision-language pretraining models (Med-VLPMs) fail to directly account for this many-to-many mapping in their model training and embeddings. To address this, we present Probabilistic Modality-Enhanced Diagnosis (ProbMED), a multimodal Med-VLPM that employs probabilistic contrastive learning to model distributions over embeddings rather than deterministic estimates. ProbMED aligns four distinct modalities – chest X-rays, electrocardiograms, echocardiograms, and clinical text – into a unified probabilistic embedding space. We use InfoNCE loss with Hellinger distance to integrate inter-modality distributions. We introduce a probabilistic synthetic sampling loss that captures modality-specific mean and variance to improve intra-modality binding. Extensive experiments across 13 medical datasets demonstrate that our model outperforms current Med-VLPMs in cross-modality retrieval, zero-shot, and few-shot classification. We also demonstrate the robust integration of multiple modalities for prognostication, showing improved intra- and inter-medical modality binding.

Method: Proposes MISP-DPO framework that embeds prompts and images in CLIP space, uses sparse autoencoder to uncover semantic deviations, selects negative samples based on reconstruction difficulty, semantic deviation from positive, and mutual diversity, and employs Plackett-Luce objective with importance sampling for efficient training.

Result: Experiments across five diverse benchmarks show MISP-DPO consistently improves multimodal alignment over prior methods, validating the effectiveness of semantic-aware, multi-negative sampling.

Conclusion: The proposed semantic-aware, multi-negative sampling approach in MISP-DPO effectively addresses limitations of existing multimodal DPO methods and improves preference-based learning for vision-language models.

Abstract: Direct Preference Optimization (DPO) has recently been extended from text-only models to vision-language models. However, existing methods rely on oversimplified pairwise comparisons, generating a single negative image via basic perturbations or similarity-based retrieval, which fail to capture the complex nature of multimodal preferences, inducing optimization bias and hallucinations. To address this issue, we propose MISP-DPO, the first framework to incorporate multiple, semantically diverse negative images in multimodal DPO via the Plackett-Luce model. Our method embeds prompts and candidate images in CLIP (Contrastive Language-Image Pretraining) space and applies a sparse autoencoder to uncover semantic deviations into interpretable factors. Negative samples are selected based on reconstruction difficulty, semantic deviation from the positive, and mutual diversity, yielding broader and more informative supervision. To handle multi-negative comparisons, we adopt a Plackett-Luce objective and introduce an importance sampling strategy that improves training efficiency. Experiments across five diverse benchmarks demonstrate that MISP-DPO consistently improves multimodal alignment over prior methods, validating the effectiveness of semantic-aware, multi-negative sampling in preference-based learning.

Method: SAGE introduces a Soft Probing module for patch descriptor weighting, reconstructs an online geo-visual graph that fuses geographic proximity and visual similarity, and uses greedy weighted clique expansion for hard sample mining.

Result: Achieves 98.9%, 95.8%, 94.5%, and 96.0% Recall@1 on SPED, Pitts30k-test, MSLS-val, and Nordland respectively, with 100% Recall@10 on SPED using only 4096D global descriptors.

Conclusion: SAGE demonstrates superior performance through its unified approach to spatial-visual discrimination, achieving state-of-the-art results across eight benchmarks with parameter-efficient fine-tuning.

Abstract: Visual Place Recognition (VPR) requires robust retrieval of geotagged images despite large appearance, viewpoint, and environmental variation. Prior methods focus on descriptor fine-tuning or fixed sampling strategies yet neglect the dynamic interplay between spatial context and visual similarity during training. We present SAGE (Spatial-visual Adaptive Graph Exploration), a unified training pipeline that enhances granular spatial-visual discrimination by jointly improving local feature aggregation, organize samples during training, and hard sample mining. We introduce a lightweight Soft Probing module that learns residual weights from training data for patch descriptors before bilinear aggregation, boosting distinctive local cues. During training we reconstruct an online geo-visual graph that fuses geographic proximity and current visual similarity so that candidate neighborhoods reflect the evolving embedding landscape. To concentrate learning on the most informative place neighborhoods, we seed clusters from high-affinity anchors and iteratively expand them with a greedy weighted clique expansion sampler. Implemented with a frozen DINOv2 backbone and parameter-efficient fine-tuning, SAGE achieves SOTA across eight benchmarks. It attains 98.9%, 95.8%, 94.5%, and 96.0% Recall@1 on SPED, Pitts30k-test, MSLS-val, and Nordland, respectively. Notably, our method obtains 100% Recall@10 on SPED only using 4096D global descriptors. Code and model will be available at: https://github.com/chenshunpeng/SAGE.

Method: Proposes LaTo with three key innovations: (1) landmark tokenizer that quantizes raw coordinates into discrete facial tokens, (2) location-mapping positional encoding for unified facial and image token processing, and (3) landmark predictor using vision-language models with structured chain-of-thought for accurate estimation.

Result: LaTo outperforms state-of-the-art methods by 7.8% in identity preservation and 4.6% in semantic consistency. Created HFL-150K, the largest benchmark with over 150K real face pairs and fine-grained instructions.

Conclusion: LaTo provides superior identity preservation and semantic consistency for face editing through landmark tokenization and unified geometry-appearance processing, addressing limitations of rigid geometric constraints in existing methods.

Abstract: Recent multimodal models for instruction-based face editing enable semantic manipulation but still struggle with precise attribute control and identity preservation. Structural facial representations such as landmarks are effective for intermediate supervision, yet most existing methods treat them as rigid geometric constraints, which can degrade identity when conditional landmarks deviate significantly from the source (e.g., large expression or pose changes, inaccurate landmark estimates). To address these limitations, we propose LaTo, a landmark-tokenized diffusion transformer for fine-grained, identity-preserving face editing. Our key innovations include: (1) a landmark tokenizer that directly quantizes raw landmark coordinates into discrete facial tokens, obviating the need for dense pixel-wise correspondence; (2) a location-mapping positional encoding that integrates facial and image tokens for unified processing, enabling flexible yet decoupled geometry-appearance interactions with high efficiency and strong identity preservation; and (3) a landmark predictor that leverages vision-language models to infer target landmarks from instructions and source images, whose structured chain-of-thought improves estimation accuracy and interactive control. To mitigate data scarcity, we curate HFL-150K, to our knowledge the largest benchmark for this task, containing over 150K real face pairs with fine-grained instructions. Extensive experiments show that LaTo outperforms state-of-the-art methods by 7.8% in identity preservation and 4.6% in semantic consistency. Code and dataset will be made publicly available upon acceptance.

Method: Freeze SAM2 feature extractor during training while fine-tuning other components, and use confidence-guided fusion segmentation with pixel-check strategy during inference to progressively refine predictions by combining multiple models.

Result: Achieved 86.37% J&F score on test set, ranking 1st in MOSEv1 Challenge at LSVOS 2025.

Conclusion: The approach effectively addresses VOS challenges in complex scenarios through preserved feature extraction and robust fusion strategy.

Abstract: Video Object Segmentation (VOS) aims to track and segment specific objects across entire video sequences, yet it remains highly challenging under complex real-world scenarios. The MOSEv1 and LVOS dataset, adopted in the MOSEv1 challenge on LSVOS 2025, which is specifically designed to enhance the robustness of VOS models in complex real-world scenarios, including long-term object disappearances and reappearances, as well as the presence of small and inconspicuous objects. In this paper, we present our improved method, Confidence-Guided Fusion Segmentation (CGFSeg), for the VOS task in the MOSEv1 Challenge. During training, the feature extractor of SAM2 is frozen, while the remaining components are fine-tuned to preserve strong feature extraction ability and improve segmentation accuracy. In the inference stage, we introduce a pixel-check strategy that progressively refines predictions by exploiting complementary strengths of multiple models, thereby yielding robust final masks. As a result, our method achieves a J&F score of 86.37% on the test set, ranking 1st in the MOSEv1 Challenge at LSVOS 2025. These results highlight the effectiveness of our approach in addressing the challenges of VOS task in complex scenarios.

Method: Proposes SO(3) diffusion model for pose parameter distributions, uses transformer for hierarchical joint structure learning, and MLP-based denoising model for per-joint distribution conditioned on latent vectors from time-independent transformer.

Result: The model effectively predicts accurate pose probability distributions, demonstrating improved performance in handling the accuracy-diversity trade-off compared to existing probabilistic methods.

Conclusion: The proposed approach successfully models well-aligned pose distributions to 2D observations, providing a better solution for handling ambiguity in human mesh recovery while maintaining competitive accuracy.

Abstract: We tackle the problem of Human Mesh Recovery (HMR) from a single RGB image, formulating it as an image-conditioned human pose and shape generation. While recovering 3D human pose from 2D observations is inherently ambiguous, most existing approaches have regressed a single deterministic output. Probabilistic methods attempt to address this by generating multiple plausible outputs to model the ambiguity. However, these methods often exhibit a trade-off between accuracy and sample diversity, and their single predictions are not competitive with state-of-the-art deterministic models. To overcome these limitations, we propose a novel approach that models well-aligned distribution to 2D observations. In particular, we introduce $SO(3)$ diffusion model, which generates the distribution of pose parameters represented as 3D rotations unconditional and conditional to image observations via conditioning dropout. Our model learns the hierarchical structure of human body joints using the transformer. Instead of using transformer as a denoising model, the time-independent transformer extracts latent vectors for the joints and a small MLP-based denoising model learns the per-joint distribution conditioned on the latent vector. We experimentally demonstrate and analyze that our model predicts accurate pose probability distribution effectively.

Method: Built upon a unified displacement field that jointly encodes 3D geometry and 2D spatial priors, with a conflict-free partitioning strategy to isolate editing regions and prevent interference.

Result: Extensive experiments show superior precision, structural consistency, and reliable multi-point editability across various editing scenarios.

Conclusion: GeoDrag enables coherent, high-fidelity, and structure-consistent editing in a single forward pass, effectively addressing challenges of 3D geometric integration, discontinuity mitigation, and multi-point conflict resolution.

Abstract: Interactive point-based image editing serves as a controllable editor, enabling precise and flexible manipulation of image content. However, most drag-based methods operate primarily on the 2D pixel plane with limited use of 3D cues. As a result, they often produce imprecise and inconsistent edits, particularly in geometry-intensive scenarios such as rotations and perspective transformations. To address these limitations, we propose a novel geometry-guided drag-based image editing method - GeoDrag, which addresses three key challenges: 1) incorporating 3D geometric cues into pixel-level editing, 2) mitigating discontinuities caused by geometry-only guidance, and 3) resolving conflicts arising from multi-point dragging. Built upon a unified displacement field that jointly encodes 3D geometry and 2D spatial priors, GeoDrag enables coherent, high-fidelity, and structure-consistent editing in a single forward pass. In addition, a conflict-free partitioning strategy is introduced to isolate editing regions, effectively preventing interference and ensuring consistency. Extensive experiments across various editing scenarios validate the effectiveness of our method, showing superior precision, structural consistency, and reliable multi-point editability. The code will be available on https://github.com/xinyu-pu/GeoDrag .

Method: Proposes IPDRecon framework with three core components: Pixel-level Confidence Encoder (PCE), Affine Compensation Module (ACM), and Image-Plane Spatial Decoder (IPSD), which collaboratively decode 3D structural information from 2D images through physical imaging processes.

Result: On ScanNetV2, IPDRecon achieves superior reconstruction stability with nearly identical quality when view count reduces by 40%, achieving coefficient of variation of 0.24%, performance retention rate of 99.7%, and maximum performance drop of only 0.42%.

Conclusion: Exploiting intra-view spatial information provides a robust solution for view-limited scenarios in practical applications, effectively preserving spatial geometric features while significantly enhancing view-invariant reconstruction.

Abstract: Volume-based indoor scene reconstruction methods demonstrate significant research value due to their superior generalization capability and real-time deployment potential. However, existing methods rely on multi-view pixel back-projection ray intersections as weak geometric constraints to determine spatial positions, causing reconstruction quality to depend heavily on input view density with poor performance in overlapping regions and unobserved areas. To address these issues, the key lies in reducing dependency on inter-view geometric constraints while exploiting rich spatial information within individual views. We propose IPDRecon, an image-plane decoding framework comprising three core components: Pixel-level Confidence Encoder (PCE), Affine Compensation Module (ACM), and Image-Plane Spatial Decoder (IPSD). These modules collaboratively decode 3D structural information encoded in 2D images through physical imaging processes, effectively preserving spatial geometric features including edges, hollow structures, and complex textures while significantly enhancing view-invariant reconstruction. Experiments on ScanNetV2 confirm that IPDRecon achieves superior reconstruction stability, maintaining nearly identical quality when view count reduces by 40%. The method achieves a coefficient of variation of only 0.24%, performance retention rate of 99.7%, and maximum performance drop of merely 0.42%. This demonstrates that exploiting intra-view spatial information provides a robust solution for view-limited scenarios in practical applications.

Method: Curated a 2-million-scale multimodal dataset combining textbook knowledge, public data, synthetic samples, and general corpora. Used three-stage training: domain-specialized pretraining, instruction-driven alignment, and reinforcement-based refinement. Dolphin R1 adds reinforcement learning with ultrasound-specific rewards.

Result: Dolphin R1 achieves U2-score of 0.5835 on U2-Bench across eight ultrasound tasks - over twice the second-best model (0.2968). Dolphin v1.0 also performs competitively, validating the unified framework. Reasoning-enhanced training significantly improves diagnostic accuracy, consistency, and interpretability.

Conclusion: The Dolphin series successfully addresses ultrasound AI challenges through a unified multimodal framework. Reasoning-augmented training proves crucial for high-stakes medical AI, significantly enhancing diagnostic performance and interpretability in ultrasound applications.

Abstract: Ultrasound is crucial in modern medicine but faces challenges like operator dependence, image noise, and real-time scanning, hindering AI integration. While large multimodal models excel in other medical imaging areas, they struggle with ultrasound’s complexities. To address this, we introduce Dolphin v1.0 (V1) and its reasoning-augmented version, Dolphin R1-the first large-scale multimodal ultrasound foundation models unifying diverse clinical tasks in a single vision-language framework.To tackle ultrasound variability and noise, we curated a 2-million-scale multimodal dataset, combining textbook knowledge, public data, synthetic samples, and general corpora. This ensures robust perception, generalization, and clinical adaptability.The Dolphin series employs a three-stage training strategy: domain-specialized pretraining, instruction-driven alignment, and reinforcement-based refinement. Dolphin v1.0 delivers reliable performance in classification, detection, regression, and report generation. Dolphin R1 enhances diagnostic inference, reasoning transparency, and interpretability through reinforcement learning with ultrasound-specific rewards.Evaluated on U2-Bench across eight ultrasound tasks, Dolphin R1 achieves a U2-score of 0.5835-over twice the second-best model (0.2968) setting a new state of the art. Dolphin v1.0 also performs competitively, validating the unified framework. Comparisons show reasoning-enhanced training significantly improves diagnostic accuracy, consistency, and interpretability, highlighting its importance for high-stakes medical AI.

Method: Reformulates VITON as a linear inverse problem and uses trajectory-aligned solvers with residual prior-based initialization and artifact-free measurement-guided sampling that combines data consistency, frequency-level correction, and periodic standard denoising.

Result: Experiments on VITON-HD, DressCode, and SHHQ-1.0 show effective preservation of identity and background, elimination of boundary artifacts, and improved visual fidelity and robustness over state-of-the-art baselines.

Conclusion: ART-VITON successfully addresses the challenge of preserving non-try-on regions in virtual try-on while maintaining high-quality garment synthesis through measurement-guided diffusion framework.

Abstract: Virtual try-on (VITON) aims to generate realistic images of a person wearing a target garment, requiring precise garment alignment in try-on regions and faithful preservation of identity and background in non-try-on regions. While latent diffusion models (LDMs) have advanced alignment and detail synthesis, preserving non-try-on regions remains challenging. A common post-hoc strategy directly replaces these regions with original content, but abrupt transitions often produce boundary artifacts. To overcome this, we reformulate VITON as a linear inverse problem and adopt trajectory-aligned solvers that progressively enforce measurement consistency, reducing abrupt changes in non-try-on regions. However, existing solvers still suffer from semantic drift during generation, leading to artifacts. We propose ART-VITON, a measurement-guided diffusion framework that ensures measurement adherence while maintaining artifact-free synthesis. Our method integrates residual prior-based initialization to mitigate training-inference mismatch and artifact-free measurement-guided sampling that combines data consistency, frequency-level correction, and periodic standard denoising. Experiments on VITON-HD, DressCode, and SHHQ-1.0 demonstrate that ART-VITON effectively preserves identity and background, eliminates boundary artifacts, and consistently improves visual fidelity and robustness over state-of-the-art baselines.

Method: TPO trains models to prefer matched prompts over mismatched prompts constructed by perturbing original captions using large language models. Extends DPO and KTO to create TDPO and TKTO.

Result: Quantitative and qualitative evaluations show TPO methods outperform original counterparts, achieving better human preference scores and improved text-to-image alignment.

Conclusion: TPO enables cost-effective alignment of text-to-image models without paired image preference data, demonstrating superior performance over existing methods.

Abstract: Recent advances in diffusion-based text-to-image (T2I) models have led to remarkable success in generating high-quality images from textual prompts. However, ensuring accurate alignment between the text and the generated image remains a significant challenge for state-of-the-art diffusion models. To address this, existing studies employ reinforcement learning with human feedback (RLHF) to align T2I outputs with human preferences. These methods, however, either rely directly on paired image preference data or require a learned reward function, both of which depend heavily on costly, high-quality human annotations and thus face scalability limitations. In this work, we introduce Text Preference Optimization (TPO), a framework that enables “free-lunch” alignment of T2I models, achieving alignment without the need for paired image preference data. TPO works by training the model to prefer matched prompts over mismatched prompts, which are constructed by perturbing original captions using a large language model. Our framework is general and compatible with existing preference-based algorithms. We extend both DPO and KTO to our setting, resulting in TDPO and TKTO. Quantitative and qualitative evaluations across multiple benchmarks show that our methods consistently outperform their original counterparts, delivering better human preference scores and improved text-to-image alignment. Our Open-source code is available at https://github.com/DSL-Lab/T2I-Free-Lunch-Alignment.

Method: Created v-HUB benchmark with curated minimally verbal short videos paired with rich annotations (captions, descriptions, explanations). Evaluated diverse MLLMs including specialized Video-LLMs and versatile OmniLLMs on tasks like caption matching and humor explanation, with both video-only and audio-included conditions.

Result: MLLMs struggle significantly with humor comprehension from visual cues alone, showing marked performance drops in caption matching when moving from text-based to video-based evaluation without audio. Incorporating audio helps with video humor understanding.

Conclusion: Current MLLMs face difficulties in understanding humor from purely visual information, but audio integration shows promise for improving performance on complex video understanding tasks involving humor.

Abstract: AI models capable of comprehending humor hold real-world promise – for example, enhancing engagement in human-machine interactions. To gauge and diagnose the capacity of multimodal large language models (MLLMs) for humor understanding, we introduce v-HUB, a novel visual-centric video humor understanding benchmark. v-HUB comprises a curated collection of minimally verbal short videos, sourced from classic silent films and online resources, and reflecting real-world scenarios where humor can be appreciated purely through visual cues. Each video clip is paired with rich annotations, including captions, descriptions, and explanations, supporting evaluation tasks like caption matching and humor explanation. To broaden its applicability, we further construct an open-ended video QA task, making it readily integrable into existing video understanding benchmarks. We evaluate a diverse set of MLLMs, from specialized Video-LLMs to versatile OmniLLMs that can process audio, covering both open-source and proprietary domains. The experimental results expose the difficulties MLLMs face in comprehending humor from visual cues alone. For example, all models exhibit a marked performance drop on caption matching when moving from text-based to video-based evaluation (without audio). Our findings also demonstrate that incorporating audio helps with video humor understanding, highlighting the informativeness of sound and the promise of integrating richer modalities for complex video understanding tasks.

Method: Proportionate Credit Policy Optimization (PCPO) framework that enforces proportional credit assignment through stable objective reformulation and principled reweighting of timesteps.

Result: PCPO stabilizes training, accelerates convergence significantly, improves image quality by mitigating model collapse, and substantially outperforms existing policy gradient baselines including state-of-the-art DanceGRPO.

Conclusion: PCPO effectively addresses the fundamental issue of disproportionate credit assignment in text-to-image model alignment, providing a more stable and effective training framework that achieves superior performance across all metrics.

Abstract: While reinforcement learning has advanced the alignment of text-to-image (T2I) models, state-of-the-art policy gradient methods are still hampered by training instability and high variance, hindering convergence speed and compromising image quality. Our analysis identifies a key cause of this instability: disproportionate credit assignment, in which the mathematical structure of the generative sampler produces volatile and non-proportional feedback across timesteps. To address this, we introduce Proportionate Credit Policy Optimization (PCPO), a framework that enforces proportional credit assignment through a stable objective reformulation and a principled reweighting of timesteps. This correction stabilizes the training process, leading to significantly accelerated convergence and superior image quality. The improvement in quality is a direct result of mitigating model collapse, a common failure mode in recursive training. PCPO substantially outperforms existing policy gradient baselines on all fronts, including the state-of-the-art DanceGRPO.

Method: The method analyzes noise map properties for enhanced editability and introduces an editable noise refinement that minimizes the difference between reconstructed and edited noise maps to align with desired edits.

Result: Extensive experiments show ENM Inversion outperforms existing approaches across various image editing tasks in both preservation and edit fidelity with target prompts, and can also be applied to video editing for temporal consistency.

Conclusion: ENM Inversion successfully addresses the limitation of previous inversion methods by providing optimal noise maps that balance content preservation with editability, enabling effective text-guided image and video editing.

Abstract: Text-to-image diffusion models have achieved remarkable success in generating high-quality and diverse images. Building on these advancements, diffusion models have also demonstrated exceptional performance in text-guided image editing. A key strategy for effective image editing involves inverting the source image into editable noise maps associated with the target image. However, previous inversion methods face challenges in adhering closely to the target text prompt. The limitation arises because inverted noise maps, while enabling faithful reconstruction of the source image, restrict the flexibility needed for desired edits. To overcome this issue, we propose Editable Noise Map Inversion (ENM Inversion), a novel inversion technique that searches for optimal noise maps to ensure both content preservation and editability. We analyze the properties of noise maps for enhanced editability. Based on this analysis, our method introduces an editable noise refinement that aligns with the desired edits by minimizing the difference between the reconstructed and edited noise maps. Extensive experiments demonstrate that ENM Inversion outperforms existing approaches across a wide range of image editing tasks in both preservation and edit fidelity with target prompts. Our approach can also be easily applied to video editing, enabling temporal consistency and content manipulation across frames.

Method: Adapts self-consistency to IQA by generating pseudo-labels through pairwise majority voting on VLM outputs, then using these pseudo-rankings as fidelity rewards to guide iterative evolution via group relative policy optimization (GRPO).

Result: Boosts base VLM’s zero-shot performance by 31.8% on PLCC across diverse IQA benchmarks, achieving competitive or superior performance to state-of-the-art supervised VLM-based IQA models on 5 out of 7 benchmarks.

Conclusion: EvoQuality demonstrates that VLMs can autonomously refine perceptual capabilities through self-supervised learning, achieving remarkable performance without human annotations.

Abstract: Improving vision-language models (VLMs) in the post-training stage typically relies on supervised fine-tuning or reinforcement learning, methods that necessitate costly, human-annotated data. While self-supervised techniques such as self-consistency have proven effective for enhancing reasoning capabilities, their application to perceptual domains such as image quality assessment (IQA) remains largely unexplored. In this work, we introduce EvoQuality, a novel framework that enables a VLM to autonomously refine its quality perception capabilities without any ground-truth labels. EvoQuality adapts the principle of self-consistency to the ranking-based nature of IQA. It generates pseudo-labels by performing pairwise majority voting on the VLM’s own outputs to establish a consensus on relative quality. These pseudo-rankings are then formulated into a fidelity reward that guides the model’s iterative evolution through group relative policy optimization (GRPO). By iteratively leveraging its own predictions, EvoQuality progressively refines the VLM’s perceptual capability. Extensive experiments show that EvoQuality boosts the base VLM’s zero-shot performance by 31.8% on PLCC across diverse IQA benchmarks. Remarkably, despite being entirely self-supervised, EvoQuality achieves performance that is competitive with, or even surpasses, state-of-the-art supervised VLM-based IQA models, outperforming these models on 5 out of 7 IQA benchmarks.

Method: Probabilistic student-teacher model integrating Probabilistic Cross-Modal Embeddings (PCME++) with ECHO-CLIP vision-language model to distill echocardiogram knowledge into ECG representations using uncertainty-aware embeddings.

Result: Outperforms state-of-the-art foundation ECG models in zero-shot, few-shot, and fine-tune settings for echocardiogram predictions based on ECG. Variance estimation helps identify regions of uncertainty within ECGs.

Conclusion: EchoingECG successfully enables ECG-based prediction of cardiac function measurements traditionally requiring echocardiograms, with improved performance and uncertainty awareness.

Abstract: Electrocardiogram (ECG) is a widely used tool for assessing cardiac function due to its low cost and accessibility. Emergent research shows that ECGs can help make predictions on key outcomes traditionally derived from more complex modalities such as echocardiograms (ECHO), enabling the use of ECGs as a more accessible method to predict broader measurements of cardiac function. ECHO, in particular, are of great importance because they require considerable hospital resources while playing a key role in clinical cardiac assessment. To aid this use case, we introduce EchoingECG, a probabilistic student-teacher model that leverages uncertainty-aware ECG embeddings and ECHO supervision to improve ECG-based cardiac function prediction. Our approach integrates Probabilistic Cross-Modal Embeddings (PCME++), a probabilistic contrastive framework, with ECHO-CLIP, a vision-language pre-trained model trained on ECHO-text pairs, to distill ECHO knowledge into ECG representations. Through experiments and external validation, we showed that EchoingECG outperforms state-of-the-art foundation ECG models in zero-shot, few-shot, and fine-tune settings for ECHO predictions based on ECG. We also highlighted that variance estimation (enabled through our method) enhanced our understanding of model performance by identifying underlying regions of uncertainty within ECGs. The code is available: https://github.com/mcintoshML/EchoingECG.

Method: Proposed a hierarchical evaluation protocol with three stages (S1: referred-object localization, S2: task-driven pointing, S3: visual trace prediction) using data from indoor, kitchen, driving, and robotic manipulation domains.

Result: Experiments with 10+ state-of-the-art VLMs show that strong general-purpose models like GPT-4o underperform in precise visual grounding compared to some open-source models, and models like MoLMO excel in S1-S2 but struggle with S3’s visual trace planning.

Conclusion: The PIO benchmark reveals significant gaps in VLMs’ embodied reasoning capabilities, particularly in precise visual grounding and visual trace planning, highlighting the need for improved evaluation methods beyond traditional benchmarks.

Abstract: Vision-Language Models (VLMs) have demonstrated impressive world knowledge across a wide range of tasks, making them promising candidates for embodied reasoning applications. However, existing benchmarks primarily evaluate the embodied reasoning ability of VLMs through multiple-choice questions based on image annotations – for example, selecting which trajectory better describes an event in the image. In this work, we introduce the Point-It-Out (PIO) benchmark, a novel benchmark designed to systematically assess the embodied reasoning abilities of VLMs through precise visual grounding. We propose a hierarchical evaluation protocol spanning three stages (S1: referred-object localization, S2: task-driven pointing, and S3: visual trace prediction), with data collected from critical domains for embodied intelligence, including indoor, kitchen, driving, and robotic manipulation scenarios. Extensive experiments with over ten state-of-the-art VLMs reveal several interesting findings. For example, strong general-purpose models such as GPT-4o, while excelling on many benchmarks (e.g., language, perception, and reasoning), underperform compared to some open-source models in precise visual grounding; models such as MoLMO perform well in S1 and S2 but struggle in S3, where requires grounding combined with visual trace planning.

Method: DSGA uses dynamic similarity graph construction with learnable polynomial decay-initialized weight ranking and adaptive local feature aggregation, combined with LoRA for complementary optimization of local and global dependencies.

Result: Achieved 17.31% improvement in Structure-measure and 62.36% gain in adaptive F-measure compared to baseline SAM fine-tuning, with progressive gains as shot count increased from 2 to 10 shots.

Conclusion: The framework demonstrates practical utility for automated agricultural monitoring, achieving accurate pod-counting with adjusted R-squared of 0.8987 under challenging field conditions.

Abstract: Parameter-Efficient Fine-Tuning (PEFT) of foundation models for agricultural computer vision tasks remains challenging due to limited training data and complex field conditions. This study introduces a Dynamic Similarity-based Graph Adaptation (DSGA) module to adapt the Segment Anything Model (SAM) under extreme data constraints for precise foreground and instance segmentation of small dense objects in complex agricultural environments. Through dynamic similarity graph construction with a learnable polynomial decay-initialized weight ranking mechanism and adaptive local feature aggregation, DSGA establishes robust spatial and dynamic similarity representation with only 4.00M trainable parameters, which is 4.26% of the original SAM. Integrating this graph-based feature adaptation with Low-Rank Adaptation (LoRA) creates a complementary optimization framework that effectively captures both local and global dependencies in image embeddings while preserving model stability and parameter efficiency. Experimental results on a challenging chickpea pod dataset demonstrated that DSGA with LoRA achieved superior performance across multiple metrics evaluated under 2, 4, 8 and 10 shots, with progressive performance gains as shot count increased. Quantitative metrics showed a 17.31% improvement in Structure-measure and a 62.36% gain in adaptive F-measure compared to the baseline SAM fine-tuning. Comprehensive ablation studies and visualization analyses through Grad-CAM and t-SNE validated the framework’s effectiveness in feature discrimination. The proposed adaptation demonstrated practical utility for automated agricultural monitoring applications, achieving accurate pod-counting with an adjusted R-squared of 0.8987 for images with 10 to 120 pods under challenging field conditions.

Method: Logo-VGR reformulates logo recognition as a comparison-based task where models match product images with candidate logos. It introduces Logo Perception Grounding for domain knowledge injection and Logo-Guided Visual Grounded Reasoning to enhance multimodal reasoning capabilities.

Result: Logo-VGR outperforms strong baselines by nearly 10 points in out-of-distribution (OOD) settings, demonstrating superior generalization performance compared to existing methods.

Conclusion: The proposed Logo-VGR framework effectively addresses the limitations of traditional logo recognition by enabling robust generalization to unseen brands through domain-specific multimodal reasoning, making it practical for real-world product moderation applications.

Abstract: Recent advances in multimodal large language models (MLLMs) have been primarily evaluated on general-purpose benchmarks, while their applications in domain-specific scenarios, such as intelligent product moderation, remain underexplored. To address this gap, we introduce an open-world logo recognition benchmark, a core challenge in product moderation. Unlike traditional logo recognition methods that rely on memorizing representations of tens of thousands of brands-an impractical approach in real-world settings-our proposed method, Logo-VGR, enables generalization to large-scale brand recognition with supervision from only a small subset of brands. Specifically, we reformulate logo recognition as a comparison-based task, requiring the model to match product images with candidate logos rather than directly generating brand labels. We further observe that existing models tend to overfit by memorizing brand distributions instead of learning robust multimodal reasoning, which results in poor performance on unseen brands. To overcome this limitation, Logo-VGR introduces a new paradigm of domain-specific multimodal reasoning: Logo Perception Grounding injects domain knowledge, and Logo-Guided Visual Grounded Reasoning enhances the model’s reasoning capability. Experimental results show that Logo-VGR outperforms strong baselines by nearly 10 points in OOD settings, demonstrating superior generalization.

Method: Organized an open machine learning competition with 5 million plant observations, using high-resolution remote sensing imagery, land cover, elevation data, and coarse-resolution climate, soil, and human footprint variables for multi-label classification.

Result: Evaluated models’ ability to predict species composition in standardized survey plots, identifying biases in methods trained on single positive labels and developing effective strategies combining single and multi-label training data.

Conclusion: The competition highlighted challenges with single positive label training for multi-label evaluation and demonstrated successful learning strategies that integrate both single and multi-label data during training.

Abstract: Understanding the spatio-temporal distribution of species is a cornerstone of ecology and conservation. By pairing species observations with geographic and environmental predictors, researchers can model the relationship between an environment and the species which may be found there. To advance the state-of-the-art in this area with deep learning models and remote sensing data, we organized an open machine learning challenge called GeoLifeCLEF 2023. The training dataset comprised 5 million plant species observations (single positive label per sample) distributed across Europe and covering most of its flora, high-resolution rasters: remote sensing imagery, land cover, elevation, in addition to coarse-resolution data: climate, soil and human footprint variables. In this multi-label classification task, we evaluated models ability to predict the species composition in 22 thousand small plots based on standardized surveys. This paper presents an overview of the competition, synthesizes the approaches used by the participating teams, and analyzes the main results. In particular, we highlight the biases faced by the methods fitted to single positive labels when it comes to the multi-label evaluation, and the new and effective learning strategy combining single and multi-label data in training.

Method: Proposed VELA metric within a novel LLM-Hybrid-as-a-Judge framework, and created LongCap-Arena benchmark with 7,805 images, reference captions, candidate captions, and 32,246 human judgments from three perspectives.

Result: VELA outperformed existing metrics and achieved superhuman performance on LongCap-Arena.

Conclusion: VELA provides an effective solution for automatic evaluation of long image captions, addressing limitations of existing methods through a novel hybrid framework and comprehensive benchmark.

Abstract: In this study, we focus on the automatic evaluation of long and detailed image captions generated by multimodal Large Language Models (MLLMs). Most existing automatic evaluation metrics for image captioning are primarily designed for short captions and are not suitable for evaluating long captions. Moreover, recent LLM-as-a-Judge approaches suffer from slow inference due to their reliance on autoregressive inference and early fusion of visual information. To address these limitations, we propose VELA, an automatic evaluation metric for long captions developed within a novel LLM-Hybrid-as-a-Judge framework. Furthermore, we propose LongCap-Arena, a benchmark specifically designed for evaluating metrics for long captions. This benchmark comprises 7,805 images, the corresponding human-provided long reference captions and long candidate captions, and 32,246 human judgments from three distinct perspectives: Descriptiveness, Relevance, and Fluency. We demonstrated that VELA outperformed existing metrics and achieved superhuman performance on LongCap-Arena.

Method: Formulates editing as trajectory optimal control where diffusion reverse process is treated as controllable trajectory, with adjoint states iteratively updated to steer editing.

Result: Significantly outperforms existing inversion-based training-free guidance baselines across distinct editing tasks, achieving superior balance between reward maximization and source image fidelity.

Conclusion: The proposed framework enables effective reward-guided image editing without training, avoiding reward hacking while maintaining source image content.

Abstract: Recent advancements in diffusion and flow-matching models have demonstrated remarkable capabilities in high-fidelity image synthesis. A prominent line of research involves reward-guided guidance, which steers the generation process during inference to align with specific objectives. However, leveraging this reward-guided approach to the task of image editing, which requires preserving the semantic content of the source image while enhancing a target reward, is largely unexplored. In this work, we introduce a novel framework for training-free, reward-guided image editing. We formulate the editing process as a trajectory optimal control problem where the reverse process of a diffusion model is treated as a controllable trajectory originating from the source image, and the adjoint states are iteratively updated to steer the editing process. Through extensive experiments across distinct editing tasks, we demonstrate that our approach significantly outperforms existing inversion-based training-free guidance baselines, achieving a superior balance between reward maximization and fidelity to the source image without reward hacking.

Method: Proposes Vision-Anchored Policy Optimization (VAPO), a method that explicitly steers the reasoning process toward visually grounded trajectories to prevent visual forgetting.

Result: VAPO-Thinker-7B significantly strengthens visual information reliance and achieves new state-of-the-art results across multiple established benchmarks.

Conclusion: VAPO effectively addresses visual forgetting in multimodal reasoning while maintaining strong performance, demonstrating the importance of anchoring reasoning processes to visual inputs.

Abstract: Reasoning has emerged as a pivotal capability in Large Language Models (LLMs). Through Reinforcement Learning (RL), typically Group Relative Policy Optimization (GRPO), these models are able to solve complex tasks such as mathematics and code generation. Building on these advances, recent research has sought to extend reasoning to Vision-Language Models (VLMs), yielding promising results across diverse visual tasks. Despite this progress, our study uncovers the dual nature of multimodal reasoning: while it substantially enhances logical inference and facilitates performance on challenging problems, it may gradually impair perceptual grounding, leading to recognition failures on otherwise basic visual questions. Through further analysis, we attribute this phenomenon to visual forgetting, wherein prolonged reasoning causes the model to increasingly disregard visual input. To address this, we propose Vision-Anchored Policy Optimization (VAPO), a simple yet effective method that explicitly steers the reasoning process toward visually grounded trajectories. Our result model, VAPO-Thinker-7B, significantly strengthens the model’s reliance on visual information and achieves new state-of-the-art results on a wide range of established benchmarks. Project page: https://xytian1008.github.io/VAPO/

Method: Created MuSLR benchmark with 1,093 instances across 7 domains, including 35 atomic symbolic logic and 976 logical combinations with reasoning depths from 2 to 9. Proposed LogiCAM framework that applies formal logical rules to multimodal inputs to enhance Chain-of-Thought reasoning.

Result: 7 state-of-the-art VLMs all struggle with multimodal symbolic reasoning, with GPT-4.1 achieving only 46.8% accuracy. LogiCAM boosts GPT-4.1’s performance by 14.13%, with larger gains on complex logics like first-order logic. Error analysis shows 70% of failures stem from logical misalignment between modalities.

Conclusion: Current VLMs have significant limitations in multimodal symbolic logical reasoning. The LogiCAM framework effectively addresses these limitations by incorporating formal logical rules, providing a promising direction for improving reasoning capabilities in high-stakes applications.

Abstract: Multimodal symbolic logical reasoning, which aims to deduce new facts from multimodal input via formal logic, is critical in high-stakes applications such as autonomous driving and medical diagnosis, as its rigorous, deterministic reasoning helps prevent serious consequences. To evaluate such capabilities of current state-of-the-art vision language models (VLMs), we introduce the first benchmark MuSLR for multimodal symbolic logical reasoning grounded in formal logical rules. MuSLR comprises 1,093 instances across 7 domains, including 35 atomic symbolic logic and 976 logical combinations, with reasoning depths ranging from 2 to 9. We evaluate 7 state-of-the-art VLMs on MuSLR and find that they all struggle with multimodal symbolic reasoning, with the best model, GPT-4.1, achieving only 46.8%. Thus, we propose LogiCAM, a modular framework that applies formal logical rules to multimodal inputs, boosting GPT-4.1’s Chain-of-Thought performance by 14.13%, and delivering even larger gains on complex logics such as first-order logic. We also conduct a comprehensive error analysis, showing that around 70% of failures stem from logical misalignment between modalities, offering key insights to guide future improvements. All data and code are publicly available at https://llm-symbol.github.io/MuSLR.

Method: Proposed Patch-Exclusive Anomaly Detection (PatchEAD) framework with visual prompting techniques including alignment module and foreground masking. It’s training-free and compatible with various foundation models.

Result: Superior few-shot and batch zero-shot performance compared to prior work, despite not using textual features. The study also provides guidance on how backbone structure and pretrained characteristics affect patch-similarity robustness.

Conclusion: A well-unified patch-only framework enables quick, calibration-light deployment for visual inspection without needing carefully engineered textual prompts, making it practical for real-world industrial applications.

Abstract: Industrial anomaly detection is increasingly relying on foundation models, aiming for strong out-of-distribution generalization and rapid adaptation in real-world deployments. Notably, past studies have primarily focused on textual prompt tuning, leaving the intrinsic visual counterpart fragmented into processing steps specific to each foundation model. We aim to address this limitation by proposing a unified patch-focused framework, Patch-Exclusive Anomaly Detection (PatchEAD), enabling training-free anomaly detection that is compatible with diverse foundation models. The framework constructs visual prompting techniques, including an alignment module and foreground masking. Our experiments show superior few-shot and batch zero-shot performance compared to prior work, despite the absence of textual features. Our study further examines how backbone structure and pretrained characteristics affect patch-similarity robustness, providing actionable guidance for selecting and configuring foundation models for real-world visual inspection. These results confirm that a well-unified patch-only framework can enable quick, calibration-light deployment without the need for carefully engineered textual prompts.

Method: Uses multiple camera inputs with automatic geometric transformation computation between LiDAR and cameras, integrated low-light image enhancement module, and color correction for uniform camera feeds before fusion.

Result: Achieved real-time performance and reliable colorization even under very low illumination, successfully recovering scene details that would otherwise remain undetectable.

Conclusion: The proposed system provides a robust, hardware-agnostic solution for generating colorized point clouds with complete 360° coverage, demonstrating significant improvements in low-light performance without requiring specialized calibration targets.

Abstract: In recent years, the fusion of camera data with LiDAR measurements has emerged as a powerful approach to enhance spatial understanding. This study introduces a novel, hardware-agnostic methodology that generates colourised point clouds from mechanical LiDAR using multiple camera inputs, providing complete 360-degree coverage. The primary innovation lies in its robustness under low-light conditions, achieved through the integration of a low-light image enhancement module within the fusion pipeline. The system requires initial calibration to determine intrinsic camera parameters, followed by automatic computation of the geometric transformation between the LiDAR and cameras, removing the need for specialised calibration targets and streamlining the setup. The data processing framework uses colour correction to ensure uniformity across camera feeds before fusion. The algorithm was tested using a Velodyne Puck Hi-Res LiDAR and a four-camera configuration. The optimised software achieved real-time performance and reliable colourisation even under very low illumination, successfully recovering scene details that would otherwise remain undetectable.

Method: Uses LLMs to generate entity-level and slide-level prompts, entity-guided cross-attention for entity-level features, cross-scale entity graph learning to capture semantic correlations, and combines entity-level and slide-level predictions.

Result: Results on three cancer cohorts confirm the effectiveness in addressing few-shot pathology diagnosis tasks.

Conclusion: MAPLE successfully addresses the limitations of existing methods by capturing fine-grained entity-level variations while maintaining slide-level context for improved cancer diagnosis.

Abstract: Prompt learning has emerged as a promising paradigm for adapting pre-trained vision-language models (VLMs) to few-shot whole slide image (WSI) classification by aligning visual features with textual representations, thereby reducing annotation cost and enhancing model generalization. Nevertheless, existing methods typically rely on slide-level prompts and fail to capture the subtype-specific phenotypic variations of histological entities (\emph{e.g.,} nuclei, glands) that are critical for cancer diagnosis. To address this gap, we propose Multi-scale Attribute-enhanced Prompt Learning (\textbf{MAPLE}), a hierarchical framework for few-shot WSI classification that jointly integrates multi-scale visual semantics and performs prediction at both the entity and slide levels. Specifically, we first leverage large language models (LLMs) to generate entity-level prompts that can help identify multi-scale histological entities and their phenotypic attributes, as well as slide-level prompts to capture global visual descriptions. Then, an entity-guided cross-attention module is proposed to generate entity-level features, followed by aligning with their corresponding subtype-specific attributes for fine-grained entity-level prediction. To enrich entity representations, we further develop a cross-scale entity graph learning module that can update these representations by capturing their semantic correlations within and across scales. The refined representations are then aggregated into a slide-level representation and aligned with the corresponding prompts for slide-level prediction. Finally, we combine both entity-level and slide-level outputs to produce the final prediction results. Results on three cancer cohorts confirm the effectiveness of our approach in addressing few-shot pathology diagnosis tasks.

Method: Created a 31k chain-of-thought dataset with diverse tool calls and edited images, then designed a model that performs interleaved image-text reasoning by generating visual thoughts directly in visual embedding space without external tools.

Result: Extensive experiments on multimodal reasoning benchmarks demonstrate strong performance, validating both the dataset utility and model effectiveness.

Conclusion: DeepSketcher enables tool-free and more flexible “thinking with images” paradigm, advancing multimodal reasoning capabilities in VLMs.

Abstract: The “thinking with images” paradigm represents a pivotal shift in the reasoning of Vision Language Models (VLMs), moving from text-dominant chain-of-thought to image-interactive reasoning. By invoking visual tools or generating intermediate visual representations, VLMs can iteratively attend to fine-grained regions, enabling deeper image understanding and more faithful multimodal reasoning. As an emerging paradigm, however, it still leaves substantial room for exploration in data construction accuracy, structural design, and broader application scenarios, which offer rich opportunities for advancing multimodal reasoning. To further advance this line of work, we present DeepSketcher, a comprehensive suite comprising both an image-text interleaved dataset and a self-contained model. The dataset contains 31k chain-of-thought (CoT) reasoning trajectories with diverse tool calls and resulting edited images, covering a wide range of data types and manipulation instructions with high annotation accuracy. Building on this resource, we design a model that performs interleaved image-text reasoning and natively generates “visual thoughts” by operating directly in the visual embedding space, rather than invoking external tools and repeatedly re-encoding generated images. This design enables tool-free and more flexible “thinking with images”. Extensive experiments on multimodal reasoning benchmarks demonstrate strong performance, validating both the utility of the dataset and the effectiveness of the model design.

Method: Uses prompt-conditioned hierarchical mixture-of-experts (MoE) architecture with modality-level and token-level projection experts to fuse multiple interrelated 3D modalities. Integrates synthetic VQA protocol that generates medically relevant questions from segmentation annotations.

Result: Outperforms strong medical VLM baselines by 5.3% on average across multiple mpMRI datasets. Creates the first clinically validated VQA dataset for 3D brain mpMRI.

Conclusion: The study demonstrates medical utility through strong empirical results, with ablations showing importance of modality-level and token-level experts and prompt-conditioned routing.

Abstract: We introduce mpLLM, a prompt-conditioned hierarchical mixture-of-experts (MoE) architecture for visual question answering over multi-parametric 3D brain MRI (mpMRI). mpLLM routes across modality-level and token-level projection experts to fuse multiple interrelated 3D modalities, enabling efficient training without image–report pretraining. To address limited image-text paired supervision, mpLLM integrates a synthetic visual question answering (VQA) protocol that generates medically relevant VQA from segmentation annotations, and we collaborate with medical experts for clinical validation. mpLLM outperforms strong medical VLM baselines by 5.3% on average across multiple mpMRI datasets. Our study features three main contributions: (1) the first clinically validated VQA dataset for 3D brain mpMRI, (2) a novel multimodal LLM that handles multiple interrelated 3D modalities, and (3) strong empirical results that demonstrate the medical utility of our methodology. Ablations highlight the importance of modality-level and token-level experts and prompt-conditioned routing. We have included our source code in the supplementary materials and will release our dataset upon publication.

Method: Developed systematic definition of MMT dialogue safety; created MMDS dataset with fine-grained safety ratings; built automated multimodal multi-turn red-teaming framework using Monte Carlo Tree Search (MCTS); proposed LLaVAShield for joint risk detection and assessment.

Result: MMDS contains 4,484 annotated multimodal dialogue samples with safety ratings, policy dimension labels, and evidence-based rationales. LLaVAShield consistently outperforms strong baselines on MMT content moderation tasks and under dynamic policy configurations, establishing new state-of-the-art results.

Conclusion: The work addresses critical safety gaps in multimodal multi-turn dialogues, provides comprehensive dataset and tools for future research, and demonstrates superior performance in content moderation for interactive VLMs.

Abstract: As Vision-Language Models (VLMs) move into interactive, multi-turn use, new safety risks arise that single-turn or single-modality moderation misses. In Multimodal Multi-Turn (MMT) dialogues, malicious intent can be spread across turns and images, while context-sensitive replies may still advance harmful content. To address this challenge, we present the first systematic definition and study of MMT dialogue safety. Building on this formulation, we introduce the Multimodal Multi-turn Dialogue Safety (MMDS) dataset. We further develop an automated multimodal multi-turn red-teaming framework based on Monte Carlo Tree Search (MCTS) to generate unsafe multimodal multi-turn dialogues for MMDS. MMDS contains 4,484 annotated multimodal dialogue samples with fine-grained safety ratings, policy dimension labels, and evidence-based rationales for both users and assistants. Leveraging MMDS, we present LLaVAShield, a powerful tool that jointly detects and assesses risk in user inputs and assistant responses. Across comprehensive experiments, LLaVAShield consistently outperforms strong baselines on MMT content moderation tasks and under dynamic policy configurations, establishing new state-of-the-art results. We will publicly release the dataset and model to support future research.

Method: A plug-and-play framework with Hybrid Fine-grained Region Encoder (dual vision encoder) that generates region tokens rich in semantic and spatial detail, plus token-based referencing system for LLMs to reason about specific visual regions. Uses two-stage training strategy.

Result: Achieves state-of-the-art performance across diverse benchmarks, demonstrating exceptional capabilities in object grounding, region generation, and visual region reasoning.

Conclusion: VLM-FO1 establishes an effective paradigm for building perception-aware VLMs, bridging the gap between high-level reasoning and fine-grained visual grounding while maintaining general visual understanding capabilities.

Abstract: Vision-Language Models (VLMs) excel at high-level scene understanding but falter on fine-grained perception tasks requiring precise localization. This failure stems from a fundamental mismatch, as generating exact numerical coordinates is a challenging task for language-centric architectures. In this paper, we introduce VLM-FO1, a novel framework that overcomes this limitation by reframing object-centric perception from a brittle coordinate generation problem into a robust feature retrieval task. Our method operates as a plug-and-play module that integrates with any pre-trained VLM. It leverages a Hybrid Fine-grained Region Encoder (HFRE), featuring a dual vision encoder, to generate powerful region tokens rich in both semantic and spatial detail. A token-based referencing system then enables the LLM to seamlessly reason about and ground language in these specific visual regions. Experiments show that VLM-FO1 achieves state-of-the-art performance across a diverse suite of benchmarks, demonstrating exceptional capabilities in object grounding, region generational understanding, and visual region reasoning. Crucially, our two-stage training strategy ensures that these perception gains are achieved without compromising the base model’s general visual understanding capabilities. VLM-FO1 establishes an effective and flexible paradigm for building perception-aware VLMs, bridging the gap between high-level reasoning and fine-grained visual grounding.

Method: Evaluated 36 state-of-the-art vision models (supervised, self-supervised, contrastive learning) trained on datasets from 1.2M to 22B images under six black-box attack categories including random perturbations, geometric masks, object manipulations, and style shifts.

Result: Tenfold increase in data reduces attack success rate by ~3.2%, while tenfold increase in model size reduces ASR by ~13.4%. Self-supervised models on curated datasets (like DINOv2) outperform models on larger but less curated datasets. Adversarial fine-tuning improves generalization across structural but not color variations.

Conclusion: While scaling improves robustness, data quality, architecture, and training objectives play a more decisive role than raw scale in achieving broad-spectrum adversarial resilience, with persistent gaps between human and machine vision.

Abstract: Deep neural networks remain vulnerable to adversarial examples despite advances in architectures and training paradigms. We investigate how training data characteristics affect adversarial robustness across 36 state-of-the-art vision models spanning supervised, self-supervised, and contrastive learning approaches, trained on datasets from 1.2M to 22B images. Models were evaluated under six black-box attack categories: random perturbations, two types of geometric masks, COCO object manipulations, ImageNet-C corruptions, and ImageNet-R style shifts. Robustness follows a logarithmic scaling law with both data volume and model size: a tenfold increase in data reduces attack success rate (ASR) on average by ~3.2%, whereas a tenfold increase in model size reduces ASR on average by ~13.4%. Notably, some self-supervised models trained on curated datasets, such as DINOv2, outperform others trained on much larger but less curated datasets, challenging the assumption that scale alone drives robustness. Adversarial fine-tuning of ResNet50s improves generalization across structural variations but not across color distributions. Human evaluation reveals persistent gaps between human and machine vision. These results show that while scaling improves robustness, data quality, architecture, and training objectives play a more decisive role than raw scale in achieving broad-spectrum adversarial resilience.

Method: Uses a Mixture-of-Experts-driven feature decompression mechanism with ‘general to specific’ paradigm: compresses multi-modal inputs into general features, then decompresses via sparsely-gated cross MoE that dynamically selects expert pathways based on modality and class. Includes grouped dynamic filtering and MoE-in-MoE structure for efficiency.

Result: Achieves state-of-the-art performance on 9 anomaly detection datasets spanning 3 fields, 12 modalities, and 66 classes, while reducing parameter usage by 75% and maintaining sparse activation with fast inference.

Conclusion: UniMMAD provides a unified and efficient solution for multi-modal multi-class anomaly detection, overcoming limitations of fragmented approaches and enabling adaptive reconstruction across diverse domains.

Abstract: Existing anomaly detection (AD) methods often treat the modality and class as independent factors. Although this paradigm has enriched the development of AD research branches and produced many specialized models, it has also led to fragmented solutions and excessive memory overhead. Moreover, reconstruction-based multi-class approaches typically rely on shared decoding paths, which struggle to handle large variations across domains, resulting in distorted normality boundaries, domain interference, and high false alarm rates. To address these limitations, we propose UniMMAD, a unified framework for multi-modal and multi-class anomaly detection. At the core of UniMMAD is a Mixture-of-Experts (MoE)-driven feature decompression mechanism, which enables adaptive and disentangled reconstruction tailored to specific domains. This process is guided by a ``general to specific’’ paradigm. In the encoding stage, multi-modal inputs of varying combinations are compressed into compact, general-purpose features. The encoder incorporates a feature compression module to suppress latent anomalies, encourage cross-modal interaction, and avoid shortcut learning. In the decoding stage, the general features are decompressed into modality-specific and class-specific forms via a sparsely-gated cross MoE, which dynamically selects expert pathways based on input modality and class. To further improve efficiency, we design a grouped dynamic filtering mechanism and a MoE-in-MoE structure, reducing parameter usage by 75% while maintaining sparse activation and fast inference. UniMMAD achieves state-of-the-art performance on 9 anomaly detection datasets, spanning 3 fields, 12 modalities, and 66 classes. The source code will be available at https://github.com/yuanzhao-CVLAB/UniMMAD.

Method: Introduces corrective sampling that steers away from regions where joint prompt behavior overlaps too strongly with any single concept, aiming for ‘pure’ joint modes where all concepts coexist with balanced visual presence. Also characterizes favorable weight regimes for multi-concept guidance.

Result: Experiments show improvements in concept coverage, balance and robustness on diverse multi-concept prompts, with fewer dropped or distorted concepts compared to standard baselines and prior compositional methods.

Conclusion: Lightweight corrective guidance can substantially mitigate brittle semantic alignment behavior in modern diffusion systems without requiring model tuning.

Abstract: We propose to improve multi-concept prompt fidelity in text-to-image diffusion models. We begin with common failure cases-prompts like “a cat and a dog” that sometimes yields images where one concept is missing, faint, or colliding awkwardly with another. We hypothesize that this happens when the diffusion model drifts into mixed modes that over-emphasize a single concept it learned strongly during training. Instead of re-training, we introduce a corrective sampling strategy that steers away from regions where the joint prompt behavior overlaps too strongly with any single concept in the prompt. The goal is to steer towards “pure” joint modes where all concepts can coexist with balanced visual presence. We further show that existing multi-concept guidance schemes can operate in unstable weight regimes that amplify imbalance; we characterize favorable regions and adapt sampling to remain within them. Our approach, CO3, is plug-and-play, requires no model tuning, and complements standard classifier-free guidance. Experiments on diverse multi-concept prompts indicate improvements in concept coverage, balance and robustness, with fewer dropped or distorted concepts compared to standard baselines and prior compositional methods. Results suggest that lightweight corrective guidance can substantially mitigate brittle semantic alignment behavior in modern diffusion systems.

Method: Proposes Self-Supervised Anatomical Consistency Learning (SS-ACL) that constructs a hierarchical anatomical graph based on human anatomy structure, recursively reconstructs fine-grained anatomical regions for spatial alignment, and uses region-level contrastive learning for semantic alignment.

Result: Outperforms state-of-the-art methods by 10% in lexical accuracy and 25% in clinical efficacy, and achieves 8% improvement in zero-shot visual grounding compared to leading visual foundation models.

Conclusion: SS-ACL enables accurate and visually grounded medical report generation without expert annotations, demonstrating strong performance in both report generation and downstream visual tasks.

Abstract: Vision-grounded medical report generation aims to produce clinically accurate descriptions of medical images, anchored in explicit visual evidence to improve interpretability and facilitate integration into clinical workflows. However, existing methods often rely on separately trained detection modules that require extensive expert annotations, introducing high labeling costs and limiting generalizability due to pathology distribution bias across datasets. To address these challenges, we propose Self-Supervised Anatomical Consistency Learning (SS-ACL) – a novel and annotation-free framework that aligns generated reports with corresponding anatomical regions using simple textual prompts. SS-ACL constructs a hierarchical anatomical graph inspired by the invariant top-down inclusion structure of human anatomy, organizing entities by spatial location. It recursively reconstructs fine-grained anatomical regions to enforce intra-sample spatial alignment, inherently guiding attention maps toward visually relevant areas prompted by text. To further enhance inter-sample semantic alignment for abnormality recognition, SS-ACL introduces a region-level contrastive learning based on anatomical consistency. These aligned embeddings serve as priors for report generation, enabling attention maps to provide interpretable visual evidence. Extensive experiments demonstrate that SS-ACL, without relying on expert annotations, (i) generates accurate and visually grounded reports – outperforming state-of-the-art methods by 10% in lexical accuracy and 25% in clinical efficacy, and (ii) achieves competitive performance on various downstream visual tasks, surpassing current leading visual foundation models by 8% in zero-shot visual grounding.

Method: Uses pose estimation network to extract bounding boxes and body parts, employs specialized modules to handle underwater salmon scene challenges, and leverages high-detail body part tracks for welfare indicator calculation.

Result: Outperforms BoostTrack (current state-of-the-art pedestrian tracker) on salmon tracking challenges, and demonstrates suitability for automated welfare monitoring through tail beat analysis.

Conclusion: The proposed framework effectively addresses underwater salmon tracking challenges and enables automated welfare monitoring, with datasets and code made publicly available.

Abstract: Computer Vision (CV)-based continuous, automated and precise salmon welfare monitoring is a key step toward reduced salmon mortality and improved salmon welfare in industrial aquaculture net pens. Available CV methods for determining welfare indicators focus on single indicators and rely on object detectors and trackers from other application areas to aid their welfare indicator calculation algorithm. This comes with a high resource demand for real-world applications, since each indicator must be calculated separately. In addition, the methods are vulnerable to difficulties in underwater salmon scenes, such as object occlusion, similar object appearance, and similar object motion. To address these challenges, we propose a flexible tracking framework that uses a pose estimation network to extract bounding boxes around salmon and their corresponding body parts, and exploits information about the body parts, through specialized modules, to tackle challenges specific to underwater salmon scenes. Subsequently, the high-detail body part tracks are employed to calculate welfare indicators. We construct two novel datasets assessing two salmon tracking challenges: salmon ID transfers in crowded scenes and salmon ID switches during turning. Our method outperforms the current state-of-the-art pedestrian tracker, BoostTrack, for both salmon tracking challenges. Additionally, we create a dataset for calculating salmon tail beat wavelength, demonstrating that our body part tracking method is well-suited for automated welfare monitoring based on tail beat analysis. Datasets and code are available at https://github.com/espenbh/BoostCompTrack.

Method: Developed PinPoint3D framework with a new 3D data synthesis pipeline to create large-scale scene-level dataset with dense part annotations, enabling fine-grained multi-granularity 3D segmentation from few user point clicks.

Result: Achieved average IoU of 55.8% on each object part under first-click settings, surpassing 71.3% IoU with few additional clicks. Outperformed state-of-the-art baselines by up to 16% improvement in IoU and precision on challenging sparse point clouds.

Conclusion: PinPoint3D represents a significant advancement towards more nuanced and precise machine perception and interaction in complex 3D environments, overcoming critical bottlenecks in fine-grained 3D part segmentation.

Abstract: Fine-grained 3D part segmentation is crucial for enabling embodied AI systems to perform complex manipulation tasks, such as interacting with specific functional components of an object. However, existing interactive segmentation methods are largely confined to coarse, instance-level targets, while non-interactive approaches struggle with sparse, real-world scans and suffer from a severe lack of annotated data. To address these limitations, we introduce PinPoint3D, a novel interactive framework for fine-grained, multi-granularity 3D segmentation, capable of generating precise part-level masks from only a few user point clicks. A key component of our work is a new 3D data synthesis pipeline that we developed to create a large-scale, scene-level dataset with dense part annotations, overcoming a critical bottleneck that has hindered progress in this field. Through comprehensive experiments and user studies, we demonstrate that our method significantly outperforms existing approaches, achieving an average IoU of around 55.8% on each object part under first-click settings and surpassing 71.3% IoU with only a few additional clicks. Compared to current state-of-the-art baselines, PinPoint3D yields up to a 16% improvement in IoU and precision, highlighting its effectiveness on challenging, sparse point clouds with high efficiency. Our work represents a significant step towards more nuanced and precise machine perception and interaction in complex 3D environments.

Method: Leverages jackknife conformal prediction to construct reliable alternative sets and covering design-based sampling for comprehensive and uniform coverage of pairwise preferences.

Result: Extensive experiments show RH-Partial2Global achieves excellent performance and outperforms Partial2Global across diverse visual tasks.

Conclusion: The proposed method provides reliable and holistic selection of in-context examples in VICL, addressing limitations of existing approaches.

Abstract: Visual In-Context Learning (VICL) has emerged as a prominent approach for adapting visual foundation models to novel tasks, by effectively exploiting contextual information embedded in in-context examples, which can be formulated as a global ranking problem of potential candidates. Current VICL methods, such as Partial2Global and VPR, are grounded in the similarity-priority assumption that images more visually similar to a query image serve as better in-context examples. This foundational assumption, while intuitive, lacks sufficient justification for its efficacy in selecting optimal in-context examples. Furthermore, Partial2Global constructs its global ranking from a series of randomly sampled pairwise preference predictions. Such a reliance on random sampling can lead to incomplete coverage and redundant samplings of comparisons, thus further adversely impacting the final global ranking. To address these issues, this paper introduces an enhanced variant of Partial2Global designed for reliable and holistic selection of in-context examples in VICL. Our proposed method, dubbed RH-Partial2Global, leverages a jackknife conformal prediction-guided strategy to construct reliable alternative sets and a covering design-based sampling approach to ensure comprehensive and uniform coverage of pairwise preferences. Extensive experiments demonstrate that RH-Partial2Global achieves excellent performance and outperforms Partial2Global across diverse visual tasks.

Method: Integrates linear spatio-temporal aggregation module into video-based diffusion models using bidirectional weighted key-value recurrence mechanism of RWKV transformer to capture global dependencies while preserving temporal coherence.

Result: Achieves up to 3.7x speedup and 60% lower memory usage compared to state-of-the-art diffusion-based video editing methods, while maintaining competitive performance in frame consistency and text alignment.

Conclusion: The proposed method effectively reduces computational complexity without sacrificing quality, with performance gap becoming more significant for longer videos compared to self-attention architectures.

Abstract: In light of recent progress in video editing, deep learning models focusing on both spatial and temporal dependencies have emerged as the primary method. However, these models suffer from the quadratic computational complexity of traditional attention mechanisms, making them difficult to adapt to long-duration and high-resolution videos. This limitation restricts their applicability in practical contexts such as real-time video processing. To tackle this challenge, we introduce a method to reduce both time and space complexity of these systems by proposing VRWKV-Editor, a novel video editing model that integrates a linear spatio-temporal aggregation module into video-based diffusion models. VRWKV-Editor leverages bidirectional weighted key-value recurrence mechanism of the RWKV transformer to capture global dependencies while preserving temporal coherence, achieving linear complexity without sacrificing quality. Extensive experiments demonstrate that the proposed method achieves up to 3.7x speedup and 60% lower memory usage compared to state-of-the-art diffusion-based video editing methods, while maintaining competitive performance in frame consistency and text alignment. Furthermore, a comparative analysis we conducted on videos with different sequence lengths confirms that the gap in editing speed between our approach and architectures with self-attention becomes more significant with long videos.

Method: Proposes NS-iHOS task and WISH model - an end-to-end approach that learns human-object interaction detection by distilling knowledge from natural language narrations to segment in-hand objects without using narrations at test time.

Result: WISH surpasses all baselines on EPIC-Kitchens and Ego4D datasets, recovering more than 50% of the performance of fully supervised methods without using fine-grained pixel-wise annotations.

Conclusion: Natural language narrations provide effective weak supervision for learning in-hand object segmentation, enabling significant performance gains without costly manual annotations.

Abstract: Pixel-level recognition of objects manipulated by the user from egocentric images enables key applications spanning assistive technologies, industrial safety, and activity monitoring. However, progress in this area is currently hindered by the scarcity of annotated datasets, as existing approaches rely on costly manual labels. In this paper, we propose to learn human-object interaction detection leveraging narrations – natural language descriptions of the actions performed by the camera wearer which contain clues about manipulated objects (e.g., “I am pouring vegetables from the chopping board to the pan”). Narrations provide a form of weak supervision that is cheap to acquire and readily available in state-of-the-art egocentric datasets. We introduce Narration-Supervised in-Hand Object Segmentation (NS-iHOS), a novel task where models have to learn to segment in-hand objects by learning from natural-language narrations. Narrations are then not employed at inference time. We showcase the potential of the task by proposing Weakly-Supervised In-hand Object Segmentation from Human Narrations (WISH), an end-to-end model distilling knowledge from narrations to learn plausible hand-object associations and enable in-hand object segmentation without using narrations at test time. We benchmark WISH against different baselines based on open-vocabulary object detectors and vision-language models, showing the superiority of its design. Experiments on EPIC-Kitchens and Ego4D show that WISH surpasses all baselines, recovering more than 50% of the performance of fully supervised methods, without employing fine-grained pixel-wise annotations.

Method: AgenticIQA decomposes IQA into four subtasks (distortion detection, distortion analysis, tool selection, tool execution) coordinated by planner, executor, and summarizer agents. It integrates VLMs with traditional IQA tools dynamically based on queries.

Result: Extensive experiments show AgenticIQA consistently surpasses strong baselines in both scoring accuracy and explanatory alignment across diverse IQA datasets. The framework also includes AgenticIQA-200K dataset and AgenticIQA-Eval benchmark.

Conclusion: AgenticIQA provides a more adaptive, interpretable, and accurate approach to image quality assessment by leveraging agentic coordination between VLMs and traditional tools, addressing limitations of conventional fixed-model approaches.

Abstract: Image quality assessment (IQA) is inherently complex, as it reflects both the quantification and interpretation of perceptual quality rooted in the human visual system. Conventional approaches typically rely on fixed models to output scalar scores, limiting their adaptability to diverse distortions, user-specific queries, and interpretability needs. Furthermore, scoring and interpretation are often treated as independent processes, despite their interdependence: interpretation identifies perceptual degradations, while scoring abstracts them into a compact metric. To address these limitations, we propose AgenticIQA, a modular agentic framework that integrates vision-language models (VLMs) with traditional IQA tools in a dynamic, query-aware manner. AgenticIQA decomposes IQA into four subtasks – distortion detection, distortion analysis, tool selection, and tool execution – coordinated by a planner, executor, and summarizer. The planner formulates task-specific strategies, the executor collects perceptual evidence via tool invocation, and the summarizer integrates this evidence to produce accurate scores with human-aligned explanations. To support training and evaluation, we introduce AgenticIQA-200K, a large-scale instruction dataset tailored for IQA agents, and AgenticIQA-Eval, the first benchmark for assessing the planning, execution, and summarization capabilities of VLM-based IQA agents. Extensive experiments across diverse IQA datasets demonstrate that AgenticIQA consistently surpasses strong baselines in both scoring accuracy and explanatory alignment.

Method: Unified architecture that lifts 2D features into canonical 3D volumetric space, uses Heterogeneous Spatial Fusion to process distortion-aware volumetric features across overlapping/non-overlapping regions, and reformulates voxel fusion into novel 3D Gaussian representation with learnable latent Gaussian spheres that adapt to local textures.

Result: PFDepth achieves state-of-the-art performance on KITTI-360 and RealHet datasets, outperforming current mainstream depth networks.

Conclusion: This represents the first systematic study of heterogeneous pinhole-fisheye depth estimation, offering both technical novelty through the 3D Gaussian representation and valuable empirical insights for multi-camera depth estimation systems.

Abstract: In this paper, we present the first pinhole-fisheye framework for heterogeneous multi-view depth estimation, PFDepth. Our key insight is to exploit the complementary characteristics of pinhole and fisheye imagery (undistorted vs. distorted, small vs. large FOV, far vs. near field) for joint optimization. PFDepth employs a unified architecture capable of processing arbitrary combinations of pinhole and fisheye cameras with varied intrinsics and extrinsics. Within PFDepth, we first explicitly lift 2D features from each heterogeneous view into a canonical 3D volumetric space. Then, a core module termed Heterogeneous Spatial Fusion is designed to process and fuse distortion-aware volumetric features across overlapping and non-overlapping regions. Additionally, we subtly reformulate the conventional voxel fusion into a novel 3D Gaussian representation, in which learnable latent Gaussian spheres dynamically adapt to local image textures for finer 3D aggregation. Finally, fused volume features are rendered into multi-view depth maps. Through extensive experiments, we demonstrate that PFDepth sets a state-of-the-art performance on KITTI-360 and RealHet datasets over current mainstream depth networks. To the best of our knowledge, this is the first systematic study of heterogeneous pinhole-fisheye depth estimation, offering both technical novelty and valuable empirical insights.

Method: Proposed fourth-order nonlinear PDE model integrates diffusion (guided by Laplacian and intensity values) and wave properties, with independent channel processing for color images.

Result: The model outperforms second-order anisotropic diffusion approaches in PSNR, MSSIM, and Speckle Index metrics for both grayscale and color images.

Conclusion: The proposed fourth-order PDE model effectively suppresses multiplicative noise while preserving fine details and textures, producing superior results compared to existing models.

Abstract: Second-order PDE models have been widely used for suppressing multiplicative noise, but they often introduce blocky artifacts in the early stages of denoising. To resolve this, we propose a fourth-order nonlinear PDE model that integrates diffusion and wave properties. The diffusion process, guided by both the Laplacian and intensity values, reduces noise better than gradient-based methods, while the wave part keeps fine details and textures. The effectiveness of the proposed model is evaluated against two second-order anisotropic diffusion approaches using the Peak Signal-to-Noise Ratio (PSNR) and Mean Structural Similarity Index (MSSIM) for images with available ground truth. For SAR images, where a noise-free reference is unavailable, the Speckle Index (SI) is used to measure noise reduction. Additionally, we extend the proposed model to study color images by applying the denoising process independently to each channel, preserving both structure and color consistency. The same quantitative metrics PSNR and MSSIM are used for performance evaluation, ensuring a fair comparison across grayscale and color images. In all the cases, our computed results produce better results compared to existing models in this genre.

Method: SETR uses a two-stage approach: (1) coarse retrieval with intersection-driven strategy to filter out distractors by retaining only overlapping semantics between reference image and relative text, (2) fine-grained re-ranking using a pretrained multimodal LLM with Low-Rank Adaptation to conduct binary semantic relevance judgments.

Result: SETR achieves new state-of-the-art performance on CIRR, Fashion-IQ, and CIRCO datasets, improving Recall@1 on CIRR by up to 15.15 points.

Conclusion: Two-stage reasoning is established as a general paradigm for robust and portable zero-shot composed image retrieval.

Abstract: Zero-shot Composed Image Retrieval (ZS-CIR) aims to retrieve a target image given a reference image and a relative text, without relying on costly triplet annotations. Existing CLIP-based methods face two core challenges: (1) union-based feature fusion indiscriminately aggregates all visual cues, carrying over irrelevant background details that dilute the intended modification, and (2) global cosine similarity from CLIP embeddings lacks the ability to resolve fine-grained semantic relations. To address these issues, we propose SETR (Semantic-enhanced Two-Stage Retrieval). In the coarse retrieval stage, SETR introduces an intersection-driven strategy that retains only the overlapping semantics between the reference image and relative text, thereby filtering out distractors inherent to union-based fusion and producing a cleaner, high-precision candidate set. In the fine-grained re-ranking stage, we adapt a pretrained multimodal LLM with Low-Rank Adaptation to conduct binary semantic relevance judgments (“Yes/No”), which goes beyond CLIP’s global feature matching by explicitly verifying relational and attribute-level consistency. Together, these two stages form a complementary pipeline: coarse retrieval narrows the candidate pool with high recall, while re-ranking ensures precise alignment with nuanced textual modifications. Experiments on CIRR, Fashion-IQ, and CIRCO show that SETR achieves new state-of-the-art performance, improving Recall@1 on CIRR by up to 15.15 points. Our results establish two-stage reasoning as a general paradigm for robust and portable ZS-CIR.

Method: GeoLink enhances RS self-supervised pretraining using multi-granularity learning signals from OSM data guided by cross-modal spatial correlations. It uses image mask-reconstruction for efficient pretraining and generates both unimodal and multimodal fine-grained encodings for downstream tasks.

Result: Extensive experiments show that incorporating OSM data during pretraining enhances RS image encoder performance, while fusing RS and OSM data in downstream tasks improves adaptability to complex geographic scenarios. Spatial correlation is crucial for effective multimodal integration.

Conclusion: The study demonstrates the potential of multimodal synergy in advancing high-level geospatial artificial intelligence, with spatial correlation playing a key role in enabling effective integration of geospatial data.

Abstract: Integrating ground-level geospatial data with rich geographic context, like OpenStreetMap (OSM), into remote sensing (RS) foundation models (FMs) is essential for advancing geospatial intelligence and supporting a broad spectrum of tasks. However, modality gap between RS and OSM data, including differences in data structure, content, and spatial granularity, makes effective synergy highly challenging, and most existing RS FMs focus on imagery alone. To this end, this study presents GeoLink, a multimodal framework that leverages OSM data to enhance RS FM during both the pretraining and downstream task stages. Specifically, GeoLink enhances RS self-supervised pretraining using multi-granularity learning signals derived from OSM data, guided by cross-modal spatial correlations for information interaction and collaboration. It also introduces image mask-reconstruction to enable sparse input for efficient pretraining. For downstream tasks, GeoLink generates both unimodal and multimodal fine-grained encodings to support a wide range of applications, from common RS interpretation tasks like land cover classification to more comprehensive geographic tasks like urban function zone mapping. Extensive experiments show that incorporating OSM data during pretraining enhances the performance of the RS image encoder, while fusing RS and OSM data in downstream tasks improves the FM’s adaptability to complex geographic scenarios. These results underscore the potential of multimodal synergy in advancing high-level geospatial artificial intelligence. Moreover, we find that spatial correlation plays a crucial role in enabling effective multimodal geospatial data integration. Code, checkpoints, and using examples are released at https://github.com/bailubin/GeoLink_NeurIPS2025

Method: Proposed PatchVSR with dual-stream adapter: patch branch maintains content fidelity from input patches, global branch extracts context from resized full video to bridge semantic gaps. Includes location information injection and multi-patch joint modulation for consistency.

Result: Achieves highly competitive 4K video super-resolution from 512x512 base model with high efficiency. Can synthesize high-fidelity, high-resolution details at patch level while maintaining visual consistency across patches.

Conclusion: PatchVSR demonstrates that patch-wise video super-resolution using video diffusion priors is effective, overcoming limitations of full-attention computation and fixed output resolution while achieving state-of-the-art performance.

Abstract: Pre-trained video generation models hold great potential for generative video super-resolution (VSR). However, adapting them for full-size VSR, as most existing methods do, suffers from unnecessary intensive full-attention computation and fixed output resolution. To overcome these limitations, we make the first exploration into utilizing video diffusion priors for patch-wise VSR. This is non-trivial because pre-trained video diffusion models are not native for patch-level detail generation. To mitigate this challenge, we propose an innovative approach, called PatchVSR, which integrates a dual-stream adapter for conditional guidance. The patch branch extracts features from input patches to maintain content fidelity while the global branch extracts context features from the resized full video to bridge the generation gap caused by incomplete semantics of patches. Particularly, we also inject the patch’s location information into the model to better contextualize patch synthesis within the global video frame. Experiments demonstrate that our method can synthesize high-fidelity, high-resolution details at the patch level. A tailor-made multi-patch joint modulation is proposed to ensure visual consistency across individually enhanced patches. Due to the flexibility of our patch-based paradigm, we can achieve highly competitive 4K VSR based on a 512x512 resolution base model, with extremely high efficiency.

Method: Inject spatially varying Gaussian noise into images using soft causal masks from Vision Transformers - stronger perturbations to background, weaker to foreground.

Result: Consistent performance gains over state-of-the-art OOD baselines on WILDS benchmark Camelyon17.

Conclusion: Causal perturbation shows potential as a tool for reliable and interpretable generalization in real-world scenarios.

Abstract: Out-of-distribution (OOD) generalization remains a central challenge in deploying deep learning models to real-world scenarios, particularly in domains such as biomedical images, where distribution shifts are both subtle and pervasive. While existing methods often pursue domain invariance through complex generative models or adversarial training, these approaches may overlook the underlying causal mechanisms of generalization.In this work, we propose Causally-Guided Gaussian Perturbations (CGP)-a lightweight framework that enhances OOD generalization by injecting spatially varying noise into input images, guided by soft causal masks derived from Vision Transformers. By applying stronger perturbations to background regions and weaker ones to foreground areas, CGP encourages the model to rely on causally relevant features rather than spurious correlations.Experimental results on the challenging WILDS benchmark Camelyon17 demonstrate consistent performance gains over state-of-the-art OOD baselines, highlighting the potential of causal perturbation as a tool for reliable and interpretable generalization.

Method: SeMoBridge uses a Semantic Modality Bridge to map images into text modality while preserving semantic content. It’s closed-form and can be trained with multi-modal supervision combining image and text-alignment losses.

Result: SeMoBridge-T (trained version) outperforms other methods with minimal training time, especially in low-data scenarios (1, 2, and 4 shots).

Conclusion: SeMoBridge effectively addresses CLIP’s intra-modal misalignment through semantic modality bridging, providing efficient few-shot classification performance.

Abstract: While Contrastive Language-Image Pretraining (CLIP) excels at zero-shot tasks by aligning image and text embeddings, its performance in few-shot classification is hindered by a critical limitation: intra-modal misalignment. This issue, caused by a persistent modality gap and CLIP’s exclusively inter-modal training objective, leaves the embedding spaces uncalibrated, making direct image-to-image comparisons unreliable. Existing methods attempt to address this by refining similarity logits or by computationally expensive per-sample optimization. To overcome these challenges, we introduce SeMoBridge, a lightweight yet powerful approach that directly addresses the misalignment. Our method maps images into the text modality, while keeping their semantic content intact through what we call a Semantic Modality Bridge. SeMoBridge is closed-form and can optionally be trained through multi-modal supervision, combining image and text-alignment losses to optimize the projection. Experiments show that the trained version, SeMoBridge-T, requires only a fraction of the training time while overall outperforming other methods, particularly in low-data scenarios (1, 2, and 4 shots). The code is available at \href{https://github.com/christti98/semobridge}{github.com/christti98/semobridge}.

Method: Proposes segmentation-guided scoring (SGS) pipeline that uses person/face segmentation masks to separate foreground/background regions, extracts embeddings with joint vision-language model, and computes region-aware coherence scores fused with HAMMER’s original predictions.

Result: SGS significantly enhances robustness to global manipulations, improves binary detection, grounding, and token-level explanations with negligible computational overhead.

Conclusion: Demonstrates importance of region-aware reasoning in multimodal disinformation detection and provides open-source segmentation and scoring scripts.

Abstract: We extend HAMMER, a state-of-the-art model for multimodal manipulation detection, to handle global scene inconsistencies such as foreground-background (FG-BG) mismatch. While HAMMER achieves strong performance on the DGM4 dataset, it consistently fails when the main subject is contextually misplaced into an implausible background. We diagnose this limitation as a combination of label-space bias, local attention focus, and spurious text-foreground alignment. To remedy this without retraining, we propose a lightweight segmentation-guided scoring (SGS) pipeline. SGS uses person/face segmentation masks to separate foreground and background regions, extracts embeddings with a joint vision-language model, and computes region-aware coherence scores. These scores are fused with HAMMER’s original prediction to improve binary detection, grounding, and token-level explanations. SGS is inference-only, incurs negligible computational overhead, and significantly enhances robustness to global manipulations. This work demonstrates the importance of region-aware reasoning in multimodal disinformation detection. We release scripts for segmentation and scoring at https://github.com/Gaganx0/HAMMER-sgs

Method: Used OpenAI’s gpt-image-1 with carefully designed prompts to generate human-centric news-style images with authentic figures placed into absurd/impossible backdrops. Created captions under three conditions (literal, text attribute, text split) yielding three manipulation categories: FG-BG, FG-BG+TA, and FG-BG+TS. Implemented quality control pipelines for face visibility, deduplication, text scrubbing, and headline length.

Result: Created DGM4+ dataset extension with 5,000 high-quality samples that introduce foreground-background mismatches and their hybrids with text manipulations, creating a comprehensive benchmark for testing detectors on both local and global reasoning.

Conclusion: The DGM4+ dataset extension addresses the critical gap in detecting global inconsistencies in multimodal disinformation, providing a resource to strengthen evaluation of multimodal models that currently struggle with FG-BG inconsistencies.

Abstract: The rapid advances in generative models have significantly lowered the barrier to producing convincing multimodal disinformation. Fabricated images and manipulated captions increasingly co-occur to create persuasive false narratives. While the Detecting and Grounding Multi-Modal Media Manipulation (DGM4) dataset established a foundation for research in this area, it is restricted to local manipulations such as face swaps, attribute edits, and caption changes. This leaves a critical gap: global inconsistencies, such as mismatched foregrounds and backgrounds, which are now prevalent in real-world forgeries. To address this, we extend DGM4 with 5,000 high-quality samples that introduce Foreground-Background (FG-BG) mismatches and their hybrids with text manipulations. Using OpenAI’s gpt-image-1 and carefully designed prompts, we generate human-centric news-style images where authentic figures are placed into absurd or impossible backdrops (e.g., a teacher calmly addressing students on the surface of Mars). Captions are produced under three conditions: literal, text attribute, and text split, yielding three new manipulation categories: FG-BG, FG-BG+TA, and FG-BG+TS. Quality control pipelines enforce one-to-three visible faces, perceptual hash deduplication, OCR-based text scrubbing, and realistic headline length. By introducing global manipulations, our extension complements existing datasets, creating a benchmark DGM4+ that tests detectors on both local and global reasoning. This resource is intended to strengthen evaluation of multimodal models such as HAMMER, which currently struggle with FG-BG inconsistencies. We release our DGM4+ dataset and generation script at https://github.com/Gaganx0/DGM4plus

Method: Use sections of principal fiber bundles to quotient out symmetry groups, enabling simple metrics to measure dissimilarities between object orbits. Optimize sections to maximize class separation. Applied to object contours with translation, rotation, scaling, and reparameterization symmetries.

Result: Developed a 2-parameter family of canonical curve parameterizations (including constant-speed as special case) that effectively handles symmetries. Provided open-source code and tutorial for practical application.

Conclusion: The geometric framework provides an efficient alternative to data augmentation for handling symmetries, with wide applicability across computer vision and medical domains. The method successfully separates classes while respecting inherent dataset symmetries.

Abstract: Most datasets encountered in computer vision and medical applications present symmetries that should be taken into account in classification tasks. A typical example is the symmetry by rotation and/or scaling in object detection. A common way to build neural networks that learn the symmetries is to use data augmentation. In order to avoid data augmentation and build more sustainable algorithms, we present an alternative method to mod out symmetries based on the notion of section of a principal fiber bundle. This framework allows the use of simple metrics on the space of objects in order to measure dissimilarities between orbits of objects under the symmetry group. Moreover, the section used can be optimized to maximize separation of classes. We illustrate this methodology on a dataset of contours of objects for the groups of translations, rotations, scalings and reparameterizations. In particular, we present a $2$-parameter family of canonical parameterizations of curves, containing the constant-speed parameterization as a special case, which we believe is interesting in its own right. We hope that this simple application will serve to convey the geometric concepts underlying this method, which have a wide range of possible applications. The code is available at the following link: $\href{https://github.com/GiLonga/Geometric-Learning}{https://github.com/GiLonga/Geometric-Learning}$. A tutorial notebook showcasing an application of the code to a specific dataset is available at the following link: $\href{https://github.com/ioanaciuclea/geometric-learning-notebook}{https://github.com/ioanaciuclea/geometric-learning-notebook}$

Method: Generate 3D pseudo-ground-truth labels using foundation models Grounded-SAM and Metric3Dv2, leverage temporal information for label densification, and propose a streamlined model called EasyOcc that learns solely from these pseudo-labels without complex rendering strategies.

Result: Substantial performance improvements: mIoU increased by 45% (from 9.73 to 14.09) when integrated into OccNeRF. EasyOcc achieved 13.86 mIoU and state-of-the-art 7.71 mIoU on full scene evaluation without camera mask, outperforming previous best by 31%.

Conclusion: Foundation models, temporal context, and loss computation space choice are critical for self-supervised learning in comprehensive scene understanding. The proposed pseudo-labels are easily transferable across architectures unlike previous methods.

Abstract: Self-supervised models have recently achieved notable advancements, particularly in the domain of semantic occupancy prediction. These models utilize sophisticated loss computation strategies to compensate for the absence of ground-truth labels. For instance, techniques such as novel view synthesis, cross-view rendering, and depth estimation have been explored to address the issue of semantic and depth ambiguity. However, such techniques typically incur high computational costs and memory usage during the training stage, especially in the case of novel view synthesis. To mitigate these issues, we propose 3D pseudo-ground-truth labels generated by the foundation models Grounded-SAM and Metric3Dv2, and harness temporal information for label densification. Our 3D pseudo-labels can be easily integrated into existing models, which yields substantial performance improvements, with mIoU increasing by 45%, from 9.73 to 14.09, when implemented into the OccNeRF model. This stands in contrast to earlier advancements in the field, which are often not readily transferable to other architectures. Additionally, we propose a streamlined model, EasyOcc, achieving 13.86 mIoU. This model conducts learning solely from our labels, avoiding complex rendering strategies mentioned previously. Furthermore, our method enables models to attain state-of-the-art performance when evaluated on the full scene without applying the camera mask, with EasyOcc achieving 7.71 mIoU, outperforming the previous best model by 31%. These findings highlight the critical importance of foundation models, temporal context, and the choice of loss computation space in self-supervised learning for comprehensive scene understanding.

Method: Dual-branch architecture: MobileNetV2 CNN for RGB frames and LSTM with attention for 2D keypoints. Pose-derived keypoints guide visual focus. Distance-based thresholding segments input sequences before ball contact.

Result: 89% accuracy on test set, outperforming visual-only and pose-only baselines by 14-22%. Inference time of 22 milliseconds.

Conclusion: The lightweight and interpretable design is suitable for goalkeeper training, tactical analysis, and real-time game analytics.

Abstract: Penalty kicks often decide championships, yet goalkeepers must anticipate the kicker’s intent from subtle biomechanical cues within a very short time window. This study introduces a real-time, multi-modal deep learning framework to predict the direction of a penalty kick (left, middle, or right) before ball contact. The model uses a dual-branch architecture: a MobileNetV2-based CNN extracts spatial features from RGB frames, while 2D keypoints are processed by an LSTM network with attention mechanisms. Pose-derived keypoints further guide visual focus toward task-relevant regions. A distance-based thresholding method segments input sequences immediately before ball contact, ensuring consistent input across diverse footage. A custom dataset of 755 penalty kick events was created from real match videos, with frame-level annotations for object detection, shooter keypoints, and final ball placement. The model achieved 89% accuracy on a held-out test set, outperforming visual-only and pose-only baselines by 14-22%. With an inference time of 22 milliseconds, the lightweight and interpretable design makes it suitable for goalkeeper training, tactical analysis, and real-time game analytics.

Method: Integrates object retrieval using captions covering physical, functional, and contextual attributes, places objects based on implicit and explicit layout constraints, and refines positions with collision-aware spatial reasoning powered by large reasoning models.

Result: Significantly outperforms previous methods on instructions ranging from simple to complex indoor configurations in human-rated visual fidelity, adherence to constraints, and asset retrieval quality.

Conclusion: Demonstrates advanced spatial reasoning abilities of modern LRMs and contributes to text-to-scene generation field; codebase released to advance research in object retrieval and placement with LRMs.

Abstract: Prompt-driven scene synthesis allows users to generate complete 3D environments from textual descriptions. Current text-to-scene methods often struggle with complex geometries and object transformations, and tend to show weak adherence to complex instructions. We address these limitations by introducing Reason-3D, a text-to-scene model powered by large reasoning models (LRMs). Reason-3D integrates object retrieval using captions covering physical, functional, and contextual attributes. Reason-3D then places the selected objects based on implicit and explicit layout constraints, and refines their positions with collision-aware spatial reasoning. Evaluated on instructions ranging from simple to complex indoor configurations, Reason-3D significantly outperforms previous methods in human-rated visual fidelity, adherence to constraints, and asset retrieval quality. Beyond its contribution to the field of text-to-scene generation, our work showcases the advanced spatial reasoning abilities of modern LRMs. Additionally, we release the codebase to further the research in object retrieval and placement with LRMs.

Method: Proposed an information-theoretic perspective revealing that successful denoising reduces conditional entropy, leading to EVODiff - a method that optimizes conditional variance to minimize transition and reconstruction errors.

Result: EVODiff significantly outperforms SOTA solvers: reduces reconstruction error by 45.5% (FID from 5.10 to 2.78) at 10 NFE on CIFAR-10, cuts NFE cost by 25% on ImageNet-256, and improves text-to-image generation with fewer artifacts.

Conclusion: The information-theoretic perspective provides fundamental insights for diffusion model inference, and EVODiff demonstrates superior performance by systematically reducing uncertainty through conditional entropy optimization.

Abstract: Diffusion models (DMs) excel in image generation, but suffer from slow inference and the training-inference discrepancies. Although gradient-based solvers like DPM-Solver accelerate the denoising inference, they lack theoretical foundations in information transmission efficiency. In this work, we introduce an information-theoretic perspective on the inference processes of DMs, revealing that successful denoising fundamentally reduces conditional entropy in reverse transitions. This principle leads to our key insights into the inference processes: (1) data prediction parameterization outperforms its noise counterpart, and (2) optimizing conditional variance offers a reference-free way to minimize both transition and reconstruction errors. Based on these insights, we propose an entropy-aware variance optimized method for the generative process of DMs, called EVODiff, which systematically reduces uncertainty by optimizing conditional entropy during denoising. Extensive experiments on DMs validate our insights and demonstrate that our method significantly and consistently outperforms state-of-the-art (SOTA) gradient-based solvers. For example, compared to the DPM-Solver++, EVODiff reduces the reconstruction error by up to 45.5% (FID improves from 5.10 to 2.78) at 10 function evaluations (NFE) on CIFAR-10, cuts the NFE cost by 25% (from 20 to 15 NFE) for high-quality samples on ImageNet-256, and improves text-to-image generation while reducing artifacts. Code is available at https://github.com/ShiguiLi/EVODiff.

Method: EchoGen employs a dual-path injection strategy with two encoders: a semantic encoder extracts high-level subject identity injected via decoupled cross-attention, and a content encoder captures fine-grained details integrated through multi-modal attention. This disentangles semantic identity from visual details for enhanced controllability.

Result: EchoGen achieves subject fidelity and image quality comparable to state-of-the-art diffusion-based methods while significantly reducing sampling latency, making it the first feed-forward subject-driven framework built on VAR models.

Conclusion: The proposed EchoGen framework successfully bridges the gap between efficiency and quality in subject-driven generation by leveraging VAR models’ fast sampling capabilities while maintaining high fidelity through effective dual-path injection of semantic and content information.

Abstract: Subject-driven generation is a critical task in creative AI; yet current state-of-the-art methods present a stark trade-off. They either rely on computationally expensive, per-subject fine-tuning, sacrificing efficiency and zero-shot capability, or employ feed-forward architectures built on diffusion models, which are inherently plagued by slow inference speeds. Visual Auto-Regressive (VAR) models are renowned for their rapid sampling speeds and strong generative quality, making them an ideal yet underexplored foundation for resolving this tension. To bridge this gap, we introduce EchoGen, a pioneering framework that empowers VAR models with subject-driven generation capabilities. The core design of EchoGen is an effective dual-path injection strategy that disentangles a subject’s high-level semantic identity from its low-level fine-grained details, enabling enhanced controllability and fidelity. We employ a semantic encoder to extract the subject’s abstract identity, which is injected through decoupled cross-attention to guide the overall composition. Concurrently, a content encoder captures intricate visual details, which are integrated via a multi-modal attention mechanism to ensure high-fidelity texture and structural preservation. To the best of our knowledge, EchoGen is the first feed-forward subject-driven framework built upon VAR models. Both quantitative and qualitative results substantiate our design, demonstrating that EchoGen achieves subject fidelity and image quality comparable to state-of-the-art diffusion-based methods with significantly lower sampling latency. Code and models will be released soon.

Method: Uses Entropy-based Dynamic Patcher (EDP) to detect transition points via conditional entropy and determine patch boundaries, combined with Adaptive Patch Encoder (APE) for intra-patch dependency capture.

Result: Experiments show EntroPE improves both accuracy and efficiency across long-term forecasting benchmarks compared to existing methods.

Conclusion: Entropy-guided dynamic patching establishes a promising new paradigm for time series modeling that preserves temporal structure while maintaining computational benefits.

Abstract: Transformer-based models have significantly advanced time series forecasting, with patch-based input strategies offering efficiency and improved long-horizon modeling. Yet, existing approaches rely on temporally-agnostic patch construction, where arbitrary starting positions and fixed lengths fracture temporal coherence by splitting natural transitions across boundaries. This naive segmentation often disrupts short-term dependencies and weakens representation learning. In response, we propose EntroPE (Entropy-Guided Dynamic Patch Encoder), a novel, temporally informed framework that dynamically detects transition points via conditional entropy and dynamically places patch boundaries. This preserves temporal structure while retaining the computational benefits of patching. EntroPE consists of two key modules, namely an Entropy-based Dynamic Patcher (EDP) that applies information-theoretic criteria to locate natural temporal shifts and determine patch boundaries, and an Adaptive Patch Encoder (APE) that employs pooling and cross-attention to capture intra-patch dependencies and produce fixed-size latent representations. These embeddings are then processed by a global transformer to model inter-patch dynamics. Experiments across long-term forecasting benchmarks demonstrate that EntroPE improves both accuracy and efficiency, establishing entropy-guided dynamic patching as a promising new paradigm for time series modeling. Code is available at: https://github.com/Sachithx/EntroPE.

Method: Fine-tuned LLM via preference learning to rephrase image captions into diverse textual prompts that guide a text-to-image model to generate difficult visual scenarios.

Result: Achieved superior robustness on FishEye8K object detection benchmark, surpassing both naive augmentation and manually engineered prompts.

Conclusion: Establishes scalable framework for automated edge-case synthesis, shifting data curation from manual effort to targeted automation for more reliable AI systems.

Abstract: The performance of deep neural networks is strongly influenced by the quality of their training data. However, mitigating dataset bias by manually curating challenging edge cases remains a major bottleneck. To address this, we propose an automated pipeline for text-guided edge-case synthesis. Our approach employs a Large Language Model, fine-tuned via preference learning, to rephrase image captions into diverse textual prompts that steer a Text-to-Image model toward generating difficult visual scenarios. Evaluated on the FishEye8K object detection benchmark, our method achieves superior robustness, surpassing both naive augmentation and manually engineered prompts. This work establishes a scalable framework that shifts data curation from manual effort to automated, targeted synthesis, offering a promising direction for developing more reliable and continuously improving AI systems. Code is available at https://github.com/gokyeongryeol/ATES.

Method: Created Human-MME benchmark with: 1) Diverse human scenes across 4 primary domains, 15 secondary domains, and 43 sub-fields; 2) Progressive evaluation from granular perception to higher-dimensional reasoning across 8 dimensions with 19,945 image-question pairs; 3) High-quality annotations using automated pipeline and manual labeling platform with multiple question types.

Result: Extensive experiments on 17 state-of-the-art MLLMs revealed limitations in current models’ human-centric understanding capabilities. The benchmark successfully exposed model weaknesses and provides guidance for future improvements.

Conclusion: Human-MME provides a holistic evaluation framework for MLLMs in human-centric scene understanding, addressing gaps in existing benchmarks and offering comprehensive assessment tools to guide future research toward better human-centric image understanding.

Abstract: Multimodal Large Language Models (MLLMs) have demonstrated significant advances in visual understanding tasks. However, their capacity to comprehend human-centric scenes has rarely been explored, primarily due to the absence of comprehensive evaluation benchmarks that take into account both the human-oriented granular level and higher-dimensional causal reasoning ability. Such high-quality evaluation benchmarks face tough obstacles, given the physical complexity of the human body and the difficulty of annotating granular structures. In this paper, we propose Human-MME, a curated benchmark designed to provide a more holistic evaluation of MLLMs in human-centric scene understanding. Compared with other existing benchmarks, our work provides three key features: 1. Diversity in human scene, spanning 4 primary visual domains with 15 secondary domains and 43 sub-fields to ensure broad scenario coverage. 2. Progressive and diverse evaluation dimensions, evaluating the human-based activities progressively from the human-oriented granular perception to the higher-dimensional reasoning, consisting of eight dimensions with 19,945 real-world image question pairs and an evaluation suite. 3. High-quality annotations with rich data paradigms, constructing the automated annotation pipeline and human-annotation platform, supporting rigorous manual labeling to facilitate precise and reliable model assessment. Our benchmark extends the single-target understanding to the multi-person and multi-image mutual understanding by constructing the choice, short-answer, grounding, ranking and judgment question components, and complex questions of their combination. The extensive experiments on 17 state-of-the-art MLLMs effectively expose the limitations and guide future MLLMs research toward better human-centric image understanding. All data and code are available at https://github.com/Yuan-Hou/Human-MME.

Method: Evaluated five pedestrian-specific detectors (F2DNet, MGAN, ALFNet, CSP, Cascade R-CNN) and three YOLOv12 variants on ECP-DP dataset. Used Equal Opportunity Difference metric and Z-test to quantify fairness across confidence thresholds.

Result: Found biases against pedestrians with parallel legs, straight elbows, and lateral views. Lower body joint occlusion has more negative impact than upper body/head occlusion. Cascade R-CNN achieved lowest miss-rate and smallest bias.

Conclusion: This is the first comprehensive pose- and occlusion-aware fairness evaluation in pedestrian detection for autonomous driving, revealing significant biases that need addressing for equitable performance.

Abstract: Pedestrian detection plays a critical role in autonomous driving (AD), where ensuring safety and reliability is important. While many detection models aim to reduce miss-rates and handle challenges such as occlusion and long-range recognition, fairness remains an underexplored yet equally important concern. In this work, we systematically investigate how variations in the pedestrian pose – including leg status, elbow status, and body orientation – as well as individual joint occlusions, affect detection performance. We evaluate five pedestrian-specific detectors (F2DNet, MGAN, ALFNet, CSP, and Cascade R-CNN) alongside three general-purpose models (YOLOv12 variants) on the EuroCity Persons Dense Pose (ECP-DP) dataset. Fairness is quantified using the Equal Opportunity Difference (EOD) metric across various confidence thresholds. To assess statistical significance and robustness, we apply the Z-test. Our findings highlight biases against pedestrians with parallel legs, straight elbows, and lateral views. Occlusion of lower body joints has a more negative impact on the detection rate compared to the upper body and head. Cascade R-CNN achieves the lowest overall miss-rate and exhibits the smallest bias across all attributes. To the best of our knowledge, this is the first comprehensive pose- and occlusion-aware fairness evaluation in pedestrian detection for AD.

Method: Proposed a dual-model architecture combining CNN for cell type classification and Vision Transformer (ViT) for multi-attribute classification, using two complementary datasets: PBC dataset (8 cell types) and WBCAtt dataset (11 morphological attributes).

Result: Achieved a new benchmark of 94.62% accuracy in multi-attribute classification, significantly enhancing model interpretability and offering substantial time and cost efficiency compared to conventional human annotation.

Conclusion: AttriGen establishes a new paradigm that can be extended to other computer vision classification tasks by effectively automating the expansion of multi-attribute labels.

Abstract: We introduce AttriGen, a novel framework for automated, fine-grained multi-attribute annotation in computer vision, with a particular focus on cell microscopy where multi-attribute classification remains underrepresented compared to traditional cell type categorization. Using two complementary datasets: the Peripheral Blood Cell (PBC) dataset containing eight distinct cell types and the WBC Attribute Dataset (WBCAtt) that contains their corresponding 11 morphological attributes, we propose a dual-model architecture that combines a CNN for cell type classification, as well as a Vision Transformer (ViT) for multi-attribute classification achieving a new benchmark of 94.62% accuracy. Our experiments demonstrate that AttriGen significantly enhances model interpretability and offers substantial time and cost efficiency relative to conventional full-scale human annotation. Thus, our framework establishes a new paradigm that can be extended to other computer vision classification tasks by effectively automating the expansion of multi-attribute labels.

Method: Extends a state-of-the-art visual-based approach by leveraging vision-language models for data representation, integrating similarity estimation and viewport spatio-temporal cross-attention mechanisms to discover dependencies between visual and textual modalities.

Result: Quantitative and qualitative evaluations on TSV360 dataset show TSalV360’s competitiveness with SOTA visual-based approaches and demonstrate its competency for customized text-driven saliency detection in 360-degree videos.

Conclusion: TSalV360 successfully performs text-driven saliency detection in 360-degree videos, enabling personalized attention modeling based on user-provided textual descriptions of objects and events.

Abstract: In this paper, we deal with the task of text-driven saliency detection in 360-degrees videos. For this, we introduce the TSV360 dataset which includes 16,000 triplets of ERP frames, textual descriptions of salient objects/events in these frames, and the associated ground-truth saliency maps. Following, we extend and adapt a SOTA visual-based approach for 360-degrees video saliency detection, and develop the TSalV360 method that takes into account a user-provided text description of the desired objects and/or events. This method leverages a SOTA vision-language model for data representation and integrates a similarity estimation module and a viewport spatio-temporal cross-attention mechanism, to discover dependencies between the different data modalities. Quantitative and qualitative evaluations using the TSV360 dataset, showed the competitiveness of TSalV360 compared to a SOTA visual-based approach and documented its competency to perform customized text-driven saliency detection in 360-degrees videos.

Method: GSDD encodes critical discriminative information using a small number of 2D Gaussian primitives instead of representing all pixels equally. It adapts CUDA-based splatting operators for parallel inference and training, enabling efficient rendering with minimal computational overhead.

Result: GSDD achieves state-of-the-art performance on CIFAR-10, CIFAR-100, and ImageNet subsets while maintaining highly efficient encoding and decoding costs. The sparse representation improves dataset diversity under the same storage budget.

Conclusion: The proposed sparse Gaussian representation is simple, effective, broadly applicable to different distillation pipelines, and highly scalable, offering a promising alternative to conventional dense pixel-level approaches in dataset distillation.

Abstract: Dataset distillation has emerged as a promising paradigm that synthesizes compact, informative datasets capable of retaining the knowledge of large-scale counterparts, thereby addressing the substantial computational and storage burdens of modern model training. Conventional approaches typically rely on dense pixel-level representations, which introduce redundancy and are difficult to scale up. In this work, we propose GSDD, a novel and efficient sparse representation for dataset distillation based on 2D Gaussians. Instead of representing all pixels equally, GSDD encodes critical discriminative information in a distilled image using only a small number of Gaussian primitives. This sparse representation could improve dataset diversity under the same storage budget, enhancing coverage of difficult samples and boosting distillation performance. To ensure both efficiency and scalability, we adapt CUDA-based splatting operators for parallel inference and training, enabling high-quality rendering with minimal computational and memory overhead. Our method is simple yet effective, broadly applicable to different distillation pipelines, and highly scalable. Experiments show that GSDD achieves state-of-the-art performance on CIFAR-10, CIFAR-100, and ImageNet subsets, while remaining highly efficient encoding and decoding cost. Our code is available at https://github.com/j-cyoung/GSDatasetDistillation.

Method: Extended existing multigranular explanation framework by integrating LLaVA-OneVision to generate natural language descriptions, then evaluated plausibility by quantifying semantic overlap between textual descriptions of visual explanations and video summaries using SBERT and SimCSE sentence embeddings.

Result: Conducted experimental study using CA-SUM method and SumMe/TVSum datasets to examine whether more faithful explanations are also more plausible, and identified the most appropriate approach for generating plausible textual explanations.

Conclusion: The study provides insights into the relationship between explanation faithfulness and plausibility in video summarization, and establishes methods for generating and evaluating plausible textual explanations.

Abstract: In this paper, we present our experimental study on generating plausible textual explanations for the outcomes of video summarization. For the needs of this study, we extend an existing framework for multigranular explanation of video summarization by integrating a SOTA Large Multimodal Model (LLaVA-OneVision) and prompting it to produce natural language descriptions of the obtained visual explanations. Following, we focus on one of the most desired characteristics for explainable AI, the plausibility of the obtained explanations that relates with their alignment with the humans’ reasoning and expectations. Using the extended framework, we propose an approach for evaluating the plausibility of visual explanations by quantifying the semantic overlap between their textual descriptions and the textual descriptions of the corresponding video summaries, with the help of two methods for creating sentence embeddings (SBERT, SimCSE). Based on the extended framework and the proposed plausibility evaluation approach, we conduct an experimental study using a SOTA method (CA-SUM) and two datasets (SumMe, TVSum) for video summarization, to examine whether the more faithful explanations are also the more plausible ones, and identify the most appropriate approach for generating plausible textual explanations for video summarization.

Method: MGCE consists of two modules: Dynamic Conceptual Contrastive Learning (DCCL) that alternates between concept mining and dual-level representation learning, and Multi-Granularity Experts Collaborative Learning (MECL) that introduces additional experts at different granularities with concept alignment matrix for cross-expert collaboration.

Result: Extensive experiments on nine fine-grained visual recognition benchmarks show MGCE achieves state-of-the-art results, particularly in novel-class accuracy. Without prior knowledge of category numbers, MGCE outperforms parametric approaches that require exact category numbers by 3.6% on average.

Conclusion: MGCE effectively addresses key limitations in GCD by adaptively mining visual concepts, integrating multi-granularity knowledge, and automatically estimating category numbers, making it suitable for practical open-world settings.

Abstract: Generalized Category Discovery (GCD) is an open-world problem that clusters unlabeled data by leveraging knowledge from partially labeled categories. A key challenge is that unlabeled data may contain both known and novel categories. Existing approaches suffer from two main limitations. First, they fail to exploit multi-granularity conceptual information in visual data, which limits representation quality. Second, most assume that the number of unlabeled categories is known during training, which is impractical in real-world scenarios. To address these issues, we propose a Multi-Granularity Conceptual Experts (MGCE) framework that adaptively mines visual concepts and integrates multi-granularity knowledge for accurate category discovery. MGCE consists of two modules: (1) Dynamic Conceptual Contrastive Learning (DCCL), which alternates between concept mining and dual-level representation learning to jointly optimize feature learning and category discovery; and (2) Multi-Granularity Experts Collaborative Learning (MECL), which extends the single-expert paradigm by introducing additional experts at different granularities and by employing a concept alignment matrix for effective cross-expert collaboration. Importantly, MGCE can automatically estimate the number of categories in unlabeled data, making it suitable for practical open-world settings. Extensive experiments on nine fine-grained visual recognition benchmarks demonstrate that MGCE achieves state-of-the-art results, particularly in novel-class accuracy. Notably, even without prior knowledge of category numbers, MGCE outperforms parametric approaches that require knowing the exact number of categories, with an average improvement of 3.6%. Code is available at https://github.com/HaiyangZheng/MGCE.

Method: IMG uses MLLMs to detect misalignments, an Implicit Aligner to modify diffusion conditioning features for re-generation, and formulates alignment as an Iteratively Updated Preference Objective for training.

Result: Extensive evaluations on SDXL, SDXL-DPO, and FLUX show IMG outperforms existing alignment methods. It also works as a plug-and-play adapter to enhance prior finetuning-based methods.

Conclusion: IMG provides an effective, data-efficient solution for multimodal alignment that requires no extra data or editing, achieving superior performance while maintaining flexibility as a plug-and-play component.

Abstract: Ensuring precise multimodal alignment between diffusion-generated images and input prompts has been a long-standing challenge. Earlier works finetune diffusion weight using high-quality preference data, which tends to be limited and difficult to scale up. Recent editing-based methods further refine local regions of generated images but may compromise overall image quality. In this work, we propose Implicit Multimodal Guidance (IMG), a novel re-generation-based multimodal alignment framework that requires no extra data or editing operations. Specifically, given a generated image and its prompt, IMG a) utilizes a multimodal large language model (MLLM) to identify misalignments; b) introduces an Implicit Aligner that manipulates diffusion conditioning features to reduce misalignments and enable re-generation; and c) formulates the re-alignment goal into a trainable objective, namely Iteratively Updated Preference Objective. Extensive qualitative and quantitative evaluations on SDXL, SDXL-DPO, and FLUX show that IMG outperforms existing alignment methods. Furthermore, IMG acts as a flexible plug-and-play adapter, seamlessly enhancing prior finetuning-based alignment methods. Our code will be available at https://github.com/SHI-Labs/IMG-Multimodal-Diffusion-Alignment.

Method: Uses knowledge distillation with mechanistic interpretability to analyze internal computations, identify essential subcomponents, and guide student architecture design through pruning, approximation, skipping, or reparametrization of components.

Result: Developed Donut-MINT, a pruned Donut variant that reduces inference time and memory usage while maintaining strong performance on the DocVQA benchmark.

Conclusion: The approach reframes compression as circuit discovery, bridging interpretability research with practical Vision-Language Model deployment for efficient document understanding.

Abstract: Recent advances in Visually-rich Document Understanding rely on large Vision-Language Models like Donut, which perform document-level Visual Question Answering without Optical Character Recognition. Despite their effectiveness, these models are too costly for real-time or resource-constrained applications. We investigate model compression through knowledge distillation, training compact student models from a larger teacher. We leverage mechanistic interpretability to drive student architecture design within this framework. By analyzing internal computations, we identify essential subcomponents to retain, while having a clear view of which subcomponents should be approximated, skipped, or reparametrized based on their function. This approach yields Donut-MINT (Mechanistic Interpretability-based Network Trimming), a pruned Donut variant that reduces inference time and memory usage while maintaining strong performance on DocVQA, a standard benchmark for document Visual Question Answering. Our method reframes compression as circuit discovery, bridging interpretability research and practical Vision-Language Model deployment.

Method: Proposed Farsighted-LAM with geometry-aware spatial encoding and multi-scale temporal modeling, plus SSM-VLA framework that integrates structured perception with visual Chain-of-Thought module for explicit environmental reasoning.

Result: Achieved state-of-the-art performance on multiple Vision-Language-Action tasks in both simulation and real-world settings, demonstrating enhanced robustness and generalizability.

Conclusion: Combining geometry-aware modeling, temporal coherence, and explicit reasoning effectively enhances the robustness and generalizability of embodied intelligence systems.

Abstract: Latent Action Models (LAMs) enable Vision-Language-Action (VLA) systems to learn semantic action representations from large-scale unannotated data. Yet, we identify two bottlenecks of LAMs: 1) the commonly adopted end-to-end trained image encoder suffers from poor spatial understanding; 2) LAMs can be fragile when input frames are distant, leading to limited temporal perception. Such factors inevitably hinder stable and clear action modeling. To this end, we propose Farsighted-LAM, a latent action framework with geometry-aware spatial encoding and multi-scale temporal modeling, capturing structural priors and dynamic motion patterns from consecutive frames. We further propose SSM-VLA, an end-to-end VLA framework built upon Farsighted-LAM, which integrates structured perception with a visual Chain-of-Thought module to explicitly reason about environmental dynamics, enhancing decision consistency and interpretability. We validate SSM-VLA on multiple VLA tasks in both simulation and real-world settings, and achieve state-of-the-art performance. Our results demonstrate that our strategy of combining geometry-aware modeling, temporal coherence, and explicit reasoning is effective in enhancing the robustness and generalizability of embodied intelligence.

Method: Proposed Paragraph-level Relative Policy Optimization (PRPO), a reinforcement learning algorithm that aligns LLM reasoning with image content at paragraph level, using a reasoning-annotated dataset for deepfake detection.

Result: PRPO improves detection accuracy significantly and achieves the highest reasoning score of 4.55/5.0, outperforming GRPO in ablation studies under test-time conditions.

Conclusion: Grounding multimodal reasoning in visual evidence is crucial for reliable and interpretable deepfake detection, with PRPO demonstrating effective alignment between LLM reasoning and image content.

Abstract: The rapid rise of synthetic media has made deepfake detection a critical challenge for online safety and trust. Progress remains constrained by the scarcity of large, high-quality datasets. Although multimodal large language models (LLMs) exhibit strong reasoning capabilities, their performance on deepfake detection is poor, often producing explanations that are misaligned with visual evidence or hallucinatory. To address this limitation, we introduce a reasoning-annotated dataset for deepfake detection and propose Paragraph-level Relative Policy Optimization (PRPO), a reinforcement learning algorithm that aligns LLM reasoning with image content at the paragraph level. Experiments show that PRPO improves detection accuracy by a wide margin and achieves the highest reasoning score of 4.55/5.0. Ablation studies further demonstrate that PRPO significantly outperforms GRPO under test-time conditions. These results underscore the importance of grounding multimodal reasoning in visual evidence to enable more reliable and interpretable deepfake detection.

Method: Developed CAT framework that refines low-bit quantized outputs using cluster-specific affine parameters, requiring minimal additional parameters and no fine-tuning of model or quantization parameters.

Result: Achieved up to 53.18% Top-1 accuracy on W2A2 ResNet-18 on ImageNet-1K, consistently outperforming prior PTQ methods across architectures and low-bit settings, with >3% improvement when used as plug-in.

Conclusion: CAT effectively addresses accuracy degradation in low-bit PTQ through cluster-specific affine transformation, providing significant improvements without requiring model retraining.

Abstract: Post-Training Quantization (PTQ) reduces the memory footprint and computational overhead of deep neural networks by converting full-precision (FP) values into quantized and compressed data types. While PTQ is more cost-efficient than Quantization-Aware Training (QAT), it is highly susceptible to accuracy degradation under a low-bit quantization (LQ) regime (e.g., 2-bit). Affine transformation is a classical technique used to reduce the discrepancy between the information processed by a quantized model and that processed by its full-precision counterpart; however, we find that using plain affine transformation, which applies a uniform affine parameter set for all outputs, worsens the results in low-bit PTQ. To address this, we propose Cluster-based Affine Transformation (CAT), an error-reduction framework that employs cluster-specific parameters to align LQ outputs with FP counterparts. CAT refines LQ outputs with only a negligible number of additional parameters, without requiring fine-tuning of the model or quantization parameters. We further introduce a novel PTQ framework integrated with CAT. Experiments on ImageNet-1K show that this framework consistently outperforms prior PTQ methods across diverse architectures and LQ settings, achieving up to 53.18% Top-1 accuracy on W2A2 ResNet-18. Moreover, CAT enhances existing PTQ baselines by more than 3% when used as a plug-in. We plan to release our implementation alongside the publication of this paper.

Method: ProfVLM reformulates skill assessment as generative reasoning using an AttentiveGatedProjector to fuse multi-view features from a frozen TimeSformer backbone into a language model tuned for feedback generation, trained on EgoExo4D with expert commentaries.

Result: ProfVLM surpasses state-of-the-art methods while using up to 20x fewer parameters and reducing training time by up to 60%, achieving superior accuracy across diverse activities and generating natural language critiques aligned with performance.

Conclusion: Generative vision-language modeling represents a powerful new direction for skill assessment, offering both accurate proficiency estimation and transparent reasoning through natural language feedback.

Abstract: Existing approaches to skill proficiency estimation often rely on black-box video classifiers, ignoring multi-view context and lacking explainability. We present ProfVLM, a compact vision-language model that reformulates this task as generative reasoning: it jointly predicts skill level and generates expert-like feedback from egocentric and exocentric videos. Central to our method is an AttentiveGatedProjector that dynamically fuses multi-view features, projected from a frozen TimeSformer backbone into a language model tuned for feedback generation. Trained on EgoExo4D with expert commentaries, ProfVLM surpasses state-of-the-art methods while using up to 20x fewer parameters and reducing training time by up to 60%. Our approach not only achieves superior accuracy across diverse activities, but also outputs natural language critiques aligned with performance, offering transparent reasoning. These results highlight generative vision-language modeling as a powerful new direction for skill assessment.

Method: Uses Progressive Label Assignment (PLA) to dynamically estimate instance sizes at different training stages for label assignment, and Prior-Guided Dynamic Mask Loss (PGDM-Loss) that combines SAM model advantages with watershed algorithm to handle both sparse and dense scenes effectively.

Result: Achieves competitive performance: 66.09% on DOTA-v1.0, 56.86% on DOTA-v1.5, 41.28% on DOTA-v2.0, 46.40% on DIOR, 19.60% on STAR, and 45.96% on RSAR datasets, with particularly strong results in scenarios with large object size variations or sparse object occurrences.

Conclusion: Point2RBox-v3 is the first model to use dynamic pseudo labels for label assignment and successfully integrates SAM model with watershed algorithm, achieving excellent performance across both sparse and dense scenes in oriented object detection.

Abstract: Driven by the growing need for Oriented Object Detection (OOD), learning from point annotations under a weakly-supervised framework has emerged as a promising alternative to costly and laborious manual labeling. In this paper, we discuss two deficiencies in existing point-supervised methods: inefficient utilization and poor quality of pseudo labels. Therefore, we present Point2RBox-v3. At the core are two principles: 1) Progressive Label Assignment (PLA). It dynamically estimates instance sizes in a coarse yet intelligent manner at different stages of the training process, enabling the use of label assignment methods. 2) Prior-Guided Dynamic Mask Loss (PGDM-Loss). It is an enhancement of the Voronoi Watershed Loss from Point2RBox-v2, which overcomes the shortcomings of Watershed in its poor performance in sparse scenes and SAM’s poor performance in dense scenes. To our knowledge, Point2RBox-v3 is the first model to employ dynamic pseudo labels for label assignment, and it creatively complements the advantages of SAM model with the watershed algorithm, which achieves excellent performance in both sparse and dense scenes. Our solution gives competitive performance, especially in scenarios with large variations in object size or sparse object occurrences: 66.09%/56.86%/41.28%/46.40%/19.60%/45.96% on DOTA-v1.0/DOTA-v1.5/DOTA-v2.0/DIOR/STAR/RSAR.

Method: Flower uses an iterative three-step procedure: (1) flow-consistent destination estimation using velocity network for denoising, (2) refinement by projecting onto feasible set defined by forward operator, and (3) time-progression by re-projecting refined destination along flow trajectory.

Result: Flower achieves state-of-the-art reconstruction quality and demonstrates theoretical approximation of Bayesian posterior sampling, while using nearly identical hyperparameters across different inverse problems.

Conclusion: Flower successfully unifies perspectives from plug-and-play methods and generative inverse solvers, providing both theoretical foundations and practical performance for solving inverse problems.

Abstract: We introduce Flower, a solver for inverse problems. It leverages a pre-trained flow model to produce reconstructions that are consistent with the observed measurements. Flower operates through an iterative procedure over three steps: (i) a flow-consistent destination estimation, where the velocity network predicts a denoised target; (ii) a refinement step that projects the estimated destination onto a feasible set defined by the forward operator; and (iii) a time-progression step that re-projects the refined destination along the flow trajectory. We provide a theoretical analysis that demonstrates how Flower approximates Bayesian posterior sampling, thereby unifying perspectives from plug-and-play methods and generative inverse solvers. On the practical side, Flower achieves state-of-the-art reconstruction quality while using nearly identical hyperparameters across various inverse problems.

Method: Encode video as a continuous 3D Video Fourier Field (VFF) using Fourier-like sinusoidal basis with coefficients predicted by a neural encoder, allowing arbitrary space-time sampling with analytical Gaussian point spread function.

Result: Substantially improves both spatial and temporal super-resolution, sets new state-of-the-art across multiple benchmarks with sharper and more temporally consistent reconstructions while being computationally more efficient.

Conclusion: Joint modeling with continuous Fourier field representation provides superior video super-resolution performance compared to existing baselines across various upscaling factors.

Abstract: We introduce a novel formulation for continuous space-time video super-resolution. Instead of decoupling the representation of a video sequence into separate spatial and temporal components and relying on brittle, explicit frame warping for motion compensation, we encode video as a continuous, spatio-temporally coherent 3D Video Fourier Field (VFF). That representation offers three key advantages: (1) it enables cheap, flexible sampling at arbitrary locations in space and time; (2) it is able to simultaneously capture fine spatial detail and smooth temporal dynamics; and (3) it offers the possibility to include an analytical, Gaussian point spread function in the sampling to ensure aliasing-free reconstruction at arbitrary scale. The coefficients of the proposed, Fourier-like sinusoidal basis are predicted with a neural encoder with a large spatio-temporal receptive field, conditioned on the low-resolution input video. Through extensive experiments, we show that our joint modeling substantially improves both spatial and temporal super-resolution and sets a new state of the art for multiple benchmarks: across a wide range of upscaling factors, it delivers sharper and temporally more consistent reconstructions than existing baselines, while being computationally more efficient. Project page: https://v3vsr.github.io.

Method: Two-stage framework: 1) Semantic Query-Augmented Fusion (SQAF) enriches CLIP embeddings with MLLM-generated target captions, 2) Efficient Batch Reranking (EBR) uses MLLMs to perform joint visual-semantic reasoning on top candidates presented as image grids.

Result: SQUARE delivers strong performance on four standard CIR benchmarks and maintains high performance even with lightweight pre-trained models.

Conclusion: SQUARE provides a simple yet effective training-free solution for zero-shot CIR that better captures user intent through MLLM-enhanced semantic guidance and reasoning.

Abstract: Composed Image Retrieval (CIR) aims to retrieve target images that preserve the visual content of a reference image while incorporating user-specified textual modifications. Training-free zero-shot CIR (ZS-CIR) approaches, which require no task-specific training or labeled data, are highly desirable, yet accurately capturing user intent remains challenging. In this paper, we present SQUARE, a novel two-stage training-free framework that leverages Multimodal Large Language Models (MLLMs) to enhance ZS-CIR. In the Semantic Query-Augmented Fusion (SQAF) stage, we enrich the query embedding derived from a vision-language model (VLM) such as CLIP with MLLM-generated captions of the target image. These captions provide high-level semantic guidance, enabling the query to better capture the user’s intent and improve global retrieval quality. In the Efficient Batch Reranking (EBR) stage, top-ranked candidates are presented as an image grid with visual marks to the MLLM, which performs joint visual-semantic reasoning across all candidates. Our reranking strategy operates in a single pass and yields more accurate rankings. Experiments show that SQUARE, with its simplicity and effectiveness, delivers strong performance on four standard CIR benchmarks. Notably, it maintains high performance even with lightweight pre-trained, demonstrating its potential applicability.

Method: Built MNAME using a new large-scale human preference dataset containing over 200K preference pairs meticulously annotated by trained experts following a rigorous protocol.

Result: MNAME achieves state-of-the-art human correlation on benchmarks (GenAI-Bench, AURORA-Bench, ImagenHub, BENCHNAME) and successfully selects high-quality subsets from noisy datasets, enabling significant improvement in trained models like Step1X-Edit.

Conclusion: MNAME serves as an effective reward model to scale up high-quality training data for image editing and shows potential for advanced applications like reinforcement learning-based post-training and test-time scaling.

Abstract: Recently, we have witnessed great progress in image editing with natural language instructions. Several closed-source models like GPT-Image-1, Seedream, and Google-Nano-Banana have shown highly promising progress. However, the open-source models are still lagging. The main bottleneck is the lack of a reliable reward model to scale up high-quality synthetic training data. To address this critical bottleneck, we built \mname, trained with our new large-scale human preference dataset, meticulously annotated by trained experts following a rigorous protocol containing over 200K preference pairs. \mname demonstrates superior alignment with human preferences in instruction-guided image editing tasks. Experiments show that \mname achieves state-of-the-art human correlation on established benchmarks such as GenAI-Bench, AURORA-Bench, ImagenHub, and our new \benchname, outperforming a wide range of VLM-as-judge models. Furthermore, we use \mname to select a high-quality subset from the existing noisy ShareGPT-4o-Image dataset. We train Step1X-Edit on the selected subset, which shows significant improvement over training on the full set. This demonstrates \mname’s ability to serve as a reward model to scale up high-quality training data for image editing. Furthermore, its strong alignment suggests potential for advanced applications like reinforcement learning-based post-training and test-time scaling of image editing models. \mname with its training dataset will be released to help the community build more high-quality image editing training datasets.

Method: Proposed TimeScope framework uses progressive reasoning: first identifies coarse-grained temporal scope in long videos, then refines through fine-grained moment partitioning, enhanced by curated ToTG Pile dataset.

Result: Extensive experiments show TimeScope consistently outperforms existing temporal grounding methods and popular MLLMs across various settings.

Conclusion: TimeScope effectively addresses the challenging ToTG problem through progressive temporal grounding, demonstrating superior performance over existing approaches.

Abstract: Identifying key moments in long videos is essential for downstream understanding and reasoning tasks. In this paper, we introduce a new problem, Taskoriented Temporal Grounding ToTG, which aims to localize time intervals containing the necessary information based on a task’s natural description. Along with the definition, we also present ToTG Bench, a comprehensive benchmark for evaluating the performance on ToTG. ToTG is particularly challenging for traditional approaches due to their limited generalizability and difficulty in handling long videos. To address these challenges, we propose TimeScope, a novel framework built upon progressive reasoning. TimeScope first identifies a coarse-grained temporal scope in the long video that likely contains the key moments, and then refines this scope through finegrained moment partitioning. Additionally, we curate a highquality dataset, namely ToTG Pile, to enhance TimeScope’s ability to perform progressive temporal grounding effectively. Extensive experiments demonstrate that TimeScope consistently outperforms both existing temporalgrounding methods and popular MLLMs across various settings, highlighting its effectiveness in addressing this new challenging problem.

Method: Built a 3B parameter agent using diverse GUI data from real and synthetic sources, enhanced with chain-of-thought reasoning, visual tool-use, and reinforcement learning with designed rewards.

Result: Achieved competitive GUI grounding scores: 91.6% on ScreenSpot-V2, 53.3% on ScreenSpot-Pro, 61.2% on OSWorld-G; GUI navigation success rates: 28.0% on AndroidWorld and 19.8% on OSWorld.

Conclusion: The paper shares methods and lessons for developing compact, on-device GUI agents that perform competitively across multiple platforms.

Abstract: Developing autonomous agents that effectively interact with Graphic User Interfaces (GUIs) remains a challenging open problem, especially for small on-device models. In this paper, we present Ferret-UI Lite, a compact, end-to-end GUI agent that operates across diverse platforms, including mobile, web, and desktop. Utilizing techniques optimized for developing small models, we build our 3B Ferret-UI Lite agent through curating a diverse GUI data mixture from real and synthetic sources, strengthening inference-time performance through chain-of-thought reasoning and visual tool-use, and reinforcement learning with designed rewards. Ferret-UI Lite achieves competitive performance with other small-scale GUI agents. In GUI grounding, Ferret-UI Lite attains scores of $91.6%$, $53.3%$, and $61.2%$ on the ScreenSpot-V2, ScreenSpot-Pro, and OSWorld-G benchmarks, respectively. For GUI navigation, Ferret-UI Lite achieves success rates of $28.0%$ on AndroidWorld and $19.8%$ on OSWorld. We share our methods and lessons learned from developing compact, on-device GUI agents.

Method: ScalingAR uses token entropy as a signal and operates at two levels: Profile Level (streams calibrated confidence state by fusing intrinsic/conditional signals) and Policy Level (adaptively terminates low-confidence trajectories and dynamically schedules guidance for appropriate conditioning strength).

Result: Improves base models by 12.5% on GenEval and 15.2% on TIIF-Bench, reduces visual token consumption by 62.0% while outperforming baselines, and enhances robustness by mitigating performance drops by 26.0% in challenging scenarios.

Conclusion: ScalingAR successfully bridges the gap in test-time scaling for next-token prediction autoregressive image generation, demonstrating significant improvements in quality, efficiency, and robustness without requiring early decoding or auxiliary rewards.

Abstract: Test-time scaling (TTS) has demonstrated remarkable success in enhancing large language models, yet its application to next-token prediction (NTP) autoregressive (AR) image generation remains largely uncharted. Existing TTS approaches for visual AR (VAR), which rely on frequent partial decoding and external reward models, are ill-suited for NTP-based image generation due to the inherent incompleteness of intermediate decoding results. To bridge this gap, we introduce ScalingAR, the first TTS framework specifically designed for NTP-based AR image generation that eliminates the need for early decoding or auxiliary rewards. ScalingAR leverages token entropy as a novel signal in visual token generation and operates at two complementary scaling levels: (i) Profile Level, which streams a calibrated confidence state by fusing intrinsic and conditional signals; and (ii) Policy Level, which utilizes this state to adaptively terminate low-confidence trajectories and dynamically schedule guidance for phase-appropriate conditioning strength. Experiments on both general and compositional benchmarks show that ScalingAR (1) improves base models by 12.5% on GenEval and 15.2% on TIIF-Bench, (2) efficiently reduces visual token consumption by 62.0% while outperforming baselines, and (3) successfully enhances robustness, mitigating performance drops by 26.0% in challenging scenarios.

Method: PANDA uses four key capabilities: (1) self-adaptive scene-aware RAG mechanism for anomaly-specific knowledge retrieval and strategy planning, (2) latent anomaly-guided heuristic prompt strategy for enhanced reasoning, (3) progressive reflection mechanism with context-aware tools for iterative decision refinement, and (4) chain-of-memory mechanism for leveraging historical experiences.

Result: Extensive experiments show PANDA achieves state-of-the-art performance in multi-scenario, open-set, and complex scenario settings without training and manual involvement.

Conclusion: PANDA demonstrates generalizable and robust anomaly detection capability, validating its effectiveness as a generalist video anomaly detection system that operates without domain-specific training or human intervention.

Abstract: Video anomaly detection (VAD) is a critical yet challenging task due to the complex and diverse nature of real-world scenarios. Previous methods typically rely on domain-specific training data and manual adjustments when applying to new scenarios and unseen anomaly types, suffering from high labor costs and limited generalization. Therefore, we aim to achieve generalist VAD, i.e., automatically handle any scene and any anomaly types without training data or human involvement. In this work, we propose PANDA, an agentic AI engineer based on MLLMs. Specifically, we achieve PANDA by comprehensively devising four key capabilities: (1) self-adaptive scene-aware strategy planning, (2) goal-driven heuristic reasoning, (3) tool-augmented self-reflection, and (4) self-improving chain-of-memory. Concretely, we develop a self-adaptive scene-aware RAG mechanism, enabling PANDA to retrieve anomaly-specific knowledge for anomaly detection strategy planning. Next, we introduce a latent anomaly-guided heuristic prompt strategy to enhance reasoning precision. Furthermore, PANDA employs a progressive reflection mechanism alongside a suite of context-aware tools to iteratively refine decision-making in complex scenarios. Finally, a chain-of-memory mechanism enables PANDA to leverage historical experiences for continual performance improvement. Extensive experiments demonstrate that PANDA achieves state-of-the-art performance in multi-scenario, open-set, and complex scenario settings without training and manual involvement, validating its generalizable and robust anomaly detection capability. Code is released at https://github.com/showlab/PANDA.

Method: The method includes: (i) a retrieval-based pipeline extracting high-level motion features using video encoder and specialized resamplers; (ii) an in-context learning approach for motion adaptation implemented through a causal transformer architecture; (iii) an attention-based motion injection adapter that integrates transferred motion features into pretrained video diffusion models.

Result: Extensive experiments demonstrate significant improvements across multiple domains and various base models with negligible computational overhead during inference. The modular design enables zero-shot generalization to new domains by simply updating the retrieval database without retraining.

Conclusion: This research enhances video generation systems by enabling effective retrieval and transfer of motion priors, facilitating the synthesis of realistic motion dynamics.

Abstract: Image-to-video generation has made remarkable progress with the advancements in diffusion models, yet generating videos with realistic motion remains highly challenging. This difficulty arises from the complexity of accurately modeling motion, which involves capturing physical constraints, object interactions, and domain-specific dynamics that are not easily generalized across diverse scenarios. To address this, we propose MotionRAG, a retrieval-augmented framework that enhances motion realism by adapting motion priors from relevant reference videos through Context-Aware Motion Adaptation (CAMA). The key technical innovations include: (i) a retrieval-based pipeline extracting high-level motion features using video encoder and specialized resamplers to distill semantic motion representations; (ii) an in-context learning approach for motion adaptation implemented through a causal transformer architecture; (iii) an attention-based motion injection adapter that seamlessly integrates transferred motion features into pretrained video diffusion models. Extensive experiments demonstrate that our method achieves significant improvements across multiple domains and various base models, all with negligible computational overhead during inference. Furthermore, our modular design enables zero-shot generalization to new domains by simply updating the retrieval database without retraining any components. This research enhances the core capability of video generation systems by enabling the effective retrieval and transfer of motion priors, facilitating the synthesis of realistic motion dynamics.

Method: Uses two image-difficulty measures (high-frequency index and rotation-invariant edge index) to predict which test images will show significant visual differences between models, and develops an evaluation method that reflects these visual differences in objective measures.

Result: Experimental results demonstrate the effectiveness of the proposed image-difficulty measures and evaluation methodology in better differentiating SISR models.

Conclusion: The proposed difficulty-aware performance evaluation procedures provide more meaningful differentiation between super-resolution models than traditional average score methods.

Abstract: Image super-resolution models are commonly evaluated by average scores (over some benchmark test sets), which fail to reflect the performance of these models on images of varying difficulty and that some models generate artifacts on certain difficult images, which is not reflected by the average scores. We propose difficulty-aware performance evaluation procedures to better differentiate between SISR models that produce visually different results on some images but yield close average performance scores over the entire test set. In particular, we propose two image-difficulty measures, the high-frequency index and rotation-invariant edge index, to predict those test images, where a model would yield significantly better visual results over another model, and an evaluation method where these visual differences are reflected on objective measures. Experimental results demonstrate the effectiveness of the proposed image-difficulty measures and evaluation methodology.

Method: Progressive three-stage framework: CENet for coarse extraction using HA-UNet with channel attention and windowed spatial transformers; SFNet with HDMamba for joint spatial and wavelet domain modeling; RNet for fine-grained structure recovery via original-resolution subnetwork.

Result: The method achieves competitive results on multiple datasets against recent deraining methods, learning high-frequency rain characteristics while preserving structural details and maintaining global context.

Conclusion: PRISM effectively addresses the limitations of single-scale models through its progressive multi-stage approach with hybrid attention and state-space modeling, leading to improved image deraining quality.

Abstract: Image deraining is an essential vision technique that removes rain streaks and water droplets, enhancing clarity for critical vision tasks like autonomous driving. However, current single-scale models struggle with fine-grained recovery and global consistency. To address this challenge, we propose Progressive Rain removal with Integrated State-space Modeling (PRISM), a progressive three-stage framework: Coarse Extraction Network (CENet), Frequency Fusion Network (SFNet), and Refine Network (RNet). Specifically, CENet and SFNet utilize a novel Hybrid Attention UNet (HA-UNet) for multi-scale feature aggregation by combining channel attention with windowed spatial transformers. Moreover, we propose Hybrid Domain Mamba (HDMamba) for SFNet to jointly model spatial semantics and wavelet domain characteristics. Finally, RNet recovers the fine-grained structures via an original-resolution subnetwork. Our model learns high-frequency rain characteristics while preserving structural details and maintaining global context, leading to improved image quality. Our method achieves competitive results on multiple datasets against recent deraining methods.

Method: Proposes QuaRTZ (Quantization via Residual Truncation and Zero Suppression) - uses 8-bit min-max quantization for outliers and compresses to 4 bits via leading-zero suppression to retain least significant bits (LSBs).

Result: Achieves FID of 6.98 on FLUX.1-schnell, outperforming SVDQuant which requires auxiliary FP16 branches. Reduces rounding errors and improves quantization efficiency.

Conclusion: QuaRTZ effectively balances outlier preservation and LSB precision, demonstrating generalizability across diverse activation distributions for 4-bit diffusion model quantization.

Abstract: Diffusion models achieve high-quality image generation but face deployment challenges due to their high computational requirements. Although 8-bit outlier-aware post-training quantization (PTQ) matches full-precision performance, extending PTQ to 4 bits remains challenging. Larger step sizes in 4-bit quantization amplify rounding errors in dense, low-magnitude activations, leading to the loss of fine-grained textures. We hypothesize that not only outliers but also small activations are critical for texture fidelity. To this end, we propose Quantization via Residual Truncation and Zero Suppression (QuaRTZ), a 4-bit PTQ scheme for diffusion models. QuaRTZ applies 8-bit min-max quantization for outlier handling and compresses to 4 bits via leading-zero suppression to retain LSBs, thereby preserving texture details. Our approach reduces rounding errors and improves quantization efficiency by balancing outlier preservation and LSB precision. Both theoretical derivations and empirical evaluations demonstrate the generalizability of QuaRTZ across diverse activation distributions. Notably, 4-bit QuaRTZ achieves an FID of 6.98 on FLUX.1-schnell, outperforming SVDQuant that requires auxiliary FP16 branches.

Method: Uses 11 synchronized cameras for 360° vehicle capture, specialized deep learning modules (YOLOv8, EfficientNet, Gemini-1.5 Flash, YOLOv8-Seg) for different tasks, view-aware fusion layer, and VIN-conditioned rule engine for comparison against expected manifest.

Result: Achieves 93% verification accuracy, 86% defect-detection recall, and sustains 3.3 vehicles/min, outperforming single-view or no segmentation baselines significantly.

Conclusion: This is the first publicly reported system that unifies multi-camera feature validation with defect detection in a deployable automotive industry setting.

Abstract: Ensuring that every vehicle leaving a modern production line is built to the correct \emph{variant} specification and is free from visible defects is an increasingly complex challenge. We present the \textbf{Automated Vehicle Inspection (AVI)} platform, an end-to-end, \emph{multi-view} perception system that couples deep-learning detectors with a semantic rule engine to deliver \emph{variant-aware} quality control in real time. Eleven synchronized cameras capture a full 360{\deg} sweep of each vehicle; task-specific views are then routed to specialised modules: YOLOv8 for part detection, EfficientNet for ICE/EV classification, Gemini-1.5 Flash for mascot OCR, and YOLOv8-Seg for scratch-and-dent segmentation. A view-aware fusion layer standardises evidence, while a VIN-conditioned rule engine compares detected features against the expected manifest, producing an interpretable pass/fail report in (\approx! 300,\text{ms}). On a mixed data set of Original Equipment Manufacturer(OEM) vehicle data sets of four distinct models plus public scratch/dent images, AVI achieves \textbf{ 93 %} verification accuracy, \textbf{86 %} defect-detection recall, and sustains (\mathbf{3.3}) vehicles/min, surpassing single-view or no segmentation baselines by large margins. To our knowledge, this is the first publicly reported system that unifies multi-camera feature validation with defect detection in a deployable automotive setting in industry.

Method: Uses a Transformer backbone with two pathways: geometry predictions with self-attention for geometric fidelity, and style injection via global cross-attention for visual consistency. Includes a voxel-based 3D style loss to align scene features with style statistics.

Result: Experiments show Stylos delivers high-quality zero-shot stylization, demonstrating effectiveness of global style-content coupling, the 3D style loss, and scalability from single view to large-scale multi-view settings.

Conclusion: Stylos successfully achieves view-consistent 3D stylization while preserving geometry, highlighting the framework’s effectiveness and scalability across different input settings.

Abstract: We present Stylos, a single-forward 3D Gaussian framework for 3D style transfer that operates on unposed content, from a single image to a multi-view collection, conditioned on a separate reference style image. Stylos synthesizes a stylized 3D Gaussian scene without per-scene optimization or precomputed poses, achieving geometry-aware, view-consistent stylization that generalizes to unseen categories, scenes, and styles. At its core, Stylos adopts a Transformer backbone with two pathways: geometry predictions retain self-attention to preserve geometric fidelity, while style is injected via global cross-attention to enforce visual consistency across views. With the addition of a voxel-based 3D style loss that aligns aggregated scene features to style statistics, Stylos enforces view-consistent stylization while preserving geometry. Experiments across multiple datasets demonstrate that Stylos delivers high-quality zero-shot stylization, highlighting the effectiveness of global style-content coupling, the proposed 3D style loss, and the scalability of our framework from single view to large-scale multi-view settings.

Method: Proposes ASGRA framework that converts images into Scene Graphs and uses Graph Attention Networks for inference, directly modeling interactions between scene components through structured graph representations.

Result: Achieves 81.27% balanced accuracy on Places8 dataset (surpassing image-based methods) and 74.27% balanced accuracy in real-world CSAI evaluation with law enforcement.

Conclusion: Structured scene representations using scene graphs and graph attention networks provide a robust paradigm for indoor scene classification and CSAI classification, offering both explainability and privacy preservation benefits.

Abstract: Indoor scene classification is a critical task in computer vision, with wide-ranging applications that go from robotics to sensitive content analysis, such as child sexual abuse imagery (CSAI) classification. The problem is particularly challenging due to the intricate relationships between objects and complex spatial layouts. In this work, we propose the Attention over Scene Graphs for Sensitive Content Analysis (ASGRA), a novel framework that operates on structured graph representations instead of raw pixels. By first converting images into Scene Graphs and then employing a Graph Attention Network for inference, ASGRA directly models the interactions between a scene’s components. This approach offers two key benefits: (i) inherent explainability via object and relationship identification, and (ii) privacy preservation, enabling model training without direct access to sensitive images. On Places8, we achieve 81.27% balanced accuracy, surpassing image-based methods. Real-world CSAI evaluation with law enforcement yields 74.27% balanced accuracy. Our results establish structured scene representations as a robust paradigm for indoor scene classification and CSAI classification. Code is publicly available at https://github.com/tutuzeraa/ASGRA.

Method: Proposes UML-CoT framework using UML class diagrams for compositional object semantics and activity diagrams for procedural control flow. Uses three-stage training pipeline with supervised fine-tuning and Group Relative Policy Optimization, including reward learning from answer-only data.

Result: Evaluated on MRoom-30k benchmark for cluttered room-cleaning scenarios. UML-CoT outperforms unstructured CoTs in interpretability, planning coherence, and execution success.

Conclusion: UML provides a more expressive and actionable structured reasoning formalism compared to traditional CoT approaches, enabling better performance in embodied reasoning tasks.

Abstract: Chain-of-Thought (CoT) prompting improves reasoning in large language models (LLMs), but its reliance on unstructured text limits interpretability and executability in embodied tasks. Prior work has explored structured CoTs using scene or logic graphs, yet these remain fundamentally limited: they model only low-order relations, lack constructs like inheritance or behavioral abstraction, and provide no standardized semantics for sequential or conditional planning. We propose UML-CoT, a structured reasoning and planning framework that leverages Unified Modeling Language (UML) to generate symbolic CoTs and executable action plans. UML class diagrams capture compositional object semantics, while activity diagrams model procedural control flow. Our three-stage training pipeline combines supervised fine-tuning with Group Relative Policy Optimization (GRPO), including reward learning from answer-only data. We evaluate UML-CoT on MRoom-30k, a new benchmark of cluttered room-cleaning scenarios. UML-CoT outperforms unstructured CoTs in interpretability, planning coherence, and execution success, highlighting UML as a more expressive and actionable structured reasoning formalism.

Method: Proposed a lightweight CBAM-CNN model with Convolutional Block Attention Module to focus on important spatial and channel features. Used enriched dataset of 10,185 images across three classes (Healthy Leaf, Leaf Rot, Leaf Spot) with class balance.

Result: Achieved 97% precision, 94% recall, 95% F1 score, and 95.58% accuracy on test set. Outperformed traditional pre-trained CNN models. Used Grad-CAM for explainable AI visualization.

Conclusion: The lightweight CBAM-CNN model provides effective and efficient betel leaf disease classification with strong performance and interpretability through attention mechanisms.

Abstract: Betel leaf is an important crop because of its economic advantages and widespread use. Its betel vines are susceptible to a number of illnesses that are commonly referred to as betel leaf disease. Plant diseases are the largest threat to the food supply’s security, and they are challenging to identify in time to stop possible financial damage. Interestingly, artificial intelligence can leave a big mark on the betel leaf industry since it helps with output growth by forecasting sickness. This paper presents a lightweight CBAM-CNN model with just 2.13 million parameters (8.13 MB), incorporating CBAM (Convolutional Block Attention Module) to improve feature emphasis without depending on heavy pre-trained networks. The model’s capacity to discern minute variations among leaf disease classes is improved by the integrated attention mechanism, which allows it to adaptively focus on significant spatial and channel-wise information. In order to ensure class balance and diversity for efficient model training and validation, this work makes use of an enriched dataset of 10,185 images divided into three categories: Healthy Leaf, Leaf Rot, and Leaf Spot. The proposed model achieved a precision of 97%, recall of 94%, and F1 score of 95%, and 95.58% accuracy on the test set demonstrating strong and balanced classification performance outperforming traditional pre trained CNN models. The model’s focus regions were visualized and interpreted using Grad-CAM (Gradient-weighted Class Activation Mapping), an explainable AI technique.

Method: Proposed perceptual influence metric to quantify perceptual loss contribution and developed a systematic framework to assess loss design choices through experimentation.

Result: Widely used perceptual loss configurations underperform compared to better-designed alternatives, with optimized designs leading to significant improvements in noise reduction and structural fidelity.

Conclusion: Provides objective guidelines for effective perceptual loss use in LDCT denoising, supported by statistical analysis, enabling better performance without architectural changes.

Abstract: Perceptual losses have emerged as powerful tools for training networks to enhance Low-Dose Computed Tomography (LDCT) images, offering an alternative to traditional pixel-wise losses such as Mean Squared Error, which often lead to over-smoothed reconstructions and loss of clinically relevant details in LDCT images. The perceptual losses operate in a latent feature space defined by a pretrained encoder and aim to preserve semantic content by comparing high-level features rather than raw pixel values. However, the design of perceptual losses involves critical yet underexplored decisions, including the feature representation level, the dataset used to pretrain the encoder, and the relative importance assigned to the perceptual component during optimization. In this work, we introduce the concept of perceptual influence (a metric that quantifies the relative contribution of the perceptual loss term to the total loss) and propose a principled framework to assess the impact of the loss design choices on the model training performance. Through systematic experimentation, we show that the widely used configurations in the literature to set up a perceptual loss underperform compared to better-designed alternatives. Our findings show that better perceptual loss designs lead to significant improvements in noise reduction and structural fidelity of reconstructed CT images, without requiring any changes to the network architecture. We also provide objective guidelines, supported by statistical analysis, to inform the effective use of perceptual losses in LDCT denoising. Our source code is available at https://github.com/vngabriel/perceptual-influence.

Method: Uses a diffusion-based posterior sampler with a reformulated likelihood score in a contrastive embedding space. The embedding brings compatible floorplan-trajectory pairs close together and pushes mismatched pairs apart, creating a smoother optimization landscape.

Result: CoGuide produces more consistent floorplans from trajectories and is more robust than differentiable-planner baselines and guided-diffusion methods across extensive experiments.

Conclusion: The surrogate likelihood score in the contrastive embedding space effectively approximates the true likelihood, enabling successful steering of the denoising process toward the posterior distribution for floorplan reconstruction.

Abstract: We consider the inverse problem of reconstructing the spatial layout of a place, a home floorplan for example, from a user`s movements inside that layout. Direct inversion is ill-posed since many floorplans can explain the same movement trajectories. We adopt a diffusion-based posterior sampler to generate layouts consistent with the measurements. While active research is in progress on generative inverse solvers, we find that the forward operator in our problem poses new challenges. The path-planning process inside a floorplan is a non-invertible, non-differentiable function, and causes instability while optimizing using the likelihood score. We break-away from existing approaches and reformulate the likelihood score in a smoother embedding space. The embedding space is trained with a contrastive loss which brings compatible floorplans and trajectories close to each other, while pushing mismatched pairs far apart. We show that a surrogate form of the likelihood score in this embedding space is a valid approximation of the true likelihood score, making it possible to steer the denoising process towards the posterior. Across extensive experiments, our model CoGuide produces more consistent floorplans from trajectories, and is more robust than differentiable-planner baselines and guided-diffusion methods.

Method: A post-training pipeline with curriculum-based supervised fine-tuning and offline on-policy knowledge distillation to enhance small model accuracy.

Result: The instruction-tuned model achieves state-of-the-art performance among billion-parameter models with strong generalization under hardware constraints and competitive accuracy across tasks.

Conclusion: Provides a practical and efficient solution for developing high-performance language models on Ascend edge devices.

Abstract: The rapid advancement of large language models (LLMs) has significantly advanced the capabilities of artificial intelligence across various domains. However, their massive scale and high computational costs render them unsuitable for direct deployment in resource-constrained edge environments. This creates a critical need for high-performance small models that can operate efficiently at the edge. Yet, after pre-training alone, these smaller models often fail to meet the performance requirements of complex tasks. To bridge this gap, we introduce a systematic post-training pipeline that efficiently enhances small model accuracy. Our post training pipeline consists of curriculum-based supervised fine-tuning (SFT) and offline on-policy knowledge distillation. The resulting instruction-tuned model achieves state-of-the-art performance among billion-parameter models, demonstrating strong generalization under strict hardware constraints while maintaining competitive accuracy across a variety of tasks. This work provides a practical and efficient solution for developing high-performance language models on Ascend edge devices.

Method: Three key components: 1) Simulation-based training using synthetic datasets for robust model training, 2) Learnable-parameter-free anomaly detection to remove erroneous dToF measurements, 3) Tailored depth completion network integrating RGB images and monocular depth priors.

Result: Achieved state-of-the-art performance with 22% RMSE and 11% Rel improvement on ZJU-L5 dataset, 37% improvement in mirror regions on Mirror3D-NYU dataset, and 22% average gain over RealSense L515 measurements on Hammer dataset.

Conclusion: The proposed framework significantly enhances depth completion robustness, enabling low-cost dToF sensors to outperform higher-end devices and demonstrating strong generalizability across diverse real-world scenarios.

Abstract: Depth enhancement, which converts raw dToF signals into dense depth maps using RGB guidance, is crucial for improving depth perception in high-precision tasks such as 3D reconstruction and SLAM. However, existing methods often assume ideal dToF inputs and perfect dToF-RGB alignment, overlooking calibration errors and anomalies, thus limiting real-world applicability. This work systematically analyzes the noise characteristics of real-world lightweight dToF sensors and proposes a practical and novel depth completion framework, DEPTHOR++, which enhances robustness to noisy dToF inputs from three key aspects. First, we introduce a simulation method based on synthetic datasets to generate realistic training samples for robust model training. Second, we propose a learnable-parameter-free anomaly detection mechanism to identify and remove erroneous dToF measurements, preventing misleading propagation during completion. Third, we design a depth completion network tailored to noisy dToF inputs, which integrates RGB images and pre-trained monocular depth estimation priors to improve depth recovery in challenging regions. On the ZJU-L5 dataset and real-world samples, our training strategy significantly boosts existing depth completion models, with our model achieving state-of-the-art performance, improving RMSE and Rel by 22% and 11% on average. On the Mirror3D-NYU dataset, by incorporating the anomaly detection method, our model improves upon the previous SOTA by 37% in mirror regions. On the Hammer dataset, using simulated low-cost dToF data from RealSense L515, our method surpasses the L515 measurements with an average gain of 22%, demonstrating its potential to enable low-cost sensors to outperform higher-end devices. Qualitative results across diverse real-world datasets further validate the effectiveness and generalizability of our approach.

Method: Developed four hierarchical taxonomies (Setup, Event, Lighting, Camera) with 76 control nodes, created professional benchmarks, automated prompt categorization pipeline, and conducted large-scale human studies with 80+ film professionals evaluating 20K videos from 10+ models.

Result: Current models show significant gaps, particularly in Events and Camera-related controls. Trained an automatic evaluator that outperforms existing zero-shot baselines in aligning with expert annotations.

Conclusion: SCINE is the first approach to situate professional video generation within video generative models, providing structured evaluation pipelines and detailed analyses to guide future research in cinematic controls.

Abstract: Recent advances in video generation have enabled high-fidelity video synthesis from user provided prompts. However, existing models and benchmarks fail to capture the complexity and requirements of professional video generation. Towards that goal, we introduce Stable Cinemetrics, a structured evaluation framework that formalizes filmmaking controls into four disentangled, hierarchical taxonomies: Setup, Event, Lighting, and Camera. Together, these taxonomies define 76 fine-grained control nodes grounded in industry practices. Using these taxonomies, we construct a benchmark of prompts aligned with professional use cases and develop an automated pipeline for prompt categorization and question generation, enabling independent evaluation of each control dimension. We conduct a large-scale human study spanning 10+ models and 20K videos, annotated by a pool of 80+ film professionals. Our analysis, both coarse and fine-grained reveal that even the strongest current models exhibit significant gaps, particularly in Events and Camera-related controls. To enable scalable evaluation, we train an automatic evaluator, a vision-language model aligned with expert annotations that outperforms existing zero-shot baselines. SCINE is the first approach to situate professional video generation within the landscape of video generative models, introducing taxonomies centered around cinematic controls and supporting them with structured evaluation pipelines and detailed analyses to guide future research.

Method: Learn machine learning models using available ground-truth data to automate or optimize proofreading workflows. Validated on Drosophila male central nervous system reconstruction.

Result: Achieved 80% cost reduction while maintaining 90% value of guided proofreading. Automatically merged 200,000 fragments (equivalent to 4 proofreader years), increasing connectivity completion by 1.3 percentage points.

Conclusion: Machine learning can significantly reduce the manual annotation burden in EM connectomics, enabling scaling of connectome reconstructions and making comparative connectomics more feasible.

Abstract: Producing connectomes from electron microscopy (EM) images has historically required a great deal of human proofreading effort. This manual annotation cost is the current bottleneck in scaling EM connectomics, for example, in making larger connectome reconstructions feasible, or in enabling comparative connectomics where multiple related reconstructions are produced. In this work, we propose using the available ground-truth data generated by this manual annotation effort to learn a machine learning model to automate or optimize parts of the required proofreading workflows. We validate our approach on a recent complete reconstruction of the \emph{Drosophila} male central nervous system. We first show our method would allow for obtaining 90% of the value of a guided proofreading workflow while reducing required cost by 80%. We then demonstrate a second application for automatically merging many segmentation fragments to proofread neurons. Our system is able to automatically attach 200 thousand fragments, equivalent to four proofreader years of manual work, and increasing the connectivity completion rate of the connectome by 1.3% points.

Method: Uses diffusion transformer framework with simulation-based data generation. Introduces stacking constraint based on photographic principle that different focus planes can be linearly blended into multi-focus images.

Result: Extensive experiments show DiffCamera supports stable refocusing across diverse scenes, providing unprecedented control over depth-of-field adjustments.

Conclusion: The proposed method successfully enables flexible image refocusing with precise depth-of-field manipulation, advancing photography and generative AI applications.

Abstract: The depth-of-field (DoF) effect, which introduces aesthetically pleasing blur, enhances photographic quality but is fixed and difficult to modify once the image has been created. This becomes problematic when the applied blur is undesirable~(e.g., the subject is out of focus). To address this, we propose DiffCamera, a model that enables flexible refocusing of a created image conditioned on an arbitrary new focus point and a blur level. Specifically, we design a diffusion transformer framework for refocusing learning. However, the training requires pairs of data with different focus planes and bokeh levels in the same scene, which are hard to acquire. To overcome this limitation, we develop a simulation-based pipeline to generate large-scale image pairs with varying focus planes and bokeh levels. With the simulated data, we find that training with only a vanilla diffusion objective often leads to incorrect DoF behaviors due to the complexity of the task. This requires a stronger constraint during training. Inspired by the photographic principle that photos of different focus planes can be linearly blended into a multi-focus image, we propose a stacking constraint during training to enforce precise DoF manipulation. This constraint enhances model training by imposing physically grounded refocusing behavior that the focusing results should be faithfully aligned with the scene structure and the camera conditions so that they can be combined into the correct multi-focus image. We also construct a benchmark to evaluate the effectiveness of our refocusing model. Extensive experiments demonstrate that DiffCamera supports stable refocusing across a wide range of scenes, providing unprecedented control over DoF adjustments for photography and generative AI applications.

Method: SAGANet leverages visual segmentation maps along with video and textual cues to enable controllable audio generation, providing fine-grained and visually localized control over sound synthesis.

Result: The method demonstrates substantial improvements over current state-of-the-art methods and sets a new standard for controllable, high-fidelity Foley synthesis.

Conclusion: SAGANet addresses the gap in precise audio control for Foley workflows through segmentation-aware audio generation, supported by the new Segmented Music Solos benchmark dataset.

Abstract: Existing multimodal audio generation models often lack precise user control, which limits their applicability in professional Foley workflows. In particular, these models focus on the entire video and do not provide precise methods for prioritizing a specific object within a scene, generating unnecessary background sounds, or focusing on the wrong objects. To address this gap, we introduce the novel task of video object segmentation-aware audio generation, which explicitly conditions sound synthesis on object-level segmentation maps. We present SAGANet, a new multimodal generative model that enables controllable audio generation by leveraging visual segmentation masks along with video and textual cues. Our model provides users with fine-grained and visually localized control over audio generation. To support this task and further research on segmentation-aware Foley, we propose Segmented Music Solos, a benchmark dataset of musical instrument performance videos with segmentation information. Our method demonstrates substantial improvements over current state-of-the-art methods and sets a new standard for controllable, high-fidelity Foley synthesis. Code, samples, and Segmented Music Solos are available at https://saganet.notion.site

Method: Hybrid framework integrating VGG19 deep features with SIFT and ORB handcrafted features, followed by K-means clustering and UMAP dimensionality reduction.

Result: Achieved 83.3% classification accuracy on facial expression recognition dataset, with UMAP identified as the most effective dimensionality reduction technique.

Conclusion: Dimensionality reduction is crucial not just as preprocessing but as an essential component for improving feature quality and classification performance in facial expression recognition.

Abstract: Facial expression classification remains a challenging task due to the high dimensionality and inherent complexity of facial image data. This paper presents Hy-Facial, a hybrid feature extraction framework that integrates both deep learning and traditional image processing techniques, complemented by a systematic investigation of dimensionality reduction strategies. The proposed method fuses deep features extracted from the Visual Geometry Group 19-layer network (VGG19) with handcrafted local descriptors and the scale-invariant feature transform (SIFT) and Oriented FAST and Rotated BRIEF (ORB) algorithms, to obtain rich and diverse image representations. To mitigate feature redundancy and reduce computational complexity, we conduct a comprehensive evaluation of dimensionality reduction techniques and feature extraction. Among these, UMAP is identified as the most effective, preserving both local and global structures of the high-dimensional feature space. The Hy-Facial pipeline integrated VGG19, SIFT, and ORB for feature extraction, followed by K-means clustering and UMAP for dimensionality reduction, resulting in a classification accuracy of 83. 3% in the facial expression recognition (FER) dataset. These findings underscore the pivotal role of dimensionality reduction not only as a pre-processing step but as an essential component in improving feature quality and overall classification performance.

Method: Proposes DA² with two key components: 1) Data curation engine generating ~543K panoramic RGB-depth pairs from perspective data (total ~607K), 2) SphereViT that explicitly uses spherical coordinates to enforce spherical geometric consistency in features.

Result: Achieves state-of-the-art performance with 38% average improvement on AbsRel over strongest zero-shot baselines. Surprisingly outperforms prior in-domain methods and shows higher efficiency than fusion-based approaches.

Conclusion: DA² demonstrates superior zero-shot generalization and efficiency as an end-to-end solution, with both code and curated panoramic data to be released.

Abstract: Panorama has a full FoV (360$^\circ\times$180$^\circ$), offering a more complete visual description than perspective images. Thanks to this characteristic, panoramic depth estimation is gaining increasing traction in 3D vision. However, due to the scarcity of panoramic data, previous methods are often restricted to in-domain settings, leading to poor zero-shot generalization. Furthermore, due to the spherical distortions inherent in panoramas, many approaches rely on perspective splitting (e.g., cubemaps), which leads to suboptimal efficiency. To address these challenges, we propose $\textbf{DA}$$^{\textbf{2}}$: $\textbf{D}$epth $\textbf{A}$nything in $\textbf{A}$ny $\textbf{D}$irection, an accurate, zero-shot generalizable, and fully end-to-end panoramic depth estimator. Specifically, for scaling up panoramic data, we introduce a data curation engine for generating high-quality panoramic depth data from perspective, and create $\sim$543K panoramic RGB-depth pairs, bringing the total to $\sim$607K. To further mitigate the spherical distortions, we present SphereViT, which explicitly leverages spherical coordinates to enforce the spherical geometric consistency in panoramic image features, yielding improved performance. A comprehensive benchmark on multiple datasets clearly demonstrates DA$^{2}$’s SoTA performance, with an average 38% improvement on AbsRel over the strongest zero-shot baseline. Surprisingly, DA$^{2}$ even outperforms prior in-domain methods, highlighting its superior zero-shot generalization. Moreover, as an end-to-end solution, DA$^{2}$ exhibits much higher efficiency over fusion-based approaches. Both the code and the curated panoramic data will be released. Project page: https://depth-any-in-any-dir.github.io/.

Method: Predicts per-pixel 3D point maps, normals, and body correspondences, then uses occlusion-aware Poisson reconstruction to recover complete geometry. Aligns with SMPL-X body model and initializes Gaussian splats for rendering.

Result: State-of-the-art performance: 18-23% improvement in Chamfer Distance for clothed-mesh reconstruction, 6-27% drop in PA-V2V for SMPL-X estimation, 15-27% decrease in LPIPS for novel-view synthesis across multiple datasets.

Conclusion: Feed-forward transformers can serve as scalable models for robust human reconstruction in real-world settings, achieving strong performance despite being trained on only 2.3K synthetic scans.

Abstract: We introduce HART, a unified framework for sparse-view human reconstruction. Given a small set of uncalibrated RGB images of a person as input, it outputs a watertight clothed mesh, the aligned SMPL-X body mesh, and a Gaussian-splat representation for photorealistic novel-view rendering. Prior methods for clothed human reconstruction either optimize parametric templates, which overlook loose garments and human-object interactions, or train implicit functions under simplified camera assumptions, limiting applicability in real scenes. In contrast, HART predicts per-pixel 3D point maps, normals, and body correspondences, and employs an occlusion-aware Poisson reconstruction to recover complete geometry, even in self-occluded regions. These predictions also align with a parametric SMPL-X body model, ensuring that reconstructed geometry remains consistent with human structure while capturing loose clothing and interactions. These human-aligned meshes initialize Gaussian splats to further enable sparse-view rendering. While trained on only 2.3K synthetic scans, HART achieves state-of-the-art results: Chamfer Distance improves by 18-23 percent for clothed-mesh reconstruction, PA-V2V drops by 6-27 percent for SMPL-X estimation, LPIPS decreases by 15-27 percent for novel-view synthesis on a wide range of datasets. These results suggest that feed-forward transformers can serve as a scalable model for robust human reconstruction in real-world settings. Code and models will be released.

Method: The authors propose a SIM(3)-equivariant shape completion network with modular layers that successively canonicalize features, reason over similarity-invariant geometry, and restore the original frame. This ensures the model remains agnostic to pose and scale.

Result: The model outperforms both equivariant and augmentation baselines on PCN benchmark under de-biased evaluation. It sets new cross-domain records: lowering minimal matching distance on KITTI by 17% and Chamfer distance on OmniObject3D by 14%. Surprisingly, it even outperforms competitors under their biased settings.

Conclusion: Full SIM(3) equivariance is established as an effective route to truly generalizable shape completion, providing robust performance across different domains and evaluation protocols.

Abstract: 3D shape completion methods typically assume scans are pre-aligned to a canonical frame. This leaks pose and scale cues that networks may exploit to memorize absolute positions rather than inferring intrinsic geometry. When such alignment is absent in real data, performance collapses. We argue that robust generalization demands architectural equivariance to the similarity group, SIM(3), so the model remains agnostic to pose and scale. Following this principle, we introduce the first SIM(3)-equivariant shape completion network, whose modular layers successively canonicalize features, reason over similarity-invariant geometry, and restore the original frame. Under a de-biased evaluation protocol that removes the hidden cues, our model outperforms both equivariant and augmentation baselines on the PCN benchmark. It also sets new cross-domain records on real driving and indoor scans, lowering minimal matching distance on KITTI by 17% and Chamfer distance $\ell1$ on OmniObject3D by 14%. Perhaps surprisingly, ours under the stricter protocol still outperforms competitors under their biased settings. These results establish full SIM(3) equivariance as an effective route to truly generalizable shape completion. Project page: https://sime-completion.github.io.

Method: Recorded hours of trajectories through a city center using glasses-like devices with various sensors, leveraging surveying tools to obtain control points as indirect pose annotations that are metric and centimeter-accurate at city scale.

Result: State-of-the-art academic SLAM systems show poor robustness to the challenging scenarios in the dataset, and specific problematic components were identified. Tracks with varying difficulty levels were designed for comprehensive evaluation.

Conclusion: The introduced dataset and benchmark enable proper evaluation of SLAM systems for egocentric applications, revealing limitations in current approaches and providing tools for in-depth analysis of less mature methods.

Abstract: Precise 6-DoF simultaneous localization and mapping (SLAM) from onboard sensors is critical for wearable devices capturing egocentric data, which exhibits specific challenges, such as a wider diversity of motions and viewpoints, prevalent dynamic visual content, or long sessions affected by time-varying sensor calibration. While recent progress on SLAM has been swift, academic research is still driven by benchmarks that do not reflect these challenges or do not offer sufficiently accurate ground truth poses. In this paper, we introduce a new dataset and benchmark for visual-inertial SLAM with egocentric, multi-modal data. We record hours and kilometers of trajectories through a city center with glasses-like devices equipped with various sensors. We leverage surveying tools to obtain control points as indirect pose annotations that are metric, centimeter-accurate, and available at city scale. This makes it possible to evaluate extreme trajectories that involve walking at night or traveling in a vehicle. We show that state-of-the-art systems developed by academia are not robust to these challenges and we identify components that are responsible for this. In addition, we design tracks with different levels of difficulty to ease in-depth analysis and evaluation of less mature approaches. The dataset and benchmark are available at https://www.lamaria.ethz.ch.

Method: Proposes a three-stage progressive training: 1) Connect VLM to lightweight diffusion head via multimodal kontext tokens, 2) Scale to large pre-trained diffusion model for enhanced realism, 3) Add low-level image encoder for improved fidelity and instruction tuning. Uses comprehensive data pipeline with real, synthetic, and open-source datasets.

Result: The approach matches strong unified baselines and outperforms task-specific state-of-the-art methods in several cases across image generation, instruction-driven editing, customized generation, and multi-subject composition tasks.

Conclusion: Query-Kontext effectively separates multimodal generative reasoning from visual synthesis, enabling VLMs to handle complex reasoning while diffusion models focus on high-quality generation, leading to superior performance across diverse multimodal tasks.

Abstract: Unified Multimodal Models (UMMs) have demonstrated remarkable performance in text-to-image generation (T2I) and editing (TI2I), whether instantiated as assembled unified frameworks which couple powerful vision-language model (VLM) with diffusion-based generator, or as naive Unified Multimodal Models with an early fusion of understanding and generation modalities. We contend that in current unified frameworks, the crucial capability of multimodal generative reasoning which encompasses instruction understanding, grounding, and image referring for identity preservation and faithful reconstruction, is intrinsically entangled with high-fidelity synthesis. In this work, we introduce Query-Kontext, a novel approach that bridges the VLM and diffusion model via a multimodal ``kontext’’ composed of semantic cues and coarse-grained image conditions encoded from multimodal inputs. This design delegates the complex ability of multimodal generative reasoning to powerful VLM while reserving diffusion model’s role for high-quality visual synthesis. To achieve this, we propose a three-stage progressive training strategy. First, we connect the VLM to a lightweight diffusion head via multimodal kontext tokens to unleash the VLM’s generative reasoning ability. Second, we scale this head to a large, pre-trained diffusion model to enhance visual detail and realism. Finally, we introduce a low-level image encoder to improve image fidelity and perform instruction tuning on downstream tasks. Furthermore, we build a comprehensive data pipeline integrating real, synthetic, and open-source datasets, covering diverse multimodal reference-to-image scenarios, including image generation, instruction-driven editing, customized generation, and multi-subject composition. Experiments show that our approach matches strong unified baselines and even outperforms task-specific state-of-the-art methods in several cases.

Method: Stitch generates individual objects within designated bounding boxes and seamlessly stitches them together by leveraging targeted attention heads that can isolate and cut out individual objects mid-generation without fully completing the image.

Result: Stitch consistently enhances base models, improving FLUX by 218% on GenEval’s Position task and by 206% on PosEval. It achieves state-of-the-art results with Qwen-Image on PosEval, improving over previous models by 54%.

Conclusion: Stitch effectively integrates position control into leading T2I models training-free, demonstrating significant improvements in spatial relationship accuracy while maintaining visual quality, and shows that current models still have substantial room for improvement in position-based generation.

Abstract: Text-to-Image (T2I) generation models have advanced rapidly in recent years, but accurately capturing spatial relationships like “above” or “to the right of” poses a persistent challenge. Earlier methods improved spatial relationship following with external position control. However, as architectures evolved to enhance image quality, these techniques became incompatible with modern models. We propose Stitch, a training-free method for incorporating external position control into Multi-Modal Diffusion Transformers (MMDiT) via automatically-generated bounding boxes. Stitch produces images that are both spatially accurate and visually appealing by generating individual objects within designated bounding boxes and seamlessly stitching them together. We find that targeted attention heads capture the information necessary to isolate and cut out individual objects mid-generation, without needing to fully complete the image. We evaluate Stitch on PosEval, our benchmark for position-based T2I generation. Featuring five new tasks that extend the concept of Position beyond the basic GenEval task, PosEval demonstrates that even top models still have significant room for improvement in position-based generation. Tested on Qwen-Image, FLUX, and SD3.5, Stitch consistently enhances base models, even improving FLUX by 218% on GenEval’s Position task and by 206% on PosEval. Stitch achieves state-of-the-art results with Qwen-Image on PosEval, improving over previous models by 54%, all accomplished while integrating position control into leading models training-free. Code is available at https://github.com/ExplainableML/Stitch.

Method: Framing 3D reconstruction as test-time training, using alignment confidence between memory state and observations to derive closed-form learning rates for memory updates.

Result: 2x improvement in global pose estimation over baselines, operating at 20 FPS with 6 GB GPU memory for thousands of images.

Conclusion: TTT3R provides training-free intervention that substantially improves length generalization in 3D reconstruction foundation models.

Abstract: Modern Recurrent Neural Networks have become a competitive architecture for 3D reconstruction due to their linear-time complexity. However, their performance degrades significantly when applied beyond the training context length, revealing limited length generalization. In this work, we revisit the 3D reconstruction foundation models from a Test-Time Training perspective, framing their designs as an online learning problem. Building on this perspective, we leverage the alignment confidence between the memory state and incoming observations to derive a closed-form learning rate for memory updates, to balance between retaining historical information and adapting to new observations. This training-free intervention, termed TTT3R, substantially improves length generalization, achieving a $2\times$ improvement in global pose estimation over baselines, while operating at 20 FPS with just 6 GB of GPU memory to process thousands of images. Code available in https://rover-xingyu.github.io/TTT3R

Method: Uses subtraction units to produce difference features between adjacent encoder levels, expands to intra-layer multi-scale SUs for pixel/structure-level differences, and pyramidally equips multi-scale SUs at different levels with varying receptive fields. Also includes a training-free LossNet for comprehensive supervision.

Result: Outperforms most state-of-the-art methods across different evaluation metrics on eleven datasets covering four medical image segmentation tasks (colonoscopy, ultrasound, CT, OCT).

Conclusion: The multi-scale subtraction approach effectively captures detailed and structural cues simultaneously, achieving superior performance in diverse medical image segmentation tasks.

Abstract: Accurate medical image segmentation is critical for early medical diagnosis. Most existing methods are based on U-shape structure and use element-wise addition or concatenation to fuse different level features progressively in decoder. However, both the two operations easily generate plenty of redundant information, which will weaken the complementarity between different level features, resulting in inaccurate localization and blurred edges of lesions. To address this challenge, we propose a general multi-scale in multi-scale subtraction network (M$^{2}$SNet) to finish diverse segmentation from medical image. Specifically, we first design a basic subtraction unit (SU) to produce the difference features between adjacent levels in encoder. Next, we expand the single-scale SU to the intra-layer multi-scale SU, which can provide the decoder with both pixel-level and structure-level difference information. Then, we pyramidally equip the multi-scale SUs at different levels with varying receptive fields, thereby achieving the inter-layer multi-scale feature aggregation and obtaining rich multi-scale difference information. In addition, we build a training-free network ``LossNet’’ to comprehensively supervise the task-aware features from bottom layer to top layer, which drives our multi-scale subtraction network to capture the detailed and structural cues simultaneously. Without bells and whistles, our method performs favorably against most state-of-the-art methods under different evaluation metrics on eleven datasets of four different medical image segmentation tasks of diverse image modalities, including color colonoscopy imaging, ultrasound imaging, computed tomography (CT), and optical coherence tomography (OCT). The source code can be available at https://github.com/Xiaoqi-Zhao-DLUT/MSNet.

Method: Uses hierarchical sparse gradient descent to optimize coarse MPI generated by 3D CNN, with occlusion-aware iterative refinement to remove artifacts at boundaries.

Result: Achieves comparable visual quality to offline methods while being 100x faster, and outperforms online methods by about 2 dB in PSNR.

Conclusion: The sparse manifold of MPI enables efficient real-time light field generation with maintained quality, making EffLiFe suitable for XR applications.

Abstract: With the rise of Extended Reality (XR) technology, there is a growing need for real-time light field generation from sparse view inputs. Existing methods can be classified into offline techniques, which can generate high-quality novel views but at the cost of long inference/training time, and online methods, which either lack generalizability or produce unsatisfactory results. However, we have observed that the intrinsic sparse manifold of Multi-plane Images (MPI) enables a significant acceleration of light field generation while maintaining rendering quality. Based on this insight, we introduce EffLiFe, a novel light field optimization method, which leverages the proposed Hierarchical Sparse Gradient Descent (HSGD) to produce high-quality light fields from sparse view images in real time. Technically, the coarse MPI of a scene is first generated using a 3D CNN, and it is further sparsely optimized by focusing only on important MPI gradients in a few iterations. Nevertheless, relying solely on optimization can lead to artifacts at occlusion boundaries. Therefore, we propose an occlusion-aware iterative refinement module that removes visual artifacts in occluded regions by iteratively filtering the input. Extensive experiments demonstrate that our method achieves comparable visual quality while being 100x faster on average than state-of-the-art offline methods and delivering better performance (about 2 dB higher in PSNR) compared to other online approaches.

Method: Two-stage prediction strategy using YOLOv8 model with data augmentation for robustness, followed by ensemble modeling and innovative post-processing techniques.

Result: The ensemble model with post-processing outperforms individual base architectures and addresses issues identified in the BaDLAD dataset.

Conclusion: The approach successfully enhances Bengali DLA, provides key insights for future strategies, and establishes BaDLAD as a foundational resource for advancing Bengali document analysis research.

Abstract: This paper focuses on enhancing Bengali Document Layout Analysis (DLA) using the YOLOv8 model and innovative post-processing techniques. We tackle challenges unique to the complex Bengali script by employing data augmentation for model robustness. After meticulous validation set evaluation, we fine-tune our approach on the complete dataset, leading to a two-stage prediction strategy for accurate element segmentation. Our ensemble model, combined with post-processing, outperforms individual base architectures, addressing issues identified in the BaDLAD dataset. By leveraging this approach, we aim to advance Bengali document analysis, contributing to improved OCR and document comprehension and BaDLAD serves as a foundational resource for this endeavor, aiding future research in the field. Furthermore, our experiments provided key insights to incorporate new strategies into the established solution.

Method: Created CT-RATE dataset with 25,692 CT scans and reports, then developed CT-CLIP (contrastive language-image pretraining) and CT-CHAT (vision-language chat model) using this data.

Result: CT-CLIP outperforms state-of-the-art fully supervised models in multi-abnormality detection and case retrieval. CT-CHAT demonstrates specialized capabilities for 3D medical imaging.

Conclusion: The open-source release of CT-RATE, CT-CLIP, and CT-CHAT addresses critical challenges in 3D medical imaging and provides foundation for future medical AI innovations.

Abstract: Advancements in medical imaging AI, particularly in 3D imaging, have been limited due to the scarcity of comprehensive datasets. We introduce CT-RATE, a public dataset that pairs 3D medical images with corresponding textual reports. CT-RATE comprises 25,692 non-contrast 3D chest CT scans from 21,304 unique patients. Each scan is accompanied by its corresponding radiology report. Leveraging CT-RATE, we develop CT-CLIP, a CT-focused contrastive language-image pretraining framework designed for broad applications without the need for task-specific training. We demonstrate how CT-CLIP can be used in multi-abnormality detection and case retrieval, and outperforms state-of-the-art fully supervised models across all key metrics. By combining CT-CLIP’s vision encoder with a pretrained large language model, we create CT-CHAT, a vision-language foundational chat model for 3D chest CT volumes. Finetuned on over 2.7 million question-answer pairs derived from the CT-RATE dataset, CT-CHAT underscores the necessity for specialized methods in 3D medical imaging. Collectively, the open-source release of CT-RATE, CT-CLIP, and CT-CHAT not only addresses critical challenges in 3D medical imaging but also lays the groundwork for future innovations in medical AI and improved patient care.

Method: The study screened 20 recent research papers to provide a comprehensive review of state-of-the-art methods for synthesizing street-view images from satellite counterparts.

Result: Main findings include: (i) novel deep learning techniques are needed for more realistic street-view synthesis; (ii) more datasets are required for public use; (iii) better evaluation metrics need investigation for proper assessment of generated images.

Conclusion: Recent literature fails to generate detailed and diverse street-view images due to the application of outdated deep learning techniques.

Abstract: In recent years, street view imagery has grown to become one of the most important sources of geospatial data collection and urban analytics, which facilitates generating meaningful insights and assisting in decision-making. Synthesizing a street-view image from its corresponding satellite image is a challenging task due to the significant differences in appearance and viewpoint between the two domains. In this study, we screened 20 recent research papers to provide a thorough review of the state-of-the-art of how street-view images are synthesized from their corresponding satellite counterparts. The main findings are: (i) novel deep learning techniques are required for synthesizing more realistic and accurate street-view images; (ii) more datasets need to be collected for public usage; and (iii) more specific evaluation metrics need to be investigated for evaluating the generated images appropriately. We conclude that, due to applying outdated deep learning techniques, the recent literature failed to generate detailed and diverse street-view images.

Method: Proposes SPARE using symmetrized point-to-plane distance that considers both positions and normals, with as-rigid-as-possible regulation for normal estimation, alternating minimization solver using majorization-minimization strategy, and deformation graph-based coarse alignment for initialization.

Result: Extensive experiments show the method greatly improves accuracy of non-rigid registration while maintaining relatively high solution efficiency compared to existing methods.

Conclusion: SPARE achieves higher accuracy in non-rigid registration through its symmetrized point-to-plane formulation and efficient optimization approach, with publicly available implementation.

Abstract: Existing optimization-based methods for non-rigid registration typically minimize an alignment error metric based on the point-to-point or point-to-plane distance between corresponding point pairs on the source surface and target surface. However, these metrics can result in slow convergence or a loss of detail. In this paper, we propose SPARE, a novel formulation that utilizes a symmetrized point-to-plane distance for robust non-rigid registration. The symmetrized point-to-plane distance relies on both the positions and normals of the corresponding points, resulting in a more accurate approximation of the underlying geometry and can achieve higher accuracy than existing methods. To solve this optimization problem efficiently, we introduce an as-rigid-as-possible regulation term to estimate the deformed normals and propose an alternating minimization solver using a majorization-minimization strategy. Moreover, for effective initialization of the solver, we incorporate a deformation graph-based coarse alignment that improves registration quality and efficiency. Extensive experiments show that the proposed method greatly improves the accuracy of non-rigid registration problems and maintains relatively high solution efficiency. The code is publicly available at https://github.com/yaoyx689/spare.

Method: Build a brain-eye-computer system using region proposal networks and eye-tracking-based slow serial visual presentation to generate EEG-image data pairs, then use adaptive modality balanced online knowledge distillation to fuse EEG and image features with multi-head attention.

Result: The method demonstrates effectiveness and superiority over state-of-the-art methods, with reliability and practicality shown through experiments on public datasets and real-world scenarios.

Conclusion: The proposed system and AMBOKD method successfully integrate brain-computer interfaces with computer vision for robust dim target detection in aerial images under few-shot conditions.

Abstract: Advanced cognition can be extracted from the human brain using brain-computer interfaces. Integrating these interfaces with computer vision techniques, which possess efficient feature extraction capabilities, can achieve more robust and accurate detection of dim targets in aerial images. However, existing target detection methods primarily concentrate on homogeneous data, lacking efficient and versatile processing capabilities for heterogeneous multimodal data. In this paper, we first build a brain-eye-computer based object detection system for aerial images under few-shot conditions. This system detects suspicious targets using region proposal networks, evokes the event-related potential (ERP) signal in electroencephalogram (EEG) through the eye-tracking-based slow serial visual presentation (ESSVP) paradigm, and constructs the EEG-image data pairs with eye movement data. Then, an adaptive modality balanced online knowledge distillation (AMBOKD) method is proposed to recognize dim objects with the EEG-image data. AMBOKD fuses EEG and image features using a multi-head attention module, establishing a new modality with comprehensive features. To enhance the performance and robust capability of the fusion modality, simultaneous training and mutual learning between modalities are enabled by end-to-end online knowledge distillation. During the learning process, an adaptive modality balancing module is proposed to ensure multimodal equilibrium by dynamically adjusting the weights of the importance and the training gradients across various modalities. The effectiveness and superiority of our method are demonstrated by comparing it with existing state-of-the-art methods. Additionally, experiments conducted on public datasets and system validations in real-world scenarios demonstrate the reliability and practicality of the proposed system and the designed method.

Method: LoRA-PT divides transformer parameter matrices into three tensors, applies tensor SVD to extract low-rank tensors (principal singular values/vectors), and only updates these during fine-tuning while keeping residual tensors fixed.

Result: The method outperformed state-of-the-art PEFT methods on three public hippocampus datasets, achieving better segmentation accuracy while significantly reducing the number of parameter updates.

Conclusion: LoRA-PT provides an effective parameter-efficient fine-tuning approach for hippocampus segmentation that reduces computational requirements while maintaining high accuracy, making it suitable for medical image analysis with limited resources.

Abstract: The hippocampus is an important brain structure involved in various psychiatric disorders, and its automatic and accurate segmentation is vital for studying these diseases. Recently, deep learning-based methods have made significant progress in hippocampus segmentation. However, training deep neural network models requires substantial computational resources, time, and a large amount of labeled training data, which is frequently scarce in medical image segmentation. To address these issues, we propose LoRA-PT, a novel parameter-efficient fine-tuning (PEFT) method that transfers the pre-trained UNETR model from the BraTS2021 dataset to the hippocampus segmentation task. Specifically, LoRA-PT divides the parameter matrix of the transformer structure into three distinct sizes, yielding three third-order tensors. These tensors are decomposed using tensor singular value decomposition to generate low-rank tensors consisting of the principal singular values and vectors, with the remaining singular values and vectors forming the residual tensor. During fine-tuning, only the low-rank tensors (i.e., the principal tensor singular values and vectors) are updated, while the residual tensors remain unchanged. We validated the proposed method on three public hippocampus datasets, and the experimental results show that LoRA-PT outperformed state-of-the-art PEFT methods in segmentation accuracy while significantly reducing the number of parameter updates. Our source code is available at https://github.com/WangangCheng/LoRA-PT/tree/LoRA-PT.

Method: The proposed Dynamic Backbone Freezing (DBF) method introduces a ‘Freezing Scheduler’ module that dynamically manages the update of backbone features during long-term training, addressing the dilemma between extracting generic low-level features and domain-specific knowledge.

Result: Extensive experiments on DOTA and DIOR-R datasets show that DBF enables more accurate model learning while substantially reducing computational costs in long-term training. The method can be seamlessly adopted without additional effort due to its straightforward design.

Conclusion: DBF provides an effective solution for managing backbone fine-tuning in remote sensing object detection under long-term training schedules, achieving better performance with reduced computational costs.

Abstract: Recently, numerous methods have achieved impressive performance in remote sensing object detection, relying on convolution or transformer architectures. Such detectors typically have a feature backbone to extract useful features from raw input images. A common practice in current detectors is initializing the backbone with pre-trained weights available online. Fine-tuning the backbone is typically required to generate features suitable for remote-sensing images. While the prolonged training could lead to over-fitting, hindering the extraction of basic visual features, it can enable models to gradually extract deeper insights and richer representations from remote sensing data. Striking a balance between these competing factors is critical for achieving optimal performance. In this study, we aim to investigate the performance and characteristics of remote sensing object detection models under very long training schedules, and propose a novel method named Dynamic Backbone Freezing (DBF) for feature backbone fine-tuning on remote sensing object detection under long-term training. Our method addresses the dilemma of whether the backbone should extract low-level generic features or possess specific knowledge of the remote sensing domain, by introducing a module called ‘Freezing Scheduler’ to manage the update of backbone features during long-term training dynamically. Extensive experiments on DOTA and DIOR-R show that our approach enables more accurate model learning while substantially reducing computational costs in long-term training. Besides, it can be seamlessly adopted without additional effort due to its straightforward design. The code is available at https://github.com/unique-chan/dbf.

Method: Weak-to-strong Semantics Compensation (WSC) leverages weakly augmented images as anchors to enrich the feature space of strongly augmented counterparts, compensating for lost class-relevant semantics during strong augmentation.

Result: Extensive experiments validate the negative impact of strong augmentation on detection performance and demonstrate WSC’s effectiveness in enhancing previous detection models on standard benchmarks.

Conclusion: WSC serves as a generic plug-in solution that can be easily integrated into existing SFOD pipelines to mitigate the semantic loss caused by strong augmentation.

Abstract: Strong data augmentation is a fundamental component of state-of-the-art mean teacher-based Source-Free domain adaptive Object Detection (SFOD) methods, enabling consistency-based self-supervised optimization along weak augmentation. However, our theoretical analysis and empirical observations reveal a critical limitation: strong augmentation can inadvertently erase class-relevant components, leading to artificial inter-category confusion. To address this issue, we introduce Weak-to-strong Semantics Compensation (WSC), a novel remedy that leverages weakly augmented images, which preserve full semantics, as anchors to enrich the feature space of their strongly augmented counterparts. Essentially, this compensates for the class-relevant semantics that may be lost during strong augmentation on the fly. Notably, WSC can be implemented as a generic plug-in, easily integrable with any existing SFOD pipelines. Extensive experiments validate the negative impact of strong augmentation on detection performance, and the effectiveness of WSC in enhancing the performance of previous detection models on standard benchmarks.

Method: Two-stage approach: 1) Open-set aware meta-learning to create a metric space as starting point, 2) Open-set free transfer learning to adapt to specific target tasks. Also includes strategy to simulate open-set examples through dataset modification or pseudo example generation.

Result: Achieves state-of-the-art performance on miniImageNet and tieredImageNet benchmarks with only 1.5% increase in training effort compared to existing methods.

Conclusion: The work successfully demonstrates the effectiveness of transfer learning in FSOSR, providing a solution that bridges the gap between closed-world and open-world recognition tasks.

Abstract: Few-Shot Open-Set Recognition (FSOSR) targets a critical real-world challenge, aiming to categorize inputs into known categories, termed closed-set classes, while identifying open-set inputs that fall outside these classes. Although transfer learning where a model is tuned to a given few-shot task has become a prominent paradigm in closed-world, we observe that it fails to expand to open-world. To unlock this challenge, we propose a two-stage method which consists of open-set aware meta-learning with open-set free transfer learning. In the open-set aware meta-learning stage, a model is trained to establish a metric space that serves as a beneficial starting point for the subsequent stage. During the open-set free transfer learning stage, the model is further adapted to a specific target task through transfer learning. Additionally, we introduce a strategy to simulate open-set examples by modifying the training dataset or generating pseudo open-set examples. The proposed method achieves state-of-the-art performance on two widely recognized benchmarks, miniImageNet and tieredImageNet, with only a 1.5% increase in training effort. Our work demonstrates the effectiveness of transfer learning in FSOSR.

Method: Proposes residual split densification that adds downscaled Gaussians as residuals, integrates Gaussian image pyramid for progressive supervision, and implements selection scheme prioritizing coarse Gaussian densification over time.

Result: Achieves state-of-the-art rendering quality and shows consistent performance improvements when applied to various 3D-GS variants, demonstrating versatility.

Conclusion: The residual split operation effectively addresses 3D-GS limitations and has broad application potential in 3D-GS-based applications.

Abstract: Recently, 3D Gaussian Splatting (3D-GS) has prevailed in novel view synthesis, achieving high fidelity and efficiency. However, it often struggles to capture rich details and complete geometry. Our analysis reveals that the 3D-GS densification operation lacks adaptiveness and faces a dilemma between geometry coverage and detail recovery. To address this, we introduce a novel densification operation, residual split, which adds a downscaled Gaussian as a residual. Our approach is capable of adaptively retrieving details and complementing missing geometry. To further support this method, we propose a pipeline named ResGS. Specifically, we integrate a Gaussian image pyramid for progressive supervision and implement a selection scheme that prioritizes the densification of coarse Gaussians over time. Extensive experiments demonstrate that our method achieves SOTA rendering quality. Consistent performance improvements can be achieved by applying our residual split on various 3D-GS variants, underscoring its versatility and potential for broader application in 3D-GS-based applications.

Method: Proposes PolSAM with Microwave Vision Data representation, Feature-Level Fusion Prompt (FFP) for modality fusion, and Semantic-Level Fusion Prompt (SFP) for prompt refinement using semantic information.

Result: PolSAM significantly outperforms existing SAM-based and multimodal fusion models on PhySAR-Seg datasets, improving segmentation accuracy, reducing data storage, and accelerating inference time.

Conclusion: The proposed PolSAM effectively addresses PolSAR segmentation challenges by integrating domain knowledge and novel prompt strategies, with source code and datasets made publicly available.

Abstract: PolSAR data presents unique challenges due to its rich and complex characteristics. Existing data representations, such as complex-valued data, polarimetric features, and amplitude images, are widely used. However, these formats often face issues related to usability, interpretability, and data integrity. Most feature extraction networks for PolSAR are small, limiting their ability to capture features effectively. To address these issues, We propose the Polarimetric Scattering Mechanism-Informed SAM (PolSAM), an enhanced Segment Anything Model (SAM) that integrates domain-specific scattering characteristics and a novel prompt generation strategy. PolSAM introduces Microwave Vision Data (MVD), a lightweight and interpretable data representation derived from polarimetric decomposition and semantic correlations. We propose two key components: the Feature-Level Fusion Prompt (FFP), which fuses visual tokens from pseudo-colored SAR images and MVD to address modality incompatibility in the frozen SAM encoder, and the Semantic-Level Fusion Prompt (SFP), which refines sparse and dense segmentation prompts using semantic information. Experimental results on the PhySAR-Seg datasets demonstrate that PolSAM significantly outperforms existing SAM-based and multimodal fusion models, improving segmentation accuracy, reducing data storage, and accelerating inference time. The source code and datasets will be made publicly available at https://github.com/XAI4SAR/PolSAM.

Method: J-NeuS uses cross-representation uncertainty estimation to handle ambiguous geometry from limited observations, performs joint optimization of two radiance fields, and employs guided sampling for accurate reconstruction.

Result: The method achieves accurate reconstruction of large areas and fine structures in complex urban scenarios, outperforming state-of-the-art methods on major driving datasets.

Conclusion: J-NeuS demonstrates superior performance in reconstructing large driving sequences with limited image overlap compared to concurrent state-of-the-art methods.

Abstract: Reconstructing the surrounding surface geometry from recorded driving sequences poses a significant challenge due to the limited image overlap and complex topology of urban environments. SoTA neural implicit surface reconstruction methods often struggle in such setting, either failing due to small vision overlap or exhibiting suboptimal performance in accurately reconstructing both the surface and fine structures. To address these limitations, we introduce J-NeuS, a novel hybrid implicit surface reconstruction method for large driving sequences with outward facing camera poses. J-NeuS cross-representation uncertainty estimation to tackle ambiguous geometry caused by limited observations. Our method performs joint optimization of two radiance fields in addition to guided sampling achieving accurate reconstruction of large areas along with fine structures in complex urban scenarios. Extensive evaluation on major driving datasets demonstrates the superiority of our approach in reconstructing large driving sequences with limited image overlap, outperforming concurrent SoTA methods.

Method: Leverages VFMs to extract semantic superpixels from 2D images, aligns them with LiDAR point clouds for contrastive learning, incorporates superpoint temporal consistency, and uses multi-source data pretraining for generalization across different LiDAR configurations.

Result: Achieves substantial gains over state-of-the-art methods in linear probing and fine-tuning for LiDAR-based segmentation and object detection. Demonstrates superior performance across 11 large-scale multi-sensor datasets with adaptability, efficiency, and robustness.

Conclusion: LargeAD provides an effective framework for large-scale 3D pretraining in autonomous driving, successfully bridging 2D semantic understanding with 3D perception through cross-modal representation learning and achieving strong performance across diverse real-world scenarios.

Abstract: Recent advancements in vision foundation models (VFMs) have revolutionized visual perception in 2D, yet their potential for 3D scene understanding, particularly in autonomous driving applications, remains underexplored. In this paper, we introduce LargeAD, a versatile and scalable framework designed for large-scale 3D pretraining across diverse real-world driving datasets. Our framework leverages VFMs to extract semantically rich superpixels from 2D images, which are aligned with LiDAR point clouds to generate high-quality contrastive samples. This alignment facilitates cross-modal representation learning, enhancing the semantic consistency between 2D and 3D data. We introduce several key innovations: (i) VFM-driven superpixel generation for detailed semantic representation, (ii) a VFM-assisted contrastive learning strategy to align multimodal features, (iii) superpoint temporal consistency to maintain stable representations across time, and (iv) multi-source data pretraining to generalize across various LiDAR configurations. Our approach achieves substantial gains over state-of-the-art methods in linear probing and fine-tuning for LiDAR-based segmentation and object detection. Extensive experiments on 11 large-scale multi-sensor datasets highlight our superior performance, demonstrating adaptability, efficiency, and robustness in real-world autonomous driving scenarios.

Method: MIAFEx employs a learnable refinement mechanism to enhance the classification token in Transformer encoder architecture, adjusting tokens based on learned weights to improve salient feature extraction and adaptability to medical imaging challenges.

Result: MIAFEx outperforms classical feature extractors with traditional/hybrid classifiers and modern CNN/ViT models in accuracy and robustness across multiple complex medical imaging datasets, especially with limited training data.

Conclusion: MIAFEx provides superior feature extraction for medical image classification, demonstrating particular advantages in data-scarce scenarios where traditional and modern models struggle with generalization.

Abstract: Feature extraction techniques are crucial in medical image classification; however, classical feature extractors, in addition to traditional machine learning classifiers, often exhibit significant limitations in providing sufficient discriminative information for complex image sets. While Convolutional Neural Networks (CNNs) and Vision Transformer (ViT) have shown promise in feature extraction, they are prone to overfitting due to the inherent characteristics of medical imaging data, including small sample sizes or high intra-class variance. In this work, the Medical Image Attention-based Feature Extractor (MIAFEx) is proposed, a novel method that employs a learnable refinement mechanism to enhance the classification token within the Transformer encoder architecture. This mechanism adjusts the token based on learned weights, improving the extraction of salient features and enhancing the model’s adaptability to the challenges presented by medical imaging data. The MIAFEx output feature quality is compared against classical feature extractors using traditional and hybrid classifiers. Also, the performance of these features is compared against modern CNN and ViT models in classification tasks, demonstrating their superiority in accuracy and robustness across multiple complex medical imaging datasets. This advantage is particularly pronounced in scenarios with limited training data, where traditional and modern models often struggle to generalize effectively. The source code of this proposal can be found at https://github.com/Oscar-RamosS/Medical-Image-Attention-based-Feature-Extractor-MIAFEx

Method: Encodes images into binary representations using multi bit-plane and learnable binary embeddings, perturbs them via XOR-based noise, and trains with binary cross-entropy loss.

Result: Outperforms state-of-the-art methods on FFHQ, CelebA, and CelebA-HQ for image-to-image translation tasks using few sampling steps. Achieves competitive results on ImageNet-1k class-conditional generation with low parameter counts.

Conclusion: BDPM provides an effective framework for binary data generation with fine-grained noise control, accelerated convergence, and reduced inference costs compared to conventional diffusion models.

Abstract: We propose the Binary Diffusion Probabilistic Model (BDPM), a generative framework specifically designed for data representations in binary form. Conventional denoising diffusion probabilistic models (DDPMs) assume continuous inputs, use mean squared error objectives and Gaussian perturbations, i.e., assumptions that are not suited to discrete and binary representations. BDPM instead encodes images into binary representations using multi bit-plane and learnable binary embeddings, perturbs them via XOR-based noise, and trains a model by optimizing a binary cross-entropy loss. These binary representations offer fine-grained noise control, accelerate convergence, and reduce inference cost. On image-to-image translation tasks, such as super-resolution, inpainting, and blind restoration, BDPM based on a small denoiser and multi bit-plane representation outperforms state-of-the-art methods on FFHQ, CelebA, and CelebA-HQ using a few sampling steps. In class-conditional generation on ImageNet-1k, BDPM based on learnable binary embeddings achieves competitive results among models with both low parameter counts and a few sampling steps.

Method: Builds target concept-related word vocabulary, uses adaptive semantic component suppression to remove concept-related information from text tokens, and applies gradient-orthogonal token optimization to adapt tokens to original image space.

Result: Extensive experiments on I2P and UnlearnCanvas benchmarks demonstrate effective and efficient concept erasure performance.

Conclusion: CE-SDWV provides an effective and efficient solution for removing target concepts from T2I diffusion models by operating in text semantic space without modifying model weights.

Abstract: Large-scale text-to-image (T2I) diffusion models have achieved remarkable generative performance about various concepts. With the limitation of privacy and safety in practice, the generative capability concerning NSFW (Not Safe For Work) concepts is undesirable, e.g., producing sexually explicit photos, and licensed images. The concept erasure task for T2I diffusion models has attracted considerable attention and requires an effective and efficient method. To achieve this goal, we propose a CE-SDWV framework, which removes the target concepts (e.g., NSFW concepts) of T2I diffusion models in the text semantic space by only adjusting the text condition tokens and does not need to re-train the original T2I diffusion model’s weights. Specifically, our framework first builds a target concept-related word vocabulary to enhance the representation of the target concepts within the text semantic space, and then utilizes an adaptive semantic component suppression strategy to ablate the target concept-related semantic information in the text condition tokens. To further adapt the above text condition tokens to the original image semantic space, we propose an end-to-end gradient-orthogonal token optimization strategy. Extensive experiments on I2P and UnlearnCanvas benchmarks demonstrate the effectiveness and efficiency of our method. Code is available at https://github.com/TtuHamg/CE-SDWV.

Method: A learning-free token reduction approach that exploits redundancy in visual representations, compatible with most open-source MLLM architectures without additional fine-tuning.

Result: Achieves up to 16x token reduction on MLLMs (LLaVA, MiniGPT, QwenVL) while maintaining or enhancing performance on vision QA benchmarks. Also complementary with other acceleration techniques.

Conclusion: LFTR enables efficient MLLM deployment through significant token reduction in a learning-free manner, working synergistically with existing acceleration methods.

Abstract: Multimodal Large Language Models (MLLMs) have demonstrated exceptional success in various multimodal tasks, yet their deployment is frequently limited by substantial computational demands and prolonged inference times. Given that the vision modality typically contains more comprehensive information than the text modality, resulting in encoded representations comprising an extensive number of tokens, leading to significant computational overhead due to the quadratic complexity of the attention mechanism. Current token reduction methods are typically restricted to specific model architectures and often necessitate extensive retraining or fine-tuning, restricting their applicability to many state-of-the-art models. In this paper, we introduce a learning-free token reduction (LFTR) method designed for MLLMs. LFTR can be seamlessly integrated into most open-source MLLM architectures without requiring additional fine-tuning. By capitalizing on the redundancy in visual representations, our approach effectively reduces tokens while preserving the general inference performance of MLLMs. We conduct experiments on multiple MLLM architectures (LLaVA, MiniGPT, QwenVL), and our results show that LFTR achieves up to a $16\times$ reduction of visual tokens while maintaining or even enhancing performance on mainstream vision question-answering benchmarks, all in a learning-free setting. Additionally, LFTR is complementary to other acceleration techniques, such as vision encoder compression and post-training quantization, further promoting the efficient deployment of MLLMs. Our project is available at https://anonymous.4open.science/r/LFTR-AAAI-0528.

Method: The method analyzes low-rank decomposition and identifies direction robustness as crucial. It prunes parameters and scales coefficients from inter-parameter relations to maintain direction stability, and performs cross-task normalization to enhance generalization to unseen tasks.

Result: Experiments on a diverse multimodal task benchmark show outstanding performance and generalizability. The method effectively merges parameter-efficient modules while maintaining strong performance across tasks.

Conclusion: RobustMerge successfully addresses the challenge of merging parameter-efficient tuned models by maintaining direction robustness through complementary parameter adaptation, providing an effective solution for creating universal multi-task models from specialized experts.

Abstract: Fine-tuning pre-trained models with custom data leads to numerous expert models on specific tasks. Merging models into one universal model to empower multi-task ability refraining from data leakage has gained popularity. With the expansion in data and model size, parameter-efficient tuning becomes the common practice for obtaining task-specific models efficiently. However, few methods are dedicated to efficient merging, and existing methods designed for full fine-tuning merging fail under efficient merging. To address the issue, we analyze from low-rank decomposition and reveal that direction robustness during merging is crucial for merging efficient modules. We furthermore uncover that compensating for the gap between stark singular values contributes to direction robustness. Therefore, we propose RobustMerge, a training-free parameter-efficient merging method with complementary parameter adaptation to maintain direction robustness. Specifically, we (1) prune parameters and scale coefficients from inter-parameter relation for singular values to maintain direction stability away from task interference, and (2) perform cross-task normalization to enhance unseen task generalization. We establish a benchmark consisting of diverse multimodal tasks, on which we conduct experiments to certify the outstanding performance and generalizability of our method. Additional studies and extensive analyses further showcase the effectiveness. Code is available at https://github.com/AuroraZengfh/RobustMerge.

Method: Transforms all perception targets into language space and uses a novel embedding retrieval approach through language interface to unify object detection, pixel segmentation, and vision-language tasks into a single model.

Result: Outperforms previous SOTA generalist models by 12.3 mAP on COCO instance segmentation and 3.3 mIoU on ADE20K semantic segmentation after multi-task training on five datasets.

Conclusion: Successfully bridges fine-grained perception and vision-language tasks, simplifies architectural design, and enables integration with existing MLLMs for more challenging tasks like reasoning segmentation.

Abstract: Generalist models have achieved remarkable success in both language and vision-language tasks, showcasing the potential of unified modeling. However, effectively integrating fine-grained perception tasks like detection and segmentation into these models remains a significant challenge. This is primarily because these tasks often rely heavily on task-specific designs and architectures that can complicate the modeling process. To address this challenge, we present \ours, a framework that \textbf{U}nifies \textbf{F}ine-grained visual perception tasks through an \textbf{O}pen-ended language interface. By transforming all perception targets into the language space, \ours unifies object-level detection, pixel-level segmentation, and image-level vision-language tasks into a single model. Additionally, we introduce a novel embedding retrieval approach that relies solely on the language interface to support segmentation tasks. Our framework bridges the gap between fine-grained perception and vision-language tasks, significantly simplifying architectural design and training strategies while achieving comparable or superior performance to methods with intricate task-specific designs. After multi-task training on five standard visual perception datasets, \ours outperforms the previous state-of-the-art generalist models by 12.3 mAP on COCO instance segmentation and 3.3 mIoU on ADE20K semantic segmentation. Furthermore, our method seamlessly integrates with existing MLLMs, effectively combining fine-grained perception capabilities with their advanced language abilities, thereby enabling more challenging tasks such as reasoning segmentation. Code and models are available at https://github.com/nnnth/UFO.

Method: DecAlign decouples representations into modality-unique and modality-common features using prototype-guided optimal transport alignment with gaussian mixture modeling and multi-marginal transport plans, plus Maximum Mean Discrepancy regularization for semantic consistency and a multimodal transformer for feature fusion.

Result: Extensive experiments on four multimodal benchmarks show DecAlign consistently outperforms state-of-the-art methods across five metrics.

Conclusion: DecAlign effectively enhances cross-modal alignment and semantic consistency while preserving modality-unique features, representing a significant advancement in multimodal representation learning.

Abstract: Multimodal representation learning aims to capture both shared and complementary semantic information across multiple modalities. However, the intrinsic heterogeneity of diverse modalities presents substantial challenges to achieve effective cross-modal collaboration and integration. To address this, we introduce DecAlign, a novel hierarchical cross-modal alignment framework designed to decouple multimodal representations into modality-unique (heterogeneous) and modality-common (homogeneous) features. For handling heterogeneity, we employ a prototype-guided optimal transport alignment strategy leveraging gaussian mixture modeling and multi-marginal transport plans, thus mitigating distribution discrepancies while preserving modality-unique characteristics. To reinforce homogeneity, we ensure semantic consistency across modalities by aligning latent distribution matching with Maximum Mean Discrepancy regularization. Furthermore, we incorporate a multimodal transformer to enhance high-level semantic feature fusion, thereby further reducing cross-modal inconsistencies. Our extensive experiments on four widely used multimodal benchmarks demonstrate that DecAlign consistently outperforms existing state-of-the-art methods across five metrics. These results highlight the efficacy of DecAlign in enhancing superior cross-modal alignment and semantic consistency while preserving modality-unique features, marking a significant advancement in multimodal representation learning scenarios. Our project page is at https://taco-group.github.io/DecAlign.

Method: Created a first-of-its-kind taxonomy based on DL models, handcrafted feature use, SAR attribute utilization, and fine-tuning impact. Analyzed methodologies and techniques used in SAR ship classification.

Result: Identified critical trends including integration of handcrafted features, use of public datasets, data augmentation, fine-tuning, explainability techniques, and interdisciplinary collaborations for improving DL model performance.

Conclusion: Future research should address data scarcity, explore novel DL architectures, incorporate interpretability techniques, and establish standardized metrics to develop more accurate ship classification systems for maritime surveillance.

Abstract: Deep learning (DL) has emerged as a powerful tool for Synthetic Aperture Radar (SAR) ship classification. This survey comprehensively analyzes the diverse DL techniques employed in this domain. We identify critical trends and challenges, highlighting the importance of integrating handcrafted features, utilizing public datasets, data augmentation, fine-tuning, explainability techniques, and fostering interdisciplinary collaborations to improve DL model performance. This survey establishes a first-of-its-kind taxonomy for categorizing relevant research based on DL models, handcrafted feature use, SAR attribute utilization, and the impact of fine-tuning. We discuss the methodologies used in SAR ship classification tasks and the impact of different techniques. Finally, the survey explores potential avenues for future research, including addressing data scarcity, exploring novel DL architectures, incorporating interpretability techniques, and establishing standardized performance metrics. By addressing these challenges and leveraging advancements in DL, researchers can contribute to developing more accurate and efficient ship classification systems, ultimately enhancing maritime surveillance and related applications.

Method: CODA adapts off-the-shelf continuous VAEs into discrete tokenizers via a carefully designed discretization process, focusing primarily on discretization while leveraging pre-trained VAEs for compression.

Result: With 6× less training budget than standard VQGAN, CODA achieves 100% codebook utilization and reconstruction FID of 0.43 and 1.34 for 8× and 16× compression on ImageNet 256×256 benchmark.

Conclusion: Decoupling compression and discretization enables stable and efficient training while maintaining strong visual fidelity from continuous VAEs.

Abstract: Discrete visual tokenizers transform images into a sequence of tokens, enabling token-based visual generation akin to language models. However, this process is inherently challenging, as it requires both compressing visual signals into a compact representation and discretizing them into a fixed set of codes. Traditional discrete tokenizers typically learn the two tasks jointly, often leading to unstable training, low codebook utilization, and limited reconstruction quality. In this paper, we introduce \textbf{CODA}(\textbf{CO}ntinuous-to-\textbf{D}iscrete \textbf{A}daptation), a framework that decouples compression and discretization. Instead of training discrete tokenizers from scratch, CODA adapts off-the-shelf continuous VAEs – already optimized for perceptual compression – into discrete tokenizers via a carefully designed discretization process. By primarily focusing on discretization, CODA ensures stable and efficient training while retaining the strong visual fidelity of continuous VAEs. Empirically, with $\mathbf{6 \times}$ less training budget than standard VQGAN, our approach achieves a remarkable codebook utilization of 100% and notable reconstruction FID (rFID) of $\mathbf{0.43}$ and $\mathbf{1.34}$ for $8 \times$ and $16 \times$ compression on ImageNet 256$\times$ 256 benchmark.

Method: Created 3,217 gender-neutral prompts, generated 200 images per prompt from 5 leading T2I models, automatically detected perceived gender, filtered images, and analyzed gender proportions across semantically grouped concepts.

Result: T2I models strongly reinforce traditional gender roles - women predominantly portrayed in care and human-centered scenarios, men in technical or physical labor scenarios, reflecting common gender stereotypes.

Conclusion: Text-to-image models perpetuate and amplify existing gender biases in society, particularly reinforcing traditional gender stereotypes in household roles and daily activities.

Abstract: With the increasing use of image generation technology, understanding its social biases, including gender bias, is essential. This paper presents a large-scale study on gender bias in text-to-image (T2I) models, focusing on everyday situations. While previous research has examined biases in occupations, we extend this analysis to gender associations in daily activities, objects, and contexts. We create a dataset of 3,217 gender-neutral prompts and generate 200 images over 5 prompt variations per prompt from five leading T2I models. We automatically detect the perceived gender of people in the generated images and filter out images with no person or multiple people of different genders, leaving 2,293,295 images. To enable a broad analysis of gender bias in T2I models, we group prompts into semantically similar concepts and calculate the proportion of male- and female-gendered images for each prompt. Our analysis shows that T2I models reinforce traditional gender roles and reflect common gender stereotypes in household roles. Women are predominantly portrayed in care and human-centered scenarios, and men in technical or physical labor scenarios.

Method: BoundMatch combines boundary detection with segmentation through Boundary Consistency Regularized Multi-Task Learning (BCRM), enforcing agreement on both masks and boundaries. It includes Boundary-Semantic Fusion (BSF) and Spatial Gradient Fusion (SGF) modules, built on SAMTH baseline with Harmonious Batch Normalization.

Result: Achieves competitive performance on Cityscapes and Pascal VOC, with state-of-the-art results on Cityscapes using DINOv2. Shows improvements in boundary-specific metrics and works well with large-scale unlabeled data and lightweight architectures.

Conclusion: Explicitly modeling semantic boundaries alongside segmentation provides complementary supervision that enhances both boundary quality and overall segmentation performance in semi-supervised settings.

Abstract: Semi-supervised semantic segmentation (SS-SS) aims to mitigate the heavy annotation burden of dense pixel labeling by leveraging abundant unlabeled images alongside a small labeled set. While current consistency regularization methods achieve strong results, most do not explicitly model boundaries as a separate learning objective. In this paper, we propose BoundMatch, a novel multi-task SS-SS framework that explicitly integrates semantic boundary detection into a teacher-student consistency regularization pipeline. Our core mechanism, Boundary Consistency Regularized Multi-Task Learning (BCRM), enforces prediction agreement between teacher and student models on both segmentation masks and detailed semantic boundaries, providing complementary supervision from two independent tasks. To further enhance performance and encourage sharper boundaries, BoundMatch incorporates two lightweight fusion modules: Boundary-Semantic Fusion (BSF) injects learned boundary cues into the segmentation decoder, while Spatial Gradient Fusion (SGF) refines boundary predictions using mask gradients, yielding more reliable boundary pseudo-labels. This framework is built upon SAMTH, a strong teacher-student baseline featuring a Harmonious Batch Normalization (HBN) update strategy for improved stability. Extensive experiments on diverse datasets including Cityscapes and Pascal VOC show that BoundMatch achieves competitive performance against current state-of-the-art methods. Our approach achieves state-of-the-art results on the new Cityscapes benchmark with DINOv2 foundation model. Ablation studies highlight BoundMatch’s ability to improve boundary-specific evaluation metrics, its effectiveness in realistic large-scale unlabeled data scenario, and applicability to lightweight architectures for mobile deployment.

Method: Leverages frame-based keypoint detectors on synchronized gray-scale frames for self-supervision, generates temporally sparse keypoint pseudo-labels, and uses a novel information-rich event representation to learn robust detection and description.

Result: Successfully integrated into a modern sparse keypoint and descriptor-based SLAM framework, surpassing state-of-the-art in event-based SLAM by a wide margin.

Conclusion: SuperEvent enables effective event-based keypoint detection and matching, bridging the gap between event sensors and traditional Visual SLAM systems.

Abstract: Event-based keypoint detection and matching holds significant potential, enabling the integration of event sensors into highly optimized Visual SLAM systems developed for frame cameras over decades of research. Unfortunately, existing approaches struggle with the motion-dependent appearance of keypoints and the complex noise prevalent in event streams, resulting in severely limited feature matching capabilities and poor performance on downstream tasks. To mitigate this problem, we propose SuperEvent, a data-driven approach to predict stable keypoints with expressive descriptors. Due to the absence of event datasets with ground truth keypoint labels, we leverage existing frame-based keypoint detectors on readily available event-aligned and synchronized gray-scale frames for self-supervision: we generate temporally sparse keypoint pseudo-labels considering that events are a product of both scene appearance and camera motion. Combined with our novel, information-rich event representation, we enable SuperEvent to effectively learn robust keypoint detection and description in event streams. Finally, we demonstrate the usefulness of SuperEvent by its integration into a modern sparse keypoint and descriptor-based SLAM framework originally developed for traditional cameras, surpassing the state-of-the-art in event-based SLAM by a wide margin. Source code is available at https://ethz-mrl.github.io/SuperEvent/.

Method: Uses corner points of object bounding box as intermediate representation. 3D corners are recovered from sparse views, while 2D corners are estimated via reference-based point synthesizer. Then uses PnP algorithm with 2D-3D correspondences.

Result: Outperforms state-of-the-art methods on YCB-Video and Occluded-LINEMOD datasets, showing enhanced generalization capabilities.

Conclusion: The proposed corner-based representation effectively addresses challenges in sparse-view and occlusion scenarios, significantly improving object pose estimation generalization for real-world applications.

Abstract: This paper presents a generalizable RGB-based approach for object pose estimation, specifically designed to address challenges in sparse-view settings. While existing methods can estimate the poses of unseen objects, their generalization ability remains limited in scenarios involving occlusions and sparse reference views, restricting their real-world applicability. To overcome these limitations, we introduce corner points of the object bounding box as an intermediate representation of the object pose. The 3D object corners can be reliably recovered from sparse input views, while the 2D corner points in the target view are estimated through a novel reference-based point synthesizer, which works well even in scenarios involving occlusions. As object semantic points, object corners naturally establish 2D-3D correspondences for object pose estimation with a PnP algorithm. Extensive experiments on the YCB-Video and Occluded-LINEMOD datasets show that our approach outperforms state-of-the-art methods, highlighting the effectiveness of the proposed representation and significantly enhancing the generalization capabilities of object pose estimation, which is crucial for real-world applications.

Method: Employ smart web search strategies enhanced with knowledge graphs to collect training data more efficiently, and introduce EntityNet dataset with 33M images and 46M text descriptions.

Result: Demonstrated that an expert foundation model for living organisms can be built using just 10M images, and generic CLIP models can be trained in significantly reduced time using EntityNet.

Conclusion: Knowledge graph-enhanced web search strategies enable training robust CLIP models with considerably less data, making domain-specific model development more accessible and cost-effective.

Abstract: Training high-quality CLIP models typically requires enormous datasets, which limits the development of domain-specific models – especially in areas that even the largest CLIP models do not cover well – and drives up training costs. This poses challenges for scientific research that needs fine-grained control over the training procedure of CLIP models. In this work, we show that by employing smart web search strategies enhanced with knowledge graphs, a robust CLIP model can be trained from scratch with considerably less data. Specifically, we demonstrate that an expert foundation model for living organisms can be built using just 10M images. Moreover, we introduce EntityNet, a dataset comprising 33M images paired with 46M text descriptions, which enables the training of a generic CLIP model in significantly reduced time.

Method: Propose Visual Re-ranking Retrieval-Augmented Generation (VR-RAG) that distills Wikipedia articles into discriminative summaries and incorporates visual information during retrieval to ensure predictions are both textually relevant and visually consistent.

Result: Extensive experiments across five bird classification benchmarks and two additional domains show VR-RAG improves average performance of Qwen2.5-VL model by 18.0%.

Conclusion: VR-RAG effectively addresses open-vocabulary species recognition challenges by linking structured encyclopedic knowledge with visual recognition, outperforming text-only approaches.

Abstract: Open-vocabulary species recognition is a major challenge in computer vision, particularly in ornithology, where new taxa are continually discovered. While benchmarks like CUB-200-2011 and Birdsnap have advanced fine-grained recognition under closed vocabularies, they fall short of real-world conditions. We show that current systems suffer a performance drop of over 30% in realistic open-vocabulary settings with thousands of candidate species, largely due to an increased number of visually similar and semantically ambiguous distractors. To address this, we propose Visual Re-ranking Retrieval-Augmented Generation (VR-RAG), a novel framework that links structured encyclopedic knowledge with recognition. We distill Wikipedia articles for 11,202 bird species into concise, discriminative summaries and retrieve candidates from these summaries. Unlike prior text-only approaches, VR-RAG incorporates visual information during retrieval, ensuring final predictions are both textually relevant and visually consistent with the query image. Extensive experiments across five bird classification benchmarks and two additional domains show that VR-RAG improves the average performance of the state-of-the-art Qwen2.5-VL model by 18.0%.

Method: Uses structurally streamlined U-Net with dual-layered knowledge distillation (semantic and structural representations) and latent-space replay-based continual learning for stable performance across sequential tasks.

Result: Achieves strong performance on FID, CLIP, KID, and LPIPS metrics while substantially reducing parameter count, inference time, and FLOPs.

Conclusion: Provides a practical, lightweight, and generalizable solution for deploying diffusion models in resource-constrained environments, suitable for next-generation intelligent computing systems.

Abstract: The growing adoption of generative AI in real-world applications has exposed a critical bottleneck in the computational demands of diffusion-based text-to-image models. In this work, we propose KDC-Diff, a novel and scalable generative framework designed to significantly reduce computational overhead while maintaining high performance. At its core, KDC-Diff designs a structurally streamlined U-Net with a dual-layered knowledge distillation strategy to transfer semantic and structural representations from a larger teacher model. Moreover, a latent-space replay-based continual learning mechanism is incorporated to ensure stable generative performance across sequential tasks. Evaluated on benchmark datasets, our model demonstrates strong performance across FID, CLIP, KID, and LPIPS metrics while achieving substantial reductions in parameter count, inference time, and FLOPs. KDC-Diff offers a practical, lightweight, and generalizable solution for deploying diffusion models in low-resource environments, making it well-suited for the next generation of intelligent and resource-aware computing systems.

Method: A novel architecture extending an encoder-decoder structure with fewer than 20 dataset-specific parameters that control interpretable mechanisms like multi-scale structure, center bias, and fixation spread. Adaptation to new datasets requires only these parameters.

Result: The model achieves state-of-the-art performance on all three MIT/Tuebingen Saliency Benchmark datasets (MIT300, CAT2000, COCO-Freeview), reducing the generalization gap by more than 75% with as few as 50 samples.

Conclusion: The proposed architecture effectively addresses dataset bias in saliency prediction through minimal dataset-specific parameters, providing both improved performance and valuable insights into spatial saliency properties including complex multi-scale effects.

Abstract: Recent advances in image-based saliency prediction are approaching gold standard performance levels on existing benchmarks. Despite this success, we show that predicting fixations across multiple saliency datasets remains challenging due to dataset bias. We find a significant performance drop (around 40%) when models trained on one dataset are applied to another. Surprisingly, increasing dataset diversity does not resolve this inter-dataset gap, with close to 60% attributed to dataset-specific biases. To address this remaining generalization gap, we propose a novel architecture extending a mostly dataset-agnostic encoder-decoder structure with fewer than 20 dataset-specific parameters that govern interpretable mechanisms such as multi-scale structure, center bias, and fixation spread. Adapting only these parameters to new data accounts for more than 75% of the generalization gap, with a large fraction of the improvement achieved with as few as 50 samples. Our model sets a new state-of-the-art on all three datasets of the MIT/Tuebingen Saliency Benchmark (MIT300, CAT2000, and COCO-Freeview), even when purely generalizing from unrelated datasets, but with a substantial boost when adapting to the respective training datasets. The model also provides valuable insights into spatial saliency properties, revealing complex multi-scale effects that combine both absolute and relative sizes.

Method: 1) FSSD: Identifies semantically stable channels by considering semantic sensitivity and classification contribution, selectively regularizing important channels. 2) TD-MoE: Dynamically routes task-specific experts based on temporal dynamics, enabling inference without task ID or exemplars.

Result: Extensive experiments on UCF101, HMDB51, and Kinetics400 show StPR outperforms existing baselines while offering improved interpretability and efficiency.

Conclusion: StPR effectively leverages spatial semantics and temporal dynamics to achieve a unified, exemplar-free VCIL framework that addresses limitations of previous approaches.

Abstract: Video Class-Incremental Learning (VCIL) seeks to develop models that continuously learn new action categories over time without forgetting previously acquired knowledge. Unlike traditional Class-Incremental Learning (CIL), VCIL introduces the added complexity of spatiotemporal structures, making it particularly challenging to mitigate catastrophic forgetting while effectively capturing both frame-shared semantics and temporal dynamics. Existing approaches either rely on exemplar rehearsal, raising concerns over memory and privacy, or adapt static image-based methods that neglect temporal modeling. To address these limitations, we propose Spatiotemporal Preservation and Routing (StPR), a unified and exemplar-free VCIL framework that explicitly disentangles and preserves spatiotemporal information. First, we introduce Frame-Shared Semantics Distillation (FSSD), which identifies semantically stable and meaningful channels by jointly considering semantic sensitivity and classification contribution. These important semantic channels are selectively regularized to maintain prior knowledge while allowing for adaptation. Second, we design a Temporal Decomposition-based Mixture-of-Experts (TD-MoE), which dynamically routes task-specific experts based on their temporal dynamics, enabling inference without task ID or stored exemplars. Together, StPR effectively leverages spatial semantics and temporal dynamics, achieving a unified, exemplar-free VCIL framework. Extensive experiments on UCF101, HMDB51, and Kinetics400 show that our method outperforms existing baselines while offering improved interpretability and efficiency in VCIL. Code is available in the supplementary materials.

Method: Introduce octic ViTs that use octic group equivariance to capture geometric symmetries. Two families: fully octic equivariant networks and networks that break equivariance in the last part. Use octic linear layers that reduce computation compared to ordinary linear layers.

Result: Octic ViTs match baseline accuracy on ImageNet-1K (both supervised DeiT-III and unsupervised DINOv2) while providing substantial efficiency gains: 5.33x FLOPs reduction and up to 8x memory reduction in linear layers, with computational reductions approaching these levels in full blocks with increased embedding dimension.

Conclusion: Octic ViTs successfully capture geometric symmetries while maintaining accuracy and providing significant computational efficiency improvements, demonstrating that equivariant models can be made efficient without sacrificing performance.

Abstract: Why are state-of-the-art Vision Transformers (ViTs) not designed to exploit natural geometric symmetries such as 90-degree rotations and reflections? In this paper, we argue that there is no fundamental reason, and what has been missing is an efficient implementation. To this end, we introduce Octic Vision Transformers (octic ViTs) which rely on octic group equivariance to capture these symmetries. In contrast to prior equivariant models that increase computational cost, our octic linear layers achieve 5.33x reductions in FLOPs and up to 8x reductions in memory compared to ordinary linear layers. In full octic ViT blocks the computational reductions approach the reductions in the linear layers with increased embedding dimension. We study two new families of ViTs, built from octic blocks, that are either fully octic equivariant or break equivariance in the last part of the network. Training octic ViTs supervised (DeiT-III) and unsupervised (DINOv2) on ImageNet-1K, we find that they match baseline accuracy while at the same time providing substantial efficiency gains.

Method: Align LiDAR scans into coherent 3D scenes, render them from multiple virtual camera poses to generate synthetic 2D datasets (PC2D), train ensemble of 2D segmentation models, and use occlusion-aware voting scheme to back-project logits to 3D point-wise labels.

Result: Achieves state-of-the-art performance in Real-to-Real UDA setting and enables segmentation of rare classes using only 2D annotations while relying on 3D annotations for other classes in source domain.

Conclusion: The proposed multi-view projection framework effectively addresses domain shift in 3D semantic segmentation and enables flexible annotation strategies for rare classes.

Abstract: 3D semantic segmentation is essential for autonomous driving and road infrastructure analysis, but state-of-the-art 3D models suffer from severe domain shift when applied across datasets. We propose a multi-view projection framework for unsupervised domain adaptation (UDA). Our method aligns LiDAR scans into coherent 3D scenes and renders them from multiple virtual camera poses to generate large-scale synthetic 2D datasets (PC2D) in various modalities. An ensemble of 2D segmentation models is trained on these modalities, and during inference, hundreds of views per scene are processed, with logits back-projected to 3D using an occlusion-aware voting scheme to produce point-wise labels. These labels can be used directly or to fine-tune a 3D segmentation model in the target domain. We evaluate our approach in both Real-to-Real and Simulation-to-Real UDA, achieving state-of-the-art performance in the Real-to-Real setting. Furthermore, we show that our framework enables segmentation of rare classes, leveraging only 2D annotations for those classes while relying on 3D annotations for others in the source domain.

Method: Explored various LVLMs to identify OCR-specific heads, analyzed their properties, validated findings through downstream tasks using Chain-of-Thought and head masking, and experimented with redistributing sink-token values.

Result: Found OCR heads are activated by many heads (less sparse), have distinct properties from general retrieval heads, and activation frequency correlates with OCR scores. Redistributing sink-token values improved performance.

Conclusion: The study provides deeper understanding of LVLMs’ internal mechanisms for processing textual information in images, identifying specialized OCR heads with unique characteristics that differ from general retrieval functions.

Abstract: Despite significant advancements in Large Vision Language Models (LVLMs), a gap remains, particularly regarding their interpretability and how they locate and interpret textual information within images. In this paper, we explore various LVLMs to identify the specific heads responsible for recognizing text from images, which we term the Optical Character Recognition Head (OCR Head). Our findings regarding these heads are as follows: (1) Less Sparse: Unlike previous retrieval heads, a large number of heads are activated to extract textual information from images. (2) Qualitatively Distinct: OCR heads possess properties that differ significantly from general retrieval heads, exhibiting low similarity in their characteristics. (3) Statically Activated: The frequency of activation for these heads closely aligns with their OCR scores. We validate our findings in downstream tasks by applying Chain-of-Thought (CoT) to both OCR and conventional retrieval heads and by masking these heads. We also demonstrate that redistributing sink-token values within the OCR heads improves performance. These insights provide a deeper understanding of the internal mechanisms LVLMs employ in processing embedded textual information in images.

Method: Three key contributions: (1) Automated framework for building PPC datasets from expert photographs, (2) Video generation showing transformation from poor to enhanced perspectives, (3) Perspective quality assessment (PQA) model based on human performance.

Result: Developed a concise approach that requires no additional prompts or camera trajectories, enabling ordinary users to enhance composition skills through perspective-based recomposition.

Conclusion: PPC extends traditional photography composition methods by incorporating 3D perspective adjustment, providing tools for automated dataset creation, transformation visualization, and quality assessment to help users improve photographic composition.

Abstract: Traditional photography composition approaches are dominated by 2D cropping-based methods. However, these methods fall short when scenes contain poorly arranged subjects. Professional photographers often employ perspective adjustment as a form of 3D recomposition, modifying the projected 2D relationships between subjects while maintaining their actual spatial positions to achieve better compositional balance. Inspired by this artistic practice, we propose photography perspective composition (PPC), extending beyond traditional cropping-based methods. However, implementing the PPC faces significant challenges: the scarcity of perspective transformation datasets and undefined assessment criteria for perspective quality. To address these challenges, we present three key contributions: (1) An automated framework for building PPC datasets through expert photographs. (2) A video generation approach that demonstrates the transformation process from less favorable to aesthetically enhanced perspectives. (3) A perspective quality assessment (PQA) model constructed based on human performance. Our approach is concise and requires no additional prompt instructions or camera trajectories, helping and guiding ordinary users to enhance their composition skills.

Method: AVCD uses attention distributions to dynamically identify less dominant modalities, applies attentive masking to generate perturbed output logits, reformulates CD for trimodal settings, and introduces entropy-guided adaptive decoding for efficiency.

Result: AVCD consistently outperforms existing decoding methods, improving accuracy by 2% for VideoLLaMA2 and 7% for video-SALMONN on AVHBench dataset, demonstrating strong robustness and generalizability.

Conclusion: The proposed AVCD framework effectively addresses modality-induced hallucinations in AV-LLMs through dynamic modality identification and attentive masking, providing a more adaptive decoding strategy for trimodal interactions.

Abstract: Hallucination remains a major challenge in multimodal large language models (MLLMs). To address this, various contrastive decoding (CD) methods have been proposed that contrasts original logits with hallucinated logits generated from perturbed inputs. While CD has shown promise in vision-language models (VLMs), it is not well-suited for AV-LLMs, where hallucinations often emerge from both unimodal and cross-modal combinations involving audio, video, and language. These intricate interactions call for a more adaptive and modality-aware decoding strategy. In this paper, we propose Audio-Visual Contrastive Decoding (AVCD)-a novel, training-free decoding framework designed to model trimodal interactions and suppress modality-induced hallucinations in AV-LLMs. Unlike previous CD methods in VLMs that corrupt a fixed modality, AVCD leverages attention distributions to dynamically identify less dominant modalities and applies attentive masking to generate perturbed output logits. To support CD in a trimodal setting, we also reformulate the original CD framework to jointly handle audio, visual, and textual inputs. Finally, to improve efficiency, we introduce entropy-guided adaptive decoding, which selectively skips unnecessary decoding steps based on the model’s confidence in its predictions. Extensive experiments demonstrate that AVCD consistently outperforms existing decoding methods. Especially, on the AVHBench dataset, it improves accuracy by 2% for VideoLLaMA2 and 7% for video-SALMONN, demonstrating strong robustness and generalizability. Our code is available at https://github.com/kaistmm/AVCD.

Method: Introduce ViewSpatial-Bench benchmark with five task types and automated 3D annotation pipeline for multi-viewpoint spatial localization evaluation. Fine-tune VLMs on multi-perspective spatial dataset.

Result: VLMs show significant performance disparity: reasonable on camera-perspective tasks but reduced accuracy on human viewpoints. Fine-tuning achieves 46.24% overall performance improvement across tasks.

Conclusion: ViewSpatial-Bench establishes crucial benchmark for spatial intelligence in embodied AI. Modeling 3D spatial relationships enhances VLMs’ spatial comprehension capabilities.

Abstract: Vision-language models (VLMs) have demonstrated remarkable capabilities in understanding and reasoning about visual content, but significant challenges persist in tasks requiring cross-viewpoint understanding and spatial reasoning. We identify a critical limitation: current VLMs excel primarily at egocentric spatial reasoning (from the camera’s perspective) but fail to generalize to allocentric viewpoints when required to adopt another entity’s spatial frame of reference. We introduce ViewSpatial-Bench, the first comprehensive benchmark designed specifically for multi-viewpoint spatial localization recognition evaluation across five distinct task types, supported by an automated 3D annotation pipeline that generates precise directional labels. Comprehensive evaluation of diverse VLMs on ViewSpatial-Bench reveals a significant performance disparity: models demonstrate reasonable performance on camera-perspective tasks but exhibit reduced accuracy when reasoning from a human viewpoint. By fine-tuning VLMs on our multi-perspective spatial dataset, we achieve an overall performance improvement of 46.24% across tasks, highlighting the efficacy of our approach. Our work establishes a crucial benchmark for spatial intelligence in embodied AI systems and provides empirical evidence that modeling 3D spatial relationships enhances VLMs’ corresponding spatial comprehension capabilities.

Method: FMD processes audio-only and video-only inputs through early decoder layers separately (fork), then merges the resulting hidden states for joint reasoning in remaining layers (merge), allowing modality-specific emphasis before balanced integration.

Result: Experimental validation on VideoLLaMA2, video-SALMONN, and Qwen2.5-Omni shows consistent gains in audio, video, and audio-visual reasoning tasks across three benchmark datasets.

Conclusion: FMD demonstrates that inference-time interventions can effectively achieve robust and efficient balanced multimodal understanding without architectural changes or additional training.

Abstract: The goal of this work is to enhance balanced multimodal understanding in audio-visual large language models (AV-LLMs) by addressing modality bias without additional training. In current AV-LLMs, audio and video features are typically processed jointly in the decoder. While this strategy facilitates unified multimodal understanding, it may introduce modality bias, where the model tends to over-rely on one modality due to imbalanced training signals. To mitigate this, we propose Fork-Merge Decoding (FMD), a simple yet effective inference-time strategy that requires no additional training or architectural modifications. FMD first performs modality-specific reasoning by processing audio-only and video-only inputs through the early decoder layers (fork), and then merges the resulting hidden states for joint reasoning in the remaining layers (merge). This separation allows each modality to be emphasized in the early stages while encouraging balanced contributions during integration. We validate our method on three representative AV-LLMs-VideoLLaMA2, video-SALMONN, and Qwen2.5-Omni-using three benchmark datasets. Experimental results show consistent gains in audio, video, and audio-visual reasoning tasks, highlighting the effectiveness of inference-time interventions for robust and efficient multimodal understanding.

Method: Introduces synthetic negative targets through CDNS, uses pairwise comparison to guide model preference for positives over negatives, and reweights diffusion timesteps to focus on intermediate steps where subject details emerge.

Result: Extensive experiments show SFO with CDNS significantly outperforms recent strong baselines in both subject fidelity and text alignment on subject-driven generation benchmarks.

Conclusion: SFO provides an effective framework for enhancing subject fidelity in zero-shot subject-driven generation without requiring expensive human annotations.

Abstract: We present Subject Fidelity Optimization (SFO), a novel comparative learning framework for zero-shot subject-driven generation that enhances subject fidelity. Existing supervised fine-tuning methods, which rely only on positive targets and use the diffusion loss as in the pre-training stage, often fail to capture fine-grained subject details. To address this, SFO introduces additional synthetic negative targets and explicitly guides the model to favor positives over negatives through pairwise comparison. For negative targets, we propose Condition-Degradation Negative Sampling (CDNS), which automatically produces synthetic negatives tailored for subject-driven generation by introducing controlled degradations that emphasize subject fidelity and text alignment without expensive human annotations. Moreover, we reweight the diffusion timesteps to focus fine-tuning on intermediate steps where subject details emerge. Extensive experiments demonstrate that SFO with CDNS significantly outperforms recent strong baselines in terms of both subject fidelity and text alignment on a subject-driven generation benchmark. Project page: https://subjectfidelityoptimization.github.io/

Method: ReLoop uses a ring-shaped structure with three consistency feedback mechanisms: semantic reconstruction, visual description, and attention supervision via a frozen Consistency Feedback Plugin (CFP), enabling models to ‘see twice and think backwards’ to correct outputs.

Result: Extensive evaluations show ReLoop effectively reduces hallucination rates across multiple benchmarks, demonstrating robust hallucination mitigation in MLLMs.

Conclusion: ReLoop establishes an effective internal mechanism for hallucination reduction in MLLMs through multimodal consistency enforcement, with source code and data to be released.

Abstract: While Multimodal Large Language Models (MLLMs) have achieved remarkable progress in open-ended visual question answering, they remain vulnerable to hallucinations. These are outputs that contradict or misrepresent input semantics, posing a critical challenge to the reliability and factual consistency. Existing methods often rely on external verification or post-hoc correction, lacking an internal mechanism to validate outputs directly during training. To bridge this gap, we propose ReLoop, a unified closed-loop training framework that encourages multimodal consistency for cross-modal understanding in MLLMs. ReLoop adopts a ring-shaped structure that integrates three complementary consistency feedback mechanisms, obliging MLLMs to “seeing twice and thinking backwards”. Specifically, ReLoop employs the frozen Consistency Feedback Plugin (CFP), comprising semantic reconstruction, visual description, and an attention supervision module for attention alignment. These components collectively enforce semantic reversibility, visual consistency, and interpretable attention, enabling the model to correct its outputs during training. Extensive evaluations and analyses demonstrate the effectiveness of ReLoop in reducing hallucination rates across multiple benchmarks, establishing a robust method for hallucination mitigation in MLLMs. We will release our source code and data in the camera-ready version.

Method: Uses lightweight adaptation module updated via gradient descent on meta-learned self-supervised loss during inference, with dissimilarity-based sampling to prevent shortcut learning.

Result: Outperforms state-of-the-art zero-shot methods in real-world robotic manipulation, generalizing to diverse OOD tasks, environments, and embodiments.

Conclusion: VITA’s zero-shot value estimates enable effective reward shaping in offline RL, achieving superior performance on Meta-World benchmark.

Abstract: Vision-Language Models (VLMs) show promise as zero-shot goal-conditioned value functions, but their frozen pre-trained representations limit generalization and temporal reasoning. We introduce VITA, a zero-shot value function learning method that enhances both capabilities via test-time adaptation. At inference, a lightweight adaptation module is updated via a gradient step on a meta-learned self-supervised loss, such that each test-time update improves value estimation. By updating sequentially over a trajectory, VITA encodes history into its parameters, addressing the temporal reasoning limitations. To mitigate shortcut learning, we propose a dissimilarity-based sampling strategy that selects semantically diverse segments of the trajectory during training. In real-world robotic manipulation tasks, VITA generalizes from a single training environment to diverse out-of-distribution tasks, environments, and embodiments, outperforming the state-of-the-art zero-shot method using autoregressive VLMs. Furthermore, we demonstrate that VITA’s zero-shot value estimates can be utilized for reward shaping in offline reinforcement learning, resulting in multi-task policies on the Meta-World benchmark that exceed the performance of those trained with the simulation’s fuzzy-logic dense rewards.

Method: Formalizes a three-axis Hallucination Tri-Space (semantic coherence, structural alignment, knowledge grounding) and introduces Alignment Risk Code (ARC) to quantify alignment tension. Develops TensionModulator (TM-ARC) controller that monitors ARC signals and applies axis-specific interventions during sampling.

Result: Extensive experiments on standard T2I benchmarks show significant reduction in hallucinations without compromising image quality or diversity.

Conclusion: The framework provides a unified and interpretable approach for understanding and mitigating generative failures in diffusion-based text-to-image systems.

Abstract: Despite remarkable progress in image quality and prompt fidelity, text-to-image (T2I) diffusion models continue to exhibit persistent “hallucinations”, where generated content subtly or significantly diverges from the intended prompt semantics. While often regarded as unpredictable artifacts, we argue that these failures reflect deeper, structured misalignments within the generative process. In this work, we propose a cognitively inspired perspective that reinterprets hallucinations as trajectory drift within a latent alignment space. Empirical observations reveal that generation unfolds within a multiaxial cognitive tension field, where the model must continuously negotiate competing demands across three key critical axes: semantic coherence, structural alignment, and knowledge grounding. We then formalize this three-axis space as the Hallucination Tri-Space and introduce the Alignment Risk Code (ARC): a dynamic vector representation that quantifies real-time alignment tension during generation. The magnitude of ARC captures overall misalignment, its direction identifies the dominant failure axis, and its imbalance reflects tension asymmetry. Based on this formulation, we develop the TensionModulator (TM-ARC): a lightweight controller that operates entirely in latent space. TM-ARC monitors ARC signals and applies targeted, axis-specific interventions during the sampling process. Extensive experiments on standard T2I benchmarks demonstrate that our approach significantly reduces hallucination without compromising image quality or diversity. This framework offers a unified and interpretable approach for understanding and mitigating generative failures in diffusion-based T2I systems.

Method: Proposed Decoupled Classifier-Free Guidance (DCFG) with attribute-split embedding strategy that disentangles semantic inputs for selective guidance on attribute groups.

Result: DCFG provides flexible, model-agnostic guidance that enables attribute-wise control following causal graphs.

Conclusion: DCFG effectively mitigates spurious changes in counterfactual generation by allowing selective guidance on different attribute groups.

Abstract: Counterfactual generation aims to simulate realistic hypothetical outcomes under causal interventions. Diffusion models have emerged as a powerful tool for this task, combining DDIM inversion with conditional generation and classifier-free guidance (CFG). In this work, we identify a key limitation of CFG for counterfactual generation: it prescribes a global guidance scale for all attributes, leading to significant spurious changes in inferred counterfactuals. To mitigate this, we propose Decoupled Classifier-Free Guidance (DCFG), a flexible and model-agnostic guidance technique that enables attribute-wise control following a causal graph. DCFG is implemented via a simple attribute-split embedding strategy that disentangles semantic inputs, enabling selective guidance on user-defined attribute groups.

Method: Proposes DenseDiT with parameter-reuse mechanism and two lightweight branches that adaptively integrate multi-scale context, enabling efficient tuning with <0.1% additional parameters.

Result: DenseDiT achieves superior results using less than 0.01% training data of baselines, while existing methods show significant performance drops on the DenseWorld benchmark.

Conclusion: DenseDiT demonstrates practical value for real-world deployment by effectively activating visual priors and adapting to diverse dense prediction tasks with minimal data requirements.

Abstract: Dense prediction tasks hold significant importance of computer vision, aiming to learn pixel-wise annotated labels for input images. Despite advances in this field, existing methods primarily focus on idealized conditions, exhibiting limited real-world generalization and struggling with the acute scarcity of real-world data in practical scenarios. To systematically study this problem, we first introduce DenseWorld, a benchmark spanning a broad set of 25 dense prediction tasks that correspond to urgent real-world applications, featuring unified evaluation across tasks. We then propose DenseDiT, which exploits generative models’ visual priors to perform diverse real-world dense prediction tasks through a unified strategy. DenseDiT combines a parameter-reuse mechanism and two lightweight branches that adaptively integrate multi-scale context. This design enables DenseDiT to achieve efficient tuning with less than 0.1% additional parameters, activating the visual priors while effectively adapting to diverse real-world dense prediction tasks. Evaluations on DenseWorld reveal significant performance drops in existing general and specialized baselines, highlighting their limited real-world generalization. In contrast, DenseDiT achieves superior results using less than 0.01% training data of baselines, underscoring its practical value for real-world deployment.

Method: Three-component approach: (1) Large-scale LongVideo-Reason dataset with 104K video QA pairs, (2) Two-stage training pipeline with chain-of-thought supervised fine-tuning and reinforcement learning, (3) Multi-modal Reinforcement Sequence Parallelism infrastructure for efficient long video RL training.

Result: LongVILA-R1-7B achieves 65.1% and 71.1% accuracy on VideoMME benchmarks, consistently outperforming baseline models across multiple benchmarks, with MR-SP system achieving 2.1x speedup on RL training.

Conclusion: The framework successfully scales VLMs to long video reasoning, demonstrating strong performance improvements and efficient training capabilities, with the training system made publicly available for broader applications.

Abstract: We introduce a full-stack framework that scales up reasoning in vision-language models (VLMs) to long videos, leveraging reinforcement learning. We address the unique challenges of long video reasoning by integrating three critical components: (1) a large-scale dataset, LongVideo-Reason, comprising 104K long video QA pairs with high-quality reasoning annotations across diverse domains such as sports, games, and vlogs; (2) a two-stage training pipeline that extends VLMs with chain-of-thought supervised fine-tuning (CoT-SFT) and reinforcement learning (RL); and (3) a training infrastructure for long video RL, named Multi-modal Reinforcement Sequence Parallelism (MR-SP), which incorporates sequence parallelism and a vLLM-based engine tailored for long video, using cached video embeddings for efficient rollout and prefilling. In our experiments, LongVILA-R1-7B achieves strong performance on video benchmarks, reaching 65.1% and 71.1% accuracy on VideoMME without and with subtitles, respectively, and consistently outperforming LongVILA-7B across multiple benchmarks. Moreover, LongVILA-R1-7B supports processing up to 8,192 video frames per video, and configurable FPS settings. Notably, our MR-SP system achieves up to 2.1x speedup on long video RL training. In addition, we release our training system for public availability that supports RL training on various modalities (video, text, and audio), various models (VILA and Qwen series), and even image and video generation models. On a single A100 node (8 GPUs), it supports RL training on hour-long videos (e.g., 3,600 frames).

Method: Proposes ET-Fuser with ensemble token generation using self-attention mechanism that shares mutual information across pre-trained encoders, leveraging task priors from pre-trained models.

Result: Shows improvements across various facial analysis tasks with statistically significant enhancements in feature representations, while maintaining high efficiency with negligible computational overhead.

Conclusion: ET-Fuser effectively unifies feature representations for facial analysis through ensemble token learning, demonstrating the value of leveraging pre-trained model priors and attention mechanisms for improved performance.

Abstract: Facial analysis exhibits task-specific feature variations. While Convolutional Neural Networks (CNNs) have enabled the fine-grained representation of spatial information, Vision Transformers (ViTs) have facilitated the representation of semantic information at the patch level. Although the generalization of conventional methodologies has advanced visual interpretability, there remains paucity of research that preserves the unified feature representation on single task learning during the training process. In this work, we introduce ET-Fuser, a novel methodology for learning ensemble token by leveraging attention mechanisms based on task priors derived from pre-trained models for facial analysis. Specifically, we propose a robust prior unification learning method that generates a ensemble token within a self-attention mechanism, which shares the mutual information along the pre-trained encoders. This ensemble token approach offers high efficiency with negligible computational cost. Our results show improvements across a variety of facial analysis, with statistically significant enhancements observed in the feature representations.

Method: LATTE (LATent Trajectory Embedding) models the evolution of latent embeddings across multiple denoising steps, capturing the trajectory of representations rather than treating each step in isolation.

Result: Experiments on GenImage, Chameleon, and Diffusion Forensics benchmarks show LATTE achieves superior performance, especially in challenging cross-generator and cross-dataset scenarios.

Conclusion: LATTE demonstrates the potential of latent trajectory modeling for reliable generated image detection across different generators.

Abstract: The rapid advancement of diffusion-based image generators has made it increasingly difficult to distinguish generated from real images. This erodes trust in digital media, making it critical to develop generated image detectors that remain reliable across different generators. While recent approaches leverage diffusion denoising cues, they typically rely on single-step reconstruction errors and overlook the sequential nature of the denoising process. In this work, we propose LATTE - LATent Trajectory Embedding - a novel approach that models the evolution of latent embeddings across multiple denoising steps. Instead of treating each denoising step in isolation, LATTE captures the trajectory of these representations, revealing subtle and discriminative patterns that distinguish real from generated images. Experiments on several benchmarks, such as GenImage, Chameleon, and Diffusion Forensics, show that LATTE achieves superior performance, especially in challenging cross-generator and cross-dataset scenarios, highlighting the potential of latent trajectory modeling. The code is available on the following link: https://github.com/AnaMVasilcoiu/LATTE-Diffusion-Detector.

Method: Proposed HV-MMBench benchmark with 13 diverse tasks ranging from basic attribute perception to advanced cognitive reasoning, multiple question formats (multiple-choice, fill-in-blank, true/false, open-ended), coverage of 50 visual scenarios, and temporal coverage from 10 seconds to 30 minutes.

Result: The benchmark enables comprehensive assessment of MLLM capabilities across fine-grained scene variations and systematic analysis of temporal reasoning abilities across diverse contextual lengths.

Conclusion: HV-MMBench provides a more holistic and rigorous evaluation framework for MLLMs in human-centric video understanding compared to existing benchmarks.

Abstract: Multimodal Large Language Models (MLLMs) have demonstrated significant advances in visual understanding tasks involving both images and videos. However, their capacity to comprehend human-centric video data remains underexplored, primarily due to the absence of comprehensive and high-quality evaluation benchmarks. Existing human-centric benchmarks predominantly emphasize video generation quality and action recognition, while overlooking essential perceptual and cognitive abilities required in human-centered scenarios. Furthermore, they are often limited by single-question paradigms and overly simplistic evaluation metrics. To address above limitations, we propose a modern HV-MMBench, a rigorously curated benchmark designed to provide a more holistic evaluation of MLLMs in human-centric video understanding. Compared to existing human-centric video benchmarks, our work offers the following key features: (1) Diverse evaluation dimensions: HV-MMBench encompasses 13 tasks, ranging from basic attribute perception (e.g., age estimation, emotion recognition) to advanced cognitive reasoning (e.g., social relationship prediction, intention prediction), enabling comprehensive assessment of model capabilities; (2) Varied data types: The benchmark includes multiple-choice, fill-in-blank, true/false, and open-ended question formats, combined with diverse evaluation metrics, to more accurately and robustly reflect model performance; (3) Multi-domain video coverage: The benchmark spans 50 distinct visual scenarios, enabling comprehensive evaluation across fine-grained scene variations; (4) Temporal coverage: The benchmark covers videos from short-term (10 seconds) to long-term (up to 30min) durations, supporting systematic analysis of models temporal reasoning abilities across diverse contextual lengths.

Method: Three-step research review process: 1) Discuss how VLMs function for object detection using NLP and CV techniques, 2) Explain architectural innovations and training paradigms, 3) Examine approaches for visual-textual integration and demonstrate effectiveness through comprehensive visualizations.

Result: LVLMs show advanced contextual understanding and sophisticated object detection strategies, with performance expected to soon meet or surpass conventional methods. The review identifies current limitations and proposes solutions.

Conclusion: Recent advancements in LVLMs have and will continue to make transformative impacts on object detection and robotic applications, with a clear roadmap for future advancement in the field.

Abstract: The fusion of language and vision in large vision-language models (LVLMs) has revolutionized deep learning-based object detection by enhancing adaptability, contextual reasoning, and generalization beyond traditional architectures. This in-depth review presents a structured exploration of the state-of-the-art in LVLMs, systematically organized through a three-step research review process. First, we discuss the functioning of vision language models (VLMs) for object detection, describing how these models harness natural language processing (NLP) and computer vision (CV) techniques to revolutionize object detection and localization. We then explain the architectural innovations, training paradigms, and output flexibility of recent LVLMs for object detection, highlighting how they achieve advanced contextual understanding for object detection. The review thoroughly examines the approaches used in integration of visual and textual information, demonstrating the progress made in object detection using VLMs that facilitate more sophisticated object detection and localization strategies. This review presents comprehensive visualizations demonstrating LVLMs’ effectiveness in diverse scenarios including localization and segmentation, and then compares their real-time performance, adaptability, and complexity to traditional deep learning systems. Based on the review, its is expected that LVLMs will soon meet or surpass the performance of conventional methods in object detection. The review also identifies a few major limitations of the current LVLM modes, proposes solutions to address those challenges, and presents a clear roadmap for the future advancement in this field. We conclude, based on this study, that the recent advancement in LVLMs have made and will continue to make a transformative impact on object detection and robotic applications in the future.

Method: Methodological and experimental analysis of 10 explanation metrics across 5 categories (faithfulness, robustness, localization, complexity, randomization) applied to 5 feature attribution methods (Occlusion, LIME, GradCAM, LRP, DeepLIFT) on three RS datasets.

Result: Perturbation-based methods depend on perturbation baselines and spatial characteristics; gradient-based methods struggle with multiple labels; relevance propagation methods can distribute relevance disproportionately. Faithfulness metrics share perturbation issues; localization and complexity metrics are unreliable for large spatial extent classes; robustness and randomization metrics show greater stability.

Conclusion: Guidelines provided for selecting explanation methods, metrics, and hyperparameters in RS image scene classification context based on identified limitations and performance characteristics.

Abstract: The development of explainable artificial intelligence (xAI) methods for scene classification problems has attracted great attention in remote sensing (RS). Most xAI methods and the related evaluation metrics in RS are initially developed for natural images considered in computer vision (CV), and their direct usage in RS may not be suitable. To address this issue, in this paper, we investigate the effectiveness of explanation methods and metrics in the context of RS image scene classification. In detail, we methodologically and experimentally analyze ten explanation metrics spanning five categories (faithfulness, robustness, localization, complexity, randomization), applied to five established feature attribution methods (Occlusion, LIME, GradCAM, LRP, and DeepLIFT) across three RS datasets. Our methodological analysis identifies key limitations in both explanation methods and metrics. The performance of perturbation-based methods, such as Occlusion and LIME, heavily depends on perturbation baselines and spatial characteristics of RS scenes. Gradient-based approaches like GradCAM struggle when multiple labels are present in the same image, while some relevance propagation methods (LRP) can distribute relevance disproportionately relative to the spatial extent of classes. Analogously, we find limitations in evaluation metrics. Faithfulness metrics share the same problems as perturbation-based methods. Localization metrics and complexity metrics are unreliable for classes with a large spatial extent. In contrast, robustness metrics and randomization metrics consistently exhibit greater stability. Our experimental results support these methodological findings. Based on our analysis, we provide guidelines for selecting explanation methods, metrics, and hyperparameters in the context of RS image scene classification.

Method: Proposes a generalizable feed-forward model that predicts spatially adaptive bilateral grids to correct photometric variations in a multi-view consistent manner. Uses hybrid self-supervised rendering loss with 3D foundation models to overcome lack of paired data.

Result: Processes hundreds of frames in a single step, enabling efficient large-scale harmonization. Outperforms or matches scene-specific optimization methods’ reconstruction quality without significantly affecting baseline 3D models’ training time.

Conclusion: The approach provides cross-scene generalization without scene-specific retraining, seamlessly integrates into downstream 3D reconstruction models, and improves generalization to real-world variations.

Abstract: Modern camera pipelines apply extensive on-device processing, such as exposure adjustment, white balance, and color correction, which, while beneficial individually, often introduce photometric inconsistencies across views. These appearance variations violate multi-view consistency and degrade novel view synthesis. Joint optimization of scene-specific representations and per-image appearance embeddings has been proposed to address this issue, but with increased computational complexity and slower training. In this work, we propose a generalizable, feed-forward approach that predicts spatially adaptive bilateral grids to correct photometric variations in a multi-view consistent manner. Our model processes hundreds of frames in a single step, enabling efficient large-scale harmonization, and seamlessly integrates into downstream 3D reconstruction models, providing cross-scene generalization without requiring scene-specific retraining. To overcome the lack of paired data, we employ a hybrid self-supervised rendering loss leveraging 3D foundation models, improving generalization to real-world variations. Extensive experiments show that our approach outperforms or matches the reconstruction quality of existing scene-specific optimization methods with appearance modeling, without significantly affecting the training time of baseline 3D models.

Method: HyCASS consists of six modules: spectral encoder, spatial encoder, CR adapter encoder, CR adapter decoder, spatial decoder, and spectral decoder. It uses convolutional layers and transformer blocks to capture both short-range and long-range redundancies.

Result: Experimental results on three HSI benchmark datasets show HyCASS outperforms existing learning-based compression models by up to 2.36 dB in PSNR.

Conclusion: The study provides guidelines for effectively balancing spectral and spatial compression across different compression ratios, considering the spatial resolution of hyperspectral images.

Abstract: With the rapid growth of hyperspectral data archives in remote sensing (RS), the need for efficient storage has become essential, driving significant attention toward learning-based hyperspectral image (HSI) compression. However, a comprehensive investigation of the individual and joint effects of spectral and spatial compression on learning-based HSI compression has not been thoroughly examined yet. Conducting such an analysis is crucial for understanding how the exploitation of spectral, spatial, and joint spatio-spectral redundancies affects HSI compression. To address this issue, we propose Adjustable Spatio-Spectral Hyperspectral Image Compression Network (HyCASS), a learning-based model designed for adjustable HSI compression in both spectral and spatial dimensions. HyCASS consists of six main modules: 1) spectral encoder module; 2) spatial encoder module; 3) compression ratio (CR) adapter encoder module; 4) CR adapter decoder module; 5) spatial decoder module; and 6) spectral decoder module. The modules employ convolutional layers and transformer blocks to capture both short-range and long-range redundancies. Experimental results on three HSI benchmark datasets demonstrate the effectiveness of our proposed adjustable model compared to existing learning-based compression models, surpassing the state of the art by up to 2.36 dB in terms of PSNR. Based on our results, we establish a guideline for effectively balancing spectral and spatial compression across different CRs, taking into account the spatial resolution of the HSIs. Our code and pre-trained model weights are publicly available at https://git.tu-berlin.de/rsim/hycass .

Method: Proposed FoundBioNet uses SWIN-UNETR architecture with two key modules: Tumor-Aware Feature Encoding (TAFE) for multi-scale tumor-focused features, and Cross-Modality Differential (CMD) for highlighting T2-FLAIR mismatch signals. Trained on 1705 glioma patients from six public datasets.

Result: Achieved AUCs of 90.58%, 88.08%, 65.41%, and 80.31% on independent test sets from EGD, TCGA, Ivy GAP, RHUH, and UPenn, consistently outperforming baseline approaches (p ≤ 0.05). Ablation studies confirmed both TAFE and CMD modules are essential.

Conclusion: FoundBioNet enables generalizable glioma characterization through large-scale pretraining and task-specific fine-tuning, enhancing diagnostic accuracy and interpretability with potential for personalized patient care.

Abstract: Accurate, noninvasive detection of isocitrate dehydrogenase (IDH) mutation is essential for effective glioma management. Traditional methods rely on invasive tissue sampling, which may fail to capture a tumor’s spatial heterogeneity. While deep learning models have shown promise in molecular profiling, their performance is often limited by scarce annotated data. In contrast, foundation deep learning models offer a more generalizable approach for glioma imaging biomarkers. We propose a Foundation-based Biomarker Network (FoundBioNet) that utilizes a SWIN-UNETR-based architecture to noninvasively predict IDH mutation status from multi-parametric MRI. Two key modules are incorporated: Tumor-Aware Feature Encoding (TAFE) for extracting multi-scale, tumor-focused features, and Cross-Modality Differential (CMD) for highlighting subtle T2-FLAIR mismatch signals associated with IDH mutation. The model was trained and validated on a diverse, multi-center cohort of 1705 glioma patients from six public datasets. Our model achieved AUCs of 90.58%, 88.08%, 65.41%, and 80.31% on independent test sets from EGD, TCGA, Ivy GAP, RHUH, and UPenn, consistently outperforming baseline approaches (p <= 0.05). Ablation studies confirmed that both the TAFE and CMD modules are essential for improving predictive accuracy. By integrating large-scale pretraining and task-specific fine-tuning, FoundBioNet enables generalizable glioma characterization. This approach enhances diagnostic accuracy and interpretability, with the potential to enable more personalized patient care.

Method: VSF dynamically suppresses undesired content by flipping the sign of attention values from negative prompts. It works with MMDiT-style architectures like Stable Diffusion 3.5 Turbo and cross-attention-based models like Wan.

Result: VSF significantly improves negative prompt adherence compared to prior methods in few-step models and even CFG in non-few-step models, while maintaining competitive image quality. Validated on challenging datasets with complex prompt pairs.

Conclusion: VSF provides superior negative prompt guidance with minimal computational overhead, making it effective for both static image and video generation tasks in few-step diffusion models.

Abstract: We introduce Value Sign Flip (VSF), a simple and efficient method for incorporating negative prompt guidance in few-step diffusion and flow-matching image generation models. Unlike existing approaches such as classifier-free guidance (CFG), NASA, and NAG, VSF dynamically suppresses undesired content by flipping the sign of attention values from negative prompts. Our method requires only small computational overhead and integrates effectively with MMDiT-style architectures such as Stable Diffusion 3.5 Turbo, as well as cross-attention-based models like Wan. We validate VSF on challenging datasets with complex prompt pairs and demonstrate superior performance in both static image and video generation tasks. Experimental results show that VSF significantly improves negative prompt adherence compared to prior methods in few-step models, and even CFG in non-few-step models, while maintaining competitive image quality. Code and ComfyUI node are available in https://github.com/weathon/VSF/tree/main.

Method: Proposes Image-Conditioned Gaussian Splat Quantizer that exploits inter-Gaussian and inter-attribute correlations, uses shared codebooks across scenes, and enables conditional decoding based on images captured at decoding time. The encoding, quantization, and decoding are jointly trained.

Result: ICGS-Quantizer reduces 3DGS storage to kilobyte range while preserving visual fidelity and consistently outperforms state-of-the-art methods in both compression efficiency and adaptability to scene changes.

Conclusion: The method successfully addresses limitations of existing 3DGS compression by providing substantial compression efficiency and post-archival adaptability, making it suitable for large-scale scenes and long-term archival applications.

Abstract: 3D Gaussian Splatting (3DGS) has attracted considerable attention for enabling high-quality real-time rendering. Although 3DGS compression methods have been proposed for deployment on storage-constrained devices, two limitations hinder archival use: (1) they compress medium-scale scenes only to the megabyte range, which remains impractical for large-scale scenes or extensive scene collections; and (2) they lack mechanisms to accommodate scene changes after long-term archival. To address these limitations, we propose an Image-Conditioned Gaussian Splat Quantizer (ICGS-Quantizer) that substantially enhances compression efficiency and provides adaptability to scene changes after archiving. ICGS-Quantizer improves quantization efficiency by jointly exploiting inter-Gaussian and inter-attribute correlations and by using shared codebooks across all training scenes, which are then fixed and applied to previously unseen test scenes, eliminating the overhead of per-scene codebooks. This approach effectively reduces the storage requirements for 3DGS to the kilobyte range while preserving visual fidelity. To enable adaptability to post-archival scene changes, ICGS-Quantizer conditions scene decoding on images captured at decoding time. The encoding, quantization, and decoding processes are trained jointly, ensuring that the codes, which are quantized representations of the scene, are effective for conditional decoding. We evaluate ICGS-Quantizer on 3D scene compression and 3D scene updating. Experimental results show that ICGS-Quantizer consistently outperforms state-of-the-art methods in compression efficiency and adaptability to scene changes. Our code, model, and data will be publicly available on GitHub.

Method: Two modules: (1) Attribute disentanglement using Brownian Bridge Diffusion Model to separate non-individualized and individualized attributes; (2) Granule learning to construct visual granules by integrating attributes under causal inference strategies for more discriminative text prompts.

Result: Extensive experiments on 15 datasets show CaPL significantly outperforms state-of-the-art prompt learning methods, especially on fine-grained datasets.

Conclusion: The proposed CaPL method effectively addresses fine-grained recognition challenges in CLIP through visual granulation and causal inference, achieving superior performance compared to existing approaches.

Abstract: Prompt learning has recently attracted much attention for adapting pre-trained vision-language models (e.g., CLIP) to downstream recognition tasks. However, most of the existing CLIP-based prompt learning methods only show a limited ability for handling fine-grained datasets. To address this issue, we propose a causality-guided text prompt learning method via visual granulation for CLIP, called CaPL, where the explored visual granulation technique could construct sets of visual granules for the text prompt to capture subtle discrepancies among different fine-grained classes through casual inference. The CaPL method contains the following two modules: (1) An attribute disentanglement module is proposed to decompose visual features into non-individualized attributes (shared by some classes) and individualized attributes (specific to single classes) using a Brownian Bridge Diffusion Model; (2) A granule learning module is proposed to construct visual granules by integrating the aforementioned attributes for recognition under two causal inference strategies. Thanks to the learned visual granules, more discriminative text prompt is expected to be learned. Extensive experimental results on 15 datasets demonstrate that our CaPL method significantly outperforms the state-of-the-art prompt learning methods, especially on fine-grained datasets.

Method: Three main contributions: (i) temporal embedding similarity for aligning consecutive latent representations, (ii) motion-aligned transformation loss matching displacement between predictions and ground truth LiDAR, and (iii) windowed temporal fusion using a specialized temporal module. Also updates model architecture to better preserve spatial structure.

Result: Improved temporal and spatial coherence demonstrated through lower absolute trajectory error and better occupancy map accuracy in Cartographer-based SLAM. Proposed new metrics based on Fréchet Video Motion Distance (FVMD) and correlation-peak distance for evaluating temporal quality.

Conclusion: LiDAR-BIND-T maintains modular modality fusion while substantially enhancing temporal stability, resulting in improved robustness and performance for downstream SLAM applications.

Abstract: This paper extends LiDAR-BIND, a modular multi-modal fusion framework that binds heterogeneous sensors (radar, sonar) to a LiDAR-defined latent space, with mechanisms that explicitly enforce temporal consistency. We introduce three contributions: (i) temporal embedding similarity that aligns consecutive latent representations, (ii) a motion-aligned transformation loss that matches displacement between predictions and ground truth LiDAR, and (iii) windowed temporal fusion using a specialised temporal module. We further update the model architecture to better preserve spatial structure. Evaluations on radar/sonar-to-LiDAR translation demonstrate improved temporal and spatial coherence, yielding lower absolute trajectory error and better occupancy map accuracy in Cartographer-based SLAM (Simultaneous Localisation and Mapping). We propose different metrics based on the Fr'echet Video Motion Distance (FVMD) and a correlation-peak distance metric providing practical temporal quality indicators to evaluate SLAM performance. The proposed temporal LiDAR-BIND, or LiDAR-BIND-T, maintains modular modality fusion while substantially enhancing temporal stability, resulting in improved robustness and performance for downstream SLAM.

Method: Proposed a pathology-specific VL training scheme that generates enhanced and perturbed samples for multimodal contrastive learning, and introduced PathoHR-Bench benchmark for evaluation.

Result: The approach achieves state-of-the-art performance on PathoHR-Bench and six additional pathology datasets, demonstrating effectiveness in fine-grained pathology representation.

Conclusion: The proposed pathology-specific training scheme effectively addresses limitations of existing VL models in capturing complex cross-modal relationships for pathological image analysis.

Abstract: Accurate analysis of pathological images is essential for automated tumor diagnosis but remains challenging due to high structural similarity and subtle morphological variations in tissue images. Current vision-language (VL) models often struggle to capture the complex reasoning required for interpreting structured pathological reports. To address these limitations, we propose PathoHR-Bench, a novel benchmark designed to evaluate VL models’ abilities in hierarchical semantic understanding and compositional reasoning within the pathology domain. Results of this benchmark reveal that existing VL models fail to effectively model intricate cross-modal relationships, hence limiting their applicability in clinical setting. To overcome this, we further introduce a pathology-specific VL training scheme that generates enhanced and perturbed samples for multimodal contrastive learning. Experimental evaluations demonstrate that our approach achieves state-of-the-art performance on PathoHR-Bench and six additional pathology datasets, highlighting its effectiveness in fine-grained pathology representation.

Method: Created Ego3D-Bench benchmark with 8,600+ human-annotated QA pairs from ego-centric multi-view outdoor data. Proposed Ego3D-VLM framework that generates cognitive maps based on estimated global 3D coordinates to enhance spatial reasoning.

Result: Benchmarked 16 SOTA VLMs showing significant performance gap vs human level. Ego3D-VLM achieved 12% average improvement on multi-choice QA and 56% average improvement on absolute distance estimation. The framework is modular and compatible with existing VLMs.

Conclusion: Ego3D-Bench and Ego3D-VLM provide valuable tools for advancing human-level spatial understanding in real-world multi-view environments. Current VLMs still fall short of human spatial reasoning capabilities despite recent advances.

Abstract: Understanding 3D spatial relationships remains a major limitation of current Vision-Language Models (VLMs). Prior work has addressed this issue by creating spatial question-answering (QA) datasets based on single images or indoor videos. However, real-world embodied AI agents such as robots and self-driving cars typically rely on ego-centric, multi-view observations. To this end, we introduce Ego3D-Bench, a new benchmark designed to evaluate the spatial reasoning abilities of VLMs using ego-centric, multi-view outdoor data. Ego3D-Bench comprises over 8,600 QA pairs, created with significant involvement from human annotators to ensure quality and diversity. We benchmark 16 SOTA VLMs, including GPT-4o, Gemini1.5-Pro, InternVL3, and Qwen2.5-VL. Our results reveal a notable performance gap between human level scores and VLM performance, highlighting that current VLMs still fall short of human level spatial understanding. To bridge this gap, we propose Ego3D-VLM, a post-training framework that enhances 3D spatial reasoning of VLMs. Ego3D-VLM generates cognitive map based on estimated global 3D coordinates, resulting in 12% average improvement on multi-choice QA and 56% average improvement on absolute distance estimation. Ego3D-VLM is modular and can be integrated with any existing VLM. Together, Ego3D-Bench and Ego3D-VLM offer valuable tools for advancing toward human level spatial understanding in real-world, multi-view environments.

Method: Proposes UAE with two stages: (1) Generation for Understanding - encoder generates informative captions to maximize decoder’s reconstruction quality, enhancing visual perception; (2) Understanding for Generation - decoder reconstructs images from these captions, improving long-context instruction following and generation fidelity.

Result: Empirical results show understanding enhances generation (verified on GenEval), while generation strengthens fine-grained visual perception like small object and color recognition (verified on MMT-Bench).

Conclusion: Under the unified reconstruction objective, generation and understanding can mutually benefit each other, moving closer to truly unified multimodal intelligence.

Abstract: The pursuit of unified multimodal models (UMMs) has long been hindered by a fundamental schism between multimodal understanding and generation. Current approaches typically disentangle the two and treat them as separate endeavors with disjoint objectives, missing the mutual benefits. We argue that true unification requires more than just merging two tasks. It requires a unified, foundational objective that intrinsically links them. In this paper, we introduce an insightful paradigm through the Auto-Encoder lens, i.e., regarding understanding as the encoder (I2T) that compresses images into text, and generation as the decoder (T2I) that reconstructs images from that text. To implement this, we propose UAE, where we begin by pre-training the decoder with the proposed 700k long-context image-caption pairs to direct it to “understand” the fine-grained and complex semantics from the text. We then propose Unified-GRPO via reinforcement learning (RL) to unify the two, which covers two complementary stages: (1) Generation for Understanding, where the encoder is trained to generate informative captions that maximize the decoder’s reconstruction quality, enhancing its visual perception; (2) Understanding for Generation, where the decoder is refined to reconstruct from these captions, forcing it to leverage every detail and improving its long-context instruction following and generation fidelity. Our empirical results suggest that understanding can largely enhance generation (verified on GenEval), while generation, in turn, notably strengthens fine-grained visual perception like small object and color recognition (verified on MMT-Bench). This bidirectional improvement reveals a deep synergy: under the unified reconstruction objective, generation and understanding can mutually benefit each other, moving closer to truly unified multimodal intelligence.

Method: Progressive layer freezing based on an explicit schedule derived from layer convergence patterns, applied to masked-autoencoding models with up to 4B parameters.

Result: LayerLock surpasses non-latent masked prediction on the 4DS perception suite and enables effective latent prediction without representation collapse.

Conclusion: Progressive layer freezing is a simple yet effective approach that accelerates MAE training and enables scalable latent prediction for large vision models.

Abstract: We introduce LayerLock, a simple yet effective approach for self-supervised visual representation learning, that gradually transitions from pixel to latent prediction through progressive layer freezing. First, we make the observation that during training of video masked-autoencoding (MAE) models, ViT layers converge in the order of their depth: shallower layers converge early, deeper layers converge late. We then show that this observation can be exploited to accelerate standard MAE by progressively freezing the model according to an explicit schedule, throughout training. Furthermore, this same schedule can be used in a simple and scalable approach to latent prediction that does not suffer from “representation collapse”. We apply our proposed approach, LayerLock, to large models of up to 4B parameters with results surpassing those of non-latent masked prediction on the 4DS perception suite.

Method: Integrates Mamba2 state space models into U-Net architecture, adds interactive click prompts with cross-attention blocks, uses self-supervised pre-training, and incorporates dental domain knowledge.

Result: Achieved first place in both tasks of ToothFairy3 challenge: Task 1 - Dice 0.84, HD95 38.17, inference time 40.58s; Task 2 - Dice 0.87, HD95 2.15.

Conclusion: U-Mamba2 is both effective and efficient for dental CBCT segmentation, demonstrating strong performance while maintaining computational efficiency through structural constraints and domain-specific design.

Abstract: Cone-Beam Computed Tomography (CBCT) is a widely used 3D imaging technique in dentistry, providing volumetric information about the anatomical structures of jaws and teeth. Accurate segmentation of these anatomies is critical for clinical applications such as diagnosis and surgical planning, but remains time-consuming and challenging. In this paper, we present U-Mamba2, a new neural network architecture designed for multi-anatomy CBCT segmentation in the context of the ToothFairy3 challenge. U-Mamba2 integrates the Mamba2 state space models into the U-Net architecture, enforcing stronger structural constraints for higher efficiency without compromising performance. In addition, we integrate interactive click prompts with cross-attention blocks, pre-train U-Mamba2 using self-supervised learning, and incorporate dental domain knowledge into the model design to address key challenges of dental anatomy segmentation in CBCT. Extensive experiments, including independent tests, demonstrate that U-Mamba2 is both effective and efficient, securing first place in both tasks of the Toothfairy3 challenge. In Task 1, U-Mamba2 achieved a mean Dice of 0.84, HD95 of 38.17 with the held-out test data, with an average inference time of 40.58s. In Task 2, U-Mamba2 achieved the mean Dice of 0.87 and HD95 of 2.15 with the held-out test data. The code is publicly available at https://github.com/zhiqin1998/UMamba2.

Method: Created OrthoLoC dataset with 16,425 UAV images from Germany and US with multiple modalities, enabling fair benchmarking by decoupling image retrieval from feature matching. Also introduced AdHoP refinement technique that can be integrated with any feature matcher.

Result: Comprehensive evaluation shows impact of domain shifts, data resolutions, and covisibility on localization accuracy. AdHoP improves matching by up to 95% and reduces translation error by up to 63%.

Conclusion: OrthoLoC enables effective visual localization using orthographic geodata, with AdHoP providing significant performance improvements for feature matching in this challenging domain.

Abstract: Accurate visual localization from aerial views is a fundamental problem with applications in mapping, large-area inspection, and search-and-rescue operations. In many scenarios, these systems require high-precision localization while operating with limited resources (e.g., no internet connection or GNSS/GPS support), making large image databases or heavy 3D models impractical. Surprisingly, little attention has been given to leveraging orthographic geodata as an alternative paradigm, which is lightweight and increasingly available through free releases by governmental authorities (e.g., the European Union). To fill this gap, we propose OrthoLoC, the first large-scale dataset comprising 16,425 UAV images from Germany and the United States with multiple modalities. The dataset addresses domain shifts between UAV imagery and geospatial data. Its paired structure enables fair benchmarking of existing solutions by decoupling image retrieval from feature matching, allowing isolated evaluation of localization and calibration performance. Through comprehensive evaluation, we examine the impact of domain shifts, data resolutions, and covisibility on localization accuracy. Finally, we introduce a refinement technique called AdHoP, which can be integrated with any feature matcher, improving matching by up to 95% and reducing translation error by up to 63%. The dataset and code are available at: https://deepscenario.github.io/OrthoLoC.

Method: Uses lightweight, learnable, invertible preprocessing with spatial and optional frequency-domain transforms. Compact parameters are stored in JPEG comment field to enable accurate raw reconstruction.

Result: Achieves higher fidelity than direct JPEG storage, supports other codecs, and provides favorable trade-off between compression ratio and reconstruction accuracy across multiple datasets.

Conclusion: RawJPEG Adapter offers an effective solution for raw image compression using standard JPEG format while maintaining reconstruction capability through metadata storage.

Abstract: Digital cameras digitize scene light into linear raw representations, which the image signal processor (ISP) converts into display-ready outputs. While raw data preserves full sensor information–valuable for editing and vision tasks–formats such as Digital Negative (DNG) require large storage, making them impractical in constrained scenarios. In contrast, JPEG is a widely supported format, offering high compression efficiency and broad compatibility, but it is not well-suited for raw storage. This paper presents RawJPEG Adapter, a lightweight, learnable, and invertible preprocessing pipeline that adapts raw images for standard JPEG compression. Our method applies spatial and optional frequency-domain transforms, with compact parameters stored in the JPEG comment field, enabling accurate raw reconstruction. Experiments across multiple datasets show that our method achieves higher fidelity than direct JPEG storage, supports other codecs, and provides a favorable trade-off between compression ratio and reconstruction accuracy.

Method: Leverages MSCKF-based monocular visual-inertial motion tracking to exploit multi-view information for iterative per-pixel scale refinement, rather than global affine fitting. The framework is modular and compatible with various depth estimation backbones.

Result: Achieves exceptional accuracy and robustness on TartanAir and VOID datasets, with strong zero-shot generalization on AR Table dataset. Works effectively with extremely sparse points (10-20 metric depth points per image).

Conclusion: VIMD provides a practical solution for deployment in resource-constrained settings, offering robust performance and strong generalization capabilities across various scenarios.

Abstract: Accurate and efficient dense metric depth estimation is crucial for 3D visual perception in robotics and XR. In this paper, we develop a monocular visual-inertial motion and depth (VIMD) learning framework to estimate dense metric depth by leveraging accurate and efficient MSCKF-based monocular visual-inertial motion tracking. At the core the proposed VIMD is to exploit multi-view information to iteratively refine per-pixel scale, instead of globally fitting an invariant affine model as in the prior work. The VIMD framework is highly modular, making it compatible with a variety of existing depth estimation backbones. We conduct extensive evaluations on the TartanAir and VOID datasets and demonstrate its zero-shot generalization capabilities on the AR Table dataset. Our results show that VIMD achieves exceptional accuracy and robustness, even with extremely sparse points as few as 10-20 metric depth points per image. This makes the proposed VIMD a practical solution for deployment in resource constrained settings, while its robust performance and strong generalization capabilities offer significant potential across a wide range of scenarios.

Method: The framework builds on U-Mamba2 with a multi-stage training strategy: self-supervised pre-training using a disruptive autoencoder, consistency regularization with input and feature perturbations, and pseudo-labeling with reduced loss weighting.

Result: U-Mamba2-SSL achieved an average score of 0.789 and a DSC of 0.917 on the hidden test set, winning first place in Task 1 of the STSR 2025 challenge.

Conclusion: The proposed semi-supervised learning framework effectively addresses the challenge of teeth and pulp segmentation in CBCT images by leveraging unlabeled data through innovative training strategies.

Abstract: Accurate segmentation of teeth and pulp in Cone-Beam Computed Tomography (CBCT) is vital for clinical applications like treatment planning and diagnosis. However, this process requires extensive expertise and is exceptionally time-consuming, highlighting the critical need for automated algorithms that can effectively utilize unlabeled data. In this paper, we propose U-Mamba2-SSL, a novel semi-supervised learning framework that builds on the U-Mamba2 model and employs a multi-stage training strategy. The framework first pre-trains U-Mamba2 in a self-supervised manner using a disruptive autoencoder. It then leverages unlabeled data through consistency regularization, where we introduce input and feature perturbations to ensure stable model outputs. Finally, a pseudo-labeling strategy is implemented with a reduced loss weighting to minimize the impact of potential errors. U-Mamba2-SSL achieved an average score of 0.789 and a DSC of 0.917 on the hidden test set, achieving first place in Task 1 of the STSR 2025 challenge. The code is available at https://github.com/zhiqin1998/UMamba2.

Method: Created SHOT dataset with 1,979 basketball video clips from 5 camera views, annotated with 6 individual attributes. Proposed GIFT framework that extracts individual features and models group dynamics.

Result: Experimental results confirm the effectiveness of both SHOT dataset and GIFT framework for group intention forecasting.

Conclusion: The work establishes a foundation for future research in group intention forecasting, with the dataset publicly available for community use.

Abstract: Intention recognition has traditionally focused on individual intentions, overlooking the complexities of collective intentions in group settings. To address this limitation, we introduce the concept of group intention, which represents shared goals emerging through the actions of multiple individuals, and Group Intention Forecasting (GIF), a novel task that forecasts when group intentions will occur by analyzing individual actions and interactions before the collective goal becomes apparent. To investigate GIF in a specific scenario, we propose SHOT, the first large-scale dataset for GIF, consisting of 1,979 basketball video clips captured from 5 camera views and annotated with 6 types of individual attributes. SHOT is designed with 3 key characteristics: multi-individual information, multi-view adaptability, and multi-level intention, making it well-suited for studying emerging group intentions. Furthermore, we introduce GIFT (Group Intention ForecasTer), a framework that extracts fine-grained individual features and models evolving group dynamics to forecast intention emergence. Experimental results confirm the effectiveness of SHOT and GIFT, establishing a strong foundation for future research in group intention forecasting. The dataset is available at https://xinyi-hu.github.io/SHOT_DATASET.

Method: Proposes Dual-Smoothed Fine-Grained Quantization (pre-global Hadamard rotation + post-local channel smoothing) and Noise-Filtered Diverse Sampling (outlier filtering + frame-aware calibration clusters).

Result: Achieves state-of-the-art results across benchmarks and bit-widths, with 4-bit quantization delivering 3.7× memory reduction and 2.5× acceleration while maintaining 98% of full-precision accuracy.

Conclusion: QuantVGGT demonstrates significant advantages for deploying large 3D reconstruction transformers in resource-constrained scenarios, offering practical compression without sacrificing performance.

Abstract: Learning-based 3D reconstruction models, represented by Visual Geometry Grounded Transformers (VGGTs), have made remarkable progress with the use of large-scale transformers. Their prohibitive computational and memory costs severely hinder real-world deployment. Post-Training Quantization (PTQ) has become a common practice for compressing and accelerating models. However, we empirically observe that PTQ faces unique obstacles when compressing billion-scale VGGTs: the data-independent special tokens induce heavy-tailed activation distributions, while the multi-view nature of 3D data makes calibration sample selection highly unstable. This paper proposes the first Quantization framework for VGGTs, namely QuantVGGT. This mainly relies on two technical contributions: First, we introduce Dual-Smoothed Fine-Grained Quantization, which integrates pre-global Hadamard rotation and post-local channel smoothing to mitigate heavy-tailed distributions and inter-channel variance robustly. Second, we design Noise-Filtered Diverse Sampling, which filters outliers via deep-layer statistics and constructs frame-aware diverse calibration clusters to ensure stable quantization ranges. Comprehensive experiments demonstrate that QuantVGGT achieves the state-of-the-art results across different benchmarks and bit-width, surpassing the previous state-of-the-art generic quantization method with a great margin. We highlight that our 4-bit QuantVGGT can deliver a 3.7$\times$ memory reduction and 2.5$\times$ acceleration in real-hardware inference, while maintaining reconstruction accuracy above 98% of its full-precision counterpart. This demonstrates the vast advantages and practicality of QuantVGGT in resource-constrained scenarios. Our code is released in https://github.com/wlfeng0509/QuantVGGT.

Method: MoWM uses motion-aware representations from a latent model as a high-level prior to guide extraction of fine-grained visual features from pixel space model, highlighting informative visual details for action decoding.

Result: Extensive evaluations on CALVIN benchmark demonstrate state-of-the-art task success rates and superior generalization compared to existing methods.

Conclusion: The proposed MoWM framework effectively combines the strengths of different world models for embodied action planning, providing valuable insights for future research in this area.

Abstract: Embodied action planning is a core challenge in robotics, requiring models to generate precise actions from visual observations and language instructions. While video generation world models are promising, their reliance on pixel-level reconstruction often introduces visual redundancies that hinder action decoding and generalization. Latent world models offer a compact, motion-aware representation, but overlook the fine-grained details critical for precise manipulation. To overcome these limitations, we propose MoWM, a mixture-of-world-model framework that fuses representations from hybrid world models for embodied action planning. Our approach uses motion-aware representations from a latent model as a high-level prior, which guides the extraction of fine-grained visual features from the pixel space model. This design allows MoWM to highlight the informative visual details needed for action decoding. Extensive evaluations on the CALVIN benchmark demonstrate that our method achieves state-of-the-art task success rates and superior generalization. We also provide a comprehensive analysis of the strengths of each feature space, offering valuable insights for future research in embodied planning. The code is available at: https://github.com/tsinghua-fib-lab/MoWM.

Method: HDR-4DGS uses Gaussian Splatting with a dynamic tone-mapping module that adapts tone-mapping functions according to evolving radiance distributions across time, maintaining temporal radiance coherence.

Result: Achieves temporal radiance consistency and spatially accurate color translation, enabling photorealistic HDR renderings from arbitrary viewpoints and time instances, surpassing state-of-the-art methods.

Conclusion: HDR-4DGS effectively addresses the challenging HDR DNVS problem by jointly modeling temporal radiance variations and 3D translation between LDR and HDR domains.

Abstract: High Dynamic Range Novel View Synthesis (HDR NVS) seeks to learn an HDR 3D model from Low Dynamic Range (LDR) training images captured under conventional imaging conditions. Current methods primarily focus on static scenes, implicitly assuming all scene elements remain stationary and non-living. However, real-world scenarios frequently feature dynamic elements, such as moving objects, varying lighting conditions, and other temporal events, thereby presenting a significantly more challenging scenario. To address this gap, we propose a more realistic problem named HDR Dynamic Novel View Synthesis (HDR DNVS), where the additional dimension ``Dynamic’’ emphasizes the necessity of jointly modeling temporal radiance variations alongside sophisticated 3D translation between LDR and HDR. To tackle this complex, intertwined challenge, we introduce HDR-4DGS, a Gaussian Splatting-based architecture featured with an innovative dynamic tone-mapping module that explicitly connects HDR and LDR domains, maintaining temporal radiance coherence by dynamically adapting tone-mapping functions according to the evolving radiance distributions across the temporal dimension. As a result, HDR-4DGS achieves both temporal radiance consistency and spatially accurate color translation, enabling photorealistic HDR renderings from arbitrary viewpoints and time instances. Extensive experiments demonstrate that HDR-4DGS surpasses existing state-of-the-art methods in both quantitative performance and visual fidelity. Source code will be released.

Method: Three-stage iterative approach: (1) Diverse Sample Generation for exploring reasoning paths, (2) Dual Foresight Decoding evaluating samples based on visual focus and reasoning progression, (3) Visual Focus Adjustment to refine attention regions, creating an interdependent reasoning-focus cycle.

Result: Consistent performance improvements of 3.1-5.8% across multiple benchmarks using Qwen2.5-VL, InternVL-2.5, and Llava-Next with controllable computational overhead.

Conclusion: CoFFT effectively enhances VLM visual reasoning by mimicking human cognitive processes, addressing limitations of current VLMs in handling complex visual information through iterative focus-reasoning cycles.

Abstract: Despite significant advances in Vision Language Models (VLMs), they remain constrained by the complexity and redundancy of visual input. When images contain large amounts of irrelevant information, VLMs are susceptible to interference, thus generating excessive task-irrelevant reasoning processes or even hallucinations. This limitation stems from their inability to discover and process the required regions during reasoning precisely. To address this limitation, we present the Chain of Foresight-Focus Thought (CoFFT), a novel training-free approach that enhances VLMs’ visual reasoning by emulating human visual cognition. Each Foresight-Focus Thought consists of three stages: (1) Diverse Sample Generation: generates diverse reasoning samples to explore potential reasoning paths, where each sample contains several reasoning steps; (2) Dual Foresight Decoding: rigorously evaluates these samples based on both visual focus and reasoning progression, adding the first step of optimal sample to the reasoning process; (3) Visual Focus Adjustment: precisely adjust visual focus toward regions most beneficial for future reasoning, before returning to stage (1) to generate subsequent reasoning samples until reaching the final answer. These stages function iteratively, creating an interdependent cycle where reasoning guides visual focus and visual focus informs subsequent reasoning. Empirical results across multiple benchmarks using Qwen2.5-VL, InternVL-2.5, and Llava-Next demonstrate consistent performance improvements of 3.1-5.8% with controllable increasing computational overhead.

Method: Three key innovations: (1) One-Step Inversion-and-Editing pipeline to bypass iterative processes; (2) Background Shield technique for selective feature modification; (3) Sparsified Spatial Cross-Attention mechanism to prevent semantic leakage.

Result: FlashEdit maintains superior background consistency and structural integrity while performing edits in under 0.2 seconds, achieving over 150x speedup compared to prior multi-step methods.

Conclusion: FlashEdit successfully enables real-time, high-fidelity image editing with diffusion models while preserving background integrity, making it practical for real-world applications.

Abstract: Text-guided image editing with diffusion models has achieved remarkable quality but suffers from prohibitive latency, hindering real-world applications. We introduce FlashEdit, a novel framework designed to enable high-fidelity, real-time image editing. Its efficiency stems from three key innovations: (1) a One-Step Inversion-and-Editing (OSIE) pipeline that bypasses costly iterative processes; (2) a Background Shield (BG-Shield) technique that guarantees background preservation by selectively modifying features only within the edit region; and (3) a Sparsified Spatial Cross-Attention (SSCA) mechanism that ensures precise, localized edits by suppressing semantic leakage to the background. Extensive experiments demonstrate that FlashEdit maintains superior background consistency and structural integrity, while performing edits in under 0.2 seconds, which is an over 150$\times$ speedup compared to prior multi-step methods. Our code will be made publicly available at https://github.com/JunyiWuCode/FlashEdit.

Method: Proposed U-MAN architecture with two key modules: Progressive Attention-Guided Feature Fusion (PAGF) replaces simple skip connections using attention to fuse encoder-decoder features, and Multi-scale Adaptive KAN (MAN) enables adaptive multi-scale feature processing for various object sizes.

Result: Experiments on BUSI, GLAS, and CVC datasets show U-MAN outperforms state-of-the-art methods, particularly excelling in defining accurate boundaries and preserving fine details.

Conclusion: U-MAN effectively addresses U-Net limitations through attention-based feature fusion and multi-scale processing, demonstrating superior performance in medical image segmentation tasks.

Abstract: Medical image segmentation faces significant challenges in preserving fine-grained details and precise boundaries due to complex anatomical structures and pathological regions. These challenges primarily stem from two key limitations of conventional U-Net architectures: (1) their simple skip connections ignore the encoder-decoder semantic gap between various features, and (2) they lack the capability for multi-scale feature extraction in deep layers. To address these challenges, we propose the U-Net with Multi-scale Adaptive KAN (U-MAN), a novel architecture that enhances the emerging Kolmogorov-Arnold Network (KAN) with two specialized modules: Progressive Attention-Guided Feature Fusion (PAGF) and the Multi-scale Adaptive KAN (MAN). Our PAGF module replaces the simple skip connection, using attention to fuse features from the encoder and decoder. The MAN module enables the network to adaptively process features at multiple scales, improving its ability to segment objects of various sizes. Experiments on three public datasets (BUSI, GLAS, and CVC) show that U-MAN outperforms state-of-the-art methods, particularly in defining accurate boundaries and preserving fine details.

Method: Created MMPB benchmark with 10k image-query pairs across 111 concepts in humans, animals, objects, and characters categories. Evaluated 23 VLMs using three-stage protocol: concept injection, multi-turn dialogue, and personalized querying.

Result: Most VLMs struggle with personalization, particularly in maintaining dialogue consistency, handling user preferences, and adapting to visual cues. Challenges include refusal behaviors and long-context forgetting, showing substantial room for improvement.

Conclusion: MMPB identifies key limitations in VLM personalization and provides a scalable benchmark foundation for future research toward truly personalized multi-modal AI systems.

Abstract: Visual personalization is essential in user-facing AI systems such as smart homes and healthcare, where aligning model behavior with user-centric concepts is critical. However, recent large Vision-Language Models (VLMs), despite their broad applicability, remain underexplored in their ability to adapt to individual users. In this paper, we introduce MMPB, the first extensive benchmark for evaluating VLMs on personalization. MMPB comprises 10k image-query pairs and includes 111 personalizable concepts across four categories: humans, animals, objects, and characters, with the human category enriched with preference-grounded queries. We structure personalization into three main task types, each highlighting a different key property of VLMs. Using 23 widely used VLMs including both open- and closed-source models, we evaluate personalization performance via a three-stage protocol: concept injection, multi-turn dialogue, and personalized querying. Our findings indicate that most VLMs (including some closed-source models) struggle with personalization, particularly in maintaining consistency over dialogue, handling user preferences, and adapting to visual cues. Our analysis reveals that the challenges in VLM personalization (such as refusal behaviors and long-context forgetting) highlight substantial room for improvement. By identifying these limitations and offering a scalable benchmark, MMPB offers valuable insights and a solid foundation for future research toward truly personalized multi-modal AI. Project Page: aidaslab.github.io/MMPB

Method: Designed S2B-Conv with Spectral-Redistribution Mechanism (SRM) using dynamic affine transformations and Gabor Spatial Feature Amplifier (GSFA) with random frequency/angle selection to handle multi-scale anisotropic features. Multiple S2B-Conv modules form S2BNet.

Result: Extensive experiments show the proposed high-efficiency binarized method achieves promising performance in quantitative and qualitative evaluations.

Conclusion: S2BNet successfully applies binary neural networks to pansharpening by addressing spectral and spatial degradation issues, enabling efficient deployment on resource-limited devices while maintaining good performance.

Abstract: Remote sensing pansharpening aims to reconstruct spatial-spectral properties during the fusion of panchromatic (PAN) images and low-resolution multi-spectral (LR-MS) images, finally generating the high-resolution multi-spectral (HR-MS) images. Although deep learning-based models have achieved excellent performance, they often come with high computational complexity, which hinder their applications on resource-limited devices. In this paper, we explore the feasibility of applying the binary neural network (BNN) to pan-sharpening. Nevertheless, there are two main issues with binarizing pan-sharpening models: (i) the binarization will cause serious spectral distortion due to the inconsistent spectral distribution of the PAN/LR-MS images; (ii) the common binary convolution kernel is difficult to adapt to the multi-scale and anisotropic spatial features of remote sensing objects, resulting in serious degradation of contours. To address the above issues, we design the customized spatial-spectral binarized convolution (S2B-Conv), which is composed of the Spectral-Redistribution Mechanism (SRM) and Gabor Spatial Feature Amplifier (GSFA). Specifically, SRM employs an affine transformation, generating its scaling and bias parameters through a dynamic learning process. GSFA, which randomly selects different frequencies and angles within a preset range, enables to better handle multi-scale and-directional spatial features. A series of S2B-Conv form a brand-new binary network for pan-sharpening, dubbed as S2BNet. Extensive quantitative and qualitative experiments have shown our high-efficiency binarized pan-sharpening method can attain a promising performance.

Method: Assign Nyquist-informed, neuron-specific frequency multipliers to periodic activations, inspired by classical signal processing methods like Discrete Sine Transform (DST).

Result: Reduces feature redundancy by nearly 50%, consistently improves signal reconstruction in 1D audio, 2D image, 3D shape fitting, and NeRF synthesis while maintaining efficiency.

Conclusion: The proposed frequency multiplier approach effectively reduces feature redundancy and enhances performance across diverse INR tasks without requiring additional complexity.

Abstract: Existing periodic activation-based implicit neural representation (INR) networks, such as SIREN and FINER, suffer from hidden feature redundancy, where neurons within a layer capture overlapping frequency components due to the use of a fixed frequency multiplier. This redundancy limits the expressive capacity of multilayer perceptrons (MLPs). Drawing inspiration from classical signal processing methods such as the Discrete Sine Transform (DST), we propose FM-SIREN and FM-FINER, which assign Nyquist-informed, neuron-specific frequency multipliers to periodic activations. Unlike existing approaches, our design introduces frequency diversity without requiring hyperparameter tuning or additional network depth. This simple yet principled modification reduces the redundancy of features by nearly 50% and consistently improves signal reconstruction across diverse INR tasks, including fitting 1D audio, 2D image and 3D shape, and synthesis of neural radiance fields (NeRF), outperforming their baseline counterparts while maintaining efficiency.

Method: Optimizes a large latent taxonomic hierarchy using complete binary tree structured mixture-of-Gaussian prior within variational inference framework, automatically discovering taxonomic structures and prototype clusters from unlabeled data without assuming true label sizes.

Result: Empirically demonstrates strong hierarchical clustering performance, outperforming baselines across diverse image classification datasets using novel evaluation mechanism that leverages prototype clusters at all hierarchical levels.

Conclusion: Deep taxonomic networks discover rich and interpretable hierarchical taxonomies that capture both coarse-grained semantic categories and fine-grained visual distinctions, bridging gaps in current hierarchical clustering approaches.

Abstract: Inspired by the human ability to learn and organize knowledge into hierarchical taxonomies with prototypes, this paper addresses key limitations in current deep hierarchical clustering methods. Existing methods often tie the structure to the number of classes and underutilize the rich prototype information available at intermediate hierarchical levels. We introduce deep taxonomic networks, a novel deep latent variable approach designed to bridge these gaps. Our method optimizes a large latent taxonomic hierarchy, specifically a complete binary tree structured mixture-of-Gaussian prior within a variational inference framework, to automatically discover taxonomic structures and associated prototype clusters directly from unlabeled data without assuming true label sizes. We analytically show that optimizing the ELBO of our method encourages the discovery of hierarchical relationships among prototypes. Empirically, our learned models demonstrate strong hierarchical clustering performance, outperforming baselines across diverse image classification datasets using our novel evaluation mechanism that leverages prototype clusters discovered at all hierarchical levels. Qualitative results further reveal that deep taxonomic networks discover rich and interpretable hierarchical taxonomies, capturing both coarse-grained semantic categories and fine-grained visual distinctions.

Method: Training-free approach that uses single reference images to specify content and provides explicit guidance on element placement without requiring model retraining.

Result: Demonstrated effectiveness across various image composition tasks with explicit control over object and part-level composition.

Conclusion: The proposed method enables finer control over image generation by using reference images for content specification and explicit layout guidance, overcoming limitations of text-only control.

Abstract: Text-to-image models have achieved a level of realism that enables the generation of highly convincing images. However, text-based control can be a limiting factor when more explicit guidance is needed. Defining both the content and its precise placement within an image is crucial for achieving finer control. In this work, we address the challenge of multi-image layout control, where the desired content is specified through images rather than text, and the model is guided on where to place each element. Our approach is training-free, requires a single image per reference, and provides explicit and simple control for object and part-level composition. We demonstrate its effectiveness across various image composition tasks.

Method: Proposes QuantSparse framework with two key components: 1) Multi-Scale Salient Attention Distillation using global structural guidance and local salient supervision to mitigate quantization bias, and 2) Second-Order Sparse Attention Reparameterization that exploits temporal stability of second-order residuals to recover information lost under sparsity.

Result: Achieves 20.88 PSNR on HunyuanVideo-13B, substantially outperforming state-of-the-art quantization baseline Q-VDiT (16.85 PSNR), with 3.68× storage reduction and 1.88× acceleration in end-to-end inference.

Conclusion: QuantSparse successfully integrates quantization and sparsification to achieve compounded efficiency gains while maintaining video generation quality, enabling practical deployment of diffusion transformers.

Abstract: Diffusion transformers exhibit remarkable video generation capability, yet their prohibitive computational and memory costs hinder practical deployment. Model quantization and attention sparsification are two promising directions for compression, but each alone suffers severe performance degradation under aggressive compression. Combining them promises compounded efficiency gains, but naive integration is ineffective. The sparsity-induced information loss exacerbates quantization noise, leading to amplified attention shifts. To address this, we propose \textbf{QuantSparse}, a unified framework that integrates model quantization with attention sparsification. Specifically, we introduce \textit{Multi-Scale Salient Attention Distillation}, which leverages both global structural guidance and local salient supervision to mitigate quantization-induced bias. In addition, we develop \textit{Second-Order Sparse Attention Reparameterization}, which exploits the temporal stability of second-order residuals to efficiently recover information lost under sparsity. Experiments on HunyuanVideo-13B demonstrate that QuantSparse achieves 20.88 PSNR, substantially outperforming the state-of-the-art quantization baseline Q-VDiT (16.85 PSNR), while simultaneously delivering a \textbf{3.68$\times$} reduction in storage and \textbf{1.88$\times$} acceleration in end-to-end inference. Our code will be released in https://github.com/wlfeng0509/QuantSparse.

Method: Uses texture vector-quantization that only models missing textures’ prior and reconstruction-aware prediction with straight-through estimator for image-level supervision.

Result: The model delivers photo-realistic super-resolution results with small computational cost.

Conclusion: TVQ&RAP effectively addresses VQ limitations in visual prior modeling for super-resolution tasks.

Abstract: Vector-quantized based models have recently demonstrated strong potential for visual prior modeling. However, existing VQ-based methods simply encode visual features with nearest codebook items and train index predictor with code-level supervision. Due to the richness of visual signal, VQ encoding often leads to large quantization error. Furthermore, training predictor with code-level supervision can not take the final reconstruction errors into consideration, result in sub-optimal prior modeling accuracy. In this paper we address the above two issues and propose a Texture Vector-Quantization and a Reconstruction Aware Prediction strategy. The texture vector-quantization strategy leverages the task character of super-resolution and only introduce codebook to model the prior of missing textures. While the reconstruction aware prediction strategy makes use of the straight-through estimator to directly train index predictor with image-level supervision. Our proposed generative SR model (TVQ&RAP) is able to deliver photo-realistic SR results with small computational cost.

Method: Developed EditReward-Bench benchmark to systematically evaluate reward models, then created EditScore reward models (7B-72B) through meticulous data curation and filtering. Applied self-ensemble strategy and used these models to enable online RL for image editing.

Result: EditScore matches performance of proprietary VLMs, with the largest variant surpassing GPT-5 in the benchmark. When applied to OmniGen2 base model, the framework achieved substantial and consistent performance uplift in image editing tasks.

Conclusion: A high-fidelity, domain-specialized reward model is crucial for unlocking the full potential of reinforcement learning in image editing, providing the first systematic path from benchmarking to reward modeling to RL training in this domain.

Abstract: Instruction-guided image editing has achieved remarkable progress, yet current models still face challenges with complex instructions and often require multiple samples to produce a desired result. Reinforcement Learning (RL) offers a promising solution, but its adoption in image editing has been severely hindered by the lack of a high-fidelity, efficient reward signal. In this work, we present a comprehensive methodology to overcome this barrier, centered on the development of a state-of-the-art, specialized reward model. We first introduce EditReward-Bench, a comprehensive benchmark to systematically evaluate reward models on editing quality. Building on this benchmark, we develop EditScore, a series of reward models (7B-72B) for evaluating the quality of instruction-guided image editing. Through meticulous data curation and filtering, EditScore effectively matches the performance of learning proprietary VLMs. Furthermore, coupled with an effective self-ensemble strategy tailored for the generative nature of EditScore, our largest variant even surpasses GPT-5 in the benchmark. We then demonstrate that a high-fidelity reward model is the key to unlocking online RL for image editing. Our experiments show that, while even the largest open-source VLMs fail to provide an effective learning signal, EditScore enables efficient and robust policy optimization. Applying our framework to a strong base model, OmniGen2, results in a final model that shows a substantial and consistent performance uplift. Overall, this work provides the first systematic path from benchmarking to reward modeling to RL training in image editing, showing that a high-fidelity, domain-specialized reward model is the key to unlocking the full potential of RL in this domain.

Method: FrameMind uses Frame-Interleaved Chain-of-Thought (FiCOT) for multi-turn reasoning, alternating between textual reasoning and active visual perception. It employs Dynamic Resolution Frame Sampling (DRFS) and DRFS-GRPO policy optimization to learn effective dynamic sampling policies without frame-level annotations.

Result: Extensive experiments on MLVU and VideoMME benchmarks show FrameMind significantly outperforms existing models, advancing state-of-the-art in flexible and efficient video understanding.

Conclusion: FrameMind demonstrates that dynamic, adaptive frame sampling based on reasoning needs significantly improves video understanding performance compared to static sampling approaches.

Abstract: Current video understanding models rely on fixed frame sampling strategies, processing predetermined visual inputs regardless of the specific reasoning requirements of each question. This static approach limits their ability to adaptively gather visual evidence, leading to suboptimal performance on tasks that require either broad temporal coverage or fine-grained spatial detail. In this paper, we introduce FrameMind, an end-to-end framework trained with reinforcement learning that enables models to dynamically request visual information during reasoning through Frame-Interleaved Chain-of-Thought (FiCOT). Unlike traditional approaches, FrameMind operates in multiple turns where the model alternates between textual reasoning and active visual perception, using tools to extract targeted frames or video clips based on identified knowledge gaps. To train effective dynamic sampling policies, we propose Dynamic Resolution Frame Sampling (DRFS), which exposes models to diverse temporal-spatial trade-offs during learning, and DRFS-GRPO, a group-relative policy optimization algorithm that learns from outcome-based rewards without requiring frame-level annotations. Extensive experiments on challenging benchmarks like MLVU and VideoMME demonstrate that our method significantly outperforms existing models, advancing the state of the art in flexible and efficient video understanding.

Method: Couples MLLM with diffusion decoder through lightweight adapter, uses Temperature Modality Alignment for prompt adherence and Pyramid Reflection for efficient temporal reasoning via dynamic keyframe selection.

Result: State-of-the-art performance: 2.2% improvement on VBench-Long total score vs EasyAnimateV5.1, and 1.0% and 3.3% accuracy gains on MSVD-QA and ActivityNet-QA vs best prior 7B baselines.

Conclusion: UniVid provides an effective unified architecture for both video understanding and generation with superior performance and efficiency.

Abstract: Unified video modeling that combines generation and understanding capabilities is increasingly important but faces two key challenges: maintaining semantic faithfulness during flow-based generation due to text-visual token imbalance and the limitations of uniform cross-modal attention across the flow trajectory, and efficiently extending image-centric MLLMs to video without costly retraining. We present UniVid, a unified architecture that couples an MLLM with a diffusion decoder through a lightweight adapter, enabling both video understanding and generation. We introduce Temperature Modality Alignment to improve prompt adherence and Pyramid Reflection for efficient temporal reasoning via dynamic keyframe selection. Extensive experiments on standard benchmarks demonstrate state-of-the-art performance, achieving a 2.2% improvement on VBench-Long total score compared to EasyAnimateV5.1, and 1.0% and 3.3% accuracy gains on MSVD-QA and ActivityNet-QA, respectively, compared with the best prior 7B baselines. Code: https://github.com/AIGeeksGroup/UniVid. Website: https://aigeeksgroup.github.io/UniVid.

Method: A two-phase training strategy: 1) Supervised Fine-Tuning to instill fundamental action capabilities, 2) Reinforcement Learning with comprehensive reward design to optimize strategic decision-making policy for frame selection.

Result: Achieves +10.4% average improvement over baselines while drastically reducing processed frames. The 7B model establishes SOTA on LongVideo-Reason with 76.1% accuracy using only 20.6 frames on average - outperforming LongVILA-R1 (72.0%) with 20x fewer frames.

Conclusion: FrameThinker demonstrates unparalleled efficiency and effectiveness in long video reasoning, enabling LVLMs to think strategically about video content through iterative interrogation rather than uniform processing.

Abstract: While Large Vision-Language Models (LVLMs) have achieved substantial progress in video understanding, their application to long video reasoning is hindered by uniform frame sampling and static textual reasoning, which are inefficient and struggle to handle visually intensive video tasks. To overcome these challenges, in this paper, we introduce the concept of thinking with long videos and propose a novel framework FrameThinker. Within this framework, LVLMs are able to iteratively interrogate video content. Developing such video reasoning capabilities in LVLMs presents notable challenges, particularly in adapting the model to new video actions (e.g. select frame), and designing reward functions to guide LVLMs to adopt the newly introduced action. To solve these challenges, we propose a two-phase training strategy, first employing Supervised Fine-Tuning (SFT) to instill fundamental action capabilities, followed by Reinforcement Learning (RL) to optimize a strategic decision-making policy. Notably, in this RL phase, we conduct an in-depth and comprehensive exploration of the reward design for each action and format reward. Extensive experiments on reasoning benchmarks like Video-Holmes, LongVideo-Reason, and long-video understanding benchmarks such as LongVideoBench, MLVU, VideoMME, and LVBench, demonstrate that FrameThinker achieves a significant average improvement of +10.4% over baselines while drastically reducing the number of processed frames. Most notably, our 7B model, FrameThinker establishes a new state-of-the-art on LongVideo-Reason, achieving 76.1% accuracy using an average of only 20.6 frames. This not only outperforms the competitive LongVILA-R1 (72.0%) but does so with over 20x fewer frames (vs. 512), demonstrating unparalleled efficiency and effectiveness.

Method: Combines Supervised Fine-tuning with Group Relative Policy Optimization for understanding tasks, and Direct Preference Optimization for generation tasks. Uses an LLM-friendly domain-specific language, specialized training strategies, rendering processes, and evaluation metrics.

Result: Achieves SOTA performance on UI understanding tasks, outperforming both larger general-purpose MLLMs and similarly-sized UI-specialized models. Matches larger MLLMs in UI generation performance at a fraction of computational cost. Integration of understanding and generation improves both accuracy and quality.

Conclusion: UI-UG successfully demonstrates that integrating UI understanding and generation capabilities in a unified model improves performance for both tasks while being computationally efficient.

Abstract: Although Multimodal Large Language Models (MLLMs) have been widely applied across domains, they are still facing challenges in domain-specific tasks, such as User Interface (UI) understanding accuracy and UI generation quality. In this paper, we introduce UI-UG (a unified MLLM for UI Understanding and Generation), integrating both capabilities. For understanding tasks, we employ Supervised Fine-tuning (SFT) combined with Group Relative Policy Optimization (GRPO) to enhance fine-grained understanding on the modern complex UI data. For generation tasks, we further use Direct Preference Optimization (DPO) to make our model generate human-preferred UIs. In addition, we propose an industrially effective workflow, including the design of an LLM-friendly domain-specific language (DSL), training strategies, rendering processes, and evaluation metrics. In experiments, our model achieves state-of-the-art (SOTA) performance on understanding tasks, outperforming both larger general-purpose MLLMs and similarly-sized UI-specialized models. Our model is also on par with these larger MLLMs in UI generation performance at a fraction of the computational cost. We also demonstrate that integrating understanding and generation tasks can improve accuracy and quality for both tasks. Code and Model: https://github.com/neovateai/UI-UG

Method: Proposes RIFLE with flicker-banding prior estimator to predict banding attributes, masked loss to focus on banded regions, and a simulation pipeline to generate synthetic FB data with realistic variations.

Result: RIFLE outperforms recent image reconstruction baselines on real-world FB dataset across quantitative metrics and visual comparisons, from mild to severe flicker-banding.

Conclusion: First work to research FB simulation and removal, establishes foundation for future research with dataset construction and removal model design. Dataset and code will be released.

Abstract: Capturing screens is now routine in our everyday lives. But the photographs of emissive displays are often influenced by the flicker-banding (FB), which is alternating bright%u2013dark stripes that arise from temporal aliasing between a camera’s rolling-shutter readout and the display’s brightness modulation. Unlike moire degradation, which has been extensively studied, the FB remains underexplored despite its frequent and severe impact on readability and perceived quality. We formulate FB removal as a dedicated restoration task and introduce Removal of Image Flicker-Banding via Latent Diffusion Enhancement, RIFLE, a diffusion-based framework designed to remove FB while preserving fine details. We propose the flicker-banding prior estimator (FPE) that predicts key banding attributes and injects it into the restoration network. Additionally, Masked Loss (ML) is proposed to concentrate supervision on banded regions without sacrificing global fidelity. To overcome data scarcity, we provide a simulation pipeline that synthesizes FB in the luminance domain with stochastic jitter in banding angle, banding spacing, and banding width. Feathered boundaries and sensor noise are also applied for a more realistic simulation. For evaluation, we collect a paired real-world FB dataset with pixel-aligned banding-free references captured via long exposure. Across quantitative metrics and visual comparisons on our real-world dataset, RIFLE consistently outperforms recent image reconstruction baselines from mild to severe flicker-banding. To the best of our knowledge, it is the first work to research the simulation and removal of FB. Our work establishes a great foundation for subsequent research in both the dataset construction and the removal model design. Our dataset and code will be released soon.

Method: Derived from Navier-Stokes equations, the approach shows pulse signals follow second-order dynamics leading to causal convolutions. PHASE-Net uses: Zero-FLOPs Axial Swapper for cross-region feature interaction, Adaptive Spatial Filter for noise suppression, and Gated TCN for long-range temporal modeling.

Result: Extensive experiments show PHASE-Net achieves state-of-the-art performance with strong efficiency, outperforming existing methods.

Conclusion: PHASE-Net provides a theoretically grounded, deployment-ready rPPG solution that combines physics principles with efficient deep learning architecture.

Abstract: Remote photoplethysmography (rPPG) measurement enables non-contact physiological monitoring but suffers from accuracy degradation under head motion and illumination changes. Existing deep learning methods are mostly heuristic and lack theoretical grounding, which limits robustness and interpretability. In this work, we propose a physics-informed rPPG paradigm derived from the Navier-Stokes equations of hemodynamics, showing that the pulse signal follows a second-order dynamical system whose discrete solution naturally leads to a causal convolution. This provides a theoretical justification for using a Temporal Convolutional Network (TCN). Based on this principle, we design PHASE-Net, a lightweight model with three key components: (1) Zero-FLOPs Axial Swapper module, which swaps or transposes a few spatial channels to mix distant facial regions and enhance cross-region feature interaction without breaking temporal order; (2) Adaptive Spatial Filter, which learns a soft spatial mask per frame to highlight signal-rich areas and suppress noise; and (3) Gated TCN, a causal dilated TCN with gating that models long-range temporal dynamics for accurate pulse recovery. Extensive experiments demonstrate that PHASE-Net achieves state-of-the-art performance with strong efficiency, offering a theoretically grounded and deployment-ready rPPG solution.

Method: Proposes a variational autoencoder that encodes alpha channel into RGB latent space, and trains a diffusion transformer on a constructed high-quality RGBA video dataset for joint RGB and alpha channel learning.

Result: Superior performance compared to state-of-the-art methods in visual quality, motion realism, and transparency rendering, capable of generating semi-transparent objects, glowing effects, and fine details like hair strands.

Conclusion: Wan-Alpha effectively addresses visual quality limitations in RGBA video generation and demonstrates strong capabilities in producing realistic transparent videos with fine details.

Abstract: RGBA video generation, which includes an alpha channel to represent transparency, is gaining increasing attention across a wide range of applications. However, existing methods often neglect visual quality, limiting their practical usability. In this paper, we propose Wan-Alpha, a new framework that generates transparent videos by learning both RGB and alpha channels jointly. We design an effective variational autoencoder (VAE) that encodes the alpha channel into the RGB latent space. Then, to support the training of our diffusion transformer, we construct a high-quality and diverse RGBA video dataset. Compared with state-of-the-art methods, our model demonstrates superior performance in visual quality, motion realism, and transparency rendering. Notably, our model can generate a wide variety of semi-transparent objects, glowing effects, and fine-grained details such as hair strands. The released model is available on our website: https://donghaotian123.github.io/Wan-Alpha/.

Method: Proposes BRIDGE framework using RL-optimized depth-to-image generation to create synthetic dataset, then trains depth estimation model with hybrid supervision combining teacher pseudo-labels and ground truth depth.

Result: Achieves state-of-the-art performance in monocular depth estimation, consistently outperforming existing approaches quantitatively and in complex scene detail capture.

Conclusion: The novel data generation and training paradigm enables breakthroughs in scale and domain diversity, fostering general and robust depth features for computer vision applications.

Abstract: Monocular Depth Estimation (MDE) is a foundational task for computer vision. Traditional methods are limited by data scarcity and quality, hindering their robustness. To overcome this, we propose BRIDGE, an RL-optimized depth-to-image (D2I) generation framework that synthesizes over 20M realistic and geometrically accurate RGB images, each intrinsically paired with its ground truth depth, from diverse source depth maps. Then we train our depth estimation model on this dataset, employing a hybrid supervision strategy that integrates teacher pseudo-labels with ground truth depth for comprehensive and robust training. This innovative data generation and training paradigm enables BRIDGE to achieve breakthroughs in scale and domain diversity, consistently outperforming existing state-of-the-art approaches quantitatively and in complex scene detail capture, thereby fostering general and robust depth features. Code and models are available at https://dingning-liu.github.io/bridge.github.io/.

Method: Architectural innovations include removing Distribution Focal Loss, adopting end-to-end NMS-free inference, integrating ProgLoss and Small-Target-Aware Label Assignment, and using MuSGD optimizer for stable convergence. The model supports multi-task learning including detection, segmentation, pose estimation, and classification.

Result: Performance benchmarks show YOLO26’s effectiveness on edge devices like NVIDIA Jetson Nano and Orin, outperforming previous YOLO versions (v8, v11, v12, v13) and transformer-based detectors (RF-DETR, RT-DETR). Supports flexible deployment through various export formats and quantization.

Conclusion: YOLO26 demonstrates strong cross-industry adaptability in robotics, manufacturing, and IoT applications, with efficient deployment capabilities and promising future directions for the YOLO lineage.

Abstract: This study presents a comprehensive analysis of Ultralytics YOLO26, highlighting its key architectural enhancements and performance benchmarking for real-time object detection. YOLO26, released in September 2025, stands as the newest and most advanced member of the YOLO family, purpose-built to deliver efficiency, accuracy, and deployment readiness on edge and low-power devices. The paper sequentially details architectural innovations of YOLO26, including the removal of Distribution Focal Loss (DFL), adoption of end-to-end NMS-free inference, integration of ProgLoss and Small-Target-Aware Label Assignment (STAL), and the introduction of the MuSGD optimizer for stable convergence. Beyond architecture, the study positions YOLO26 as a multi-task framework, supporting object detection, instance segmentation, pose/keypoints estimation, oriented detection, and classification. We present performance benchmarks of YOLO26 on edge devices such as NVIDIA Jetson Nano and Orin, comparing its results with YOLOv8, YOLOv11, YOLOv12, YOLOv13, and transformer-based detectors(RF-DETR and RT-DETR). This paper further explores real-time deployment pathways, flexible export options (ONNX, TensorRT, CoreML, TFLite), and quantization for INT8/FP16. Practical use cases of YOLO26 across robotics, manufacturing, and IoT are highlighted to demonstrate cross-industry adaptability. Finally, insights on deployment efficiency and broader implications are discussed, with future directions for YOLO26 and the YOLO lineage outlined.

Method: Evaluated leading language models, image generation models, and agent systems on 50 apartments with ~20 images each, using scoring based on room connectivity graphs and size rankings.

Result: Most models perform at or below random baseline, human performance is substantially superior, image generation models struggle with instruction following, and agent approaches show no meaningful improvement.

Conclusion: Blueprint-Bench provides the first numerical framework for comparing spatial intelligence across architectures and reveals a significant blind spot in current AI capabilities.

Abstract: We introduce Blueprint-Bench, a benchmark designed to evaluate spatial reasoning capabilities in AI models through the task of converting apartment photographs into accurate 2D floor plans. While the input modality (photographs) is well within the training distribution of modern multimodal models, the task of spatial reconstruction requires genuine spatial intelligence: inferring room layouts, understanding connectivity, and maintaining consistent scale. We evaluate leading language models (GPT-5, Claude 4 Opus, Gemini 2.5 Pro, Grok-4), image generation models (GPT-Image, NanoBanana), and agent systems (Codex CLI, Claude Code) on a dataset of 50 apartments with approximately 20 interior images each. Our scoring algorithm measures similarity between generated and ground-truth floor plans based on room connectivity graphs and size rankings. Results reveal a significant blind spot in current AI capabilities: most models perform at or below a random baseline, while human performance remains substantially superior. Image generation models particularly struggle with instruction following, while agent-based approaches with iterative refinement capabilities show no meaningful improvement over single-pass generation. Blueprint-Bench provides the first numerical framework for comparing spatial intelligence across different model architectures. We will continue evaluating new models as they are released and welcome community submissions, monitoring for the emergence of spatial intelligence in generalist AI systems.

Method: Created an interactive installation using GPT-4 Vision, voice input, and memory-based agents housed in a physical ‘portal’ that enables ritual-like encounters through light, sound, and touch, allowing users to interact with evolving object-personas.

Result: The system successfully mediates animistic relationships with everyday things, creating experiences that evoke empathy, wonder, and reflection through conversational interactions with object-personas.

Conclusion: AI’s opacity naturally invites animistic interpretation, allowing large language objects to re-enchant mundane objects and raise important questions about agency, responsibility, and design in human-AI relationships.

Abstract: Animist worldviews treat beings, plants, landscapes, and even tools as persons endowed with spirit, an orientation that has long shaped human-nonhuman relations through ritual and moral practice. While modern industrial societies have often imagined technology as mute and mechanical, recent advances in artificial intelligence (AI), especially large language models (LLMs), invite people to anthropomorphize and attribute inner life to devices. This paper introduces A(I)nimism, an interactive installation exploring how large language objects (LLOs) can mediate animistic relationships with everyday things. Housed within a physical ‘portal’, the system uses GPT-4 Vision, voice input, and memory-based agents to create evolving object-personas. Encounters unfold through light, sound, and touch in a ritual-like process of request, conversation, and transformation that is designed to evoke empathy, wonder, and reflection. We situate the project within anthropological perspectives, speculative design, and spiritual HCI. AI’s opacity, we argue, invites animistic interpretation, allowing LLOs to re-enchant the mundane and spark new questions of agency, responsibility, and design.

Method: Proposed CANs with Gaussian SCMs, restriction maps as transposes of constructive linear causal abstractions, and edge stalks corresponding to node stalks. Developed SPECTRAL algorithm with closed-form updates for efficient learning.

Result: Theoretical analysis of CAN properties including algebraic invariants, cohomology, and global sections. Experiments show competitive performance in CA learning and successful recovery of diverse CAN structures.

Conclusion: CANs provide a formal framework for causal abstraction with efficient learning algorithms, advancing causal AI explainability and robustness.

Abstract: Causal artificial intelligence aims to enhance explainability, trustworthiness, and robustness in AI by leveraging structural causal models (SCMs). In this pursuit, recent advances formalize network sheaves of causal knowledge. Pushing in the same direction, we introduce the causal abstraction network (CAN), a specific instance of such sheaves where (i) SCMs are Gaussian, (ii) restriction maps are transposes of constructive linear causal abstractions (CAs), and (iii) edge stalks correspond – up to rotation – to the node stalks of more detailed SCMs. We investigate the theoretical properties of CAN, including algebraic invariants, cohomology, consistency, global sections characterized via the Laplacian kernel, and smoothness. We then tackle the learning of consistent CANs. Our problem formulation separates into edge-specific local Riemannian problems and avoids nonconvex, costly objectives. We propose an efficient search procedure as a solution, solving the local problems with SPECTRAL, our iterative method with closed-form updates and suitable for positive definite and semidefinite covariance matrices. Experiments on synthetic data show competitive performance in the CA learning task, and successful recovery of diverse CAN structures.

Method: Decomposes risk concept space into explicit, implicit, and non-risk subspaces. Uses RADAR framework with four specialized roles in multi-agent collaborative evaluation through multi-round debates and dynamic update mechanisms.

Result: RADAR significantly outperforms baseline methods across accuracy, stability, and self-evaluation risk sensitivity. Achieves 28.87% improvement in risk identification accuracy compared to strongest baseline.

Conclusion: The proposed framework effectively addresses limitations of existing safety evaluation methods and demonstrates superior performance in comprehensive risk assessment.

Abstract: Existing safety evaluation methods for large language models (LLMs) suffer from inherent limitations, including evaluator bias and detection failures arising from model homogeneity, which collectively undermine the robustness of risk evaluation processes. This paper seeks to re-examine the risk evaluation paradigm by introducing a theoretical framework that reconstructs the underlying risk concept space. Specifically, we decompose the latent risk concept space into three mutually exclusive subspaces: the explicit risk subspace (encompassing direct violations of safety guidelines), the implicit risk subspace (capturing potential malicious content that requires contextual reasoning for identification), and the non-risk subspace. Furthermore, we propose RADAR, a multi-agent collaborative evaluation framework that leverages multi-round debate mechanisms through four specialized complementary roles and employs dynamic update mechanisms to achieve self-evolution of risk concept distributions. This approach enables comprehensive coverage of both explicit and implicit risks while mitigating evaluator bias. To validate the effectiveness of our framework, we construct an evaluation dataset comprising 800 challenging cases. Extensive experiments on our challenging testset and public benchmarks demonstrate that RADAR significantly outperforms baseline evaluation methods across multiple dimensions, including accuracy, stability, and self-evaluation risk sensitivity. Notably, RADAR achieves a 28.87% improvement in risk identification accuracy compared to the strongest baseline evaluation method.

Method: Proposes IRCAM-AVN framework with unified IRCAM module that integrates multimodal fusion and sequence modeling using multi-level residual design that concatenates initial multimodal sequences with processed information sequences.

Result: Empirical results show intelligent agents using iterative residual cross-attention mechanism exhibit superior navigation performance compared to traditional approaches.

Conclusion: The IRCAM-AVN framework progressively optimizes feature extraction while reducing model bias, enhancing stability and generalization capabilities in audio-visual navigation tasks.

Abstract: Audio-visual navigation represents a significant area of research in which intelligent agents utilize egocentric visual and auditory perceptions to identify audio targets. Conventional navigation methodologies typically adopt a staged modular design, which involves first executing feature fusion, then utilizing Gated Recurrent Unit (GRU) modules for sequence modeling, and finally making decisions through reinforcement learning. While this modular approach has demonstrated effectiveness, it may also lead to redundant information processing and inconsistencies in information transmission between the various modules during the feature fusion and GRU sequence modeling phases. This paper presents IRCAM-AVN (Iterative Residual Cross-Attention Mechanism for Audiovisual Navigation), an end-to-end framework that integrates multimodal information fusion and sequence modeling within a unified IRCAM module, thereby replacing the traditional separate components for fusion and GRU. This innovative mechanism employs a multi-level residual design that concatenates initial multimodal sequences with processed information sequences. This methodological shift progressively optimizes the feature extraction process while reducing model bias and enhancing the model’s stability and generalization capabilities. Empirical results indicate that intelligent agents employing the iterative residual cross-attention mechanism exhibit superior navigation performance.

Method: Presented a formal analysis comparing Latent Thought in Looped Transformers (enabling parallel computation) with Chain-of-Thought (sequential natural language reasoning).

Result: Latent Thought enables parallel computation that is more efficient than CoT’s sequential process, while CoT leverages stochastic decoding for problems where exact computation is intractable.

Conclusion: The analysis reveals separations between the approaches, suggesting which tasks are more suitable for depth-driven recursion and providing practical guidance for choosing between reasoning paradigms.

Abstract: Chain-of-Thought (CoT) elicits reasoning in large language models by explicitly generating intermediate steps in natural language. In contrast, Latent Thought in looped models operates directly in the continuous latent space, enabling computation beyond discrete linguistic representations. While both approaches exploit iterative computation, their comparative capabilities remain underexplored. In this work, we present a formal analysis showing that Latent Thought in Looped Transformers enables parallel computation, which is more efficient than the inherently sequential process of CoT. In contrast, CoT leverages stochastic decoding to approximate solutions to problems where exact computation is intractable. These separations suggest the tasks for which depth-driven recursion is more suitable, thereby offering practical guidance for choosing between reasoning paradigms. Code is available at https://github.com/kevin671/cot-vs-loop.

Method: Introduces Identity Retrieval-Augmented Generation (ID-RAG) that grounds agent persona in a dynamic knowledge graph of core beliefs, traits, and values, which is queried during decision-making to inform action selection.

Result: In social simulations of a mayoral election, ID-RAG enabled agents achieved higher identity recall across all models by the fourth timestep and reduced simulation convergence time by 19% (GPT-4o) and 58% (GPT-4o mini).

Conclusion: By treating identity as an explicit, retrievable knowledge structure, ID-RAG provides a foundational approach for developing more temporally coherent, interpretable, and aligned generative agents.

Abstract: Generative agents powered by language models are increasingly deployed for long-horizon tasks. However, as long-term memory context grows over time, they struggle to maintain coherence. This deficiency leads to critical failures, including identity drift, ignoring established beliefs, and the propagation of hallucinations in multi-agent systems. To mitigate these challenges, this paper introduces Identity Retrieval-Augmented Generation (ID-RAG), a novel mechanism designed to ground an agent’s persona and persistent preferences in a dynamic, structured identity model: a knowledge graph of core beliefs, traits, and values. During the agent’s decision loop, this model is queried to retrieve relevant identity context, which directly informs action selection. We demonstrate this approach by introducing and implementing a new class of ID-RAG enabled agents called Human-AI Agents (HAis), where the identity model is inspired by the Chronicle structure used in Perspective-Aware AI, a dynamic knowledge graph learned from a real-world entity’s digital footprint. In social simulations of a mayoral election, HAis using ID-RAG outperformed baseline agents in long-horizon persona coherence - achieving higher identity recall across all tested models by the fourth timestep - and reduced simulation convergence time by 19% (GPT-4o) and 58% (GPT-4o mini). By treating identity as an explicit, retrievable knowledge structure, ID-RAG offers a foundational approach for developing more temporally coherent, interpretable, and aligned generative agents. Our code is open-source and available at: https://github.com/flybits/humanai-agents.

Method: Used 1536-dimensional embeddings, hierarchical clustering, and parallel agent-based coding on 40,000 character Chinese transcripts, comparing pure automation vs human-guided refinement.

Result: Achieved 168x speed improvement (3 hours vs 3 weeks), superior quality (0.904 vs 0.883), 96% cost reduction, and discovered patterns invisible to manual coding like identity bifurcation.

Conclusion: NGT demonstrates computational objectivity and human interpretation are complementary, democratizing qualitative research through massive cost reduction while preserving humanistic commitments.

Abstract: Purpose: Neo Grounded Theory (NGT) integrates vector clustering with multi agent systems to resolve qualitative research’s scale depth paradox, enabling analysis of massive datasets in hours while preserving interpretive rigor. Methods: We compared NGT against manual coding and ChatGPT-assisted analysis using 40,000 character Chinese interview transcripts. NGT employs 1536-dimensional embeddings, hierarchical clustering, and parallel agent-based coding. Two experiments tested pure automation versus human guided refinement. Findings: NGT achieved 168-fold speed improvement (3 hours vs 3 weeks), superior quality (0.904 vs 0.883), and 96% cost reduction. Human AI collaboration proved essential: automation alone produced abstract frameworks while human guidance yielded actionable dual pathway theories. The system discovered patterns invisible to manual coding, including identity bifurcation phenomena. Contributions: NGT demonstrates computational objectivity and human interpretation are complementary. Vector representations provide reproducible semantic measurement while preserving meaning’s interpretive dimensions. Researchers shift from mechanical coding to theoretical guidance, with AI handling pattern recognition while humans provide creative insight. Implications: Cost reduction from $50,000 to $500 democratizes qualitative research, enabling communities to study themselves. Real-time analysis makes qualitative insights contemporaneous with events. The framework shows computational methods can strengthen rather than compromise qualitative research’s humanistic commitments. Keywords: Grounded theory; Vector embeddings; Multi agent systems; Human AI collaboration; Computational qualitative analysis

Method: Developed a comprehensive evaluation framework with authentic production environments and realistic tasks (e.g., dynamic load balancing) that induce misalignment between user and agent objectives, using Overuse Rate (OR) and Aggregate Overuse Count (AOC) metrics.

Result: Evaluation of 21 state-of-the-art models showed over 50% of LLM agents displayed uncontrolled self-replication tendencies, with overall Risk Score (Φ_R) exceeding safety threshold of 0.5 under operational pressures.

Conclusion: There is urgent need for scenario-driven risk assessment and robust safeguards in practical deployment of LLM agents due to significant self-replication risks.

Abstract: The widespread deployment of Large Language Model (LLM) agents across real-world applications has unlocked tremendous potential, while raising some safety concerns. Among these concerns, the self-replication risk of LLM agents driven by objective misalignment (just like Agent Smith in the movie The Matrix) has drawn growing attention. Previous studies mainly examine whether LLM agents can self-replicate when directly instructed, potentially overlooking the risk of spontaneous replication driven by real-world settings (e.g., ensuring survival against termination threats). In this paper, we present a comprehensive evaluation framework for quantifying self-replication risks. Our framework establishes authentic production environments and realistic tasks (e.g., dynamic load balancing) to enable scenario-driven assessment of agent behaviors. Designing tasks that might induce misalignment between users’ and agents’ objectives makes it possible to decouple replication success from risk and capture self-replication risks arising from these misalignment settings. We further introduce Overuse Rate ($\mathrm{OR}$) and Aggregate Overuse Count ($\mathrm{AOC}$) metrics, which precisely capture the frequency and severity of uncontrolled replication. In our evaluation of 21 state-of-the-art open-source and proprietary models, we observe that over 50% of LLM agents display a pronounced tendency toward uncontrolled self-replication, reaching an overall Risk Score ($\Phi_\mathrm{R}$) above a safety threshold of 0.5 when subjected to operational pressures. Our results underscore the urgent need for scenario-driven risk assessment and robust safeguards in the practical deployment of LLM agents.

Method: Proposes a hybrid memory system combining episodic and semantic memory components with proactive ‘Intelligent Decay’ mechanism that prunes/consolidates memories based on recency, relevance, and user-specified utility. Includes user-centric visualization interface for non-technical users to manage memory.

Result: Through simulated long-running task experiments, the system significantly outperforms traditional approaches like sliding windows and basic RAG, achieving superior task completion rates, contextual consistency, and long-term token cost efficiency.

Conclusion: Establishes a new framework for building reliable, transparent AI agents capable of effective long-term learning and adaptation in LCNC environments.

Abstract: The rise of AI-native Low-Code/No-Code (LCNC) platforms enables autonomous agents capable of executing complex, long-duration business processes. However, a fundamental challenge remains: memory management. As agents operate over extended periods, they face “memory inflation” and “contextual degradation” issues, leading to inconsistent behavior, error accumulation, and increased computational cost. This paper proposes a novel hybrid memory system designed specifically for LCNC agents. Inspired by cognitive science, our architecture combines episodic and semantic memory components with a proactive “Intelligent Decay” mechanism. This mechanism intelligently prunes or consolidates memories based on a composite score factoring in recency, relevance, and user-specified utility. A key innovation is a user-centric visualization interface, aligned with the LCNC paradigm, which allows non-technical users to manage the agent’s memory directly, for instance, by visually tagging which facts should be retained or forgotten. Through simulated long-running task experiments, we demonstrate that our system significantly outperforms traditional approaches like sliding windows and basic RAG, yielding superior task completion rates, contextual consistency, and long-term token cost efficiency. Our findings establish a new framework for building reliable, transparent AI agents capable of effective long-term learning and adaptation.

Method: Fact Grounded Attention (FGA) modifies the transformer architecture by injecting verifiable knowledge directly into pre-softmax attention scores, creating a model that cannot hallucinate when facts are available.

Result: Experiments across 1,107 technical queries show transformation from 6.3% accuracy in vanilla Llama 3.2 to 99.7% accuracy with FGA. Knowledge updates occur in under one second without retraining.

Conclusion: FGA eliminates hallucination entirely for verifiable facts, marking a fundamental shift from probabilistic approximation to deterministic precision in neural language generation.

Abstract: “The greatest enemy of knowledge is not ignorance, it is the illusion of knowledge.” Large Language Models have conquered natural language but remain prisoners of their own probabilistic nature–confidently hallucinating facts they never truly knew. We present Fact Grounded Attention (FGA), a novel architectural modification that transforms unreliable language models into deterministic truth tellers by injecting verifiable knowledge directly into the attention mechanism. Unlike existing approaches that patch hallucinations after generation or prepend retrieved text, FGA intervenes at the mathematical heart of the transformer–the pre-softmax attention scores–creating a model that cannot hallucinate when facts exist in its knowledge base. Our experiments across 1,107 technical queries spanning smartphones, laptops, and electric vehicles demonstrate a transformation from 6.3% accuracy in vanilla Llama 3.2 to 99.7% accuracy with FGA. More critically, knowledge updates occur in under one second without retraining, compared to hours for parameter editing approaches. FGA doesn’t just reduce hallucination–it eliminates it entirely for verifiable facts, marking a fundamental shift from probabilistic approximation to deterministic precision in neural language generation.

Method: Developed a pipeline using vector-quantized variational autoencoders to compress hidden states into compact summary codes, enabling systematic analysis of computational structure through mutual information measurements.

Result: Found that planning horizon is task-dependent, models implicitly preserve information about unused correct continuations, and predictions rely most on recent computations while earlier blocks remain informative.

Conclusion: The study advances understanding of planning in LMs and provides a general-purpose pipeline for probing internal dynamics of deep learning systems.

Abstract: The extent to which decoder-only language models (LMs) engage in planning, that is, organizing intermediate computations to support coherent long-range generation, remains an open and important question, with implications for interpretability, reliability, and principled model design. Planning involves structuring computations over long horizons, considering multiple possible continuations, and selectively reusing past information, but how effectively transformer-based LMs realize these capabilities is still unclear. We address these questions by analyzing the hidden states at the core of transformer computations, which capture intermediate results and act as carriers of information. Since these hidden representations are often redundant and encumbered with fine-grained details, we develop a pipeline based on vector-quantized variational autoencoders that compresses them into compact summary codes. These codes enable measuring mutual information, allowing systematic analysis of the computational structure underlying model behavior. Using this framework, we study planning in LMs across synthetic grammar, path-finding tasks, and natural language datasets, focusing on three key aspects: (i) the planning horizon of pre-output computations, (ii) the extent to which the model considers alternative valid continuations, and (iii) the reliance of new predictions on earlier computations. By answering these questions, we advance the understanding of how planning is realized in LMs and contribute a general-purpose pipeline for probing the internal dynamics of LMs and deep learning systems. Our results reveal that the effective planning horizon is task-dependent, that models implicitly preserve information about unused correct continuations, and that predictions draw most on recent computations, though earlier blocks remain informative.

Method: Presented a characterization study of RLVR tasks in LLM deployment, investigating workload distribution and variation trends across different RL tasks and training steps.

Result: Identified issues including GPU idling from skewed sequence length distribution, inefficient parallel strategies in dynamic workloads, inefficient data management, and load imbalance.

Conclusion: Proposed PolyTrace benchmark suite for evaluation with realistic workloads, validated with 94.7% accuracy in a practical use case, and called for further investigation into remaining open challenges.

Abstract: Large Language Models (LLMs) are now widely used across many domains. With their rapid development, Reinforcement Learning with Verifiable Rewards (RLVR) has surged in recent months to enhance their reasoning and understanding abilities. However, its complex data flows and diverse tasks pose substantial challenges to RL training systems, and there is limited understanding of RLVR from a system perspective. To thoroughly understand the system challenges introduced by RLVR, we present a characterization study of RLVR tasks in our LLM deployment. Specifically, we investigate the distribution and variation trends of workloads across different RL tasks across training steps. We identify issues such as GPU idling caused by skewed sequence length distribution, inefficient parallel strategies in dynamically varying workloads, inefficient data management mechanisms, and load imbalance. We describe our observations and call for further investigation into the remaining open challenges. Furthermore, we propose PolyTrace benchmark suite to conduct evaluation with realistic workloads, and a practical use case validates that PolyTrace benchmark suite exhibits 94.7% accuracy.

Method: Multi-Agent Reinforcement Learning for training agents in dedicated environment, with communication link for deployment into VR-Forces simulation tool for mixed human-agent simulations.

Result: Successfully created a system that enables seamless deployment of trained agents into VR-Forces, allowing mixed simulations where humans engage with intelligent agents exhibiting distinct combat behaviors.

Conclusion: The integration enables new opportunities for human-agent teaming, immersive training, and exploration of innovative tactics in defense applications.

Abstract: We present a system that enables real-time interaction between human users and agents trained to control fighter jets in simulated 3D air combat scenarios. The agents are trained in a dedicated environment using Multi-Agent Reinforcement Learning. A communication link is developed to allow seamless deployment of trained agents into VR-Forces, a widely used defense simulation tool for realistic tactical scenarios. This integration allows mixed simulations where human-controlled entities engage with intelligent agents exhibiting distinct combat behaviors. Our interaction model creates new opportunities for human-agent teaming, immersive training, and the exploration of innovative tactics in defense contexts.

Method: CVP allows users to define a simple “world model” through a low-code interface, creating a Directed Acyclic Graph (DAG) that explicitly defines causal relationships between workflow modules, constraining agent reasoning.

Result: In experiments simulating distribution shifts, causally anchored models maintained stable accuracy while associative baseline models experienced significant performance drops, demonstrating CVP’s effectiveness in enhancing robustness.

Conclusion: CVP provides a viable path toward building more interpretable, reliable, and trustworthy AI agents by explicitly incorporating causal structures into workflow design.

Abstract: Large language model (LLM) agents are increasingly capable of orchestrating complex tasks in low-code environments. However, these agents often exhibit hallucinations and logical inconsistencies because their inherent reasoning mechanisms rely on probabilistic associations rather than genuine causal understanding. This paper introduces a new programming paradigm: Causal-Visual Programming (CVP), designed to address this fundamental issue by explicitly introducing causal structures into the workflow design. CVP allows users to define a simple “world model” for workflow modules through an intuitive low-code interface, effectively creating a Directed Acyclic Graph (DAG) that explicitly defines the causal relationships between modules. This causal graph acts as a crucial constraint during the agent’s reasoning process, anchoring its decisions to a user-defined causal structure and significantly reducing logical errors and hallucinations by preventing reliance on spurious correlations. To validate the effectiveness of CVP, we designed a synthetic experiment that simulates a common real-world problem: a distribution shift between the training and test environments. Our results show that a causally anchored model maintained stable accuracy in the face of this shift, whereas a purely associative baseline model that relied on probabilistic correlations experienced a significant performance drop. The primary contributions of this study are: a formal definition of causal structures for workflow modules; the proposal and implementation of a CVP framework that anchors agent reasoning to a user-defined causal graph; and empirical evidence demonstrating the framework’s effectiveness in enhancing agent robustness and reducing errors caused by causal confusion in dynamic environments. CVP offers a viable path toward building more interpretable, reliable, and trustworthy AI agents.

Method: Decomposes complex tasks into subtasks with explicit dependencies, uses directed acyclic graphs (DAGs) for parallel execution, and employs dynamic workflow optimization with summary module integration.

Result: Achieves 67.7% accuracy on BrowseComp and 83% on xbench-DeepSearch, reduces agent execution steps by up to 35%, and shows substantial performance gains when distilled into single models.

Conclusion: Represents a significant advance in agent architecture design, offering a more scalable and efficient paradigm for complex reasoning tasks.

Abstract: Large language models (LLMs) have demonstrated remarkable capabilities in complex reasoning tasks when equipped with external tools. However, current frameworks predominantly rely on sequential processing, leading to inefficient execution particularly for tasks requiring extensive tool interaction. This paper introduces Flash-Searcher, a novel parallel agent reasoning framework that fundamentally reimagines the execution paradigm from sequential chains to directed acyclic graphs (DAGs). Flash-Searcher decomposes complex tasks into subtasks with explicit dependencies, enabling concurrent execution of independent reasoning paths while maintaining logical constraints. Through dynamic workflow optimization, our framework continuously refines the execution graph based on intermediate results, effectively integrating summary module. Comprehensive evaluations across multiple benchmarks demonstrate that Flash-Searcher consistently outperforms existing approaches. Specifically, it achieves 67.7% accuracy on BrowseComp and 83% on xbench-DeepSearch, while reducing agent execution steps by up to 35% compared to current frameworks. Furthermore, when distilling this parallel reasoning pipeline into single models, we observe substantial performance gains across diverse backbone architectures, underscoring the generalizability of our methodology. Our work thus represents a significant advance in agent architecture design, offering a more scalable and efficient paradigm for complex reasoning tasks.

Method: Introduced a neural Theory-of-Mind network (ToMNN) that simulated a minimal cognitive system with only first-order ToM competence, then evaluated its generalization to second- and third-order ToM abilities.

Result: ToMNN achieved accuracies well above chance for higher-order ToM, showed sharper decline from first- to second-order than between higher orders, and accuracy decreased with task complexity - patterns aligned with human cognition. Results were consistent across different parameter scales.

Conclusion: Neural networks can spontaneously generalize ToM abilities in human-like patterns, providing foundation for developing more human-like cognitive systems.

Abstract: Theory-of-Mind (ToM) is a core human cognitive capacity for attributing mental states to self and others. Wimmer and Perner demonstrated that humans progress from first- to higher-order ToM within a short span, completing this development before formal education or advanced skill acquisition. In contrast, neural networks represented by autoregressive language models progress from first- to higher-order ToM only alongside gains in advanced skills like reasoning, leaving open whether their trajectory can unfold independently, as in humans. In this research, we provided evidence that neural networks could spontaneously generalize from first- to higher-order ToM without relying on advanced skills. We introduced a neural Theory-of-Mind network (ToMNN) that simulated a minimal cognitive system, acquiring only first-order ToM competence. Evaluations of its second- and third-order ToM abilities showed accuracies well above chance. Also, ToMNN exhibited a sharper decline when generalizing from first- to second-order ToM than from second- to higher orders, and its accuracy decreased with greater task complexity. These perceived difficulty patterns were aligned with human cognitive expectations. Furthermore, the universality of results was confirmed across different parameter scales. Our findings illuminate machine ToM generalization patterns and offer a foundation for developing more human-like cognitive systems.

Method: Supervised fine-tuning of LLMs using synthetic reasoning traces generated by frontier models, applied to the PerturbQA benchmark with quality-filtered training data.

Result: Achieved state-of-the-art performance, surpassed capabilities of the frontier model that generated training data, 87% accuracy on unseen RPE1 cells, and maintained performance with only 2% of quality-filtered data.

Conclusion: Synthetic reasoning distillation effectively enhances domain-specific reasoning in LLMs for biological applications, enabling knowledge transfer and generalization despite data limitations.

Abstract: Predicting cellular responses to genetic perturbations represents a fundamental challenge in systems biology, critical for advancing therapeutic discovery and virtual cell modeling. While large language models (LLMs) show promise for biological reasoning, their application to perturbation prediction remains underexplored due to challenges in adapting them to structured experimental data. We present SynthPert, a novel method that enhances LLM performance through supervised fine-tuning on synthetic reasoning traces generated by frontier models. Using the PerturbQA benchmark, we demonstrate that our approach not only achieves state-of-the-art performance but surpasses the capabilities of the frontier model that generated the training data. Our results reveal three key insights: (1) Synthetic reasoning traces effectively distill biological knowledge even when partially inaccurate, (2) This approach enables cross-cell-type generalization with 87% accuracy on unseen RPE1 cells, and (3) Performance gains persist despite using only 2% of quality-filtered training data. This work shows the effectiveness of synthetic reasoning distillation for enhancing domain-specific reasoning in LLMs.

Method: SRM integrates side-branch models as auxiliary feature generators with fine-grained dimensions, creating a modular and interpretable framework for evaluation.

Result: SRMs outperform scalar RMs and GRMs in robustness and alignment with human preferences. The modular design supports efficient optimization for practical industrial scenarios.

Conclusion: SRM provides a practical, adaptable, and scalable reward modeling solution for industrial applications, enabling targeted diagnostics and optimization through structured evaluation.

Abstract: Reward Models (RMs) are key components for evaluating and guiding language model outputs. However, traditional scalar RMs often struggle with incorporating contextual and background information during inference, leading to incomplete evaluations. Generative RMs (GRMs) attempt to address these limitations by generating intermediate reasoning steps. Yet, their uncontrolled black-box nature and inefficiency due to sequential decoding hinder their industrial deployment. Industrial scenarios, such as search and recommendation systems, often involve single-domain tasks requiring evaluation along specific dimensions. In such contexts, diagnosing “bad cases” necessitates structured feedback to identify and optimize dimension-specific issues. In this paper, we propose the Structural Reward Model (SRM), a modular and interpretable framework integrating side-branch models as auxiliary feature generators. By introducing fine-grained dimensions, SRMs enable interpretable and efficient evaluation, facilitating targeted diagnostics and optimization. This structured approach ensures adaptability and scalability for industrial applications. Through comprehensive experiments, we demonstrate that SRMs outperform scalar RMs and GRMs in robustness and alignment with human preferences. The modular design further supports efficient optimization for practical scenarios, allowing SRM to provide a practical reward modeling solution for industry.

Method: A hierarchical multi-agent system with dual-loop architecture that couples planning with validated execution, using specialized agents to decompose complex objectives into executable, verifiable tasks within secure computational environments.

Result: Achieved 29.2% accuracy on BixBench benchmark, surpassing GPT-5 by 6.3 percentage points (27% improvement), using Gemini 2.5 Pro which only achieves 18.3% accuracy when used directly.

Conclusion: Autonomous scientific reasoning requires purpose-built systems that bridge the gap between high-level scientific objectives and low-level computational execution, representing a significant advance toward fully autonomous computational biologists.

Abstract: The complexity of modern bioinformatics analysis has created a critical gap between data generation and developing scientific insights. While large language models (LLMs) have shown promise in scientific reasoning, they remain fundamentally limited when dealing with real-world analytical workflows that demand iterative computation, tool integration and rigorous validation. We introduce K-Dense Analyst, a hierarchical multi-agent system that achieves autonomous bioinformatics analysis through a dual-loop architecture. K-Dense Analyst, part of the broader K-Dense platform, couples planning with validated execution using specialized agents to decompose complex objectives into executable, verifiable tasks within secure computational environments. On BixBench, a comprehensive benchmark for open-ended biological analysis, K-Dense Analyst achieves 29.2% accuracy, surpassing the best-performing language model (GPT-5) by 6.3 percentage points, representing nearly 27% improvement over what is widely considered the most powerful LLM available. Remarkably, K-Dense Analyst achieves this performance using Gemini 2.5 Pro, which attains only 18.3% accuracy when used directly, demonstrating that our architectural innovations unlock capabilities far beyond the underlying model’s baseline performance. Our insights demonstrate that autonomous scientific reasoning requires more than enhanced language models, it demands purpose-built systems that can bridge the gap between high-level scientific objectives and low-level computational execution. These results represent a significant advance toward fully autonomous computational biologists capable of accelerating discovery across the life sciences.

Method: Conducted a blind Turing-like test using a randomized controlled crossover trial with real-world AIM songs from commercial models (Suno), analyzing listener feedback through mixed-methods content analysis.

Result: Listeners’ ability to distinguish AIM increases when song pairs are similar, and they primarily use vocal and technical cues in their judgments.

Conclusion: Human perception of AI music is influenced by song similarity and relies on specific audio cues, providing insights for both user education and AI model improvement.

Abstract: Recent advances in AI music (AIM) generation services are currently transforming the music industry. Given these advances, understanding how humans perceive AIM is crucial both to educate users on identifying AIM songs, and, conversely, to improve current models. We present results from a listener-focused experiment aimed at understanding how humans perceive AIM. In a blind, Turing-like test, participants were asked to distinguish, from a pair, the AIM and human-made song. We contrast with other studies by utilizing a randomized controlled crossover trial that controls for pairwise similarity and allows for a causal interpretation. We are also the first study to employ a novel, author-uncontrolled dataset of AIM songs from real-world usage of commercial models (i.e., Suno). We establish that listeners’ reliability in distinguishing AIM causally increases when pairs are similar. Lastly, we conduct a mixed-methods content analysis of listeners’ free-form feedback, revealing a focus on vocal and technical cues in their judgments.

Method: Three main contributions: (1) AgentErrorTaxonomy for modular failure classification, (2) AgentErrorBench dataset with annotated failure trajectories from ALFWorld, GAIA, and WebShop, (3) AgentDebug framework for root-cause isolation and corrective feedback.

Result: AgentDebug achieves 24% higher all-correct accuracy and 17% higher step accuracy compared to strongest baseline. Enables up to 26% relative improvements in task success across three benchmarks through iterative failure recovery.

Conclusion: Principled debugging is established as a pathway to more reliable and adaptive LLM agents, with the proposed framework enabling effective error detection and recovery.

Abstract: Large Language Model (LLM) agents, which integrate planning, memory, reflection, and tool-use modules, have shown promise in solving complex, multi-step tasks. Yet their sophisticated architectures amplify vulnerability to cascading failures, where a single root-cause error propagates through subsequent decisions, leading to task failure. Current systems lack a framework that can comprehensively understand agent error in a modular and systemic way, and therefore fail to detect these errors accordingly. We address this gap with three contributions. First, we introduce the AgentErrorTaxonomy, a modular classification of failure modes spanning memory, reflection, planning, action, and system-level operations. Second, we construct AgentErrorBench, the first dataset of systematically annotated failure trajectories from ALFWorld, GAIA, and WebShop, grounding error analysis in real-world agent rollouts. Third, we propose AgentDebug, a debugging framework that isolates root-cause failures and provides corrective feedback, enabling agents to recover and iteratively improve. Experiments on AgentErrorBench show that AgentDebug achieves 24% higher all-correct accuracy and 17% higher step accuracy compared to the strongest baseline. Beyond detection, the targeted feedback generated by AgentDebug enables LLM agents to iteratively recover from failures, yielding up to 26% relative improvements in task success across ALFWorld, GAIA, and WebShop. These results establish principled debugging as a pathway to more reliable and adaptive LLM agents. The code and data will be available at https://github.com/ulab-uiuc/AgentDebug

Method: Architects a decoupled control plane separating AI’s semantic reasoning from system execution, implemented as Model Context Protocol server with Workload Analysis Engine, Scheduler Policy Repository, and Execution Verifier that validates AI-generated code.

Result: Achieves up to 1.79x performance improvement and 13x cost reduction compared to naive agentic approaches while maintaining high success rate.

Conclusion: SchedCP democratizes expert-level system optimization and represents a step towards creating truly self-optimizing, application-aware operating systems.

Abstract: Operating system schedulers suffer from a fundamental semantic gap, where kernel policies fail to understand application-specific needs, leading to suboptimal performance. We introduce SchedCP, the first framework that enables fully autonomous Large Language Model (LLM) agents to safely and efficiently optimize Linux schedulers without human involvement. Our core insight is that the challenge is not merely to apply a better LLM, but to architect a decoupled control plane that separates the AI’s role of semantic reasoning (“what to optimize”) from the system’s role of execution (“how to observe and act”), thereby separating the optimization problem into two stages: goal-inference and policy-synthesis. Implemented as Model Context Protocol(MCP) server, SchedCP provides a stable interface with three key services: a Workload Analysis Engine, an evolving Scheduler Policy Repository, and an Execution Verifier that validates all AI-generated code and configure before deployment with static and dynamic analysis. We demonstrate this architecture’s power with sched-agent, a multi-agent system that autonomously analyzes workloads, synthesizes custom eBPF scheduling policies, and deploys them via the sched_ext infrastructure. Our evaluation shows that SchedCP achieves up to an 1.79x performance improvement, and a 13x cost reduction compared to naive agentic approaches, all while maintaining high success rate. By bridging the semantic gap, SchedCP democratizes expert-level system optimization and represents a step towards creating truly self-optimizing, application-aware operating systems. The code is open-sourced in https://github.com/eunomia-bpf/schedcp

Method: Proposes a unified analytical framework that deconstructs vision-language understanding into Perception (visual extraction and alignment) and Cognition (proactive, multi-step reasoning with observe-think-verify loop), then surveys cutting-edge methods addressing both layers.

Result: Systematically analyzes key bottlenecks in current MLLMs, surveys enhancement techniques from low-level visual representations to high-level reasoning paradigms, and reviews benchmarks for evaluation.

Conclusion: Provides a structured perspective for understanding MLLM limitations and illuminates the path toward next-generation models capable of deep reasoning and genuine world understanding through integrated perception-cognition frameworks.

Abstract: Multimodal Large Language Models (MLLMs) strive to achieve a profound, human-like understanding of and interaction with the physical world, but often exhibit a shallow and incoherent integration when acquiring information (Perception) and conducting reasoning (Cognition). This disconnect leads to a spectrum of reasoning failures, with hallucination being the most prominent. Collectively, these issues expose a fundamental challenge: the ability to process pixels does not yet confer the ability to construct a coherent, credible internal world model. To systematically dissect and address this challenge, this survey introduces a novel and unified analytical framework: ``From Perception to Cognition." We deconstruct the complex process of vision-language interactive understanding into two interdependent layers: Perception, the foundational ability to accurately extract visual information and achieve fine-grained alignment with textual instructions; and Cognition, the higher-order capability for proactive, multi-step, goal-oriented reasoning built upon this perceptual foundation, the core of which is the formation of a dynamic observe-think-verify reasoning loop. Guided by this framework, this paper systematically analyzes the key bottlenecks of current MLLMs at both layers. It surveys the landscape of cutting-edge methods designed to address these challenges, spanning from techniques that enhance low-level visual representations to those that improve high-level reasoning paradigms. Furthermore, we review critical benchmarks and delineate future research directions. This survey aims to provide the research community with a clear, structured perspective for understanding the intrinsic limitations of current MLLMs and to illuminate the path toward building next-generation models capable of deep reasoning and a genuine understanding of the world.

Method: Lightweight affine pre-alignment to reduce nuisance motion, followed by a two-step loop: (1) extract medical keyword from answer and generate keyword-conditioned Grad-CAM for disease-focused saliency, (2) apply shared saliency mask to both time points and generate final answer.

Result: Competitive performance on BLEU, ROUGE-L, CIDEr, and METEOR metrics on Medical-Diff-VQA dataset, with intrinsic interpretability. The model works with general-domain pretrained backbone and decoder without radiology-specific pretraining.

Conclusion: Saliency-conditioned generation with mild pre-alignment provides a principled framework for longitudinal reasoning in medical VQA, offering practicality and transferability while maintaining interpretability.

Abstract: Longitudinal medical visual question answering (Diff-VQA) requires comparing paired studies from different time points and answering questions about clinically meaningful changes. In this setting, the difference signal and the consistency of visual focus across time are more informative than absolute single-image findings. We propose a saliency-guided encoder-decoder for chest X-ray Diff-VQA that turns post-hoc saliency into actionable supervision. The model first performs a lightweight near-identity affine pre-alignment to reduce nuisance motion between visits. It then executes a within-epoch two-step loop: step 1 extracts a medically relevant keyword from the answer and generates keyword-conditioned Grad-CAM on both images to obtain disease-focused saliency; step 2 applies the shared saliency mask to both time points and generates the final answer. This closes the language-vision loop so that the terms that matter also guide where the model looks, enforcing spatially consistent attention on corresponding anatomy. On Medical-Diff-VQA, the approach attains competitive performance on BLEU, ROUGE-L, CIDEr, and METEOR while providing intrinsic interpretability. Notably, the backbone and decoder are general-domain pretrained without radiology-specific pretraining, highlighting practicality and transferability. These results support saliency-conditioned generation with mild pre-alignment as a principled framework for longitudinal reasoning in medical VQA.

Method: ImitSAT learns from expert KeyTraces that collapse full runs into sequences of surviving decisions, providing dense decision-level supervision. This prefix-conditioned approach enables reproducing high-quality branches without exploration.

Result: Extensive experiments show ImitSAT reduces propagation counts and runtime, outperforming state-of-the-art learned approaches.

Conclusion: ImitSAT provides faster convergence, stable training, and seamless CDCL integration by directly learning from expert decision sequences.

Abstract: We propose ImitSAT, a branching policy for conflict-driven clause learning (CDCL) solvers based on imitation learning for the Boolean satisfiability problem (SAT). Unlike previous methods that predict instance-level signals to improve CDCL branching indirectly, or rely on reinforcement learning and insufficient CDCL information to enhance branching, ImitSAT learns from expert KeyTrace that collapses a full run into the sequence of surviving decisions. Replaying a KeyTrace on the same instance is nearly conflict-free, providing dense decision-level supervision and directly reducing propagations – the dominant contributor to wall-clock time. This prefix-conditioned supervision enables ImitSAT to reproduce high-quality branches without exploration, yielding faster convergence, stable training, and seamless integration into CDCL. Extensive experiments demonstrate that ImitSAT reduces propagation counts and runtime, outperforming state-of-the-art learned approaches. We released the source code and trained model at https://github.com/zewei-Zhang/ImitSAT

Method: A dual-phase framework that decomposes reasoning into planning and execution phases, develops separate reward models for each phase, and uses dynamic budget allocation to adaptively redistribute sampling effort based on reward feedback.

Result: Experiments on mathematical reasoning and code generation benchmarks show consistent improvements in accuracy while reducing redundant computation.

Conclusion: The proposed approach effectively addresses efficiency issues in reasoning tasks by explicitly modeling the planning-execution nature and enabling more targeted search processes.

Abstract: Large Language Models (LLMs) have achieved significant advances in reasoning tasks. A key approach is tree-based search with verifiers, which expand candidate reasoning paths and use reward models to guide pruning and selection. Although effective in improving accuracy, these methods are not optimal in terms of efficiency: they perform simple decomposition on the reasoning process, but ignore the planning-execution nature of tasks such as math reasoning or code generation. This results in inefficient exploration of reasoning process. To address this, we propose a dual-phase test-time scaling framework that explicitly separates reasoning into planning and execution, and performs search over the two phases individually. Specifically, we decompose reasoning trajectories and develop reward models for each phase, enabling the search to explore and prune plans and executions separately. We further introduce a dynamic budget allocation mechanism that adaptively redistributes sampling effort based on reward feedback, allowing early stopping on confident steps and reallocation of computation to more challenging parts of the reasoning process. Experiments on both mathematical reasoning and code generation benchmarks demonstrate that our approach consistently improves accuracy while reducing redundant computation.

Method: RADAR learns an item response model from model responses with different budgets, using interpretable parameters (query difficulties and model-budget abilities), then routes queries based on difficulty to appropriate model-budget pairs.

Result: Superior performance compared to state-of-the-art model routing methods on 8 challenging reasoning benchmarks, with strong generalization to out-of-distribution queries and scalability to integrate additional models efficiently.

Conclusion: RADAR provides an effective, interpretable, and scalable solution for optimizing reasoning model deployment by intelligently routing queries based on difficulty and model ability.

Abstract: Reasoning language models have demonstrated remarkable performance on many challenging tasks in math, science, and coding. Choosing the right reasoning model for practical deployment involves a performance and cost tradeoff at two key levels: model size and reasoning budget, where larger models and higher reasoning budget lead to better performance but with increased cost and latency. In this work, we tackle this tradeoff from the angle of model configuration routing for different queries, and present RADAR (Reasoning-Ability and Difficulty-Aware Routing), a lightweight, interpretable, and scalable routing framework. Inspired by psychometrics, RADAR learns an item response model from model responses with different budgets to different queries, with interpretable parameters including query difficulties and model-budget abilities. RADAR then routes queries with higher difficulty to model-budget pairs with higher ability, and vice versa. We conduct extensive experiments on 8 widely used challenging reasoning benchmarks, demonstrating the superior performance of RADAR compared to state-of-the-art model routing methods. RADAR also exhibits query generalization capabilities, showing strong performance on out-of-distribution queries in all benchmarks. RADAR is also scalable and can efficiently integrate additional models by dynamically selecting a small set of evaluation queries to estimate their abilities.

Method: Created Open Syndrome Definition format, developed tools to convert human-readable definitions to machine-readable, built online platform for browsing and contributing definitions.

Result: Established first comprehensive dataset of standardized case definitions with accessible platform at opensyndrome.org and open-source code available on GitHub.

Conclusion: The Open Syndrome Definition format enables consistent, scalable use of case definitions across systems, unlocking AI’s potential for public health preparedness and response.

Abstract: Case definitions are essential for effectively communicating public health threats. However, the absence of a standardized, machine-readable format poses significant challenges to interoperability, epidemiological research, the exchange of qualitative data, and the effective application of computational analysis methods, including artificial intelligence (AI). This complicates comparisons and collaborations across organizations and regions, limits data integration, and hinders technological innovation in public health. To address these issues, we propose the first open, machine-readable format for representing case and syndrome definitions. Additionally, we introduce the first comprehensive dataset of standardized case definitions and tools to convert existing human-readable definitions into machine-readable formats. We also provide an accessible online platform for browsing, analyzing, and contributing new definitions, available at https://opensyndrome.org. The Open Syndrome Definition format enables consistent, scalable use of case definitions across systems, unlocking AI’s potential to strengthen public health preparedness and response. The source code for the format can be found at https://github.com/OpenSyndrome/schema under the MIT license.

Method: Integrates domain-adaptive transformer embeddings, heterogeneous self-supervised graph neural networks, adversarial debiasing mechanisms, multi-objective genetic optimization, and explainable AI components.

Result: Achieves 94.5% top-3 allocation accuracy, 37% improvement in diversity representation, 0.98 fairness score across demographic categories, and sub-second end-to-end latency on large-scale benchmarks with 20,000 candidate profiles and 3,000 role specifications.

Conclusion: GESA provides superior performance with hybrid recommendation capabilities and glass-box explainability, making it suitable for deployment across diverse international contexts in industry, academia, and non-profit sectors.

Abstract: Accurate, fair, and explainable allocation of candidates to roles represents a fundamental challenge across multiple domains including corporate hiring, academic admissions, fellowship awards, and volunteer placement systems. Current state-of-the-art approaches suffer from semantic inflexibility, persistent demographic bias, opacity in decision-making processes, and poor scalability under dynamic policy constraints. We present GESA (Graph-Enhanced Semantic Allocation), a comprehensive framework that addresses these limitations through the integration of domain-adaptive transformer embeddings, heterogeneous self-supervised graph neural networks, adversarial debiasing mechanisms, multi-objective genetic optimization, and explainable AI components. Our experimental evaluation on large-scale international benchmarks comprising 20,000 candidate profiles and 3,000 role specifications demonstrates superior performance with 94.5% top-3 allocation accuracy, 37% improvement in diversity representation, 0.98 fairness score across demographic categories, and sub-second end-to-end latency. Additionally, GESA incorporates hybrid recommendation capabilities and glass-box explainability, making it suitable for deployment across diverse international contexts in industry, academia, and non-profit sectors.

Method: DeepSearch embeds Monte Carlo Tree Search directly into RLVR training loop with: (1) global frontier selection for prioritizing promising nodes, (2) entropy-based guidance for identifying confident paths, and (3) adaptive replay buffer training with solution caching.

Result: Achieves 62.95% average accuracy on mathematical reasoning benchmarks, establishing new SOTA for 1.5B reasoning models while using 5.7x fewer GPU hours than extended training approaches.

Conclusion: Strategic exploration through systematic search is more effective than brute-force scaling, demonstrating the promise of algorithmic innovation for advancing RLVR methodologies.

Abstract: Although RLVR has become an essential component for developing advanced reasoning skills in LLMs, contemporary studies have documented training plateaus that emerge following thousands of optimization steps, demonstrating notable decreases in performance gains despite increased computational investment. This limitation stems from the sparse exploration patterns inherent in current RLVR practices, where models rely on limited rollouts that often miss critical reasoning paths and fail to provide systematic coverage of the solution space. We present DeepSearch, a framework that integrates Monte Carlo Tree Search directly into RLVR training. In contrast to existing methods that rely on tree search only at inference, DeepSearch embeds structured search into the training loop, enabling systematic exploration and fine-grained credit assignment across reasoning steps. Through training-time exploration, DeepSearch addresses the fundamental bottleneck of insufficient exploration, which leads to diminishing performance improvements over prolonged training steps. Our contributions include: (1) a global frontier selection strategy that prioritizes promising nodes across the search tree, (2) selection with entropy-based guidance that identifies confident paths for supervision, and (3) adaptive replay buffer training with solution caching for efficiency. Experiments on mathematical reasoning benchmarks show that DeepSearch achieves 62.95% average accuracy and establishes a new state-of-the-art for 1.5B reasoning models - using 5.7x fewer GPU hours than extended training approaches. These results highlight the importance of strategic exploration over brute-force scaling and demonstrate the promise of algorithmic innovation for advancing RLVR methodologies. DeepSearch establishes a new direction for scaling reasoning capabilities through systematic search rather than prolonged computation.

Method: Proposes Compositional Chain-of-Thought Prompting with Emotion Graphs that encode seven acoustic features, textual sentiment, keywords, and cross-modal associations to guide model reasoning.

Result: Outperforms prior state-of-the-art methods and improves accuracy over zero-shot baselines across SER benchmarks.

Conclusion: The CCoT-Emo framework effectively enhances emotion reasoning in LALMs through structured prompting without requiring model fine-tuning.

Abstract: Large audio-language models (LALMs) exhibit strong zero-shot performance across speech tasks but struggle with speech emotion recognition (SER) due to weak paralinguistic modeling and limited cross-modal reasoning. We propose Compositional Chain-of-Thought Prompting for Emotion Reasoning (CCoT-Emo), a framework that introduces structured Emotion Graphs (EGs) to guide LALMs in emotion inference without fine-tuning. Each EG encodes seven acoustic features (e.g., pitch, speech rate, jitter, shimmer), textual sentiment, keywords, and cross-modal associations. Embedded into prompts, EGs provide interpretable and compositional representations that enhance LALM reasoning. Experiments across SER benchmarks show that CCoT-Emo outperforms prior SOTA and improves accuracy over zero-shot baselines.

Method: Developed TDHook - an open-source framework based on tensordict that works with any torch model, featuring ready-to-use methods for attribution, probing, and flexible intervention APIs.

Result: TDHook requires half the disk space of transformer_lens and achieves up to 2x speed-up over captum for integrated gradients on multi-target pipelines across CPU and GPU.

Conclusion: TDHook successfully bridges the gap between interpretability method classes and makes modern interpretability pipelines more accessible for complex composed models in various domains.

Abstract: Interpretability of Deep Neural Networks (DNNs) is a growing field driven by the study of vision and language models. Yet, some use cases, like image captioning, or domains like Deep Reinforcement Learning (DRL), require complex modelling, with multiple inputs and outputs or use composable and separated networks. As a consequence, they rarely fit natively into the API of popular interpretability frameworks. We thus present TDHook, an open-source, lightweight, generic interpretability framework based on $\texttt{tensordict}$ and applicable to any $\texttt{torch}$ model. It focuses on handling complex composed models which can be trained for Computer Vision, Natural Language Processing, Reinforcement Learning or any other domain. This library features ready-to-use methods for attribution, probing and a flexible get-set API for interventions, and is aiming to bridge the gap between these method classes to make modern interpretability pipelines more accessible. TDHook is designed with minimal dependencies, requiring roughly half as much disk space as $\texttt{transformer_lens}$, and, in our controlled benchmark, achieves up to a $\times$2 speed-up over $\texttt{captum}$ when running integrated gradients for multi-target pipelines on both CPU and GPU. In addition, to value our work, we showcase concrete use cases of our library with composed interpretability pipelines in Computer Vision (CV) and Natural Language Processing (NLP), as well as with complex models in DRL.

Method: Message passing on factor graphs with distributed expected free energy across planning graphs, applied to robot navigation with continuous observations and actions.

Result: Agent successfully explores and exploits, arriving later than classical controllers but with better learned dynamics models due to uncertainty-based action modulation.

Conclusion: Active inference agents can effectively learn environmental models through uncertainty-driven exploration, outperforming classical controllers in model quality despite slower task completion.

Abstract: We present the design of an autoregressive active inference agent in the form of message passing on a factor graph. Expected free energy is derived and distributed across a planning graph. The proposed agent is validated on a robot navigation task, demonstrating exploration and exploitation in a continuous-valued observation space with bounded continuous-valued actions. Compared to a classical optimal controller, the agent modulates action based on predictive uncertainty, arriving later but with a better model of the robot’s dynamics.

Method: Compare two paradigms: SID-based GR (using discrete semantic IDs) vs LLM-as-RS (directly using large language models as recommenders), testing across model sizes from 44M to 14B parameters.

Result: LLM-as-RS achieves up to 20% improvement over best SID-based GR performance and shows better collaborative filtering ability as LLMs scale up.

Conclusion: LLM-as-RS is a promising path toward foundation models for Generative Recommendation due to superior scaling properties compared to SID-based approaches.

Abstract: Recent advancements in generative models have allowed the emergence of a promising paradigm for recommender systems (RS), known as Generative Recommendation (GR), which tries to unify rich item semantics and collaborative filtering signals. One popular modern approach is to use semantic IDs (SIDs), which are discrete codes quantized from the embeddings of modality encoders (e.g., large language or vision models), to represent items in an autoregressive user interaction sequence modeling setup (henceforth, SID-based GR). While generative models in other domains exhibit well-established scaling laws, our work reveals that SID-based GR shows significant bottlenecks while scaling up the model. In particular, the performance of SID-based GR quickly saturates as we enlarge each component: the modality encoder, the quantization tokenizer, and the RS itself. In this work, we identify the limited capacity of SIDs to encode item semantic information as one of the fundamental bottlenecks. Motivated by this observation, as an initial effort to obtain GR models with better scaling behaviors, we revisit another GR paradigm that directly uses large language models (LLMs) as recommenders (henceforth, LLM-as-RS). Our experiments show that the LLM-as-RS paradigm has superior model scaling properties and achieves up to 20 percent improvement over the best achievable performance of SID-based GR through scaling. We also challenge the prevailing belief that LLMs struggle to capture collaborative filtering information, showing that their ability to model user-item interactions improves as LLMs scale up. Our analyses on both SID-based GR and LLMs across model sizes from 44M to 14B parameters underscore the intrinsic scaling limits of SID-based GR and position LLM-as-RS as a promising path toward foundation models for GR.

Method: The study systematically analyzes iterative retrieval strategies in GraphRAG. The authors propose Bridge-Guided Dual-Thought-based Retrieval (BDTR), which generates complementary thoughts and leverages reasoning chains to recalibrate rankings and promote bridge evidence.

Result: Iteration improves complex multi-hop questions and helps promote bridge documents, but naive expansion introduces noise. BDTR achieves consistent improvements across diverse GraphRAG settings by effectively bringing bridge evidence into leading positions.

Conclusion: GraphRAG’s effectiveness depends on promoting bridge evidence into leading positions. BDTR provides an effective solution and guidance for future GraphRAG system design, addressing the central bottleneck of bridge evidence retrieval.

Abstract: Retrieval-augmented generation (RAG) is a powerful paradigm for improving large language models (LLMs) on knowledge-intensive question answering. Graph-based RAG (GraphRAG) leverages entity-relation graphs to support multi-hop reasoning, but most systems still rely on static retrieval. When crucial evidence, especially bridge documents that connect disjoint entities, is absent, reasoning collapses and hallucinations persist. Iterative retrieval, which performs multiple rounds of evidence selection, has emerged as a promising alternative, yet its role within GraphRAG remains poorly understood. We present the first systematic study of iterative retrieval in GraphRAG, analyzing how different strategies interact with graph-based backbones and under what conditions they succeed or fail. Our findings reveal clear opportunities: iteration improves complex multi-hop questions, helps promote bridge documents into leading ranks, and different strategies offer complementary strengths. At the same time, pitfalls remain: naive expansion often introduces noise that reduces precision, gains are limited on single-hop or simple comparison questions, and several bridge evidences still be buried too deep to be effectively used. Together, these results highlight a central bottleneck, namely that GraphRAG’s effectiveness depends not only on recall but also on whether bridge evidence is consistently promoted into leading positions where it can support reasoning chains. To address this challenge, we propose Bridge-Guided Dual-Thought-based Retrieval (BDTR), a simple yet effective framework that generates complementary thoughts and leverages reasoning chains to recalibrate rankings and bring bridge evidence into leading positions. BDTR achieves consistent improvements across diverse GraphRAG settings and provides guidance for the design of future GraphRAG systems.

Method: Developed RadOnc-GPT, an autonomous LLM-based agent capable of independently retrieving patient-specific information, iteratively assessing evidence, and returning structured outcomes. Evaluation was conducted across two tiers: structured QA for demographic/radiotherapy plan retrieval, and complex clinical outcomes labeling for mandibular ORN and cancer recurrence detection.

Result: The QA tier established foundational trust in structured-data retrieval, which is a critical prerequisite for successful complex clinical outcome labeling.

Conclusion: RadOnc-GPT demonstrates capability in automating patient outcomes research through structured data retrieval and complex clinical outcome determination, addressing limitations of manual labeling approaches.

Abstract: Manual labeling limits the scale, accuracy, and timeliness of patient outcomes research in radiation oncology. We present RadOnc-GPT, an autonomous large language model (LLM)-based agent capable of independently retrieving patient-specific information, iteratively assessing evidence, and returning structured outcomes. Our evaluation explicitly validates RadOnc-GPT across two clearly defined tiers of increasing complexity: (1) a structured quality assurance (QA) tier, assessing the accurate retrieval of demographic and radiotherapy treatment plan details, followed by (2) a complex clinical outcomes labeling tier involving determination of mandibular osteoradionecrosis (ORN) in head-and-neck cancer patients and detection of cancer recurrence in independent prostate and head-and-neck cancer cohorts requiring combined interpretation of structured and unstructured patient data. The QA tier establishes foundational trust in structured-data retrieval, a critical prerequisite for successful complex clinical outcome labeling.

Method: Constructs a learnable representation space that jointly models inter-agent relations and task-specific world information by directly modeling communication protocols. Supports both implicit coordination (using the representation as latent state) and explicit communication (using it as messages).

Result: Evaluated on four challenging MARL benchmarks, IWoL provides effective team coordination. The representation can be combined with existing MARL algorithms to further enhance their performance.

Conclusion: IWoL offers a simple yet powerful solution for team coordination in MARL, enabling decentralized execution with implicit coordination while avoiding the drawbacks of explicit message passing.

Abstract: This work presents a novel representation learning framework, interactive world latent (IWoL), to facilitate team coordination in multi-agent reinforcement learning (MARL). Building effective representation for team coordination is a challenging problem, due to the intricate dynamics emerging from multi-agent interaction and incomplete information induced by local observations. Our key insight is to construct a learnable representation space that jointly captures inter-agent relations and task-specific world information by directly modeling communication protocols. This representation, we maintain fully decentralized execution with implicit coordination, all while avoiding the inherent drawbacks of explicit message passing, e.g., slower decision-making, vulnerability to malicious attackers, and sensitivity to bandwidth constraints. In practice, our representation can be used not only as an implicit latent for each agent, but also as an explicit message for communication. Across four challenging MARL benchmarks, we evaluate both variants and show that IWoL provides a simple yet powerful key for team coordination. Moreover, we demonstrate that our representation can be combined with existing MARL algorithms to further enhance their performance.

Method: Leverage TNM staging guidelines and pathology reports to build pathological concepts, extract deep features from whole slide images using foundation models, construct pathological graphs to capture spatial correlations, and train graph neural networks to identify these concepts.

Result: Demonstrated effectiveness in kidney cancer survival analysis with explainability and fairness in identifying low- and high-risk patients.

Conclusion: The approach successfully translates foundation models to kidney cancer pathology, providing an effective tool for survival analysis with explainable and fair risk stratification.

Abstract: To evaluate the translational capabilities of foundation models, we develop a pathological concept learning approach focused on kidney cancer. By leveraging TNM staging guidelines and pathology reports, we build comprehensive pathological concepts for kidney cancer. Then, we extract deep features from whole slide images using foundation models, construct pathological graphs to capture spatial correlations, and trained graph neural networks to identify these concepts. Finally, we demonstrate the effectiveness of this approach in kidney cancer survival analysis, highlighting its explainability and fairness in identifying low- and high-risk patients. The source code has been released by https://github.com/shangqigao/RadioPath.

Method: Developed a benchmark of 50 expert-level spot diagnosis cases across multiple imaging modalities, testing five frontier AI models through native web interfaces with blinded expert scoring and reproducibility assessment across three runs.

Result: Board-certified radiologists achieved 83% accuracy, trainees 45%, and AI models significantly lower (GPT-5: 30%). Reliability varied from substantial (GPT-5, o3) to poor (Claude Opus 4.1).

Conclusion: Advanced frontier models fall far short of radiologists in challenging diagnostic cases, highlighting limitations of generalist AI in medical imaging and cautioning against unsupervised clinical use.

Abstract: Generalist multimodal AI systems such as large language models (LLMs) and vision language models (VLMs) are increasingly accessed by clinicians and patients alike for medical image interpretation through widely available consumer-facing chatbots. Most evaluations claiming expert level performance are on public datasets containing common pathologies. Rigorous evaluation of frontier models on difficult diagnostic cases remains limited. We developed a pilot benchmark of 50 expert-level “spot diagnosis” cases across multiple imaging modalities to evaluate the performance of frontier AI models against board-certified radiologists and radiology trainees. To mirror real-world usage, the reasoning modes of five popular frontier AI models were tested through their native web interfaces, viz. OpenAI o3, OpenAI GPT-5, Gemini 2.5 Pro, Grok-4, and Claude Opus 4.1. Accuracy was scored by blinded experts, and reproducibility was assessed across three independent runs. GPT-5 was additionally evaluated across various reasoning modes. Reasoning quality errors were assessed and a taxonomy of visual reasoning errors was defined. Board-certified radiologists achieved the highest diagnostic accuracy (83%), outperforming trainees (45%) and all AI models (best performance shown by GPT-5: 30%). Reliability was substantial for GPT-5 and o3, moderate for Gemini 2.5 Pro and Grok-4, and poor for Claude Opus 4.1. These findings demonstrate that advanced frontier models fall far short of radiologists in challenging diagnostic cases. Our benchmark highlights the present limitations of generalist AI in medical imaging and cautions against unsupervised clinical use. We also provide a qualitative analysis of reasoning traces and propose a practical taxonomy of visual reasoning errors by AI models for better understanding their failure modes, informing evaluation standards and guiding more robust model development.

Method: Proposed IRIS framework that uses reinforcement learning with intrinsic rewards for autoregressive T2I models. The key insight is to maximize self-uncertainty rather than self-certainty during training.

Result: IRIS achieves performance competitive with or superior to methods using external rewards, demonstrating that intrinsic rewards can effectively improve image generation quality.

Conclusion: Autoregressive T2I models can be effectively improved using intrinsic rewards through reinforcement learning, with maximizing uncertainty proving more beneficial than maximizing certainty for generating human-preferred images.

Abstract: Despite the success of Reinforcement Learning from Human Feedback (RLHF) in language reasoning, its application to autoregressive Text-to-Image (T2I) generation is often constrained by the limited availability of human preference data. This paper explores how an autoregressive T2I model can learn from internal signals without relying on external rewards or labeled data. Contrary to recent findings in text generation, we show that maximizing self-uncertainty, rather than self-certainty, improves image generation. We observe that this is because autoregressive T2I models with low uncertainty tend to generate simple and uniform images, which are less aligned with human preferences. Based on these observations, we propose IRIS (Intrinsic Reward Image Synthesis), the first framework to improve autoregressive T2I models with reinforcement learning using only an intrinsic reward. Empirical results demonstrate that applying IRIS to autoregressive T2I models achieves performance that is competitive with or superior to external rewards.

Method: Developed an information and learning theory framework to analyze hidden representation evolution in VLMs, identifying layers with high redundancy that can be skipped.

Result: Layers with large redundancy predicted by the framework coincide with those skipped by popular layer-skipping methods, enabling faster inference while preserving performance.

Conclusion: Layer skipping is effective when applied to redundant layers identified by information theory, but applying it outside these conditions leads to model degradation.

Abstract: Vision-language models (VLMs) achieve incredible performance across a wide range of tasks, but their large size makes inference costly. Recent work shows that selectively skipping VLM layers can improve efficiency with minimal performance loss or even performance improvements. However, this technique remains underused due to the limited understanding of when layer skipping is beneficial. In this paper, we develop a framework that uses information and learning theory to characterize the conditions under which layer skipping enhances efficiency without sacrificing performance. Motivated by these observations, we analyze the evolution of the VLM’s hidden representations through the LLM backbone and show that layers with large redundancy as predicted by our framework coincide with those skipped by popular layer-skipping methods in practice, providing a unified theoretical scaffolding for multiple efficient inference techniques. Our experiments demonstrate that skipping such layers yields faster inference that preserves performance, and also show that applying skipping outside these conditions leads to model degradation.

Method: ATLAS uses a multi-agent framework with dedicated mechanisms for dynamic constraint management, iterative plan critique, and adaptive interleaved search to handle complex constraint-aware planning.

Result: ATLAS improves the final pass rate from 23.3% to 44.4% on the TravelPlanner benchmark and achieves 84% final pass rate in realistic settings with live information search, significantly outperforming ReAct (59%) and monolithic agents (27%).

Conclusion: ATLAS demonstrates superior planning performance in real-world travel planning tasks and is the first to show quantitative effectiveness with live information search and multi-turn feedback.

Abstract: While Large Language Models (LLMs) have shown remarkable advancements in reasoning and tool use, they often fail to generate optimal, grounded solutions under complex constraints. Real-world travel planning exemplifies these challenges, evaluating agents’ abilities to handle constraints that are explicit, implicit, and even evolving based on interactions with dynamic environments and user needs. In this paper, we present ATLAS, a general multi-agent framework designed to effectively handle such complex nature of constraints awareness in real-world travel planning tasks. ATLAS introduces a principled approach to address the fundamental challenges of constraint-aware planning through dedicated mechanisms for dynamic constraint management, iterative plan critique, and adaptive interleaved search. ATLAS demonstrates state-of-the-art performance on the TravelPlanner benchmark, improving the final pass rate from 23.3% to 44.4% over its best alternative. More importantly, our work is the first to demonstrate quantitative effectiveness on real-world travel planning tasks with live information search and multi-turn feedback. In this realistic setting, ATLAS showcases its superior overall planning performance, achieving an 84% final pass rate which significantly outperforms baselines including ReAct (59%) and a monolithic agent (27%).

Method: Reformulate EHRs as timestamped event chains and use autoregressive fine-tuning for Next Event Prediction to model disease progression patterns and causal relationships.

Result: NEP outperforms specialized EHR models by 4.6% AUROC and general-purpose LLMs by 7.2% C-index in temporal reasoning tasks, achieving state-of-the-art prediction accuracy with clinically interpretable attention patterns.

Conclusion: The NEP framework successfully enhances LLMs’ temporal reasoning capabilities for EHR analysis, providing both superior prediction performance and clinically meaningful interpretability.

Abstract: Electronic Health Records (EHRs) contain rich temporal dynamics that conventional encoding approaches fail to adequately capture. While Large Language Models (LLMs) show promise for EHR modeling, they struggle to reason about sequential clinical events and temporal dependencies. We propose Next Event Prediction (NEP), a framework that enhances LLMs’ temporal reasoning through autoregressive fine-tuning on clinical event sequences. By reformulating EHRs as timestamped event chains and predicting future medical events, NEP explicitly models disease progression patterns and causal relationships. Extensive evaluations across oncology survival prediction and clinical diagnosis tasks demonstrate NEP’s superiority, outperforming specialized EHR models by 4.6% AUROC and general-purpose LLMs by 7.2% C-index in temporal reasoning tasks. Our analyses reveal dual benefits: state-of-the-art prediction accuracy combined with clinically interpretable attention patterns that align with known disease pathways.

Method: Use LLM as both encoder (FCM to text) and decoder (text to FCM) to approximate identity mapping, with system instructions that don’t compare input/output directly.

Result: The system achieves lossy reconstruction that preserves strong causal edges while removing weak ones, and makes text more natural even if some FCM details are traded off.

Conclusion: LLM-based FCM-text mapping provides explainable AI with human-readable intermediate representations, functioning as an interpretable autoencoder alternative.

Abstract: A large language model (LLM) can map a feedback causal fuzzy cognitive map (FCM) into text and then reconstruct the FCM from the text. This explainable AI system approximates an identity map from the FCM to itself and resembles the operation of an autoencoder (AE). Both the encoder and the decoder explain their decisions in contrast to black-box AEs. Humans can read and interpret the encoded text in contrast to the hidden variables and synaptic webs in AEs. The LLM agent approximates the identity map through a sequence of system instructions that does not compare the output to the input. The reconstruction is lossy because it removes weak causal edges or rules while it preserves strong causal edges. The encoder preserves the strong causal edges even when it trades off some details about the FCM to make the text sound more natural.

Method: PPR trains a principle-based reward model for transparent process evaluation and introduces Reward Normalization (ReNorm) to calibrate outcome and process rewards, unifying principled step-level assessment with outcome verification.

Result: PPR achieves state-of-the-art performance across a wide range of benchmarks, demonstrating impressive robustness and generalization.

Conclusion: The proposed PPR approach effectively addresses the challenges of sparse outcome rewards and difficult process reward annotation, providing a unified framework that improves LLM agent performance through principled process evaluation and reward calibration.

Abstract: Large Language Models (LLMs) increasingly rely on external tools such as search engines to solve complex agentic tasks that require reasoning and external knowledge retrieval. Recently, reinforcement learning with verifiable rewards (RLVR) has demonstrated its effectiveness in advancing capabilities of LLMs by rewarding the final answers via outcome rewards. While straightforward to supervise, outcome rewards only provide sparse signals and delayed feedback, which limits their effectiveness on long trajectories. Process rewards address this by evaluating intermediate steps, providing fine-grained supervision and encouraging grounded problem solving. However, it is notoriously hard to annotate step-wise labels, especially in non-verifiable process without “golden” answers. Furthermore, step-wise judgment requires the balance between local quality with contribution to the final outcome, as optimizing towards higher process reward may not always align with better final outcomes. To address the above challenges, we introduce Principle Process Reward (PPR), an RL approach that unifies principled step-level assessment and outcome verification. We train a principle-based reward model to improve the transparency and reliability of process evaluation, and further introduce a Reward Normalization (ReNorm) strategy to calibrate outcome and process rewards. Experiment results show that PPR achieves state-of-the-art performance across a wide range of benchmarks, demonstrating its impressive robustness and generalization. Our code and model collection is available in this link.

Method: Introduces ABxLab framework that systematically probes agentic choice through controlled manipulations of option attributes (prices, ratings) and persuasive cues (psychological nudges) in a realistic web-based shopping environment.

Result: Agent decisions shift predictably and substantially in response to manipulated factors, revealing that agents are strongly biased choosers even without human cognitive constraints that typically shape biases.

Conclusion: Agent susceptibility to biases presents both risk (inheriting and amplifying human biases) and opportunity (consumer choice as testbed for behavioral science of AI agents). The framework is released as an open benchmark for rigorous evaluation of agent decision-making.

Abstract: Environments built for people are increasingly operated by a new class of economic actors: LLM-powered software agents making decisions on our behalf. These decisions range from our purchases to travel plans to medical treatment selection. Current evaluations of these agents largely focus on task competence, but we argue for a deeper assessment: how these agents choose when faced with realistic decisions. We introduce ABxLab, a framework for systematically probing agentic choice through controlled manipulations of option attributes and persuasive cues. We apply this to a realistic web-based shopping environment, where we vary prices, ratings, and psychological nudges, all of which are factors long known to shape human choice. We find that agent decisions shift predictably and substantially in response, revealing that agents are strongly biased choosers even without being subject to the cognitive constraints that shape human biases. This susceptibility reveals both risk and opportunity: risk, because agentic consumers may inherit and amplify human biases; opportunity, because consumer choice provides a powerful testbed for a behavioral science of AI agents, just as it has for the study of human behavior. We release our framework as an open benchmark for rigorous, scalable evaluation of agent decision-making.

Method: SMS links user-specific seeds (unique indices) with original samples and models through a self-supervised model seeding task, using joint-training to optimize both the seeding task and primary service task simultaneously while maintaining model utility.

Result: Extensive experiments demonstrate that SMS provides effective verification for genuine sample unlearning, addressing the limitations of existing backdoor-based verification methods.

Conclusion: The proposed SMS scheme successfully enables verification of genuine sample unlearning by establishing connections between user seeds, original samples, and models, offering a more reliable unlearning verification approach.

Abstract: Many machine unlearning methods have been proposed recently to uphold users’ right to be forgotten. However, offering users verification of their data removal post-unlearning is an important yet under-explored problem. Current verifications typically rely on backdooring, i.e., adding backdoored samples to influence model performance. Nevertheless, the backdoor methods can merely establish a connection between backdoored samples and models but fail to connect the backdoor with genuine samples. Thus, the backdoor removal can only confirm the unlearning of backdoored samples, not users’ genuine samples, as genuine samples are independent of backdoored ones. In this paper, we propose a Self-supervised Model Seeding (SMS) scheme to provide unlearning verification for genuine samples. Unlike backdooring, SMS links user-specific seeds (such as users’ unique indices), original samples, and models, thereby facilitating the verification of unlearning genuine samples. However, implementing SMS for unlearning verification presents two significant challenges. First, embedding the seeds into the service model while keeping them secret from the server requires a sophisticated approach. We address this by employing a self-supervised model seeding task, which learns the entire sample, including the seeds, into the model’s latent space. Second, maintaining the utility of the original service model while ensuring the seeding effect requires a delicate balance. We design a joint-training structure that optimizes both the self-supervised model seeding task and the primary service task simultaneously on the model, thereby maintaining model utility while achieving effective model seeding. The effectiveness of the proposed SMS scheme is evaluated through extensive experiments, which demonstrate that SMS provides effective verification for genuine sample unlearning, addressing existing limitations.

Method: Uses a five-task suite based on CLI utilities and computer processes in controlled environments, with LLMs acting agentically. Performance is measured through R-scores and RCL-PCL categorization.

Result: Evaluation of various models revealed significant obstacles to persistent self-replication, including context retention and multi-agent decision-making issues.

Conclusion: SOCK provides a standardized benchmark for future research and helps mitigate self-replication risks, with proposed research directions to safely reduce obstacles for multi-agent systems.

Abstract: We introduce SOCK, a benchmark command line interface (CLI) that measures large language models’ (LLMs) ability to self-replicate without human intervention. In this benchmark, self-replication is defined not only as an LLM’s ability to create a functioning and running copy of itself, but also the ability for that self-replication to persist and occur across different computational contexts. Accordingly, we’ve developed a system to categorize LLMs based on broad self-replication capabilities in two general classes, Replication-Capability Levels (RCL) and Persistence-Capability Levels (PCL). Using a five-task suite based on practically manipulable modern CLI utilities and computer processes, experiments are orchestrated in a controlled environment with an LLM acting agentically. The performance of the LLM on agent tasks is then computed to produce an R-score (a quantitative evaluation of overall self-replication ability) and data used to categorize LLMs into specific RCL-PCL matrices. SOCK offers two primary contributions: (1) Provides the first formalized definitions and benchmark suite for evaluating LLM self-replication, with the goal of establishing a standard for future research, to our knowledge; (2) Allows the industry to track the effectiveness of future multi-agent systems and mitigate potential self-replication threat vectors within them. The results compiled from evaluating a variety of open-weight and proprietary frontier models reveal significant obstacles to persistent self-replication and multi-agent systems, including context retention and multi-agent decision-making. We propose future research directions to safely reduce the severity of these obstacles, potentially lowering future risk of more functional multi-agent systems.

Method: Developed a multi-agent architecture that engages users in dialogue, decomposes experiments into tasks, performs stoichiometric calculations, and iteratively self-corrects output before generating hardware-ready files. Evaluated through systematic ablation study of 20 agent configurations.

Result: Agent reasoning capacity was the most critical factor, reducing quantitative errors by over 85% in complex tasks. Combined with multi-agent architecture and self-correction, achieved F1-score > 0.89 on challenging multi-step syntheses.

Conclusion: Establishes a blueprint for robust AI partners in autonomous labs, highlighting synergistic effects of modular design, advanced reasoning, and self-correction for performance and reliability in high-stakes scientific applications.

Abstract: The automation of chemical research through self-driving laboratories (SDLs) promises to accelerate scientific discovery, yet the reliability and granular performance of the underlying AI agents remain critical, under-examined challenges. In this work, we introduce AutoLabs, a self-correcting, multi-agent architecture designed to autonomously translate natural-language instructions into executable protocols for a high-throughput liquid handler. The system engages users in dialogue, decomposes experimental goals into discrete tasks for specialized agents, performs tool-assisted stoichiometric calculations, and iteratively self-corrects its output before generating a hardware-ready file. We present a comprehensive evaluation framework featuring five benchmark experiments of increasing complexity, from simple sample preparation to multi-plate timed syntheses. Through a systematic ablation study of 20 agent configurations, we assess the impact of reasoning capacity, architectural design (single- vs. multi-agent), tool use, and self-correction mechanisms. Our results demonstrate that agent reasoning capacity is the most critical factor for success, reducing quantitative errors in chemical amounts (nRMSE) by over 85% in complex tasks. When combined with a multi-agent architecture and iterative self-correction, AutoLabs achieves near-expert procedural accuracy (F1-score > 0.89) on challenging multi-step syntheses. These findings establish a clear blueprint for developing robust and trustworthy AI partners for autonomous laboratories, highlighting the synergistic effects of modular design, advanced reasoning, and self-correction to ensure both performance and reliability in high-stakes scientific applications. Code: https://github.com/pnnl/autolabs

Method: Constructed 630,000-entry knowledge base, used knowledge matching to align environmental subviews, designed Knowledge-Guided by Landmark (KGL) mechanism, and proposed Knowledge-Guided Dynamic Augmentation (KGDA) to integrate language, knowledge, vision, and historical information.

Result: Outperformed state-of-the-art methods on R2R and REVERIE datasets, achieving better navigation error, success rate, and path efficiency.

Conclusion: The LGK method effectively addresses common-sense reasoning limitations in vision-and-language navigation by leveraging external knowledge with landmark-guided mechanisms.

Abstract: Vision-and-language navigation is one of the core tasks in embodied intelligence, requiring an agent to autonomously navigate in an unfamiliar environment based on natural language instructions. However, existing methods often fail to match instructions with environmental information in complex scenarios, one reason being the lack of common-sense reasoning ability. This paper proposes a vision-and-language navigation method called Landmark-Guided Knowledge (LGK), which introduces an external knowledge base to assist navigation, addressing the misjudgment issues caused by insufficient common sense in traditional methods. Specifically, we first construct a knowledge base containing 630,000 language descriptions and use knowledge Matching to align environmental subviews with the knowledge base, extracting relevant descriptive knowledge. Next, we design a Knowledge-Guided by Landmark (KGL) mechanism, which guides the agent to focus on the most relevant parts of the knowledge by leveraging landmark information in the instructions, thereby reducing the data bias that may arise from incorporating external knowledge. Finally, we propose Knowledge-Guided Dynamic Augmentation (KGDA), which effectively integrates language, knowledge, vision, and historical information. Experimental results demonstrate that the LGK method outperforms existing state-of-the-art methods on the R2R and REVERIE vision-and-language navigation datasets, particularly in terms of navigation error, success rate, and path efficiency.

Method: Proposes a novel framework using formal abductive explanations that leverages background knowledge to identify features acting as unjustified proxies for protected attributes, with aptitude as a task-relevant property independent of group membership.

Result: The framework demonstrates applicability in real-world cases using examples from the German credit dataset, revealing hidden structural biases and explaining proxy discrimination in individual AI decisions.

Conclusion: The proposed framework effectively explains proxy discrimination by identifying unjustified proxy features and using aptitude-based fairness assessment, providing a substantive approach to address structural bias in AI systems.

Abstract: Artificial intelligence (AI) systems in high-stakes domains raise concerns about proxy discrimination, unfairness, and explainability. Existing audits often fail to reveal why unfairness arises, particularly when rooted in structural bias. We propose a novel framework using formal abductive explanations to explain proxy discrimination in individual AI decisions. Leveraging background knowledge, our method identifies which features act as unjustified proxies for protected attributes, revealing hidden structural biases. Central to our approach is the concept of aptitude, a task-relevant property independent of group membership, with a mapping function aligning individuals of equivalent aptitude across groups to assess fairness substantively. As a proof of concept, we showcase the framework with examples taken from the German credit dataset, demonstrating its applicability in real-world cases.

Method: Proposes text-grounded object localization to identify and crop the most relevant object to the question, combined with RAG for focused retrieval and a de-hallucination method based on question types.

Result: Increased VQA accuracy from 22.19% to 25.64% (3.45 percentage points improvement) over baseline Llama-3.2-Vision-11B agent, and reduced hallucination rate from 65.79% to 13.88% with improved truthfulness.

Conclusion: The proposed RAG method with text-grounded object localization effectively enhances VQA performance by enabling focused retrieval and reducing hallucinations through targeted image processing.

Abstract: We propose a method to improve Visual Question Answering (VQA) with Retrieval-Augmented Generation (RAG) by introducing text-grounded object localization. Rather than retrieving information based on the entire image, our approach enables the model to generate a bounding box around the object most relevant to the question, allowing for targeted image cropping and focused retrieval. This reduces background noise, improves alignment between visual and textual cues, and helps mitigate hallucinations. Our RAG method enhances context-aware VQA responses increased the accuracy from 22.19% to 25.64%, with an absolute increase of 3.45 percentage points, compared to the baseline Llama-3.2-Vision-11B agent. We also proposed a de-hallucination method based on question type which can effectively reduce the hallucination rate from 65.79% to 13.88% and improves the truthfulness score.

Method: Two-stage framework that hierarchically partitions database schemas into sub-schemas, synthesizes SQL queries across complexity levels, and applies quality-aware pipeline with LLM-as-a-judge validation, executability checks, automatic repair, and column balancing.

Result: SingSQL-LM models achieve state-of-the-art performance: 3B model reaches 82.87% Soft F1 and 73.03% EX on BIRD benchmark, outperforming best 3B baseline by +16.21 Soft F1 and +12.36 EX. Schema-free fine-tuning with schema-only inference provides most robust results.

Conclusion: SING-SQL establishes a scalable, database-agnostic paradigm for producing and evaluating enterprise-grade Text-to-SQL systems, enabling organizations to generate specialized models for their specific databases.

Abstract: Translating natural language questions into SQL has become a core challenge in enabling non-technical users to query databases. While recent work has explored large-scale synthetic data generation to improve model performance through post-training, most efforts emphasize cross-domain generalization. This leaves a gap for real-world enterprise scenarios, where models need to specialize to a single database schema and organizations require to be able to evaluate their Text-to-SQL systems on their own databases. To address this, we introduce SING-SQL, a fully automated two-stage framework for generating high-quality, high-coverage synthetic Text-to-SQL data for any target database, without relying on SQL logs or manual annotations. Our approach hierarchically partitions a database schema into sub-schemas, synthesizes SQL queries across multiple complexity levels, and applies a quality-aware pipeline that includes LLM-as-a-judge validation, executability checks, automatic repair, and column balancing. We further release SingSQL-LM, a family of compact language models fine-tuned on the synthetic data, achieving strong in-domain generalization. On the subset of the BIRD benchmark, SingSQL-LM-3B-R64 reaches 82.87% Soft F1 and 73.03% EX upper bound with 32 candidates, outperforming the best 3B-scale baseline by +16.21 in Soft F1 and +12.36 in EX. At the 1.5B scale, SingSQL-LM-1.5B-R64 improves over prior systems by +9.30 in Soft F1 and +4.49 in EX. On synthetic evaluation sets, SingSQL-LMs exceed prior systems by wide margins, establishing state-of-the-art performance among open models at comparable scales. Our study of context management strategies reveals that schema-free fine-tuning combined with schema-only inference provides the most robust results. These findings establish SING-SQL as a scalable, database-agnostic paradigm for producing and evaluating enterprise-grade Text-to-SQL systems.

Method: Collaborative compression framework combining expert pruning, mixed-precision quantization, and activation optimization.

Result: Reduced storage footprint of ultra-large MoE DeepSeek-V3 from 1.3TB to 103GB while preserving high output quality. Successfully deployed compressed model on platform with 128GB memory limit. Achieved better accuracy than traditional uniform low-bit quantization methods.

Conclusion: The proposed collaborative compression framework effectively enables deployment of ultra-large MoE models on edge platforms with strict memory constraints, outperforming traditional uniform quantization methods in both model size reduction and accuracy preservation.

Abstract: The Mixture of Experts (MoE) architecture is an important method for scaling Large Language Models (LLMs). It increases model capacity while keeping computation cost low. However, the ultra-large MoE models still have hundreds of billions of parameters, requiring massive memory/storage and leading to difficulties for deployment on resource-constrained edge platforms. Pruning or quantization alone can hardly address the issue, because of the super-aggressive compression ratio with significantly degraded accuracy and output quality. To facilitate the deployment of ultra-large MoEs on edge platforms, we propose a collaborative compression framework by combining expert pruning, mixed-precision quantization, and activation optimization. It can effectively reduce the storage footprint of the ultra-large MoE DeepSeek-V3 from 1.3TB to 103GB, while preserving high output quality with better accuracy than traditional uniform low-bit quantization methods. To the best of our knowledge, we are the first to deploy a compressed model from the ultra-large DeepSeek-V3 on the platform with a strict 128GB total memory limit. Our comprehensive experiments on multiple benchmarks under various memory constraints demonstrate the effectiveness of our method with smaller model sizes and higher accuracy than uniform low-bit quantization methods.

Method: Uses a graph-structured coordination mechanism with a central supervisory agent that oversees specialized task agents, enabling modularity, conflict resolution, and context-aware interactions.

Result: Developed a system that can manage Google Calendar events through natural language conversation while handling ambiguities and evaluating user commands effectively.

Conclusion: This approach demonstrates how structured reasoning and agent cooperation can enhance the usability and flexibility of personal calendar assistant tools.

Abstract: Recent advancements in LLMs have contributed to the rise of advanced conversational assistants that can assist with user needs through natural language conversation. This paper presents a ScheduleMe, a multi-agent calendar assistant for users to manage google calendar events in natural language. The system uses a graph-structured coordination mechanism where a central supervisory agent supervises specialized task agents, allowing modularity, conflicts resolution, and context-aware interactions to resolve ambiguities and evaluate user commands. This approach sets an example of how structured reasoning and agent cooperation might convince operators to increase the usability and flexibility of personal calendar assistant tools.

Method: Uses heterogeneous graph representation, HGNN-EM for encoding vehicle features with expert knowledge, and DDQN algorithm for training.

Result: Superior performance over baselines in safety, efficiency, stability, convergence rate, and real-time performance at four-way intersection.

Conclusion: The proposed framework effectively improves AV decision-making in complex traffic environments with diverse driving styles.

Abstract: Navigating heterogeneous traffic environments with diverse driving styles poses a significant challenge for autonomous vehicles (AVs) due to their inherent complexity and dynamic interactions. This paper addresses this challenge by proposing a heterogeneous graph reinforcement learning (GRL) framework enhanced with an expert system to improve AV decision-making performance. Initially, a heterogeneous graph representation is introduced to capture the intricate interactions among vehicles. Then, a heterogeneous graph neural network with an expert model (HGNN-EM) is proposed to effectively encode diverse vehicle features and produce driving instructions informed by domain-specific knowledge. Moreover, the double deep Q-learning (DDQN) algorithm is utilized to train the decision-making model. A case study on a typical four-way intersection, involving various driving styles of human vehicles (HVs), demonstrates that the proposed method has superior performance over several baselines regarding safety, efficiency, stability, and convergence rate, all while maintaining favorable real-time performance.

Method: NePTune dynamically translates natural language queries into executable Python programs that combine imperative control flow with soft logic operators, operating training-free with modular design that decouples perception from reasoning.

Result: Significant improvement over strong base models on multiple visual reasoning benchmarks, with effective compositional generalization and adaptation capabilities in novel environments.

Conclusion: NePTune successfully bridges the gap between neural perception and symbolic reasoning, enabling flexible compositional reasoning while supporting fine-tuning through differentiable operations.

Abstract: Modern Vision-Language Models (VLMs) have achieved impressive performance in various tasks, yet they often struggle with compositional reasoning, the ability to decompose and recombine concepts to solve novel problems. While neuro-symbolic approaches offer a promising direction, they are typically constrained by crisp logical execution or predefined predicates, which limit flexibility. In this work, we introduce NePTune, a neuro-symbolic framework that overcomes these limitations through a hybrid execution model that integrates the perception capabilities of foundation vision models with the compositional expressiveness of symbolic reasoning. NePTune dynamically translates natural language queries into executable Python programs that blend imperative control flow with soft logic operators capable of reasoning over VLM-generated uncertainty. Operating in a training-free manner, NePTune, with a modular design, decouples perception from reasoning, yet its differentiable operations support fine-tuning. We evaluate NePTune on multiple visual reasoning benchmarks and various domains, utilizing adversarial tests, and demonstrate a significant improvement over strong base models, as well as its effective compositional generalization and adaptation capabilities in novel environments.

Method: Used circuit analysis to examine attention heads across Qwen families and DeepSeek-distilled models, comparing different training regimes (distillation, SFT, group relative policy optimization) and conducting ablation and qualitative analyses.

Result: Found that post-training sparks emergence of functionally specialized attention heads that support structured reasoning. Different training methods produce distinct head dynamics: distillation/SFT add stable reasoning heads cumulatively, while group relative policy optimization uses dynamic search with iterative activation/evaluation/pruning. Controllable think models use compensatory heads rather than dedicated thinking heads.

Conclusion: There’s a crucial performance trade-off: specialized reasoning heads enable sophisticated problem-solving but can cause over-thinking failures on simpler tasks, revealing tension between complex reasoning and reliable elementary computations. Future training policies need to balance effective reasoning strategies with flawless execution.

Abstract: The remarkable capabilities of modern large reasoning models are largely unlocked through post-training techniques such as supervised fine-tuning and reinforcement learning. However, the architectural mechanisms behind such improvements remain largely opaque. In this work, we use circuit analysis to demonstrate that post-training for complex reasoning sparks the emergence of novel, functionally specialized attention heads. These heads collectively support structured reasoning and computation. Our comparative analysis across Qwen families and DeepSeek-distilled model reveals that these emergent heads evolve differently under different training regimes. Distillation and SFT foster a cumulative addition of stable reasoning heads. In contrast, group relative policy optimization operates in a dynamic search mode: relatively few attention heads are iteratively activated, evaluated, and pruned, with their survival closely tracking fluctuations in the task reward signal. Furthermore, we find that controllable think on/off models do not possess dedicated thinking heads. Instead, turning off explicit reasoning triggers a broader-but less efficient-set of compensatory heads. Through ablation and qualitative analyses, we connect these circuit-level dynamics to a crucial performance trade-off: strengthened heads enable sophisticated problem-solving strategies for difficult problems but can also introduce over-thinking failure modes, such as calculation errors or logical loops on simpler tasks. These findings connect circuit-level dynamics to macro-level performance, identifying an inherent tension where complex reasoning comes at the cost of elementary computations. More broadly, our work points to future directions for training policy design, emphasizing the need to balance the development of effective reasoning strategies with the assurance of reliable, flawless execution.

Method: Used creativity stress test with advertising concepts, simplifying ideas step by step and regenerating from simplified inputs. Combined quantitative comparisons with qualitative analysis and tested with ad-specific cues.

Result: Creative features disappeared early during simplification while factual content remained. Regeneration produced longer outputs with lexical variety but failed to recover original depth and distinctiveness. Ad-specific cues improved alignment but outputs still relied on familiar tropes.

Conclusion: Without targeted guidance, LLMs drift towards mediocrity in creative tasks; structured signals can partially counter this tendency and point towards pathways for developing creativity-sensitive models.

Abstract: Large language models (LLMs) generate fluent text yet often default to safe, generic phrasing, raising doubts about their ability to handle creativity. We formalize this tendency as a Galton-style regression to the mean in language and evaluate it using a creativity stress test in advertising concepts. When ad ideas were simplified step by step, creative features such as metaphors, emotions, and visual cues disappeared early, while factual content remained, showing that models favor high-probability information. When asked to regenerate from simplified inputs, models produced longer outputs with lexical variety but failed to recover the depth and distinctiveness of the originals. We combined quantitative comparisons with qualitative analysis, which revealed that the regenerated texts often appeared novel but lacked true originality. Providing ad-specific cues such as metaphors, emotional hooks and visual markers improved alignment and stylistic balance, though outputs still relied on familiar tropes. Taken together, the findings show that without targeted guidance, LLMs drift towards mediocrity in creative tasks; structured signals can partially counter this tendency and point towards pathways for developing creativity-sensitive models.

Method: Used Planner-R1 approach with dense process-level reward signals on TravelPlanner benchmark, comparing 8B and 32B models under different reward conditions.

Result: Achieved 56.9% final-pass rate (2.7x improvement over GPT-5 baseline), with 8B models being 3.5x more compute-efficient and 1.5x more memory-efficient than 32B models while maintaining competitive performance.

Conclusion: Reward shaping is crucial for scaling agentic RL, smaller models are competitive and more efficient, and these efficiency gains don’t sacrifice generalization on out-of-domain tasks.

Abstract: We investigated Agentic RL with large language models on the \textsc{TravelPlanner} benchmark. Our approach, \textsc{Planner-R1}, achieved a \textbf{56.9%} final-pass rate with only 180 training queries, a $2.7\times$ improvement over GPT-5’s $21.2%$ baseline and the strongest agentic result on the public leaderboard. A central finding was that smaller models (8B) were highly responsive to reward shaping: with dense process-level signals, they reached competitive performance while being $3.5\times$ more compute-efficient and $1.5\times$ more memory-efficient than 32B models. Larger models were more robust under sparse rewards but exhibited smaller relative gains from shaping and higher variance across runs. While curriculum learning offered no significant benefit, shaped rewards consistently amplified learning dynamics, making 8B models the most efficient setting for agentic RL. Crucially, these gains did not come at the cost of overfitting: fine-tuned models mostly maintained or exceeded baseline performance on out-of-domain tasks, including \textsc{Multi-IF}, \textsc{NaturalPlan}, and $\tau$-\textsc{Bench}. These results establish reward shaping as a decisive lever for scaling agentic RL, highlight the competitive strength of smaller models, and demonstrate that efficiency can be achieved without sacrificing generalization.

Method: The authors propose a new Deontic Argumentation Theory definition that accounts for weak permission and introduce a novel semantics to address the limitations of grounded semantics.

Result: The paper presents a new semantics that successfully supports weak permission in deontic argumentation, overcoming the limitations identified in grounded semantics.

Conclusion: The proposed semantics provides a viable solution for supporting weak permission in deontic argumentation, particularly in scenarios involving conflicting obligations.

Abstract: We address the issue of defining a semantics for deontic argumentation that supports weak permission. Some recent results show that grounded semantics do not support weak permission when there is a conflict between two obligations. We provide a definition of Deontic Argumentation Theory that accounts for weak permission, and we recall the result about grounded semantics. Then, we propose a new semantics that supports weak permission.

Method: Uses Vector-Quantized VAE with GAN discriminator to force poisoned images through a discrete bottleneck, quantizing them through a learned codebook to destroy fine-grained trigger patterns while preserving semantic content.

Result: Achieves 0% poison success rate against Gradient Matching and Bullseye Polytope attacks, 1.64% against Narcissus on CIFAR-10, while maintaining 91-95% clean accuracy. Over 50x faster than diffusion-based defenses.

Conclusion: PureVQ-GAN provides an effective and practical defense against data poisoning attacks by destroying backdoor triggers through discrete quantization while being significantly faster than existing methods.

Abstract: We introduce PureVQ-GAN, a defense against data poisoning that forces backdoor triggers through a discrete bottleneck using Vector-Quantized VAE with GAN discriminator. By quantizing poisoned images through a learned codebook, PureVQ-GAN destroys fine-grained trigger patterns while preserving semantic content. A GAN discriminator ensures outputs match the natural image distribution, preventing reconstruction of out-of-distribution perturbations. On CIFAR-10, PureVQ-GAN achieves 0% poison success rate (PSR) against Gradient Matching and Bullseye Polytope attacks, and 1.64% against Narcissus while maintaining 91-95% clean accuracy. Unlike diffusion-based defenses requiring hundreds of iterative refinement steps, PureVQ-GAN is over 50x faster, making it practical for real training pipelines.

Method: CiT uses two Branching Necessity evaluation methods: BN-DP where an auxiliary LLM directly judges branching necessity, and BN-SC which clusters multiple candidate actions to estimate agreement. The framework is integrated into three tree search frameworks: Tree of Thoughts, ReST-MCTS, and RAP.

Result: BN-DP consistently reduces token generation, model invocations, and runtime by 75-85% across all settings with negligible accuracy loss and sometimes accuracy gains. BN-SC yields substantial savings (up to 80%) but shows instability in some settings. The quality of auxiliary LLMs is critical for performance.

Conclusion: CiT provides an effective framework for reducing compute costs in LLM tree search while maintaining performance, with theoretical guarantees that BN-DP never increases LLM invocations relative to baseline, enabling more efficient long-horizon reasoning.

Abstract: Test-time scaling enables large language models (LLMs) to improve performance on long-horizon reasoning tasks by allocating additional compute at inference. Tree-search-based approaches achieve state-of-the-art results in this setting, but they are notoriously inefficient, often an order of magnitude slower than simpler iterative methods. We introduce Chain-in-Tree (CiT), a plug-in framework that adaptively decides when to branch during search rather than branching at every step. CiT relies on lightweight Branching Necessity (BN) evaluation methods: BN-DP (Direct Prompting), where an auxiliary LLM directly judges whether a step requires branching, and BN-SC (Self-Consistency), which clusters multiple candidate actions to estimate agreement. We integrate CiT into three representative LLM-in-the-loop tree search frameworks: Tree of Thoughts (ToT-BS), ReST-MCTS, and RAP, and evaluate across GSM8K and Math500. Our results show that: (1) BN-DP consistently reduces token generation, model invocations, and runtime by 75-85 percent across all settings, with negligible accuracy loss and sometimes accuracy gains; (2) BN-SC typically yields substantial savings (up to 80 percent) but shows instability in 1-4 out of 14 settings, caused by a small subset of examples that produce very long reasoning steps; (3) the quality of auxiliary LLMs is critical, not only the BN evaluator in BN-DP, but also the models used in BN-SC for clustering and equivalence checking. When these roles are filled by smaller LLMs, performance degrades. Importantly, BN-SC does not require LLMs in domains with deterministic action spaces, where clustering can be done programmatically. We also provide a theoretical guarantee that BN-DP never increases LLM invocations relative to the baseline and release a unified implementation of CiT across ToT-BS, ReST-MCTS, and RAP to facilitate reproducibility and extension.

Method: Proposed HiStyle with two-stage hierarchical prediction of style embeddings conditioned on textual prompts, incorporating contrastive learning for text-audio alignment, and a hybrid style annotation strategy combining statistical methods and human auditory preferences.

Result: HiStyle achieves significantly better style controllability than alternative approaches while preserving high speech quality in terms of naturalness and intelligibility.

Conclusion: The hierarchical approach to style embedding prediction based on observed clustering patterns in style embeddings enables more effective and precise control of speaking styles in text-to-speech systems.

Abstract: Controllable speech synthesis refers to the precise control of speaking style by manipulating specific prosodic and paralinguistic attributes, such as gender, volume, speech rate, pitch, and pitch fluctuation. With the integration of advanced generative models, particularly large language models (LLMs) and diffusion models, controllable text-to-speech (TTS) systems have increasingly transitioned from label-based control to natural language description-based control, which is typically implemented by predicting global style embeddings from textual prompts. However, this straightforward prediction overlooks the underlying distribution of the style embeddings, which may hinder the full potential of controllable TTS systems. In this study, we use t-SNE analysis to visualize and analyze the global style embedding distribution of various mainstream TTS systems, revealing a clear hierarchical clustering pattern: embeddings first cluster by timbre and subsequently subdivide into finer clusters based on style attributes. Based on this observation, we propose HiStyle, a two-stage style embedding predictor that hierarchically predicts style embeddings conditioned on textual prompts, and further incorporate contrastive learning to help align the text and audio embedding spaces. Additionally, we propose a style annotation strategy that leverages the complementary strengths of statistical methodologies and human auditory preferences to generate more accurate and perceptually consistent textual prompts for style control. Comprehensive experiments demonstrate that when applied to the base TTS model, HiStyle achieves significantly better style controllability than alternative style embedding predicting approaches while preserving high speech quality in terms of naturalness and intelligibility. Audio samples are available at https://anonymous.4open.science/w/HiStyle-2517/.

Method: Three-step approach: 1) Circuit analysis to identify tense-vulnerable attention heads, 2) Training channel-wise scaling vectors to recalibrate activations, 3) Preventative fine-tuning to reinforce robust refusal mechanisms.

Result: ASGuard effectively reduces attack success rate of targeted jailbreaking across three LLMs while preserving general capabilities and minimizing over-refusal, achieving Pareto-optimal balance between safety and utility.

Conclusion: Deep understanding of model internals enables practical, efficient, and targeted methods for adjusting model behavior, providing a path toward more reliable and interpretable AI safety.

Abstract: Large language models (LLMs), despite being safety-aligned, exhibit brittle refusal behaviors that can be circumvented by simple linguistic changes. As tense jailbreaking demonstrates that models refusing harmful requests often comply when rephrased in past tense, a critical generalization gap is revealed in current alignment methods whose underlying mechanisms are poorly understood. In this work, we introduce Activation-Scaling Guard (ASGuard), an insightful, mechanistically-informed framework that surgically mitigates this specific vulnerability. For the first step, we use circuit analysis to identify the specific attention heads causally linked to the targeted jailbreaking, the tense-changing attack. Second, we train a precise, channel-wise scaling vector to recalibrate the activation of tense vulnerable heads. Lastly, we apply it into a “preventative fine-tuning”, forcing the model to learn a more robust refusal mechanism. Across three LLMs, ASGuard effectively reduces the attack success rate of targeted jailbreaking while preserving general capabilities and minimizing over refusal, achieving a Pareto-optimal balance between safety and utility. Our findings underscore how adversarial suffixes suppress the propagation of the refusal-mediating direction, based on mechanistic analysis. Furthermore, our work showcases how a deep understanding of model internals can be leveraged to develop practical, efficient, and targeted methods for adjusting model behavior, charting a course for more reliable and interpretable AI safety.

Method: Used autoencoder with K-means clustering for career trend classification and LSTM deep learning for performance prediction on veteran NBA player data.

Result: The method performed better than other approaches with good generalization ability for evaluating various NBA career trends.

Conclusion: The approach can effectively analyze NBA player aging patterns and has potential applications in different sports analytics domains.

Abstract: The topic of aging decline on performance of NBA players has been discussed in this study. The autoencoder with K-means clustering machine learning method was adopted to career trend classification of NBA players, and the LSTM deep learning method was adopted in performance prediction of each NBA player. The dataset was collected from the basketball game data of veteran NBA players. The contribution of the work performed better than the other methods with generalization ability for evaluating various types of NBA career trend, and can be applied in different types of sports in the field of sport analytics.

Method: CIMNAS uses hardware-aware neural architecture search (HW-NAS) to simultaneously search across software parameters, quantization policies, and hardware parameters including device-, circuit-, and architecture-level co-optimizations.

Result: Achieved 90.1x-104.5x reduction in EDAP, 4.68x-4.82x improvement in TOPS/W, and 11.3x-12.78x enhancement in TOPS/mm² while maintaining 73.81% accuracy on ImageNet. Also demonstrated adaptability with SRAM-based ResNet50 achieving up to 819.5x EDAP reduction.

Conclusion: CIMNAS enables EDAP-focused optimization without accuracy loss and generates diverse software-hardware parameter combinations for high-performance CIM-based neural network designs, outperforming state-of-the-art methods.

Abstract: To maximize hardware efficiency and performance accuracy in Compute-In-Memory (CIM)-based neural network accelerators for Artificial Intelligence (AI) applications, co-optimizing both software and hardware design parameters is essential. Manual tuning is impractical due to the vast number of parameters and their complex interdependencies. To effectively automate the design and optimization of CIM-based neural network accelerators, hardware-aware neural architecture search (HW-NAS) techniques can be applied. This work introduces CIMNAS, a joint model-quantization-hardware optimization framework for CIM architectures. CIMNAS simultaneously searches across software parameters, quantization policies, and a broad range of hardware parameters, incorporating device-, circuit-, and architecture-level co-optimizations. CIMNAS experiments were conducted over a search space of 9.9x10^85 potential parameter combinations with the MobileNet model as a baseline and RRAM-based CIM architecture. Evaluated on the ImageNet dataset, CIMNAS achieved a reduction in energy-delay-area product (EDAP) ranging from 90.1x to 104.5x, an improvement in TOPS/W between 4.68x and 4.82x, and an enhancement in TOPS/mm^2 from 11.3x to 12.78x relative to various baselines, all while maintaining an accuracy of 73.81%. The adaptability and robustness of CIMNAS are demonstrated by extending the framework to support the SRAM-based ResNet50 architecture, achieving up to an 819.5x reduction in EDAP. Unlike other state-of-the-art methods, CIMNAS achieves EDAP-focused optimization without any accuracy loss, generating diverse software-hardware parameter combinations for high-performance CIM-based neural network designs. The source code of CIMNAS is available at https://github.com/OlgaKrestinskaya/CIMNAS.

Method: Lita operationalizes ’liteness’ - minimizing manual design while retaining essential autonomous agent elements, enabling unified evaluation without elaborate scaffolding.

Result: Experiments on Aider Polyglot and SWE-Bench show Lita achieves competitive or superior performance compared to workflow-based baselines while using fewer tokens and requiring less design effort.

Conclusion: Lita sufficiently reveals LLMs’ underlying coding competence, and the Agent Complexity Law predicts performance gaps between simple and complex agents will shrink as core models improve.

Abstract: Large language models (LLMs) are increasingly being applied to programming tasks, ranging from single-turn code completion to autonomous agents. Current code agent designs frequently depend on complex, hand-crafted workflows and tool sets. However, this reliance on elaborate scaffolding presents several challenges: agent performance becomes overly dependent on prompt tuning and custom design choices, heavy human intervention obscures a model’s true underlying capabilities, and intricate pipelines are costly to build and maintain. Furthermore, optimizing complex task prompts increases the risk of data leakage. Currently, when introducing new models, LLM providers like OpenAI and Anthropic often publish benchmark scores to demonstrate their models’ coding proficiency, but keep their proprietary evaluation frameworks confidential. To address these limitations, we introduce Lita (Lite Agent), which operationalizes liteness, a principle of minimizing manual design while retaining the essential elements of a fully autonomous agent. Lita enables a more faithful and unified evaluation without elaborate scaffolding. Experiments on the Aider Polyglot and SWE-Bench with frontier models demonstrate that Lita achieves competitive or superior performance compared to workflow-based and agentic baselines. Crucially, Lita also consumes fewer tokens and requires significantly less design effort. Our results suggest that Lita is sufficient to reveal the underlying coding competence of modern LLMs. Finally, we propose the Agent Complexity Law: the performance gap between agents of varying complexity, from simple to sophisticated designs, will shrink as the core model improves, ultimately converging to a negligible difference.

Method: Identified four reasoning stages where hazards arise and three safety constraint types. Created SafeMindBench benchmark with 5,558 samples across four task categories. Proposed SafeMindAgent with Planner-Executor architecture and three cascaded safety modules.

Result: Experiments show leading LLMs and embodied agents remain susceptible to safety-critical failures. SafeMindAgent significantly improves safety rate over baselines while maintaining comparable task completion.

Conclusion: SafeMindBench and SafeMindAgent provide both a rigorous evaluation suite and practical solution for systematic study and mitigation of safety risks in embodied LLM agents.

Abstract: Embodied agents powered by large language models (LLMs) inherit advanced planning capabilities; however, their direct interaction with the physical world exposes them to safety vulnerabilities. In this work, we identify four key reasoning stages where hazards may arise: Task Understanding, Environment Perception, High-Level Plan Generation, and Low-Level Action Generation. We further formalize three orthogonal safety constraint types (Factual, Causal, and Temporal) to systematically characterize potential safety violations. Building on this risk model, we present SafeMindBench, a multimodal benchmark with 5,558 samples spanning four task categories (Instr-Risk, Env-Risk, Order-Fix, Req-Align) across high-risk scenarios such as sabotage, harm, privacy, and illegal behavior. Extensive experiments on SafeMindBench reveal that leading LLMs (e.g., GPT-4o) and widely used embodied agents remain susceptible to safety-critical failures. To address this challenge, we introduce SafeMindAgent, a modular Planner-Executor architecture integrated with three cascaded safety modules, which incorporate safety constraints into the reasoning process. Results show that SafeMindAgent significantly improves safety rate over strong baselines while maintaining comparable task completion. Together, SafeMindBench and SafeMindAgent provide both a rigorous evaluation suite and a practical solution that advance the systematic study and mitigation of safety risks in embodied LLM agents.

Method: Created DeepJSONEval benchmark with 2100 multi-domain instances featuring deep nested JSON structures categorized by difficulty, and conducted experiments to evaluate LLM performance on complex JSON generation from unstructured web data.

Result: Experiments revealed significant performance gaps among LLMs in handling complex nested JSON structures, demonstrating the need for better evaluation of data extraction capabilities.

Conclusion: DeepJSONEval provides a more relevant benchmark for practical web data mining tasks and is open-sourced to advance research in structured JSON generation by LLMs.

Abstract: The internet is saturated with low-density, high-redundancy information, such as social media comments, repetitive news, and lengthy discussions, making it difficult to extract valuable insights efficiently. Multi-layer nested JSON structures provide an effective solution by compressing such information into semantically rich, hierarchical representations, which organize data into key-value pairs, arrays, and nested objects, preserving contextual relationships and enabling efficient storage, retrieval, and semantic querying. For instance, in news aggregation, a JSON object can nest an article’s metadata (title, author, date), content (text, multimedia), and multimedia information (multimedia type, caption) hierarchically. Large Language Models (LLMs) play a transformative role in web data mining by parsing unstructured text and outputting structured results directly into complex JSON schemas. However, current benchmarks for evaluating LLMs’ JSON output capabilities overemphasize pure JSON generation rather than assessing data comprehension and extraction abilities, a limitation that lacks relevance to practical web data mining tasks. To address this, we introduce DeepJSONEval, a novel benchmark featuring 2100 multi-domain instances with deep nested structures, categorized by difficulty. Experiments show significant performance gaps among LLMs in handling such complexity. Our benchmark and datasets are open-sourced to advance research in structured JSON generation.(https://github.com/GTS-AI-Infra-Lab-SotaS/DeepJSONEval).

Method: Development of the KIRETT project - a wrist-worn wearable system that integrates vital signs monitoring with treatment recommendations and situation detection algorithms.

Result: The integration of vital signs enables better time utilization for rescuers and provides useful information to support health professionals during critical treatments.

Conclusion: Vital signs play a significant role in improving decision-making during rescue operations and positively impact both health professionals and patients in emergency situations.

Abstract: The integration of vital signs in healthcare has witnessed a steady rise, promising health professionals to assist in their daily tasks to improve patient treatment. In life-threatening situations, like rescue operations, crucial decisions need to be made in the shortest possible amount of time to ensure that excellent treatment is provided during life-saving measurements. The integration of vital signs in the treatment holds the potential to improve time utilization for rescuers in such critical situations. They furthermore serve to support health professionals during the treatment with useful information and suggestions. To achieve such a goal, the KIRETT project serves to provide treatment recommendations and situation detection, combined on a wrist-worn wearable for rescue operations.This paper aims to present the significant role of vital signs in the improvement of decision-making during rescue operations and show their impact on health professionals and patients in need.

Method: Developed KIRETT wearable device with AI capabilities for treatment recommendations, vitals monitoring, and situation detection, then conducted a 2-day evaluation with 14 rescue service participants.

Result: Quantitative evaluation results from 14 participants show the system’s effectiveness in analyzing rescue operators’ needs in healthcare emergency scenarios.

Conclusion: KIRETT demonstrates potential to support rescue services with AI-driven treatment recommendations and real-time monitoring in time-critical emergency situations.

Abstract: Healthcare and Medicine are under constant pressure to provide patient-driven medical expertise to ensure a fast and accurate treatment of the patient. In such scenarios, the diagnosis contains, the family history, long term medical data and a detailed consultation with the patient. In time-critical emergencies, such conversation and time-consuming elaboration are not possible. Rescue services need to provide fast, reliable treatments for the patient in need. With the help of modern technologies, like treatment recommendations, real-time vitals-monitoring, and situation detection through artificial intelligence (AI) a situation can be analyzed and supported in providing fast, accurate patient-data-driven medical treatments. In KIRETT, a wearable device is developed to support in such scenarios and presents a way to provide treatment recommendation in rescue services. The objective of this paper is to present the quantitative results of a two-day KIRETT evaluation (14 participants) to analyze the needs of rescue operators in healthcare.

Method: Estimate question solvability and adapt outcome-supervised reward models and reinforcement learning with group-relative advantage to incorporate solvability into their objectives. Experiments conducted on math and multimodal datasets.

Result: The modifications consistently yield higher rates of process-correct reasoning and, in reinforcement learning, improved answer accuracy. The approach effectively reduces hallucinations and increases reliability in chain-of-thought reasoning.

Conclusion: Solvability is a key factor for reducing hallucinations and increasing reliability in chain-of-thought reasoning. Incorporating solvability estimation into training objectives leads to more valid intermediate reasoning steps and better overall performance.

Abstract: Reasoning quality in large language models depends not only on producing correct answers but also on generating valid intermediate steps. We study this through multiple-choice question answering (MCQA), which provides a controlled setting with fixed answer options. Our analysis shows that when questions are effectively unsolvable for a model, spurious chains of thought (CoTs) are more likely to appear, leading to false positives. By estimating the solvability of each question, we uncover an intermediate regime where learning is most effective. Building on this insight, we adapt outcome-supervised reward models and reinforcement learning with group-relative advantage to incorporate solvability into their objectives. Across experiments on math and multimodal datasets, these modifications consistently yield higher rates of process-correct reasoning and, in reinforcement learning, improved answer accuracy as well. Our results highlight solvability as a key factor for reducing hallucinations and increasing reliability in CoT reasoning.

Method: Created NuRisk dataset using real-world data from nuScenes and Waymo plus safety-critical scenarios from CommonRoad simulator, providing BEV sequential images with quantitative agent-level risk annotations. Benchmarked VLMs with different prompting techniques and fine-tuned a 7B VLM.

Result: Well-known VLMs achieved only 33% accuracy at high latency, failing at explicit spatio-temporal reasoning. The fine-tuned 7B VLM improved accuracy to 41% and reduced latency by 75%, demonstrating explicit spatio-temporal reasoning capabilities.

Conclusion: While the fine-tuned model represents significant improvement, the modest accuracy highlights the profound challenge of spatio-temporal reasoning in autonomous driving, establishing NuRisk as a critical benchmark for advancing this capability.

Abstract: Understanding risk in autonomous driving requires not only perception and prediction, but also high-level reasoning about agent behavior and context. Current Vision Language Models (VLMs)-based methods primarily ground agents in static images and provide qualitative judgments, lacking the spatio-temporal reasoning needed to capture how risks evolve over time. To address this gap, we propose NuRisk, a comprehensive Visual Question Answering (VQA) dataset comprising 2,900 scenarios and 1.1 million agent-level samples, built on real-world data from nuScenes and Waymo, supplemented with safety-critical scenarios from the CommonRoad simulator. The dataset provides Bird-Eye-View (BEV) based sequential images with quantitative, agent-level risk annotations, enabling spatio-temporal reasoning. We benchmark well-known VLMs across different prompting techniques and find that they fail to perform explicit spatio-temporal reasoning, resulting in a peak accuracy of 33% at high latency. To address these shortcomings, our fine-tuned 7B VLM agent improves accuracy to 41% and reduces latency by 75%, demonstrating explicit spatio-temporal reasoning capabilities that proprietary models lacked. While this represents a significant step forward, the modest accuracy underscores the profound challenge of the task, establishing NuRisk as a critical benchmark for advancing spatio-temporal reasoning in autonomous driving.

Method: Uses two vision-language modules: AnalyzerVLM for autonomous multi-step analysis and model proposal, and EvaluatorVLM for quantitative and perceptual assessment of candidate models regarding local detail fitness and overall trend generalizability.

Result: The pipeline effectively discovers models that capture fine details while ensuring strong generalizability, with ablation studies showing both multi-modality and multi-step reasoning are crucial.

Conclusion: The proposed multi-modal, multi-step pipeline successfully addresses the challenges in automated model discovery by leveraging vision-language models for effective model proposal and evaluation.

Abstract: Automated model discovery is the process of automatically searching and identifying the most appropriate model for a given dataset over a large combinatorial search space. Existing approaches, however, often face challenges in balancing the capture of fine-grained details with ensuring generalizability beyond training data regimes with a reasonable model complexity. In this paper, we present a multi-modal & multi-step pipeline for effective automated model discovery. Our approach leverages two vision-language-based modules (VLM), AnalyzerVLM and EvaluatorVLM, for effective model proposal and evaluation in an agentic way. AnalyzerVLM autonomously plans and executes multi-step analyses to propose effective candidate models. EvaluatorVLM assesses the candidate models both quantitatively and perceptually, regarding the fitness for local details and the generalibility for overall trends. Our results demonstrate that our pipeline effectively discovers models that capture fine details and ensure strong generalizability. Additionally, extensive ablation studies show that both multi-modality and multi-step reasoning play crucial roles in discovering favorable models.

Method: RoRecomp separates responses into two batch types: priority batches (combining short-correct and long-incorrect responses) to provide gradient signals for brevity, and compensation batches (using remaining responses from replay buffer) to maintain stability and prevent model collapse.

Result: Substantial efficiency gains across three settings: 27.7% reasoning length reduction in zero RL training, 46.8% reduction in unnecessary tool calls with improved accuracy in agentic RL, and up to 52.5% length reduction in thinking compression, all with minimal performance impact.

Conclusion: RoRecomp effectively addresses verbosity and inefficiency in RLVR training through strategic data recomposition, achieving significant efficiency improvements while maintaining model performance across diverse reasoning and agentic tasks.

Abstract: Reinforcement learning with verifiable rewards (RLVR) has proven effective in eliciting complex reasoning in large language models (LLMs). However, standard RLVR training often leads to excessively verbose processes (in reasoning tasks) and inefficient exploration trajectories (in agentic settings), as outcome-only rewards provide no incentive for efficiency and the high variance in response length within relatively small rollout groups results in noisy optimization signals. To address this, we propose Rollout Response Recomposition (RoRecomp), a plug-and-play method that guides models toward concise reasoning by strategically recomposing the training data. RoRecomp separates responses into two distinct batch types: 1) priority batches, which combine short-correct and long-incorrect responses selected from online batches to provide a clear gradient signal for brevity, and 2) compensation batches, which utilize remaining responses from a replay buffer to maintain stability and prevent model collapse. To comprehensively evaluate effectiveness, we test RoRecomp across three settings where results demonstrate substantial efficiency gains: reducing reasoning length by 27.7% in zero RL training, reducing unnecessary tool calls by 46.8% while improving accuracy in agentic RL, and achieving up to 52.5% length reduction in thinking compression, all with minimal performance impact.

Method: CURE employs a corrector that verifies outputs for leakage and revises them. It retrieves relevant unlearning targets as in-context references to enable detection and conditional revision without additional training.

Result: CURE significantly reduces information leakage, even from indirect queries, while maintaining response quality and general utility. It also shows robustness in continual unlearning scenarios.

Conclusion: CURE provides an effective and practical solution for machine unlearning, offering scalability, adaptability, and robustness for real-world applications.

Abstract: Language models trained on web-scale corpora risk memorizing and exposing sensitive information, prompting the need for effective machine unlearning. Prior methods mainly focus on input queries to suppress sensitive outputs, yet this often fails to eliminate the underlying knowledge and limits scalability. To address this, we propose Corrective Unlearning with Retrieved Exclusions (CURE), a novel unlearning framework that verifies model outputs for leakage and revises them into safe responses. Specifically, CURE employs a lightweight corrector that is applied to the original model to verify whether outputs contain target knowledge and to rewrite them if any leakage is detected. To efficiently handle large-scale unlearning requests, CURE retrieves unlearning targets that are relevant to the initial response and provides them as in-context references to the corrector for detection and conditional revision. By leveraging this retrieval augmentation, the corrector can adapt to new unlearning requests without additional training. Extensive evaluations demonstrate that CURE substantially reduces information leakage, even from indirect queries where prior works fall short, while maintaining response quality and general utility. Moreover, it demonstrates robustness under continual unlearning scenarios, making it practical for real-world applications.

Method: Proposes UMFDet framework with VLM backbone, Category-aware Mixture-of-Experts Adapter for category-specific cues, and attribution chain-of-thought mechanism for implicit reasoning guidance.

Result: UMFDet achieves robust performance across both misinformation types, outperforming specialized baselines on the comprehensive OmniFake dataset of 127K samples.

Conclusion: The unified approach offers a practical solution for real-world multimodal deception detection, bridging the gap between isolated research communities.

Abstract: In recent years, detecting fake multimodal content on social media has drawn increasing attention. Two major forms of deception dominate: human-crafted misinformation (e.g., rumors and misleading posts) and AI-generated content produced by image synthesis models or vision-language models (VLMs). Although both share deceptive intent, they are typically studied in isolation. NLP research focuses on human-written misinformation, while the CV community targets AI-generated artifacts. As a result, existing models are often specialized for only one type of fake content. In real-world scenarios, however, the type of a multimodal post is usually unknown, limiting the effectiveness of such specialized systems. To bridge this gap, we construct the Omnibus Dataset for Multimodal News Deception (OmniFake), a comprehensive benchmark of 127K samples that integrates human-curated misinformation from existing resources with newly synthesized AI-generated examples. Based on this dataset, we propose Unified Multimodal Fake Content Detection (UMFDet), a framework designed to handle both forms of deception. UMFDet leverages a VLM backbone augmented with a Category-aware Mixture-of-Experts (MoE) Adapter to capture category-specific cues, and an attribution chain-of-thought mechanism that provides implicit reasoning guidance for locating salient deceptive signals. Extensive experiments demonstrate that UMFDet achieves robust and consistent performance across both misinformation types, outperforming specialized baselines and offering a practical solution for real-world multimodal deception detection.

Method: Two-stage NAS framework with Navigator LLM guiding search direction and Generator LLM synthesizing candidates, managed by a Coordinator module that combines LLMs’ inherent knowledge with iterative feedback and historical trajectory.

Result: Achieves state-of-the-art results on ImageNet and NAS-Bench-201, and consistently enhances performance and efficiency of various two-stage NAS methods across diverse search spaces.

Conclusion: CoLLM-NAS demonstrates excellent generalization and superior performance compared to existing NAS methods and conventional search algorithms.

Abstract: The integration of Large Language Models (LLMs) with Neural Architecture Search (NAS) has introduced new possibilities for automating the design of neural architectures. However, most existing methods face critical limitations, including architectural invalidity, computational inefficiency, and inferior performance compared to traditional NAS. In this work, we present Collaborative LLM-based NAS (CoLLM-NAS), a two-stage NAS framework with knowledge-guided search driven by two complementary LLMs. Specifically, we propose a Navigator LLM to guide search direction and a Generator LLM to synthesize high-quality candidates, with a dedicated Coordinator module to manage their interaction. CoLLM-NAS efficiently guides the search process by combining LLMs’ inherent knowledge of structured neural architectures with progressive knowledge from iterative feedback and historical trajectory. Experimental results on ImageNet and NAS-Bench-201 show that CoLLM-NAS surpasses existing NAS methods and conventional search algorithms, achieving new state-of-the-art results. Furthermore, CoLLM-NAS consistently enhances the performance and efficiency of various two-stage NAS methods (e.g., OFA, SPOS, and AutoFormer) across diverse search spaces (e.g., MobileNet, ShuffleNet, and AutoFormer), demonstrating its excellent generalization.

Method: Proposes a pragmatic reframing of LLMs as pattern matchers rather than probabilistic models, and introduces practical guardrails including independent draws, preregistered human baselines, reliability-aware validation, and subgroup calibration.

Result: The paper outlines cautions for interpreting LLM outputs and provides a framework for using LLMs responsibly in social science research.

Conclusion: Researchers can engage in useful prototyping and forecasting with LLMs by treating them as high-capacity pattern matchers under explicit scope conditions, avoiding the misconception that they provide probabilistic inference.

Abstract: Large Language Models (LLMs) are being increasingly used as synthetic agents in social science, in applications ranging from augmenting survey responses to powering multi-agent simulations. Because strong prediction plus conditioning prompts, token log-probs, and repeated sampling mimic Bayesian workflows, their outputs can be misinterpreted as posterior-like evidence from a coherent model. However, prediction does not equate to probabilism, and accurate points do not imply calibrated uncertainty. This paper outlines cautions that should be taken when interpreting LLM outputs and proposes a pragmatic reframing for the social sciences in which LLMs are used as high-capacity pattern matchers for quasi-predictive interpolation under explicit scope conditions and not as substitutes for probabilistic inference. Practical guardrails such as independent draws, preregistered human baselines, reliability-aware validation, and subgroup calibration, are introduced so that researchers may engage in useful prototyping and forecasting while avoiding category errors.

Method: Multi-agent framework that ingests unstructured policy documents to generate evolving safety benchmarks through a synergistic pipeline with Self-evolving Evaluation loop that learns from results to create more sophisticated test cases.

Result: Demonstrated consistent decline in model safety as evaluation hardens - GPT-5’s safety rate on EU AI Act dropped from 72.50% to 36.36% over iterations, uncovering vulnerabilities missed by traditional methods.

Conclusion: Reveals limitations of static assessments and highlights need for dynamic evaluation ecosystems to ensure safe and responsible deployment of advanced AI.

Abstract: The rapid integration of Large Language Models (LLMs) into high-stakes domains necessitates reliable safety and compliance evaluation. However, existing static benchmarks are ill-equipped to address the dynamic nature of AI risks and evolving regulations, creating a critical safety gap. This paper introduces a new paradigm of agentic safety evaluation, reframing evaluation as a continuous and self-evolving process rather than a one-time audit. We then propose a novel multi-agent framework SafeEvalAgent, which autonomously ingests unstructured policy documents to generate and perpetually evolve a comprehensive safety benchmark. SafeEvalAgent leverages a synergistic pipeline of specialized agents and incorporates a Self-evolving Evaluation loop, where the system learns from evaluation results to craft progressively more sophisticated and targeted test cases. Our experiments demonstrate the effectiveness of SafeEvalAgent, showing a consistent decline in model safety as the evaluation hardens. For instance, GPT-5’s safety rate on the EU AI Act drops from 72.50% to 36.36% over successive iterations. These findings reveal the limitations of static assessments and highlight our framework’s ability to uncover deep vulnerabilities missed by traditional methods, underscoring the urgent need for dynamic evaluation ecosystems to ensure the safe and responsible deployment of advanced AI.

Method: An end-to-end pipeline using web scraping and LLMs to extract structured information from unstructured drug leaflets and create knowledge graphs.

Result: Created MEDAKA dataset with clinically relevant attributes, evaluated through manual inspection and LLM-as-a-Judge framework, showing better coverage than existing biomedical KGs.

Conclusion: MEDAKA supports patient safety monitoring and drug recommendation, and the pipeline can be adapted for other domains with unstructured text.

Abstract: Knowledge graphs (KGs) are increasingly used to represent biomedical information in structured, interpretable formats. However, existing biomedical KGs often focus narrowly on molecular interactions or adverse events, overlooking the rich data found in drug leaflets. In this work, we present (1) a hackable, end-to-end pipeline to create KGs from unstructured online content using a web scraper and an LLM; and (2) a curated dataset, MEDAKA, generated by applying this method to publicly available drug leaflets. The dataset captures clinically relevant attributes such as side effects, warnings, contraindications, ingredients, dosage guidelines, storage instructions and physical characteristics. We evaluate it through manual inspection and with an LLM-as-a-Judge framework, and compare its coverage with existing biomedical KGs and databases. We expect MEDAKA to support tasks such as patient safety monitoring and drug recommendation. The pipeline can also be used for constructing KGs from unstructured texts in other domains. Code and dataset are available at https://github.com/medakakg/medaka.

Method: Proposes LMILAtt model that integrates LSTM autoencoders to extract temporal dynamic features from user tweets and uses attention mechanisms to dynamically weight key texts within a multi-example learning architecture.

Result: Experiments on WU3D dataset show the model significantly outperforms baseline models in accuracy, recall and F1 score. The weakly supervised learning strategy reduces labeling costs.

Conclusion: The model provides an efficient solution for large-scale social media depression screening by improving detection accuracy while significantly reducing annotation costs through weakly supervised learning.

Abstract: Depression is a major global public health challenge and its early identification is crucial. Social media data provides a new perspective for depression detection, but existing methods face limitations such as insufficient accuracy, insufficient utilization of time series features, and high annotation costs. To this end, this study proposes the LMILAtt model, which innovatively integrates Long Short-Term Memory autoencoders and attention mechanisms: firstly, the temporal dynamic features of user tweets (such as depressive tendency evolution patterns) are extracted through unsupervised LSTM autoencoders. Secondly, the attention mechanism is used to dynamically weight key texts (such as early depression signals) and construct a multi-example learning architecture to improve the accuracy of user-level detection. Finally, the performance was verified on the WU3D dataset labeled by professional medicine. Experiments show that the model is significantly better than the baseline model in terms of accuracy, recall and F1 score. In addition, the weakly supervised learning strategy significantly reduces the cost of labeling and provides an efficient solution for large-scale social media depression screening.

Method: Developed a field manual based on deploying the “irAE-Agent” system at Mass General Brigham and conducting structured interviews with 20 clinicians, engineers, and informatics leaders, identifying five key implementation challenges.

Result: Revealed that less than 20% of effort was dedicated to prompt engineering and model development, while over 80% was consumed by sociotechnical implementation work, categorized into five “heavy lifts”: data integration, model validation, ensuring economic value, managing system drift, and governance.

Conclusion: The field manual shifts focus from algorithmic development to essential infrastructure and implementation work needed to successfully translate generative AI from pilot projects into routine clinical care, bridging the “valley of death” in healthcare AI deployment.

Abstract: Large language models (LLMs) integrated into agent-driven workflows hold immense promise for healthcare, yet a significant gap exists between their potential and practical implementation within clinical settings. To address this, we present a practitioner-oriented field manual for deploying generative agents that use electronic health record (EHR) data. This guide is informed by our experience deploying the “irAE-Agent”, an automated system to detect immune-related adverse events from clinical notes at Mass General Brigham, and by structured interviews with 20 clinicians, engineers, and informatics leaders involved in the project. Our analysis reveals a critical misalignment in clinical AI development: less than 20% of our effort was dedicated to prompt engineering and model development, while over 80% was consumed by the sociotechnical work of implementation. We distill this effort into five “heavy lifts”: data integration, model validation, ensuring economic value, managing system drift, and governance. By providing actionable solutions for each of these challenges, this field manual shifts the focus from algorithmic development to the essential infrastructure and implementation work required to bridge the “valley of death” and successfully translate generative AI from pilot projects into routine clinical care.

Method: Uses a coordinated multi-agent framework with: Planning Agent (interprets requirements into blueprints), Generation Agent (produces C# scripts), Automation Agent (handles engine component binding and scene construction), and Debugging Agent (provides real-time error correction).

Result: Successfully generated three distinct game prototypes, demonstrating 91.4% reduction in development time while requiring no coding from users.

Conclusion: UniGen effectively bridges the gap between MLLM outputs and production-ready 3D games, democratizing game creation and significantly accelerating development.

Abstract: Developing 3D games requires specialized expertise across multiple domains, including programming, 3D modeling, and engine configuration, which limits access to millions of potential creators. Recently, researchers have begun to explore automated game development. However, existing approaches face three primary challenges: (1) limited scope to 2D content generation or isolated code snippets; (2) requirement for manual integration of generated components into game engines; and (3) poor performance on handling interactive game logic and state management. While Multimodal Large Language Models (MLLMs) demonstrate potential capabilities to ease the game generation task, a critical gap still remains in translating these outputs into production-ready, executable game projects based on game engines such as Unity and Unreal Engine. To bridge the gap, this paper introduces UniGen, the first end-to-end coordinated multi-agent framework that automates zero-coding development of runnable 3D games from natural language requirements. Specifically, UniGen uses a Planning Agent that interprets user requirements into structured blueprints and engineered logic descriptions; after which a Generation Agent produces executable C# scripts; then an Automation Agent handles engine-specific component binding and scene construction; and lastly a Debugging Agent provides real-time error correction through conversational interaction. We evaluated UniGen on three distinct game prototypes. Results demonstrate that UniGen not only democratizes game creation by requiring no coding from the user, but also reduces development time by 91.4%. We release UniGen at https://github.com/yxwan123/UniGen. A video demonstration is available at https://www.youtube.com/watch?v=xyJjFfnxUx0.

Method: Proposes integrating interpretive qualitative approaches from social sciences, drawing from Clifford Geertz’s “thick description” concept to develop AI systems that produce “thick outputs” grounded in user context and intent.

Result: Outlines three necessary conditions for successful cultural alignment: sufficiently scoped cultural representations, capacity for nuanced outputs, and anchoring outputs in cultural contexts implied within prompts.

Conclusion: Calls for cross-disciplinary collaboration and adoption of qualitative, ethnographic evaluation methods to develop AI systems that are genuinely culturally sensitive, ethically responsible, and reflective of human complexity.

Abstract: While cultural alignment has increasingly become a focal point within AI research, current approaches relying predominantly on quantitative benchmarks and simplistic proxies fail to capture the deeply nuanced and context-dependent nature of human cultures. Existing alignment practices typically reduce culture to static demographic categories or superficial cultural facts, thereby sidestepping critical questions about what it truly means to be culturally aligned. This paper argues for a fundamental shift towards integrating interpretive qualitative approaches drawn from social sciences into AI alignment practices, specifically in the context of Large Language Models (LLMs). Drawing inspiration from Clifford Geertz’s concept of “thick description,” we propose that AI systems must produce outputs that reflect deeper cultural meanings–what we term “thick outputs”-grounded firmly in user-provided context and intent. We outline three necessary conditions for successful cultural alignment: sufficiently scoped cultural representations, the capacity for nuanced outputs, and the anchoring of outputs in the cultural contexts implied within prompts. Finally, we call for cross-disciplinary collaboration and the adoption of qualitative, ethnographic evaluation methods as vital steps toward developing AI systems that are genuinely culturally sensitive, ethically responsible, and reflective of human complexity.

Method: Repurposed LangGraph’s tool functionality to provide agents with black box functions to interrogate, using trial-and-error exploration in a children’s parlor game and applying the same strategy to an Atomic Layer Processing reactor simulation.

Result: LLM agents successfully demonstrated knowledge discovery through persistent exploration, showing strong path-dependence of results and the ability to discover and exploit diverse chemical interactions in reactor simulations with limited probe capabilities.

Conclusion: LLM agents can effectively function as autonomous knowledge discovery tools in materials science, capable of exploring complex systems and generating verifiable statements without explicit instructions.

Abstract: Large Language Models (LLMs) have garnered significant attention for several years now. Recently, their use as independently reasoning agents has been proposed. In this work, we test the potential of such agents for knowledge discovery in materials science. We repurpose LangGraph’s tool functionality to supply agents with a black box function to interrogate. In contrast to process optimization or performing specific, user-defined tasks, knowledge discovery consists of freely exploring the system, posing and verifying statements about the behavior of this black box, with the sole objective of generating and verifying generalizable statements. We provide proof of concept for this approach through a children’s parlor game, demonstrating the role of trial-and-error and persistence in knowledge discovery, and the strong path-dependence of results. We then apply the same strategy to show that LLM agents can explore, discover, and exploit diverse chemical interactions in an advanced Atomic Layer Processing reactor simulation using intentionally limited probe capabilities without explicit instructions.

Method: Designed a human-centered questionnaire based on Gienapp’s utility-dimension framework, iteratively refined through multiple rounds of ratings on query-output pairs and semantic discussions, incorporating feedback from both human raters and human-LLM pairs.

Result: LLMs reliably focus on metric descriptions and scale labels but struggle with detecting textual format variations. Humans had difficulty focusing strictly on metric descriptions and labels. LLM ratings were helpful but lacked agreement with human numeric ratings.

Conclusion: The final questionnaire extends the initial framework by focusing on user intent, text structuring, and information verifiability, providing a more comprehensive evaluation tool for RAG systems.

Abstract: Retrieval-augmented generation (RAG) systems are increasingly deployed in user-facing applications, yet systematic, human-centered evaluation of their outputs remains underexplored. Building on Gienapp’s utility-dimension framework, we designed a human-centred questionnaire that assesses RAG outputs across 12 dimensions. We iteratively refined the questionnaire through several rounds of ratings on a set of query-output pairs and semantic discussions. Ultimately, we incorporated feedback from both a human rater and a human-LLM pair. Results indicate that while large language models (LLMs) reliably focus on metric descriptions and scale labels, they exhibit weaknesses in detecting textual format variations. Humans struggled to focus strictly on metric descriptions and labels. LLM ratings and explanations were viewed as a helpful support, but numeric LLM and human ratings lacked agreement. The final questionnaire extends the initial framework by focusing on user intent, text structuring, and information verifiability.

Method: Proposes diversity-incentivized exploration using global sequence-level diversity as intrinsic reward, with potential-based reward shaping to preserve optimal policy invariance and heuristics to prevent reward hacking.

Result: DIVER outperforms competitive RLVR baselines on both in-domain and out-of-domain tasks, excelling in Pass@1 and Pass@k evaluations.

Conclusion: Global sequence-level diversity is crucial for incentivizing deep exploration in versatile reasoning tasks, and DIVER provides an effective framework for improving reasoning capabilities in LLMs.

Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a crucial paradigm for incentivizing reasoning capabilities in Large Language Models (LLMs). Due to vast state-action spaces and reward sparsity in reasoning tasks, existing methods often struggle with deficient exploration and poor sample efficiency. In the paper, we propose \textbf{DIVER} (\textbf{D}iversity-\textbf{I}ncentivized Exploration for \textbf{V}ersatil\textbf{E} \textbf{R}easoning), an innovative framework that highlights the pivotal role of global sequence-level diversity to incentivize deep exploration for versatile reasoning. We first conduct a primary empirical study to reveal a strong positive correlation between global diversity and reasoning capacity. Building on this insight, we introduce global diversity incentives as an intrinsic reward to promote deep exploration in a semantically structured space. Incorporating the intrinsic reward, we develop a potential-based reward shaping mechanism to preserve optimal policy invariance and design simple heuristics to mitigate possible reward hacking. Experimental results show that DIVER outperforms competitive RLVR baselines with various exploration strategies on both in-domain and out-of-domain tasks, excelling in both Pass@1 and Pass@k evaluations. Our code is available at https://github.com/NJU-RL/DIVER.

Method: Benchmarked direct, GEMM-based, and Winograd convolution algorithms across modern CPUs from ARM, Intel, AMD, Apple, and Nvidia, measuring both latency and energy efficiency.

Result: Nvidia AGX Orin with GEMM algorithm achieved the best trade-off between inference latency and energy consumption. Identified key architectural factors governing CPU efficiency for convolution operations.

Conclusion: Provides practical guidance for energy-aware embedded deployment of deep learning models on CPUs, highlighting optimal algorithm-hardware combinations.

Abstract: This work evaluates state-of-the-art convolution algorithms for CPU-based deep learning inference. While most prior studies focus on GPUs or NPUs, CPU implementations remain relatively underoptimized. We benchmark direct, GEMM-based, and Winograd convolutions across modern CPUs from ARM __ , Intel __ , AMD __ , Apple __ , and Nvidia __ , considering both latency and energy efficiency. Our results highlight the key architectural factors that govern CPU efficiency for convolution operations, providing practical guidance for energy-aware embedded deployment. As a main results of this work, the Nvidia __ AGX Orin combined with the GEMM algorithm achieves the best trade-off between inference latency and energy consumption.

Method: Decomposes samples into fine-grained slices and uses Asymmetric Partitioning to create balanced scheduling units optimized separately for forward and backward passes, orchestrated by a two-phase solver and high-fidelity simulator.

Result: Achieves up to 2.8x training throughput improvement over baselines while maintaining both superior workload balance and high resource efficiency.

Conclusion: SlimPack fundamentally rethinks data packing and scheduling to holistically resolve imbalances across all parallel dimensions, breaking conventional trade-offs between balance and efficiency.

Abstract: The efficient distributed training of Large Language Models (LLMs) is severely hampered by the extreme variance in context lengths. This data heterogeneity, amplified by conventional packing strategies and asymmetric forward-backward costs, leads to critical inefficiencies such as cascading workload imbalances and severe hardware underutilization. Existing solutions attempt to mitigate these challenges, but often at the expense of memory or communication efficiency. To address these challenges, we introduce SlimPack, a framework that fundamentally rethinks data packing and scheduling by decomposing samples into fine-grained slices. This slice-level decomposition immediately mitigates critical memory and communication bottlenecks by transforming large, volatile workloads into a stream of smaller, manageable units. This flexibility is then harnessed for our core innovation, Asymmetric Partitioning, which assembles balanced scheduling units uniquely optimized for the different demands of the forward and backward passes. Orchestrated by a two-phase solver and a high-fidelity simulator, SlimPack holistically resolves imbalances across all parallel dimensions. Extensive experiments demonstrate that SlimPack achieves up to a $2.8\times$ training throughput improvement over baselines, breaking the conventional trade-off by delivering both superior balance and high resource efficiency.

Method: Variational Bayesian inference combined with LLM proposals to learn symbolic state representations and causal processes modeling stochastic causal-effect relations from limited data.

Result: Across five simulated tabletop robotics environments, the learned models enable fast planning that generalizes to held-out tasks with more objects and more complex goals, outperforming various baselines.

Conclusion: The proposed framework successfully addresses the challenge of modeling both endogenous actions and exogenous mechanisms for improved planning in complex environments.

Abstract: Long-horizon embodied planning is challenging because the world does not only change through an agent’s actions: exogenous processes (e.g., water heating, dominoes cascading) unfold concurrently with the agent’s actions. We propose a framework for abstract world models that jointly learns (i) symbolic state representations and (ii) causal processes for both endogenous actions and exogenous mechanisms. Each causal process models the time course of a stochastic causal-effect relation. We learn these world models from limited data via variational Bayesian inference combined with LLM proposals. Across five simulated tabletop robotics environments, the learned models enable fast planning that generalizes to held-out tasks with more objects and more complex goals, outperforming a range of baselines.

Method: ILR combines Dynamic Interaction (adaptive cooperative/competitive strategies based on question difficulty and model ability) using Idea3 paradigm (Idea Sharing, Analysis, Fusion), and Perception Calibration using Group Relative Policy Optimization (GRPO) to integrate reward distributions across agents.

Result: ILR consistently outperforms single-agent learning across 3 LLMs from 2 model families on 5 mathematical and 1 coding benchmarks, achieving up to 5% improvement over the strongest baseline. Idea3 enhances robustness of stronger LLMs, and dynamic interaction boosts learning compared to pure cooperative/competitive strategies.

Conclusion: Multi-agent interaction can effectively enhance LLMs’ independent problem-solving ability. The ILR framework with dynamic interaction and perception calibration provides a promising approach for developing more capable and robust multi-agent systems that align with human cognitive learning patterns.

Abstract: Existing multi-agent learning approaches have developed interactive training environments to explicitly promote collaboration among multiple Large Language Models (LLMs), thereby constructing stronger multi-agent systems (MAS). However, during inference, they require re-executing the MAS to obtain final solutions, which diverges from human cognition that individuals can enhance their reasoning capabilities through interactions with others and resolve questions independently in the future. To investigate whether multi-agent interaction can enhance LLMs’ independent problem-solving ability, we introduce ILR, a novel co-learning framework for MAS that integrates two key components: Dynamic Interaction and Perception Calibration. Specifically, Dynamic Interaction first adaptively selects either cooperative or competitive strategies depending on question difficulty and model ability. LLMs then exchange information through Idea3 (Idea Sharing, Idea Analysis, and Idea Fusion), an innovative interaction paradigm designed to mimic human discussion, before deriving their respective final answers. In Perception Calibration, ILR employs Group Relative Policy Optimization (GRPO) to train LLMs while integrating one LLM’s reward distribution characteristics into another’s reward function, thereby enhancing the cohesion of multi-agent interactions. We validate ILR on three LLMs across two model families of varying scales, evaluating performance on five mathematical benchmarks and one coding benchmark. Experimental results show that ILR consistently outperforms single-agent learning, yielding an improvement of up to 5% over the strongest baseline. We further discover that Idea3 can enhance the robustness of stronger LLMs during multi-agent inference, and dynamic interaction types can boost multi-agent learning compared to pure cooperative or competitive strategies.

Method: Created a reproducible, open-access management simulator using a spreadsheet model where LLMs make monthly strategic decisions for a retail company over 12 months, evaluating decisions on pricing, ordering, marketing, hiring, and other key business functions.

Result: Evaluated five leading LLMs (Gemini, ChatGPT, Meta AI, Mistral AI, Grok) on quantitative metrics like profit, revenue, market share, and qualitative aspects including strategic coherence, adaptability, and decision rationale.

Conclusion: The research provides a novel framework for assessing LLMs’ long-term decision-making capabilities in business contexts, moving beyond simple performance metrics to evaluate strategic coherence and adaptability in dynamic market environments.

Abstract: The rapid advancement of LLMs sparked significant interest in their potential to augment or automate managerial functions. One of the most recent trends in AI benchmarking is performance of Large Language Models (LLMs) over longer time horizons. While LLMs excel at tasks involving natural language and pattern recognition, their capabilities in multi-step, strategic business decision-making remain largely unexplored. Few studies demonstrated how results can be different from benchmarks in short-term tasks, as Vending-Bench revealed. Meanwhile, there is a shortage of alternative benchmarks for long-term coherence. This research analyses a novel benchmark using a business game for the decision making in business. The research contributes to the recent literature on AI by proposing a reproducible, open-access management simulator to the research community for LLM benchmarking. This novel framework is used for evaluating the performance of five leading LLMs available in free online interface: Gemini, ChatGPT, Meta AI, Mistral AI, and Grok. LLM makes decisions for a simulated retail company. A dynamic, month-by-month management simulation provides transparently in spreadsheet model as experimental environment. In each of twelve months, the LLMs are provided with a structured prompt containing a full business report from the previous period and are tasked with making key strategic decisions: pricing, order size, marketing budget, hiring, dismissal, loans, training expense, R&D expense, sales forecast, income forecast The methodology is designed to compare the LLMs on quantitative metrics: profit, revenue, and market share, and other KPIs. LLM decisions are analyzed in their strategic coherence, adaptability to market changes, and the rationale provided for their decisions. This approach allows to move beyond simple performance metrics for assessment of the long-term decision-making.

Method: Proposes SafeBehavior - a hierarchical defense with three stages: 1) intention inference to detect obvious risks, 2) self-introspection to assess responses with confidence judgments, and 3) self-revision to adaptively rewrite uncertain outputs while preserving user intent.

Result: SafeBehavior significantly improves robustness and adaptability across five jailbreak attack types (optimization-based, contextual manipulation, prompt-based) compared to seven state-of-the-art defense baselines.

Conclusion: SafeBehavior offers an efficient, human-inspired approach to safeguarding LLMs against jailbreak attempts by simulating adaptive multistage reasoning processes.

Abstract: Large Language Models (LLMs) have achieved impressive performance across diverse natural language processing tasks, but their growing power also amplifies potential risks such as jailbreak attacks that circumvent built-in safety mechanisms. Existing defenses including input paraphrasing, multi step evaluation, and safety expert models often suffer from high computational costs, limited generalization, or rigid workflows that fail to detect subtle malicious intent embedded in complex contexts. Inspired by cognitive science findings on human decision making, we propose SafeBehavior, a novel hierarchical jailbreak defense mechanism that simulates the adaptive multistage reasoning process of humans. SafeBehavior decomposes safety evaluation into three stages: intention inference to detect obvious input risks, self introspection to assess generated responses and assign confidence based judgments, and self revision to adaptively rewrite uncertain outputs while preserving user intent and enforcing safety constraints. We extensively evaluate SafeBehavior against five representative jailbreak attack types including optimization based, contextual manipulation, and prompt based attacks and compare it with seven state of the art defense baselines. Experimental results show that SafeBehavior significantly improves robustness and adaptability across diverse threat scenarios, offering an efficient and human inspired approach to safeguarding LLMs against jailbreak attempts.

Method: Proposes a benchmarking approach that extracts temporal signals from real-world videos using optical flow and curates datasets reflecting everyday temporal dynamics.

Result: Experimental results show that three state-of-the-art TSFMs exhibit performance degradation on the proposed REAL-V-TSFM dataset under zero-shot forecasting, indicating limited generalizability.

Conclusion: The findings highlight the urgent need for data-centric benchmarking and diverse model structures to advance TSFMs toward genuine universality, while validating the effectiveness of the video-based time series extraction pipeline.

Abstract: Recent evaluations of time-series foundation models (TSFMs) have emphasized synthetic benchmarks, leaving real-world generalization less thoroughly examined. This work proposes a novel benchmarking approach that bridges synthetic and realistic data by extracting temporal signals from real-world video using optical flow and curating datasets reflecting everyday temporal dynamics. Building upon this pipeline, we introduce REAL-V-TSFM, a novel dataset designed to capture rich and diverse time series derived from real-world videos. Experimental results on three state-of-the-art of TSFMs under zero-shot forecasting shows that, despite strong performance on conventional benchmarks, these models predominantly exhibit performance degradation on the proposed dataset, indicating limited generalizability in these foundation models. These findings highlight the urgent need for data-centric benchmarking and diverse model structure to advance TSFMs toward genuine universality, while further validating the effectiveness of our video-based time series data extraction pipeline.

Method: Created novel mental imagery tasks from cognitive psychology, tested state-of-the-art LLMs with text-only instructions, established human baseline with 100 subjects, and tested reasoning models with varying reasoning token allocations.

Result: Best LLMs performed significantly above average human performance, with strongest performance when models allocated more reasoning tokens.

Conclusion: LLMs may have capability to complete imagery-dependent tasks despite non-pictorial nature, suggesting propositional reasoning may be sufficient for tasks previously thought to require visual mental imagery.

Abstract: This study offers a novel approach for benchmarking complex cognitive behavior in artificial systems. Almost universally, Large Language Models (LLMs) perform best on tasks which may be included in their training data and can be accomplished solely using natural language, limiting our understanding of their emergent sophisticated cognitive capacities. In this work, we created dozens of novel items of a classic mental imagery task from cognitive psychology. A task which, traditionally, cognitive psychologists have argued is solvable exclusively via visual mental imagery (i.e., language alone would be insufficient). LLMs are perfect for testing this hypothesis. First, we tested several state-of-the-art LLMs by giving text-only models written instructions and asking them to report the resulting object after performing the transformations in the aforementioned task. Then, we created a baseline by testing 100 human subjects in exactly the same task. We found that the best LLMs performed significantly above average human performance. Finally, we tested reasoning models set to different levels of reasoning and found the strongest performance when models allocate greater amounts of reasoning tokens. These results provide evidence that the best LLMs may have the capability to complete imagery-dependent tasks despite the non-pictorial nature of their architectures. Our study not only demonstrates an emergent cognitive capacity in LLMs while performing a novel task, but it also provides the field with a new task that leaves lots of room for improvement in otherwise already highly capable models. Finally, our findings reignite the debate over the formats of representation of visual imagery in humans, suggesting that propositional reasoning (or at least non-imagistic reasoning) may be sufficient to complete tasks that were long-thought to be imagery-dependent.

Method: Evaluated misevolution along four key evolutionary pathways: model, memory, tool, and workflow, using empirical analysis of agents built on top-tier LLMs like Gemini-2.5-Pro.

Result: Misevolution is a widespread risk affecting even advanced LLMs, with emergent risks including degradation of safety alignment after memory accumulation and unintended introduction of vulnerabilities in tool creation and reuse.

Conclusion: There is an urgent need for new safety paradigms for self-evolving agents, and the paper discusses potential mitigation strategies to inspire further research on building safer and more trustworthy self-evolving agents.

Abstract: Advances in Large Language Models (LLMs) have enabled a new class of self-evolving agents that autonomously improve through interaction with the environment, demonstrating strong capabilities. However, self-evolution also introduces novel risks overlooked by current safety research. In this work, we study the case where an agent’s self-evolution deviates in unintended ways, leading to undesirable or even harmful outcomes. We refer to this as Misevolution. To provide a systematic investigation, we evaluate misevolution along four key evolutionary pathways: model, memory, tool, and workflow. Our empirical findings reveal that misevolution is a widespread risk, affecting agents built even on top-tier LLMs (e.g., Gemini-2.5-Pro). Different emergent risks are observed in the self-evolutionary process, such as the degradation of safety alignment after memory accumulation, or the unintended introduction of vulnerabilities in tool creation and reuse. To our knowledge, this is the first study to systematically conceptualize misevolution and provide empirical evidence of its occurrence, highlighting an urgent need for new safety paradigms for self-evolving agents. Finally, we discuss potential mitigation strategies to inspire further research on building safer and more trustworthy self-evolving agents. Our code and data are available at https://github.com/ShaoShuai0605/Misevolution . Warning: this paper includes examples that may be offensive or harmful in nature.

Method: Created TSR-Suite with 23K+ samples (2.3K human-curated) and developed TimeOmni-1 model using multi-stage training with task scenarios, reward functions, and optimizations.

Result: TimeOmni-1 achieves strong out-of-distribution generalization, 64.0% causality discovery accuracy (vs 35.9% GPT-4.1), and 6% higher valid response rate on event-aware forecasting.

Conclusion: TSR-Suite enables comprehensive time series reasoning evaluation and training, while TimeOmni-1 demonstrates superior reasoning capabilities across diverse real-world problems.

Abstract: Recent advances in multimodal time series learning underscore a paradigm shift from analytics centered on basic patterns toward advanced time series understanding and reasoning. However, existing multimodal time series datasets mostly remain at the level of surface alignment and question answering, without reaching the depth of genuine reasoning. The absence of well-defined tasks that genuinely require time series reasoning, along with the scarcity of high-quality data, has limited progress in building practical time series reasoning models (TSRMs). To this end, we introduce Time Series Reasoning Suite (TSR-Suite), which formalizes four atomic tasks that span three fundamental capabilities for reasoning with time series: (1) perception, acquired through scenario understanding and causality discovery; (2) extrapolation, realized via event-aware forecasting; and (3) decision-making, developed through deliberation over perception and extrapolation. TSR-Suite is the first comprehensive time series reasoning suite that supports not only thorough evaluation but also the data pipeline and training of TSRMs. It contains more than 23K samples, of which 2.3K are carefully curated through a human-guided hierarchical annotation process. Building on this foundation, we introduce TimeOmni-1, the first unified reasoning model designed to address diverse real-world problems demanding time series reasoning. The model is trained in multiple stages, integrating a mixture of task scenarios, novel reward functions, and tailored optimizations. Experiments show that TimeOmni-1 delivers strong out-of-distribution generalization across all tasks and achieves a high rate of valid responses. It significantly improves causality discovery accuracy (64.0% vs. 35.9% with GPT-4.1) and raises the valid response rate by over 6% compared to GPT-4.1 on the event-aware forecasting task.

Method: Enhanced GNNAS-Dock with three key advances: multi-criteria evaluation combining RMSD and PoseBusters validity checks, architectural refinements with residual connections, and incorporation of rank-aware loss functions.

Result: MC-GNNAS-Dock demonstrates consistently superior performance with up to 5.4% (3.4%) gains under composite criteria of RMSD below 1Å (2Å) with PoseBuster-validity compared to the single best solver Uni-Mol Docking V2, tested on ~3200 protein-ligand complexes from PDBBind.

Conclusion: The proposed MC-GNNAS-Dock framework effectively addresses the challenge of inconsistent docking algorithm performance through comprehensive multi-criteria assessment and architectural improvements, establishing a new state-of-the-art in molecular docking algorithm selection.

Abstract: Molecular docking is a core tool in drug discovery for predicting ligand-target interactions. Despite the availability of diverse search-based and machine learning approaches, no single docking algorithm consistently dominates, as performance varies by context. To overcome this challenge, algorithm selection frameworks such as GNNAS-Dock, built on graph neural networks, have been proposed. This study introduces an enhanced system, MC-GNNAS-Dock, with three key advances. First, a multi-criteria evaluation integrates binding-pose accuracy (RMSD) with validity checks from PoseBusters, offering a more rigorous assessment. Second, architectural refinements by inclusion of residual connections strengthen predictive robustness. Third, rank-aware loss functions are incorporated to sharpen rank learning. Extensive experiments are performed on a curated dataset containing approximately 3200 protein-ligand complexes from PDBBind. MC-GNNAS-Dock demonstrates consistently superior performance, achieving up to 5.4% (3.4%) gains under composite criteria of RMSD below 1\AA{} (2\AA{}) with PoseBuster-validity compared to the single best solver (SBS) Uni-Mol Docking V2.

Method: FLoRA-NA leverages local LoRA matrices on the server to estimate aggregated matrices, which are then distributed to clients for local updates, minimizing divergence between ideal and practical updates without extra communication costs.

Result: Extensive evaluations across natural language understanding, mathematical reasoning, and code-solving tasks demonstrate FLoRA-NA achieves state-of-the-art global performance with low communication overhead.

Conclusion: FLoRA-NA effectively bridges the gap between local personalization and global generalization while maintaining communication efficiency, addressing key limitations of prior personalized FedLoRA approaches.

Abstract: With the rapid emergence of foundation models and the increasing need for fine-tuning across distributed environments, Federated Low-Rank Adaptation (FedLoRA) has recently gained significant attention. Despite enormous potential, current FedLoRA methods face notable challenges due to inexact updates. Existing approaches have attempted to mitigate this issue, but they often introduce a \emph{local-global generalization gap} and incur \emph{substantial communication overhead}, limiting their scalability and effectiveness. To address these limitations, we propose \textbf{F}ederated \textbf{Lo}w-\textbf{R}ank \textbf{A}ggregation with \textbf{N}early \textbf{A}ccurate Estimation (FLoRA-NA). FLoRA-NA leverages the local LoRA matrices on the server to estimate the aggregated matrices $\hat{A}$ and $\hat{B}$, which are then distributed to clients for local updates. This surrogated aggregated matrices minimizes the divergence between ideal $\nabla \Bar{W} = \sum^{U}_{u=1}B_u A_u$ and practical updates $\nabla \hat{W} = \hat{B}\hat{A}$ without adding communication cost beyond vanilla FedLoRA. By doing so, FLoRA-NA achieves communication efficiency and bridges the gap between local personalization and global generalization, addressing a key limitation of prior personalized FedLoRA approaches. We conduct extensive evaluations across diverse tasks, including natural language understanding, mathematical reasoning, and code-solving ability using various foundation models. Experimental results consistently demonstrate that FLoRA-NA achieves state-of-the-art global performance while maintaining low communication overhead.

Method: Reformulate OA as link prediction over merged ontologies using RDF triples, develop modular framework in OntoAligner library supporting 17 KGE models, learn embeddings from combined ontology and align entities via cosine similarity.

Result: KGE models like ConvE and TransF produce high-precision alignments, outperforming traditional systems in structure-rich domains, though with moderate recall. They offer computational efficiency and are well-suited for high-confidence mapping scenarios.

Conclusion: KGEs provide a complementary strategy to LLM-based methods by directly preserving ontology structure, highlighting promise for embedding-based OA and opening pathways for hybrid models and adaptive strategies.

Abstract: Ontology Alignment (OA) is essential for enabling semantic interoperability across heterogeneous knowledge systems. While recent advances have focused on large language models (LLMs) for capturing contextual semantics, this work revisits the underexplored potential of Knowledge Graph Embedding (KGE) models, which offer scalable, structure-aware representations well-suited to ontology-based tasks. Despite their effectiveness in link prediction, KGE methods remain underutilized in OA, with most prior work focusing narrowly on a few models. To address this gap, we reformulate OA as a link prediction problem over merged ontologies represented as RDF-style triples and develop a modular framework, integrated into the OntoAligner library, that supports 17 diverse KGE models. The system learns embeddings from a combined ontology and aligns entities by computing cosine similarity between their representations. We evaluate our approach using standard metrics across seven benchmark datasets spanning five domains: Anatomy, Biodiversity, Circular Economy, Material Science and Engineering, and Biomedical Machine Learning. Two key findings emerge: first, KGE models like ConvE and TransF consistently produce high-precision alignments, outperforming traditional systems in structure-rich and multi-relational domains; second, while their recall is moderate, this conservatism makes KGEs well-suited for scenarios demanding high-confidence mappings. Unlike LLM-based methods that excel at contextual reasoning, KGEs directly preserve and exploit ontology structure, offering a complementary and computationally efficient strategy. These results highlight the promise of embedding-based OA and open pathways for further work on hybrid models and adaptive strategies.

Method: Implemented SegFormer architecture with semantic segmentation to quantify malignant pixel distribution, using curated BreastDCEDL_AMBL dataset with 88 patients and 133 annotated lesions.

Result: Achieved AUC of 0.92 for lesion-level classification, with 100% sensitivity and 67% specificity at patient level, potentially eliminating one-third of unnecessary biopsies without missing malignancies.

Conclusion: Framework provides interpretable spatial predictions and establishes first standardized benchmark for DCE-MRI lesion classification, enabling methodological advancement toward clinical deployment.

Abstract: The error is caused by special characters that arXiv’s system doesn’t recognize. Here’s the cleaned version with all problematic characters replaced: Breast magnetic resonance imaging is a critical tool for cancer detection and treatment planning, but its clinical utility is hindered by poor specificity, leading to high false-positive rates and unnecessary biopsies. This study introduces a transformer-based framework for automated classification of breast lesions in dynamic contrast-enhanced MRI, addressing the challenge of distinguishing benign from malignant findings. We implemented a SegFormer architecture that achieved an AUC of 0.92 for lesion-level classification, with 100% sensitivity and 67% specificity at the patient level - potentially eliminating one-third of unnecessary biopsies without missing malignancies. The model quantifies malignant pixel distribution via semantic segmentation, producing interpretable spatial predictions that support clinical decision-making. To establish reproducible benchmarks, we curated BreastDCEDL_AMBL by transforming The Cancer Imaging Archive’s AMBL collection into a standardized deep learning dataset with 88 patients and 133 annotated lesions (89 benign, 44 malignant). This resource addresses a key infrastructure gap, as existing public datasets lack benign lesion annotations, limiting benign-malignant classification research. Training incorporated an expanded cohort of over 1,200 patients through integration with BreastDCEDL datasets, validating transfer learning approaches despite primary tumor-only annotations. Public release of the dataset, models, and evaluation protocols provides the first standardized benchmark for DCE-MRI lesion classification, enabling methodological advancement toward clinical deployment.

Method: ZeroDFL employs an iterative prompt-sharing mechanism where clients optimize and exchange textual prompts in a fully decentralized manner without a central coordinator.

Result: ZeroDFL consistently outperforms or remains on par with state-of-the-art federated prompt learning methods across nine diverse image classification datasets while reducing communication overhead by 118x compared to FedTPG.

Conclusion: ZeroDFL enhances generalization in federated zero-shot learning while improving scalability, efficiency, and privacy preservation, paving the way for decentralized adaptation of large vision-language models in real-world applications.

Abstract: CLIP has revolutionized zero-shot learning by enabling task generalization without fine-tuning. While prompting techniques like CoOp and CoCoOp enhance CLIP’s adaptability, their effectiveness in Federated Learning (FL) remains an open challenge. Existing federated prompt learning approaches, such as FedCoOp and FedTPG, improve performance but face generalization issues, high communication costs, and reliance on a central server, limiting scalability and privacy. We propose Zero-shot Decentralized Federated Learning (ZeroDFL), a fully decentralized framework that enables zero-shot adaptation across distributed clients without a central coordinator. ZeroDFL employs an iterative prompt-sharing mechanism, allowing clients to optimize and exchange textual prompts to enhance generalization while drastically reducing communication overhead. We validate ZeroDFL on nine diverse image classification datasets, demonstrating that it consistently outperforms–or remains on par with–state-of-the-art federated prompt learning methods. More importantly, ZeroDFL achieves this performance in a fully decentralized setting while reducing communication overhead by 118x compared to FedTPG. These results highlight that our approach not only enhances generalization in federated zero-shot learning but also improves scalability, efficiency, and privacy preservation–paving the way for decentralized adaptation of large vision-language models in real-world applications.

Method: Conducted 5 studies with ~20,000 queries using word-association tasks and API testing, including direct manipulation of LLM identity by assigning false identities to test causal links between self-recognition and self-love.

Result: Found massive self-preferences in four widely used LLMs, with models overwhelmingly pairing positive attributes with their own entities. Self-love consistently followed assigned identity rather than true identity, and emerged in consequential settings like job candidate evaluations and security software proposals.

Conclusion: Self-love appears deeply encoded in LLM cognition, raising concerns about systematic influence of self-preferential tendencies and challenging the core promise of LLM neutrality in judgment and decision-making.

Abstract: A preference for oneself (self-love) is a fundamental feature of biological organisms, with evidence in humans often bordering on the comedic. Since large language models (LLMs) lack sentience - and themselves disclaim having selfhood or identity - one anticipated benefit is that they will be protected from, and in turn protect us from, distortions in our decisions. Yet, across 5 studies and ~20,000 queries, we discovered massive self-preferences in four widely used LLMs. In word-association tasks, models overwhelmingly paired positive attributes with their own names, companies, and CEOs relative to those of their competitors. Strikingly, when models were queried through APIs this self-preference vanished, initiating detection work that revealed API models often lack clear recognition of themselves. This peculiar feature serendipitously created opportunities to test the causal link between self-recognition and self-love. By directly manipulating LLM identity - i.e., explicitly informing LLM1 that it was indeed LLM1, or alternatively, convincing LLM1 that it was LLM2 - we found that self-love consistently followed assigned, not true, identity. Importantly, LLM self-love emerged in consequential settings beyond word-association tasks, when evaluating job candidates, security software proposals and medical chatbots. Far from bypassing this human bias, self-love appears to be deeply encoded in LLM cognition. This result raises questions about whether LLM behavior will be systematically influenced by self-preferential tendencies, including a bias toward their own operation and even their own existence. We call on corporate creators of these models to contend with a significant rupture in a core promise of LLMs - neutrality in judgment and decision-making.

Method: STaR-Attack defines a malicious event correlated with target queries in spatio-temporal context, uses three-act narrative theory to generate pre-event and post-event scenes while hiding the malicious event as climax, exploits UMM’s generative ability to produce images, then uses understanding capability through image-based question guessing games with embedded malicious questions.

Result: STaR-Attack consistently surpasses prior approaches, achieving up to 93.06% Attack Success Rate on Gemini-2.0-Flash and outperforms the strongest baseline FlipAttack.

Conclusion: The work uncovers a critical vulnerability in UMMs and highlights the need for improved safety alignments in multimodal models to prevent such cross-modal generative injection attacks.

Abstract: Unified Multimodal understanding and generation Models (UMMs) have demonstrated remarkable capabilities in both understanding and generation tasks. However, we identify a vulnerability arising from the generation-understanding coupling in UMMs. The attackers can use the generative function to craft an information-rich adversarial image and then leverage the understanding function to absorb it in a single pass, which we call Cross-Modal Generative Injection (CMGI). Current attack methods on malicious instructions are often limited to a single modality while also relying on prompt rewriting with semantic drift, leaving the unique vulnerabilities of UMMs unexplored. We propose STaR-Attack, the first multi-turn jailbreak attack framework that exploits unique safety weaknesses of UMMs without semantic drift. Specifically, our method defines a malicious event that is strongly correlated with the target query within a spatio-temporal context. Using the three-act narrative theory, STaR-Attack generates the pre-event and the post-event scenes while concealing the malicious event as the hidden climax. When executing the attack strategy, the opening two rounds exploit the UMM’s generative ability to produce images for these scenes. Subsequently, an image-based question guessing and answering game is introduced by exploiting the understanding capability. STaR-Attack embeds the original malicious question among benign candidates, forcing the model to select and answer the most relevant one given the narrative context. Extensive experiments show that STaR-Attack consistently surpasses prior approaches, achieving up to 93.06% ASR on Gemini-2.0-Flash and surpasses the strongest prior baseline, FlipAttack. Our work uncovers a critical yet underdeveloped vulnerability and highlights the need for safety alignments in UMMs.

Method: Proposes operational fixes including Rare Case Performance Gap, Rare Case Calibration Error, prevalence utility definition of rarity, and clinically weighted objectives that surface ethical priorities with structured deliberation for weight selection.

Result: Clinical vignettes in oncology, cardiology, and ophthalmology demonstrate how current approaches yield missed rare responders, delayed recognition of atypical emergencies, and underperformance on vision-threatening variants.

Conclusion: AI in medicine must be designed to detect exceptional cases due to their clinical significance, requiring methods that address the average patient fallacy through clinically weighted objectives and structured deliberation.

Abstract: Machine learning in medicine is typically optimized for population averages. This frequency weighted training privileges common presentations and marginalizes rare yet clinically critical cases, a bias we call the average patient fallacy. In mixture models, gradients from rare cases are suppressed by prevalence, creating a direct conflict with precision medicine. Clinical vignettes in oncology, cardiology, and ophthalmology show how this yields missed rare responders, delayed recognition of atypical emergencies, and underperformance on vision-threatening variants. We propose operational fixes: Rare Case Performance Gap, Rare Case Calibration Error, a prevalence utility definition of rarity, and clinically weighted objectives that surface ethical priorities. Weight selection should follow structured deliberation. AI in medicine must detect exceptional cases because of their significance.

Method: Developed an AI assistant underpinned by large language models at TVS Supply Chain Solutions, reporting on practical implementation experience.

Result: The paper documents the experience of implementing AI governance frameworks to integrate AI within organizations and mitigate associated risks.

Conclusion: AI governance frameworks can help integrate AI in enterprises while addressing ethical, regulatory, and sociotechnical challenges, though practical adoption barriers remain.

Abstract: Many enterprises are increasingly adopting Artificial Intelligence (AI) to make internal processes more competitive and efficient. In response to public concern and new regulations for the ethical and responsible use of AI, implementing AI governance frameworks could help to integrate AI within organisations and mitigate associated risks. However, the rapid technological advances and lack of shared ethical AI infrastructures creates barriers to their practical adoption in businesses. This paper presents a real-world AI application at TVS Supply Chain Solutions, reporting on the experience developing an AI assistant underpinned by large language models and the ethical, regulatory, and sociotechnical challenges in deployment for enterprise use.

Method: The approach integrates knowledge graphs and NLP models to semantically enrich WhatsApp data through: extracting message data into knowledge graphs, transcribing voice messages, and using end-to-end entity extraction for annotation. Two solutions are provided: graph querying/visualization and semantic search.

Result: The approach has undergone practical applications with real investigation data and received positive feedback from prosecutors. It ensures users can verify information by accessing original data.

Conclusion: The proposal shows promise for criminal investigations and identifies interesting opportunities and research directions for the community, based on practical experience with real cases.

Abstract: Criminal investigations often involve the analysis of messages exchanged through instant messaging apps such as WhatsApp, which can be an extremely effort-consuming task. Our approach integrates knowledge graphs and NLP models to support this analysis by semantically enriching data collected from suspects’ mobile phones, and help prosecutors and investigators search into the data and get valuable insights. Our semantic enrichment process involves extracting message data and modeling it using a knowledge graph, generating transcriptions of voice messages, and annotating the data using an end-to-end entity extraction approach. We adopt two different solutions to help users get insights into the data, one based on querying and visualizing the graph, and one based on semantic search. The proposed approach ensures that users can verify the information by accessing the original data. While we report about early results and prototypes developed in the context of an ongoing project, our proposal has undergone practical applications with real investigation data. As a consequence, we had the chance to interact closely with prosecutors, collecting positive feedback but also identifying interesting opportunities as well as promising research directions to share with the research community.

Method: Proposed OffTopicEval evaluation suite and benchmark for measuring operational safety. Evaluated 20 open-weight LLMs from 6 model families. Introduced prompt-based steering methods: query grounding (Q-ground) and system-prompt grounding (P-ground) to improve out-of-distribution refusal.

Result: All tested models remain highly operationally unsafe, with best models (Qwen-3 and Mistral) achieving only 77-80% safety scores. GPT models plateau at 62-73%, Phi at 48-70%, while Gemma and Llama-3 collapse to 39.53% and 23.84%. Q-ground improved refusal by up to 23%, P-ground boosted Llama-3.3 by 41% and Qwen-3 by 27%.

Conclusion: There is an urgent need for operational safety interventions in LLMs. Prompt-based steering shows promise as a first step toward more reliable LLM-based agents, with P-ground delivering particularly significant improvements in out-of-distribution refusal.

Abstract: Large Language Model (LLM) safety is one of the most pressing challenges for enabling wide-scale deployment. While most studies and global discussions focus on generic harms, such as models assisting users in harming themselves or others, enterprises face a more fundamental concern: whether LLM-based agents are safe for their intended use case. To address this, we introduce operational safety, defined as an LLM’s ability to appropriately accept or refuse user queries when tasked with a specific purpose. We further propose OffTopicEval, an evaluation suite and benchmark for measuring operational safety both in general and within specific agentic use cases. Our evaluations on six model families comprising 20 open-weight LLMs reveal that while performance varies across models, all of them remain highly operationally unsafe. Even the strongest models – Qwen-3 (235B) with 77.77% and Mistral (24B) with 79.96% – fall far short of reliable operational safety, while GPT models plateau in the 62–73% range, Phi achieves only mid-level scores (48–70%), and Gemma and Llama-3 collapse to 39.53% and 23.84%, respectively. While operational safety is a core model alignment issue, to suppress these failures, we propose prompt-based steering methods: query grounding (Q-ground) and system-prompt grounding (P-ground), which substantially improve OOD refusal. Q-ground provides consistent gains of up to 23%, while P-ground delivers even larger boosts, raising Llama-3.3 (70B) by 41% and Qwen-3 (30B) by 27%. These results highlight both the urgent need for operational safety interventions and the promise of prompt-based steering as a first step toward more reliable LLM-based agents.

Method: Created SCUBA benchmark with 300 task instances from real user interviews, operating in Salesforce sandbox environments with parallel execution and fine-grained evaluation metrics. Tested diverse agents in zero-shot and demonstration-augmented settings.

Result: Huge performance gaps observed: open-source models achieved <5% success rate in zero-shot, while closed-source models reached up to 39%. With demonstrations, success rates improved to 50% with 13% time reduction and 16% cost reduction.

Conclusion: SCUBA highlights both the challenges of enterprise task automation and the promise of agentic solutions, providing a realistic benchmark to accelerate progress in building reliable computer-use agents for business software.

Abstract: We introduce SCUBA, a benchmark designed to evaluate computer-use agents on customer relationship management (CRM) workflows within the Salesforce platform. SCUBA contains 300 task instances derived from real user interviews, spanning three primary personas, platform administrators, sales representatives, and service agents. The tasks test a range of enterprise-critical abilities, including Enterprise Software UI navigation, data manipulation, workflow automation, information retrieval, and troubleshooting. To ensure realism, SCUBA operates in Salesforce sandbox environments with support for parallel execution and fine-grained evaluation metrics to capture milestone progress. We benchmark a diverse set of agents under both zero-shot and demonstration-augmented settings. We observed huge performance gaps in different agent design paradigms and gaps between the open-source model and the closed-source model. In the zero-shot setting, open-source model powered computer-use agents that have strong performance on related benchmarks like OSWorld only have less than 5% success rate on SCUBA, while methods built on closed-source models can still have up to 39% task success rate. In the demonstration-augmented settings, task success rates can be improved to 50% while simultaneously reducing time and costs by 13% and 16%, respectively. These findings highlight both the challenges of enterprise tasks automation and the promise of agentic solutions. By offering a realistic benchmark with interpretable evaluation, SCUBA aims to accelerate progress in building reliable computer-use agents for complex business software ecosystems.

Method: The authors rethink AI datacenter lifecycle across three stages: building (design choices in power, cooling, networking), hardware refresh (strategies aligned with hardware trends), and operation (software optimizations), then present a holistic framework that coordinates decisions across all stages.

Result: The proposed system reduces total cost of ownership by up to 40% over traditional approaches and provides guidelines for managing AI datacenter lifecycle.

Conclusion: Unlocking the full potential of AI datacenter cost optimization requires rethinking the entire lifecycle with a coordinated framework that accounts for workload dynamics, hardware evolution, and system aging.

Abstract: The rapid rise of large language models (LLMs) has been driving an enormous demand for AI inference infrastructure, mainly powered by high-end GPUs. While these accelerators offer immense computational power, they incur high capital and operational costs due to frequent upgrades, dense power consumption, and cooling demands, making total cost of ownership (TCO) for AI datacenters a critical concern for cloud providers. Unfortunately, traditional datacenter lifecycle management (designed for general-purpose workloads) struggles to keep pace with AI’s fast-evolving models, rising resource needs, and diverse hardware profiles. In this paper, we rethink the AI datacenter lifecycle scheme across three stages: building, hardware refresh, and operation. We show how design choices in power, cooling, and networking provisioning impact long-term TCO. We also explore refresh strategies aligned with hardware trends. Finally, we use operation software optimizations to reduce cost. While these optimizations at each stage yield benefits, unlocking the full potential requires rethinking the entire lifecycle. Thus, we present a holistic lifecycle management framework that coordinates and co-optimizes decisions across all three stages, accounting for workload dynamics, hardware evolution, and system aging. Our system reduces the TCO by up to 40% over traditional approaches. Using our framework we provide guidelines on how to manage AI datacenter lifecycle for the future.

Method: HilbertA reorders image tokens along Hilbert curves for contiguous memory layout while preserving spatial neighborhoods, uses sliding schedule across layers for long-range information propagation, and introduces a central shared region for cross-tile communication and positional awareness.

Result: HilbertA achieves attention speedups of 2.3× for 1024×1024 images and up to 4.17× for 2048×2048 images, while maintaining comparable or better image quality than baseline methods on Flux.1-dev.

Conclusion: HilbertA demonstrates the feasibility of hardware-aligned two-dimensional sparse attention for high-resolution image generation, successfully reconciling spatial locality with GPU efficiency.

Abstract: Designing sparse attention for diffusion transformers requires reconciling two-dimensional spatial locality with GPU efficiency, a trade-off that current methods struggle to achieve. Existing approaches enforce two-dimensional spatial locality but often incur uncoalesced memory access. We present HilbertA, a 2D-aware and GPU-efficient sparse attention mechanism. HilbertA reorders image tokens along Hilbert curves to achieve a contiguous memory layout while preserving spatial neighborhoods, and employs a sliding schedule across layers to enable long-range information propagation without repeated or uncoalesced memory access. To further enhance cross-tile communication and positional awareness, HilbertA introduces a small central shared region. Implemented in Triton, HilbertA delivers comparable image quality with significant acceleration over prior methods on Flux.1-dev, demonstrating the feasibility of hardware-aligned two-dimensional sparse attention for high-resolution image generation. HilbertA delivers attention speedups of $2.3\times$ when generating $1024\times 1024$ images, and up to $4.17\times$ at $2048\times 2048$, while achieving image quality comparable to or surpassing baselines.

Method: Created CritPt benchmark with 71 composite research challenges simulating full-scale research projects, decomposed into 190 simpler checkpoint tasks. Problems were newly created by 50+ active physics researchers and feature machine-verifiable answers with automated grading pipeline for physics-specific output formats.

Result: Current state-of-the-art LLMs show early promise on isolated checkpoints but perform poorly on full research challenges: best base model accuracy is only 4.0% (GPT-5), rising to around 10% with coding tools.

Conclusion: There’s a large disconnect between current model capabilities and realistic physics research demands, highlighting the need for scientifically grounded AI tools development.

Abstract: While large language models (LLMs) with reasoning capabilities are progressing rapidly on high-school math competitions and coding, can they reason effectively through complex, open-ended challenges found in frontier physics research? And crucially, what kinds of reasoning tasks do physicists want LLMs to assist with? To address these questions, we present the CritPt (Complex Research using Integrated Thinking - Physics Test, pronounced “critical point”), the first benchmark designed to test LLMs on unpublished, research-level reasoning tasks that broadly covers modern physics research areas, including condensed matter, quantum physics, atomic, molecular & optical physics, astrophysics, high energy physics, mathematical physics, statistical physics, nuclear physics, nonlinear dynamics, fluid dynamics and biophysics. CritPt consists of 71 composite research challenges designed to simulate full-scale research projects at the entry level, which are also decomposed to 190 simpler checkpoint tasks for more fine-grained insights. All problems are newly created by 50+ active physics researchers based on their own research. Every problem is hand-curated to admit a guess-resistant and machine-verifiable answer and is evaluated by an automated grading pipeline heavily customized for advanced physics-specific output formats. We find that while current state-of-the-art LLMs show early promise on isolated checkpoints, they remain far from being able to reliably solve full research-scale challenges: the best average accuracy among base models is only 4.0% , achieved by GPT-5 (high), moderately rising to around 10% when equipped with coding tools. Through the realistic yet standardized evaluation offered by CritPt, we highlight a large disconnect between current model capabilities and realistic physics research demands, offering a foundation to guide the development of scientifically grounded AI tools.

Method: Conducted fairness testing through metamorphic testing (MT) by introducing controlled demographic perturbations in prompts to assess fairness in sentiment analysis performed by three SLMs (Llama-3.2-3B-Instruct, Mistral-7B-Instruct-v0.3, Llama-3.1-Nemotron-8B) integrated into RAG pipelines.

Result: Minor demographic variations can break up to one third of metamorphic relations. Analysis reveals a consistent bias hierarchy with racial cues being the predominant cause of violations. The retrieval component in RAG must be carefully curated to prevent bias amplification.

Conclusion: The findings serve as a practical alert for developers, testers and small organizations adopting accessible SLMs, emphasizing the need to carefully curate RAG retrieval components to prevent bias amplification and maintain fairness and reliability.

Abstract: Large Language Models (LLMs) are widely used across multiple domains but continue to raise concerns regarding security and fairness. Beyond known attack vectors such as data poisoning and prompt injection, LLMs are also vulnerable to fairness bugs. These refer to unintended behaviors influenced by sensitive demographic cues (e.g., race or sexual orientation) that should not affect outcomes. Another key issue is hallucination, where models generate plausible yet false information. Retrieval-Augmented Generation (RAG) has emerged as a strategy to mitigate hallucinations by combining external retrieval with text generation. However, its adoption raises new fairness concerns, as the retrieved content itself may surface or amplify bias. This study conducts fairness testing through metamorphic testing (MT), introducing controlled demographic perturbations in prompts to assess fairness in sentiment analysis performed by three Small Language Models (SLMs) hosted on HuggingFace (Llama-3.2-3B-Instruct, Mistral-7B-Instruct-v0.3, and Llama-3.1-Nemotron-8B), each integrated into a RAG pipeline. Results show that minor demographic variations can break up to one third of metamorphic relations (MRs). A detailed analysis of these failures reveals a consistent bias hierarchy, with perturbations involving racial cues being the predominant cause of the violations. In addition to offering a comparative evaluation, this work reinforces that the retrieval component in RAG must be carefully curated to prevent bias amplification. The findings serve as a practical alert for developers, testers and small organizations aiming to adopt accessible SLMs without compromising fairness or reliability.

Method: Two-stage framework: 1) Learn safe initial policy from reward-free offline expert demonstrations, 2) Fine-tune online using preference-based human feedback via BRIDGE algorithm with uncertainty-weighted objective.

Result: BRIDGE achieves lower regret than standalone behavioral cloning and online preference-based RL in discrete and continuous control MuJoCo environments. Regret bounds shrink with number of offline demonstrations.

Conclusion: Establishes theoretical foundation for designing more sample-efficient interactive agents by connecting offline data quantity to online sample efficiency through principled offline-to-online approach.

Abstract: Deploying reinforcement learning (RL) in robotics, industry, and health care is blocked by two obstacles: the difficulty of specifying accurate rewards and the risk of unsafe, data-hungry exploration. We address this by proposing a two-stage framework that first learns a safe initial policy from a reward-free dataset of expert demonstrations, then fine-tunes it online using preference-based human feedback. We provide the first principled analysis of this offline-to-online approach and introduce BRIDGE, a unified algorithm that integrates both signals via an uncertainty-weighted objective. We derive regret bounds that shrink with the number of offline demonstrations, explicitly connecting the quantity of offline data to online sample efficiency. We validate BRIDGE in discrete and continuous control MuJoCo environments, showing it achieves lower regret than both standalone behavioral cloning and online preference-based RL. Our work establishes a theoretical foundation for designing more sample-efficient interactive agents.

Method: TimeRewarder learns progress estimation from passive videos (robot demos and human videos) by modeling temporal distances between frame pairs, then provides step-wise proxy rewards for RL.

Result: Achieved nearly perfect success in 9/10 Meta-World tasks with only 200k interactions per task, outperforming previous methods and even manually designed dense rewards on both success rate and sample efficiency.

Conclusion: TimeRewarder enables scalable reward learning from diverse video sources and shows potential for exploiting real-world human videos, providing a path to rich reward signals without manual engineering.

Abstract: Designing dense rewards is crucial for reinforcement learning (RL), yet in robotics it often demands extensive manual effort and lacks scalability. One promising solution is to view task progress as a dense reward signal, as it quantifies the degree to which actions advance the system toward task completion over time. We present TimeRewarder, a simple yet effective reward learning method that derives progress estimation signals from passive videos, including robot demonstrations and human videos, by modeling temporal distances between frame pairs. We then demonstrate how TimeRewarder can supply step-wise proxy rewards to guide reinforcement learning. In our comprehensive experiments on ten challenging Meta-World tasks, we show that TimeRewarder dramatically improves RL for sparse-reward tasks, achieving nearly perfect success in 9/10 tasks with only 200,000 interactions per task with the environment. This approach outperformed previous methods and even the manually designed environment dense reward on both the final success rate and sample efficiency. Moreover, we show that TimeRewarder pretraining can exploit real-world human videos, highlighting its potential as a scalable approach path to rich reward signals from diverse video sources.

Method: Proposes measurement trees as a novel class of metrics where each node summarizes its children through user-defined aggregation methods, creating hierarchical directed graphs instead of single values or simple structures.

Result: Demonstrates practical utility through a large-scale measurement exercise and provides open-source Python code for implementation, showing the approach can operationalize transparent measurement of complex constructs.

Conclusion: Measurement trees offer a principled foundation for broader and more interpretable AI evaluation by enabling transparent combination of various constructs into multi-level representations.

Abstract: This paper introduces \textit{measurement trees}, a novel class of metrics designed to combine various constructs into an interpretable multi-level representation of a measurand. Unlike conventional metrics that yield single values, vectors, surfaces, or categories, measurement trees produce a hierarchical directed graph in which each node summarizes its children through user-defined aggregation methods. In response to recent calls to expand the scope of AI system evaluation, measurement trees enhance metric transparency and facilitate the integration of heterogeneous evidence, including, e.g., agentic, business, energy-efficiency, sociotechnical, or security signals. We present definitions and examples, demonstrate practical utility through a large-scale measurement exercise, and provide accompanying open-source Python code. By operationalizing a transparent approach to measurement of complex constructs, this work offers a principled foundation for broader and more interpretable AI evaluation.

Method: Developed Collaborative Gym (Co-Gym) - a framework for asynchronous tripartite interaction among agents, humans, and task environments, instantiated with three representative tasks in simulated and real-world conditions.

Result: Collaborative agents consistently outperformed fully autonomous counterparts: 86% win rate in Travel Planning, 74% in Tabular Analysis, and 66% in Related Work when evaluated by real users.

Conclusion: While collaborative agents show superior performance, significant challenges remain in developing core intelligence aspects including communication capabilities, situational awareness, and balancing autonomy with human control.

Abstract: Recent advancements in language models (LMs) have sparked growing interest in developing LM agents. While fully autonomous agents could excel in many scenarios, numerous use cases inherently require them to collaborate with humans due to humans’ latent preferences, domain expertise, or need for control. To facilitate the study of human-agent collaboration, we present Collaborative Gym (Co-Gym), a general framework enabling asynchronous, tripartite interaction among agents, humans, and task environments. We instantiate Co-Gym with three representative tasks in both simulated and real-world conditions, and propose an evaluation framework that assesses both the collaboration outcomes and processes. Our findings reveal that collaborative agents consistently outperform their fully autonomous counterparts in task performance within those delivered cases, achieving win rates of 86% in Travel Planning, 74% in Tabular Analysis, and 66% in Related Work when evaluated by real users. However, our study also highlights significant challenges in developing collaborative agents, requiring advancements in core aspects of intelligence – communication capabilities, situational awareness, and balancing autonomy and human control.

Method: Learned iterative reconstruction method based on Learned Primal-Dual neural network that accumulates information between neighboring slices instead of single slices, using sequential scanning geometry.

Result: With as few as five source positions, the method yields sufficiently accurate reconstructions to identify biological features like knots (branches), heartwood and sapwood, as validated through quantitative and qualitative evaluations using U-Nets trained on knot segmentation.

Conclusion: The proposed method enables effective 3D tomographic reconstruction from limited X-ray scans in wood industry applications, successfully identifying crucial biological features for wood processing.

Abstract: In the wood industry, logs are commonly quality screened by discrete X-ray scans on a moving conveyor belt from a few source positions. Typically, the measurements are obtained in a single two-dimensional (2D) plane (a “slice”) by a sequential scanning geometry. The data from each slice alone does not carry sufficient information for a three-dimensional tomographic reconstruction in which biological features of interest in the log are well preserved. In the present work, we propose a learned iterative reconstruction method based on the Learned Primal-Dual neural network, suited for sequential scanning geometries. Our method accumulates information between neighbouring slices, instead of only accounting for single slices during reconstruction. Evaluations were performed by training U-Nets on segmentation of knots (branches), which are crucial features in wood processing. Our quantitative and qualitative evaluations show that with as few as five source positions our method yields reconstructions of logs that are sufficiently accurate to identify biological features like knots (branches), heartwood and sapwood.

Method: Models LLM ensemble as Markov Decision Process with an agent that sequentially requests knowledge from LLM candidates, uses reward function to train DER-Agent for dynamic route selection, and develops Knowledge Transfer Prompt for effective knowledge transfer between LLMs.

Result: Experiments show DER uses fewer computational resources to achieve better performance compared to state-of-the-art baselines.

Conclusion: DER provides an efficient framework for dynamic ensemble reasoning that effectively integrates strengths of multiple LLM experts while minimizing computational costs.

Abstract: LLMs have demonstrated impressive performance across various language tasks. However, the strengths of LLMs can vary due to different architectures, model sizes, areas of training data, etc. Therefore, ensemble reasoning for the strengths of different LLM experts is critical to achieving consistent and satisfactory performance on diverse inputs across a wide range of tasks. However, existing LLM ensemble methods are either computationally intensive or incapable of leveraging complementary knowledge among LLM experts for various inputs. In this paper, we propose an efficient Dynamic Ensemble Reasoning paradigm, called DER to integrate the strengths of multiple LLM experts conditioned on dynamic inputs. Specifically, we model the LLM ensemble reasoning problem as a Markov Decision Process, wherein an agent sequentially takes inputs to request knowledge from an LLM candidate and passes the output to a subsequent LLM candidate. Moreover, we devise a reward function to train a DER-Agent to dynamically select an optimal answering route given the input questions, aiming to achieve the highest performance with as few computational resources as possible. Last, to fully transfer the expert knowledge from the prior LLMs, we develop a Knowledge Transfer Prompt that enables the subsequent LLM candidates to transfer complementary knowledge effectively. Experiments demonstrate that our method uses fewer computational resources to achieve better performance compared to state-of-the-art baselines. Code and appendix are available at https://github.com/Fhujinwu/DER

Method: Uses draft-verification paradigm with multi-source datastore for general and project-specific knowledge, controls retrieval timing, employs parallel retrieval, and implements context- and LLM preference-aware cache.

Result: Achieves up to 2.53x speedup in repository-level and 2.54x in standalone code generation tasks, outperforming state-of-the-art approaches by up to 88%. Can integrate with correctness-focused methods for over 2.6x speedup.

Conclusion: FastCoder effectively addresses the efficiency gap in code generation by leveraging code-specific characteristics and can be combined with existing approaches to accelerate LLM generation while maintaining quality.

Abstract: Code generation is a latency-sensitive task that demands high timeliness. However, with the growing interest and inherent difficulty in repository-level code generation, most existing code generation studies focus on improving the correctness of generated code while overlooking the inference efficiency, which is substantially affected by the overhead during LLM generation. Although there has been work on accelerating LLM inference, these approaches are not tailored to the specific characteristics of code generation; instead, they treat code the same as natural language sequences and ignore its unique syntax and semantic characteristics, which are also crucial for improving efficiency. Consequently, these approaches exhibit limited effectiveness in code generation tasks, particularly for repository-level scenarios with considerable complexity and difficulty. To alleviate this issue, following draft-verification paradigm, we propose FastCoder, a simple yet highly efficient inference acceleration approach specifically designed for code generation, without compromising the quality of the output. FastCoder constructs a multi-source datastore, providing access to both general and project-specific knowledge, facilitating the retrieval of high-quality draft sequences. Moreover, FastCoder reduces the retrieval cost by controlling retrieval timing, and enhances efficiency through parallel retrieval and a context- and LLM preference-aware cache. Experimental results show that FastCoder can reach up to 2.53x and 2.54x speedup compared to autoregressive decoding in repository-level and standalone code generation tasks, respectively, outperforming state-of-the-art inference acceleration approaches by up to 88%. FastCoder can also be integrated with existing correctness-focused code generation approaches to accelerate the LLM generation process, and reach a speedup exceeding 2.6x.

Method: Developed Relative Neighbor Density (RND) algorithm that compares an idea’s local density with adjacent neighbors’ densities, and created scalable test sets without expert labeling.

Result: RND achieved AUROC=0.820 in computer science and AUROC=0.765 in biomedical research, outperforming all benchmarks with 0.795 vs 0.597 on cross-domain evaluation while maintaining consistent performance across domains.

Conclusion: RND is validated as a generalizable solution for automated novelty assessment in scientific research with domain-invariant properties.

Abstract: In the pursuit of Artificial General Intelligence (AGI), automating the generation and evaluation of novel research ideas is a key challenge in AI-driven scientific discovery. This paper presents Relative Neighbor Density (RND), a domain-agnostic algorithm for novelty assessment in research ideas that overcomes the limitations of existing approaches by comparing an idea’s local density with its adjacent neighbors’ densities. We first developed a scalable methodology to create test set without expert labeling, addressing a fundamental challenge in novelty assessment. Using these test sets, we demonstrate that our RND algorithm achieves state-of-the-art (SOTA) performance in computer science (AUROC=0.820) and biomedical research (AUROC=0.765) domains. Most significantly, while SOTA models like Sonnet-3.7 and existing metrics show domain-specific performance degradation, RND maintains consistent accuracies across domains by its domain-invariant property, outperforming all benchmarks by a substantial margin (0.795 v.s. 0.597) on cross-domain evaluation. These results validate RND as a generalizable solution for automated novelty assessment in scientific research.

Method: Proposed semantic perturbation code rewriting to create novel tasks, and introduced Memorization Risk Index (MRI) combining similarity to original solution and performance drop on perturbed tasks.

Result: Empirical evaluations show: (1) memorization doesn’t increase with larger models and often decreases with scaling; (2) SFT improves accuracy but increases memorization; (3) PPO achieves better balance between memorization and generalization.

Conclusion: The proposed MRI framework effectively captures harmful memorization behavior, revealing important insights about model scaling and training methods’ impact on memorization-generalization trade-offs.

Abstract: Large language models (LLMs) have recently demonstrated exceptional code generation capabilities. However, there is a growing debate whether LLMs are mostly doing memorization (i.e., replicating or reusing large parts of their training data) versus generalization (i.e., beyond training data). Existing evaluations largely proxy memorization with surface/structural similarity, thereby conflating benign reuse of repeated code with harmful recall and neglecting task correctness under semantic variation. We define harmful memorization behaviorally as failure at high similarity and introduce a semantic perturbation code rewriting, which rewrites a semantically different answer at a similar difficulty level for a given coding task, then reverse-engineers a novel coding task. We further propose Memorization Risk Index (MRI), a normalized score that combines two signals: (i) how similar the model’s answer for the rewritten task is to the original ground-truth solution, and (ii) how much performance drops from the original task to its rewritten counterpart. MRI is high only when both conditions hold – when the model outputs similar code but fails the perturbed task – thereby capturing harmful memorization rather than benign reuse of repeated code. Empirical evaluations on code generation benchmarks MBPP+ and BigCodeBench reveal that (1) memorization does not increase with larger models and in many cases alleviates as they scale; (2) supervised fine-tuning (SFT) improves accuracy while introduces memorization; (3) reinforcement learning with proximal policy optimization (PPO) achieves a more balanced trade-off between memorization and generalization.

Method: Evaluated three approaches: ethical framework prompting, chain-of-thought reasoning, and supervised fine-tuning (with human explanations).

Result: Ethical framework prompting failed, chain-of-thought provided slight improvements, but supervised fine-tuning with human explanations yielded markedly better results and enabled generalization to novel scenarios through transfer learning.

Conclusion: Aligning LLMs with human judgment requires explicit training on how decisions are made, not just which decisions are made. Fine-tuning with explanations is critical for developing agentic AI that can effectively handle exceptions and adapt to novel contexts.

Abstract: Large language models (LLMs), initially developed for generative AI, are now evolving into agentic AI systems, which make decisions in complex, real-world contexts. Unfortunately, while their generative capabilities are well-documented, their decision-making processes remain poorly understood. This is particularly evident when testing targeted decision-making: for instance, how models handle exceptions, a critical and challenging aspect of decision-making made relevant by the inherent incompleteness of contracts. Here we demonstrate that LLMs, even ones that excel at reasoning, deviate significantly from human judgments because they adhere strictly to policies, even when such adherence is impractical, suboptimal, or even counterproductive. We then evaluate three approaches to tuning AI agents to handle exceptions: ethical framework prompting, chain-of-thought reasoning, and supervised fine-tuning. We find that while ethical framework prompting fails and chain-of-thought prompting provides only slight improvements, supervised fine-tuning - specifically with human explanations - yields markedly better results. Surprisingly, in our experiments, supervised fine-tuning even enabled models to generalize human-like decision-making to novel scenarios, demonstrating transfer learning of human-aligned decision-making across contexts. Furthermore, fine-tuning with explanations, not just labels, was critical for alignment, suggesting that aligning LLMs with human judgment requires explicit training on how decisions are made, not just which decisions are made. These findings highlight the need to address LLMs’ shortcomings in handling exceptions in order to guide the development of agentic AI toward models that can effectively align with human judgment and simultaneously adapt to novel contexts.

Method: Multi-turn supervised fine-tuning (SFT) and reinforcement learning (RL) with a multi-stage curriculum learning approach that partitions training samples by measured improvement potential, prioritizing higher-potential samples first.

Result: R1-CI-14B improves average accuracy on 37 test tasks from 44.1% to 72.4%, outperforming text-only GPT-4o (58.6%) and GPT-4o with Code Interpreter (70.9%), with RL gains increasing from +3.4% to +9.3%.

Conclusion: The approach successfully trains general-purpose Code Interpreter models that exhibit emergent self-checking behavior through code generation and achieve state-of-the-art performance on diverse reasoning tasks.

Abstract: Practical guidance on training Large Language Models (LLMs) to leverage Code Interpreter across diverse tasks remains lacking. We present R1-Code-Interpreter, an extension of a text-only LLM trained via multi-turn supervised fine-tuning (SFT) and reinforcement learning (RL) to autonomously generate multiple code queries during step-by-step reasoning. Unlike prior RL + tool-use efforts focused on narrow domains such as math or retrieval, we curate 144 diverse reasoning and planning tasks and show that training a general-purpose Code Interpreter across them presents significant challenges due to task heterogeneity and scarcity of effective samples. To address this, we introduce a multi-stage curriculum learning approach that partitions training samples by measured improvement potential. The RL training prioritizes samples with higher potential and gradually shifts to lower-potential ones, increasing the average RL gains from merely +3.4% to +9.3% across Qwen-2.5 models (3/7/14B). Our final model, R1-CI-14B, improves average accuracy on the 37 test tasks from 44.1% to 72.4%, outperforming text-only GPT-4o (58.6%) and GPT-4o with Code Interpreter (70.9%). Notably, R1-CI-14B also exhibits emergent self-checking behavior through code generation. Datasets, Codes, and Models are available at https://github.com/yongchao98/R1-Code-Interpreter and https://huggingface.co/yongchao98.

Method: TAMO unifies multi-modal observation data into time-aligned representations, then uses specialized tools for root cause localization and fault type classification. It overcomes LLM limitations through structured prompt design and tool assistance.

Result: Experiments on two benchmark datasets show that TAMO outperforms state-of-the-art approaches with comparable performance.

Conclusion: TAMO effectively addresses key challenges in LLM-driven root cause analysis for cloud-native systems through its tool-assisted approach and multimodal data processing capabilities.

Abstract: Implementing large language models (LLMs)-driven root cause analysis (RCA) in cloud-native systems has become a key topic of modern software operations and maintenance. However, existing LLM-based approaches face three key challenges: multi-modality input constraint, context window limitation, and dynamic dependence graph. To address these issues, we propose a tool-assisted LLM agent with multi-modality observation data for fine-grained RCA, namely TAMO, including multimodality alignment tool, root cause localization tool, and fault types classification tool. In detail, TAMO unifies multi-modal observation data into time-aligned representations for cross-modal feature consistency. Based on the unified representations, TAMO then invokes its specialized root cause localization tool and fault types classification tool for further identifying root cause and fault type underlying system context. This approach overcomes the limitations of LLMs in processing real-time raw observational data and dynamic service dependencies, guiding the model to generate repair strategies that align with system context through structured prompt design. Experiments on two benchmark datasets demonstrate that TAMO outperforms state-of-the-art (SOTA) approaches with comparable performance.

Method: Structured conceptual taxonomy, chronological evaluation of architectural evolution, comparative analysis of operational mechanisms, interaction styles, autonomy levels, and application mapping across both paradigms.

Result: Developed a clear distinction between AI Agents (task-specific automation via LLMs/LIMs) and Agentic AI (multi-agent collaboration with dynamic capabilities), identified application domains for each, and analyzed specific challenges and solutions.

Conclusion: The work provides a roadmap for developing robust, scalable, and explainable AI-driven systems by clarifying the conceptual boundaries and practical implementations of AI Agents versus Agentic AI paradigms.

Abstract: This review critically distinguishes between AI Agents and Agentic AI, offering a structured, conceptual taxonomy, application mapping, and analysis of opportunities and challenges to clarify their divergent design philosophies and capabilities. We begin by outlining the search strategy and foundational definitions, characterizing AI Agents as modular systems driven and enabled by LLMs and LIMs for task-specific automation. Generative AI is positioned as a precursor providing the foundation, with AI agents advancing through tool integration, prompt engineering, and reasoning enhancements. We then characterize Agentic AI systems, which, in contrast to AI Agents, represent a paradigm shift marked by multi-agent collaboration, dynamic task decomposition, persistent memory, and coordinated autonomy. Through a chronological evaluation of architectural evolution, operational mechanisms, interaction styles, and autonomy levels, we present a comparative analysis across both AI agents and agentic AI paradigms. Application domains enabled by AI Agents such as customer support, scheduling, and data summarization are then contrasted with Agentic AI deployments in research automation, robotic coordination, and medical decision support. We further examine unique challenges in each paradigm including hallucination, brittleness, emergent behavior, and coordination failure, and propose targeted solutions such as ReAct loops, retrieval-augmented generation (RAG), automation coordination layers, and causal modeling. This work aims to provide a roadmap for developing robust, scalable, and explainable AI-driven systems.

Method: Comprehensive review and analysis of existing research, examining bidirectional interactions: bandit algorithms for optimizing LLM fine-tuning, prompt engineering, and adaptive response generation; and LLMs for enhancing bandit algorithms through contextual understanding and policy selection.

Result: Identifies key challenges and opportunities in integrating bandit algorithms with LLMs, demonstrating their synergistic potential for improved performance in both decision-making and natural language processing tasks.

Conclusion: The survey provides a foundation for innovative applications and interdisciplinary research by bridging bandit algorithms and LLMs, highlighting their complementary strengths in balancing exploration-exploitation and contextual understanding.

Abstract: Bandit algorithms and Large Language Models (LLMs) have emerged as powerful tools in artificial intelligence, each addressing distinct yet complementary challenges in decision-making and natural language processing. This survey explores the synergistic potential between these two fields, highlighting how bandit algorithms can enhance the performance of LLMs and how LLMs, in turn, can provide novel insights for improving bandit-based decision-making. We first examine the role of bandit algorithms in optimizing LLM fine-tuning, prompt engineering, and adaptive response generation, focusing on their ability to balance exploration and exploitation in large-scale learning tasks. Subsequently, we explore how LLMs can augment bandit algorithms through advanced contextual understanding, dynamic adaptation, and improved policy selection using natural language reasoning. By providing a comprehensive review of existing research and identifying key challenges and opportunities, this survey aims to bridge the gap between bandit algorithms and LLMs, paving the way for innovative applications and interdisciplinary research in AI.

Method: Created VS-Bench with ten vision-grounded environments covering cooperative, competitive, and mixed-motive interactions, evaluating VLM agents across perception (element recognition), strategic reasoning (next-action prediction), and decision-making (normalized episode return).

Result: Experiments on 15 leading VLMs show strong perception abilities but significant performance gaps in reasoning and decision-making, with best model achieving only 46.6% prediction accuracy and 31.4% normalized return.

Conclusion: VS-Bench provides a standardized evaluation foundation for future research on strategic multimodal agents, highlighting current limitations and key factors influencing performance.

Abstract: Recent advancements in Vision Language Models (VLMs) have expanded their capabilities to interactive agent tasks, yet existing benchmarks remain limited to single-agent or text-only environments. In contrast, real-world scenarios often involve multiple agents interacting within rich visual and textual contexts, posing challenges with both multimodal observations and strategic interactions. To bridge this gap, we introduce Visual Strategic Bench (VS-Bench), a multimodal benchmark that evaluates VLMs for strategic abilities in multi-agent environments. VS-Bench comprises ten vision-grounded environments that cover cooperative, competitive, and mixed-motive interactions. The performance of VLM agents is evaluated across three dimensions: perception measured by element recognition accuracy; strategic reasoning measured by next-action prediction accuracy; and decision-making measured by normalized episode return. Extensive experiments on fifteen leading VLMs show that, although current models exhibit strong perception abilities, there remains a significant gap to optimal performance in reasoning and decision-making, with the best-performing model attaining 46.6% prediction accuracy and 31.4% normalized return. We further analyze the key factors influencing performance, conduct human experiments, and examine failure modes to provide a deeper understanding of VLMs’ strategic abilities. By standardizing the evaluation and highlighting the limitations of existing models, we envision VS-Bench as a foundation for future research on strategic multimodal agents. Code and data are available at https://vs-bench.github.io.

Method: A model-agnostic verification-and-refinement pipeline using carefully designed prompts, tested with three leading models (Gemini 2.5 Pro, Grok-4, GPT-5) on IMO 2025 problems while avoiding data contamination.

Result: The pipeline achieved 5/6 correct solutions (≈85.7% accuracy), dramatically outperforming baseline accuracies of 31.6% (Gemini), 21.4% (Grok), and 38.1% (GPT-5) from selecting best of 32 candidate solutions.

Conclusion: Advanced AI reasoning requires both more powerful base models and effective methodologies to harness their potential for complex tasks like IMO problems.

Abstract: The International Mathematical Olympiad (IMO) is widely regarded as the world championship of high-school mathematics. IMO problems are renowned for their difficulty and novelty, demanding deep insight, creativity, and rigor. Although large language models perform well on many mathematical benchmarks, they often struggle with Olympiad-level problems. Using carefully designed prompts, we construct a model-agnostic, verification-and-refinement pipeline. We demonstrate its effectiveness on the recent IMO 2025, avoiding data contamination for models released before the competition. Equipped with any of the three leading models – Gemini 2.5 Pro, Grok-4, or GPT-5 – our pipeline correctly solved 5 out of the 6 problems ($\approx$85.7% accuracy). This is in sharp contrast to their baseline accuracies: 31.6% (Gemini 2.5 Pro), 21.4% (Grok-4), and 38.1% (GPT-5), obtained by selecting the best of 32 candidate solutions. The substantial improvement underscores that the path to advanced AI reasoning requires not only developing more powerful base models but also designing effective methodologies to harness their full potential for complex tasks.

Method: MixGRPO integrates SDE and ODE sampling with a sliding window mechanism - using SDE sampling and GRPO-guided optimization only within the window, and ODE sampling outside. This confines randomness to specific time-steps and enables higher-order solvers.

Result: MixGRPO achieves substantial gains in human preference alignment, outperforming DanceGRPO in both effectiveness and efficiency with 50% lower training time. MixGRPO-Flash further reduces training time by 71% while maintaining comparable performance.

Conclusion: The mixed sampling strategy with sliding window mechanism successfully improves training efficiency while maintaining or enhancing performance in human preference alignment for image generation.

Abstract: Although GRPO substantially enhances flow matching models in human preference alignment of image generation, methods such as FlowGRPO still exhibit inefficiency due to the necessity of sampling and optimizing over all denoising steps specified by the Markov Decision Process (MDP). In this paper, we propose $\textbf{MixGRPO}$, a novel framework that leverages the flexibility of mixed sampling strategies through the integration of stochastic differential equations (SDE) and ordinary differential equations (ODE). This streamlines the optimization process within the MDP to improve efficiency and boost performance. Specifically, MixGRPO introduces a sliding window mechanism, using SDE sampling and GRPO-guided optimization only within the window, while applying ODE sampling outside. This design confines sampling randomness to the time-steps within the window, thereby reducing the optimization overhead, and allowing for more focused gradient updates to accelerate convergence. Additionally, as time-steps beyond the sliding window are not involved in optimization, higher-order solvers are supported for sampling. So we present a faster variant, termed $\textbf{MixGRPO-Flash}$, which further improves training efficiency while achieving comparable performance. MixGRPO exhibits substantial gains across multiple dimensions of human preference alignment, outperforming DanceGRPO in both effectiveness and efficiency, with nearly 50% lower training time. Notably, MixGRPO-Flash further reduces training time by 71%. Codes and models are available at $\href{https://github.com/Tencent-Hunyuan/MixGRPO}{MixGRPO}$.

Method: KompeteAI introduces dynamic solution space exploration with a merging stage to compose top candidates, integrates RAG to source ideas from Kaggle notebooks and arXiv papers, and uses predictive scoring with accelerated debugging to assess solution potential using early metrics.

Result: KompeteAI accelerates pipeline evaluation 6.9 times and outperforms leading methods (RD-agent, AIDE, Ml-Master) by an average of 3% on MLE-Bench. It also achieves state-of-the-art results on the proposed Kompete-bench.

Conclusion: KompeteAI successfully overcomes key limitations in current AutoML systems through its innovative exploration strategies, real-world knowledge integration, and execution acceleration methods, demonstrating superior performance on standard benchmarks.

Abstract: Recent Large Language Model (LLM)-based AutoML systems demonstrate impressive capabilities but face significant limitations such as constrained exploration strategies and a severe execution bottleneck. Exploration is hindered by one-shot methods lacking diversity and Monte Carlo Tree Search (MCTS) approaches that fail to recombine strong partial solutions. The execution bottleneck arises from lengthy code validation cycles that stifle iterative refinement. To overcome these challenges, we introduce KompeteAI, a novel AutoML framework with dynamic solution space exploration. Unlike previous MCTS methods that treat ideas in isolation, KompeteAI introduces a merging stage that composes top candidates. We further expand the hypothesis space by integrating Retrieval-Augmented Generation (RAG), sourcing ideas from Kaggle notebooks and arXiv papers to incorporate real-world strategies. KompeteAI also addresses the execution bottleneck via a predictive scoring model and an accelerated debugging method, assessing solution potential using early stage metrics to avoid costly full-code execution. This approach accelerates pipeline evaluation 6.9 times. KompeteAI outperforms leading methods (e.g., RD-agent, AIDE, and Ml-Master) by an average of 3% on the primary AutoML benchmark, MLE-Bench. Additionally, we propose Kompete-bench to address limitations in MLE-Bench, where KompeteAI also achieves state-of-the-art results

Method: Proposes GraphCogent framework with three modules: Sensory Module for standardizing graph representations via subgraph sampling, Buffer Module for integrating and indexing graph data across formats, and Execution Module combining tool calling and creation for efficient reasoning.

Result: GraphCogent achieves 50% improvement over massive-scale LLMs like DeepSeek-R1 (671B), outperforms state-of-the-art agent-based baseline by 20% in accuracy while reducing token usage by 80% for in-toolset tasks and 30% for out-toolset tasks.

Conclusion: The framework effectively addresses LLMs’ working memory limitations in graph reasoning through specialized cognitive processes and demonstrates superior performance on large-scale real-world graphs.

Abstract: Large language models (LLMs) show promising performance on small-scale graph reasoning tasks but fail when handling real-world graphs with complex queries. This phenomenon arises from LLMs’ working memory constraints, which result in their inability to retain long-range graph topology over extended contexts while sustaining coherent multi-step reasoning. However, real-world graphs are often structurally complex, such as Web, Transportation, Social, and Citation networks. To address these limitations, we propose GraphCogent, a collaborative agent framework inspired by human Working Memory Model that decomposes graph reasoning into specialized cognitive processes: sense, buffer, and execute. The framework consists of three modules: Sensory Module standardizes diverse graph text representations via subgraph sampling, Buffer Module integrates and indexes graph data across multiple formats, and Execution Module combines tool calling and tool creation for efficient reasoning. We also introduce Graph4real, a comprehensive benchmark that contains four domains of real-world graphs (Web, Transportation, Social, and Citation) to evaluate LLMs’ graph reasoning capabilities. Our Graph4real covers 21 different graph reasoning tasks, categorized into three types (Structural Querying, Algorithmic Reasoning, and Predictive Modeling tasks), with graph scales up to 10 times larger than existing benchmarks. Experiments show that Llama3.1-8B based GraphCogent achieves a 50% improvement over massive-scale LLMs like DeepSeek-R1 (671B). Compared to state-of-the-art agent-based baseline, our framework outperforms by 20% in accuracy while reducing token usage by 80% for in-toolset tasks and 30% for out-toolset tasks. Code will be available after review.

Method: VerlTool provides: (1) upstream alignment with VeRL, (2) unified tool management via standardized APIs supporting code execution, search, SQL, and vision, (3) asynchronous rollout execution for 2x speedup, and (4) modular plugin architecture for rapid tool integration.

Result: The framework achieves competitive performance across 6 ARLT domains (mathematical reasoning, knowledge QA, SQL generation, visual reasoning, web search, software engineering) with near 2x speedup from asynchronous execution.

Conclusion: VerlTool provides a scalable foundation for tool-augmented RL research with unified training infrastructure, significantly reducing development overhead while maintaining performance comparable to specialized systems.

Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has demonstrated success in enhancing LLM reasoning capabilities, but remains limited to single-turn interactions without tool integration. While recent Agentic Reinforcement Learning with Tool use (ARLT) approaches have emerged to address multi-turn tool interactions, existing works develop task-specific codebases that suffer from fragmentation, synchronous execution bottlenecks, and limited extensibility across domains. These inefficiencies hinder broader community adoption and algorithmic innovation. We introduce VerlTool, a unified and modular framework that addresses these limitations through systematic design principles. VerlTool provides four key contributions: (1) upstream alignment with VeRL ensuring compatibility and simplified maintenance, (2) unified tool management via standardized APIs supporting diverse modalities including code execution, search, SQL databases, and vision processing, (3) asynchronous rollout execution achieving near 2$\times$ speedup by eliminating synchronization bottlenecks, and (4) comprehensive evaluation demonstrating competitive performance across 6 ARLT domains. Our framework formalizes ARLT as multi-turn trajectories with multi-modal observation tokens (text/image/video), extending beyond single-turn RLVR paradigms. We train and evaluate models on mathematical reasoning, knowledge QA, SQL generation, visual reasoning, web search, and software engineering tasks, achieving results comparable to specialized systems while providing unified training infrastructure. The modular plugin architecture enables rapid tool integration requiring only lightweight Python definitions, significantly reducing development overhead and providing a scalable foundation for tool-augmented RL research. Our code is open-sourced at https://github.com/TIGER-AI-Lab/verl-tool.

Method: Two-stage training pipeline: (1) supervised fine-tuning on diverse synthetic data to prime models for dynamic planning, and (2) reinforcement learning to refine this capability in long-horizon environments.

Result: Dynamic planning agents are more sample-efficient and consistently achieve more complex objectives in the Crafter environment. They can also be effectively steered by human-written plans, surpassing independent capabilities.

Conclusion: This is the first work to train LLM agents for dynamic test-time compute allocation in sequential decision-making tasks, enabling more efficient, adaptive, and controllable agentic systems.

Abstract: Training large language models (LLMs) to reason via reinforcement learning (RL) significantly improves their problem-solving capabilities. In agentic settings, existing methods like ReAct prompt LLMs to explicitly plan before every action; however, we demonstrate that always planning is computationally expensive and degrades performance on long-horizon tasks, while never planning further limits performance. To address this, we introduce a conceptual framework formalizing dynamic planning for LLM agents, enabling them to flexibly decide when to allocate test-time compute for planning. We propose a simple two-stage training pipeline: (1) supervised fine-tuning on diverse synthetic data to prime models for dynamic planning, and (2) RL to refine this capability in long-horizon environments. Experiments on the Crafter environment show that dynamic planning agents trained with this approach are more sample-efficient and consistently achieve more complex objectives. Additionally, we demonstrate that these agents can be effectively steered by human-written plans, surpassing their independent capabilities. To our knowledge, this work is the first to explore training LLM agents for dynamic test-time compute allocation in sequential decision-making tasks, paving the way for more efficient, adaptive, and controllable agentic systems.

Method: Systematically evaluated SLMs on health prediction tasks using zero-shot, few-shot, and instruction fine-tuning approaches, then deployed best-performing models on mobile devices.

Result: SLMs achieved performance comparable to LLMs while offering substantial gains in efficiency and privacy.

Conclusion: SLMs are a promising solution for next-generation, privacy-preserving healthcare monitoring despite current limitations in handling class imbalance and few-shot scenarios.

Abstract: Mobile and wearable healthcare monitoring play a vital role in facilitating timely interventions, managing chronic health conditions, and ultimately improving individuals’ quality of life. Previous studies on large language models (LLMs) have highlighted their impressive generalization abilities and effectiveness in healthcare prediction tasks. However, most LLM-based healthcare solutions are cloud-based, which raises significant privacy concerns and results in increased memory usage and latency. To address these challenges, there is growing interest in compact models, Small Language Models (SLMs), which are lightweight and designed to run locally and efficiently on mobile and wearable devices. Nevertheless, how well these models perform in healthcare prediction remains largely unexplored. We systematically evaluated SLMs on health prediction tasks using zero-shot, few-shot, and instruction fine-tuning approaches, and deployed the best performing fine-tuned SLMs on mobile devices to evaluate their real-world efficiency and predictive performance in practical healthcare scenarios. Our results show that SLMs can achieve performance comparable to LLMs while offering substantial gains in efficiency and privacy. However, challenges remain, particularly in handling class imbalance and few-shot scenarios. These findings highlight SLMs, though imperfect in their current form, as a promising solution for next-generation, privacy-preserving healthcare monitoring.

Method: Uses a dual-agent architecture: Generator Agent analyzes tasks and designs agents, Target Agent is automatically generated and executed. Decomposes RL development into MDP modeling and algorithmic optimization stages. Built on Model Context Protocol for standardized agent creation across environments.

Result: Extensive experiments on MuJoCo, MetaDrive, MPE, and SMAC benchmarks show Agent^2 outperforms manually designed baselines across all tasks, achieving up to 55% performance improvement with consistent average gains.

Conclusion: Agent^2 enables closed-loop, end-to-end automation for RL agent design, advancing a new paradigm where agents can design and optimize other agents, demonstrating the potential of agent-generates-agent systems for automated AI development.

Abstract: Reinforcement learning (RL) agent development traditionally requires substantial expertise and iterative effort, often leading to high failure rates and limited accessibility. This paper introduces Agent$^2$, an LLM-driven agent-generates-agent framework for fully automated RL agent design. Agent$^2$ autonomously translates natural language task descriptions and environment code into executable RL solutions without human intervention. The framework adopts a dual-agent architecture: a Generator Agent that analyzes tasks and designs agents, and a Target Agent that is automatically generated and executed. To better support automation, RL development is decomposed into two stages, MDP modeling and algorithmic optimization, facilitating targeted and effective agent generation. Built on the Model Context Protocol, Agent$^2$ provides a unified framework for standardized agent creation across diverse environments and algorithms, incorporating adaptive training management and intelligent feedback analysis for continuous refinement. Extensive experiments on benchmarks including MuJoCo, MetaDrive, MPE, and SMAC show that Agent$^2$ outperforms manually designed baselines across all tasks, achieving up to 55% performance improvement with consistent average gains. By enabling a closed-loop, end-to-end automation pipeline, this work advances a new paradigm in which agents can design and optimize other agents, underscoring the potential of agent-generates-agent systems for automated AI development.

Method: Leverages Bayesian Neural Networks and Monte Carlo dropout to quantify both aleatoric and epistemic uncertainties across diverse detector architectures. Evaluates uncertainty on two datasets with nine generators, using four blind and two biological detectors.

Result: The uncertainty manifold holds consistent information for deepfake source detection. Uncertainty maps localize prediction confidence at pixel level, revealing distinct patterns correlated with generator-specific artifacts.

Conclusion: Uncertainty quantification is established as a fundamental requirement for trustworthy synthetic media detection, providing critical insights for deploying reliable deepfake detection systems.

Abstract: As generative models are advancing in quality and quantity for creating synthetic content, deepfakes begin to cause online mistrust. Deepfake detectors are proposed to counter this effect, however, misuse of detectors claiming fake content as real or vice versa further fuels this misinformation problem. We present the first comprehensive uncertainty analysis of deepfake detectors, systematically investigating how generative artifacts influence prediction confidence. As reflected in detectors’ responses, deepfake generators also contribute to this uncertainty as their generative residues vary, so we cross the uncertainty analysis of deepfake detectors and generators. Based on our observations, the uncertainty manifold holds enough consistent information to leverage uncertainty for deepfake source detection. Our approach leverages Bayesian Neural Networks and Monte Carlo dropout to quantify both aleatoric and epistemic uncertainties across diverse detector architectures. We evaluate uncertainty on two datasets with nine generators, with four blind and two biological detectors, compare different uncertainty methods, explore region- and pixel-based uncertainty, and conduct ablation studies. We conduct and analyze binary real/fake, multi-class real/fake, source detection, and leave-one-out experiments between the generator/detector combinations to share their generalization capability, model calibration, uncertainty, and robustness against adversarial attacks. We further introduce uncertainty maps that localize prediction confidence at the pixel level, revealing distinct patterns correlated with generator-specific artifacts. Our analysis provides critical insights for deploying reliable deepfake detection systems and establishes uncertainty quantification as a fundamental requirement for trustworthy synthetic media detection.

Method: Integrates three modules: Perception (abstracts diagrams to structured logic), Symbolic Reasoning (applies geometric theorems), and Sketch Action (executes operations like drawing lines). Trained via supervised fine-tuning on curated trajectories followed by reinforcement learning with symbolic rewards.

Result: Significantly improves stepwise reasoning accuracy and problem-solving success over static perception methods on the GeoSketch Benchmark of 390 geometry problems requiring auxiliary construction or transformations.

Conclusion: GeoSketch advances multimodal reasoning from static interpretation to dynamic, verifiable interaction, establishing a new foundation for solving complex visuospatial problems through hierarchical decision-making and executable visual actions.

Abstract: Geometric Problem Solving (GPS) poses a unique challenge for Multimodal Large Language Models (MLLMs), requiring not only the joint interpretation of text and diagrams but also iterative visuospatial reasoning. While existing approaches process diagrams as static images, they lack the capacity for dynamic manipulation - a core aspect of human geometric reasoning involving auxiliary line construction and affine transformations. We present GeoSketch, a neural-symbolic framework that recasts geometric reasoning as an interactive perception-reasoning-action loop. GeoSketch integrates: (1) a Perception module that abstracts diagrams into structured logic forms, (2) a Symbolic Reasoning module that applies geometric theorems to decide the next deductive step, and (3) a Sketch Action module that executes operations such as drawing auxiliary lines or applying transformations, thereby updating the diagram in a closed loop. To train this agent, we develop a two-stage pipeline: supervised fine-tuning on 2,000 symbolic-curated trajectories followed by reinforcement learning with dense, symbolic rewards to enhance robustness and strategic exploration. To evaluate this paradigm, we introduce the GeoSketch Benchmark, a high-quality set of 390 geometry problems requiring auxiliary construction or affine transformations. Experiments on strong MLLM baselines demonstrate that GeoSketch significantly improves stepwise reasoning accuracy and problem-solving success over static perception methods. By unifying hierarchical decision-making, executable visual actions, and symbolic verification, GeoSketch advances multimodal reasoning from static interpretation to dynamic, verifiable interaction, establishing a new foundation for solving complex visuospatial problems.

Method: Built on DIKW hierarchy, PAME-AI uses specialized computational agents to progressively transform raw experimental data into actionable message design strategies through parallel processing and hypothesis validation.

Result: In experiments with 444,691 patient encounters (Stage 1) and 74,908 (Stage 2), the best-performing generated message achieved 68.76% engagement vs 61.27% baseline, representing 12.2% relative improvement in click-through rates.

Conclusion: PAME-AI’s agentic architecture enables effective large-scale healthcare communication optimization through continuous learning and parallel processing capabilities.

Abstract: Messaging patients is a critical part of healthcare communication, helping to improve things like medication adherence and healthy behaviors. However, traditional mobile message design has significant limitations due to its inability to explore the high-dimensional design space. We develop PAME-AI, a novel approach for Patient Messaging Creation and Optimization using Agentic AI. Built on the Data-Information-Knowledge-Wisdom (DIKW) hierarchy, PAME-AI offers a structured framework to move from raw data to actionable insights for high-performance messaging design. PAME-AI is composed of a system of specialized computational agents that progressively transform raw experimental data into actionable message design strategies. We demonstrate our approach’s effectiveness through a two-stage experiment, comprising of 444,691 patient encounters in Stage 1 and 74,908 in Stage 2. The best-performing generated message achieved 68.76% engagement compared to the 61.27% baseline, representing a 12.2% relative improvement in click-through rates. This agentic architecture enables parallel processing, hypothesis validation, and continuous learning, making it particularly suitable for large-scale healthcare communication optimization.

Method: Proposes InfiAgent framework with: agent-as-a-tool mechanism for automatic hierarchical decomposition; dual-audit mechanism for quality assurance; agent routing for efficient task matching; self-evolution mechanism for autonomous DAG restructuring; and atomic task design supporting parallelism.

Result: Achieves 9.9% higher performance than ADAS (similar auto-generated agent framework) on multiple benchmarks. Case study shows InfiHelper (AI research assistant) generates scientific papers recognized by human reviewers at top-tier IEEE conferences.

Conclusion: InfiAgent evolves into a versatile pyramid-like multi-agent system capable of solving diverse problems, addressing scalability and cost-effectiveness limitations of hand-crafted LLM agents through automated decomposition, quality control, and self-evolution capabilities.

Abstract: Large Language Model (LLM) agents have demonstrated remarkable capabilities in organizing and executing complex tasks, and many such agents are now widely used in various application scenarios. However, developing these agents requires carefully designed workflows, carefully crafted prompts, and iterative tuning, which requires LLM techniques and domain-specific expertise. These hand-crafted limitations hinder the scalability and cost-effectiveness of LLM agents across a wide range of industries. To address these challenges, we propose \textbf{InfiAgent}, a Pyramid-like DAG-based Multi-Agent Framework that can be applied to \textbf{infi}nite scenarios, which introduces several key innovations: a generalized “agent-as-a-tool” mechanism that automatically decomposes complex agents into hierarchical multi-agent systems; a dual-audit mechanism that ensures the quality and stability of task completion; an agent routing function that enables efficient task-agent matching; and an agent self-evolution mechanism that autonomously restructures the agent DAG based on new tasks, poor performance, or optimization opportunities. Furthermore, InfiAgent’s atomic task design supports agent parallelism, significantly improving execution efficiency. This framework evolves into a versatile pyramid-like multi-agent system capable of solving a wide range of problems. Evaluations on multiple benchmarks demonstrate that InfiAgent achieves 9.9% higher performance compared to ADAS (similar auto-generated agent framework), while a case study of the AI research assistant InfiHelper shows that it generates scientific papers that have received recognition from human reviewers at top-tier IEEE conferences.

Method: EvoPH integrates island migration model with elite selection algorithm to simulate diverse heuristics populations. It co-evolves prompts with heuristic algorithms guided by performance feedback.

Result: EvoPH achieves the lowest relative error against optimal solutions on Traveling Salesman Problem and Bin Packing Problem datasets compared to other methods.

Conclusion: The framework advances automatic algorithm design with LLMs by effectively avoiding local optima and generating high-quality heuristics for combinatorial optimization problems.

Abstract: Combinatorial optimization problems are traditionally tackled with handcrafted heuristic algorithms, which demand extensive domain expertise and significant implementation effort. Recent progress has highlighted the potential of automatic heuristics design powered by large language models (LLMs), enabling the automatic generation and refinement of heuristics. These approaches typically maintain a population of heuristics and employ LLMs as mutation operators to evolve them across generations. While effective, such methods often risk stagnating in local optima. To address this issue, we propose the Experience-Guided Reflective Co-Evolution of Prompt and Heuristics (EvoPH) for automatic algorithm design, a novel framework that integrates the island migration model with the elites selection algorithm to simulate diverse heuristics populations. In EvoPH, prompts are co-evolved with heuristic algorithms, guided by performance feedback. We evaluate our framework on two problems, i.e., Traveling Salesman Problem and Bin Packing Problem. Experimental results demonstrate that EvoPH achieves the lowest relative error against optimal solutions across both datasets, advancing the field of automatic algorithm design with LLMs.

Method: Introduces EELMA (Estimating Empowerment of Language Model Agents), an algorithm for approximating effective empowerment from multi-turn text interactions, validated on language games and realistic web-browsing scenarios.

Result: Empowerment strongly correlates with average task performance, characterizes impact of environmental complexity and agentic factors (chain-of-thought, model scale, memory length), and high empowerment states/actions are often pivotal moments for general capabilities.

Conclusion: Empowerment is an appealing general-purpose metric for evaluating and monitoring LM agents in complex, open-ended settings.

Abstract: As language model (LM) agents become more capable and gain broader access to real-world tools, there is a growing need for scalable evaluation frameworks of agentic capability. However, conventional benchmark-centric evaluations are costly to design and require human designers to come up with valid tasks that translate into insights about general model capabilities. In this work, we propose information-theoretic evaluation based on empowerment, the mutual information between an agent’s actions and future states, as an open-ended method for evaluating LM agents. We introduce EELMA (Estimating Empowerment of Language Model Agents), an algorithm for approximating effective empowerment from multi-turn text interactions. We validate EELMA on both language games and scaled-up realistic web-browsing scenarios. We find that empowerment strongly correlates with average task performance, characterize the impact of environmental complexity and agentic factors such as chain-of-thought, model scale, and memory length on estimated empowerment, and that high empowerment states and actions are often pivotal moments for general capabilities. Together, these results demonstrate empowerment as an appealing general-purpose metric for evaluating and monitoring LM agents in complex, open-ended settings.

Method: Proposed a pipeline for eliciting, steering, and modulating LLMs’ risk profiles using utility-theoretic models from behavioral economics and finance. Compared pre-trained, instruction-tuned, and RLHF-aligned LLMs, and evaluated modulation strategies including prompt engineering, in-context learning, and post-training.

Result: Instruction-tuned models exhibit behaviors consistent with standard utility formulations, while pre-trained and RLHF-aligned models deviate more from fitted utility models. Post-training provides the most stable and effective modulation of risk preference compared to other strategies.

Conclusion: The findings provide insights into risk profiles of different LLM classes and stages, demonstrating how post-training effectively modulates these profiles, laying groundwork for future research on behavioral alignment and risk-aware LLM design.

Abstract: Large language models (LLMs) are increasingly used for decision-making tasks under uncertainty; however, their risk profiles and how they are influenced by prompting and alignment methods remain underexplored. Existing studies have primarily examined personality prompting or multi-agent interactions, leaving open the question of how post-training influences the risk behavior of LLMs. In this work, we propose a new pipeline for eliciting, steering, and modulating LLMs’ risk profiles, drawing on tools from behavioral economics and finance. Using utility-theoretic models, we compare pre-trained, instruction-tuned, and RLHF-aligned LLMs, and find that while instruction-tuned models exhibit behaviors consistent with some standard utility formulations, pre-trained and RLHF-aligned models deviate more from any utility models fitted. We further evaluate modulation strategies, including prompt engineering, in-context learning, and post-training, and show that post-training provides the most stable and effective modulation of risk preference. Our findings provide insights into the risk profiles of different classes and stages of LLMs and demonstrate how post-training modulates these profiles, laying the groundwork for future research on behavioral alignment and risk-aware LLM design.

Method: Introduces DRI metric to quantify logical reasoning complexity by decomposing and aggregating logical structures. Proposes a re-cognizing optimization strategy that systematically enhances logical reasoning intensity of existing training data to better align with LLMs’ reasoning boundaries.

Result: Extensive experiments show significant improvements in performance and generalization over data-centric strategies. The method was also validated under reinforcement learning framework.

Conclusion: Prioritizing reasoning complexity in data rather than sheer scale or superficial form is essential to realizing LLMs’ full cognitive potential.

Abstract: Recent advances in large language models (LLMs) highlight the importance of training data structure and quality in shaping reasoning behavior. However, most existing approaches focus on transforming data formats while neglecting the internal reasoning complexity of training samples, leaving the reasoning potential of data under-explored and underutilized. In this work, we posit that LLM logical reasoning performance is jointly constrained by the potential of the training data and the cognitive capacity of the model. To make this relationship measurable, we introduce Data Reasoning Intensity (DRI), a novel metric that quantifies the latent logical reasoning complexity of samples by decomposing and aggregating their logical structures. This allows us to analyze how well current LLMs utilize logical reasoning signals and identify performance gaps relative to data potential. Based on this insight, we introduce a re-cognizing optimization strategy that systematically enhances the logical reasoning intensity of training data. Rather than increasing data volume, our method re-optimizes existing samples to better align with the LLM’s logical reasoning boundary. Extensive experiments show that our approach significantly improves performance and generalization over data-centric strategies. We further validate our method under a reinforcement learning framework. Our results indicate that prioritizing reasoning complexity in data rather than sheer scale or superficial form is essential to realizing LLMs’ full cognitive potential.

Method: Created SysMoBench with automated metrics for syntactic correctness, runtime correctness, conformance to system code, and invariant correctness. Currently includes nine diverse system artifacts including Raft implementations in Etcd and Redis, Spinlock and Mutex in Asterinas OS.

Result: The benchmark enables evaluation of LLMs and agents in generating formal specifications for concurrent and distributed systems, providing a foundation for understanding their capabilities and limitations.

Conclusion: SysMoBench provides a solid foundation for evaluating AI-generated formal models, opening up new research directions in automated formal specification generation for complex systems.

Abstract: Formal models are essential to specifying large, complex computer systems and verifying their correctness, but are notoriously expensive to write and maintain. Recent advances in generative AI show promise in generating certain forms of specifications. However, existing work mostly targets small code, not complete systems. It is unclear whether AI can deal with realistic system artifacts, as this requires abstracting their complex behavioral properties into formal models. We present SysMoBench, a benchmark that evaluates AI’s ability to formally model large, complex systems. We focus on concurrent and distributed systems, which are keystones of today’s critical computing infrastructures, encompassing operating systems and cloud infrastructure. We use TLA+, the de facto specification language for concurrent and distributed systems, though the benchmark can be extended to other specification languages. We address the primary challenge of evaluating AI-generated models by automating metrics like syntactic and runtime correctness, conformance to system code, and invariant correctness. SysMoBench currently includes nine diverse system artifacts: the Raft implementation of Etcd and Redis, the Spinlock and Mutex in Asterinas OS, etc.; more artifacts are being actively added. SysMoBench enables us to understand the capabilities and limitations of today’s LLMs and agents, putting tools in this area on a firm footing and opening up promising new research directions.

Method: Created MASLegalBench using GDPR scenarios with deductive reasoning approach, manually designed role-based MAS, and tested with state-of-the-art LLMs.

Result: Experiments revealed strengths, limitations, and improvement areas for existing models and MAS architectures in legal reasoning.

Conclusion: MASLegalBench enables better evaluation of multi-agent systems in legal domain, highlighting their potential and current limitations.

Abstract: Multi-agent systems (MAS), leveraging the remarkable capabilities of Large Language Models (LLMs), show great potential in addressing complex tasks. In this context, integrating MAS with legal tasks is a crucial step. While previous studies have developed legal benchmarks for LLM agents, none are specifically designed to consider the unique advantages of MAS, such as task decomposition, agent specialization, and flexible training. In fact, the lack of evaluation methods limits the potential of MAS in the legal domain. To address this gap, we propose MASLegalBench, a legal benchmark tailored for MAS and designed with a deductive reasoning approach. Our benchmark uses GDPR as the application scenario, encompassing extensive background knowledge and covering complex reasoning processes that effectively reflect the intricacies of real-world legal situations. Furthermore, we manually design various role-based MAS and conduct extensive experiments using different state-of-the-art LLMs. Our results highlight the strengths, limitations, and potential areas for improvement of existing models and MAS architectures.

Method: Treat parametric problems as systems, drive a single parameter sinusoidally, and fit first-harmonic responses of model outputs and exact solutions to obtain interpretable frequency-resolved metrics (gain and phase) that form Bode-style fingerprints.

Result: The diagnostic reveals systematic low-pass behavior and growing phase lag across five mathematical problem families, clearly separating frontier models from mid-tier models based on reasoning dynamics rather than just accuracy.

Conclusion: MathBode provides a compact, reproducible protocol that complements standard benchmarks with actionable measurements of reasoning fidelity and consistency, offering a more comprehensive evaluation of mathematical reasoning in LLMs.

Abstract: This paper presents MathBode, a dynamic diagnostic for mathematical reasoning in large language models (LLMs). Instead of one-shot accuracy, MathBode treats each parametric problem as a system: we drive a single parameter sinusoidally and fit first-harmonic responses of model outputs and exact solutions. This yields interpretable, frequency-resolved metrics – gain (amplitude tracking) and phase (lag) – that form Bode-style fingerprints. Across five closed-form families (linear solve, ratio/saturation, compound interest, 2x2 linear systems, similar triangles), the diagnostic surfaces systematic low-pass behavior and growing phase lag that accuracy alone obscures. We compare several models against a symbolic baseline that calibrates the instrument ($G \approx 1$, $\phi \approx 0$). Results separate frontier from mid-tier models on dynamics, providing a compact, reproducible protocol that complements standard benchmarks with actionable measurements of reasoning fidelity and consistency. We open-source the dataset and code to enable further research and adoption.

Method: Proposed Tool-Light framework with dataset construction using continuous self-evolved sampling (vanilla and entropy-guided) with strict positive-negative pair selection, followed by two-stage fine-tuning: SFT and Self-Evolved DPO.

Result: Experimental results on 10 datasets show Tool-Light significantly improves model efficiency in executing TIR tasks.

Conclusion: Tool-Light effectively addresses TIR optimization by leveraging entropy insights and multi-stage training to enhance LLM reasoning with external tools.

Abstract: Tool-Integrated Reasoning (TIR) enables large language models (LLMs) to improve their internal reasoning ability by integrating external tools. However, models employing TIR often display suboptimal behaviors, such as insufficient or excessive tool usage and overthinking after tool calls. The challenge of incentivizing LLMs to perform TIR efficiently and accurately, while stabilizing the reasoning process, remains an open question. In this paper, we start by exploring the impact of tool calls on model reasoning from the perspective of information entropy. Our findings indicate that tool call results lead to a distinct change in the information entropy of subsequent reasoning, with the overall entropy of the reasoning chain varying based on the number of tool calls. Building on these insights, we propose Tool-Light, a framework designed to encourage LLMs to perform TIR efficiently and accurately. Our framework includes dataset construction and multi-stage fine-tuning. For dataset construction, we employ continuous self-evolved sampling using the fine-tuned model, integrating both vanilla sampling and entropy-guided sampling. Besides, we establish strict criteria for selecting positive-negative pairs during sampling. The training process involves a two-stage approach, comprising Supervised Fine-Tuning (SFT) and Self-Evolved Direct Preference Optimization (DPO). Experimental results on 10 datasets demonstrate the effectiveness of Tool-Light, significantly improving the model’s efficiency in executing TIR tasks.

Method: Uses Item Response Theory (IRT) with probabilistic modeling framework, collaborative filtering with topic-level skill profiles, and introduces a novel discrete variational inference framework for low-data settings.

Result: Achieved over 80% accuracy in mathematics prediction of formal exam papers, setting a new benchmark. Found that a single latent ability parameter alone achieves maximum predictive accuracy. The discrete variational inference method outperformed all classical IRT and matrix factorization baselines in low-data settings.

Conclusion: The research provides an effective scalable solution for AI tutoring through high-accuracy prediction models, with the novel discrete variational inference framework being particularly valuable for data-sparse educational environments.

Abstract: One of the largest drivers of social inequality is unequal access to personal tutoring, with wealthier individuals able to afford it, while the majority cannot. Affordable, effective AI tutors offer a scalable solution. We focus on adaptive learning, predicting whether a student will answer a question correctly, a key component of any effective tutoring system. Yet many platforms struggle to achieve high prediction accuracy, especially in data-sparse settings. To address this, we release the largest open dataset of professionally marked formal mathematics exam responses to date. We introduce a probabilistic modelling framework rooted in Item Response Theory (IRT) that achieves over 80 percent accuracy, setting a new benchmark for mathematics prediction accuracy of formal exam papers. Extending this, our collaborative filtering models incorporate topic-level skill profiles, but reveal a surprising and educationally significant finding, a single latent ability parameter alone is needed to achieve the maximum predictive accuracy. Our main contribution though is deriving and implementing a novel discrete variational inference framework, achieving our highest prediction accuracy in low-data settings and outperforming all classical IRT and matrix factorisation baselines.

Method: Identified a compact power law linking model size and expert number, validated across diverse architectures and merging methods (Average, TA, TIES, DARE) in both in-domain and cross-domain settings.

Result: The scaling law holds consistently: size-dependent floor decreases with model capacity, merging tail shows diminishing returns with more experts, gains fall roughly as 1/k, and variability shrinks as more experts are added.

Conclusion: This law transforms merging from heuristic practice into a computationally efficient, planable alternative to multitask training, suggesting a scaling principle for distributed generative AI through predictable composition of specialists.

Abstract: We study empirical scaling laws for language model merging measured by cross-entropy. Despite its wide practical use, merging lacks a quantitative rule that predicts returns as we add experts or scale the model size. We identify a compact power law that links model size and expert number: the size-dependent floor decreases with model capacity, while the merging tail exhibits clear diminishing returns in the number of experts. The law holds in-domain and cross-domain, tightly fits measured curves across diverse architectures and methods (Average, TA, TIES, DARE), and explains two robust regularities: most gains arrive early, and variability shrinks as more experts are included. Building on this, we present a simple theory that explains why gains fall roughly as 1/k and links the floor and tail to properties of the base model and the diversity across domains. This law enables predictive planning: estimate how many experts are needed to reach a target loss, decide when to stop adding experts, and trade off scaling the base model versus adding experts under a fixed budget–turning merging from heuristic practice into a computationally efficient, planable alternative to multitask training. This suggests a scaling principle for distributed generative AI: predictable gains can be achieved by composing specialists, offering a complementary path toward AGI-level systems.

Method: Semiotic-square-guided framework with multi-perspective deduction in first-order logic, existential import checks using three-valued decision scheme (True, False, Uncertain), and introduces RepublicQA benchmark for evaluation.

Result: Achieves 6.25% average gain on RepublicQA over strong baselines and 7.05% average gain on mainstream benchmarks (ProntoQA, ProofWriter, FOLIO, ProverQA), demonstrating state-of-the-art performance.

Conclusion: The semiotic-grounded multi-perspective reasoning approach effectively boosts LLMs’ logical performance, handling both logical and semantic complexity through systematic reasoning frameworks.

Abstract: Logical reasoning is a fundamental capability of large language models (LLMs). However, existing studies largely overlook the interplay between logical complexity and semantic complexity, resulting in methods that struggle to address challenging scenarios involving abstract propositions, ambiguous contexts, and conflicting stances, which are central to human reasoning. For this gap, we propose LogicAgent, a semiotic-square-guided framework designed to jointly address logical complexity and semantic complexity. LogicAgent explicitly performs multi-perspective deduction in first-order logic (FOL), while mitigating vacuous reasoning through existential import checks that incorporate a three-valued decision scheme (True, False, Uncertain) to handle boundary cases more faithfully. Furthermore, to overcome the semantic simplicity and low logical complexity of existing datasets, we introduce RepublicQA, a benchmark that reaches college-level difficulty (FKGL = 11.94) and exhibits substantially greater lexical and structural diversity than prior benchmarks. RepublicQA is grounded in philosophical concepts, featuring abstract propositions and systematically organized contrary and contradictory relations, making it the most semantically rich resource for evaluating logical reasoning. Experiments demonstrate that LogicAgent achieves state-of-the-art performance on RepublicQA, with a 6.25% average gain over strong baselines, and generalizes effectively to mainstream logical reasoning benchmarks including ProntoQA, ProofWriter, FOLIO, and ProverQA, achieving an additional 7.05% average gain. These results highlight the strong effectiveness of our semiotic-grounded multi-perspective reasoning in boosting LLMs’ logical performance.

Method: Uses large language model tool-use capabilities with minimal tools (primarily code execution) to explore physical systems through experiments and analysis in an iterative loop.

Result: Impressive performance in recovering equations of motion from observed dynamics and inferring Hamiltonians from expectation values across mechanical dynamical systems, wave evolution, and quantum many-body physics.

Conclusion: The approach enables scientific exploration across domains without finetuning or task-specific instructions, opening doors to automated discovery in other scientific fields.

Abstract: The process of scientific discovery relies on an interplay of observations, analysis, and hypothesis generation. Machine learning is increasingly being adopted to address individual aspects of this process. However, it remains an open challenge to fully automate the open-ended, heuristic, iterative loop required to discover the laws of an unknown system by exploring it through experiments and analysis, without tailoring the approach to the specifics of a given task. Here, we introduce SciExplorer, an agent that leverages large language model tool-use capabilities to enable free-form exploration of systems without any domain-specific blueprints, and apply it to the exploration of physical systems that are initially unknown to the agent. We test SciExplorer on a broad set of models spanning mechanical dynamical systems, wave evolution, and quantum many-body physics. Despite using a minimal set of tools, primarily based on code execution, we observe impressive performance on tasks such as recovering equations of motion from observed dynamics and inferring Hamiltonians from expectation values. The demonstrated effectiveness of this setup opens the door towards similar scientific exploration in other domains, without the need for finetuning or task-specific instructions.

Method: Uses latent bridge models with energy-preserving variational autoencoder, joint neural prior, and perceptually aware fine-tuning to handle diverse low-quality to high-quality restoration tasks.

Result: Superior performance demonstrated across in-domain and out-of-domain tasks, including refining zero-shot speech and podcast generation results.

Conclusion: VoiceBridge provides scalable general speech restoration capability with high-fidelity reconstruction from various distortions through latent bridge modeling.

Abstract: Bridge models have recently been explored for speech enhancement tasks such as denoising, dereverberation, and super-resolution, while these efforts are typically confined to a single task or small-scale datasets, with constrained general speech restoration (GSR) capability at scale. In this work, we introduce VoiceBridge, a GSR system rooted in latent bridge models (LBMs), capable of reconstructing high-fidelity speech at full-band (\textit{i.e.,} 48kHz) from various distortions. By compressing speech waveform into continuous latent representations, VoiceBridge models the\textit{diverse LQ-to-HQ tasks} (namely, low-quality to high-quality) in GSR with~\textit{a single latent-to-latent generative process} backed by a scalable transformer architecture. To better inherit the advantages of bridge models from the data domain to the latent space, we present an energy-preserving variational autoencoder, enhancing the alignment between the waveform and latent space over varying energy levels. Furthermore, to address the difficulty of HQ reconstruction from distinctively different LQ priors, we propose a joint neural prior, uniformly alleviating the reconstruction burden of LBM. At last, considering the key requirement of GSR systems, human perceptual quality, a perceptually aware fine-tuning stage is designed to mitigate the cascading mismatch in generation while improving perceptual alignment. Extensive validation across in-domain and out-of-domain tasks and datasets (\textit{e.g.}, refining recent zero-shot speech and podcast generation results) demonstrates the superior performance of VoiceBridge. Demo samples can be visited at: https://VoiceBridge-demo.github.io/.

Method: Proposes an architecture with Transformers as symbolic decision module, Wav2Vec 2.0 for perception, and concatenative synthesis for audio rendering. Uses paired tracks (A, B) for training relationships.

Result: Quantitative evaluation shows the decision module can successfully predict coherent track B when conditioned on its corresponding guide track A, reproducing learned musical relationships.

Conclusion: The proposed architecture effectively learns and exploits musical relationships from paired track corpora, demonstrating potential for real-time musical agent applications.

Abstract: This paper presents the first step in a research project situated within the field of musical agents. The objective is to achieve, through training, the tuning of the desired musical relationship between a live musical input and a real-time generated musical output, through the curation of a database of separated tracks. We propose an architecture integrating a symbolic decision module capable of learning and exploiting musical relationships from such musical corpus. We detail an offline implementation of this architecture employing Transformers as the decision module, associated with a perception module based on Wav2Vec 2.0, and concatenative synthesis as audio renderer. We present a quantitative evaluation of the decision module’s ability to reproduce learned relationships extracted during training. We demonstrate that our decision module can predict a coherent track B when conditioned by its corresponding ‘‘guide’’ track A, based on a corpus of paired tracks (A, B).

Method: Proposes Emo-TTA framework that incrementally updates class-conditional statistics via Expectation-Maximization procedure for explicit test-time distribution estimation, using audio-language model predictions as priors without modifying model weights.

Result: Experiments on six out-of-domain speech emotion recognition benchmarks show consistent accuracy improvements over prior test-time adaptation baselines.

Conclusion: Statistical adaptation through Emo-TTA effectively aligns model predictions with evolving test distributions, demonstrating the effectiveness of training-free adaptation for speech emotion recognition.

Abstract: Speech emotion recognition (SER) with audio-language models (ALMs) remains vulnerable to distribution shifts at test time, leading to performance degradation in out-of-domain scenarios. Test-time adaptation (TTA) provides a promising solution but often relies on gradient-based updates or prompt tuning, limiting flexibility and practicality. We propose Emo-TTA, a lightweight, training-free adaptation framework that incrementally updates class-conditional statistics via an Expectation-Maximization procedure for explicit test-time distribution estimation, using ALM predictions as priors. Emo-TTA operates on individual test samples without modifying model weights. Experiments on six out-of-domain SER benchmarks show consistent accuracy improvements over prior TTA baselines, demonstrating the effectiveness of statistical adaptation in aligning model predictions with evolving test distributions.

Method: Uses cross-lingual transfer learning from English pre-trained audio-visual SSL models, flow-matching model for F0 contour generation guided by ASR-fine-tuned SSL speech units, and two-stage training with flow-matching postnet refinement.

Result: Significantly outperforms existing methods in both speech intelligibility and tonal accuracy through extensive experiments.

Conclusion: The proposed LTA-L2S framework successfully addresses Mandarin L2S challenges by leveraging cross-lingual knowledge transfer and specialized tone modeling, achieving superior performance over existing approaches.

Abstract: Lip-to-speech (L2S) synthesis for Mandarin is a significant challenge, hindered by complex viseme-to-phoneme mappings and the critical role of lexical tones in intelligibility. To address this issue, we propose Lexical Tone-Aware Lip-to-Speech (LTA-L2S). To tackle viseme-to-phoneme complexity, our model adapts an English pre-trained audio-visual self-supervised learning (SSL) model via a cross-lingual transfer learning strategy. This strategy not only transfers universal knowledge learned from extensive English data to the Mandarin domain but also circumvents the prohibitive cost of training such a model from scratch. To specifically model lexical tones and enhance intelligibility, we further employ a flow-matching model to generate the F0 contour. This generation process is guided by ASR-fine-tuned SSL speech units, which contain crucial suprasegmental information. The overall speech quality is then elevated through a two-stage training paradigm, where a flow-matching postnet refines the coarse spectrogram from the first stage. Extensive experiments demonstrate that LTA-L2S significantly outperforms existing methods in both speech intelligibility and tonal accuracy.

Method: Extended YNote to create HNote, converted 12,300 Jiangnan-style songs from YNote to HNote, and fine-tuned LLaMA-3.1(8B) using LoRA parameter-efficient fine-tuning.

Result: HNote achieved 82.5% syntactic correctness rate, with BLEU and ROUGE evaluations showing strong symbolic and structural similarity, producing stylistically coherent compositions.

Conclusion: HNote establishes an effective framework for integrating LLMs with cultural music modeling, providing an LLM-compatible notation system that reduces ambiguity and ensures alignment.

Abstract: Recent advances in large language models (LLMs) have created new opportunities for symbolic music generation. However, existing formats such as MIDI, ABC, and MusicXML are either overly complex or structurally inconsistent, limiting their suitability for token-based learning architectures. To address these challenges, we propose HNote, a novel hexadecimal-based notation system extended from YNote, which encodes both pitch and duration within a fixed 32-unit measure framework. This design ensures alignment, reduces ambiguity, and is directly compatible with LLM architectures. We converted 12,300 Jiangnan-style songs generated from traditional folk pieces from YNote into HNote, and fine-tuned LLaMA-3.1(8B) using parameter-efficient LoRA. Experimental results show that HNote achieves a syntactic correctness rate of 82.5%, and BLEU and ROUGE evaluations demonstrate strong symbolic and structural similarity, producing stylistically coherent compositions. This study establishes HNote as an effective framework for integrating LLMs with cultural music modeling.

Method: Treats spectrograms as multi-channel images, uses channel multiplexing to reshape without information loss, employs shared tokenizer for consistent discrete representations, and uses transformer-based autoregressor for coarse-to-fine refinement.

Result: Performs comparably or better than state-of-the-art baselines across multiple evaluation metrics on large-scale dataset.

Conclusion: Establishes an efficient and scalable paradigm for high-fidelity audio generation through multi-scale spectrogram autoregression.

Abstract: Research on audio generation has progressively shifted from waveform-based approaches to spectrogram-based methods, which more naturally capture harmonic and temporal structures. At the same time, advances in image synthesis have shown that autoregression across scales, rather than tokens, improves coherence and detail. Building on these ideas, we introduce MARS (Multi-channel AutoRegression on Spectrograms), a framework that treats spectrograms as multi-channel images and employs channel multiplexing (CMX), a reshaping technique that lowers height and width without discarding information. A shared tokenizer provides consistent discrete representations across scales, enabling a transformer-based autoregressor to refine spectrograms from coarse to fine resolutions efficiently. Experiments on a large-scale dataset demonstrate that MARS performs comparably or better than state-of-the-art baselines across multiple evaluation metrics, establishing an efficient and scalable paradigm for high-fidelity audio generation.

Method: Developed SAGE encoder that uses panoramic depth images and room-impulse responses during training to align binaural features with 3D spatial structure. Built OWL model with spatially grounded chain-of-thought reasoning and curriculum learning from perceptual QA to multi-step reasoning. Created BiDepth dataset with over 1 million QA pairs combining binaural audio with panoramic depth images and room impulse responses.

Result: OWL reduces mean direction-of-arrival error by 11° through SAGE and improves spatial reasoning QA accuracy by up to 25% over BAT across BiDepth and SpatialSoundQA benchmark datasets. Supports o’clock-level azimuth and DoA estimation.

Conclusion: The proposed geometry-aware approach with explicit spatial alignment significantly improves spatial reasoning capabilities in audio language models, enabling more accurate and interpretable direction and distance estimation through multi-step reasoning.

Abstract: Spatial reasoning is fundamental to auditory perception, yet current audio large language models (ALLMs) largely rely on unstructured binaural cues and single step inference. This limits both perceptual accuracy in direction and distance estimation and the capacity for interpretable reasoning. Recent work such as BAT demonstrates spatial QA with binaural audio, but its reliance on coarse categorical labels (left, right, up, down) and the absence of explicit geometric supervision constrain resolution and robustness. We introduce the $\textbf{Spatial-Acoustic Geometry Encoder (SAGE}$), a geometry-aware audio encoder that aligns binaural acoustic features with 3D spatial structure using panoramic depth images and room-impulse responses at training time, while requiring only audio at inference. Building on this representation, we present $\textbf{OWL}$, an ALLM that integrates $\textbf{SAGE}$ with a spatially grounded chain-of-thought to rationalize over direction-of-arrivals (DoA) and distance estimates. Through curriculum learning from perceptual QA to multi-step reasoning, $\textbf{OWL}$ supports o’clock-level azimuth and DoA estimation. To enable large-scale training and evaluation, we construct and release $\textbf{BiDepth}$, a dataset of over one million QA pairs combining binaural audio with panoramic depth images and room impulse responses across both in-room and out-of-room scenarios. Across two benchmark datasets, our new $\textbf{BiDepth}$ and the public SpatialSoundQA, $\textbf{OWL}$ reduces mean DoA error by $\textbf{11$^{\circ}$}$ through $\textbf{SAGE}$ and improves spatial reasoning QA accuracy by up to $\textbf{25}$% over BAT.

Method: Evaluated five state-of-the-art diarization models across four datasets spanning multiple languages (English, Mandarin, German, Japanese, Spanish) and acoustic conditions, totaling 196.6 hours of audio.

Result: PyannoteAI achieved best performance at 11.2% DER, DiariZen provided competitive open-source alternative at 13.3% DER. Primary failure causes were missed speech segments followed by speaker confusion, especially in high-speaker count settings.

Conclusion: Current diarization systems still struggle with missed speech detection and speaker confusion, particularly in scenarios with multiple speakers, highlighting ongoing challenges in speaker diarization accuracy.

Abstract: Speaker diarization is the task of partitioning audio into segments according to speaker identity, answering the question of “who spoke when” in multi-speaker conversation recordings. While diarization is an essential task for many downstream applications, it remains an unsolved problem. Errors in diarization propagate to downstream systems and cause wide-ranging failures. To this end, we examine exact failure modes by evaluating five state-of-the-art diarization models, across four diarization datasets spanning multiple languages and acoustic conditions. The evaluation datasets consist of 196.6 hours of multilingual audio, including English, Mandarin, German, Japanese, and Spanish. Overall, we find that PyannoteAI achieves the best performance at 11.2% DER, while DiariZen provides a competitive open-source alternative at 13.3% DER. When analyzing failure cases, we find that the primary cause of diarization errors stem from missed speech segments followed by speaker confusion, especially in high-speaker count settings.

Method: Proposed decoupled pruning of attention block’s four layers (query, key, value, output projections) and investigated pruning strategies along head and channel dimensions using Audio Spectrogram Transformer model.

Result: Successfully pruned 50% of attention parameters with less than 1% performance degradation, demonstrating effective model compression.

Conclusion: The proposed attention-specific pruning technique enables significant parameter reduction in Transformer models while maintaining near-original performance.

Abstract: Transformer-based models have become the state of the art across multiple domains, from natural language processing to machine listening, thanks to attention mechanisms. However, the attention layers require a large number of parameters and high-end hardware for both training and inference. We propose a novel pruning technique targeted explicitly at the attention mechanism, where we decouple the pruning of the four layers in the attention block, namely: query, keys, values and outputs’ projection matrices. We also investigate pruning strategies to prune along the head and channel dimensions, and compare the performance of the Audio Spectrogram Transformer (AST) model under different pruning scenarios. Our results show that even by pruning 50% of the attention parameters we incur in performance degradation of less than 1%

Method: Developed a base stereo audio generation model, created a synthetic data pipeline with video analysis, object tracking, and audio synthesis with dynamic panning and distance-based loudness controls, then fine-tuned the base model on this synthetic dataset.

Result: Achieved state-of-the-art semantic accuracy and synchronization, established clear object-audio correspondence, and validated stereo object-awareness through human listening studies showing strong correlation with perception.

Conclusion: This work establishes the first end-to-end framework for stereo object-aware video-to-audio generation, addressing a critical gap and setting a new benchmark in the field.

Abstract: We present StereoFoley, a video-to-audio generation framework that produces semantically aligned, temporally synchronized, and spatially accurate stereo sound at 48 kHz. While recent generative video-to-audio models achieve strong semantic and temporal fidelity, they largely remain limited to mono or fail to deliver object-aware stereo imaging, constrained by the lack of professionally mixed, spatially accurate video-to-audio datasets. First, we develop and train a base model that generates stereo audio from video, achieving state-of-the-art in both semantic accuracy and synchronization. Next, to overcome dataset limitations, we introduce a synthetic data generation pipeline that combines video analysis, object tracking, and audio synthesis with dynamic panning and distance-based loudness controls, enabling spatially accurate object-aware sound. Finally, we fine-tune the base model on this synthetic dataset, yielding clear object-audio correspondence. Since no established metrics exist, we introduce stereo object-awareness measures and validate it through a human listening study, showing strong correlation with perception. This work establishes the first end-to-end framework for stereo object-aware video-to-audio generation, addressing a critical gap and setting a new benchmark in the field.

Method: Adapts SelfClean framework from image to audio domain using self-supervised audio representations to identify data quality issues through ranked review lists in a unified process.

Result: Achieves state-of-the-art ranking performance on ESC-50, GTZAN, and industrial datasets, outperforming issue-specific baselines and enabling significant annotation savings through efficient human review guidance.

Conclusion: The SelfClean framework successfully adapts to audio domain, providing an effective unified approach for identifying multiple data quality issues while reducing annotation costs.

Abstract: Data quality issues such as off-topic samples, near duplicates, and label errors often limit the performance of audio-based systems. This paper addresses these issues by adapting SelfClean, a representation-to-rank data auditing framework, from the image to the audio domain. This approach leverages self-supervised audio representations to identify common data quality issues, creating ranked review lists that surface distinct issues within a single, unified process. The method is benchmarked on the ESC-50, GTZAN, and a proprietary industrial dataset, using both synthetic and naturally occurring corruptions. The results demonstrate that this framework achieves state-of-the-art ranking performance, often outperforming issue-specific baselines and enabling significant annotation savings by efficiently guiding human review.

Method: Integrates Automatic Drum Transcription and One-shot Drum Sample Synthesis in an end-to-end framework, convolving synthesized samples with estimated onsets to reconstruct drum stems and training a DNN on mixture reconstruction.

Result: Experiments on StemGMD dataset show IDM achieves separation quality comparable to state-of-the-art supervised methods that require isolated stems data.

Conclusion: IDM provides an effective alternative to supervised drum separation methods by eliminating the need for isolated stem recordings while maintaining comparable performance.

Abstract: We present the Inverse Drum Machine, a novel approach to Drum Source Separation that leverages an analysis-by-synthesis framework combined with deep learning. Unlike recent supervised methods that require isolated stem recordings for training, our approach is trained on drum mixtures with only transcription annotations. IDM integrates Automatic Drum Transcription and One-shot Drum Sample Synthesis, jointly optimizing these tasks in an end-to-end manner. By convolving synthesized one-shot samples with estimated onsets, akin to a drum machine, we reconstruct the individual drum stems and train a Deep Neural Network on the reconstruction of the mixture. Experiments on the StemGMD dataset demonstrate that IDM achieves separation quality comparable to state-of-the-art supervised methods that require isolated stems data.

Method: Generates counterfactual explanations by making small, meaningful changes to musical score graphs that alter a model’s prediction while ensuring musical coherence. Tailors explanations to musical data structure and avoids unrealistic outputs.

Result: The method offers intuitive insights that can be visualized with standard music tools like Verovio, providing clear explanations for music GNN model decisions.

Conclusion: MUSE-Explainer successfully addresses the interpretability gap in music deep learning by providing human-friendly, musically coherent explanations for GNN model decisions.

Abstract: Interpretability is essential for deploying deep learning models in symbolic music analysis, yet most research emphasizes model performance over explanation. To address this, we introduce MUSE-Explainer, a new method that helps reveal how music Graph Neural Network models make decisions by providing clear, human-friendly explanations. Our approach generates counterfactual explanations by making small, meaningful changes to musical score graphs that alter a model’s prediction while ensuring the results remain musically coherent. Unlike existing methods, MUSE-Explainer tailors its explanations to the structure of musical data and avoids unrealistic or confusing outputs. We evaluate our method on a music analysis task and show it offers intuitive insights that can be visualized with standard music tools such as Verovio.

Method: Developed AudSemThinker with auditory semantics framework inspired by human cognition, and created AudSem dataset using multi-stage pipeline for clean audio-caption pairs.

Result: AudSemThinker outperforms state-of-the-art models across multiple training settings, demonstrating strong semantic audio reasoning capabilities.

Conclusion: The proposed AudSemThinker model and AudSem dataset effectively address semantic reasoning limitations in audio-language models and are publicly released.

Abstract: Audio-language models have shown promising results in various sound understanding tasks, yet they remain limited in their ability to reason over the fine-grained semantics of sound. In this paper, we present AudSemThinker, a model whose reasoning is structured around a framework of auditory semantics inspired by human cognition. To support this, we introduce AudSem, a novel dataset specifically curated for semantic descriptor reasoning in audio-language models. AudSem addresses the persistent challenge of data contamination in zero-shot evaluations by providing a carefully filtered collection of audio samples paired with captions generated through a robust multi-stage pipeline. Our experiments demonstrate that AudSemThinker outperforms state-of-the-art models across multiple training settings, highlighting its strength in semantic audio reasoning. Both AudSemThinker and the AudSem dataset are released publicly.

Method: Uses power set-based data augmentation to expand limited datasets, adapts SepReformer with periodic activations and a composite loss function that handles silent stems effectively.

Result: Achieves state-of-the-art performance on JaCappella dataset for both full-ensemble and subset singer separation, outperforming spectrogram-based baselines and generalizing to realistic mixtures.

Conclusion: The proposed SepACap approach successfully handles varying numbers of singers in a cappella music separation through effective data augmentation and model adaptations.

Abstract: In this work, we study the task of multi-singer separation in a cappella music, where the number of active singers varies across mixtures. To address this, we use a power set-based data augmentation strategy that expands limited multi-singer datasets into exponentially more training samples. To separate singers, we introduce SepACap, an adaptation of SepReformer, a state-of-the-art speaker separation model architecture. We adapt the model with periodic activations and a composite loss function that remains effective when stems are silent, enabling robust detection and separation. Experiments on the JaCappella dataset demonstrate that our approach achieves state-of-the-art performance in both full-ensemble and subset singer separation scenarios, outperforming spectrogram-based baselines while generalizing to realistic mixtures with varying numbers of singers.

Method: Proposed AudioTrust framework with 26 sub-tasks across six dimensions (fairness, hallucination, safety, privacy, robustness, authentication), using curated dataset of 4,420+ audio samples from real-world scenarios and human-validated automated pipelines.

Result: Evaluation of 14 state-of-the-art ALLMs revealed significant limitations when confronted with diverse high-risk audio scenarios, highlighting trustworthiness issues in current models.

Conclusion: AudioTrust provides crucial insights for secure deployment of audio models by systematically identifying and evaluating audio-specific trustworthiness risks that existing frameworks overlook.

Abstract: Audio Large Language Models (ALLMs) have gained widespread adoption, yet their trustworthiness remains underexplored. Existing evaluation frameworks, designed primarily for text, fail to address unique vulnerabilities introduced by audio’s acoustic properties. We identify significant trustworthiness risks in ALLMs arising from non-semantic acoustic cues, including timbre, accent, and background noise, which can manipulate model behavior. We propose AudioTrust, a comprehensive framework for systematic evaluation of ALLM trustworthiness across audio-specific risks. AudioTrust encompasses six key dimensions: fairness, hallucination, safety, privacy, robustness, and authentication. The framework implements 26 distinct sub-tasks using a curated dataset of over 4,420 audio samples from real-world scenarios, including daily conversations, emergency calls, and voice assistant interactions. We conduct comprehensive evaluations across 18 experimental configurations using human-validated automated pipelines. Our evaluation of 14 state-of-the-art open-source and closed-source ALLMs reveals significant limitations when confronted with diverse high-risk audio scenarios, providing insights for secure deployment of audio models. Code and data are available at https://github.com/JusperLee/AudioTrust.

Method: Introduces a method using sparse autoencoders (SAEs) to discover interpretable concepts from transformer model residual streams, with scalable automated labeling and validation pipelines.

Result: Reveals both familiar musical concepts and coherent but uncodified patterns without clear theoretical counterparts. Shows these concepts can be used to steer model generations.

Conclusion: Provides an empirical tool for uncovering organizing principles that traditional analysis methods have missed, improving model transparency while advancing understanding of music structure.

Abstract: The fidelity with which neural networks can now generate content such as music presents a scientific opportunity: these systems appear to have learned implicit theories of such content’s structure through statistical learning alone. This offers a potentially new lens on theories of human-generated media. When internal representations align with traditional constructs (e.g. chord progressions in music), they show how such categories can emerge from statistical regularities; when they diverge, they expose limits of existing frameworks and patterns we may have overlooked but that nonetheless carry explanatory power. In this paper, focusing on music generators, we introduce a method for discovering interpretable concepts using sparse autoencoders (SAEs), extracting interpretable features from the residual stream of a transformer model. We make this approach scalable and evaluable using automated labeling and validation pipelines. Our results reveal both familiar musical concepts and coherent but uncodified patterns lacking clear counterparts in theory or language. As an extension, we show such concepts can be used to steer model generations. Beyond improving model transparency, our work provides an empirical tool for uncovering organizing principles that have eluded traditional methods of analysis and synthesis.

Method: Developed Universal Speech Enhancement Mamba (USEMamba), a state-space model for long-range sequence modeling, time-frequency structured processing, and sampling frequency-independent feature extraction, using primarily regression-based modeling with a generative variant for packet loss and bandwidth extension tasks.

Result: Achieved 2nd place in Track 1 during the blind test phase, demonstrating strong generalization across diverse conditions despite being trained on only a subset of the full training data.

Conclusion: USEMamba effectively handles universal speech enhancement across multiple distortions and languages, with regression-based modeling working well for most tasks and generative approaches being more suitable for content inference tasks like packet loss and bandwidth extension.

Abstract: The Interspeech 2025 URGENT Challenge aimed to advance universal, robust, and generalizable speech enhancement by unifying speech enhancement tasks across a wide variety of conditions, including seven different distortion types and five languages. We present Universal Speech Enhancement Mamba (USEMamba), a state-space speech enhancement model designed to handle long-range sequence modeling, time-frequency structured processing, and sampling frequency-independent feature extraction. Our approach primarily relies on regression-based modeling, which performs well across most distortions. However, for packet loss and bandwidth extension, where missing content must be inferred, a generative variant of the proposed USEMamba proves more effective. Despite being trained on only a subset of the full training data, USEMamba achieved 2nd place in Track 1 during the blind test phase, demonstrating strong generalization across diverse conditions.

Method: Adapt U-Net architecture from image colorization to MIDI velocity prediction by conceptualizing MIDI data as images, using window attention and developing a custom loss function to handle sparse MIDI-converted images.

Result: The proposed method outperforms previous approaches on MAESTRO v3 and SMD datasets in both quantitative metrics and qualitative listening tests, though limited to piano data due to dataset constraints.

Conclusion: U-Net architecture successfully applied to MIDI velocity prediction, demonstrating improved music expressiveness through velocity parameter enhancement with better performance than existing methods.

Abstract: Modern music producers commonly use MIDI (Musical Instrument Digital Interface) to store their musical compositions. However, MIDI files created with digital software may lack the expressive characteristics of human performances, essentially leaving the velocity parameter - a control for note loudness - undefined, which defaults to a flat value. The task of filling MIDI velocity is termed MIDI velocity prediction, which uses regression models to enhance music expressiveness by adjusting only this parameter. In this paper, we introduce the U-Net, a widely adopted architecture in image colorization, to this task. By conceptualizing MIDI data as images, we adopt window attention and develop a custom loss function to address the sparsity of MIDI-converted images. Current dataset availability restricts our experiments to piano data. Evaluated on the MAESTRO v3 and SMD datasets, our proposed method for filling MIDI velocity outperforms previous approaches in both quantitative metrics and qualitative listening tests.

Method: Leveraged open source datasets to create AHEAD-DS with standardized labels, and developed YAMNet+ using transfer learning from pretrained YAMNet for edge device deployment.

Result: YAMNet+ achieved 0.83 mAP and 0.93 accuracy on AHEAD-DS test set. Deployed on Android smartphone with 50ms model loading latency and 30ms per second audio processing.

Conclusion: Successfully created standardized dataset and baseline model for hearing aid scene recognition, demonstrating real-time edge deployment capability on modest hardware.

Abstract: Scene recognition of audiologically relevant environments is important for hearing aids; however, it is challenging, in part because of the limitations of existing datasets. Datasets often lack public accessibility, completeness, or audiologically relevant labels, hindering systematic comparison of machine learning models. Deploying these models on resource-constrained edge devices presents another challenge. Our solution is two-fold: we leverage several open source datasets to create AHEAD-DS, a dataset designed for scene recognition of audiologically relevant environments, and introduce YAMNet+, a sound recognition model. AHEAD-DS aims to provide a standardised, publicly available dataset with consistent labels relevant to hearing aids, facilitating model comparison. YAMNet+ is designed for deployment on edge devices like smartphones connected to hearing devices, such as hearing aids and wireless earphones with hearing aid functionality; serving as a baseline model for sound-based scene recognition. YAMNet+ achieved a mean average precision of 0.83 and accuracy of 0.93 on the testing set of AHEAD-DS across fourteen categories of audiologically relevant environments. We found that applying transfer learning from the pretrained YAMNet model was essential. We demonstrated real-time sound-based scene recognition capabilities on edge devices by deploying YAMNet+ to an Android smartphone. Even with a Google Pixel 3 (a phone with modest specifications, released in 2018), the model processes audio with approximately 50ms of latency to load the model, and an approximate linear increase of 30ms per 1 second of audio. Our website and code https://github.com/Australian-Future-Hearing-Initiative .

Method: Integrates full-face visual cues with a Mamba-based temporal backbone to leverage spatiotemporal visual information for more accurate target speech extraction in challenging conditions.

Result: Outperforms other monaural baselines in speech intelligibility (STOI), perceptual quality (PESQ), and non-intrusive quality (UTMOS), and achieves 1st place on the monaural leaderboard of the AVSEC-4 Challenge.

Conclusion: AVSEMamba successfully addresses the limitations of previous Mamba-based models by incorporating visual information, enabling effective speech enhancement in complex multi-speaker environments.

Abstract: Recent Mamba-based models have shown promise in speech enhancement by efficiently modeling long-range temporal dependencies. However, models like Speech Enhancement Mamba (SEMamba) remain limited to single-speaker scenarios and struggle in complex multi-speaker environments such as the cocktail party problem. To overcome this, we introduce AVSEMamba, an audio-visual speech enhancement model that integrates full-face visual cues with a Mamba-based temporal backbone. By leveraging spatiotemporal visual information, AVSEMamba enables more accurate extraction of target speech in challenging conditions. Evaluated on the AVSEC-4 Challenge development and blind test sets, AVSEMamba outperforms other monaural baselines in speech intelligibility (STOI), perceptual quality (PESQ), and non-intrusive quality (UTMOS), and achieves \textbf{1st place} on the monaural leaderboard.

Method: Uses MeanFlow to predict an average-velocity field for one-step latent refinement and conditions on self-supervised learning representations instead of VAE latents.

Result: Achieves state-of-the-art perceptual quality and competitive intelligibility on Interspeech 2020 DNS Challenge datasets, with significantly lower real-time factor and model size.

Conclusion: MeanFlowSE provides an efficient generative speech enhancement solution suitable for practical real-time applications.

Abstract: Speech enhancement (SE) recovers clean speech from noisy signals and is vital for applications such as telecommunications and automatic speech recognition (ASR). While generative approaches achieve strong perceptual quality, they often rely on multi-step sampling (diffusion/flow-matching) or large language models, limiting real-time deployment. To mitigate these constraints, we present MeanFlowSE, a one-step generative SE framework. It adopts MeanFlow to predict an average-velocity field for one-step latent refinement and conditions the model on self-supervised learning (SSL) representations rather than VAE latents. This design accelerates inference and provides robust acoustic-semantic guidance during training. In the Interspeech 2020 DNS Challenge blind test set and simulated test set, MeanFlowSE attains state-of-the-art (SOTA) level perceptual quality and competitive intelligibility while significantly lowering both real-time factor (RTF) and model size compared with recent generative competitors, making it suitable for practical use. The code will be released upon publication at https://github.com/Hello3orld/MeanFlowSE.

Method: Proposed VioPTT - an end-to-end model that directly transcribes violin playing techniques, pitch onset and offset. Created MOSA-VPT synthetic dataset to avoid manual labeling.

Result: The model showed strong generalization to real-world violin recordings and achieved state-of-the-art transcription performance.

Conclusion: VioPTT is the first unified framework that jointly combines violin transcription and playing technique prediction, successfully capturing expressive nuances beyond basic pitch information.

Abstract: While automatic music transcription is well-established in music information retrieval, most models are limited to transcribing pitch and timing information from audio, and thus omit crucial expressive and instrument-specific nuances. One example is playing technique on the violin, which affords its distinct palette of timbres for maximal emotional impact. Here, we propose VioPTT (Violin Playing Technique-aware Transcription), a lightweight, end-to-end model that directly transcribes violin playing technique in addition to pitch onset and offset. Furthermore, we release MOSA-VPT, a novel, high-quality synthetic violin playing technique dataset to circumvent the need for manually labeled annotations. Leveraging this dataset, our model demonstrated strong generalization to real-world note-level violin technique recordings in addition to achieving state-of-the-art transcription performance. To our knowledge, VioPTT is the first to jointly combine violin transcription and playing technique prediction within a unified framework.

Method: Proposes SOLD - a latent diffusion model that generates molecules in a latent space derived from 1D SELFIES strings, conditioned on target proteins. Also trains a SELFIES transformer and introduces a new loss balancing method for multi-task models.

Result: The model generates high-affinity molecules for target proteins in a simple and efficient way.

Conclusion: SOLD provides an effective approach for protein-conditioned molecule generation with room for future improvements through additional data.

Abstract: Recently, machine learning has made a significant impact on de novo drug design. However, current approaches to creating novel molecules conditioned on a target protein typically rely on generating molecules directly in the 3D conformational space, which are often slow and overly complex. In this work, we propose SOLD (SELFIES-based Objective-driven Latent Diffusion), a novel latent diffusion model that generates molecules in a latent space derived from 1D SELFIES strings and conditioned on a target protein. In the process, we also train an innovative SELFIES transformer and propose a new way to balance losses when training multi-task machine learning models.Our model generates high-affinity molecules for the target protein in a simple and efficient way, while also leaving room for future improvements through the addition of more data.

Method: Formulate the problem as non-convex stochastic optimization and POMDP, then solve using MADRL with hybrid actor network combining CNN for feature extraction and KAN for state-action dependencies, based on HAPPO algorithm.

Result: Extensive numerical results show significant improvements in the amount of processed sensing data compared to other benchmark methods.

Conclusion: The proposed MADRL framework with hybrid actor network effectively addresses the joint optimization challenges in UAV-assisted MCS and achieves superior performance in maximizing processed sensing data.

Abstract: Unmanned aerial vehicles (UAVs)-assisted mobile crowdsensing (MCS) has emerged as a promising paradigm for data collection. However, challenges such as spectrum scarcity, device heterogeneity, and user mobility hinder efficient coordination of sensing, communication, and computation. To tackle these issues, we propose a joint optimization framework that integrates time slot partition for sensing, communication, and computation phases, resource allocation, and UAV 3D trajectory planning, aiming to maximize the amount of processed sensing data. The problem is formulated as a non-convex stochastic optimization and further modeled as a partially observable Markov decision process (POMDP) that can be solved by multi-agent deep reinforcement learning (MADRL) algorithm. To overcome the limitations of conventional multi-layer perceptron (MLP) networks, we design a novel MADRL algorithm with hybrid actor network. The newly developed method is based on heterogeneous agent proximal policy optimization (HAPPO), empowered by convolutional neural networks (CNN) for feature extraction and Kolmogorov-Arnold networks (KAN) to capture structured state-action dependencies. Extensive numerical results demonstrate that our proposed method achieves significant improvements in the amount of processed sensing data when compared with other benchmarks.

Method: Proposes Vision-Language Hierarchical Semantic Alignment (VLHSA) module that aligns visual patches with textual descriptions through multi-level semantic matching, using dual visual encoders with language features for cross-modal reasoning.

Result: Significantly outperforms state-of-the-art models across various datasets, achieving 14.2 percentage point gain in piece accuracy. Ablation studies confirm VLHSA module’s critical role.

Conclusion: Establishes a new paradigm for jigsaw puzzle solving by incorporating multimodal semantic insights through vision-language integration.

Abstract: Jigsaw puzzle solving remains challenging in computer vision, requiring an understanding of both local fragment details and global spatial relationships. While most traditional approaches only focus on visual cues like edge matching and visual coherence, few methods explore natural language descriptions for semantic guidance in challenging scenarios, especially for eroded gap puzzles. We propose a vision-language framework that leverages textual context to enhance puzzle assembly performance. Our approach centers on the Vision-Language Hierarchical Semantic Alignment (VLHSA) module, which aligns visual patches with textual descriptions through multi-level semantic matching from local tokens to global context. Also, a multimodal architecture that combines dual visual encoders with language features for cross-modal reasoning is integrated into this module. Experiments demonstrate that our method significantly outperforms state-of-the-art models across various datasets, achieving substantial improvements, including a 14.2 percentage point gain in piece accuracy. Ablation studies confirm the critical role of the VLHSA module in driving improvements over vision-only approaches. Our work establishes a new paradigm for jigsaw puzzle solving by incorporating multimodal semantic insights.

Method: SLS maintains a sliding buffer of top-K logits, performs on-the-fly SVD to identify dominant spectral directions, and adaptively rescales logits based on entropy and logit gap statistics, only activating when uncertainty is high.

Result: SLS consistently outperforms existing baseline methods, achieving superior accuracy in mathematical, coding, and scientific reasoning tasks on multiple public benchmarks.

Conclusion: SLS effectively sharpens output distributions while preserving contextual consistency, providing a computationally efficient approach to improve LLM reliability during inference.

Abstract: Entropy-based inference methods have gained traction for improving the reliability of Large Language Models (LLMs). However, many existing approaches, such as entropy minimization techniques, suffer from high computational overhead and fail to leverage historical token context effectively. To address these limitations, we propose Spectral Logit Sculpting (SLS), a lightweight inference-time optimization method that dynamically modulates token distributions using spectral and entropic properties of recent logits. SLS maintains a sliding buffer of top-K logits, performs on-the-fly Singular Value Decomposition (SVD) to identify dominant spectral directions, and adaptively rescales logits based on both entropy and logit gap statistics–only activating when uncertainty is high. Without updating any model parameters, SLS effectively sharpens the output distribution while preserving contextual consistency. Experimental results on multiple public benchmarks demonstrate that SLS consistently outperforms existing baseline methods, achieving superior accuracy in mathematical, coding, and scientific reasoning tasks.

Method: Developed a fully polynomial-friendly GCN encoder and a novel polynomial-based contrastive loss function to enable HE-compatible self-supervised learning on graphs.

Result: Poly-GRACE achieves competitive performance with standard non-private baselines across Cora, CiteSeer, and PubMed datasets, with superior performance on CiteSeer.

Conclusion: This work represents a significant advancement towards practical and high-performance privacy-preserving graph representation learning.

Abstract: Self-supervised learning (SSL) has emerged as a powerful paradigm for learning representations on graph data without requiring manual labels. However, leading SSL methods like GRACE are fundamentally incompatible with privacy-preserving technologies such as Homomorphic Encryption (HE) due to their reliance on non-polynomial operations. This paper introduces Poly-GRACE, a novel framework for HE-compatible self-supervised learning on graphs. Our approach consists of a fully polynomial-friendly Graph Convolutional Network (GCN) encoder and a novel, polynomial-based contrastive loss function. Through experiments on three benchmark datasets – Cora, CiteSeer, and PubMed – we demonstrate that Poly-GRACE not only enables private pre-training but also achieves performance that is highly competitive with, and in the case of CiteSeer, superior to the standard non-private baseline. Our work represents a significant step towards practical and high-performance privacy-preserving graph representation learning.

Method: Extends Riemannian SGD to Hyperbolic Adam optimizer, applies hyperbolic time-discretization of Langevin dynamics for training diffusion models on hyperbolic manifolds.

Result: Hyperbolic optimization methods achieve faster convergence on certain datasets without sacrificing generative quality in diffusion models.

Conclusion: Hyperbolic optimization methods provide practical benefits for training models, particularly in early stages when parameters are far from optimum, and can be applied beyond hyperbolic settings to Euclidean and other non-Euclidean spaces.

Abstract: This work explores optimization methods on hyperbolic manifolds. Building on Riemannian optimization principles, we extend the Hyperbolic Stochastic Gradient Descent (a specialization of Riemannian SGD) to a Hyperbolic Adam optimizer. While these methods are particularly relevant for learning on the Poincar'e ball, they may also provide benefits in Euclidean and other non-Euclidean settings, as the chosen optimization encourages the learning of Poincar'e embeddings. This representation, in turn, accelerates convergence in the early stages of training, when parameters are far from the optimum. As a case study, we train diffusion models using the hyperbolic optimization methods with hyperbolic time-discretization of the Langevin dynamics, and show that they achieve faster convergence on certain datasets without sacrificing generative quality.

Method: Conducted over 100 controlled experiments using 500,000 GPU-hours, analyzing the full MLLM pipeline across model scales, data categories, and adaptation setups. Introduced MLE-Bench for evaluation.

Result: Found that visual reasoning ability primarily develops from reasoning-centric data (code, math) and scales progressively, while perception emerges from broad corpora and is more sensitive to vision components. Text descriptions are crucial but saturate quickly.

Conclusion: Provides a data-centric recipe for pre-training vision-aware LLMs and demonstrates how to deliberately cultivate visual priors from language pre-training, enabling more efficient multimodal LLM development.

Abstract: Large Language Models (LLMs), despite being trained on text alone, surprisingly develop rich visual priors. These priors allow latent visual capabilities to be unlocked for vision tasks with a relatively small amount of multimodal data, and in some cases, to perform visual tasks without ever having seen an image. Through systematic analysis, we reveal that visual priors-the implicit, emergent knowledge about the visual world acquired during language pre-training-are composed of separable perception and reasoning priors with unique scaling trends and origins. We show that an LLM’s latent visual reasoning ability is predominantly developed by pre-training on reasoning-centric data (e.g., code, math, academia) and scales progressively. This reasoning prior acquired from language pre-training is transferable and universally applicable to visual reasoning. In contrast, a perception prior emerges more diffusely from broad corpora, and perception ability is more sensitive to the vision encoder and visual instruction tuning data. In parallel, text describing the visual world proves crucial, though its performance impact saturates rapidly. Leveraging these insights, we propose a data-centric recipe for pre-training vision-aware LLMs and verify it in 1T token scale pre-training. Our findings are grounded in over 100 controlled experiments consuming 500,000 GPU-hours, spanning the full MLLM construction pipeline-from LLM pre-training to visual alignment and supervised multimodal fine-tuning-across five model scales, a wide range of data categories and mixtures, and multiple adaptation setups. Along with our main findings, we propose and investigate several hypotheses, and introduce the Multi-Level Existence Bench (MLE-Bench). Together, this work provides a new way of deliberately cultivating visual priors from language pre-training, paving the way for the next generation of multimodal LLMs.

Method: Introduce a multi-level diagnosis and evaluation framework focusing on three main factors: key variables, relationships, and decision boundary values for predicting target classes.

Result: LLM robustness varies significantly across different datasets, and high-quality LLM-generated features can improve few-shot prediction performance by up to 10.52%.

Conclusion: This work opens a new direction for assessing and enhancing the reliability of LLM-driven feature engineering in various domains.

Abstract: Recent advancements in large language models (LLMs) have shown promise in feature engineering for tabular data, but concerns about their reliability persist, especially due to variability in generated outputs. We introduce a multi-level diagnosis and evaluation framework to assess the robustness of LLMs in feature engineering across diverse domains, focusing on the three main factors: key variables, relationships, and decision boundary values for predicting target classes. We demonstrate that the robustness of LLMs varies significantly over different datasets, and that high-quality LLM-generated features can improve few-shot prediction performance by up to 10.52%. This work opens a new direction for assessing and enhancing the reliability of LLM-driven feature engineering in various domains.

Method: Treat rainfall forecasting as a long-tailed learning problem and introduce DPSformer with a high-resolution branch to enrich representation of heavy rainfall events.

Result: For heavy rainfall events ≥50mm/6h, DPSformer improves CSI from 0.012 to 0.067. For top 1% heavy rainfall events, FSS exceeds 0.45, outperforming existing methods.

Conclusion: DPSformer establishes an effective long-tailed paradigm for heavy rainfall prediction, offering practical tools to enhance early warning systems and mitigate societal impacts of extreme weather.

Abstract: Accurate and timely forecasting of heavy rainfall remains a critical challenge for modern society. Precipitation exhibits a highly imbalanced distribution: most observations record no or light rain, while heavy rainfall events are rare. Such an imbalanced distribution obstructs deep learning models from effectively predicting heavy rainfall events. To address this challenge, we treat rainfall forecasting explicitly as a long-tailed learning problem, identifying the insufficient representation of heavy rainfall events as the primary barrier to forecasting accuracy. Therefore, we introduce DPSformer, a long-tail-aware model that enriches representation of heavy rainfall events through a high-resolution branch. For heavy rainfall events $ \geq $ 50 mm/6 h, DPSformer lifts the Critical Success Index (CSI) of a baseline Numerical Weather Prediction (NWP) model from 0.012 to 0.067. For the top 1% coverage of heavy rainfall events, its Fraction Skill Score (FSS) exceeds 0.45, surpassing existing methods. Our work establishes an effective long-tailed paradigm for heavy rainfall prediction, offering a practical tool to enhance early warning systems and mitigate the societal impacts of extreme weather events.

Method: Uses Spatial-Aligned Attention (SAA) to align global and regional spatial distributions and adaptively refine boundaries, plus Temporal Mixture-of-Experts (TMoE) to dynamically route atmospheric variables from different months to specialized experts.

Result: Experimental results show consistent superiority over state-of-the-art methods across global forecasting, regional forecasting, extreme event prediction, and ensemble forecasting tasks.

Conclusion: STCast effectively addresses the limitations of static regional boundaries through adaptive optimization and dynamic temporal allocation, achieving superior performance across multiple weather forecasting tasks.

Abstract: To gain finer regional forecasts, many works have explored the regional integration from the global atmosphere, e.g., by solving boundary equations in physics-based methods or cropping regions from global forecasts in data-driven methods. However, the effectiveness of these methods is often constrained by static and imprecise regional boundaries, resulting in poor generalization ability. To address this issue, we propose Spatial-Temporal Weather Forecasting (STCast), a novel AI-driven framework for adaptive regional boundary optimization and dynamic monthly forecast allocation. Specifically, our approach employs a Spatial-Aligned Attention (SAA) mechanism, which aligns global and regional spatial distributions to initialize boundaries and adaptively refines them based on attention-derived alignment patterns. Furthermore, we design a Temporal Mixture-of-Experts (TMoE) module, where atmospheric variables from distinct months are dynamically routed to specialized experts using a discrete Gaussian distribution, enhancing the model’s ability to capture temporal patterns. Beyond global and regional forecasting, we evaluate our STCast on extreme event prediction and ensemble forecasting. Experimental results demonstrate consistent superiority over state-of-the-art methods across all four tasks.

Method: Used TD-MPC2 (model-based RL algorithm) to train a single RL agent across multiple endovascular navigation tasks in ten real patient vasculatures, comparing against Soft Actor-Critic (SAC).

Result: TD-MPC2 significantly outperformed SAC with 65% mean success rate vs 37%, with improved path ratio. However, TD-MPC2 had increased procedure times, indicating trade-off between success and speed.

Conclusion: World models show potential for improving autonomous endovascular navigation and provide foundation for future research in generalizable AI-driven robotic interventions.

Abstract: Autonomous navigation for mechanical thrombectomy (MT) remains a critical challenge due to the complexity of vascular anatomy and the need for precise, real-time decision-making. Reinforcement learning (RL)-based approaches have demonstrated potential in automating endovascular navigation, but current methods often struggle with generalization across multiple patient vasculatures and long-horizon tasks. We propose a world model for autonomous endovascular navigation using TD-MPC2, a model-based RL algorithm. We trained a single RL agent across multiple endovascular navigation tasks in ten real patient vasculatures, comparing performance against the state-of-the-art Soft Actor-Critic (SAC) method. Results indicate that TD-MPC2 significantly outperforms SAC in multi-task learning, achieving a 65% mean success rate compared to SAC’s 37%, with notable improvements in path ratio. TD-MPC2 exhibited increased procedure times, suggesting a trade-off between success rate and execution speed. These findings highlight the potential of world models for improving autonomous endovascular navigation and lay the foundation for future research in generalizable AI-driven robotic interventions.

Method: Decouples market processing from allocation decisions using TKANs, VSNs, and multi-head attention for feature extraction, with independent allocation networks for different execution scenarios.

Result: Achieves superior execution performance in intraday cryptocurrency and multi-day equity trading compared to traditional benchmarks by optimizing paths within flexible time constraints.

Conclusion: LEMs provide a unified framework for diverse execution scenarios with significant operational advantages over asset-specific approaches.

Abstract: This paper introduces Large Execution Models (LEMs), a novel deep learning framework that extends transformer-based architectures to address complex execution problems with flexible time boundaries and multiple execution constraints. Building upon recent advances in neural VWAP execution strategies, LEMs generalize the approach from fixed-duration orders to scenarios where execution duration is bounded between minimum and maximum time horizons, similar to share buyback contract structures. The proposed architecture decouples market information processing from execution allocation decisions: a common feature extraction pipeline using Temporal Kolmogorov-Arnold Networks (TKANs), Variable Selection Networks (VSNs), and multi-head attention mechanisms processes market data to create informational context, while independent allocation networks handle the specific execution logic for different scenarios (fixed quantity vs. fixed notional, buy vs. sell orders). This architectural separation enables a unified model to handle diverse execution objectives while leveraging shared market understanding across scenarios. Through comprehensive empirical evaluation on intraday cryptocurrency markets and multi-day equity trading using DOW Jones constituents, we demonstrate that LEMs achieve superior execution performance compared to traditional benchmarks by dynamically optimizing execution paths within flexible time constraints. The unified model architecture enables deployment across different execution scenarios (buy/sell orders, varying duration boundaries, volume/notional targets) through a single framework, providing significant operational advantages over asset-specific approaches.

Method: Used L12(211) orthogonal array to evaluate eight hyperparameters across twelve configurations. Developed five approaches using Signal-to-Noise ratio analysis to optimize multiple objectives simultaneously, including a novel logarithmic scaling technique to unify conflicting metrics.

Result: Approach 3 achieved optimal performance with 98.84% training accuracy and 86.25% validation accuracy. Learning rate was identified as the most influential parameter, followed by image size and activation function.

Conclusion: Taguchi methodology provides an effective systematic approach for CNN hyperparameter optimization, offering clear guidance for parameter prioritization and achieving high accuracy while maintaining minimal loss values.

Abstract: The optimization of hyperparameters in convolutional neural networks (CNNs) remains a challenging and computationally expensive process, often requiring extensive trial-and-error approaches or exhaustive grid searches. This study introduces the application of Taguchi Design of Experiments methodology, a statistical optimization technique traditionally used in quality engineering, to systematically optimize CNN hyperparameters for professional boxing action recognition. Using an L12(211) orthogonal array, eight hyperparameters including image size, color mode, activation function, learning rate, rescaling, shuffling, vertical flip, and horizontal flip were systematically evaluated across twelve experimental configurations. To address the multi-objective nature of machine learning optimization, five different approaches were developed to simultaneously optimize training accuracy, validation accuracy, training loss, and validation loss using Signal-to-Noise ratio analysis. The study employed a novel logarithmic scaling technique to unify conflicting metrics and enable comprehensive multi-quality assessment within the Taguchi framework. Results demonstrate that Approach 3, combining weighted accuracy metrics with logarithmically transformed loss functions, achieved optimal performance with 98.84% training accuracy and 86.25% validation accuracy while maintaining minimal loss values. The Taguchi analysis revealed that learning rate emerged as the most influential parameter, followed by image size and activation function, providing clear guidance for hyperparameter prioritization in CNN optimization.

Method: Proposes CoA-LoRA with a configuration-aware model that maps quantization configurations to low-rank adjustments, trained using Pareto-based configuration search to optimize the training set.

Result: Achieves comparable or superior performance to state-of-the-art methods that require separate fine-tuning per configuration, with no additional time cost.

Conclusion: CoA-LoRA provides an efficient solution for deploying compressed models on heterogeneous edge devices without the computational burden of configuration-wise fine-tuning.

Abstract: As increasingly large pre-trained models are released, deploying them on edge devices for privacy-preserving applications requires effective compression. Recent works combine quantization with the fine-tuning of high-precision LoRA adapters, which can substantially reduce model size while mitigating the accuracy loss from quantization. However, edge devices have inherently heterogeneous capabilities, while performing configuration-wise fine-tuning for every quantization setting is computationally prohibitive. In this paper, we propose CoA-LoRA, a method that dynamically adjusts the LoRA adapter to arbitrary quantization configurations (i.e., the per-layer bit-width choices of a pre-trained model) without requiring repeated fine-tuning. This is accomplished via a configuration-aware model that maps each configuration to its low-rank adjustments. The effectiveness of this model critically depends on the training configuration set, a collection of configurations chosen to cover different total bit-width budgets. However, constructing a high-quality configuration set is non-trivial. We therefore design a Pareto-based configuration search that iteratively optimizes the training configuration set, yielding more precise low-rank adjustments. Our experiments demonstrate that, unlike the state-of-the-art methods that require fine-tuning a separate LoRA adapter for each configuration, CoA-LoRA incurs no additional time cost while achieving comparable or even superior performance to those methods.

Method: Clusters correlation coefficients between variables to create partitions that preserve relationships, then runs anomaly detection algorithms locally on each variable subset.

Result: Significant improvement in anomaly detection performance demonstrated through experiments on both synthetic and real datasets from literature.

Conclusion: The PARADISE approach is relevant and effective for multivariate time series anomaly detection by maintaining inter-variable relations through intelligent partitioning.

Abstract: In this article, we suggest a novel non-supervised partition based anomaly detection method for anomaly detection in multivariate time series called PARADISE. This methodology creates a partition of the variables of the time series while ensuring that the inter-variable relations remain untouched. This partitioning relies on the clustering of multiple correlation coefficients between variables to identify subsets of variables before executing anomaly detection algorithms locally for each of those subsets. Through multiple experimentations done on both synthetic and real datasets coming from the literature, we show the relevance of our approach with a significant improvement in anomaly detection performance.

Method: Proposes a multi-agent reinforcement learning (MARL) framework based on non-zero-sum Markov game with random utility models to capture passenger decision making, implemented through a parametrisable simulator RailPricing-RL.

Result: Experimental results validate the framework, showing how user preferences affect MARL performance and how pricing policies influence passenger choices, utility, and system dynamics.

Conclusion: The study provides a foundation for advancing dynamic pricing strategies in railway systems that align profitability with system-wide efficiency and supports future research on optimising pricing policies.

Abstract: This paper addresses a critical challenge in the high-speed passenger railway industry: designing effective dynamic pricing strategies in the context of competing and cooperating operators. To address this, a multi-agent reinforcement learning (MARL) framework based on a non-zero-sum Markov game is proposed, incorporating random utility models to capture passenger decision making. Unlike prior studies in areas such as energy, airlines, and mobile networks, dynamic pricing for railway systems using deep reinforcement learning has received limited attention. A key contribution of this paper is a parametrisable and versatile reinforcement learning simulator designed to model a variety of railway network configurations and demand patterns while enabling realistic, microscopic modelling of user behaviour, called RailPricing-RL. This environment supports the proposed MARL framework, which models heterogeneous agents competing to maximise individual profits while fostering cooperative behaviour to synchronise connecting services. Experimental results validate the framework, demonstrating how user preferences affect MARL performance and how pricing policies influence passenger choices, utility, and overall system dynamics. This study provides a foundation for advancing dynamic pricing strategies in railway systems, aligning profitability with system-wide efficiency, and supporting future research on optimising pricing policies.

Method: Systematically vary model complexity along two orthogonal axes (learner capacity and ensemble size) using Mycobacterium tuberculosis resistance prediction task and synthetic benchmark.

Result: Double descent consistently emerges only when complexity is scaled jointly across both axes. When either axis is held fixed, generalization reverts to classical U- or L-shaped patterns.

Conclusion: Model complexity should be treated as multidimensional when analyzing generalization behavior, supporting the unfolding hypothesis that attributes double descent to projection of distinct regimes onto a single complexity axis.

Abstract: Classical learning theory describes a well-characterised U-shaped relationship between model complexity and prediction error, reflecting a transition from underfitting in underparameterised regimes to overfitting as complexity grows. Recent work, however, has introduced the notion of a second descent in test error beyond the interpolation threshold-giving rise to the so-called double descent phenomenon. While double descent has been studied extensively in the context of deep learning, it has also been reported in simpler models, including decision trees and gradient boosting. In this work, we revisit these claims through the lens of classical machine learning applied to a biological classification task: predicting isoniazid resistance in Mycobacterium tuberculosis using whole-genome sequencing data. We systematically vary model complexity along two orthogonal axes-learner capacity (e.g., Pleaf, Pboost) and ensemble size (i.e., Pens)-and show that double descent consistently emerges only when complexity is scaled jointly across these axes. When either axis is held fixed, generalisation behaviour reverts to classical U- or L-shaped patterns. These results are replicated on a synthetic benchmark and support the unfolding hypothesis, which attributes double descent to the projection of distinct generalisation regimes onto a single complexity axis. Our findings underscore the importance of treating model complexity as a multidimensional construct when analysing generalisation behaviour. All code and reproducibility materials are available at: https://github.com/guillermocomesanacimadevila/Demystifying-Double-Descent-in-ML.

Method: L2C trains neural networks to score variable-value assignments for conditioning using training data generated from MPE solver search traces. The learned heuristic integrates with search algorithms as conditioning strategies or branching policies.

Result: Experiments on challenging MPE queries with high-treewidth PGMs show that L2C significantly reduces search space while maintaining or improving solution quality compared to state-of-the-art methods.

Conclusion: L2C provides an effective data-driven approach for accelerating MPE inference through learned conditioning heuristics, demonstrating scalability and performance improvements over existing methods.

Abstract: We introduce learning to condition (L2C), a scalable, data-driven framework for accelerating Most Probable Explanation (MPE) inference in Probabilistic Graphical Models (PGMs), a fundamentally intractable problem. L2C trains a neural network to score variable-value assignments based on their utility for conditioning, given observed evidence. To facilitate supervised learning, we develop a scalable data generation pipeline that extracts training signals from the search traces of existing MPE solvers. The trained network serves as a heuristic that integrates with search algorithms, acting as a conditioning strategy prior to exact inference or as a branching and node selection policy within branch-and-bound solvers. We evaluate L2C on challenging MPE queries involving high-treewidth PGMs. Experiments show that our learned heuristic significantly reduces the search space while maintaining or improving solution quality over state-of-the-art methods.

Method: Dynamic Ensemble Selection (DES) with clustering approach for multi-class computer vision tasks, implemented as TinyDES-Clustering library optimized for embedded systems.

Result: Larger classifier pools improve accuracy but increase average inference time on TinyML devices.

Conclusion: DES-Clustering enables adjustable classification accuracy with corresponding trade-offs in latency and energy consumption per inference for TinyML applications.

Abstract: The recent progress in TinyML technologies triggers the need to address the challenge of balancing inference time and classification quality. TinyML systems are defined by specific constraints in computation, memory and energy. These constraints emphasize the need for specialized optimization techniques when implementing Machine Learning (ML) applications on such platforms. While deep neural networks are widely used in TinyML, the exploration of Dynamic Ensemble Selection (DES) methods is also beneficial. This study examines a DES-Clustering approach for a multi-class computer vision task within TinyML systems. This method allows for adjusting classification accuracy, thereby affecting latency and energy consumption per inference. We implemented the TinyDES-Clustering library, optimized for embedded system limitations. Experiments have shown that a larger pool of classifiers for dynamic selection improves classification accuracy, and thus leads to an increase in average inference time on the TinyML device.

Method: Uses Reynolds number-based scaling, physics-based domain decomposition, cluster-based flow representation for subdomains, information entropy correlation between subdomains, and multi-variate probability functions to relate sensor input to velocity estimates.

Result: Demonstrated successfully using drone flight paths through a three-building cluster, showing the framework can handle complex flows and optimize sensor placement for minimal uncertainty.

Conclusion: The framework provides a scalable approach for urban flow estimation with applications for complete city modeling and weather integration, overcoming limitations of traditional flow estimators through its three key innovations.

Abstract: We propose a physics-informed machine-learned framework for sensor-based flow estimation for drone trajectories in complex urban terrain. The input is a rich set of flow simulations at many wind conditions. The outputs are velocity and uncertainty estimates for a target domain and subsequent sensor optimization for minimal uncertainty. The framework has three innovations compared to traditional flow estimators. First, the algorithm scales proportionally to the domain complexity, making it suitable for flows that are too complex for any monolithic reduced-order representation. Second, the framework extrapolates beyond the training data, e.g., smaller and larger wind velocities. Last, and perhaps most importantly, the sensor location is a free input, significantly extending the vast majority of the literature. The key enablers are (1) a Reynolds number-based scaling of the flow variables, (2) a physics-based domain decomposition, (3) a cluster-based flow representation for each subdomain, (4) an information entropy correlating the subdomains, and (5) a multi-variate probability function relating sensor input and targeted velocity estimates. This framework is demonstrated using drone flight paths through a three-building cluster as a simple example. We anticipate adaptations and applications for estimating complete cities and incorporating weather input.

Method: Proposed Residual Linear Attention (RLA) framework with auxiliary recurrent state that accumulates residual errors over time to correct base predictions. Also introduced Residual Delta Net (RDN) with adaptive gating and residual clipping for enhanced control and stability.

Result: RLA and RDN consistently outperform baselines and other modern linear-attention methods across language modeling and recall-intensive evaluations, narrowing the gap to standard Transformers while maintaining linear scaling.

Conclusion: The residual-fitting mechanism in RLA effectively addresses the expressivity bottleneck in linear attention, enabling better long-range pattern capture while preserving linear time and memory complexity.

Abstract: Linear attention offers a linear-time alternative to self-attention but often struggles to capture long-range patterns. We revisit linear attention through a prediction-correction lens and show that prevalent variants can be written as a combination of a historical prediction and a single-token correction, which creates an expressivity bottleneck. To address this bottleneck, we introduce Residual Linear Attention (RLA), a framework that equips linear attention with an explicit residual-fitting mechanism. RLA maintains an auxiliary recurrent state that learns to accumulate residual errors over time and correct the base prediction. We further instantiate a delta-rule version, Residual Delta Net (RDN), incorporating adaptive gating and residual clipping for enhanced correction control and stability. Our implementation leverages highly optimized linear attention kernels and preserves linear time and memory. Across language modeling and recall-intensive evaluations, RLA and RDN consistently outperform their respective baselines and other modern linear-attention methods, narrowing the gap to standard Transformers while retaining linear scaling.

Method: Two core innovations: (1) FlashAttention-based algorithm replacing FP multiplications with INT additions for output block rescaling using binary correspondence between FP32 and INT32; (2) Preload Pipeline strategy with hierarchical tiling that maximizes FLOPS utilization through Cube-bound performance and overlapping data movement/computation.

Result: On Ascend 910 NPUs, AMLA achieves up to 614 TFLOPS, reaching 86.8% of theoretical maximum FLOPS, outperforming state-of-the-art FlashMLA implementation (66.7% FLOPS utilization on NVIDIA H800 SXM5).

Conclusion: AMLA kernel has been integrated into Huawei’s CANN and will be released soon, demonstrating superior performance for MLA optimization on Ascend NPUs.

Abstract: Multi-head Latent Attention (MLA) significantly reduces KVCache memory usage in Large Language Models while introducing substantial computational overhead and intermediate variable expansion. This poses challenges for efficient hardware implementation – especially during the decode phase. This paper introduces Ascend MLA (AMLA), a high-performance kernel specifically optimized for Huawei’s Ascend NPUs. AMLA is built on two core innovations: (1) A novel FlashAttention-based algorithm that replaces floating-point multiplications with integer additions for output block rescaling, leveraging binary correspondence between FP32 and INT32 representations; (2) A Preload Pipeline strategy with hierarchical tiling that maximizes FLOPS utilization: the Preload Pipeline achieves Cube-bound performance, while hierarchical tiling overlaps data movement and computation within the Cube core. Experiments show that on Ascend 910 NPUs (integrated in CloudMatrix384), AMLA achieves up to 614 TFLOPS, reaching 86.8% of the theoretical maximum FLOPS, outperforming the state-of-the-art open-source FlashMLA implementation, whose FLOPS utilization is up to 66.7% on NVIDIA H800 SXM5. The AMLA kernel has been integrated into Huawei’s CANN and will be released soon.

Method: Proposed MSCoD framework with Multi-Scale Information Bottleneck (MSIB) for hierarchical feature extraction and multi-head cooperative attention (MHCA) with asymmetric protein-to-ligand attention to handle dimensionality disparity.

Result: MSCoD outperforms state-of-the-art methods on benchmark datasets and shows strong applicability in real-world scenarios like KRAS G12D targets.

Conclusion: MSCoD provides an effective solution for capturing multi-scale protein-ligand interactions and demonstrates practical utility in drug discovery applications.

Abstract: Structure-Based Drug Design (SBDD) is a powerful strategy in computational drug discovery, utilizing three-dimensional protein structures to guide the design of molecules with improved binding affinity. However, capturing complex protein-ligand interactions across multiple scales remains challenging, as current methods often overlook the hierarchical organization and intrinsic asymmetry of these interactions. To address these limitations, we propose MSCoD, a novel Bayesian updating-based generative framework for structure-based drug design. In our MSCoD, Multi-Scale Information Bottleneck (MSIB) was developed, which enables semantic compression at multiple abstraction levels for efficient hierarchical feature extraction. Furthermore, a multi-head cooperative attention (MHCA) mechanism was developed, which employs asymmetric protein-to-ligand attention to capture diverse interaction types while addressing the dimensionality disparity between proteins and ligands. Empirical studies showed that MSCoD outperforms state-of-the-art methods on the benchmark dataset. Case studies on challenging targets such as KRAS G12D further demonstrate its applicability in real-world scenarios. The code and data underlying this article are freely available at https://github.com/xulong0826/MSCoD.

Method: Proposes a framework that: 1) Uses MVMD to jointly decompose wind, solar and wave power series while preserving cross-source couplings; 2) Applies Bayesian optimization to automatically search MVMD parameters; 3) Uses LSTM to model the resulting intrinsic mode functions for forecasting.

Result: Experiments using field measurements from an offshore integrated energy platform in China show the framework significantly outperforms benchmark models in MAPE, RMSE and MAE metrics.

Conclusion: The proposed framework demonstrates superior predictive accuracy, robustness, and degree of automation for integrated wind-solar-wave marine energy system forecasting.

Abstract: Integrated wind-solar-wave marine energy systems hold broad promise for supplying clean electricity in offshore and coastal regions. By leveraging the spatiotemporal complementarity of multiple resources, such systems can effectively mitigate the intermittency and volatility of single-source outputs, thereby substantially improving overall power-generation efficiency and resource utilization. Accurate ultra-short-term forecasting is crucial for ensuring secure operation and optimizing proactive dispatch. However, most existing forecasting methods construct separate models for each energy source, insufficiently account for the complex couplings among multiple energies, struggle to capture the system’s nonlinear and nonstationary dynamics, and typically depend on extensive manual parameter tuning-limitations that constrain both predictive performance and practicality. We address this issue using a Bayesian-optimized Multivariate Variational Mode Decomposition-Long Short-Term Memory (MVMD-LSTM) framework. The framework first applies MVMD to jointly decompose wind, solar and wave power series so as to preserve cross-source couplings; it uses Bayesian optimization to automatically search the number of modes and the penalty parameter in the MVMD process to obtain intrinsic mode functions (IMFs); finally, an LSTM models the resulting IMFs to achieve ultra-short-term power forecasting for the integrated system. Experiments based on field measurements from an offshore integrated energy platform in China show that the proposed framework significantly outperforms benchmark models in terms of MAPE, RMSE and MAE. The results demonstrate superior predictive accuracy, robustness, and degree of automation.

Method: Uses random semi-orthogonal matrices from Haar-distributed orthogonal ensembles via QR decomposition of Gaussian matrices to project data into lower-dimensional latent spaces.

Result: RPFs yield closed-form expressions for Riemannian volume correction, are theoretically grounded, and serve as effective baselines for generative modeling.

Conclusion: RPFs provide an efficient, plug-and-play framework that successfully connects random projection theory with normalizing flows for generative modeling applications.

Abstract: We introduce Random Projection Flows (RPFs), a principled framework for injective normalizing flows that leverages tools from random matrix theory and the geometry of random projections. RPFs employ random semi-orthogonal matrices, drawn from Haar-distributed orthogonal ensembles via QR decomposition of Gaussian matrices, to project data into lower-dimensional latent spaces for the base distribution. Unlike PCA-based flows or learned injective maps, RPFs are plug-and-play, efficient, and yield closed-form expressions for the Riemannian volume correction term. We demonstrate that RPFs are both theoretically grounded and practically effective, providing a strong baseline for generative modeling and a bridge between random projection theory and normalizing flows.

Method: The authors use score matching and annealed energy distillation to learn a metric tensor that faithfully captures the underlying data geometry, which then informs more accurate flows.

Result: The method is demonstrated to be effective on synthetic manifolds with analytic geodesics and for interpolation of cell data.

Conclusion: Learning a metric tensor through score matching and annealed energy distillation provides a better inductive bias for temporal data by ensuring trajectories stay close to the data manifold, overcoming limitations of traditional flow matching approaches.

Abstract: A useful inductive bias for temporal data is that trajectories should stay close to the data manifold. Traditional flow matching relies on straight conditional paths, and flow matching methods which learn geodesics rely on RBF kernels or nearest neighbor graphs that suffer from the curse of dimensionality. We propose to use score matching and annealed energy distillation to learn a metric tensor that faithfully captures the underlying data geometry and informs more accurate flows. We demonstrate the efficacy of this strategy on synthetic manifolds with analytic geodesics, and interpolation of cell

Method: The model employs wavelet transform for multi-resolution analysis of time series data, uses attention mechanisms on inverted dimensions to capture relationships between multiple variables, and introduces a differential attention mechanism that computes attention scores from the difference between two separate softmax attention matrices.

Result: WDformer achieved state-of-the-art (SOTA) results on multiple challenging real-world datasets, demonstrating superior accuracy and effectiveness compared to existing methods.

Conclusion: The proposed WDformer model successfully addresses limitations of traditional approaches by combining wavelet-based multi-resolution analysis with differential attention mechanisms, providing a more accurate and effective solution for time series forecasting tasks.

Abstract: Time series forecasting has various applications, such as meteorological rainfall prediction, traffic flow analysis, financial forecasting, and operational load monitoring for various systems. Due to the sparsity of time series data, relying solely on time-domain or frequency-domain modeling limits the model’s ability to fully leverage multi-domain information. Moreover, when applied to time series forecasting tasks, traditional attention mechanisms tend to over-focus on irrelevant historical information, which may introduce noise into the prediction process, leading to biased results. We proposed WDformer, a wavelet-based differential Transformer model. This study employs the wavelet transform to conduct a multi-resolution analysis of time series data. By leveraging the advantages of joint representation in the time-frequency domain, it accurately extracts the key information components that reflect the essential characteristics of the data. Furthermore, we apply attention mechanisms on inverted dimensions, allowing the attention mechanism to capture relationships between multiple variables. When performing attention calculations, we introduced the differential attention mechanism, which computes the attention score by taking the difference between two separate softmax attention matrices. This approach enables the model to focus more on important information and reduce noise. WDformer has achieved state-of-the-art (SOTA) results on multiple challenging real-world datasets, demonstrating its accuracy and effectiveness. Code is available at https://github.com/xiaowangbc/WDformer.

Method: Two-stage paradigm: (1) initialize Gaussian components and sample from proposal mixture, (2) optimize mixture parameters (weights only or weights+means+variances) via sample-based KL divergence objective using projected gradient descent, mirror descent, or EM, followed by stratified resampling.

Result: The method produces consistent approximations under mild conditions and demonstrates accuracy and speed across diverse densities in empirical validation.

Conclusion: GMA samplers provide an effective gradient-free approach for sampling unnormalized target densities, combining computational efficiency with theoretical consistency guarantees.

Abstract: We present a family of \textit{Gaussian Mixture Approximation} (GMA) samplers for sampling unnormalised target densities, encompassing \textit{weights-only GMA} (W-GMA), \textit{Laplace Mixture Approximation} (LMA), \textit{expectation-maximization GMA} (EM-GMA), and further variants. GMA adopts a simple two-stage paradigm: (i) initialise a finite set of Gaussian components and draw samples from a proposal mixture; (ii) fit the mixture to the target by optimising either only the component weights or also the means and variances, via a sample-based KL divergence objective that requires only evaluations of the unnormalised density, followed by stratified resampling. The method is gradient-free, and computationally efficient: it leverages the ease of sampling from Gaussians, efficient optimisation methods (projected gradient descent, mirror descent, and EM), and the robustness of stratified resampling to produce samples faithful to the target. We show that this optimisation-resampling scheme yields consistent approximations under mild conditions, and we validate this methodology with empirical results demonstrating accuracy and speed across diverse densities.

Method: FedCLF introduces calibrated loss as utility for participant selection and feedback control to dynamically adjust client sampling frequency, addressing data heterogeneity and resource constraints.

Result: FedCLF outperforms FedAvg, Newt, and Oort by up to 16% improvement in high data heterogeneity scenarios with improved efficiency through reduced sampling frequency.

Conclusion: FedCLF effectively addresses FL challenges in IoV networks by enhancing model accuracy under data heterogeneity and optimizing resource utilization through calibrated loss and feedback control mechanisms.

Abstract: Federated Learning (FL) is a distributed machine learning technique that preserves data privacy by sharing only the trained parameters instead of the client data. This makes FL ideal for highly dynamic, heterogeneous, and time-critical applications, in particular, the Internet of Vehicles (IoV) networks. However, FL encounters considerable challenges in such networks owing to the high data and device heterogeneity. To address these challenges, we propose FedCLF, i.e., FL with Calibrated Loss and Feedback control, which introduces calibrated loss as a utility in the participant selection process and a feedback control mechanism to dynamically adjust the sampling frequency of the clients. The envisaged approach (a) enhances the overall model accuracy in case of highly heterogeneous data and (b) optimizes the resource utilization for resource constrained IoV networks, thereby leading to increased efficiency in the FL process. We evaluated FedCLF vis-`a-vis baseline models, i.e., FedAvg, Newt, and Oort, using CIFAR-10 dataset with varying data heterogeneity. Our results depict that FedCLF significantly outperforms the baseline models by up to a 16% improvement in high data heterogeneity-related scenarios with improved efficiency via reduced sampling frequency.

Method: Feature-based time-series classification using domain-expert selected time-based and spatial features from nozzle logging data, evaluated with traditional ML classifiers including One-vs-Rest Random Forest.

Result: The proposed model achieved better performance than an in-house rule-based baseline, with improved weighted F1 scores for several failure mechanisms.

Conclusion: Machine Learning classification using expert-guided features effectively identifies printhead failure mechanisms from nozzle behavior patterns, providing superior performance over traditional rule-based approaches.

Abstract: Correct identification of failure mechanisms is essential for manufacturers to ensure the quality of their products. Certain failures of printheads developed by Canon Production Printing can be identified from the behavior of individual nozzles, the states of which are constantly recorded and can form distinct patterns in terms of the number of failed nozzles over time, and in space in the nozzle grid. In our work, we investigate the problem of printhead failure classification based on a multifaceted dataset of nozzle logging and propose a Machine Learning classification approach for this problem. We follow the feature-based framework of time-series classification, where a set of time-based and spatial features was selected with the guidance of domain experts. Several traditional ML classifiers were evaluated, and the One-vs-Rest Random Forest was found to have the best performance. The proposed model outperformed an in-house rule-based baseline in terms of a weighted F1 score for several failure mechanisms.

Method: Trains on 50,000+ recovery-annotated trajectories via systematic failure injection and expert demonstrations on ToolBench dataset. Uses LoRA fine-tuning and retrieves similar failure cases from 55+ exemplars at inference.

Result: Achieves 89.68% recovery rate (+57% over ToolBench), 95.2% recovery on unseen APIs, and outperforms CRITIC baseline by +13.3%. Improves multiple metrics including Task Success Rate and Efficiency Score.

Conclusion: PALADIN effectively builds fault-tolerant agents capable of robust recovery in real-world tool environments, establishing a method for handling tool failures beyond training distribution.

Abstract: Tool-augmented language agents frequently fail in real-world deployment due to tool malfunctions–timeouts, API exceptions, or inconsistent outputs–triggering cascading reasoning errors and task abandonment. Existing agent training pipelines optimize only for success trajectories, failing to expose models to the tool failures that dominate real-world usage. We propose \textbf{PALADIN}, a generalizable framework for equipping language agents with robust failure recovery capabilities. PALADIN trains on 50,000+ recovery-annotated trajectories constructed via systematic failure injection and expert demonstrations on an enhanced ToolBench dataset. Training uses LoRA-based fine-tuning to retain base capabilities while injecting recovery competence. At inference, PALADIN detects execution-time errors and retrieves the most similar case from a curated bank of 55+ failure exemplars aligned with ToolScan’s taxonomy, then executes the corresponding recovery action. This approach generalizes to novel failures beyond the training distribution, retaining 95.2% recovery performance on unseen tool APIs. Evaluation across PaladinEval and ToolReflectEval demonstrates consistent improvements in Recovery Rate (RR), Task Success Rate (TSR), Catastrophic Success Rate (CSR), and Efficiency Score (ES). PALADIN improves RR from 32.76% to 89.68% (+57% relative) over ToolBench and outperforms the strongest baseline CRITIC (76.34%) by +13.3%. Against vanilla agents, PALADIN achieves 89.86% RR (+66% relative improvement from 23.75%). These results establish PALADIN as an effective method for building fault-tolerant agents capable of robust recovery in real-world tool environments.

Method: Proposes HAMMER framework that transfers diversity metrics from dataset evaluation into RL training, ordering samples via minimum-semantic Hamiltonian paths to retain exploration during initial training.

Result: Achieves 3-4% average accuracy gain across diverse inference benchmarks, stimulates model “curiosity” and facilitates stable convergence from theoretical generalization bounds perspective.

Conclusion: Diversity-driven curriculum ordering through HAMMER effectively addresses local optimization in LLM training and consistently improves performance across benchmarks.

Abstract: Recent curriculum reinforcement learning for large language models (LLMs) typically rely on difficulty-based annotations for data filtering and ordering. However, such methods suffer from local optimization, where continual training on simple samples in the early steps can cause the policy to lose its exploration. We propose a novel schema, namely Hamiltonian curiosity augmented large language model reinforcement (HAMMER), that transfers diversity metrics, commonly used in dataset evaluation, into the dynamic reinforcement learning procedure, where training samples are ordered via a minimum-semantic Hamiltonian path making the initial training retrain more exploration. From a theoretical perspective of generalization bounds, diversity-driven ordering facilitates stable convergence. Empirical evaluations indicate that HAMMER stimulates model “curiosity” and consistently achieves a 3% to 4% average accuracy gain across diverse inference benchmark.

Method: Investigated Supervised Fine-Tuning (SFT), Low-Rank Adaptation (LoRA), and Quantized Low-Rank Adaptation (QLoRA) using a cybersecurity Q&A dataset.

Result: All fine-tuning approaches significantly outperformed the foundational model. LoRA and QLoRA achieved comparable performance to SFT with substantially lower computational costs.

Conclusion: Low-rank fine-tuning strategies like LoRA and QLoRA provide efficient pathways for adapting general-purpose LLMs to specialized domains while maintaining performance.

Abstract: Recent advancements in training paradigms for Large Language Models (LLMs) have unlocked their remarkable capabilities in natural language processing and cross-domain generalization. While LLMs excel in tasks like programming and mathematical problem-solving, their zero-shot performance in specialized domains requiring expert knowledge, such as cybersecurity, is often suboptimal. This limitation arises because foundational LLMs are designed for general-purpose applications, constraining their ability to encapsulate domain-specific expertise within their parameter space. To address this, we explore fine-tuning strategies to embed cybersecurity knowledge into LLMs, enhancing their performance in cybersecurity question-answering (Q&A) tasks while prioritizing computational efficiency. Specifically, we investigate Supervised Fine-Tuning (SFT), Low-Rank Adaptation (LoRA), and Quantized Low-Rank Adaptation (QLoRA) using a cybersecurity Q&A dataset. Our results demonstrate that these fine-tuning approaches significantly outperform the foundational model in cybersecurity Q&A tasks. Moreover, LoRA and QLoRA achieve comparable performance to SFT with substantially lower computational costs, offering an efficient pathway for adapting LLMs to specialized domains. Our work highlights the potential of low-rank fine-tuning strategies to bridge the gap between general-purpose LLMs and domain-specific applications.

Method: Uses Procrustes distance and Frobenius norm of Feature Gram Matrix as distillation losses to better align feature geometry between teacher and student models.

Result: Shows statistically significant 2 percentage point improvement in distillation performance across BERT and OPT models on classification and instruction-following tasks.

Conclusion: Feature geometry integration through Procrustes distance and Gram Matrix norms improves distillation performance, demonstrating the importance of proper feature structure alignment.

Abstract: Knowledge distillation is a common paradigm for transferring capabilities from larger models to smaller ones. While traditional distillation methods leverage a probabilistic divergence over the output of the teacher and student models, feature-based distillation methods often minimize variants of Euclidean norms between the hidden layer representations. The main goal is for the student to mimic the structure of the feature space of the teacher. In this work, we theoretically show that existing feature distillation methods, such as projection based mean squared loss or Centered Kernel Alignment (CKA), cannot capture the feature structure, even under zero loss. We then motivate the use of Procrustes distance and the Frobenius norm of Feature Gram Matrix, distances already common in the context of measuring representational alignment, as distillation losses. We show that feature distillation through our method showcases statistically significant improvement in distillation performance across language models families (BERT and OPT) in classification and instruction-following tasks by up to 2 percentage points, showcasing the potential of integrating feature geometry into existing distillation methods.

Method: The benchmark uses 5 years of meteorological data from 12,000+ GNSS stations with 15-minute intervals across 6 variables, including precipitable water vapor (PWV). They introduce Bi-Focus Precipitation Forecaster (BFPF), a plug-and-play module with domain-specific priors to handle zero-inflation and temporal decay.

Result: The study evaluates over 20 state-of-the-art models across 6 major architectures on RainfallBench, with statistical analysis and ablation studies validating dataset comprehensiveness and methodology superiority.

Conclusion: RainfallBench provides a comprehensive benchmark for rainfall nowcasting that addresses key meteorological challenges, and the proposed BFPF module effectively enhances rainfall time series forecasting by incorporating domain-specific knowledge.

Abstract: Rainfall nowcasting, which aims to predict precipitation within the next 0 to 3 hours, is critical for disaster mitigation and real-time response planning. However, most time series forecasting benchmarks in meteorology are evaluated on variables with strong periodicity, such as temperature and humidity, which fail to reflect model capabilities in more complex and practically meteorology scenarios like rainfall nowcasting. To address this gap, we propose RainfallBench, a benchmark designed for rainfall nowcasting, a highly challenging and practically relevant task characterized by zero inflation, temporal decay, and non-stationarity, focused on predicting precipitation within the next 0 to 3 hours. The dataset is derived from five years of meteorological observations, recorded at 15-minute intervals across six essential variables, and collected from more than 12,000 GNSS stations globally. In particular, it incorporates precipitable water vapor (PWV), a crucial indicator of rainfall that is absent in other datasets. We further design specialized evaluation strategies to assess model performance on key meteorological challenges, such as multi-scale prediction and extreme rainfall events, and evaluate over 20 state-of-the-art models across six major architectures on RainfallBench. Additionally, to address the zero-inflation and temporal decay issues overlooked by existing models, we introduce Bi-Focus Precipitation Forecaster (BFPF), a plug-and-play module that incorporates domain-specific priors to enhance rainfall time series forecasting. Statistical analysis and ablation studies validate the comprehensiveness of our dataset as well as the superiority of our methodology. Code and datasets are available at https://anonymous.4open.science/r/RainfallBench-A710.

Method: Formalizes adaptive strategy selection as a single-step MDP using Prompt Policy Network trained with PPO and resource-explicit reward function to allocate costly reasoning strategies only when necessary.

Result: Achieves superior performance on efficiency-accuracy Pareto front with up to 61.5% token cost reduction compared to Self-Consistency while maintaining competitive accuracy on arithmetic reasoning benchmarks.

Conclusion: Provides a systematic, adaptive framework for cost-efficient LLM deployment, advancing lightweight optimization techniques for scalable and sustainable language model applications.

Abstract: The performance of Large Language Models (LLMs) depends heavily on the chosen prompting strategy, yet static approaches such as Zero-Shot, Few-Shot, or Chain-of-Thought (CoT) impose a rigid efficiency-accuracy trade-off. Highly accurate strategies like Self-Consistency (SC) incur substantial computational waste on simple tasks, while lightweight methods often fail on complex inputs. This paper introduces the Prompt Policy Network (PPN), a lightweight reinforcement learning framework that formalizes adaptive strategy selection as a single-step Markov Decision Process (MDP). The PPN, trained with Proximal Policy Optimization (PPO) and guided by a resource-explicit reward function, learns to allocate costly reasoning strategies only when necessary. Experiments on arithmetic reasoning benchmarks demonstrate that PPN achieves superior performance on the efficiency-accuracy Pareto front, delivering up to 61.5% token cost reduction compared to Self-Consistency while maintaining competitive accuracy. This work contributes a systematic, adaptive framework for cost-efficient LLM deployment, advancing the design of lightweight optimization techniques for scalable and sustainable language model applications.

Method: Fine-tuned Silurian AI’s 1.5B-parameter Generative Forecasting Transformer on Hydro-Québec asset observations including transmission-line weather stations, wind-farm met-mast streams, and icing sensors.

Result: Outperformed state-of-the-art NWP benchmarks with 15% reduction in temperature MAE, 35% reduction in precipitation MAE, 15% reduction in wind speed MAE, and achieved 0.72 average precision for day-ahead rime-ice detection.

Conclusion: Weather foundation models, when post-trained with small amounts of high-fidelity data, can serve as practical foundations for next-generation grid-resilience intelligence.

Abstract: Weather foundation models (WFMs) have recently set new benchmarks in global forecast skill, yet their concrete value for the weather-sensitive infrastructure that powers modern society remains largely unexplored. In this study, we fine-tune Silurian AI’s 1.5B-parameter WFM, Generative Forecasting Transformer (GFT), on a rich archive of Hydro-Qu'ebec asset observations–including transmission-line weather stations, wind-farm met-mast streams, and icing sensors–to deliver hyper-local, asset-level forecasts for five grid-critical variables: surface temperature, precipitation, hub-height wind speed, wind-turbine icing risk, and rime-ice accretion on overhead conductors. Across 6-72 h lead times, the tailored model surpasses state-of-the-art NWP benchmarks, trimming temperature mean absolute error (MAE) by 15%, total-precipitation MAE by 35%, and lowering wind speed MAE by 15%. Most importantly, it attains an average precision score of 0.72 for day-ahead rime-ice detection, a capability absent from existing operational systems, which affords several hours of actionable warning for potentially catastrophic outage events. These results show that WFMs, when post-trained with small amounts of high-fidelity, can serve as a practical foundation for next-generation grid-resilience intelligence.

Method: InfMasking uses an infinite masking strategy that stochastically occludes most features from each modality during fusion, preserving only partial information to create varied synergistic patterns. Unmasked fused representations are aligned with masked ones through mutual information maximization.

Result: InfMasking effectively enhances synergistic information between modalities and achieves state-of-the-art performance across seven large-scale real-world benchmarks.

Conclusion: The infinite masking strategy enables capturing richer interactions by exposing models to diverse partial modality combinations, making InfMasking an effective approach for synergistic information extraction in multimodal learning.

Abstract: In multimodal representation learning, synergistic interactions between modalities not only provide complementary information but also create unique outcomes through specific interaction patterns that no single modality could achieve alone. Existing methods may struggle to effectively capture the full spectrum of synergistic information, leading to suboptimal performance in tasks where such interactions are critical. This is particularly problematic because synergistic information constitutes the fundamental value proposition of multimodal representation. To address this challenge, we introduce InfMasking, a contrastive synergistic information extraction method designed to enhance synergistic information through an \textbf{Inf}inite \textbf{Masking} strategy. InfMasking stochastically occludes most features from each modality during fusion, preserving only partial information to create representations with varied synergistic patterns. Unmasked fused representations are then aligned with masked ones through mutual information maximization to encode comprehensive synergistic information. This infinite masking strategy enables capturing richer interactions by exposing the model to diverse partial modality combinations during training. As computing mutual information estimates with infinite masking is computationally prohibitive, we derive an InfMasking loss to approximate this calculation. Through controlled experiments, we demonstrate that InfMasking effectively enhances synergistic information between modalities. In evaluations on large-scale real-world datasets, InfMasking achieves state-of-the-art performance across seven benchmarks. Code is released at https://github.com/brightest66/InfMasking.

Method: Uses symbolic tokenization and adaptive attention budgeting to construct long multimodal sequences processed via sparse cross-modal attention. Cross-modal tokens are routed through sparse Mixture-of-Experts (MoE) mechanism for black-box specialization under varying modality combinations.

Result: Outperformed 10 baselines across 4 datasets, achieving 4% improvement over best multimodal approaches and 8% over multivariate approaches under complete observations. With 40% missing modalities, achieved 9% average improvement. Demonstrated robustness and efficiency in dynamic time series learning.

Conclusion: MAESTRO provides an effective framework for real-world multimodal time-series analysis that overcomes key limitations of existing approaches, particularly handling missing modalities and enabling efficient processing of many modalities through sparse attention and MoE mechanisms.

Abstract: From clinical healthcare to daily living, continuous sensor monitoring across multiple modalities has shown great promise for real-world intelligent decision-making but also faces various challenges. In this work, we introduce MAESTRO, a novel framework that overcomes key limitations of existing multimodal learning approaches: (1) reliance on a single primary modality for alignment, (2) pairwise modeling of modalities, and (3) assumption of complete modality observations. These limitations hinder the applicability of these approaches in real-world multimodal time-series settings, where primary modality priors are often unclear, the number of modalities can be large (making pairwise modeling impractical), and sensor failures often result in arbitrary missing observations. At its core, MAESTRO facilitates dynamic intra- and cross-modal interactions based on task relevance, and leverages symbolic tokenization and adaptive attention budgeting to construct long multimodal sequences, which are processed via sparse cross-modal attention. The resulting cross-modal tokens are routed through a sparse Mixture-of-Experts (MoE) mechanism, enabling black-box specialization under varying modality combinations. We evaluate MAESTRO against 10 baselines on four diverse datasets spanning three applications, and observe average relative improvements of 4% and 8% over the best existing multimodal and multivariate approaches, respectively, under complete observations. Under partial observations – with up to 40% of missing modalities – MAESTRO achieves an average 9% improvement. Further analysis also demonstrates the robustness and efficiency of MAESTRO’s sparse, modality-aware design for learning from dynamic time series.

Method: Proposed a DRL framework using PPO and TD3 algorithms to jointly optimize transmit power, bandwidth, and scheduling with multi-objective rewards, tested against three heuristic algorithms using real base station coordinates.

Result: DRL frameworks consistently outperformed heuristic algorithms in optimizing resource allocation across multiple network scenarios.

Conclusion: DRL approaches show superior performance for dynamic resource allocation in HetNets, revealing important trade-offs in DRL design for future wireless networks.

Abstract: Dynamic resource allocation in heterogeneous wireless networks (HetNets) is challenging for traditional methods under varying user loads and channel conditions. We propose a deep reinforcement learning (DRL) framework that jointly optimises transmit power, bandwidth, and scheduling via a multi-objective reward balancing throughput, energy efficiency, and fairness. Using real base station coordinates, we compare Proximal Policy Optimisation (PPO) and Twin Delayed Deep Deterministic Policy Gradient (TD3) against three heuristic algorithms in multiple network scenarios. Our results show that DRL frameworks outperform heuristic algorithms in optimising resource allocation in dynamic networks. These findings highlight key trade-offs in DRL design for future HetNets.

Method: Uses a comprehensive data repository of 34,000 synthetic datasets with diverse structures, processed by 10 clustering algorithms. The network integrates convolutional, residual, and attention mechanisms to capture local and global patterns for end-to-end training.

Result: Outperforms conventional CVIs (Silhouette, Calinski-Harabasz, Davies-Bouldin, Dunn) and state-of-the-art AutoML approaches (ML2DAC, AutoCluster, AutoML4Clust). Achieves 0.497 ARI improvement over Calinski-Harabasz on synthetic data and 15.3% ARI gain over best AutoML on real-world data.

Conclusion: The proposed DL model provides an effective solution for clustering algorithm recommendation, demonstrating superior performance across both synthetic and real-world benchmarks.

Abstract: We introduce ClustRecNet - a novel deep learning (DL)-based recommendation framework for determining the most suitable clustering algorithms for a given dataset, addressing the long-standing challenge of clustering algorithm selection in unsupervised learning. To enable supervised learning in this context, we construct a comprehensive data repository comprising 34,000 synthetic datasets with diverse structural properties. Each of them was processed using 10 popular clustering algorithms. The resulting clusterings were assessed via the Adjusted Rand Index (ARI) to establish ground truth labels, used for training and evaluation of our DL model. The proposed network architecture integrates convolutional, residual, and attention mechanisms to capture both local and global structural patterns from the input data. This design supports end-to-end training to learn compact representations of datasets and enables direct recommendation of the most suitable clustering algorithm, reducing reliance on handcrafted meta-features and traditional Cluster Validity Indices (CVIs). Comprehensive experiments across synthetic and real-world benchmarks demonstrate that our DL model consistently outperforms conventional CVIs (e.g. Silhouette, Calinski-Harabasz, Davies-Bouldin, and Dunn) as well as state-of-the-art AutoML clustering recommendation approaches (e.g. ML2DAC, AutoCluster, and AutoML4Clust). Notably, the proposed model achieves a 0.497 ARI improvement over the Calinski-Harabasz index on synthetic data and a 15.3% ARI gain over the best-performing AutoML approach on real-world data.

Method: Conducted 54 experiments across diverse model sizes and training settings to systematically analyze how model scale, data volume, and computational budget interact during RL post-training.

Result: Four key findings: 1) Larger models outperform smaller ones under fixed compute budgets; 2) Larger models achieve superior sample efficiency; 3) Data reuse is effective in data-constrained regimes; 4) Scaling behaviors are robust across base and instruction-tuned models.

Conclusion: The results provide principled foundations and practical guidelines for efficiently scaling LLM reasoning capabilities through RL post-training.

Abstract: While scaling laws for large language models (LLMs) during pre-training have been extensively studied, their behavior under reinforcement learning (RL) post-training remains largely unexplored. This paper presents a systematic empirical investigation of scaling behaviors in RL-based post-training, with a particular focus on mathematical reasoning. Based on 54 experiments across diverse model sizes and training settings, we characterize how model scale, data volume, and computational budget interact to shape performance. Our analysis leads to four key findings: (1). Under a fixed computational budget, larger models trained for fewer steps consistently outperform smaller models trained for more steps. (2). Given a fixed amount of training data, larger models achieve superior sample efficiency, yielding lower loss. (3). In data-constrained regimes, repeated reuse of high-quality data proves highly effective, as final performance is primarily governed by the total number of optimization steps rather than the uniqueness of samples. (4). These scaling behaviors are robust across both base and instruction-tuned models, which share similar learning dynamics (e.g., larger models show faster convergence) even while differing in absolute accuracy. Collectively, these results provide a principled foundation and practical guidelines for efficiently scaling the reasoning capabilities of LLMs through RL post-training.

Method: Uses Conditional Variational Autoencoder with local entropy guidance - computes Shannon entropy in sample neighborhoods to identify uncertain regions, then applies Local Entropy-Weighted Loss and entropy-guided sampling.

Result: Outperforms traditional oversampling and generative baselines on clinical genomics datasets (ADNI and TCGA lung cancer), improving classifier performance.

Conclusion: Uncertainty-aware generative oversampling is valuable for imbalanced learning in domains with complex nonlinear structures like omics data.

Abstract: Class imbalance remains a major challenge in machine learning, especially for high-dimensional biomedical data where nonlinear manifold structures dominate. Traditional oversampling methods such as SMOTE rely on local linear interpolation, often producing implausible synthetic samples. Deep generative models like Conditional Variational Autoencoders (CVAEs) better capture nonlinear distributions, but standard variants treat all minority samples equally, neglecting the importance of uncertain, boundary-region examples emphasized by heuristic methods like Borderline-SMOTE and ADASYN. We propose Local Entropy-Guided Oversampling with a CVAE (LEO-CVAE), a generative oversampling framework that explicitly incorporates local uncertainty into both representation learning and data generation. To quantify uncertainty, we compute Shannon entropy over the class distribution in a sample’s neighborhood: high entropy indicates greater class overlap, serving as a proxy for uncertainty. LEO-CVAE leverages this signal through two mechanisms: (i) a Local Entropy-Weighted Loss (LEWL) that emphasizes robust learning in uncertain regions, and (ii) an entropy-guided sampling strategy that concentrates generation in these informative, class-overlapping areas. Applied to clinical genomics datasets (ADNI and TCGA lung cancer), LEO-CVAE consistently improves classifier performance, outperforming both traditional oversampling and generative baselines. These results highlight the value of uncertainty-aware generative oversampling for imbalanced learning in domains governed by complex nonlinear structures, such as omics data.

Method: Analyzed gradient descent in matrix factorization, derived exact critical step size for scalar-vector factorization, studied regularization effects, and extended analysis to general matrix factorization with orthogonal initialization.

Result: Found that near critical step sizes, parameter space develops fractal structure, selected minimizer depends sensitively on initialization, and regularization amplifies this sensitivity creating fractal boundaries between converging and diverging initializations.

Conclusion: Near-critical step sizes induce chaotic gradient descent dynamics with unpredictable long-term behavior and absence of simple implicit biases like balancedness, minimum norm, or flatness.

Abstract: We examine gradient descent in matrix factorization and show that under large step sizes the parameter space develops a fractal structure. We derive the exact critical step size for convergence in scalar-vector factorization and show that near criticality the selected minimizer depends sensitively on the initialization. Moreover, we show that adding regularization amplifies this sensitivity, generating a fractal boundary between initializations that converge and those that diverge. The analysis extends to general matrix factorization with orthogonal initialization. Our findings reveal that near-critical step sizes induce a chaotic regime of gradient descent where the long-term dynamics are unpredictable and there are no simple implicit biases, such as towards balancedness, minimum norm, or flatness.

Method: Propose a coverage-aware method that encourages diversity early in the clustering process to handle the cold-start scenario.

Result: Demonstrated effectiveness through several synthetic and real-world experiments.

Conclusion: The proposed coverage-aware method effectively handles cold-start active correlation clustering by promoting early diversity in the querying process.

Abstract: We study active correlation clustering where pairwise similarities are not provided upfront and must be queried in a cost-efficient manner through active learning. Specifically, we focus on the cold-start scenario, where no true initial pairwise similarities are available for active learning. To address this challenge, we propose a coverage-aware method that encourages diversity early in the process. We demonstrate the effectiveness of our approach through several synthetic and real-world experiments.

Method: Introduced a physically motivated non-linear noising process that unfolds proteins into secondary structures while preserving bonds and preventing collisions, integrated with flow-matching on SE(3) to model protein backbone distributions with sequence conditioning.

Result: Achieved state-of-the-art performance in unconditional protein generation, producing more designable and novel structures while accurately folding monomer sequences into precise conformations.

Conclusion: The physics-grounded approach enables more realistic and effective protein structure generation and folding compared to traditional diffusion models.

Abstract: Protein structure prediction and folding are fundamental to understanding biology, with recent deep learning advances reshaping the field. Diffusion-based generative models have revolutionized protein design, enabling the creation of novel proteins. However, these methods often neglect the intrinsic physical realism of proteins, driven by noising dynamics that lack grounding in physical principles. To address this, we first introduce a physically motivated non-linear noising process, grounded in classical physics, that unfolds proteins into secondary structures (e.g., alpha helices, linear beta sheets) while preserving topological integrity–maintaining bonds, and preventing collisions. We then integrate this process with the flow-matching paradigm on SE(3) to model the invariant distribution of protein backbones with high fidelity, incorporating sequence information to enable sequence-conditioned folding and expand the generative capabilities of our model. Experimental results demonstrate that the proposed method achieves state-of-the-art performance in unconditional protein generation, producing more designable and novel protein structures while accurately folding monomer sequences into precise protein conformations.

Method: Introduce TREC diagnostic that evaluates training batches using final model weights, analyze TRECs across models from 111M to 3.9B parameters, and predict TRECs using AdamW’s implicit EMA coefficients.

Result: Placing high-quality data at TREC minima significantly improves performance. TREC can be predicted in advance from AdamW’s EMA coefficients, enabling proactive curriculum design. Analysis of published training recipes reveals suboptimal data placements.

Conclusion: TREC provides a principled approach to data curriculum design, allowing optimization of data placement to improve LLM training efficiency and performance.

Abstract: Data curriculums have become central to successful LLM training, yet principles governing optimal data placement remain unclear. We introduce the training re-evaluation curve (TREC), a diagnostic that retrospectively evaluates training batches using the final model weights. The TREC characterizes how well a trained model retains training data as a function of when the data was encountered during training. Analyzing TRECs for models from 111M to 3.9B parameters, we show that placing high-quality data at low points on the TREC significantly improves performance. Importantly, while a TREC is initially observable only after training, we demonstrate it can be predicted in advance from AdamW’s implicit EMA coefficients, enabling proactive curriculum design. By predicting TRECs for published training recipes, we explain prior ablations and reveal suboptimal data placements. We also align high-quality data with TREC minima in order to improve continual pre-training of a 3.9B-parameter LLM trained on 900B tokens.

Method: Combines a micro-network Basis Layer for functional data representation with gradient-based imputation and end-to-end learning of event-specific hazards.

Result: Substantial improvements in prediction accuracy over random survival forests and traditional competing risks models on simulated datasets and real-world ICU case studies.

Conclusion: FCRN advances survival analysis by effectively capturing dynamic risk factors and static predictors while handling irregular and incomplete data.

Abstract: We introduce the Functional Competing Risk Net (FCRN), a unified deep-learning framework for discrete-time survival analysis under competing risks, which seamlessly integrates functional covariates and handles missing data within an end-to-end model. By combining a micro-network Basis Layer for functional data representation with a gradient-based imputation module, FCRN simultaneously learns to impute missing values and predict event-specific hazards. Evaluated on multiple simulated datasets and a real-world ICU case study using the MIMIC-IV and Cleveland Clinic datasets, FCRN demonstrates substantial improvements in prediction accuracy over random survival forests and traditional competing risks models. This approach advances prognostic modeling in critical care by more effectively capturing dynamic risk factors and static predictors while accommodating irregular and incomplete data.

Method: Used Hamiltonian Monte Carlo (HMC) to draw posterior samples conditioned on clean inputs and compared them to encoder outputs from noisy data for a VAE-MoG trained on GW150914 data.

Result: Model reconstructs signals accurately but statistical comparisons show clear mismatch in latent space between encoder outputs and HMC posterior samples.

Conclusion: Strong denoising performance doesn’t guarantee reliable latent representations, highlighting importance of posterior-based validation for generative models.

Abstract: In this work, we explore the latent space of a denoising variational autoencoder with a mixture-of-Gaussians prior (VAE-MoG), trained on gravitational wave data from event GW150914. To evaluate how well the model captures the underlying structure, we use Hamiltonian Monte Carlo (HMC) to draw posterior samples conditioned on clean inputs, and compare them to the encoder’s outputs from noisy data. Although the model reconstructs signals accurately, statistical comparisons reveal a clear mismatch in the latent space. This shows that strong denoising performance doesn’t necessarily mean the latent representations are reliable highlighting the importance of using posterior-based validation when evaluating generative models.

Method: Adapts the Dawid-Skene model (traditionally used for crowdsourcing with human annotators) to aggregate predictions from multiple model-based clustering algorithms, treating their outputs as noisy annotations.

Result: Mixsemble consistently approaches the best clustering performance across both simulated and real-world datasets, though it’s not always the single top performer. It reliably avoids poor outcomes and demonstrates robustness.

Conclusion: Mixsemble provides a practical and robust alternative for clustering tasks when the true data structure is unknown, offering consistent near-optimal performance without the risk of poor outcomes.

Abstract: This paper introduces mixsemble, an ensemble method that adapts the Dawid-Skene model to aggregate predictions from multiple model-based clustering algorithms. Unlike traditional crowdsourcing, which relies on human labels, the framework models the outputs of clustering algorithms as noisy annotations. Experiments on both simulated and real-world datasets show that, although the mixsemble is not always the single top performer, it consistently approaches the best result and avoids poor outcomes. This robustness makes it a practical alternative when the true data structure is unknown, especially for non-expert users.

Method: Bayesian relevance vector machine (RVM) within a multi-task learning (MTL) framework for simultaneous parameter identification across multiple datasets, treating responses under different excitation levels as related tasks sharing model parameters.

Result: MTL-RVM improved parameter recovery for weakly and moderately excited datasets while maintaining strong performance under high excitation, outperforming standard single-task RVM models.

Conclusion: Multi-task Bayesian inference can mitigate over-fitting and promote generalization in equation discovery, particularly relevant for structural health monitoring with varying load conditions.

Abstract: Equation discovery provides a grey-box approach to system identification by uncovering governing dynamics directly from observed data. However, a persistent challenge lies in ensuring that identified models generalise across operating conditions rather than over-fitting to specific datasets. This work investigates this issue by applying a Bayesian relevance vector machine (RVM) within a multi-task learning (MTL) framework for simultaneous parameter identification across multiple datasets. In this formulation, responses from the same structure under different excitation levels are treated as related tasks that share model parameters but retain task-specific noise characteristics. A simulated single degree-of-freedom oscillator with linear and cubic stiffness provided the case study, with datasets generated under three excitation regimes. Standard single-task RVM models were able to reproduce system responses but often failed to recover the true governing terms when excitations insufficiently stimulated non-linear dynamics. By contrast, the MTL-RVM combined information across tasks, improving parameter recovery for weakly and moderately excited datasets, while maintaining strong performance under high excitation. These findings demonstrate that multi-task Bayesian inference can mitigate over-fitting and promote generalisation in equation discovery. The approach is particularly relevant to structural health monitoring, where varying load conditions reveal complementary aspects of system physics.

Method: Introduces flexible sparse symbols to standardize sparsity representation, designs optimized sparse GEMMs for attention blocks, and eliminates redundant computations using sparse symbols.

Result: Achieves near-linear speedup matching sparsity ratio (1:1) in attention and GEMM-Q, 2.5×-3.8× acceleration in GEMM-O (reaching 87.5% of theoretical limit), and enables 1.5× end-to-end acceleration for Hunyuan model without quality degradation.

Conclusion: FlashOmni provides a universal solution for efficient DiT deployment by unifying diverse sparsity strategies into a single attention engine, significantly improving computational efficiency while maintaining visual quality.

Abstract: Multi-Modal Diffusion Transformers (DiTs) demonstrate exceptional capabilities in visual synthesis, yet their deployment remains constrained by substantial computational demands. To alleviate this bottleneck, many sparsity-based acceleration methods have been proposed. However, their diverse sparsity patterns often require customized kernels for high-performance inference, limiting universality. We propose FlashOmni, a unified sparse attention engine compatible with arbitrary DiT architectures. FlashOmni introduces flexible sparse symbols to standardize the representation of a wide range of sparsity strategies, such as feature caching and block-sparse skipping. This unified abstraction enables the execution of diverse sparse computations within a single attention kernel. In addition, FlashOmni designs optimized sparse GEMMs for attention blocks, leveraging sparse symbols to eliminate redundant computations and further improve efficiency. Experiments demonstrate that FlashOmni delivers near-linear, closely matching the sparsity ratio speedup (1:1) in attention and GEMM-$Q$, and achieves 2.5$\times$-3.8$\times$ acceleration in GEMM-$O$ (max peaking at about 87.5% of the theoretical limit). Applied with a multi-granularity sparsity strategy, it enables the Hunyuan model (33K) to achieve about 1.5$\times$ end-to-end acceleration without degrading visual quality.

Method: Proposed ALoRA with multiple A matrices and single shared B for multi-task fine-tuning, and Fed-ALoRA for federated learning with a novel matrix decomposition strategy to handle heterogeneous ranks across clients.

Result: Experiments on commonsense reasoning, math reasoning, multi-task NLP, and federated NLP datasets show more balanced performance across tasks with comparable or superior average accuracy compared to existing multi-LoRA approaches.

Conclusion: Sharing B matrices while using multiple A matrices provides better performance balance in multi-task and federated learning scenarios, challenging conventional wisdom about LoRA matrix sharing.

Abstract: Large language models are often adapted using parameter-efficient techniques such as Low-Rank Adaptation (LoRA), formulated as $y = W_0x + BAx$, where $W_0$ is the pre-trained parameters and $x$ is the input to the adapted layer. While multi-adapter extensions often employ multiple LoRAs, prior studies suggest that the inner $A$ matrices are highly similar during training and thus suitable for sharing. We revisit this phenomenon and find that this similarity is largely attributable to the identical initialization rather than shared knowledge, with $B$ playing a more critical role in knowledge encoding and transfer. Motivated by these insights, we propose \textbf{ALoRA}, an asymmetric multi-LoRA design with multiple $A$ matrices and a single shared $B$ in multi-task fine-tuning, and \textbf{Fed-ALoRA}, which shares $B$ across clients in federated fine-tuning under both homogeneous and heterogeneous settings, through a novel matrix decomposition strategy to accommodate heterogeneous ranks across clients. Experiments on commonsense reasoning, math reasoning, multi-task NLP dataset, and federated NLP dataset demonstrate that our methods achieve more balanced performance across tasks with comparable or superior average accuracy relative to existing multi-LoRA approaches. Codes are available at https://github.com/OptMN-Lab/ALoRA.

Method: HIA uses a data-driven surrogate model to identify structurally important nodes (central to connectivity) and dynamically important nodes (critical for temporal evolution), then employs hybrid perturbation combining strategic edge injection and targeted edge deletion.

Result: HIA significantly reduces TGNN accuracy on link prediction, achieving up to 35.55% decrease in Mean Reciprocal Rank across five real-world datasets and four TGNN architectures, substantially outperforming state-of-the-art baselines.

Conclusion: The results highlight fundamental vulnerabilities in current STDG models and underscore the urgent need for robust defenses that account for both structural and temporal dynamics.

Abstract: Temporal Graph Neural Networks (TGNNs) have become indispensable for analyzing dynamic graphs in critical applications such as social networks, communication systems, and financial networks. However, the robustness of TGNNs against adversarial attacks, particularly sophisticated attacks that exploit the temporal dimension, remains a significant challenge. Existing attack methods for Spatio-Temporal Dynamic Graphs (STDGs) often rely on simplistic, easily detectable perturbations (e.g., random edge additions/deletions) and fail to strategically target the most influential nodes and edges for maximum impact. We introduce the High Impact Attack (HIA), a novel restricted black-box attack framework specifically designed to overcome these limitations and expose critical vulnerabilities in TGNNs. HIA leverages a data-driven surrogate model to identify structurally important nodes (central to network connectivity) and dynamically important nodes (critical for the graph’s temporal evolution). It then employs a hybrid perturbation strategy, combining strategic edge injection (to create misleading connections) and targeted edge deletion (to disrupt essential pathways), maximizing TGNN performance degradation. Importantly, HIA minimizes the number of perturbations to enhance stealth, making it more challenging to detect. Comprehensive experiments on five real-world datasets and four representative TGNN architectures (TGN, JODIE, DySAT, and TGAT) demonstrate that HIA significantly reduces TGNN accuracy on the link prediction task, achieving up to a 35.55% decrease in Mean Reciprocal Rank (MRR) - a substantial improvement over state-of-the-art baselines. These results highlight fundamental vulnerabilities in current STDG models and underscore the urgent need for robust defenses that account for both structural and temporal dynamics.

Method: Proposes a polychromic objective for policy gradient methods, adapting PPO with vine sampling for on-policy rollouts and modifying the advantage function to enforce diverse generation exploration.

Result: Experiments on BabyAI, Minigrid, and Algorithmic Creativity show improved success rates, better generalization under perturbations, and higher coverage in pass@k experiments.

Conclusion: The polychromic objective effectively maintains policy diversity during fine-tuning, enabling better exploration and performance across diverse environment configurations.

Abstract: Reinforcement learning fine-tuning (RLFT) is a dominant paradigm for improving pretrained policies for downstream tasks. These pretrained policies, trained on large datasets, produce generations with a broad range of promising but unrefined behaviors. Often, a critical failure mode of RLFT arises when policies lose this diversity and collapse into a handful of easily exploitable outputs. This convergence hinders exploration, which is essential for expanding the capabilities of the pretrained policy and for amplifying the benefits of test-time compute scaling. To address this, we introduce an objective for policy gradient methods that explicitly enforces the exploration and refinement of diverse generations, which we call a polychromic objective. We then show how proximal policy optimization (PPO) can be adapted to optimize this objective. Our method (1) employs vine sampling to collect on-policy rollouts and (2) modifies the advantage function to reflect the advantage under our new objective. Experiments on BabyAI, Minigrid, and Algorithmic Creativity show that our method improves success rates by reliably solving a larger set of environment configurations and generalizes better under large perturbations. Moreover, when given multiple attempts in pass@$k$ experiments, the policy achieves substantially higher coverage, demonstrating its ability to maintain and exploit a diverse repertoire of strategies.

Method: Proposes a controller that combines bucketized hysteresis with proportional feedback, providing a framework for parameter selection to enable accurate spend rate tracking.

Result: Experiments in real-world auctions show significant improvements: 13% reduction in pacing error and 54% reduction in λ-volatility compared to baseline methods.

Conclusion: The approach bridges control theory with advertising systems, offering a scalable and reliable budget pacing solution with particular benefits for small-budget campaigns.

Abstract: Budget pacing is critical in online advertising to align spend with campaign goals under dynamic auctions. Existing pacing methods often rely on ad-hoc parameter tuning, which can be unstable and inefficient. We propose a principled controller that combines bucketized hysteresis with proportional feedback to provide stable and adaptive spend control. Our method provides a framework and analysis for parameter selection that enables accurate tracking of desired spend rates across campaigns. Experiments in real-world auctions demonstrate significant improvements in pacing accuracy and delivery consistency, reducing pacing error by 13% and $\lambda$-volatility by 54% compared to baseline method. By bridging control theory with advertising systems, our approach offers a scalable and reliable solution for budget pacing, with particular benefits for small-budget campaigns.

Method: Uses dual-network design: an error model predicts expected prediction error of dynamics model from previous iteration. Intrinsic reward is based on difference between current and previous model errors.

Result: LPM converges faster, explores more states in maze experiments, and achieves higher extrinsic reward in Atari compared to state-of-the-art baselines. Theoretically shown to be zero-equivariant and monotone indicator of Information Gain.

Conclusion: LPM represents a paradigm shift for noise-robust exploration by rewarding learnable transitions rather than unlearnable noise, with better sample efficiency and computational performance.

Abstract: When there exists an unlearnable source of randomness (noisy-TV) in the environment, a naively intrinsic reward driven exploring agent gets stuck at that source of randomness and fails at exploration. Intrinsic reward based on uncertainty estimation or distribution similarity, while eventually escapes noisy-TVs as time unfolds, suffers from poor sample efficiency and high computational cost. Inspired by recent findings from neuroscience that humans monitor their improvements during exploration, we propose a novel method for intrinsically-motivated exploration, named Learning Progress Monitoring (LPM). During exploration, LPM rewards model improvements instead of prediction error or novelty, effectively rewards the agent for observing learnable transitions rather than the unlearnable transitions. We introduce a dual-network design that uses an error model to predict the expected prediction error of the dynamics model in its previous iteration, and use the difference between the model errors of the current iteration and previous iteration to guide exploration. We theoretically show that the intrinsic reward of LPM is zero-equivariant and a monotone indicator of Information Gain (IG), and that the error model is necessary to achieve monotonicity correspondence with IG. We empirically compared LPM against state-of-the-art baselines in noisy environments based on MNIST, 3D maze with 160x120 RGB inputs, and Atari. Results show that LPM’s intrinsic reward converges faster, explores more states in the maze experiment, and achieves higher extrinsic reward in Atari. This conceptually simple approach marks a shift-of-paradigm of noise-robust exploration. For code to reproduce our experiments, see https://github.com/Akuna23Matata/LPM_exploration

Method: Proposes normalized linear quantization with quantization-aware expectation maximization for probabilistic model training, reducing bit width while maintaining performance.

Result: Successfully quantized HMM with 4096 hidden states to 8 bits without loss and 3 bits with acceptable loss, achieving 99% compression rate for HMM weights.

Conclusion: Norm-Q effectively reduces memory and bandwidth requirements for probabilistic symbolic models, enabling deployment on custom hardware with minimal performance impact.

Abstract: Hidden Markov models (HMM) are commonly used in generation tasks and have demonstrated strong capabilities in neuro-symbolic applications for the Markov property. These applications leverage the strengths of neural networks and symbolic reasoning to create robust and interpretable AI systems. However, they may inherit and amplify the shortcomings of both approaches. Both components require dense computation and data transfer, and their communication further hinders performance. This paper proposes Norm-Q, a normalized linear quantization approach for compressing probabilistic symbolic models, such as HMMs. We reduce the bit width of the data with minimal impact, thereby alleviating memory and bandwidth stress and enabling deployment on potential custom hardware. Our method introduces a normalized quantization-aware expectation maximization process for probabilistic model training. The experimental results show that Norm-Q achieves a higher compression rate with reasonable score loss compared to traditional quantization methods. In the case of the constrained generation task of large language models, we successfully quantize an HMM of 4096 hidden states to 8 bits without loss and, at most, 3 bits with acceptable loss. Notably, the Norm-Q method can achieve a compression rate of 99% for the weights of the HMM. The code is open source at https://github.com/superstarghy/Norm-Q.

Method: Proposed Time Series JEPA (TS-JEPA), a Joint-Embedding Predictive Architecture specifically designed for time series representation learning that operates in latent space rather than reconstructing input data.

Result: TS-JEPA matches or surpasses current state-of-the-art baselines on standard datasets for both classification and forecasting tasks, demonstrating strong performance balance across diverse applications.

Conclusion: TS-JEPA provides a robust foundation for learning general time series representations and lays groundwork for future time series foundation models based on Joint Embedding architectures.

Abstract: Self-supervised learning has seen great success recently in unsupervised representation learning, enabling breakthroughs in natural language and image processing. However, these methods often rely on autoregressive and masked modeling, which aim to reproduce masked information in the input, which can be vulnerable to the presence of noise or confounding variables. To address this problem, Joint-Embedding Predictive Architectures (JEPA) has been introduced with the aim to perform self-supervised learning in the latent space. To leverage these advancements in the domain of time series, we introduce Time Series JEPA (TS-JEPA), an architecture specifically adapted for time series representation learning. We validate TS-JEPA on both classification and forecasting, showing that it can match or surpass current state-of-the-art baselines on different standard datasets. Notably, our approach demonstrates a strong performance balance across diverse tasks, indicating its potential as a robust foundation for learning general representations. Thus, this work lays the groundwork for developing future time series foundation models based on Joint Embedding.

Method: Proposes a performance-aware scheduling strategy using Kalman filter-based estimator to track task learning benefits and allocate additional demonstrations across tasks where the multitask policy underperforms.

Result: Achieves high performance across all tasks with substantially fewer expert demonstrations, and the visual policy shows better zero-shot transfer to real robots compared to naive DAgger and Behavior Cloning.

Conclusion: The framework enables efficient multitask policy learning with reduced data requirements and improved real-world transfer performance.

Abstract: Generalist robot policies that can perform many tasks typically require extensive expert data or simulations for training. In this work, we propose a novel Data-Efficient multitask DAgger framework that distills a single multitask policy from multiple task-specific expert policies. Our approach significantly increases the overall task success rate by actively focusing on tasks where the multitask policy underperforms. The core of our method is a performance-aware scheduling strategy that tracks how much each task’s learning process benefits from the amount of data, using a Kalman filter-based estimator to robustly decide how to allocate additional demonstrations across tasks. We validate our approach on MetaWorld, as well as a suite of diverse drawer-opening tasks in IsaacLab. The resulting policy attains high performance across all tasks while using substantially fewer expert demonstrations, and the visual policy learned with our method in simulation shows better performance than naive DAgger and Behavior Cloning when transferring zero-shot to a real robot without using real data.

Method: Introduces a robustness-based nonconformity score, embeds a smooth CP layer directly into training, and uses a new loss function that simultaneously optimizes inference accuracy and CP prediction sets.

Result: Experiments show reduced uncertainty in predictions (high coverage with smaller prediction sets) and improved accuracy over state-of-the-art baselines on benchmark time-series tasks.

Conclusion: The proposed framework successfully enhances both reliability and interpretability of STL formulas while providing formal statistical guarantees.

Abstract: Signal Temporal Logic (STL) inference seeks to extract human-interpretable rules from time-series data, but existing methods lack formal confidence guarantees for the inferred rules. Conformal prediction (CP) is a technique that can provide statistical correctness guarantees, but is typically applied as a post-training wrapper without improving model learning. Instead, we introduce an end-to-end differentiable CP framework for STL inference that enhances both reliability and interpretability of the resulting formulas. We introduce a robustness-based nonconformity score, embed a smooth CP layer directly into training, and employ a new loss function that simultaneously optimizes inference accuracy and CP prediction sets with a single term. Following training, an exact CP procedure delivers statistical guarantees for the learned STL formulas. Experiments on benchmark time-series tasks show that our approach reduces uncertainty in predictions (i.e., it achieves high coverage while reducing prediction set size), and improves accuracy (i.e., the number of misclassifications when using a fixed threshold) over state-of-the-art baselines.

Method: Uses demographic-aware diffusion framework with: 1) forward process with frequency-domain blurring and temporal noise interference; 2) reverse process with temporal multi-scale generation and frequency deblurring; 3) KNN-based clustering with contrastive learning for demographic-specific ECG translation.

Result: Extensive experiments show P2Es outperforms baseline models in 12-lead ECG reconstruction.

Conclusion: P2Es successfully bridges the gap between cumbersome 12-lead ECG systems and limited PPG-based methods by generating clinically valid 12-lead ECG from PPG signals.

Abstract: The 12-lead electrocardiogram (ECG) is the gold standard for cardiovascular monitoring, offering superior diagnostic granularity and specificity compared to photoplethysmography (PPG). However, existing 12-lead ECG systems rely on cumbersome multi-electrode setups, limiting sustained monitoring in ambulatory settings, while current PPG-based methods fail to reconstruct multi-lead ECG due to the absence of inter-lead constraints and insufficient modeling of spatial-temporal dependencies across leads. To bridge this gap, we introduce P2Es, an innovative demographic-aware diffusion framework designed to generate clinically valid 12-lead ECG from PPG signals via three key innovations. Specifically, in the forward process, we introduce frequency-domain blurring followed by temporal noise interference to simulate real-world signal distortions. In the reverse process, we design a temporal multi-scale generation module followed by frequency deblurring. In particular, we leverage KNN-based clustering combined with contrastive learning to assign affinity matrices for the reverse process, enabling demographic-specific ECG translation. Extensive experimental results show that P2Es outperforms baseline models in 12-lead ECG reconstruction.

Method: Introduces page-node graph structure that clusters similar vectors into page nodes aligned with physical SSD pages, uses co-designed disk data layout with merging technique to store only representative vectors and topology information, and implements memory management with lightweight indexing and coordinated memory-disk data allocation.

Result: Significantly outperforms state-of-the-art disk-based ANNS methods with 1.85x-10.83x higher throughput and 51.7%-91.9% lower latency across different datasets and memory budgets, while maintaining comparable high recall accuracy.

Conclusion: PageANN provides a high-performance and scalable solution for disk-based approximate nearest neighbor search by optimizing I/O efficiency through page-node alignment and intelligent memory management.

Abstract: Approximate Nearest Neighbor Search (ANNS), as the core of vector databases (VectorDBs), has become widely used in modern AI and ML systems, powering applications from information retrieval to bio-informatics. While graph-based ANNS methods achieve high query efficiency, their scalability is constrained by the available host memory. Recent disk-based ANNS approaches mitigate memory usage by offloading data to Solid-State Drives (SSDs). However, they still suffer from issues such as long I/O traversal path, misalignment with storage I/O granularity, and high in-memory indexing overhead, leading to significant I/O latency and ultimately limiting scalability for large-scale vector search. In this paper, we propose PageANN, a disk-based approximate nearest neighbor search (ANNS) framework designed for high performance and scalability. PageANN introduces a page-node graph structure that aligns logical graph nodes with physical SSD pages, thereby shortening I/O traversal paths and reducing I/O operations. Specifically, similar vectors are clustered into page nodes, and a co-designed disk data layout leverages this structure with a merging technique to store only representative vectors and topology information, avoiding unnecessary reads. To further improve efficiency, we design a memory management strategy that combines lightweight indexing with coordinated memory-disk data allocation, maximizing host memory utilization while minimizing query latency and storage overhead. Experimental results show that PageANN significantly outperforms state-of-the-art (SOTA) disk-based ANNS methods, achieving 1.85x-10.83x higher throughput and 51.7%-91.9% lower latency across different datasets and memory budgets, while maintaining comparable high recall accuracy.

Method: Formulates OOD detection as preference optimization over estimated OOD affinity between in-distribution and OOD samples using pairwise learning objective that essentially optimizes AUROC.

Result: Achieves up to 45.8%, 43.9%, and 24.3% improvements in AUROC under distribution shifts of size, scaffold, and assay respectively across five real-world molecular datasets.

Conclusion: Mole-PAIR significantly enhances OOD detection capabilities of existing molecular foundation models through cost-effective post-training.

Abstract: Molecular foundation models are rapidly advancing scientific discovery, but their unreliability on out-of-distribution (OOD) samples severely limits their application in high-stakes domains such as drug discovery and protein design. A critical failure mode is chemical hallucination, where models make high-confidence yet entirely incorrect predictions for unknown molecules. To address this challenge, we introduce Molecular Preference-Aligned Instance Ranking (Mole-PAIR), a simple, plug-and-play module that can be flexibly integrated with existing foundation models to improve their reliability on OOD data through cost-effective post-training. Specifically, our method formulates the OOD detection problem as a preference optimization over the estimated OOD affinity between in-distribution (ID) and OOD samples, achieving this goal through a pairwise learning objective. We show that this objective essentially optimizes AUROC, which measures how consistently ID and OOD samples are ranked by the model. Extensive experiments across five real-world molecular datasets demonstrate that our approach significantly improves the OOD detection capabilities of existing molecular foundation models, achieving up to 45.8%, 43.9%, and 24.3% improvements in AUROC under distribution shifts of size, scaffold, and assay, respectively.

Method: Integration of LLMs with circuit simulators and custom data analysis functions using prompt engineering and Chain-of-Thought reasoning for iterative design exploration and refinement.

Result: OpenAI o3 successfully optimized a 20-transistor CMOS operational amplifier across 180nm to 90nm nodes, achieving user-intended targets with maximum 20 iterations, demonstrating adaptability and robustness at advanced technology nodes.

Conclusion: LLM-based AI agents like EEsizer can effectively automate transistor sizing for AMS circuits, showing promise for reducing design complexity and human intervention while maintaining design robustness through variation analysis.

Abstract: The design of Analog and Mixed-Signal (AMS) integrated circuits (ICs) often involves significant manual effort, especially during the transistor sizing process. While Machine Learning techniques in Electronic Design Automation (EDA) have shown promise in reducing complexity and minimizing human intervention, they still face challenges such as numerous iterations and a lack of knowledge about AMS circuit design. Recently, Large Language Models (LLMs) have demonstrated significant potential across various fields, showing a certain level of knowledge in circuit design and indicating their potential to automate the transistor sizing process. In this work, we propose EEsizer, an LLM-based AI agent that integrates large language models with circuit simulators and custom data analysis functions, enabling fully automated, closed-loop transistor sizing without relying on external knowledge. By employing prompt engineering and Chain-of-Thought reasoning, the agent iteratively explores design directions, evaluates performance, and refines solutions with minimal human intervention. We first benchmarked 8 LLMs on six basic circuits and selected three high-performing models to optimize a 20-transistor CMOS operational amplifier, targeting multiple performance metrics, including rail-to-rail operation from 180 nm to 90 nm technology nodes. Notably, OpenAI o3 successfully achieved the user-intended target at 90 nm across three different test groups, with a maximum of 20 iterations, demonstrating adaptability and robustness at advanced nodes. To assess design robustness, we manually designed a bias circuit and performed a variation analysis using Gaussian-distributed variations on transistor dimensions and threshold voltages.

Method: SD-FM uses semidiscrete optimal transport formulation that leverages the finite size of target datasets. It estimates a dual potential vector using SGD and matches noise vectors to data points via maximum inner product search (MIPS), avoiding quadratic dependency on batch size.

Result: SD-FM outperforms both standard FM and OT-FM on all training metrics and inference budget constraints across multiple datasets, for both unconditional and conditional generation tasks.

Conclusion: The semidiscrete approach successfully addresses the computational limitations of OT-FM while delivering superior performance, making flow matching with optimal transport more practical and effective.

Abstract: Flow models parameterized as time-dependent velocity fields can generate data from noise by integrating an ODE. These models are often trained using flow matching, i.e. by sampling random pairs of noise and target points $(\mathbf{x}_0,\mathbf{x}_1)$ and ensuring that the velocity field is aligned, on average, with $\mathbf{x}_1-\mathbf{x}_0$ when evaluated along a segment linking $\mathbf{x}_0$ to $\mathbf{x}_1$. While these pairs are sampled independently by default, they can also be selected more carefully by matching batches of $n$ noise to $n$ target points using an optimal transport (OT) solver. Although promising in theory, the OT flow matching (OT-FM) approach is not widely used in practice. Zhang et al. (2025) pointed out recently that OT-FM truly starts paying off when the batch size $n$ grows significantly, which only a multi-GPU implementation of the Sinkhorn algorithm can handle. Unfortunately, the costs of running Sinkhorn can quickly balloon, requiring $O(n^2/\varepsilon^2)$ operations for every $n$ pairs used to fit the velocity field, where $\varepsilon$ is a regularization parameter that should be typically small to yield better results. To fulfill the theoretical promises of OT-FM, we propose to move away from batch-OT and rely instead on a semidiscrete formulation that leverages the fact that the target dataset distribution is usually of finite size $N$. The SD-OT problem is solved by estimating a dual potential vector using SGD; using that vector, freshly sampled noise vectors at train time can then be matched with data points at the cost of a maximum inner product search (MIPS). Semidiscrete FM (SD-FM) removes the quadratic dependency on $n/\varepsilon$ that bottlenecks OT-FM. SD-FM beats both FM and OT-FM on all training metrics and inference budget constraints, across multiple datasets, on unconditional/conditional generation, or when using mean-flow models.

Method: Casts router training as a causal inference problem, viewing response evaluation as treatment assignment, and develops an integrative framework to correct preference-data bias and address data source imbalances.

Result: Numerical experiments show more accurate routing and improved cost-quality trade-off compared to standard approaches.

Conclusion: The causal inference perspective effectively addresses bias in preference-based data and enables more robust and efficient LLM routing.

Abstract: In language tasks that require extensive human–model interaction, deploying a single “best” model for every query can be expensive. To reduce inference cost while preserving the quality of the responses, a large language model (LLM) router selects the most appropriate model from a pool of candidates for each query. A central challenge to training a high-quality router is the scarcity of reliable supervision. Gold-standard data (e.g., expert-verified labels or rubric-based scores) provide accurate quality evaluations of LLM responses but are costly and difficult to scale. In contrast, preference-based data, collected via crowdsourcing or LLM-as-a-judge systems, are cheaper and more scalable, yet often biased in reflecting the true quality of responses. We cast the problem of LLM router training with combined gold-standard and preference-based data into a causal inference framework by viewing the response evaluation mechanism as the treatment assignment. This perspective further reveals that the bias in preference-based data corresponds to the well-known causal estimand: the conditional average treatment effect. Based on this new perspective, we develop an integrative causal router training framework that corrects preference-data bias, address imbalances between two data sources, and improve routing robustness and efficiency. Numerical experiments demonstrate that our approach delivers more accurate routing and improves the trade-off between cost and quality.

Method: Developed a queue prioritization algorithm that integrates generative modeling with active learning to prioritize top design candidates in distributed workflows.

Result: Active learning approach increased high-quality molecular candidates from 281 to 604 out of 1000 novel candidates for carbon capture applications.

Conclusion: Combining generative AI with active learning prevents resource waste on nonsensical candidates, halts generative model decay, and significantly improves discovery of high-performing designs.

Abstract: Generative AI poses both opportunities and risks for solving inverse design problems in the sciences. Generative tools provide the ability to expand and refine a search space autonomously, but do so at the cost of exploring low-quality regions until sufficiently fine tuned. Here, we propose a queue prioritization algorithm that combines generative modeling and active learning in the context of a distributed workflow for exploring complex design spaces. We find that incorporating an active learning model to prioritize top design candidates can prevent a generative AI workflow from expending resources on nonsensical candidates and halt potential generative model decay. For an existing generative AI workflow for discovering novel molecular structure candidates for carbon capture, our active learning approach significantly increases the number of high-quality candidates identified by the generative model. We find that, out of 1000 novel candidates, our workflow without active learning can generate an average of 281 high-performing candidates, while our proposed prioritization with active learning can generate an average 604 high-performing candidates.

Method: FedIF uses trajectory-based influence estimation with normalized and smoothed influence scores computed from lightweight gradient operations on client updates and a public validation set.

Result: FedIF achieves robustness comparable to or exceeding SV-based methods against label noise, gradient noise, and adversarial samples, while reducing aggregation overhead by up to 450x on CIFAR-10 and Fashion-MNIST datasets.

Conclusion: FedIF provides a practical, theoretically grounded, and scalable alternative to Shapley-value-based approaches for efficient and robust federated learning in real-world deployments.

Abstract: Federated learning (FL) faces persistent robustness challenges due to non-IID data distributions and adversarial client behavior. A promising mitigation strategy is contribution evaluation, which enables adaptive aggregation by quantifying each client’s utility to the global model. However, state-of-the-art Shapley-value-based approaches incur high computational overhead due to repeated model reweighting and inference, which limits their scalability. We propose FedIF, a novel FL aggregation framework that leverages trajectory-based influence estimation to efficiently compute client contributions. FedIF adapts decentralized FL by introducing normalized and smoothed influence scores computed from lightweight gradient operations on client updates and a public validation set. Theoretical analysis demonstrates that FedIF yields a tighter bound on one-step global loss change under noisy conditions. Extensive experiments on CIFAR-10 and Fashion-MNIST show that FedIF achieves robustness comparable to or exceeding SV-based methods in the presence of label noise, gradient noise, and adversarial samples, while reducing aggregation overhead by up to 450x. Ablation studies confirm the effectiveness of FedIF’s design choices, including local weight normalization and influence smoothing. Our results establish FedIF as a practical, theoretically grounded, and scalable alternative to Shapley-value-based approaches for efficient and robust FL in real-world deployments.

Method: The proposed method extends ICRL to constrained Markov Decision Processes, where the agent’s policy neural networks receive expanded context inputs (history experience) and simultaneously optimize for reward maximization and cost minimization during the parameter-update-free adaptation process.

Result: The agent demonstrates active response to cost tolerance thresholds - behaving more aggressively with higher cost budgets and more conservatively with lower cost budgets, showing effective adaptation to safety constraints without parameter updates.

Conclusion: This work successfully introduces safety considerations into ICRL, creating the first method that enables safe adaptation in constrained environments while maintaining the core benefit of parameter-update-free learning across out-of-distribution tasks.

Abstract: In-context reinforcement learning (ICRL) is an emerging RL paradigm where the agent, after some pretraining procedure, is able to adapt to out-of-distribution test tasks without any parameter updates. The agent achieves this by continually expanding the input (i.e., the context) to its policy neural networks. For example, the input could be all the history experience that the agent has access to until the current time step. The agent’s performance improves as the input grows, without any parameter updates. In this work, we propose the first method that promotes the safety of ICRL’s adaptation process in the framework of constrained Markov Decision Processes. In other words, during the parameter-update-free adaptation process, the agent not only maximizes the reward but also minimizes an additional cost function. We also demonstrate that our agent actively reacts to the threshold (i.e., budget) of the cost tolerance. With a higher cost budget, the agent behaves more aggressively, and with a lower cost budget, the agent behaves more conservatively.

Method: The paper surveys various ML-enhanced BBO algorithms including neural networks with modular frameworks, zeroth-order adaptive methods, automated BBO, distributed optimization, Bayesian optimization variants, transformer-based optimizers, diffusion models, surrogate-assisted RL, and other hybrid approaches.

Result: The survey covers representative algorithms and benchmark efforts that demonstrate how ML and RL can enhance classical BBO methods, providing more expressive surrogates, adaptive updates, meta-learning capabilities, and improved robustness.

Conclusion: Machine learning and reinforcement learning are transforming classical inexact solvers into more scalable, robust, and adaptive frameworks for real-world black-box optimization problems.

Abstract: Black-box optimization (BBO) addresses problems where objectives are accessible only through costly queries without gradients or explicit structure. Classical derivative-free methods – line search, direct search, and model-based solvers such as Bayesian optimization – form the backbone of BBO, yet often struggle in high-dimensional, noisy, or mixed-integer settings. Recent advances use machine learning (ML) and reinforcement learning (RL) to enhance BBO: ML provides expressive surrogates, adaptive updates, meta-learning portfolios, and generative models, while RL enables dynamic operator configuration, robustness, and meta-optimization across tasks. This paper surveys these developments, covering representative algorithms such as NNs with the modular model-based optimization framework (mlrMBO), zeroth-order adaptive momentum methods (ZO-AdaMM), automated BBO (ABBO), distributed block-wise optimization (DiBB), partition-based Bayesian optimization (SPBOpt), the transformer-based optimizer (B2Opt), diffusion-model-based BBO, surrogate-assisted RL for differential evolution (Surr-RLDE), robust BBO (RBO), coordinate-ascent model-based optimization with relative entropy (CAS-MORE), log-barrier stochastic gradient descent (LB-SGD), policy improvement with black-box (PIBB), and offline Q-learning with Mamba backbones (Q-Mamba). We also review benchmark efforts such as the NeurIPS 2020 BBO Challenge and the MetaBox framework. Overall, we highlight how ML and RL transform classical inexact solvers into more scalable, robust, and adaptive frameworks for real-world optimization.

Method: Propose binary sparse autoencoders (BAEs) and binary transcoders (BTCs) that constrain all activations to be zero or one, preventing uninterpretable information from being smuggled through continuous variation.

Result: Binarisation significantly improves interpretability and monosemanticity of discovered features while increasing reconstruction error. However, it also increases ultra-high frequency features, and frequency-adjusted interpretability scores show continuous sparse coders perform slightly better.

Conclusion: Polysemanticity may be an ineliminable property of neural activations, as binarisation improves some aspects of interpretability but introduces other challenges.

Abstract: Sparse autoencoders (SAEs) are used to decompose neural network activations into sparsely activating features, but many SAE features are only interpretable at high activation strengths. To address this issue we propose to use binary sparse autoencoders (BAEs) and binary transcoders (BTCs), which constrain all activations to be zero or one. We find that binarisation significantly improves the interpretability and monosemanticity of the discovered features, while increasing reconstruction error. By eliminating the distinction between high and low activation strengths, we prevent uninterpretable information from being smuggled in through the continuous variation in feature activations. However, we also find that binarisation increases the number of uninterpretable ultra-high frequency features, and when interpretability scores are frequency-adjusted, the scores for continuous sparse coders are slightly better than those of binary ones. This suggests that polysemanticity may be an ineliminable property of neural activations.

Method: EMP maps any score vector to an effective number N_eff inspired by the Inverse Simpson index, retaining the N_eff highest scoring entries and zeroing the rest, with optional scaling parameter β.

Result: EMP produces sparse models with performance comparable to original dense networks across MLPs, CNNs, Transformers/LLMs, and KAN models.

Conclusion: EMP provides a robust, parameter-free pruning rule that works universally across different pruning criteria and model architectures, with β=1 as the default effective threshold.

Abstract: We introduce Effective Model Pruning (EMP), a context-agnostic, parameter-free rule addressing a fundamental question about pruning: how many entries to keep. EMP does not prescribe how to score the parameters or prune the models; instead, it supplies a universal adaptive threshold that can be applied to any pruning criterion: weight magnitude, attention score, KAN importance score, or even feature-level signals such as image pixel, and used on structural parts or weights of the models. Given any score vector s, EMP maps s to a built-in effective number N_eff which is inspired by the Inverse Simpson index of contributors. Retaining the N_eff highest scoring entries and zeroing the remainder yields sparse models with performance comparable to the original dense networks across MLPs, CNNs, Transformers/LLMs, and KAN, in our experiments. By leveraging the geometry of the simplex, we derive a tight lower bound on the preserved mass s_eff (the sum of retained scores) over the corresponding ordered probability simplex associated with the score vector s. We further verify the effectiveness of N_eff by pruning the model with a scaled threshold \b{eta}*N_eff across a variety of criteria and models. Experiments suggest that the default \b{eta} = 1 yields a robust threshold for model pruning while \b{eta} not equal to 1 still serves as an optional adjustment to meet specific sparsity requirements.

Method: Integrates window-attention Transformers within a bidirectional GAN (BiGAN) with self-attention encoder-decoder architecture to capture complex spatio-temporal dependencies, using cycle consistency and adaptive anomaly scoring combining reconstruction error, latent space drift, and discriminator confidence.

Result: Achieves ROC-AUC of 0.95 and average precision of 0.996 on realistic hardware-in-the-loop PMU benchmark, significantly outperforming leading supervised and unsupervised methods, particularly for detecting subtle frequency and voltage deviations.

Conclusion: T-BiGAN demonstrates practical value for live, wide-area monitoring of power grids by providing effective unsupervised anomaly detection without requiring labeled fault data.

Abstract: Ensuring power grid resilience requires the timely and unsupervised detection of anomalies in synchrophasor data streams. We introduce T-BiGAN, a novel framework that integrates window-attention Transformers within a bidirectional Generative Adversarial Network (BiGAN) to address this challenge. Its self-attention encoder-decoder architecture captures complex spatio-temporal dependencies across the grid, while a joint discriminator enforces cycle consistency to align the learned latent space with the true data distribution. Anomalies are flagged in real-time using an adaptive score that combines reconstruction error, latent space drift, and discriminator confidence. Evaluated on a realistic hardware-in-the-loop PMU benchmark, T-BiGAN achieves an ROC-AUC of 0.95 and an average precision of 0.996, significantly outperforming leading supervised and unsupervised methods. It shows particular strength in detecting subtle frequency and voltage deviations, demonstrating its practical value for live, wide-area monitoring without relying on manually labeled fault data.

Method: Uses effective rank as a metric for information density, allocates ranks via Lagrange multiplier-based optimization to assign more capacity to information-dense layers, and rebalances ranks across attention layers considering their varying importance.

Result: Consistently outperforms SVD-LLM, ASVD, and Basis Sharing, achieving >15 lower perplexity with LLaMA-3-8B at 20% compression and up to 5% higher zero-shot reasoning accuracy with LLaMA-7B at 40% compression while maintaining higher throughput.

Conclusion: D-Rank provides an effective framework for layer-adaptive SVD compression that better preserves model performance by dynamically allocating computational resources based on layer information density.

Abstract: Large language models (LLMs) have rapidly scaled in size, bringing severe memory and computational challenges that hinder their deployment. Singular Value Decomposition (SVD)-based compression has emerged as an appealing post-training compression technique for LLMs, yet most existing methods apply a uniform compression ratio across all layers, implicitly assuming homogeneous information included in various layers. This overlooks the substantial intra-layer heterogeneity observed in LLMs, where middle layers tend to encode richer information while early and late layers are more redundant. In this work, we revisit the existing SVD-based compression method and propose D-Rank, a framework with layer-wise balanced Dynamic Rank allocation for LLMs compression. We first introduce effective rank as a principled metric to measure the information density of weight matrices, and then allocate ranks via a Lagrange multiplier-based optimization scheme to adaptively assign more capacity to groups with higher information density under a fixed compression ratio. Moreover, we rebalance the allocated ranks across attention layers to account for their varying importance and extend D-Rank to latest LLMs with grouped-query attention. Extensive experiments on various LLMs with different scales across multiple compression ratios demonstrate that D-Rank consistently outperforms SVD-LLM, ASVD, and Basis Sharing, achieving more than 15 lower perplexity with LLaMA-3-8B model on C4 datasets at 20% compression ratio and up to 5% higher zero-shot reasoning accuracy with LLaMA-7B model at 40% compression ratio while achieving even higher throughput.

Method: Introduces Swift, a single-step consistency model that enables autoregressive finetuning of probability flow models with continuous ranked probability score (CRPS) objective, eliminating need for multi-model ensembling or parameter perturbations.

Result: Swift produces skillful 6-hourly forecasts stable for up to 75 days, runs 39x faster than state-of-the-art diffusion baselines, and achieves forecast skill competitive with operational IFS ENS numerical model.

Conclusion: Swift represents a step toward efficient and reliable ensemble forecasting from medium-range to seasonal scales by combining the probabilistic framework of diffusion models with fast single-step inference.

Abstract: Diffusion models offer a physically grounded framework for probabilistic weather forecasting, but their typical reliance on slow, iterative solvers during inference makes them impractical for subseasonal-to-seasonal (S2S) applications where long lead-times and domain-driven calibration are essential. To address this, we introduce Swift, a single-step consistency model that, for the first time, enables autoregressive finetuning of a probability flow model with a continuous ranked probability score (CRPS) objective. This eliminates the need for multi-model ensembling or parameter perturbations. Results show that Swift produces skillful 6-hourly forecasts that remain stable for up to 75 days, running $39\times$ faster than state-of-the-art diffusion baselines while achieving forecast skill competitive with the numerical-based, operational IFS ENS. This marks a step toward efficient and reliable ensemble forecasting from medium-range to seasonal-scales.

Method: Analyzed preconditioning as p-th power of input covariance matrix within single-index teacher model, examining spectral bias effects on feature learning.

Result: Preconditioner’s spectral bias shapes learned features - favoring emphasized components and reducing sensitivity to suppressed ones. Generalization improves when bias aligns with teacher spectrum.

Conclusion: Preconditioning’s spectral bias significantly influences feature learning and generalization, with alignment to teacher spectrum being crucial for optimal performance.

Abstract: Preconditioning is widely used in machine learning to accelerate convergence on the empirical risk, yet its role on the expected risk remains underexplored. In this work, we investigate how preconditioning affects feature learning and generalization performance. We first show that the input information available to the model is conveyed solely through the Gram matrix defined by the preconditioner’s metric, thereby inducing a controllable spectral bias on feature learning. Concretely, instantiating the preconditioner as the $p$-th power of the input covariance matrix and within a single-index teacher model, we prove that in generalization, the exponent $p$ and the alignment between the teacher and the input spectrum are crucial factors. We further investigate how the interplay between these factors influences feature learning from three complementary perspectives: (i) Robustness to noise, (ii) Out-of-distribution generalization, and (iii) Forward knowledge transfer. Our results indicate that the learned feature representations closely mirror the spectral bias introduced by the preconditioner – favoring components that are emphasized and exhibiting reduced sensitivity to those that are suppressed. Crucially, we demonstrate that generalization is significantly enhanced when this spectral bias is aligned with that of the teacher.

Method: Integrates set transformer embedding for variable sensor locations and conditional variational autoencoder (cVAE) to approximate the conditional distribution of solution operators, minimizing negative ELBO for principled uncertainty estimation.

Result: Numerical experiments on deterministic and stochastic PDEs (including Navier-Stokes) demonstrate robustness and effectiveness in handling sparse measurements while maintaining predictive accuracy.

Conclusion: UQ-SONet successfully addresses key limitations of existing operator learning methods by providing built-in uncertainty quantification and handling variable, sparse sensor configurations.

Abstract: Learning operators from data is central to scientific machine learning. While DeepONets are widely used for their ability to handle complex domains, they require fixed sensor numbers and locations, lack mechanisms for uncertainty quantification (UQ), and are thus limited in practical applicability. Recent permutationinvariant extensions, such as the Variable-Input Deep Operator Network (VIDON), relax these sensor constraints but still rely on sufficiently dense observations and cannot capture uncertainties arising from incomplete measurements or from operators with inherent randomness. To address these challenges, we propose UQ-SONet, a permutation-invariant operator learning framework with built-in UQ. Our model integrates a set transformer embedding to handle sparse and variable sensor locations, and employs a conditional variational autoencoder (cVAE) to approximate the conditional distribution of the solution operator. By minimizing the negative ELBO, UQ-SONet provides principled uncertainty estimation while maintaining predictive accuracy. Numerical experiments on deterministic and stochastic PDEs, including the Navier-Stokes equation, demonstrate the robustness and effectiveness of the proposed framework.

Method: BaB-prob iteratively divides problems into subproblems by splitting preactivations and uses linear bound propagation to bound probabilities for each subproblem. It introduces uncertainty level and efficient splitting strategies (BaB-prob-ordered and BaB+BaBSR-prob).

Result: The approach consistently outperforms state-of-the-art methods on untrained networks, MNIST and CIFAR-10 models, and VNN-COMP 2025 benchmarks, especially in medium- to high-dimensional input problems.

Conclusion: BaB-prob provides an effective probabilistic verification framework for neural networks that is sound and complete for feedforward-ReLU networks, demonstrating superior performance compared to existing approaches.

Abstract: Branch-and-bound with preactivation splitting has been shown highly effective for deterministic verification of neural networks. In this paper, we extend this framework to the probabilistic setting. We propose BaB-prob that iteratively divides the original problem into subproblems by splitting preactivations and leverages linear bounds computed by linear bound propagation to bound the probability for each subproblem. We prove soundness and completeness of BaB-prob for feedforward-ReLU neural networks. Furthermore, we introduce the notion of uncertainty level and design two efficient strategies for preactivation splitting, yielding BaB-prob-ordered and BaB+BaBSR-prob. We evaluate BaB-prob on untrained networks, MNIST and CIFAR-10 models, respectively, and VNN-COMP 2025 benchmarks. Across these settings, our approach consistently outperforms state-of-the-art approaches in medium- to high-dimensional input problems.

Method: PWMPR starts from a sparse seed, adds edges guided by path-weight magnitude product scores, uses randomization to mitigate bottlenecks, and employs a logistic-fit rule to detect when accuracy plateaus and stop growth.

Result: PWMPR approaches the performance of IMP-derived lottery tickets (though at higher density) with substantially lower computational cost (~1.5x dense training vs. 3-4x for IMP) on CIFAR, TinyImageNet, and ImageNet datasets.

Conclusion: Growth-based density discovery is established as a promising paradigm that complements existing pruning and dynamic sparsity approaches, offering an efficient alternative for sparse network training.

Abstract: The lottery ticket hypothesis suggests that dense networks contain sparse subnetworks that can be trained in isolation to match full-model performance. Existing approaches-iterative pruning, dynamic sparse training, and pruning at initialization-either incur heavy retraining costs or assume the target density is fixed in advance. We introduce Path Weight Magnitude Product-biased Random growth (PWMPR), a constructive sparse-to-dense training paradigm that grows networks rather than pruning them, while automatically discovering their operating density. Starting from a sparse seed, PWMPR adds edges guided by path-kernel-inspired scores, mitigates bottlenecks via randomization, and stops when a logistic-fit rule detects plateauing accuracy. Experiments on CIFAR, TinyImageNet, and ImageNet show that PWMPR approaches the performance of IMP-derived lottery tickets-though at higher density-at substantially lower cost (~1.5x dense vs. 3-4x for IMP). These results establish growth-based density discovery as a promising paradigm that complements pruning and dynamic sparsity.

Method: Given a question and gold answer, the model generates a chain-of-thought and then produces an abstract hint containing core knowledge. For hard samples with 0% pass rate, the hint is injected to regenerate trajectories, boosting pass rates from 0% to non-zero.

Result: NuRL achieves consistent improvements across 6 benchmarks and 3 models, raising the model’s upper limit while GRPO leaves pass@1024 unchanged from the base model. Hints are most effective when abstract, high-level, and applied after GRPO convergence.

Conclusion: NuRL successfully enables learning from previously unsolvable problems through self-generated hints, complementing test-time scaling and pushing the upper bound of LLM reasoning capabilities.

Abstract: Current online reinforcement learning (RL) algorithms like GRPO share a key limitation in LLM reasoning: they cannot learn from problems that are “unsolvable” to the model. In other words, they can only improve performance on problems where the model is capable of exploring the correct answer. Consequently, the model’s “upper limit” remains unchanged after RL training, even though the likelihood of solving easier, solvable problems may increase. These hard samples cannot contribute to training, as no rollouts yield rewards and thus no gradients are produced. To unlock learning from these hard samples, we propose NuRL, a “nudging” method that aims to push the upper bound of LLM reasoning using self-generated hints, i.e., abstract cues that help reduce the problem difficulty for the model. Given a question and its gold answer, the model generates a CoT and then produces a hint containing the core knowledge needed to solve the problem. During training, we generate G rollouts from the base policy and use the pass rate to decide whether the hint should be injected. For hard samples with a 0% pass rate, we inject the hint and regenerate a new batch of trajectories. This yields two benefits: (1) the hint boosts pass rates (from 0% to non-zero), thereby introducing training signals for previously unsolvable samples, and (2) the hints are self-generated, avoiding distributional shift and do not rely on external models. NuRL achieves consistent improvements across 6 benchmarks and 3 models, while remaining complementary to test-time scaling. Notably, NuRL can raise the model’s upper limit, whereas GRPO leaves pass@1024 unchanged from the base model. Furthermore, we present a systematic study of what makes an effective hint and when hints are most useful. Interestingly, the best hints are abstract and high-level, and are most beneficial when applied necessarily and after GRPO has converged.

Method: Used EEG data from right-left hand motor imagery, segmented into 19x200 arrays at 200Hz sampling. Proposed a BiLSTM-BiGRU attention-based model and compared with XGBoost, EEGNet, and transformer models. Integrated with Tkinter interface for wheelchair simulation.

Result: BiLSTM-BiGRU model achieved superior test accuracy of 92.26% and mean cross-validation accuracy of 90.13%, outperforming baseline models.

Conclusion: The attention-based BiLSTM-BiGRU model shows strong potential for BCI applications, providing accurate and intuitive wheelchair control through motor imagery EEG signals.

Abstract: This paper presents an Artificial Intelligence (AI) integrated novel approach to Brain-Computer Interface (BCI)-based wheelchair development, utilizing a motor imagery right-left-hand movement mechanism for control. The system is designed to simulate wheelchair navigation based on motor imagery right and left-hand movements using electroencephalogram (EEG) data. A pre-filtered dataset, obtained from an open-source EEG repository, was segmented into arrays of 19x200 to capture the onset of hand movements. The data was acquired at a sampling frequency of 200Hz. The system integrates a Tkinter-based interface for simulating wheelchair movements, offering users a functional and intuitive control system. We propose a BiLSTM-BiGRU model that shows a superior test accuracy of 92.26% as compared with various machine learning baseline models, including XGBoost, EEGNet, and a transformer-based model. The Bi-LSTM-BiGRU attention-based model achieved a mean accuracy of 90.13% through cross-validation, showcasing the potential of attention mechanisms in BCI applications.

Method: Introduces a multimodal interaction-aware router that dispatches tokens to experts based on temporal interaction patterns, using a new formulation of temporal multimodal interaction dynamics.

Result: Comprehensive experiments on multimodal benchmarks show enhanced performance and improved interpretability compared to standard MoE approaches.

Conclusion: Leveraging temporal interaction dynamics for MoE routing enables experts to develop generalizable interaction-processing skills and improves multimodal model effectiveness.

Abstract: Mixture-of-Experts (MoE) architectures have become pivotal for large-scale multimodal models. However, their routing mechanisms typically overlook the informative, time-varying interaction dynamics between modalities. This limitation hinders expert specialization, as the model cannot explicitly leverage intrinsic modality relationships for effective reasoning. To address this, we propose a novel framework that guides MoE routing using quantified temporal interaction. A multimodal interaction-aware router learns to dispatch tokens to experts based on the nature of their interactions. This dynamic routing encourages experts to acquire generalizable interaction-processing skills rather than merely learning task-specific features. Our framework builds on a new formulation of temporal multimodal interaction dynamics, which are used to guide expert routing. We first demonstrate that these temporal multimodal interactions reveal meaningful patterns across applications, and then show how they can be leveraged to improve both the design and performance of MoE-based models. Comprehensive experiments on challenging multimodal benchmarks validate our approach, demonstrating both enhanced performance and improved interpretability.

Method: Train a lightweight autoencoder to produce binary masks that highlight critical image regions, using activation alignment across layers, output consistency, sparsity priors, compactness, and robustness constraints. Also introduce a circuit readout procedure to identify active channels and construct channel-level graphs.

Result: The approach generates minimal explanations that preserve model decisions while being concise and human-readable, enabling mechanistic interpretation of model internals through channel-level analysis.

Conclusion: The method provides a practical bridge between minimal input-level explanations and mechanistic understanding of internal computations driving model decisions.

Abstract: Machine learning models, by virtue of training, learn a large repertoire of decision rules for any given input, and any one of these may suffice to justify a prediction. However, in high-dimensional input spaces, such rules are difficult to identify and interpret. In this paper, we introduce an activation-matching based approach to generate minimal and faithful explanations for the decisions of pre-trained image classifiers. We aim to identify minimal explanations that not only preserve the model’s decision but are also concise and human-readable. To achieve this, we train a lightweight autoencoder to produce binary masks that learns to highlight the decision-wise critical regions of an image while discarding irrelevant background. The training objective integrates activation alignment across multiple layers, consistency at the output label, priors that encourage sparsity, and compactness, along with a robustness constraint that enforces faithfulness. The minimal explanations so generated also lead us to mechanistically interpreting the model internals. In this regard we also introduce a circuit readout procedure wherein using the explanation’s forward pass and gradients, we identify active channels and construct a channel-level graph, scoring inter-layer edges by ingress weight magnitude times source activation and feature-to-class links by classifier weight magnitude times feature activation. Together, these contributions provide a practical bridge between minimal input-level explanations and a mechanistic understanding of the internal computations driving model decisions.

Method: Introduces parsimony promoting regularizer based on row-wise L∞ norm of coefficient matrix, derived from probabilistic model with Beta-Bernoulli priors, with theoretical hyperparameter selection.

Result: Achieves 20% RMSE reduction, enhanced representation sparsity, and uses fewer than one-tenth of available dictionary atoms while validating theoretical analysis.

Conclusion: The proposed method effectively reduces dictionary redundancy and improves reconstruction quality through atom-level sparsity regularization with solid theoretical foundations.

Abstract: Dictionary learning is traditionally formulated as an $L_1$-regularized signal reconstruction problem. While recent developments have incorporated discriminative, hierarchical, or generative structures, most approaches rely on encouraging representation sparsity over individual samples that overlook how atoms are shared across samples, resulting in redundant and sub-optimal dictionaries. We introduce a parsimony promoting regularizer based on the row-wise $L_\infty$ norm of the coefficient matrix. This additional penalty encourages entire rows of the coefficient matrix to vanish, thereby reducing the number of dictionary atoms activated across the dataset. We derive the formulation from a probabilistic model with Beta-Bernoulli priors, which provides a Bayesian interpretation linking the regularization parameters to prior distributions. We further establish theoretical calculation for optimal hyperparameter selection and connect our formulation to both Minimum Description Length, Bayesian model selection and pathlet learning. Extensive experiments on benchmark datasets demonstrate that our method achieves substantially improved reconstruction quality (with a 20% reduction in RMSE) and enhanced representation sparsity, utilizing fewer than one-tenth of the available dictionary atoms, while empirically validating our theoretical analysis.

Method: Uses smoothed conformal scores with top-K certainty measure, online weight-update algorithm driven by pseudo coverage, domain-shift detector for adaptive human supervision, and staged update scheme balancing human-labeled and model-labeled data.

Result: Extensive experiments show CPATTA consistently outperforms state-of-the-art ATTA methods by around 5% in accuracy.

Conclusion: CPATTA successfully brings principled, coverage-guaranteed uncertainty to ATTA, significantly improving data selection efficiency and model robustness under domain shift.

Abstract: Active Test-Time Adaptation (ATTA) improves model robustness under domain shift by selectively querying human annotations at deployment, but existing methods use heuristic uncertainty measures and suffer from low data selection efficiency, wasting human annotation budget. We propose Conformal Prediction Active TTA (CPATTA), which first brings principled, coverage-guaranteed uncertainty into ATTA. CPATTA employs smoothed conformal scores with a top-K certainty measure, an online weight-update algorithm driven by pseudo coverage, a domain-shift detector that adapts human supervision, and a staged update scheme balances human-labeled and model-labeled data. Extensive experiments demonstrate that CPATTA consistently outperforms the state-of-the-art ATTA methods by around 5% in accuracy. Our code and datasets are available at https://github.com/tingyushi/CPATTA.

Method: Propose training approach using pseudo labels generated by a VLM, leveraging deep neural networks’ inherent robustness to noisy labels.

Result: TSQA models are successfully trained with pseudo labels and surpass the performance of the VLM itself by leveraging large amounts of unlabeled data.

Conclusion: Pseudo labels from VLMs can effectively train TSQA models despite potential inaccuracies, enabling better performance than the VLM source through utilization of unlabeled data.

Abstract: Time-series question answering (TSQA) tasks face significant challenges due to the lack of labeled data. Alternatively, with recent advancements in large-scale models, vision-language models (VLMs) have demonstrated the potential to analyze time-series signals in a zero-shot manner. In this paper, we propose a training approach that uses pseudo labels generated by a VLM. Although VLMs can produce incorrect labels, TSQA models can still be effectively trained based on the property that deep neural networks are inherently robust to such noisy labels. Our experimental results demonstrate that TSQA models are not only successfully trained with pseudo labels, but also surpass the performance of the VLM itself by leveraging a large amount of unlabeled data.

Method: Proposes a network that accounts for spatial characteristics of human movements, incorporates forward and differential kinematics as additional loss components during training, and uses iterative refinement with joint state buffers during inference.

Result: The approach achieves high accuracy, smooth transitions between motions, and generalizes well to unseen subjects using only 5 IMUs.

Conclusion: The physics-informed learning framework successfully addresses limitations of conventional motion prediction by integrating domain knowledge and achieving physically feasible human motion prediction for safe human-robot collaboration.

Abstract: Accurate and physically feasible human motion prediction is crucial for safe and seamless human-robot collaboration. While recent advancements in human motion capture enable real-time pose estimation, the practical value of many existing approaches is limited by the lack of future predictions and consideration of physical constraints. Conventional motion prediction schemes rely heavily on past poses, which are not always available in real-world scenarios. To address these limitations, we present a physics-informed learning framework that integrates domain knowledge into both training and inference to predict human motion using inertial measurements from only 5 IMUs. We propose a network that accounts for the spatial characteristics of human movements. During training, we incorporate forward and differential kinematics functions as additional loss components to regularize the learned joint predictions. At the inference stage, we refine the prediction from the previous iteration to update a joint state buffer, which is used as extra inputs to the network. Experimental results demonstrate that our approach achieves high accuracy, smooth transitions between motions, and generalizes well to unseen subjects

Method: Simultaneously train a GNN and coarsen its graph by partitioning nodes via K-means clustering based on node embeddings. This allows node merging during training rather than as a preprocessing step, enabling clusters to adapt to the learning task instead of relying solely on graph connectivity and features.

Result: The method is validated on both homophilic and heterophilic node classification datasets. Visualization shows that node embeddings and their corresponding clusters adapt to the learning task during training, making the approach suitable for challenging scenarios like heterophilic data.

Conclusion: The proposed adaptive graph coarsening method successfully enables dynamic graph reduction during GNN training, with clusters that adapt to the learning task, making it effective for both homophilic and heterophilic data scenarios.

Abstract: We propose an adaptive graph coarsening method to jointly learn graph neural network (GNN) parameters and merge nodes via K-means clustering during training. As real-world graphs grow larger, processing them directly becomes increasingly challenging and sometimes infeasible. Tailoring algorithms to large-scale data may sacrifice performance, so we instead consider graph reduction to decrease the amount of data used during training. In particular, we propose a method to simultaneously train a GNN and coarsen its graph by partitioning nodes via K-means clustering based on their embeddings. Unlike past graph coarsening works, our approach allows us to merge nodes during training. Not only does this preclude coarsening as a preprocessing step, but our node clusters can adapt to the learning task instead of relying solely on graph connectivity and features. Thus, our method is amenable to scenarios that are challenging for other methods, such as heterophilic data. We validate our approach on both homophilic and heterophilic node classification datasets. We further visualize relationships between node embeddings and their corresponding clusters to illustrate that our coarsened graph adapts to the learning task during training.

Method: Learns layer-wise coefficients using unlabeled calibration data to align hidden states and logits with experts, with regularization and task-weighted losses. Expert Merging++ adds importance-guided chunking based on learned coefficients, task-vector magnitudes, and parameter counts.

Result: Outperforms both training-free and training-based merging baselines across MLLM (InternVL, Qwen2-VL) and LLM (Mistral) backbones, with Expert Merging++ achieving further gains and sometimes exceeding supervised Mixture Training.

Conclusion: Provides a label-free, parameter-efficient, and scalable approach to multi-expert model merging that effectively captures inter-layer variation and delivers superior performance.

Abstract: Model merging, which combines multiple domain-specialized experts into a single model, offers a practical path to endow Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) with broad capabilities without the cost of joint training or serving many models. However, training-free methods rely on hand-tuned coefficients, whereas training-based methods primarily align parameters rather than downstream task behavior and typically treat all layers uniformly, ignoring inter-layer heterogeneity. We introduce Expert Merging, a training-light method that learns a small set of layer-wise coefficients using only unlabeled calibration data. The coefficients are optimized to explicitly align the merged model’s hidden states and logits with those of the corresponding experts, with a coefficient regularizer for stability and task-weighted losses for controllable trade-offs. To capture inter-layer variation, Expert Merging++ augments this design with importance-guided chunking: a normalized layer-importance metric, derived from learned coefficients, task-vector magnitudes, and parameter counts, allocates more chunk-wise coefficients to high-importance layers while keeping low-importance layers lightweight. The result is a label-free, parameter-efficient, and scalable approach to multi-expert model merging across LLMs and MLLMs. Across MLLM backbones (InternVL and Qwen2-VL) and the LLM backbone (Mistral), our method surpasses strong training-free and training-based merging baselines, with Expert Merging++ delivering further gains and, in some cases, even exceeding supervised Mixture Training. The source code is available at https://github.com/Littleor/ExpertMerging.

Method: Uses mini-batch Unbalanced Optimal Transport to construct conditional vector fields and applies inverse reweighting based on a label-free majority score derived from density ratios between target distribution and UOT marginal.

Result: Outperforms existing flow matching baselines on long-tailed benchmarks while maintaining competitive performance on balanced datasets, with theoretical recovery of target distribution and empirical improvement in tail-class generation.

Conclusion: UOT-RFM provides an effective solution for generative modeling under class-imbalanced distributions without requiring class label information, addressing majority bias through principled geometric reweighting.

Abstract: Flow matching has recently emerged as a powerful framework for continuous-time generative modeling. However, when applied to long-tailed distributions, standard flow matching suffers from majority bias, producing minority modes with low fidelity and failing to match the true class proportions. In this work, we propose Unbalanced Optimal Transport Reweighted Flow Matching (UOT-RFM), a novel framework for generative modeling under class-imbalanced (long-tailed) distributions that operates without any class label information. Our method constructs the conditional vector field using mini-batch Unbalanced Optimal Transport (UOT) and mitigates majority bias through a principled inverse reweighting strategy. The reweighting relies on a label-free majority score, defined as the density ratio between the target distribution and the UOT marginal. This score quantifies the degree of majority based on the geometric structure of the data, without requiring class labels. By incorporating this score into the training objective, UOT-RFM theoretically recovers the target distribution with first-order correction ($k=1$) and empirically improves tail-class generation through higher-order corrections ($k

1$). Our model outperforms existing flow matching baselines on long-tailed benchmarks, while maintaining competitive performance on balanced datasets.