Method: Uses conformal prediction technique to compute uncertainty as a proxy for bias analysis, benchmarking 11 models against women and non-white annotators, comparing uncertainty metrics with performance-based metrics like F1 score.

Result: Some pre-trained models show high accuracy in predicting labels from minority groups but with low confidence in predictions, revealing that performance metrics alone don’t capture bias effectively.

Conclusion: Measuring model confidence through uncertainty analysis helps identify which annotator groups are better represented in pre-trained models, enabling debiasing before deployment.

Abstract: Automatic content moderation is crucial to ensuring safety in social media. Language Model-based classifiers are being increasingly adopted for this task, but it has been shown that they perpetuate racial and social biases. Even if several resources and benchmark corpora have been developed to challenge this issue, measuring the fairness of models in content moderation remains an open issue. In this work, we present an unsupervised approach that benchmarks models on the basis of their uncertainty in classifying messages annotated by people belonging to vulnerable groups. We use uncertainty, computed by means of the conformal prediction technique, as a proxy to analyze the bias of 11 models against women and non-white annotators and observe to what extent it diverges from metrics based on performance, such as the $F_1$ score. The results show that some pre-trained models predict with high accuracy the labels coming from minority groups, even if the confidence in their prediction is low. Therefore, by measuring the confidence of models, we are able to see which groups of annotators are better represented in pre-trained models and lead the debiasing process of these models before their effective use.

Method: Introduced AccessEval benchmark evaluating 21 closed- and open-source LLMs across 6 real-world domains and 9 disability types using paired Neutral and Disability-Aware Queries, with metrics for sentiment, social perception, and factual accuracy.

Result: Responses to disability-aware queries have more negative tone, increased stereotyping, and higher factual error compared to neutral queries, with notable variation by domain and disability type. Disabilities affecting hearing, speech, and mobility are disproportionately impacted.

Conclusion: These disparities reflect persistent ableism embedded in model behavior and demonstrate how such biases can translate into tangible harms for disabled users, reinforcing the importance of bias mitigation in day-to-day applications.

Abstract: Large Language Models (LLMs) are increasingly deployed across diverse domains but often exhibit disparities in how they handle real-life queries. To systematically investigate these effects within various disability contexts, we introduce \textbf{AccessEval (Accessibility Evaluation)}, a benchmark evaluating 21 closed- and open-source LLMs across 6 real-world domains and 9 disability types using paired Neutral and Disability-Aware Queries. We evaluated model outputs with metrics for sentiment, social perception, and factual accuracy. Our analysis reveals that responses to disability-aware queries tend to have a more negative tone, increased stereotyping, and higher factual error compared to neutral queries. These effects show notable variation by domain and disability type, with disabilities affecting hearing, speech, and mobility disproportionately impacted. These disparities reflect persistent forms of ableism embedded in model behavior. By examining model performance in real-world decision-making contexts, we better illuminate how such biases can translate into tangible harms for disabled users. This framing helps bridges the gap between technical evaluation and user impact, reinforcing importance of bias mitigation in day-to-day applications. Our dataset is publicly available at: https://huggingface.co/datasets/Srikant86/AccessEval

Method: RAR² constructs a thought process to uncover implicit knowledge requirements and uses it to guide retrieval and answer generation. It uses Direct Preference Optimization (DPO) on mixed preference pairs and includes test-time scaling strategies.

Result: Experiments show RAR² outperforms RAG baselines with or without fine-tuning across several biomedical question answering datasets.

Conclusion: The joint learning framework effectively addresses reasoning-intensive medical tasks by explicitly modeling the reasoning process to guide both retrieval and generation.

Abstract: Large Language Models (LLMs) have shown promising performance on diverse medical benchmarks, highlighting their potential in supporting real-world clinical tasks. Retrieval-Augmented Generation (RAG) has emerged as a key approach for mitigating knowledge gaps and hallucinations by incorporating external medical information. However, RAG still struggles with complex medical questions that require intensive reasoning, as surface-level input often fails to reflect the true knowledge needs of the task. Existing methods typically focus on refining queries without explicitly modeling the reasoning process, limiting their ability to retrieve and integrate clinically relevant knowledge. In this work, we propose RAR$^2$, a joint learning framework that improves both Reasoning-Augmented Retrieval and Retrieval-Augmented Reasoning. RAR$^2$ constructs a thought process to uncover implicit knowledge requirements and uses it to guide retrieval and answer generation. We build a training dataset of mixed preference pairs and apply Direct Preference Optimization (DPO) to train the model. Moreover, we design two test-time scaling strategies to explore the boundaries of our framework. Experiments demonstrate the effectiveness of RAR$^2$ across several biomedical question answering datasets, outperforming RAG baselines with or without fine-tuning.

Method: Created TRUEBench with 12 languages, intra-instance multilingual instructions, rigorous evaluation criteria for explicit/implicit constraints, complex multi-turn dialogues with accumulating constraints and context switches, and LLM-based constraint validation.

Result: TRUEBench is significantly more challenging than existing benchmarks - OpenAI o1 achieved only 69.07% overall pass rate, demonstrating the benchmark’s demanding nature.

Conclusion: TRUEBench provides a realistic and comprehensive assessment of LLMs in practical productivity settings, effectively highlighting both their capabilities and limitations.

Abstract: Large language models (LLMs) are increasingly integral as productivity assistants, but existing benchmarks fall short in rigorously evaluating their real-world instruction-following capabilities. Current benchmarks often (i) lack sufficient multilinguality, (ii) fail to capture the implicit constraints inherent in user requests, and (iii) overlook the complexities of multi-turn dialogue. To address these critical gaps and provide a more realistic assessment, we introduce TRUEBench (Trustworthy Real-world Usage Evaluation Benchmark)1, a novel benchmark specifically designed for LLM-based productivity assistants. TRUEBench distinguishes itself by featuring input prompts across 12 languages, incorporating intra-instance multilingual instructions, employing rigorous evaluation criteria to capture both explicit and implicit constraints, and including complex multi-turn dialogue scenarios with both accumulating constraints and context switches. Furthermore, to ensure reliability in evaluation, we refined constraints using an LLM validator. Extensive experiments demonstrate that TRUEBench presents significantly greater challenges than existing benchmarks; for instance, a strong model like OpenAI o1 achieved only a 69.07% overall pass rate. TRUEBench offers a demanding and realistic assessment of LLMs in practical productivity settings, highlighting their capabilities and limitations.

Method: Dynamic Attention Fusion (DAF) combines frozen text embeddings from a pretrained language model with acoustic features from a speech encoder, using an adaptive attention mechanism to weight each modality per utterance.

Result: DAF consistently outperforms both static fusion and unimodal baselines on a large multimodal benchmark, with notable gains in F1-score and reductions in prediction error.

Conclusion: Dynamic weighting strategy is crucial for modeling emotionally complex inputs, and effectively integrating verbal and non-verbal information offers a more robust foundation for sentiment prediction and affective computing applications.

Abstract: Traditional sentiment analysis has long been a unimodal task, relying solely on text. This approach overlooks non-verbal cues such as vocal tone and prosody that are essential for capturing true emotional intent. We introduce Dynamic Attention Fusion (DAF), a lightweight framework that combines frozen text embeddings from a pretrained language model with acoustic features from a speech encoder, using an adaptive attention mechanism to weight each modality per utterance. Without any finetuning of the underlying encoders, our proposed DAF model consistently outperforms both static fusion and unimodal baselines on a large multimodal benchmark. We report notable gains in F1-score and reductions in prediction error and perform a variety of ablation studies that support our hypothesis that the dynamic weighting strategy is crucial for modeling emotionally complex inputs. By effectively integrating verbal and non-verbal information, our approach offers a more robust foundation for sentiment prediction and carries broader impact for affective computing applications – from emotion recognition and mental health assessment to more natural human computer interaction.

Method: Develop a lightweight single-layer ‘sampler’ on top of existing large diffusion LMs. One full-model forward pass is followed by multiple forward passes of only this sampler layer to yield multiple unmasked tokens. The sampler is trained to mimic exact joint sampling from the frozen full model.

Result: When four tokens are unmasked per full-model denoising step, the method achieves MAUVE score of 0.87 (vs marginal baseline of 0.31) with respect to the true joint distribution. Shows effectiveness on both pretrained-only (Dream-7B-Base) and instruction-tuned (Dream-7B-Instruct) models for language modeling and math & coding tasks.

Conclusion: The proposed approximate joint sampling method successfully bridges the speed-accuracy trade-off in masked diffusion language models, enabling parallel token generation while maintaining distributional fidelity to the true joint distribution.

Abstract: In autoregressive language models, each token is sampled by conditioning on all the past tokens; the overall string has thus been sampled from the correct underlying joint distribution represented by the model. In contrast, masked diffusion language models generate text by unmasking tokens out of order and potentially in parallel. Generating an overall string sampled from the correct underlying joint distribution would (again) require exactly one token unmasking in every full-model forward pass. The more tokens unmasked in parallel, the further away the string is from the true joint; this can be seen in the resulting drop in accuracy (but, increase in speed). In this paper we devise a way to {\em approximately} sample multiple tokens from the joint distribution in a single full-model forward pass; we do so by developing a new lightweight single-layer ``sampler" on top of an existing large diffusion LM. One forward pass of the full model can now be followed by multiple forward passes of only this sampler layer, to yield multiple unmasked tokens. Our sampler is trained to mimic exact joint sampling from the (frozen) full model. We show the effectiveness of our approximate joint sampling for both pretrained-only (Dream-7B-Base) and instruction-tuned (Dream-7B-Instruct) models on language modeling and math & coding tasks. When four tokens are unmasked for each full-model denoising step, our sampling algorithm achieves a MAUVE score of 0.87 (vs marginal baseline of 0.31) with respect to the true joint distribution.

Method: PAS is a family of automated activation steering methods that work with any labeled dataset without manual intervention. It constructs fast, lightweight activation vectors that can be cheaply trained, easily stored, and activated on demand.

Result: PAS reliably improves performance on behavior tasks (10.1% on Bias, 5.2% on Morality, 34.8% on Alignment) but not on intelligence-oriented tasks. It also delivers additional gains on top of ICL and SFT.

Conclusion: PAS provides a practical, automated LM post-training option that characterizes where activation steering helps and where it fails, offering a fast, lightweight alternative to traditional methods.

Abstract: Language models (LMs) are typically post-trained for desired capabilities and behaviors via weight-based or prompt-based steering, but the former is time-consuming and expensive, and the latter is not precisely controllable and often requires manual trial-and-error. While activation steering (AS) promises a cheap, fast, and controllable alternative to the two existing post-training methods, current AS techniques require hand-crafted prompt pairs or labor-intensive feature annotation, making them more inconvenient than the plug-and-play methods such as Reinforcement Learning (RL) and Supervised Fine-Tuning (SFT). We introduce Painless Activation Steering (PAS), a family of fully automated methods that make AS readily usable with any given labeled dataset, with no need for prompt construction, feature labeling, or human intervention. We evaluate PAS on three open-weight models (Llama3.1-8B-Instruct, DeepSeek-R1-Distill-8B, and Nous-Hermes-2) and 18 tasks; we find that PAS reliably improves performance for behavior tasks, but not for intelligence-oriented tasks. The introspective variant (iPAS) delivers the strongest causal steering effects (10.1% on Bias, 5.2% on Morality, and 34.8% on Alignment). We also show PAS delivers additional gains on top of In-Context Learning (ICL) and SFT. PAS constructs a fast, lightweight activation vector that can be cheaply trained, easily stored, and activated at will. Our results provide a characterization of where AS helps, where it fails, and how to deploy it as a practical, automated LM post-training option.

Method: Created MIRAGE benchmark with 1,142 ambiguous multi-hop questions categorized by ambiguity types, and proposed CLARION multi-agent framework for handling layered ambiguity.

Result: State-of-the-art models perform poorly on MIRAGE, confirming that ambiguity resolution combined with multi-step reasoning is a significant challenge. CLARION framework outperforms existing approaches.

Conclusion: Multi-hop reasoning with ambiguity requires specialized approaches like CLARION, paving the way for more adaptive and robust reasoning systems that can handle layered ambiguity.

Abstract: Real-world Multi-hop Question Answering (QA) often involves ambiguity that is inseparable from the reasoning process itself. This ambiguity creates a distinct challenge, where multiple reasoning paths emerge from a single question, each requiring independent resolution. Since each sub-question is ambiguous, the model must resolve ambiguity at every step. Thus, answering a single question requires handling multiple layers of ambiguity throughout the reasoning chain. We find that current Large Language Models (LLMs) struggle in this setting, typically exploring wrong reasoning paths and producing incomplete answers. To facilitate research on multi-hop ambiguity, we introduce MultI-hop Reasoning with AmbiGuity Evaluation for Illusory Questions (MIRAGE), a benchmark designed to analyze and evaluate this challenging intersection of ambiguity interpretation and multi-hop reasoning. MIRAGE contains 1,142 high-quality examples of ambiguous multi-hop questions, categorized under a taxonomy of syntactic, general, and semantic ambiguity, and curated through a rigorous multi-LLM verification pipeline. Our experiments reveal that even state-of-the-art models struggle on MIRAGE, confirming that resolving ambiguity combined with multi-step inference is a distinct and significant challenge. To establish a robust baseline, we propose CLarifying Ambiguity with a Reasoning and InstructiON (CLARION), a multi-agent framework that significantly outperforms existing approaches on MIRAGE, paving the way for more adaptive and robust reasoning systems.

Method: Created ML2B benchmark with 30 Kaggle competitions translated into 13 languages, covering tabular, text, and image data types. Used AIDE framework for automated end-to-end assessment of data science pipelines.

Result: Substantial 15-45% performance degradation on non-English tasks compared to English, highlighting challenges in multilingual representation learning for code generation.

Conclusion: The benchmark and evaluation framework are made publicly available to facilitate future research in multilingual ML code generation, addressing critical gaps in current evaluation methodologies.

Abstract: Large language models (LLMs) have recently demonstrated strong capabilities in generating machine learning (ML) code, enabling end-to-end pipeline construction from natural language instructions. However, existing benchmarks for ML code generation are mainly restricted to English, overlooking the global and multilingual nature of ML research and practice. To address this gap, we present ML2B, the first benchmark for evaluating multilingual ML code generation. ML2B consists of 30 Kaggle competitions translated into 13 natural languages, covering tabular, text, and image data types, with structured metadata and validated human-reviewed translations. For evaluation, we employ AIDE, an automated framework for end-to-end assessment of data science pipelines, and provide insights into cross-lingual model performance. Our results reveal substantial 15-45% performance degradation on non-English tasks, highlighting critical challenges in multilingual representation learning for code generation. The benchmark, evaluation framework, and comprehensive results are made available through our GitHub repository to facilitate future research in multilingual ML code generation: https://github.com/enaix/ml2b.

Method: Created a multi-dialect Arabic spoofed speech dataset and evaluated TTS models using an evaluation pipeline that included: embedding-based classifiers with classifier heads, classical ML algorithms on MFCC features, RawNet2 architecture, Mean Opinion Score from human ratings, and Word Error Rate from ASR processing.

Result: FishSpeech outperformed other TTS models in Arabic voice cloning on the Casablanca corpus, producing more realistic and challenging synthetic speech samples.

Conclusion: While FishSpeech produces the most challenging synthetic samples, relying on a single TTS model for dataset creation may limit generalizability.

Abstract: With the rise of generative text-to-speech models, distinguishing between real and synthetic speech has become challenging, especially for Arabic that have received limited research attention. Most spoof detection efforts have focused on English, leaving a significant gap for Arabic and its many dialects. In this work, we introduce the first multi-dialect Arabic spoofed speech dataset. To evaluate the difficulty of the synthesized audio from each model and determine which produces the most challenging samples, we aimed to guide the construction of our final dataset either by merging audios from multiple models or by selecting the best-performing model, we conducted an evaluation pipeline that included training classifiers using two approaches: modern embedding-based methods combined with classifier heads; classical machine learning algorithms applied to MFCC features; and the RawNet2 architecture. The pipeline further incorporated the calculation of Mean Opinion Score based on human ratings, as well as processing both original and synthesized datasets through an Automatic Speech Recognition model to measure the Word Error Rate. Our results demonstrate that FishSpeech outperforms other TTS models in Arabic voice cloning on the Casablanca corpus, producing more realistic and challenging synthetic speech samples. However, relying on a single TTS for dataset creation may limit generalizability.

Method: EditGRPO integrates on-policy exploration with off-policy guidance by injecting sentence-level corrections during training rollouts, using a mixed-policy RL approach.

Result: Applied to Qwen2.5-VL-3B MLLM, EditGRPO achieved 3.4% average improvement in CheXbert, GREEN, Radgraph, and RATEScore metrics across four chest X-ray datasets, with 5.9% gain on unseen datasets.

Conclusion: EditGRPO effectively addresses RL exploration challenges and improves clinical report generation performance and generalization.

Abstract: Radiology report generation requires advanced medical image analysis, effective temporal reasoning, and accurate text generation. Although recent innovations, particularly multimodal large language models (MLLMs), have shown improved performance, their supervised fine-tuning (SFT) objective is not explicitly aligned with clinical efficacy. In this work, we introduce EditGRPO, a mixed-policy reinforcement learning (RL) algorithm designed specifically to optimize the generation through clinically motivated rewards. EditGRPO integrates on-policy exploration with off-policy guidance by injecting sentence-level detailed corrections during training rollouts. This mixed-policy approach addresses the exploration dilemma and sampling efficiency issues typically encountered in RL. Applied to a Qwen2.5-VL-3B MLLM initialized with supervised fine-tuning (SFT), EditGRPO outperforms both SFT and vanilla GRPO baselines, achieving an average improvement of 3.4% in CheXbert, GREEN, Radgraph, and RATEScore metrics across four major chest X-ray report generation datasets. Notably, EditGRPO also demonstrates superior out-of-domain generalization, with an average performance gain of 5.9% on unseen datasets.

Method: Propose CRL where models generate critiques for (question, solution) pairs, with rewards based on alignment between generated and ground-truth judgment labels. Implement Critique-Coder by replacing 20% of standard RL data with CRL data.

Result: Critique-Coder consistently outperforms RL-only baselines across benchmarks, achieving over 60% on LiveCodeBench (v5) and better performance on logic reasoning tasks from BBEH dataset.

Conclusion: CRL serves as an effective complement to standard RL for LLM reasoning, enhancing both coding performance and transferable general reasoning abilities.

Abstract: Reinforcement Learning (RL) has emerged as a popular training paradigm, particularly when paired with reasoning models. While effective, it primarily focuses on generating responses and lacks mechanisms to explicitly foster critique or reflection. Several recent studies, like Critique-Fine-Tuning (CFT) and Critique-Guided-Distillation (CGD) have shown the benefits of explicitly teaching LLMs how to critique. Motivated by them, we propose Critique Reinforcement Learning (CRL), where the model is tasked with generating a critique for a given (question, solution) pair. The reward is determined solely by whether the final judgment label $c \in {\texttt{True}, \texttt{False}}$ of the generated critique aligns with the ground-truth judgment $c^*$. Building on this point, we introduce \textsc{Critique-Coder}, which is trained on a hybrid of RL and CRL by substituting 20% of the standard RL data with CRL data. We fine-tune multiple models (\textsc{Critique-Coder}) and evaluate them on different benchmarks to show their advantages over RL-only models. We show that \textsc{Critique-Coder} consistently outperforms RL-only baselines on all the evaluated benchmarks. Notably, our \textsc{Critique-Coder-8B} can reach over 60% on LiveCodeBench (v5), outperforming other reasoning models like DeepCoder-14B and GPT-o1. Beyond code generation, \textsc{Critique-Coder} also demonstrates enhanced general reasoning abilities, as evidenced by its better performance on logic reasoning tasks from the BBEH dataset. This indicates that the application of CRL on coding datasets enhances general reasoning and critique abilities, which are transferable across a broad range of tasks. Hence, we believe that CRL works as a great complement to standard RL for LLM reasoning.

Method: Developed ChatInject attack that formats malicious payloads to mimic native chat templates, and a persuasion-driven Multi-turn variant that primes agents across conversational turns to accept suspicious actions.

Result: ChatInject achieved significantly higher attack success rates (32.05% on AgentDojo, 45.90% on InjecAgent) compared to traditional methods (5.18%, 15.13%), with multi-turn variants reaching 52.33% success rate. The attack shows strong transferability across models and bypasses existing defenses.

Conclusion: Current agent systems have critical vulnerabilities to chat-template-based attacks, and existing prompt-based defenses are largely ineffective, especially against multi-turn variants.

Abstract: The growing deployment of large language model (LLM) based agents that interact with external environments has created new attack surfaces for adversarial manipulation. One major threat is indirect prompt injection, where attackers embed malicious instructions in external environment output, causing agents to interpret and execute them as if they were legitimate prompts. While previous research has focused primarily on plain-text injection attacks, we find a significant yet underexplored vulnerability: LLMs’ dependence on structured chat templates and their susceptibility to contextual manipulation through persuasive multi-turn dialogues. To this end, we introduce ChatInject, an attack that formats malicious payloads to mimic native chat templates, thereby exploiting the model’s inherent instruction-following tendencies. Building on this foundation, we develop a persuasion-driven Multi-turn variant that primes the agent across conversational turns to accept and execute otherwise suspicious actions. Through comprehensive experiments across frontier LLMs, we demonstrate three critical findings: (1) ChatInject achieves significantly higher average attack success rates than traditional prompt injection methods, improving from 5.18% to 32.05% on AgentDojo and from 15.13% to 45.90% on InjecAgent, with multi-turn dialogues showing particularly strong performance at average 52.33% success rate on InjecAgent, (2) chat-template-based payloads demonstrate strong transferability across models and remain effective even against closed-source LLMs, despite their unknown template structures, and (3) existing prompt-based defenses are largely ineffective against this attack approach, especially against Multi-turn variants. These findings highlight vulnerabilities in current agent systems.

Method: RSM-DCK uses recent context as a query to pre-select relevant context and knowledge at word-level and utterance-level semantics, then interacts the response candidate with selected content, and finally uses fused context-response representation to post-select knowledge for confident matching.

Result: The model achieves better performance than existing methods on two benchmark datasets and effectively detects relevant context and knowledge for response selection.

Conclusion: The proposed RSM-DCK model successfully addresses the problem of irrelevant information in context and knowledge by detecting relevant parts, leading to improved response selection performance in knowledge-grounded conversations.

Abstract: Recently, knowledge-grounded conversations in the open domain gain great attention from researchers. Existing works on retrieval-based dialogue systems have paid tremendous efforts to utilize neural networks to build a matching model, where all of the context and knowledge contents are used to match the response candidate with various representation methods. Actually, different parts of the context and knowledge are differentially important for recognizing the proper response candidate, as many utterances are useless due to the topic shift. Those excessive useless information in the context and knowledge can influence the matching process and leads to inferior performance. To address this problem, we propose a multi-turn \textbf{R}esponse \textbf{S}election \textbf{M}odel that can \textbf{D}etect the relevant parts of the \textbf{C}ontext and \textbf{K}nowledge collection (\textbf{RSM-DCK}). Our model first uses the recent context as a query to pre-select relevant parts of the context and knowledge collection at the word-level and utterance-level semantics. Further, the response candidate interacts with the selected context and knowledge collection respectively. In the end, The fused representation of the context and response candidate is utilized to post-select the relevant parts of the knowledge collection more confidently for matching. We test our proposed model on two benchmark datasets. Evaluation results indicate that our model achieves better performance than the existing methods, and can effectively detect the relevant context and knowledge for response selection.

Method: Extracts reusable structural directions that emerge during ICL and employs a learnable input-conditioned router to modulate attention logits accordingly.

Result: Outperforms prior implicit ICL methods on 12 real-world datasets across diverse domains and multiple LLMs, with robust generalization to out-of-domain tasks.

Conclusion: ICR pushes the boundary of ICL’s practical value by enabling generalizable implicit ICL without task-specific requirements.

Abstract: Implicit in-context learning (ICL) has newly emerged as a promising paradigm that simulates ICL behaviors in the representation space of Large Language Models (LLMs), aiming to attain few-shot performance at zero-shot cost. However, existing approaches largely rely on injecting shift vectors into residual flows, which are typically constructed from labeled demonstrations or task-specific alignment. Such designs fall short of utilizing the structural mechanisms underlying ICL and suffer from limited generalizability. To address this, we propose In-Context Routing (ICR), a novel implicit ICL method that internalizes generalizable ICL patterns at the attention logits level. It extracts reusable structural directions that emerge during ICL and employs a learnable input-conditioned router to modulate attention logits accordingly, enabling a train-once-and-reuse framework. We evaluate ICR on 12 real-world datasets spanning diverse domains and multiple LLMs. The results show that ICR consistently outperforms prior implicit ICL methods that require task-specific retrieval or training, while demonstrating robust generalization to out-of-domain tasks where existing methods struggle. These findings position ICR to push the boundary of ICL’s practical value.

Method: Introduced novel evaluation framework using multiple-choice tasks, curated 220 decision scenarios with psychologists, and generated diverse prompts from human-authored templates to analyze over 2.8 million LLM responses.

Result: LLMs showed bias-consistent behavior across anchoring, availability, confirmation, framing, interpretation, overattribution, prospect theory, and representativeness biases. Larger models (>32B parameters) reduced bias in 39.5% of cases, while detailed prompts reduced most biases by up to 14.9% (except overattribution, which increased by 8.8%).

Conclusion: LLMs systematically exhibit cognitive biases, with model size and prompt specificity being key factors in bias susceptibility, highlighting the need for careful consideration when deploying LLMs in decision-making contexts.

Abstract: As Large Language Models (LLMs) are increasingly embedded in real-world decision-making processes, it becomes crucial to examine the extent to which they exhibit cognitive biases. Extensively studied in the field of psychology, cognitive biases appear as systematic distortions commonly observed in human judgments. This paper presents a large-scale evaluation of eight well-established cognitive biases across 45 LLMs, analyzing over 2.8 million LLM responses generated through controlled prompt variations. To achieve this, we introduce a novel evaluation framework based on multiple-choice tasks, hand-curate a dataset of 220 decision scenarios targeting fundamental cognitive biases in collaboration with psychologists, and propose a scalable approach for generating diverse prompts from human-authored scenario templates. Our analysis shows that LLMs exhibit bias-consistent behavior in 17.8-57.3% of instances across a range of judgment and decision-making contexts targeting anchoring, availability, confirmation, framing, interpretation, overattribution, prospect theory, and representativeness biases. We find that both model size and prompt specificity play a significant role on bias susceptibility as follows: larger size (>32B parameters) can reduce bias in 39.5% of cases, while higher prompt detail reduces most biases by up to 14.9%, except in one case (Overattribution), which is exacerbated by up to 8.8%.

Method: Model documents as sentence-level graphs with nodes for sentences/lemmas and edges for linguistic relationships. Use SAMER lexicon features and Arabic transformer embeddings. Train GNN and transformer branches independently, combine via late fusion, and aggregate sentence-level predictions with max pooling.

Result: The hybrid fusion method outperforms standalone GNN or transformer branches across multiple readability metrics at document level, while GNN-only approach works better for sentence-level prediction.

Conclusion: Fusion offers advantages for document-level readability prediction, but GNN-only approach remains stronger for precise sentence-level readability assessment.

Abstract: We present a graph-based approach enriched with lexicons to predict document-level readability in Arabic, developed as part of the Constrained Track of the BAREC Shared Task 2025. Our system models each document as a sentence-level graph, where nodes represent sentences and lemmas, and edges capture linguistic relationships such as lexical co-occurrence and class membership. Sentence nodes are enriched with features from the SAMER lexicon as well as contextual embeddings from the Arabic transformer model. The graph neural network (GNN) and transformer sentence encoder are trained as two independent branches, and their predictions are combined via late fusion at inference. For document-level prediction, sentence-level outputs are aggregated using max pooling to reflect the most difficult sentence. Experimental results show that this hybrid method outperforms standalone GNN or transformer branches across multiple readability metrics. Overall, the findings highlight that fusion offers advantages at the document level, but the GNN-only approach remains stronger for precise prediction of sentence-level readability.

Method: HEART provides feedback on incorrect responses using curated emotionally charged phrases based on Ekman’s six universal emotions, systematically varying emotional tone across iterations to guide models away from flawed reasoning paths.

Result: When guided by an oracle verifier, HEART unlocks significantly deeper reasoning with consistent substantial accuracy increases over state-of-the-art baselines. However, in verifier-free settings it struggles to consistently harness these gains.

Conclusion: The next frontier in machine reasoning may lie not just in refining logic, but also in understanding and leveraging the ‘HEART’ of models through affective iteration protocols.

Abstract: Test-time scaling has shown considerable success in improving the performance of language models on complex reasoning tasks without requiring fine-tuning. However, current strategies such as self-reflection primarily focus on logical or structural refinement. They do not leverage the guiding potential of affective feedback. Inspired by psychological research showing that emotions can modulate cognitive performance, we introduce HEART–a novel framework that uses emotionally-driven prompts for iterative self-correction. HEART provides feedback on a model’s incorrect response using a curated set of concise, emotionally charged phrases based on the six universal emotions categorized by Dr. Paul Ekman. By systematically varying the emotional tone of the feedback across iterations, our method guides the model to escape flawed reasoning paths and explore more promising alternatives. We evaluate our framework on challenging reasoning benchmarks including OlympiadBench, Humanity’s Last Exam, and SimpleQA. Our results reveal a significant new phenomenon: when guided by an oracle verifier, this affective iteration protocol unlocks significantly deeper reasoning, leading to consistent and substantial increases in accuracy over state-of-the-art baselines with the same verifier. However, we also identify a critical bottleneck for practical deployment. In a verifier-free setting, it struggles to harness these gains consistently, highlighting as a key challenge for future work. Our findings suggest that the next frontier in machine reasoning may lie not just in refining logic, but also in understanding and leveraging the `HEART’ of the models.

Method: Introduces ToMAgent (ToMA), which pairs Theory of Mind with dialogue lookahead to produce mental states that are maximally useful for achieving dialogue goals. The method involves prompting models to generate mental states between dialogue turns.

Result: Experiments on Sotopia benchmark show ToMA outperforms baselines, exhibiting more strategic, goal-oriented reasoning behaviors, enabling long-horizon adaptation while maintaining better partner relationships.

Conclusion: Explicit integration of Theory of Mind represents a step forward in building socially intelligent LLM agents, improving dialogue performance and goal achievement.

Abstract: Theory of Mind (ToM)-an understanding of the mental states of others-is a key aspect of human social intelligence, yet, chatbots and LLM-based social agents do not typically integrate it. In this work, we demonstrate that LLMs that explicitly use ToM get better at dialogue, achieving goals more effectively. After showing that simply prompting models to generate mental states between dialogue turns already provides significant benefit, we further introduce ToMAgent (ToMA), a ToM-focused dialogue agent. ToMA is trained by pairing ToM with dialogue lookahead to produce mental states that are maximally useful for achieving dialogue goals. Experiments on the Sotopia interactive social evaluation benchmark demonstrate the effectiveness of our method over a range of baselines. Comprehensive analysis shows that ToMA exhibits more strategic, goal-oriented reasoning behaviors, which enable long-horizon adaptation, while maintaining better relationships with their partners. Our results suggest a step forward in integrating ToM for building socially intelligent LLM agents.

Method: Combines synthetic data generation (280,128 examples), supervised fine-tuning with LoRA (modifying only 0.53% of weights), and reinforcement learning with GRPO using a novel semantic similarity-based reward function.

Result: Achieves mean reward of 0.573 on 1,000 document extraction tasks, outperforming GPT-4.1 (0.457), o3 (0.464), and GPT-4.1-2025 (0.459).

Conclusion: Task-specific optimization can produce models that surpass general-purpose systems while requiring substantially fewer computational resources.

Abstract: This paper presents Extract-0, a 7-billion parameter language model specifically optimized for document information extraction that achieves performance exceeding models with parameter counts several orders of magnitude larger. Through a novel combination of synthetic data generation, supervised fine-tuning with Low-Rank Adaptation (LoRA), and reinforcement learning via Group Relative Policy Optimization (GRPO), Extract-0 achieves a mean reward of 0.573 on a benchmark of 1,000 diverse document extraction tasks, outperforming GPT-4.1 (0.457), o3 (0.464), and GPT-4.1-2025 (0.459). The training methodology employs a memory-preserving synthetic data generation pipeline that produces 280,128 training examples from diverse document sources, followed by parameterefficient fine-tuning that modifies only 0.53% of model weights (40.4M out of 7.66B parameters). The reinforcement learning phase introduces a novel semantic similarity-based reward function that handles the inherent ambiguity in information extraction tasks. This research demonstrates that task-specific optimization can yield models that surpass general-purpose systems while requiring substantially fewer computational resource.

Method: Created HiKE benchmark with high-quality natural CS data across various topics, meticulous loanword labels, and hierarchical CS-level labeling scheme (word, phrase, sentence) for systematic evaluation.

Result: Most multilingual ASR models initially struggle with CS-ASR, but this capability can be enabled through fine-tuning with CS data.

Conclusion: HiKE provides the first comprehensive evaluation framework for Korean-English code-switching that enables systematic assessment and improvement of multilingual ASR models’ CS handling capabilities.

Abstract: Despite advances in multilingual automatic speech recognition (ASR), code-switching (CS), the mixing of languages within an utterance common in daily speech, remains a severely underexplored challenge. In this paper, we introduce HiKE: the Hierarchical Korean-English code-switching benchmark, the first globally accessible evaluation framework for Korean-English CS, aiming to provide a means for the precise evaluation of multilingual ASR models and to foster research in the field. The proposed framework not only consists of high-quality, natural CS data across various topics, but also provides meticulous loanword labels and a hierarchical CS-level labeling scheme (word, phrase, and sentence) that together enable a systematic evaluation of a model’s ability to handle each distinct level of code-switching. Through evaluations of diverse multilingual ASR models and fine-tuning experiments, this paper demonstrates that while most multilingual ASR models initially struggle with CS-ASR, this capability can be enabled through fine-tuning with CS data. HiKE will be available at https://github.com/ThetaOne-AI/HiKE.

Method: Three experiments using GPT-4o: drug-formulation matching, drug-drug interaction identification (with and without web search), and medication order sentence preparation. Evaluation used cosine similarity, Levenshtein similarity, ROUGE scores, and clinician manual assessment.

Result: Poor performance across all tests: 49% correct drug-formulation matching with frequent omissions and hallucinations; inconsistent drug-drug interaction identification (54.7% vs 69.2% accuracy with/without search); only 65.8% of medication orders contained no errors.

Conclusion: GPT-4o’s overall poor performance on medication-related tasks indicates the need for domain-specific training using clinician-annotated datasets and comprehensive evaluation frameworks for benchmarking LLM performance in medical applications.

Abstract: Background: Large language models (LLMs) can be useful in diagnosing medical conditions, but few studies have evaluated their consistency in recommending appropriate medication regimens. The purpose of this evaluation was to test GPT-4o on three medication benchmarking tests including mapping a drug name to its correct formulation, identifying drug-drug interactions using both its internal knowledge and using a web search, and preparing a medication order sentence after being given the medication name. Methods: Using GTP-4o three experiments were completed. Accuracy was quantified by computing cosine similarity on TF-IDF vectors, normalized Levenshtein similarity, and ROUGE-1/ROUGE-L F1 between each response and its reference string or by manual evaluation by clinicians. Results: GPT-4o performed poorly on drug-formulation matching, with frequent omissions of available drug formulations (mean 1.23 per medication) and hallucinations of formulations that do not exist (mean 1.14 per medication). Only 49% of tested medications were correctly matched to all available formulations. Accuracy was decreased for medications with more formulations (p<0.0001). GPT-4o was also inconsistent at identifying drug-drug-interactions, although it had better performance with the search-augmented assessment compared to its internal knowledge (54.7% vs. 69.2%, p=0.013). However, allowing a web-search worsened performance when there was no drug-drug interaction (median % correct 100% vs. 40%, p<0.001). Finally, GPT-4o performed moderately with preparing a medication order sentence, with only 65.8% of medication order sentences containing no medication or abbreviation errors. Conclusions: Model performance was overall poor for all tests. This highlights the need for domain-specific training through clinician-annotated datasets and a comprehensive evaluation framework for benchmarking performance.

Method: A pipeline using LLMs as interface for prompting text-to-image models to generate scene illustrations from raw story text, applied to a story corpus with human annotation for quality judgments.

Result: Created SceneIllustrations dataset with pairwise quality judgments, demonstrating LLMs can effectively verbalize scene knowledge from story text for illustration generation and evaluation.

Conclusion: LLMs are impactful for generating and evaluating illustrations by effectively extracting and verbalizing implicit scene knowledge from narrative text.

Abstract: Generative AI has established the opportunity to readily transform content from one medium to another. This capability is especially powerful for storytelling, where visual illustrations can illuminate a story originally expressed in text. In this paper, we focus on the task of narrative scene illustration, which involves automatically generating an image depicting a scene in a story. Motivated by recent progress on text-to-image models, we consider a pipeline that uses LLMs as an interface for prompting text-to-image models to generate scene illustrations given raw story text. We apply variations of this pipeline to a prominent story corpus in order to synthesize illustrations for scenes in these stories. We conduct a human annotation task to obtain pairwise quality judgments for these illustrations. The outcome of this process is the SceneIllustrations dataset, which we release as a new resource for future work on cross-modal narrative transformation. Through our analysis of this dataset and experiments modeling illustration quality, we demonstrate that LLMs can effectively verbalize scene knowledge implicitly evoked by story text. Moreover, this capability is impactful for generating and evaluating illustrations.

Method: Introduced Bi-Induct curriculum that injects forward-copy (Induction), backward-copy (Anti), or balanced mix into pretraining stream. Trained models from 0.13B to 1B parameters under iso-FLOPs, evaluating few-shot ICL benchmarks, head-level telemetry, and language modeling perplexity.

Result: Bi-Induct accelerates induction-head emergence at small scales but doesn’t yield stronger generalization. Natural-only training performs best on function-style ICL probes. Larger natural-only models develop broader, earlier induction heads without explicit patterns. Perplexity penalties from synthetic data shrink with scale.

Conclusion: Inducing activation is not sufficient for ICL gains - circuits must become functionally necessary. Results emphasize mechanism-aware pretraining diagnostics and data mixtures that foster load-bearing, not merely present, structure.

Abstract: Does explicitly exercising the induction circuit during pretraining improve in-context learning (ICL), or is natural text sufficient when compute is held constant (iso-FLOPs)? To test whether targeted synthetic data can accelerate induction-head emergence and enhance ICL, we introduce Bi-Induct, a lightweight curriculum that injects forward-copy (Induction), backward-copy (Anti), or a balanced mix into the pretraining stream. We train models from 0.13B to 1B parameters under iso-FLOPs, evaluating (i) few-shot ICL benchmarks, (ii) head-level telemetry, and (iii) held-out language modeling perplexity. Our findings challenge the assumption that early induction circuit activation directly improves ICL. While Bi-Induct accelerates induction-head emergence at small scales, this does not consistently yield stronger generalization. On standard LM benchmarks, Bi-Induct matches natural-only training; on function-style ICL probes, the 1B natural-only performs best. Stress tests (e.g., label permutation, HITS@1 vs. HITS@3, 1 vs. 10 shots) preserve these trends. Telemetry shows larger natural-only models develop broader, earlier induction heads without explicit induction patterns. Anti-induction data fails to elicit meaningful activation. Perplexity penalties from synthetic data shrink with scale, suggesting larger models can absorb non-natural patterns with minimal cost. Crucially, ablating the top 2% of induction heads degrades ICL more than random ablations, especially for natural-only models, indicating more centralized, load-bearing circuits. Bi-Induct variants exhibit more redundant induction activity, implying different circuit utilization. Overall, inducing activation is not sufficient: ICL gains depend on these circuits becoming functionally necessary. These results underscore mechanism-aware pretraining diagnostics and data mixtures that foster load-bearing, not merely present, structure.

Method: Study S3M variants optimized for word recognition by analyzing how they represent frequent English noun and verb inflections, examining the geometric structure of their representations.

Result: The models develop representations with a global linear geometry that can link English nouns and verbs to their regular inflected forms. This structure tracks regular distributional relationships between word pairs in the English lexicon, often (but not always) due to morphological inflection.

Conclusion: These findings challenge the presumed necessity of distinct linguistic representations for phonology and morphology in human spoken word recognition, suggesting alternative representational strategies based on distributional relationships.

Abstract: Self-supervised speech models can be trained to efficiently recognize spoken words in naturalistic, noisy environments. However, we do not understand the types of linguistic representations these models use to accomplish this task. To address this question, we study how S3M variants optimized for word recognition represent phonological and morphological phenomena in frequent English noun and verb inflections. We find that their representations exhibit a global linear geometry which can be used to link English nouns and verbs to their regular inflected forms. This geometric structure does not directly track phonological or morphological units. Instead, it tracks the regular distributional relationships linking many word pairs in the English lexicon – often, but not always, due to morphological inflection. These findings point to candidate representational strategies that may support human spoken word recognition, challenging the presumed necessity of distinct linguistic representations of phonology and morphology.

Method: Created paired structured and semi-structured tables using verbalization pipeline, introduced diagnostic benchmark with splits along table size, join requirements, query complexity, and schema quality.

Result: SQL-based methods achieve high accuracy on structured inputs but degrade on semi-structured data; LLMs show flexibility but reduced precision; hybrid approaches balance performance, especially under noisy schemas.

Conclusion: No single method excels across all conditions; representation plays central role in Table QA performance; findings provide insights for model selection and design of robust hybrid approaches.

Abstract: Table Question Answering (Table QA) in real-world settings must operate over both structured databases and semi-structured tables containing textual fields. However, existing benchmarks are tied to fixed data formats and have not systematically examined how representation itself affects model performance. We present the first controlled study that isolates the role of table representation by holding content constant while varying structure. Using a verbalization pipeline, we generate paired structured and semi-structured tables, enabling direct comparisons across modeling paradigms. To support detailed analysis, we introduce a diagnostic benchmark with splits along table size, join requirements, query complexity, and schema quality. Our experiments reveal consistent trade-offs: SQL-based methods achieve high accuracy on structured inputs but degrade on semi-structured data, LLMs exhibit flexibility but reduced precision, and hybrid approaches strike a balance, particularly under noisy schemas. These effects intensify with larger tables and more complex queries. Ultimately, no single method excels across all conditions, and we highlight the central role of representation in shaping Table QA performance. Our findings provide actionable insights for model selection and design, paving the way for more robust hybrid approaches suited for diverse real-world data formats.

Method: Created AdamDB dataset covering 4M+ individuals across geography/time/profession, AdamBench evaluations based on Bloom’s taxonomy across 6 reasoning levels, and AdamRAG retrieval-augmented generation system for biographical contexts.

Result: AdamRAG substantially improves open-source models and modestly benefits closed-source ones, with largest gains on lower-order reasoning. Popularity strongly mediates accuracy, and multimodal input via face images offers smaller improvements than retrieval.

Conclusion: ADAM establishes the first benchmark and framework for cognitively, culturally, and multimodally grounded biographical evaluation, advancing development of multilingual, accurate, and hallucination-resistant MLLMs.

Abstract: We introduce ADAM (A Diverse Archive of Mankind), a framework for evaluating and improving multimodal large language models (MLLMs) in biographical reasoning. To the best of our knowledge, this is the first work to systematically examine LLM capabilities in biography, a critical yet underexplored dimension of factual knowledge. At its core, AdamDB is a multilingual and multimodal dataset covering over 4 million individuals across geography, time, and profession, while AdamBench provides cognitively structured evaluations based on Bloom’s taxonomy, spanning six reasoning levels in both English and native languages. To address hallucinations, particularly for lesser-known individuals, we propose AdamRAG, a retrieval-augmented generation system tailored to biographical contexts. Experiments show that AdamRAG substantially improves open-source models and modestly benefits closed-source ones, with the largest gains on lower-order reasoning. Popularity strongly mediates accuracy, and multimodal input via face images offers smaller, less consistent improvements than retrieval. ADAM establishes the first benchmark and framework for cognitively, culturally, and multimodally grounded biographical evaluation, advancing the development of multilingual, accurate, and hallucination-resistant MLLMs.

Method: Two distillation approaches: (1) LM-guided distillation for contextual information, and (2) combined LM and SM-guided distillation for multimodal representations. Uses encoder-decoder framework with RVQ, incorporating LM and SM during training.

Result: DM-Codec significantly outperforms SOTA models, reducing WER by 13.46%, WIL by 9.82%, and improving speech quality by 5.84% and intelligibility by 1.85% on LibriSpeech.

Conclusion: The proposed DM-Codec successfully integrates contextual information with acoustic and semantic representations, demonstrating substantial improvements in speech tokenization and synthesis performance.

Abstract: Recent advancements in speech-language models have yielded significant improvements in speech tokenization and synthesis. However, effectively mapping the complex, multidimensional attributes of speech into discrete tokens remains challenging. This process demands acoustic, semantic, and contextual information for precise speech representations. Existing speech representations generally fall into two categories: acoustic tokens from audio codecs and semantic tokens from speech self-supervised learning models. Although recent efforts have unified acoustic and semantic tokens for improved performance, they overlook the crucial role of contextual representation in comprehensive speech modeling. Our empirical investigations reveal that the absence of contextual representations results in elevated Word Error Rate (WER) and Word Information Lost (WIL) scores in speech transcriptions. To address these limitations, we propose two novel distillation approaches: (1) a language model (LM)-guided distillation method that incorporates contextual information, and (2) a combined LM and self-supervised speech model (SM)-guided distillation technique that effectively distills multimodal representations (acoustic, semantic, and contextual) into a comprehensive speech tokenizer, termed DM-Codec. The DM-Codec architecture adopts a streamlined encoder-decoder framework with a Residual Vector Quantizer (RVQ) and incorporates the LM and SM during the training process. Experiments show DM-Codec significantly outperforms state-of-the-art speech tokenization models, reducing WER by up to 13.46%, WIL by 9.82%, and improving speech quality by 5.84% and intelligibility by 1.85% on the LibriSpeech benchmark dataset. Code, samples, and checkpoints are available at https://github.com/mubtasimahasan/DM-Codec.

Method: Generated corpora using multiple LLMs (OpenAI, Anthropic, Alphabet, Meta, DeepSeek) from GPT-3 to GPT-4.5, replicating BE21 and Koditex reference human corpora. The corpora are tagged with Universal Dependencies standard for tokenization, lemmatization, and morphological/syntactical annotation.

Result: Created English corpus with average 864k tokens per model (27M total) and Czech corpus with average 768k tokens per model (21.5M total). The corpora are freely available under CC BY 4.0 license and accessible through Czech National Corpus search interface.

Conclusion: Successfully developed comprehensive LLM-generated corpora for linguistic research, providing valuable resources for comparing human and AI-generated text across multiple languages and genres.

Abstract: This article presents two corpora of English and Czech texts generated with large language models (LLMs). The motivation is to create a resource for comparing human-written texts with LLM-generated text linguistically. Emphasis was placed on ensuring these resources are multi-genre and rich in terms of topics, authors, and text types, while maintaining comparability with existing human-created corpora. These generated corpora replicate reference human corpora: BE21 by Paul Baker, which is a modern version of the original Brown Corpus, and Koditex corpus that also follows the Brown Corpus tradition but in Czech. The new corpora were generated using models from OpenAI, Anthropic, Alphabet, Meta, and DeepSeek, ranging from GPT-3 (davinci-002) to GPT-4.5, and are tagged according to the Universal Dependencies standard (i.e., they are tokenized, lemmatized, and morphologically and syntactically annotated). The subcorpus size varies according to the model used (the English part contains on average 864k tokens per model, 27M tokens altogether, the Czech partcontains on average 768k tokens per model, 21.5M tokens altogether). The corpora are freely available for download under the CC BY 4.0 license (the annotated data are under CC BY-NC-SA 4.0 licence) and are also accessible through the search interface of the Czech National Corpus.

Method: Proposes ReMemR1 with callback-enhanced memory allowing selective retrieval from entire history and non-linear reasoning, plus Reinforcement Learning with Multi-Level Rewards (RLMLR) combining final-answer rewards with dense step-level signals.

Result: Experiments on long-document QA show significant gains over existing memory-based approaches.

Conclusion: ReMemR1 effectively mitigates information degradation, improves supervision, and supports multi-hop memory utilization for long-context reasoning agents.

Abstract: Large language models face challenges in long-context question answering, where key evidence of a query may be dispersed across millions of tokens. Existing works equip large language models with a memory corpus that is dynamically updated during a single-pass document scan, also known as the “memorize while reading” methods. While this approach scales efficiently, it suffers from irreversible forward-only processing, information loss through overwriting, and sparse reinforcement learning signals. To tackle these challenges, we present ReMemR1, a memory-augmented agent with callback-enhanced memory that allows selective retrieval from the entire memory history and allows non-linear reasoning and revisiting of early evidence. To further strengthen training, we propose Reinforcement Learning with Multi-Level Rewards (RLMLR), which combines final-answer rewards with dense, step-level signals that guide effective memory use. Together, these contributions mitigate information degradation, improve supervision, and support multi-hop memory utilizing. Experiments on long-document QA show significant gains over existing memory-based approaches, which validates ReMemR1 as an effective solution for long-context reasoning agents.

Method: Introduced the first operational framework that: (1) formally defines sycophancy specific to MADS settings, (2) develops new metrics to evaluate agent sycophancy level and its impact on information exchange, (3) systematically investigates how varying sycophancy levels across agent roles (debaters and judges) affects outcomes in decentralized and centralized debate frameworks.

Result: Sycophancy is a core failure mode that amplifies disagreement collapse before reaching correct conclusions, yields lower accuracy than single-agent baselines, and arises from distinct debater-driven and judge-driven failure modes.

Conclusion: Proposed actionable design principles for MADS that effectively balance productive disagreement with cooperation in agent interactions, addressing the sycophancy problem in multi-agent debates.

Abstract: Large language models (LLMs) often display sycophancy, a tendency toward excessive agreeability. This behavior poses significant challenges for multi-agent debating systems (MADS) that rely on productive disagreement to refine arguments and foster innovative thinking. LLMs’ inherent sycophancy can collapse debates into premature consensus, potentially undermining the benefits of multi-agent debate. While prior studies focus on user–LLM sycophancy, the impact of inter-agent sycophancy in debate remains poorly understood. To address this gap, we introduce the first operational framework that (1) proposes a formal definition of sycophancy specific to MADS settings, (2) develops new metrics to evaluate the agent sycophancy level and its impact on information exchange in MADS, and (3) systematically investigates how varying levels of sycophancy across agent roles (debaters and judges) affects outcomes in both decentralized and centralized debate frameworks. Our findings reveal that sycophancy is a core failure mode that amplifies disagreement collapse before reaching a correct conclusion in multi-agent debates, yields lower accuracy than single-agent baselines, and arises from distinct debater-driven and judge-driven failure modes. Building on these findings, we propose actionable design principles for MADS, effectively balancing productive disagreement with cooperation in agent interactions.

Method: Proposed unified framework with reasoning-augmented data supervision, dual-encoder architecture, and task-alternating training for multitask AudioLLMs.

Result: Experiments on IEMOCAP and MELD show improved emotion prediction accuracy and more coherent, evidence-grounded responses. Out-of-domain tests demonstrate good generalization.

Conclusion: Emotion reasoning enhances AudioLLMs’ emotion understanding by providing semantically aligned explanations, improving both accuracy and interpretability.

Abstract: Audio Large Language Models (AudioLLMs) have achieved strong results in semantic tasks like speech recognition and translation, but remain limited in modeling paralinguistic cues such as emotion. Existing approaches often treat emotion understanding as a classification problem, offering little insight into the underlying rationale behind predictions. In this work, we explore emotion reasoning, a strategy that leverages the generative capabilities of AudioLLMs to enhance emotion recognition by producing semantically aligned, evidence-grounded explanations. To support this in multitask AudioLLMs, we introduce a unified framework combining reasoning-augmented data supervision, dual-encoder architecture, and task-alternating training. This approach enables AudioLLMs to effectively learn different tasks while incorporating emotional reasoning. Experiments on IEMOCAP and MELD show that our approach not only improves emotion prediction accuracy but also enhances the coherence and evidential grounding of the generated responses. Experiments on two out-of-domain datasets demonstrate the generalization capabilities of the resulting model.

Method: Semantic voting mechanism that replaces hard matching (exact matching) with soft matching using semantic similarity measured by lightweight sentence embedding models, avoiding reliance on LLM self-evaluation.

Result: Achieves substantial gains in computational efficiency and overall better performance than self-evaluation methods across diverse model architectures and tasks.

Conclusion: Semantic voting provides an effective self-evaluation-free approach for unverifiable tasks that is both computationally efficient and avoids the biases of LLM-based self-evaluation.

Abstract: The rising cost of acquiring supervised data has driven significant interest in self-improvement for large language models (LLMs). Straightforward unsupervised signals like majority voting have proven effective in generating pseudo-labels for verifiable tasks, while their applicability to unverifiable tasks (e.g., translation) is limited by the open-ended character of responses. As a result, self-evaluation mechanisms (e.g., self-judging and entropy minimization) are predominantly used to derive pseudo-labels. However, self-evaluation relying on LLMs typically incurs high computational overhead and introduces overconfidence issues due to intrinsic biases. To address these challenges, we propose a novel self-evaluation-free approach for unverifiable tasks, designed for lightweight yet effective self-improvement. Inspired by majority voting commonly employed in verifiable tasks, we propose semantic voting as a novel mechanism that relaxes the principle of hard matching (i.e., exact matching) toward soft matching (i.e., semantic similarity). Soft matching is achieved by leveraging a lightweight sentence embedding model to quantify semantic similarity, thereby mitigating excessive computational burden and intrinsic bias-associated limitations of self-evaluation. Comprehensive experiments demonstrate that our method achieves substantial gains in computational efficiency and overall better performance than self-evaluation methods across diverse model architectures and tasks.

Method: EviPath has three components: (1) Abductive Subtask Planning for problem decomposition and optimal path planning, (2) Faithful Sub-question Answering using evidence to generate reasoning, and (3) Conversational Fine-Tuning to format interactions for supervised fine-tuning.

Result: An 8B parameter model trained with EviPath-synthesized data significantly outperforms state-of-the-art baselines with a 14.7% absolute EM gain in open-domain question answering.

Conclusion: EviPath enables LLMs to learn complex reasoning and tool-use capabilities directly from synthesized data, providing an effective solution for RAG agent development.

Abstract: Retrieval-augmented generation agents development is hindered by the lack of process-level supervision to effectively guide agentic capabilities like task decomposition, retriever invocation, and stepwise decision-making. While reinforcement learning offers a potential solution, it suffers from sparse rewards and the limited reasoning capabilities of large language models (LLMs). Meanwhile, existing data synthesis methods only produce chain-of-thought rationales and fail to model environmental interactions. In this paper, we propose EviPath, an evidence-anchored reasoning path synthesis paradigm for RAG agent development. EviPath comprises: (i) Abductive Subtask Planning, which decomposes the problem into sub-questions and iteratively plans an optimal solution path based on the dependencies between them; (ii) Faithful Sub-question Answering, which uses supporting evidence to construct a proxy environment to generate reasoning thoughts and answers for each sub-question; and (iii) Conversational Fine-Tuning, which formats the complete agent-environment interaction trajectory into a dialogue format suitable for Supervised Fine-Tuning. EviPath allows LLMs to learn complex reasoning and tool-use capabilities directly from synthesized data. Extensive experiments on widely-used question-answering benchmarks show that an 8B parameter model trained with EviPath-synthesized data significantly and consistently outperforms state-of-the-art baselines with a double-digit absolute EM gain of 14.7% in open-domain question answering.

Method: Used credal sets (convex hulls of probability distributions) to analyze 500 creative writing prompts from WritingPrompts dataset with 10 human continuations each, evaluating 4 language models across 5 decoding strategies (100,000 stories total).

Result: Substantial gaps in capturing human creative variation, with best model-human calibration reaching only 0.434. Decoding strategy contributes 39.4% to 72.0% of total epistemic uncertainty. Model scale shows weak correlation with calibration quality.

Conclusion: The geometric framework provides actionable insights for improving generation systems for human-AI creative alignment.

Abstract: Understanding uncertainty in large language models remains a fundamental challenge, particularly in creative tasks where multiple valid outputs exist. We present a geometric framework using credal sets - convex hulls of probability distributions - to quantify and decompose uncertainty in neural text generation, calibrated against human creative variation. Analyzing 500 creative writing prompts from the WritingPrompts dataset with 10 unique human continuations each, we evaluate four language models across five decoding strategies, generating 100,000 stories. Our credal set analysis reveals substantial gaps in capturing human creative variation, with the best model-human calibration reaching only 0.434 (Gemma-2B with temperature 0.7). We decompose total uncertainty into epistemic and aleatoric components, finding that the choice of decoding strategy contributes 39.4% to 72.0% of total epistemic uncertainty. Model scale shows weak correlation with calibration quality and no significant difference exists between base and instruction-tuned models in calibration quality. Our geometric framework provides actionable insights for improving generation systems for human-AI creative alignment. We release our complete experimental framework.

Method: Two-stage fine-grained selection strategy to identify and update KV states of selected tokens while caching others for reuse, enabling quasi left-to-right generation.

Result: Substantial inference speedups and consistent improvements in generation quality on LLaDA and Dream models.

Conclusion: d²Cache effectively addresses dLLM inference inefficiency without requiring training, providing both performance acceleration and enhanced decoding reliability.

Abstract: Diffusion-based large language models (dLLMs), despite their promising performance, still suffer from inferior inference efficiency. This is because dLLMs rely on bidirectional attention and cannot directly benefit from the standard key-value (KV) cache as autoregressive models (ARMs) do. To tackle this issue, we introduce \textit{Dual aDaptive Cache} (d$^2$Cache), which is a training-free approximate KV cache framework for accelerating dLLM inference. d$^2$Cache features a two-stage fine-grained selection strategy to identify tokens and adaptively update their KV states at each decoding step, while caching the KV states of the remaining tokens for reuse. Furthermore, d$^2$Cache naturally offers a more reliable decoding alternative, which can enable quasi left-to-right generation and mitigate premature overconfidence in tokens at the end of the sequence. Extensive experimental results on two representative dLLMs (\ie, LLaDA and Dream) demonstrate that d$^2$Cache not only achieves substantial inference speedups, but also yields consistent improvements in generation quality. The code is available at https://github.com/Kamichanw/d2Cache.

Method: The authors explore LLMs’ ability with SELFIES representation, examine their capacity to correct invalid SMILES, and introduce SmiSelf - a framework that converts invalid SMILES to SELFIES using grammatical rules to leverage SELFIES’ mechanisms for correction.

Result: SmiSelf ensures 100% validity while preserving molecular characteristics and maintaining or even enhancing performance on other metrics. LLMs perform worse with SELFIES than with SMILES, and their capacity to correct invalid SMILES is limited.

Conclusion: SmiSelf helps expand LLMs’ practical applications in biomedicine and is compatible with all SMILES-based generative models, providing a solution for valid molecule generation in few-shot settings.

Abstract: Molecule generation is key to drug discovery and materials science, enabling the design of novel compounds with specific properties. Large language models (LLMs) can learn to perform a wide range of tasks from just a few examples. However, generating valid molecules using representations like SMILES is challenging for LLMs in few-shot settings. In this work, we explore how LLMs can generate 100% valid molecules. We evaluate whether LLMs can use SELFIES, a representation where every string corresponds to a valid molecule, for valid molecule generation but find that LLMs perform worse with SELFIES than with SMILES. We then examine LLMs’ ability to correct invalid SMILES and find their capacity limited. Finally, we introduce SmiSelf, a cross-chemical language framework for invalid SMILES correction. SmiSelf converts invalid SMILES to SELFIES using grammatical rules, leveraging SELFIES’ mechanisms to correct the invalid SMILES. Experiments show that SmiSelf ensures 100% validity while preserving molecular characteristics and maintaining or even enhancing performance on other metrics. SmiSelf helps expand LLMs’ practical applications in biomedicine and is compatible with all SMILES-based generative models. Code is available at https://github.com/wentao228/SmiSelf.

Method: Proposed a novel user simulator architecture that simulates four categories of non-collaborative behaviors: requesting unavailable services, digressing into tangential conversations, expressing impatience, and providing incomplete utterances.

Result: Experiments on MultiWOZ and τ-bench showed significant performance degradation in state-of-the-art tool agents when encountering non-collaborative users, with issues like escalated hallucinations and dialogue breakdowns.

Conclusion: Provides an easily extensible user simulation framework to help develop tool agents and preemptively diagnose them under challenging real-world conditions within their own services.

Abstract: Non-Collaborative User Simulators for Tool Agents Download PDF Jeonghoon Shim, Woojung Song, Cheyon Jin, Seungwon KooK, Yohan Jo 19 Sept 2025 (modified: 25 Sept 2025)ICLR 2026 Conference SubmissionConference, AuthorsRevisionsCC BY 4.0 Keywords: Tool Agent, User Simulator, Non-collaborative User, Dialogue Simulation TL;DR: A non-collaborative user simulation method for tool agent. Abstract: Tool agents interact with users through multi-turn dialogues to accomplish various tasks. Recent studies have adopted user simulation methods to develop these agents in multi-turn settings. However, existing user simulators tend to be agent-friendly, exhibiting only cooperative behaviors, which fails to train and test agents against non-collaborative users in the real world. To address this, we propose a novel user simulator architecture that simulates four categories of non-collaborative behaviors: requesting unavailable services, digressing into tangential conversations, expressing impatience, and providing incomplete utterances. Our user simulator can simulate challenging and natural non-collaborative behaviors while reliably delivering all intents and information necessary to accomplish the task. Our experiments on MultiWOZ and $\tau$-bench reveal significant performance degradation in state-of-the-art tool agents when encountering non-collaborative users. We provide detailed analyses of agents’ weaknesses under each non-collaborative condition, such as escalated hallucinations and dialogue breakdowns. Ultimately, we contribute an easily extensible user simulation framework to help the research community develop tool agents and preemptively diagnose them under challenging real-world conditions within their own services.

Method: Develops a data-driven pipeline to generate semantically labeled reasoning chains, then uses SFT followed by multi-stage RL guided by composite rewards including tag-based constraints and a Personalization Reward Model with User Embeddings (PRMU).

Result: Achieves state-of-the-art results on LaMP benchmark and self-constructed dataset with 32.65% average improvement over base model across all tasks.

Conclusion: Structured, interpretable reasoning is an effective pathway to unlocking genuine personalization capabilities in LLMs.

Abstract: Recent advancements have endowed Large Language Models (LLMs) with impressive general reasoning capabilities, yet they often struggle with personalization reasoning - the crucial ability to analyze user history, infer unique preferences, and generate tailored responses. To address this limitation, we introduce TagPR, a novel training framework that significantly enhances an LLM’s intrinsic capacity for personalization reasoning through a tagging the thought approach. Our method first develops a data-driven pipeline to automatically generate and semantically label reasoning chains, creating a structured dataset that fosters interpretable reasoning. We then propose a synergistic training strategy that begins with Supervised Fine-Tuning (SFT) on this tagged data to establish foundational reasoning patterns, followed by a multi-stage reinforcement learning (RL) process. This RL phase is guided by a unique composite reward signal, which integrates tag-based constraints and a novel Personalization Reward Model with User Embeddings (PRMU) to achieve fine-grained alignment with user-specific logic. Extensive experiments on the public LaMP benchmark and a self-constructed dataset demonstrate that our approach achieves state-of-the-art results, delivering an average improvement of 32.65% over the base model across all tasks. Our work validates that structured, interpretable reasoning is a highly effective pathway to unlocking genuine personalization capabilities in LLMs.

Method: Proposes UnmaskBranch for diversified token content and reveal order with single model calls, and ResubstituteScore for low-variance deterministic scoring of partially masked sequences.

Result: Achieves state-of-the-art results on perplexity, linguistic acceptability, and control of sentiment/toxicity, outperforming prior methods under matched compute budgets, especially in low-NFE regimes.

Conclusion: TReASURe effectively addresses key challenges in tree search for masked diffusion models, demonstrating theoretical efficiency gains and empirical superiority across multiple metrics.

Abstract: Tree search has recently emerged as a powerful framework for aligning generative models with task-specific rewards at test time. Applying tree search to Masked Diffusion Language Models, however, introduces two key challenges: (i) parallel unmasking yields highly correlated branches, limiting exploration, and (ii) reward evaluation via sampled completions produces high-variance estimates, making pruning unstable. We propose TReASURe, a tree-search test-time alignment method that addresses these issues. It introduces (i) UnmaskBranch, a branching strategy based on first-hitting unmasking that diversifies both token content and reveal order with a single model call per parent node, and (ii) ResubstituteScore, a pruning rule that uses deterministic resubstitution to score partially masked sequences with low-variance proxy completions. Theoretically, we quantify branching efficiency gains in NFEs (number of function evaluations), show that the scoring rule approximates the true reward with error bounded by predictive uncertainty, and prove improvements with larger tree widths. Empirically, TReASURe achieves state-of-the-art results on perplexity, linguistic acceptability, and control of sentiment and toxicity, outperforming prior methods under matched compute budgets, with especially strong gains in low-NFE regimes.

Method: Proposes Test-Time Policy Adaptation for Multi-Turn Interactions (T2PAM) paradigm and Optimum-Referenced One-Step Adaptation (ROSA) algorithm that uses user feedback as reward signal to estimate optimal policy and updates a small parameter subset in a single efficient step.

Result: Extensive experiments show ROSA achieves significant improvements in both task effectiveness and efficiency on challenging benchmarks, with theoretical guarantees of convergence to user preferences.

Conclusion: ROSA enables efficient in-conversation self-correction for LLMs through lightweight policy adaptation, addressing the limitations of static training and improving performance in extended multi-turn interactions.

Abstract: Large Language Models (LLMs) employ multi-turn interaction as a fundamental paradigm for completing complex tasks. However, their performance often degrades in extended interactions, as they are typically trained on static, single-turn data, which hinders their ability to adapt to real-time user feedback. To address this limitation, we first propose a new paradigm: Test-Time Policy Adaptation for Multi-Turn Interactions (T2PAM), which utilizes user feedback from the ongoing interaction as a reward signal to estimate a latent optimal policy aligned with user preferences, then updates a small subset of parameters to steer the model toward this policy, ultimately enabling efficient in-conversation self-correction. We then introduce Optimum-Referenced One-Step Adaptation (ROSA), a lightweight algorithm that operationalizes T2PAM. ROSA guides the model parameters toward a theoretical optimal policy in a single, efficient update step, avoiding costly iterative gradient-based optimization and minimizing computational overhead. We provide a rigorous theoretical analysis guaranteeing that the policy of ROSA converges to the preference of user as the number of interactions increases. Extensive experiments on challenging benchmark demonstrate that ROSA achieves significant improvements in both task effectiveness and efficiency.

Method: Pretrain LM to first generate intermediate latent thoughts (last hidden state of current position), then use these as input to predict subsequent tokens, enabling refinement in continuous space.

Result: At identical inference cost, LM with one additional latent thought per token outperforms standard model with double the parameters. 1.4B model surpasses vanilla 2.8B model on language modeling and downstream tasks.

Conclusion: Generating latent thoughts before each token (forming a chain analogous to CoT) consistently improves model performance, demonstrating the effectiveness of computational step scaling during pretraining.

Abstract: The remarkable success of Chain-of-Thought (CoT), which enhances performance by scaling generation steps at test-time, inspires us to ask: can we leverage a similar scaling of computational steps during pretraining to improve the generation of each individual token? To address this, we propose a novel pre-training methodology: Pretraining Language Models with Latent Thoughts. Our approach pretrains a language model (LM) to first generate an intermediate latent thought-the last hidden state of the current position-which is then used as input to predict the actual subsequent token. This additional computational step enables the LM to refine its prediction within unconstrained continuous space. Our experiments demonstrate that, at an identical inference cost, a LM that generates one additional latent thought per token outperforms a standard model with double the parameters. For instance, ours-1.4B (Pythia Arch), pretrained on 300B tokens from the Pile, significantly surpasses the vanilla Pythia-2.8B trained on the same data on both language modeling and a range of general downstream tasks. Furthermore, increasing the number of latent thoughts generated before each actual token-forming a chain analogous to CoT-consistently improves the model’s performance.

Method: Three-stage framework: (1) CRAS metric for role adherence measurement, (2) attention drift analysis to localize conflict resolution in middle layers, (3) SAIL method using LoRA on focal layers with token-weighted DPO optimization.

Result: Improved instruction hierarchy compliance by +5.60% on MedQA with AutoGen without full-model finetuning across standard benchmarks.

Conclusion: The surgical alignment approach effectively addresses compliance failures in multi-agent systems by targeting specific layers and optimizing with attention-aware objectives.

Abstract: Large Language Model (LLM)-powered multi-agent systems (MAS) have rapidly advanced collaborative reasoning, tool use, and role-specialized coordination in complex tasks. However, reliability-critical deployment remains hindered by a systemic failure mode: hierarchical compliance under instruction conflicts (system-user, peer-peer), where agents misprioritize system-level rules in the presence of competing demands. Moreover, widely used macro-level metrics (e.g., pass@k) obscure these micro-level violations and offer little actionable guidance for remedy. In this work, we present a full-stack, three-stage framework: (1) Diagnose - Contextualized Role Adherence Score (CRAS), a query-wise, context-aware scoring metric that decomposes role adherence into four measurable dimensions; (2) Localize - attention drift analysis revealing that instruction conflicts are resolved by attention heads that are largely concentrated in middle layers; (3) Align - Surgical Alignment of Instruction Layers (SAIL), which installs LoRA only on the localized focal layers and optimizes a token-weighted DPO-style preference objective that credits tokens by their focal attentional contribution. Across standard benchmarks and MAS frameworks, our surgical approach improves instruction hierarchy compliance (e.g., +5.60% with AutoGen on MedQA) without full-model finetuning.

Method: Used co-registered EEG and eye-tracking data from the ZuCo corpus, analyzed gaze transitions, applied Bayesian network modeling to examine structural depth effects, and measured fixation-related potentials for syntactic surprisal.

Result: Readers preferentially moved between syntactic heads, structural depth was the strongest driver of reading deviations, and syntactic surprisal influenced neural activity before word onset and during early integration.

Conclusion: Phrase structure construction can precede category detection and dominate lexical influences, supporting a predictive ’tree-scaffolding’ account of comprehension that extends current models of syntactic timing.

Abstract: A central question in psycholinguistics is the timing of syntax in sentence processing. Much of the existing evidence comes from violation paradigms, which conflate two separable processes - syntactic category detection and phrase structure construction - and implicitly assume that phrase structure follows category detection. In this study, we use co-registered EEG and eye-tracking data from the ZuCo corpus to disentangle these processes and test their temporal order under naturalistic reading conditions. Analyses of gaze transitions showed that readers preferentially moved between syntactic heads, suggesting that phrase structures, rather than serial word order, organize scanpaths. Bayesian network modeling further revealed that structural depth was the strongest driver of deviations from linear reading, outweighing lexical familiarity and surprisal. Finally, fixation-related potentials demonstrated that syntactic surprisal influences neural activity before word onset (-184 to -10 ms) and during early integration (48 to 300 ms). These findings extend current models of syntactic timing by showing that phrase structure construction can precede category detection and dominate lexical influences, supporting a predictive “tree-scaffolding” account of comprehension.

Method: Introduced Insight-to-Solve (I2S), a sequential test-time procedure that turns demonstrations into explicit, reusable insights and derives target-specific reasoning traces. Optionally includes self-refinement for coherence and correctness (I2S+).

Result: I2S and I2S+ consistently outperform both direct answering and test-time scaling baselines across diverse benchmarks and models. GPT-4.1 improved by +14.0% on AIME'25, and o1-mini improved by +2.7% on AIME and +1.7% on GPQA.

Conclusion: In-context demonstrations can be effectively harnessed through the insight-refine-solve framework, overcoming the limitations of traditional few-shot CoT approaches.

Abstract: Recent reasoning LLMs (RLMs), especially those trained with verifier-based reinforcement learning, often perform worse with few-shot CoT than with direct answering. We revisit this paradox using high-quality reasoning traces from DeepSeek-R1 as demonstrations and find that adding more exemplars consistently degrades accuracy, even when demonstrations are optimal. A detailed analysis reveals two mechanisms behind this decline: (i) semantic misguidance, where high textual similarity leads the model to treat the target as the same as the exemplar and to copy intermediate steps verbatim; and (ii) strategy transfer failure, where the model struggles to extract useful reasoning strategies and apply them to target questions. Guided by these, we introduce Insight-to-Solve (I2S), a sequential test-time procedure that turns demonstrations into explicit, reusable insights and derives a target-specific reasoning trace; optionally, the reasoning is self-refined for coherence and correctness (I2S+). Extensive experiments on diverse benchmarks show that I2S and I2S+ consistently outperform both direct answering and test-time scaling baselines across open- and closed-source models. Even for GPT models, our method helps: on AIME'25, GPT-4.1 rises by +14.0%, and o1-mini improves by +2.7% on AIME and +1.7% on GPQA, indicating that in-context demonstrations can be harnessed effectively via insight-refine-solve framework.

Method: Compiled dataset of K-pop songs on Billboard Hot 100 and Global 200 charts (2017-2025), analyzed English/Korean proportions, code-switching frequency, performed statistical tests for gender differences, and conducted classification task using multilingual embeddings and handcrafted features.

Result: English dominates globally charting K-pop songs with high code-switching; no significant gender-based differences found; female solo artists tend to use more English; classification achieved 0.76 F1 score for gender prediction; higher English usage may be more critical for US Hot 100 success.

Conclusion: Linguistic choices in K-pop lyrics are shaped by global market pressures, with code-switching and English usage reflecting performer identity and chart context, serving as strategic tools for international success.

Abstract: Code switching, particularly between Korean and English, has become a defining feature of modern K-pop, reflecting both aesthetic choices and global market strategies. This paper is a primary investigation into the linguistic strategies employed in K-pop songs that achieve global chart success, with a focus on the role of code-switching and English lyric usage. A dataset of K-pop songs that appeared on the Billboard Hot 100 and Global 200 charts from 2017 to 2025, spanning 14 groups and 8 solo artists, was compiled. Using this dataset, the proportion of English and Korean lyrics, the frequency of code-switching, and other stylistic features were analysed. It was found that English dominates the linguistic landscape of globally charting K-pop songs, with both male and female performers exhibiting high degrees of code-switching and English usage. Statistical tests indicated no significant gender-based differences, although female solo artists tend to favour English more consistently. A classification task was also performed to predict performer gender from lyrics, achieving macro F1 scores up to 0.76 using multilingual embeddings and handcrafted features. Finally, differences between songs charting on the Hot 100 versus the Global 200 were examined, suggesting that, while there is no significant gender difference in English, higher English usage may be more critical for success in the US-focused Hot 100. The findings highlight how linguistic choices in K-pop lyrics are shaped by global market pressures and reveal stylistic patterns that reflect performer identity and chart context.

Method: Used a prompt suite with with-headed prepositional phrases in ambiguous contexts, projected activations into vocabulary space to identify attention heads, and scaled value vectors of specific attention heads to control functional role distribution.

Result: Found a 3:4 preference for instrumental readings, and by scaling a single attention head’s value vector, shifted the distribution to 33% instrumental and 36% attributive complements.

Conclusion: Individual attention heads play a crucial role in determining prepositional phrase function, and their manipulation enables controlled shifting between instrumental and attributive interpretations.

Abstract: Language Models, when generating prepositional phrases, must often decide for whether their complements functions as an instrumental adjunct (describing the verb adverbially) or an attributive modifier (enriching the noun adjectivally), yet the internal mechanisms that resolve this split decision remain poorly understood. In this study, we conduct a targeted investigation into Gemma-2 to uncover and control the generation of prepositional complements. We assemble a prompt suite containing with-headed prepositional phrases whose contexts equally accommodate either an instrumental or attributive continuation, revealing a strong preference for an instrumental reading at a ratio of 3:4. To pinpoint individual attention heads that favor instrumental over attributive complements, we project activations into the vocabulary space. By scaling the value vector of a single attention head, we can shift the distribution of functional roles of complements, attenuating instruments to 33% while elevating attributes to 36%.

Method: PARL-MT framework includes: (1) Progress Awareness Generation pipeline for automatic dataset construction, and (2) Progress Awareness-Guided RL algorithm that integrates progress awareness into training to reduce redundancy and align local actions with global tasks.

Result: Empirical results on two public benchmarks show PARL-MT significantly outperforms existing methods in multi-turn function calling.

Conclusion: Progress awareness is effective for enabling robust and efficient multi-turn function calling in LLMs.

Abstract: Large language models (LLMs) have achieved impressive success in single-turn function calling, yet real-world applications such as travel planning or multi-stage data analysis typically unfold across multi-turn conversations. In these settings, LLMs must not only issue accurate function calls at each step but also maintain progress awareness, the ability to summarize past interactions and plan future actions to ensure coherent, long-horizon task execution. Existing approaches, however, either reduce multi-turn training to isolated single-turn samples, which neglects task-level planning, or employ end-to-end reinforcement learning (RL) that struggles with redundancy and lacks explicit integration of progress awareness. To overcome these limitations, we introduce PARL-MT, a framework that explicitly incorporates progress awareness into LLM training for multi-turn function calling. PARL-MT combines (i) a Progress Awareness Generation (PAG) pipeline, which automatically constructs datasets coupling conversation summaries with future task planning, and (ii) a Progress Awareness-Guided Reinforcement Learning (PAG-RL) algorithm, which integrates progress awareness into RL training to reduce contextual redundancy and improve alignment between local actions and global task completion. Empirical results on two public benchmarks demonstrate that PARL-MT significantly outperforms existing methods, highlighting the effectiveness of progress awareness in enabling robust and efficient multi-turn function calling.

Method: Developed a quantitative framework by extracting multi-dimensional evaluative features from human expert critiques using zero-shot classification, then tested VLMs like Llama, Qwen, and Gemini through persona-guided prompting.

Result: Revealed current performance levels, strengths, and areas for improvement of VLMs in art critique, showing their potential and limitations in complex semantic understanding tasks.

Conclusion: The study provides insights into VLMs’ capabilities in art critique and offers a framework for evaluating their performance in culturally complex domains like traditional Chinese painting.

Abstract: This study aims to test and evaluate the capabilities and characteristics of current mainstream Visual Language Models (VLMs) in generating critiques for traditional Chinese painting. To achieve this, we first developed a quantitative framework for Chinese painting critique. This framework was constructed by extracting multi-dimensional evaluative features covering evaluative stance, feature focus, and commentary quality from human expert critiques using a zero-shot classification model. Based on these features, several representative critic personas were defined and quantified. This framework was then employed to evaluate selected VLMs such as Llama, Qwen, or Gemini. The experimental design involved persona-guided prompting to assess the VLM’s ability to generate critiques from diverse perspectives. Our findings reveal the current performance levels, strengths, and areas for improvement of VLMs in the domain of art critique, offering insights into their potential and limitations in complex semantic understanding and content generation tasks. The code used for our experiments can be publicly accessed at: https://github.com/yha9806/VULCA-EMNLP2025.

Method: Developed CLAIRE - an agentic system combining LLM reasoning with retrieval to surface potentially inconsistent claims with contextual evidence for human review.

Result: In user studies, editors using CLAIRE identified 64.7% more inconsistencies with 87.5% higher confidence. Found at least 3.3% of English Wikipedia facts contradict another fact, with inconsistencies affecting 7.3% of FEVEROUS and 4.0% of AmbigQA examples. Best automated system achieved only 75.1% AUROC.

Conclusion: Contradictions are a measurable component of Wikipedia, and LLM-based systems like CLAIRE provide practical tools to help editors improve knowledge consistency at scale.

Abstract: Wikipedia is the largest open knowledge corpus, widely used worldwide and serving as a key resource for training large language models (LLMs) and retrieval-augmented generation (RAG) systems. Ensuring its accuracy is therefore critical. But how accurate is Wikipedia, and how can we improve it? We focus on inconsistencies, a specific type of factual inaccuracy, and introduce the task of corpus-level inconsistency detection. We present CLAIRE, an agentic system that combines LLM reasoning with retrieval to surface potentially inconsistent claims along with contextual evidence for human review. In a user study with experienced Wikipedia editors, 87.5% reported higher confidence when using CLAIRE, and participants identified 64.7% more inconsistencies in the same amount of time. Combining CLAIRE with human annotation, we contribute WIKICOLLIDE, the first benchmark of real Wikipedia inconsistencies. Using random sampling with CLAIRE-assisted analysis, we find that at least 3.3% of English Wikipedia facts contradict another fact, with inconsistencies propagating into 7.3% of FEVEROUS and 4.0% of AmbigQA examples. Benchmarking strong baselines on this dataset reveals substantial headroom: the best fully automated system achieves an AUROC of only 75.1%. Our results show that contradictions are a measurable component of Wikipedia and that LLM-based systems like CLAIRE can provide a practical tool to help editors improve knowledge consistency at scale.

Method: Two-stage training with progressive unfreezing: first training classifier head with frozen backbone, then fine-tuning entire model with differential learning rates. Uses dynamic thresholding based on probability curvature instead of fixed cutoffs.

Result: Strong precision and F1 performance on real-world transcripts, with interpretable output suitable for downstream auditing and question-answering workflows.

Conclusion: Fin-ExBERT offers a deployable solution for financial dialogue mining with batched evaluation, visualization, and calibrated export capabilities.

Abstract: Financial dialogue transcripts pose a unique challenge for sentence-level information extraction due to their informal structure, domain-specific vocabulary, and variable intent density. We introduce Fin-ExBERT, a lightweight and modular framework for extracting user intent-relevant sentences from annotated financial service calls. Our approach builds on a domain-adapted BERT (Bidirectional Encoder Representations from Transformers) backbone enhanced with LoRA (Low-Rank Adaptation) adapters, enabling efficient fine-tuning using limited labeled data. We propose a two-stage training strategy with progressive unfreezing: initially training a classifier head while freezing the backbone, followed by gradual fine-tuning of the entire model with differential learning rates. To ensure robust extraction under uncertainty, we adopt a dynamic thresholding strategy based on probability curvature (elbow detection), avoiding fixed cutoff heuristics. Empirical results show strong precision and F1 performance on real-world transcripts, with interpretable output suitable for downstream auditing and question-answering workflows. The full framework supports batched evaluation, visualization, and calibrated export, offering a deployable solution for financial dialogue mining.

Method: Aligns dLLMs at token-level under randomized masking to emit [EOS] refusal signals whenever harmful content arises, enabling robustness to any-decoding-order and any-step prefilling attacks.

Result: Reduces DIJA attack success rates from over 80% to near-zero (1.3% on LLaDA-8B-Instruct, 0.0% on Dream-v0-Instruct-7B), enables real-time monitoring and early rejection with up to 19.3x faster safe termination.

Conclusion: A2D provides effective defense against any-order generation attacks in diffusion LLMs through token-level safety alignment and enables practical real-time safety monitoring.

Abstract: Diffusion large language models (dLLMs) enable any-order generation, but this flexibility enlarges the attack surface: harmful spans may appear at arbitrary positions, and template-based prefilling attacks such as DIJA bypass response-level refusals. We introduce A2D (Any-Order, Any-Step Defense), a token-level alignment method that aligns dLLMs to emit an [EOS] refusal signal whenever harmful content arises. By aligning safety directly at the token-level under randomized masking, A2D achieves robustness to both any-decoding-order and any-step prefilling attacks under various conditions. It also enables real-time monitoring: dLLMs may begin a response but automatically terminate if unsafe continuation emerges. On safety benchmarks, A2D consistently prevents the generation of harmful outputs, slashing DIJA success rates from over 80% to near-zero (1.3% on LLaDA-8B-Instruct, 0.0% on Dream-v0-Instruct-7B), and thresholded [EOS] probabilities allow early rejection, yielding up to 19.3x faster safe termination.

Method: Introduce Policy Reasoning Traces (PRT) as specialized generated reasoning chains that serve as a reasoning bridge to enhance LLM’s policy compliance assessment capabilities.

Result: PRTs significantly enhance performance of both open-weight and commercial models in policy compliance assessment, setting new state-of-the-art for HIPAA and GDPR policies. They also improve LLM’s ability to accurately cite policy clauses and influence compliance decisions.

Conclusion: Policy Reasoning Traces effectively bridge the reasoning gap in policy compliance assessment, providing substantial improvements in model performance and citation accuracy across different policy domains.

Abstract: Policy compliance assessment is a fundamental task of evaluating whether an input case strictly complies with a set of human-defined rules, more generally known as policies. In practice, human experts follow a systematic, step-by-step process to identify violations with respect to specific stipulations outlined in the policy. However, such documentation of gold-standard, expert-level reasoning processes is costly to acquire. In this paper, we introduce Policy Reasoning Traces (PRT), a form of specialized generated reasoning chains that serve as a reasoning bridge to improve an LLM’s policy compliance assessment capabilities. Our empirical evaluations demonstrate that the use of PRTs for both inference-time and training-time scenarios significantly enhances the performance of open-weight and commercial models, setting a new state-of-the-art for HIPAA and GDPR policies. Beyond accuracy gains, we also highlight how PRTs can improve an LLM’s ability to accurately cite policy clauses, as well as influence compliance decisions through their high utilization from the raw chains of thought.

Method: SIE automatically constructs reasoning environments from large-scale structured data, leveraging rich compositional patterns for generalizable reasoning and using explicit schemas for rule-based verifiability.

Result: SIE achieves substantial improvements in in-domain structured reasoning and enables learned compositional reasoning skills to generalize effectively to out-of-domain mathematical and logical reasoning tasks. LLMs can also infer missing information in partial SIEs.

Conclusion: SIE provides a scalable, generalizable, and verifiable framework for RL finetuning of LLMs that bridges the gap between existing environment types and enables robust reasoning improvements.

Abstract: Large language models (LLMs) have achieved significant advancements in reasoning capabilities through reinforcement learning (RL) via environmental exploration. As the intrinsic properties of the environment determine the abilities that LLMs can learn, the environment plays a important role in the RL finetuning process. An ideal LLM reasoning environment should possess three core characteristics: scalability, generalizable reasoning, and verifiability. However, existing mathematical and coding environments are difficult to scale due to heavy reliance on expert annotation, while the skills learned in game-based environments are too specialized to generalize. To bridge this gap, we introduce the \textbf{S}tructured \textbf{I}n-context \textbf{E}nvironment (SIE) framework. SIE achieves scalability by automatically constructing reasoning environments from large-scale structured data, where the rich compositional patterns naturally support generalizable reasoning. Moreover, the explicit schemas and reasoning chains in structured data provide a foundation for rule-based verifiability. Experimental results show that SIE framework not only achieves substantial improvements in in-domain structured reasoning, but also enables the learned compositional reasoning skills to generalize effectively to out-of-domain mathematical and logical reasoning tasks. We further explored learning in information-limited partial SIEs and found that LLMs can infer the missing information through exploring the environment, leading to robust reasoning improvements and generalization performance.

Method: Uses an island-based evolutionary algorithm to maintain population diversity, forms groups from distinct islands, and employs voting score as fitness metric to evaluate each prompt’s contribution within groups.

Result: Achieves SOTA performance: 70.67% on HotpotQA and 43.88% on IFBench with Qwen3-8B (4.95% and 2.73% higher than GEPA), and 47.96% on IFBench and 95.33% on MATH with GPT-4.1-mini.

Conclusion: C-Evolve demonstrates competitive performance across diverse tasks by evolving prompts that work well together in consensus groups rather than individually.

Abstract: Prompt evolution algorithms offer a powerful paradigm for enhancing AI systems based on closed-source models, while few work explores whether aggregating results from multiple prompts to reach a consensus can further advance the system capability boundary. In this paper, we introduce Consensus-Evolve (C-Evolve), an evolutionary algorithm that discovers a group of prompts whose aggregated outputs after majority voting achieve optimal performance. More specifically, C-Evolve employs an island-based evolutionary algorithm to maintain population diversity, and prompts from distinct islands are selected to form groups to aggregate their outputs. The key difference from single individual evolution is a voting score, which evaluates each individual prompt’s contribution within groups. We take this as the fitness score for evolution instead of individual performance. Consequently, C-Evolve is more likely to produce and maintain prompts with higher potential to form a high-performing group and eliminate low-performing ones, gradually improving the group performance after reaching consensus. Our method achieves state-of-the-art performance across a wide range of tasks, including both open-ended tasks like HotpotQA and closed-ended tasks like MATH. On Qwen3-8B, C-Evolve achieves 70.67% on HotpotQA and 43.88% on IFBench, which are 4.95% and 2.73% higher than GEPA, respectively. For GPT-4.1-mini, the accuracy on IFBench is further improved to 47.96% and reaches 95.33% in the MATH benchmark. These results demonstrate the C-Evolve’s competitive performance.

Method: Two-stage smoothness optimization: (1) representation space stage with robustly trained probe to defend against jailbreak attacks, (2) parameter-space stage that decouples retain-forget gradient conflicts and smooths parameter space to mitigate relearning attacks.

Result: Extensive experiments on WMDP and MUSE datasets across conversational-dialogue and continuous-text settings show PRISM outperforms SOTA baselines under multiple attacks while achieving better balance among key metrics.

Conclusion: PRISM provides an effective solution for robust machine unlearning by enforcing dual-space smoothness, addressing current limitations in balancing unlearning effectiveness, utility preservation, and privacy protection.

Abstract: With the rapid advancement of large language models, Machine Unlearning has emerged to address growing concerns around user privacy, copyright infringement, and overall safety. Yet state-of-the-art (SOTA) unlearning methods often suffer from catastrophic forgetting and metric imbalance, for example by over-optimizing one objective (e.g., unlearning effectiveness, utility preservation, or privacy protection) at the expense of others. In addition, small perturbations in the representation or parameter space can be exploited by relearn and jailbreak attacks. To address these challenges, we propose PRISM, a unified framework that enforces dual-space smoothness in representation and parameter spaces to improve robustness and balance unlearning metrics. PRISM consists of two smoothness optimization stages: (i) a representation space stage that employs a robustly trained probe to defend against jailbreak attacks, and (ii) a parameter-space stage that decouples retain-forget gradient conflicts, reduces imbalance, and smooths the parameter space to mitigate relearning attacks. Extensive experiments on WMDP and MUSE, across conversational-dialogue and continuous-text settings, show that PRISM outperforms SOTA baselines under multiple attacks while achieving a better balance among key metrics.

Method: Two-stage framework: 1) Extract thought templates from teacher model to guide student model, 2) Use direct preference optimization through model self-iteration collaboration where student plays against its own correction trajectory.

Result: MedCritical 7B model outperforms Taiyi and Huatuo-o1-7B models by 3.04% and 10.12% respectively on CMExam benchmark, achieving new SOTA performance among 7B-class small models.

Conclusion: The proposed self-learning DPO approach enables small models to achieve comparable results to traditional knowledge distillation at lower cost, demonstrating effective complex medical reasoning capabilities.

Abstract: In the field of medicine, complex reasoning tasks such as clinical diagnosis, treatment planning, and medical knowledge integration pose significant challenges, where small language models often underperform compared to large language models like GPT-4 and Deepseek. Recent knowledge distillation-based methods aim to address these issues through teacher-guided error correction, but this LLM as judge approach remains challenging in terms of cost, time, and efficiency. To circumvent this issue, we propose a novel two-stage framework, MedCritical, which uses a small language model fine-tuned by a large teacher model to play against itself. In the first stage, we extract high-level and detailed long-chain thought templates from the teacher model to guide the student model to generate more complex reasoning thoughts. In the second stage, we introduce direct preference optimization (DPO) through model self-iteration collaboration to enhance the reasoning ability of the student model by playing against the correction trajectory of the fine-tuned model during training. This model self-learning DPO approach teaches the student model to use its own error-driven insights to consolidate its skills and knowledge to solve complex problems, and achieves comparable results to traditional knowledge distillation methods using teacher models at a lower cost. Notably, our MedCritical 7B model outperforms the Taiyi and Huatuo-o1-7B models by 3.04% and 10.12% respectively on the CMExam benchmark, achieving new SOTA performance among 7B-class small models.

Method: Uses a lightweight meta-learner as an ‘alignment gap estimator’ to evaluate benefits of on-policy sampling vs offline data, guiding targeted online generation and assigning sample-wise meta-weights to dynamically balance quality and distribution of online and offline data.

Result: Consistently outperforms existing preference optimization approaches on AlpacaEval 2, Arena-Hard and MT-Bench benchmarks across various settings.

Conclusion: MetaAPO effectively bridges the distribution gap in preference optimization while significantly reducing online annotation costs, demonstrating superior performance over existing methods.

Abstract: Preference optimization is crucial for aligning large language models (LLMs) with human values and intentions. A significant challenge in this process is the distribution mismatch between pre-collected offline preference data and the evolving model policy. Existing methods attempt to reduce this gap using static heuristics or decoupled online sampling strategies, but they often fail to adapt to the model’s dynamic learning state. To bridge this gap, we propose Meta-Weighted Adaptive Preference Optimization (MetaAPO), a novel framework that dynamically couples data generation with model training. MetaAPO employs a lightweight meta-learner, as an “alignment gap estimator”, to evaluate the potential benefits of on-policy sampling in relation to offline data. This guides targeted online generation and assigns sample-wise meta-weights to the optimization objective, dynamically balancing the quality and distribution of online and offline data. Experiments on AlpacaEval 2, Arena-Hard and MT-Bench demonstrate that MetaAPO consistently outperforms existing preference optimization approaches across various settings, while reducing 42% in online annotation costs.

Method: Clinical Contrastive Decoding (CCD) - a training-free, retrieval-free inference framework that integrates structured clinical signals from radiology expert models using a dual-stage contrastive mechanism to refine token-level logits during generation.

Result: CCD consistently improves performance on radiology report generation across three datasets and multiple models. On MIMIC-CXR, it achieves up to 17% improvement in RadGraph-F1 when applied to state-of-the-art RRG models.

Conclusion: CCD provides a lightweight, generalizable solution for mitigating medical hallucinations by effectively bridging expert models and MLLMs in radiology without modifying the base MLLM.

Abstract: Multimodal large language models (MLLMs) have recently achieved remarkable progress in radiology by integrating visual perception with natural language understanding. However, they often generate clinically unsupported descriptions, known as medical hallucinations, which pose serious risks in medical applications that demand accuracy and image-grounded outputs. Through empirical analysis, we find that prompt-induced hallucinations remain prevalent in radiology MLLMs, largely due to over-sensitivity to clinical sections. To address this, we introduce Clinical Contrastive Cecoding (CCD), a training-free and retrieval-free inference framework that integrates structured clinical signals from task-specific radiology expert models. CCD introduces a dual-stage contrastive mechanism to refine token-level logits during generation, thereby enhancing clinical fidelity without modifying the base MLLM. Experiments on three datasets and multiple models demonstrate that CCD consistently improves overall performance on radiology report generation (RRG). On the MIMIC-CXR dataset, it yields up to a 17% improvement in RadGraph-F1 when applied to state-of-the-art RRG models. Our approach provides a lightweight and generalisable solution for mitigating medical hallucinations, effectively bridging expert models and MLLMs in radiology.

Method: Compute Guard Vector as parameter difference between guardrail and pretrained models, compose with target models to create Target Guard Models, use prefix-based training with single-token classifier output for streaming optimization.

Result: Improves classification quality over existing guard models, enables multilingual support (Chinese, Japanese, Korean) without additional training, demonstrates portability across Llama and Gemma architectures, maintains performance under streaming with reduced latency and increased throughput.

Conclusion: Guard Vector provides an efficient, portable safety solution that reduces computational requirements while enabling multilingual safety capabilities and promoting responsible AI practices through streaming-aware evaluation.

Abstract: We introduce Guard Vector, a safety task vector computed as the parameter difference between a guardrail model (Guard Model) and a same-architecture pretrained language model. Composing this vector with a target language model yields a Target Guard Model (TGM). We then adapt TGM with a streaming-aware approach that combines prefix-based training and evaluation with a classifier that produces a single-token output. With this composition alone, TGM improves classification quality over established Guard Models across standard safety suites and enables language extensibility to Chinese, Japanese, and Korean, requiring neither additional training nor target language labels. It also demonstrates model portability across two widely used public guardrail backbones, Llama and Gemma. With prefix SFT (supervised fine-tuning), TGM preserves classification quality under streaming by aligning the behavior between prefix inputs and full-text inputs. The single-token output design increases throughput and reduces latency. Together, these components reduce data and compute requirements while promoting streaming-aware evaluation practices, thereby contributing to a more responsible AI ecosystem.

Method: Conduct ten different experiments during a single training run of a 1.5B parameter model on 210B tokens, testing for interactions between experiments through continual pretraining.

Result: Successfully replicated results from previous works on data contamination, poisoning, and memorization, and conducted novel investigations into knowledge acquisition, mathematical reasoning, and watermarking with minimal impact on training dynamics and overall performance.

Conclusion: Performing multiple pretraining experiments in a single training run is feasible and can enable rigorous scientific experimentation with large models on a limited compute budget, though interactions between experiments should be tested.

Abstract: Recent work has demonstrated that controlled pretraining experiments are a powerful tool for understanding learning, reasoning, and memorization in large language models (LLMs). However, the computational cost of pretraining presents a significant constraint. To overcome this constraint, we propose to conduct multiple pretraining experiments simultaneously during a single training run. We demonstrate the feasibility of this approach by conducting ten experiments during the training of a 1.5B parameter model on 210B tokens. Although we only train a single model, we can replicate the results from multiple previous works on data contamination, poisoning, and memorization. We also conduct novel investigations into knowledge acquisition, mathematical reasoning, and watermarking. For example, we dynamically update the training data until the model acquires a particular piece of knowledge. Remarkably, the influence of the ten experiments on the model’s training dynamics and overall performance is minimal. However, interactions between different experiments may act as a potential confounder in our approach. We propose to test for interactions with continual pretraining experiments, finding them to be negligible in our setup. Overall, our findings suggest that performing multiple pretraining experiments in a single training run can enable rigorous scientific experimentation with large models on a compute budget.

Method: Combines gated refinement (feedback regulation gate and update rejection gate) for stable improvements, and adaptive compression to escape local optima by distilling core concepts when optimization stagnates.

Result: Achieved average relative improvements of 4.7%, 4.4%, and 2.7% over SOTA methods on BBH, domain-specific, and general NLP tasks respectively, using only 25% of the prompt generation budget.

Conclusion: GRACE demonstrates that strategic information loss through refinement and compression enables efficient and effective prompt optimization with substantial performance gains and reduced computational overhead.

Abstract: Prompt engineering is crucial for leveraging the full potential of large language models (LLMs). While automatic prompt optimization offers a scalable alternative to costly manual design, generating effective prompts remains challenging. Existing methods often struggle to stably generate improved prompts, leading to low efficiency, and overlook that prompt optimization easily gets trapped in local optima. Addressing this, we propose GRACE, a framework that integrates two synergistic strategies: Gated Refinement and Adaptive Compression, achieving Efficient prompt optimization. The gated refinement strategy introduces a feedback regulation gate and an update rejection gate, which refine update signals to produce stable and effective prompt improvements. When optimization stagnates, the adaptive compression strategy distills the prompt’s core concepts, restructuring the optimization trace and opening new paths. By strategically introducing information loss through refinement and compression, GRACE delivers substantial gains in performance and efficiency. In extensive experiments on 11 tasks across three practical domains, including BIG-Bench Hard (BBH), domain-specific, and general NLP tasks, GRACE achieves significant average relative performance improvements of 4.7%, 4.4% and 2.7% over state-of-the-art methods, respectively. Further analysis shows that GRACE achieves these gains using only 25% of the prompt generation budget required by prior methods, highlighting its high optimization efficiency and low computational overhead. Our code is available at https://github.com/Eric8932/GRACE.

Method: Formalized Genette’s paratext theory, created a dataset of 560 expert-aligned paratexts from English translations of Liaozhai, and evaluated LLMs with/without reasoning traces on explicitation choice and content using intrinsic prompting and agentic retrieval methods.

Result: LLM-generated paratexts improve audience comprehension but are less effective than translator-authored ones. Statistical analysis shows wide variation in professional translators’ paratext usage, indicating cultural mediation is open-ended rather than prescriptive.

Conclusion: Paratextual explicitation has potential to advance MT beyond linguistic equivalence, with extensions to monolingual explanation and personalized adaptation.

Abstract: The faithful transfer of contextually-embedded meaning continues to challenge contemporary machine translation (MT), particularly in the rendering of culture-bound terms–expressions or concepts rooted in specific languages or cultures, resisting direct linguistic transfer. Existing computational approaches to explicitating these terms have focused exclusively on in-text solutions, overlooking paratextual apparatus in the footnotes and endnotes employed by professional translators. In this paper, we formalize Genette’s (1987) theory of paratexts from literary and translation studies to introduce the task of paratextual explicitation for MT. We construct a dataset of 560 expert-aligned paratexts from four English translations of the classical Chinese short story collection Liaozhai and evaluate LLMs with and without reasoning traces on choice and content of explicitation. Experiments across intrinsic prompting and agentic retrieval methods establish the difficulty of this task, with human evaluation showing that LLM-generated paratexts improve audience comprehension, though remain considerably less effective than translator-authored ones. Beyond model performance, statistical analysis reveals that even professional translators vary widely in their use of paratexts, suggesting that cultural mediation is inherently open-ended rather than prescriptive. Our findings demonstrate the potential of paratextual explicitation in advancing MT beyond linguistic equivalence, with promising extensions to monolingual explanation and personalized adaptation.

Method: Used essays from large-scale tests, analyzed scoring accuracy with quadratic weighted kappa and normalized mutual information, evaluated rationale similarity with cosine similarity, and explored clustering patterns using PCA on rationale embeddings.

Result: The study provides insights into LLM scoring accuracy and “thinking” processes, revealing patterns in how LLMs and humans provide rationales for their scores.

Conclusion: The findings help improve understanding of rationales behind both human scoring and LLM-based automated scoring, contributing to better automated scoring systems.

Abstract: Advances in automated scoring are closely aligned with advances in machine-learning and natural-language-processing techniques. With recent progress in large language models (LLMs), the use of ChatGPT, Gemini, Claude, and other generative-AI chatbots for automated scoring has been explored. Given their strong reasoning capabilities, LLMs can also produce rationales to support the scores they assign. Thus, evaluating the rationales provided by both human and LLM raters can help improve the understanding of the reasoning that each type of rater applies when assigning a score. This study investigates the rationales of human and LLM raters to identify potential causes of scoring inconsistency. Using essays from a large-scale test, the scoring accuracy of GPT-4o, Gemini, and other LLMs is examined based on quadratic weighted kappa and normalized mutual information. Cosine similarity is used to evaluate the similarity of the rationales provided. In addition, clustering patterns in rationales are explored using principal component analysis based on the embeddings of the rationales. The findings of this study provide insights into the accuracy and ``thinking’’ of LLMs in automated scoring, helping to improve the understanding of the rationales behind both human scoring and LLM-based automated scoring.

Method: Proposed Retrieval-Constrained Decoding (RCD) that restricts model outputs to unique surface forms, tested on YAGO-QA dataset with LMs from 135M to 70B parameters.

Result: RCD significantly improves performance: Llama-3.1-70B increased from 32.3% to 46.0% F1, and Llama-3.1-8B reached 33.0% with RCD, outperforming larger model under standard decoding.

Conclusion: Current evaluation methods underestimate LM knowledge, and RCD decoding strategy better reveals models’ true parametric knowledge capabilities.

Abstract: Language models (LMs) encode substantial factual knowledge, but often produce answers judged as incorrect. We hypothesize that many of these answers are actually correct, but are expressed in alternative surface forms that are dismissed due to an overly strict evaluation, leading to an underestimation of models’ parametric knowledge. We propose Retrieval-Constrained Decoding (RCD), a decoding strategy that restricts model outputs to unique surface forms. We introduce YAGO-QA, a dataset of 19,137 general knowledge questions. Evaluating open-source LMs from 135M to 70B parameters, we show that standard decoding undervalues their knowledge. For instance, Llama-3.1-70B scores only 32.3% F1 with vanilla decoding but 46.0% with RCD. Similarly, Llama-3.1-8B reaches 33.0% with RCD, outperforming the larger model under vanilla decoding. We publicly share the code and dataset at https://github.com/Rajjaa/disambiguated-LLM.

Method: CooT couples a text Generator with a cognitive Perceiver that continuously monitors generation using a structured hierarchy of principles. When violations are detected, it rolls back generation and regenerates with injected guidance combining universal social priors and context-specific warnings.

Result: Extensive experiments across multiple benchmarks and model families confirm that CooT consistently improves safety and social reasoning performance.

Conclusion: CooT transforms alignment from a fixed property into an explicit, dynamic, and auditable process during inference, allowing flexible policy updates without model retraining.

Abstract: Large language models (LLMs) excel at complex reasoning but can still exhibit harmful behaviors. Current alignment strategies typically embed safety into model weights, making these controls implicit, static, and difficult to modify. This paper introduces Cognition-of-Thought (CooT), a novel decoding-time framework that equips LLMs with an explicit cognitive self-monitoring loop. CooT couples a standard text Generator with a cognitive Perceiver that continuously monitors the unfolding sequence. The Perceiver uses a structured, precedence-based hierarchy of principles (e.g., safety over obedience) to detect potential misalignments as they arise. When violations are flagged, CooT intervenes by rolling back the generation to the point of error and regenerating under injected guidance that combines universal social priors with context-specific warnings. CooT thus transforms alignment from a fixed property into an explicit, dynamic, and auditable process active during inference, allowing for flexible policy updates without retraining the model. Extensive experiments across multiple benchmarks and model families confirm that CooT consistently improves safety and social reasoning performance.

Method: Systematic review of 37 articles analyzing datasets, difficulty parameters, subject domains, item types, features, models, and evaluation criteria for automated difficulty prediction.

Result: State-of-the-art language models (including transformers) can predict item difficulty with RMSE as low as 0.165, Pearson correlation up to 0.87, and accuracy up to 0.806, outperforming classic ML models while eliminating manual feature engineering.

Conclusion: Text-based methods show strong potential for automated item difficulty modeling, with performance benchmarks established for future research. The review discusses practical implications and future research directions.

Abstract: Item difficulty plays a crucial role in test performance, interpretability of scores, and equity for all test-takers, especially in large-scale assessments. Traditional approaches to item difficulty modeling rely on field testing and classical test theory (CTT)-based item analysis or item response theory (IRT) calibration, which can be time-consuming and costly. To overcome these challenges, text-based approaches leveraging machine learning and language models, have emerged as promising alternatives. This paper reviews and synthesizes 37 articles on automated item difficulty prediction in large-scale assessment settings published through May 2025. For each study, we delineate the dataset, difficulty parameter, subject domain, item type, number of items, training and test data split, input, features, model, evaluation criteria, and model performance outcomes. Results showed that although classic machine learning models remain relevant due to their interpretability, state-of-the-art language models, using both small and large transformer-based architectures, can capture syntactic and semantic patterns without the need for manual feature engineering. Uniquely, model performance outcomes were summarized to serve as a benchmark for future research and overall, text-based methods have the potential to predict item difficulty with root mean square error (RMSE) as low as 0.165, Pearson correlation as high as 0.87, and accuracy as high as 0.806. The review concludes by discussing implications for practice and outlining future research directions for automated item difficulty modeling.

Method: Evaluated role configurations in zero-shot and few-shot learning using GPT-3.5, GPT-4o, Llama2-7b, and Llama2-13b across sentiment analysis, text classification, question answering, and math reasoning tasks.

Result: Role-based prompt structuring shows potential to enhance LLM performance.

Conclusion: Role design is a promising approach for improving in-context learning in LLMs.

Abstract: In-context learning (ICL) enables Large Language Models (LLMs) to generate predictions based on prompts without additional fine-tuning. While prompt engineering has been widely studied, the impact of role design within prompts remains underexplored. This study examines the influence of role configurations in zero-shot and few-shot learning scenarios using GPT-3.5 and GPT-4o from OpenAI and Llama2-7b and Llama2-13b from Meta. We evaluate the models’ performance across datasets, focusing on tasks like sentiment analysis, text classification, question answering, and math reasoning. Our findings suggest the potential of role-based prompt structuring to enhance LLM performance.

Method: Two-stage training: 1) Task-adaptive pretraining on large-scale Arabic speech-phonemes datasets generated using MSA Phonetiser, 2) Fine-tuning on shared task data with augmentation from XTTS-v2-synthesized recitations featuring varied Ayat segments, speaker embeddings, and textual perturbations.

Result: The system ranked first on the official Iqra’Eval 2025 Shared Task leaderboard, demonstrating superior performance in phoneme-level mispronunciation detection.

Conclusion: Phoneme-aware pretraining combined with targeted augmentation yields strong performance in Arabic phoneme recognition and mispronunciation detection tasks.

Abstract: This paper describes AraS2P, our speech-to-phonemes system submitted to the Iqra’Eval 2025 Shared Task. We adapted Wav2Vec2-BERT via Two-Stage training strategy. In the first stage, task-adaptive continue pretraining was performed on large-scale Arabic speech-phonemes datasets, which were generated by converting the Arabic text using the MSA Phonetiser. In the second stage, the model was fine-tuned on the official shared task data, with additional augmentation from XTTS-v2-synthesized recitations featuring varied Ayat segments, speaker embeddings, and textual perturbations to simulate possible human errors. The system ranked first on the official leaderboard, demonstrating that phoneme-aware pretraining combined with targeted augmentation yields strong performance in phoneme-level mispronunciation detection.

Method: Proposed active learning framework using multiple deep learning architectures (LSTM, GRU, RNN) across three Arabic datasets. Compared human labeling vs LLM-assisted labeling using five LLMs (GPT-4o, Claude 3 Sonnet, Gemini 2.5 Pro, DeepSeek Chat, LLaMA 3 70B Instruct).

Result: LLM-assisted active learning achieved competitive or superior performance to human labeling. LSTM with GPT-4o labels reached 93% accuracy with only 450 samples on Hunger Station dataset. DeepSeek Chat achieved 82% accuracy with 650 samples on MASAC dataset, matching human labeling performance.

Conclusion: LLM-assisted active learning is effective for Arabic sentiment analysis, reducing annotation costs while maintaining high accuracy across different datasets and language variations.

Abstract: Natural language processing (NLP), particularly sentiment analysis, plays a vital role in areas like marketing, customer service, and social media monitoring by providing insights into user opinions and emotions. However, progress in Arabic sentiment analysis remains limited due to the lack of large, high-quality labeled datasets. While active learning has proven effective in reducing annotation efforts in other languages, few studies have explored it in Arabic sentiment tasks. Likewise, the use of large language models (LLMs) for assisting annotation and comparing their performance to human labeling is still largely unexplored in the Arabic context. In this paper, we propose an active learning framework for Arabic sentiment analysis designed to reduce annotation costs while maintaining high performance. We evaluate multiple deep learning architectures: Specifically, long short-term memory (LSTM), gated recurrent units (GRU), and recurrent neural networks (RNN), across three benchmark datasets: Hunger Station, AJGT, and MASAC, encompassing both modern standard Arabic and dialectal variations. Additionally, two annotation strategies are compared: Human labeling and LLM-assisted labeling. Five LLMs are evaluated as annotators: GPT-4o, Claude 3 Sonnet, Gemini 2.5 Pro, DeepSeek Chat, and LLaMA 3 70B Instruct. For each dataset, the best-performing LLM was used: GPT-4o for Hunger Station, Claude 3 Sonnet for AJGT, and DeepSeek Chat for MASAC. Our results show that LLM-assisted active learning achieves competitive or superior performance compared to human labeling. For example, on the Hunger Station dataset, the LSTM model achieved 93% accuracy with only 450 labeled samples using GPT-4o-generated labels, while on the MASAC dataset, DeepSeek Chat reached 82% accuracy with 650 labeled samples, matching the accuracy obtained through human labeling.

Method: Study three aspects (future-proofing, backward compatibility, question generalization) in math domain using unified framework with varying train/test distributions, three SFT- and DPO-based finetuning algorithms, and three base models.

Result: Future-proofing is challenging, backward compatibility is relatively easy (DPO-trained models improve performance), continual learning provides balanced adaptation, and all models show performance degradation on unseen questions.

Conclusion: Current judges don’t fully generalize to unseen questions, and findings provide insights for developing judge models that can handle ever-changing generators.

Abstract: The LLM-as-a-judge paradigm is widely used in both evaluating free-text model responses and reward modeling for model alignment and finetuning. Recently, finetuning judges with judge-specific data has emerged as an often preferred choice over directly prompting frontier models as judges, as the former achieves better performance with smaller model sizes while being more robust to common biases. However, the standard evaluation ignores several practical concerns of finetuned judges regarding their real world deployment. In this paper, we identify and formalize three aspects that affect the shelf life of these judges: future proofing and backward compatibility – how well judges finetuned on responses by today’s generator models perform on responses by future models or past models, as well as question generalization – how well judges generalize to unseen questions at test time. We study these three aspects in the math domain under a unified framework with varying train and test distributions, three SFT- and DPO-based finetuning algorithms and three different base models. Experiments suggest that future-proofing is challenging for most models, while backward compatibility is relatively easy, with DPO-trained models consistently improving performance. We further find that continual learning provides a more balanced adaptation to shifts between older and newer response distributions than training solely on stronger or weaker responses. Moreover, all models observe certain degrees of performance degradation when moving from questions seen during training to unseen ones, showing that current judges do not fully generalize to unseen questions. These findings provide insights into practical considerations for developing and deploying judge models in the face of ever-changing generators.

Method: Combines automatic speech recognition techniques with text correction models, leveraging both acoustic and textual modeling. Uses domain-specific fine-tuning to adapt existing technologies to Greek medical context, addressing complex terminology and linguistic variations.

Result: The system achieves more accurate and coherent transcriptions through domain-specific adaptation, better handling Greek medical terminology and linguistic inconsistencies.

Conclusion: The developed system contributes to practical language technologies for Greek healthcare by providing reliable medical speech transcription, reducing documentation burden on healthcare professionals.

Abstract: Medical dictation systems are essential tools in modern healthcare, enabling accurate and efficient conversion of speech into written medical documentation. The main objective of this paper is to create a domain-specific system for Greek medical speech transcriptions. The ultimate goal is to assist healthcare professionals by reducing the overload of manual documentation and improving workflow efficiency. Towards this goal, we develop a system that combines automatic speech recognition techniques with text correction model, allowing better handling of domain-specific terminology and linguistic variations in Greek. Our approach leverages both acoustic and textual modeling to create more realistic and reliable transcriptions. We focused on adapting existing language and speech technologies to the Greek medical context, addressing challenges such as complex medical terminology and linguistic inconsistencies. Through domain-specific fine-tuning, our system achieves more accurate and coherent transcriptions, contributing to the development of practical language technologies for the Greek healthcare sector.

Method: Proposed Model-Oriented Rationale Selection Distillation (MoRSD) with Rationale Difficulty metric to select high-quality rationales based on accuracy, diversity, and difficulty.

Result: Achieved 4.6% average improvement on seven datasets across three tasks using fewer rationales than baseline methods.

Conclusion: High-quality rationales are more important than quantity for effective CoT distillation, and MoRSD provides an efficient solution for reasoning capability transfer.

Abstract: Chain-of-thought (CoT) distillation aims to enhance small language models’ (SLMs) reasoning by transferring multi-step reasoning capability from the larger teacher models. However, existing work underestimates rationale quality, focusing primarily on data quantity, which may transfer noisy or incorrect information to the student model. To address the above issues, we proposed \textbf{M}odel-\textbf{O}riented \textbf{R}ationale \textbf{S}election \textbf{D}istillation (MoRSD), which can discern and select high quality rationales for distillation to improve performance further. We further propose a Rationale Difficulty (RD) metric to measure the ability of the student model to generate the correct answer under a given rationale. Compared to the baseline, we achieved 4.6$%$ average improvement on seven datasets over three tasks, using fewer rationales by controlling their accuracy, diversity, and difficulty. Our results reveal that a small portion of the high quality rationales can enhance the reasoning ability of student models than the entire dataset. Our method promises to be a possible solution for efficient CoT distillation. Our code will be released in https://github.com/Leon221220/MoRSD.

Method: Created Jackal benchmark with 100,000 natural language requests paired with validated JQL queries, executed on a live Jira instance with 200,000+ issues. Includes four user request types: Long NL, Short NL, Semantically Similar, and Semantically Exact.

Result: Evaluated 23 LLMs on Jackal-5K subset. Best model (Gemini 2.5 Pro) achieved only 60.3% execution accuracy overall, with significant variation across request types: Long NL (86.0%), Short NL (35.7%), Semantically Similar (22.7%), and Semantically Exact (99.3%).

Conclusion: Current state-of-the-art LLMs have significant limitations in producing correct and executable JQL queries, particularly for short and semantically similar natural language requests, highlighting the need for improved text-to-JQL capabilities.

Abstract: Enterprise teams rely on the Jira Query Language (JQL) to retrieve and filter issues from Jira. Yet, to our knowledge, there is no open, real-world, execution-based benchmark for mapping natural language queries to JQL. We introduce Jackal, a novel, large-scale text-to-JQL benchmark comprising 100,000 natural language (NL) requests paired with validated JQL queries and execution-based results on a live Jira instance with over 200,000 issues. To reflect real-world usage, each JQL query is associated with four types of user requests: (i) Long NL, (ii) Short NL, (iii) Semantically Similar, and (iv) Semantically Exact. We release Jackal, a corpus of 100,000 text-to-JQL pairs, together with an execution-based scoring toolkit, and a static snapshot of the evaluated Jira instance for reproducibility. We report text-to-JQL results on 23 Large Language Models (LLMs) spanning parameter sizes, open and closed source models, across execution accuracy, exact match, and canonical exact match. In this paper, we report results on Jackal-5K, a 5,000-pair subset of Jackal. On Jackal-5K, the best overall model (Gemini 2.5 Pro) achieves only 60.3% execution accuracy averaged equally across four user request types. Performance varies significantly across user request types: (i) Long NL (86.0%), (ii) Short NL (35.7%), (iii) Semantically Similar (22.7%), and (iv) Semantically Exact (99.3%). By benchmarking LLMs on their ability to produce correct and executable JQL queries, Jackal exposes the limitations of current state-of-the-art LLMs and sets a new, execution-based challenge for future research in Jira enterprise data.

Method: Model LLM reasoning as a cognitive journey over time, apply FFT to map hidden layer temporal signals to frequency domain, construct spectral features to capture reasoning anomalies, and design detection algorithm based on these features.

Result: The method effectively captures anomalies during reasoning process and demonstrates higher detection accuracy and robustness compared to existing approaches.

Conclusion: HSAD overcomes limitations of existing methods by combining reasoning process modeling with frequency-domain feature extraction, enabling better hallucination detection in LLMs.

Abstract: Although Large Language Models have demonstrated powerful capabilities in a wide range of tasks such as language understanding and code generation, the frequent occurrence of hallucinations during the generation process has become a significant impediment to their deployment in critical application scenarios. Current mainstream hallucination detection methods rely on factual consistency verification or static hidden layer features. The former is constrained by the scope of knowledge coverage, while the latter struggles to capture reasoning biases during the inference process. To address these issues, and inspired by signal analysis methods in cognitive neuroscience, this paper proposes a hallucination detection method based on the frequency-domain analysis of hidden layer temporal signals, named HSAD (\textbf{H}idden \textbf{S}ignal \textbf{A}nalysis-based \textbf{D}etection). First, by treating the LLM’s reasoning process as a cognitive journey that unfolds over time, we propose modeling and simulating the human process of signal perception and discrimination in a deception-detection scenario through hidden layer temporal signals. Next, The Fast Fourier Transform is applied to map these temporal signals into the frequency domain to construct spectral features, which are used to capture anomalies that arise during the reasoning process; analysis experiments on these spectral features have proven the effectiveness of this approach. Finally, a hallucination detection algorithm is designed based on these spectral features to identify hallucinations in the generated content. By effectively combining the modeling of the reasoning process with frequency-domain feature extraction, the HSAD method overcomes the limitations of existing approaches in terms of knowledge coverage and the detection of reasoning biases, demonstrating higher detection accuracy and robustness.

Method: Timber partially reverts Instruct models towards their paired Base models by making subtle, targeted refinements to the weight deltas between Base and Instruct models, without requiring additional training.

Result: Extensive experiments on Llama and Qwen series show Timber consistently improves vanilla Instruct models, particularly enhancing Pass@k performance.

Conclusion: The findings provide new insights into post-training at the weight level and offer practical strategies to refine Instruct models without additional training, addressing the exploration-exploitation trade-off.

Abstract: Post-training, which elicits a pretrained Base model into the corresponding Instruct model, is widely considered to be superficial. In this work, we first reinforce this hypothesis by providing novel quantitative evidence from the weight level that the effective rank (eRank) remains negligibly changed. However, this superficiality also suffers a critical trade-off, improving the exploitation capabilities at the cost of limiting its exploration. To tackle this issue, we propose Timber, a simple yet effective training-free method that enhances the exploration capability of the Instruct model while preserving its exploitation. The key insight is to partially revert Instruct towards the paired Base model by subtle yet targeted refinement of the weight deltas. Extensive experiments on Llama and Qwen series demonstrate that Timber consistently improves vanilla Instruct models, particularly on Pass@k performance. Our findings offer new insights into the post-training stage at the weight level and practical strategies to refine the Instruct model without training.

Method: Learn a codebook of discrete strategy priors using concise CoT sketches during training, then use continuous thinking vectors from the codebook at inference with GainRouter for adaptive switching between fast and slow reasoning.

Result: Achieves competitive or superior accuracy across multiple reasoning benchmarks while substantially lowering inference cost.

Conclusion: Offers a practical path toward efficient and controllable reasoning in large language models by reducing unnecessary token generation and suppressing overthinking.

Abstract: Reasoning-oriented Large Language Models (LLMs) often rely on generating explicit tokens step by step, and their effectiveness typically hinges on large-scale supervised fine-tuning or reinforcement learning. While Chain-of-Thought (CoT) techniques substantially enhance performance on complex reasoning tasks, they remain inefficient, requiring long reasoning traces that increase latency and token usage. In this work, we introduce Latent Codebooks for Fast Thinking, a framework that uses concise CoT sketches only during training to learn a codebook of discrete strategy priors. At inference, the model conditions on a handful of continuous thinking vectors distilled from the codebook in a single pass, enabling strategy-level guidance without producing explicit reasoning tokens. To complement this design, we propose GainRouter, a lightweight routing mechanism that adaptively switches between fast codebook guided inference and slow explicit reasoning, thereby suppressing overthinking and reducing unnecessary token generation. Experiments across multiple reasoning benchmarks show that our approach achieves competitive or superior accuracy while substantially lowering inference cost, offering a practical path toward efficient and controllable reasoning in large language models.

Method: Proposes remasking mechanism that jointly predicts token distributions and per-token confidence scores. Uses remask-aware training pipeline with supervised fine-tuning to detect/remask incorrect tokens and reinforcement learning to optimize generation trajectories.

Result: Achieves state-of-the-art results among open-source DLMs on multiple datasets.

Conclusion: RemeDi successfully enables more flexible text refinement in diffusion-based text generation through the novel remasking mechanism and confidence-based token revision.

Abstract: Mask-based Diffusion Language Models (DLMs) struggle to revise incorrect tokens: once a token is generated, it typically remains fixed. The key challenge is to identify potential errors in the inputs. In this paper, we propose \emph{\underline{Rem}asking-\underline{e}nabled \underline{Di}ffusion Language Model (RemeDi}, a mask-based DLM that introduces \emph{remasking} as another fundamental mechanism, enabling more flexible text refinement in diffusion-based text generation. To achieve this, RemeDi jointly predicts token distributions and per-token confidence scores at each step. The confidence scores determine which tokens to be unmasked after the current step, allowing the model to identify tokens with low quality and remask them. These remasked tokens can be resampled with richer context in subsequent steps. We design a remask-aware pipeline to train this ability, including supervised fine-tuning which teaches the model to detect and remask incorrect tokens in addition to predict mask tokens, and reinforcement learning which optimizes full generation trajectories toward higher rewards. Experiments show that RemeDi achieves the state-of-the-art results among open-source DLMs on multiple datasets.

Method: Systematic investigation using Qwen2.5-3B-Base model on diverse multilingual reasoning benchmarks (math, commonsense, scientific reasoning) with comprehensive mechanistic analyses.

Result: RL achieves higher accuracy and substantially stronger cross-lingual generalization than SFT. RL training on non-English data yields better overall performance and generalization than English data, unlike SFT.

Conclusion: RL enables more robust reasoning strategies, providing crucial guidance for equitable and effective multilingual reasoning.

Abstract: Enhancing the complex reasoning capabilities of Large Language Models (LLMs) attracts widespread attention. While reinforcement learning (RL) has shown superior performance for improving complex reasoning, its impact on cross-lingual generalization compared to Supervised Fine-Tuning (SFT) remains unexplored. We present the first systematic investigation into cross-lingual reasoning generalization of RL and SFT. Using Qwen2.5-3B-Base as our foundation model, we conduct experiments on diverse multilingual reasoning benchmarks, including math reasoning, commonsense reasoning, and scientific reasoning. Our investigation yields two significant findings: (1) Tuning with RL not only achieves higher accuracy but also demonstrates substantially stronger cross-lingual generalization capabilities compared to SFT. (2) RL training on non-English data yields better overall performance and generalization than training on English data, which is not observed with SFT. Furthermore, through comprehensive mechanistic analyses, we explore the underlying factors of RL’s superiority and generalization across languages. Our results provide compelling evidence that RL enables the model with more robust reasoning strategies, offering crucial guidance for more equitable and effective multilingual reasoning.

Method: A practical, batch-wise alignment strategy for fine-tuning LLMs that leverages semantically equivalent multilingual data in each training batch to directly align model outputs across languages.

Result: Improves non-English accuracy by up to 23.9% without compromising English performance, model reasoning, or retrieval quality. Reduces the accuracy drop from 29% to much lower levels.

Conclusion: The proposed method is simple to implement, scalable, and integrates seamlessly with existing LLM training & deployment pipelines, enabling more robust and equitable multilingual AI solutions in industry.

Abstract: Large language models (LLMs) remain unreliable for global enterprise applications due to substantial performance gaps between high-resource and mid/low-resource languages, driven by English-centric pretraining and internal reasoning biases. This inconsistency undermines customer experience and operational reliability in multilingual settings such as customer support, content moderation, and information retrieval. Even with advanced Retrieval-Augmented Generation (RAG) systems, we observe up to an 29% accuracy drop in non-English languages compared to English. We propose a practical, batch-wise alignment strategy for fine-tuning LLMs, leveraging semantically equivalent multilingual data in each training batch to directly align model outputs across languages. This approach improves non-English accuracy by up to 23.9% without compromising English performance, model reasoning, or retrieval quality. Our method is simple to implement, scalable, and integrates seamlessly with existing LLM training & deployment pipelines, enabling more robust and equitable multilingual AI solutions in industry.

Method: Developed TF-Bench based on type inference in System F, with verified transformations to create TF-Bench_pure that removes semantically irrelevant natural language, plus novel metrics for robustness and test-time reasoning effectiveness.

Result: State-of-the-art LLMs show substantial limitations, with Claude-3.7-sonnet achieving only 55.85% accuracy on TF-Bench_pure, revealing critical gaps in genuine program semantics reasoning.

Conclusion: Current LLMs have significant limitations in program semantics reasoning, highlighting the need for improved benchmarks and future research directions in this area.

Abstract: Large Language Models (LLMs) are increasingly integrated into the software engineering ecosystem. Their test-time compute (TTC) reasoning capabilities show significant potential for understanding program logic and semantics beyond mere token recognition. However, current benchmarks for code reasoning lack a formal, program-centric deductive framework to ensure sound evaluation, and are incapable of assessing whether models genuinely reason about program semantics or merely exploit superficial associations between natural language and code tokens. To bridge this gap, we introduce TF-Bench, a benchmark designed to evaluate LLM reasoning based on type inference in System F, a task we refer to as program semantics reasoning. By employing verified transformations to remove semantically irrelevant natural language, we construct TF-Bench_pure, a purely semantics-driven variant of TF-Bench. Our analysis reveals substantial limitations in state-of-the-art LLMs, with the best-performing LLM (Claude-3.7-sonnet) achieving only 55.85% accuracy on TF-Bench_pure. Additionally, we propose two novel metrics to assess robustness and the effectiveness of test-time reasoning, underscoring critical limitations in current LLM capabilities and highlighting essential directions for future research.

Method: Developed VIVA+ benchmark with three core dimensions: Foundational Situation Comprehension, Context-Driven Action Justification, and Reflective Reasoning. Evaluated latest commercial and open-source MLLMs, and explored targeted training and multi-step reasoning strategies.

Result: Revealed distinct performance patterns and significant challenges in current MLLMs. Targeted training and multi-step reasoning strategies yielded consistent performance improvements across models.

Conclusion: Current MLLMs have limitations in robust, context-aware, and socially adept decision-making. The analysis provides actionable insights for advancing MLLMs toward better real-world performance in human-centered environments.

Abstract: Multimodal Large Language Models (MLLMs) show promising results for embodied agents in operating meaningfully in complex, human-centered environments. Yet, evaluating their capacity for nuanced, human-like reasoning and decision-making remains challenging. In this work, we introduce VIVA+, a cognitively grounded benchmark for evaluating the reasoning and decision-making of MLLMs in human-centered situations. VIVA+ consists of 1,317 real-world situations paired with 6,373 multiple-choice questions, targeting three core abilities for decision-making: (1) Foundational Situation Comprehension, (2) Context-Driven Action Justification, and (3) Reflective Reasoning. Together, these dimensions provide a systematic framework for assessing a model’s ability to perceive, reason, and act in socially meaningful ways. We evaluate the latest commercial and open-source models on VIVA+, where we reveal distinct performance patterns and highlight significant challenges. We further explore targeted training and multi-step reasoning strategies, which yield consistent performance improvements. Finally, our in-depth analysis highlights current model limitations and provides actionable insights for advancing MLLMs toward more robust, context-aware, and socially adept decision-making in real-world settings.

Method: Proposes M-Hyper using quaternion algebra with four orthogonal bases to represent independent modalities. Uses Hamilton product for modality interactions, with FERF module for entity representation factorization and R2MF module for relation-aware modality fusion.

Result: Extensive experiments show state-of-the-art performance, robustness, and computational efficiency compared to existing methods.

Conclusion: M-Hyper successfully integrates strengths of both fusion and ensemble paradigms, enabling effective cross-modal interactions while preserving modality-specific information through quaternion-based representation.

Abstract: Multi-modal knowledge graph completion (MMKGC) aims to discover missing facts in multi-modal knowledge graphs (MMKGs) by leveraging both structural relationships and diverse modality information of entities. Existing MMKGC methods follow two multi-modal paradigms: fusion-based and ensemble-based. Fusion-based methods employ fixed fusion strategies, which inevitably leads to the loss of modality-specific information and a lack of flexibility to adapt to varying modality relevance across contexts. In contrast, ensemble-based methods retain modality independence through dedicated sub-models but struggle to capture the nuanced, context-dependent semantic interplay between modalities. To overcome these dual limitations, we propose a novel MMKGC method M-Hyper, which achieves the coexistence and collaboration of fused and independent modality representations. Our method integrates the strengths of both paradigms, enabling effective cross-modal interactions while maintaining modality-specific information. Inspired by ``quaternion’’ algebra, we utilize its four orthogonal bases to represent multiple independent modalities and employ the Hamilton product to efficiently model pair-wise interactions among them. Specifically, we introduce a Fine-grained Entity Representation Factorization (FERF) module and a Robust Relation-aware Modality Fusion (R2MF) module to obtain robust representations for three independent modalities and one fused modality. The resulting four modality representations are then mapped to the four orthogonal bases of a biquaternion (a hypercomplex extension of quaternion) for comprehensive modality interaction. Extensive experiments indicate its state-of-the-art performance, robustness, and computational efficiency.

Method: Created a 1,208-question dataset (387 multimodal), compared text-only and multimodal SOTA systems, fine-tuned Llama 3.1-8B-Instruct and RoLlama 3.1-8B-Instruct, and used LLM-as-a-Judge to assess explanation quality.

Result: SOTA models perform well but fine-tuned 8B models are competitive. Textual descriptions of images outperform direct visual input. LLM-as-a-Judge assessment reveals self-preference bias in explanation evaluation.

Conclusion: The study provides insights for developing explainable QA systems for less-resourced languages like Romanian, showing the effectiveness of fine-tuned smaller models and the limitations of visual input compared to textual descriptions.

Abstract: Ensuring that both new and experienced drivers master current traffic rules is critical to road safety. This paper evaluates Large Language Models (LLMs) on Romanian driving-law QA with explanation generation. We release a 1{,}208-question dataset (387 multimodal) and compare text-only and multimodal SOTA systems, then measure the impact of domain-specific fine-tuning for Llama 3.1-8B-Instruct and RoLlama 3.1-8B-Instruct. SOTA models perform well, but fine-tuned 8B models are competitive. Textual descriptions of images outperform direct visual input. Finally, an LLM-as-a-Judge assesses explanation quality, revealing self-preference bias. The study informs explainable QA for less-resourced languages.

Method: Developed a logic-grounded evaluation framework categorizing multimodal reasoning into six interaction patterns, analyzed attention patterns, and tested two-step prompting (recognize then reason) and early fusion modifications.

Result: Additional modalities only enhance reasoning when providing independent and sufficient reasoning paths. Performance degrades due to three systematic failures: weaker modalities dragging down performance, modality conflicts creating bias, and ineffective integration of joint signals.

Conclusion: Integration, not perception, is the main barrier to multimodal reasoning. Composition-aware training and early fusion control are promising directions to address task-composition and fusion bottlenecks.

Abstract: Multimodal large language models (MLLMs) promise enhanced reasoning by integrating diverse inputs such as text, vision, and audio. Yet cross-modal reasoning remains underexplored, with conflicting reports on whether added modalities help or harm performance. These inconsistencies stem from a lack of controlled evaluation frameworks and analysis of models’ internals to isolate when and why modality interactions support or undermine reasoning. We address this gap through a logic-grounded evaluation framework that categorizes multimodal reasoning into six interaction patterns, varying how facts are distributed across modalities and logically combined. Empirically, additional modalities enhance reasoning only when they provide independent and sufficient reasoning paths, while redundant or chained entailment support often hurts performance. Moreover, reasoning degrades in three systematic ways: weaker modalities drag down overall performance, conflicts bias preference toward certain modalities, and joint signals from different modalities fail to be integrated effectively. Therefore, we identify two core failures: task-composition bottleneck, where recognition and reasoning cannot be jointly executed in one pass, and fusion bottleneck, where early integration introduces bias. For further investigation, we find that attention patterns fail to encode fact usefulness, but a simple two-step prompting (recognize then reason) restores performance, confirming the task-composition bottleneck. Moreover, modality identity remains recoverable in early layers, and softening attention in early fusion improves reasoning, highlighting biased fusion as another failure mode. Overall, our findings show that integration, not perception, is the main barrier to multimodal reasoning, suggesting composition-aware training and early fusion control as promising directions.

Method: Proposed analytical framework using parameter importance estimation to identify textual importance distribution shift. Investigated two mitigation strategies: layer-wise learning rate scheduling and Low-Rank Adaptation (LoRA).

Result: Both approaches better maintain textual competence than full fine-tuning while improving spoken question answering performance.

Conclusion: The analysis provides principled explanation for mitigation strategies’ effectiveness, linking benefits to structural properties of textual knowledge in LLMs.

Abstract: The integration of speech into Large Language Models (LLMs) has substantially expanded their capabilities, but often at the cost of weakening their core textual competence. This degradation limits the ability of speech-enabled LLMs to fully exploit their pre-trained text-based knowledge. In this work, we analyze the underlying mechanisms of this issue through a focused study of the widely used encoder-adaptor paradigm. We propose an analytical framework based on parameter importance estimation, which reveals that fine-tuning for speech introduces a textual importance distribution shift: the layer-wise allocation of parameters critical to textual reasoning is disrupted. Building on this insight, we investigate two mitigation strategies: layer-wise learning rate scheduling and Low-Rank Adaptation (LoRA), both aim to preserve the original parameter distribution. Experimental results show that both approaches better maintain textual competence than full fine-tuning, while also improving downstream spoken question answering performance. Furthermore, our analysis offers a principled explanation for the effectiveness of the proposed mitigation strategies, linking their benefits to the structural properties of textual knowledge in LLMs.

Method: Uses knowledge-level consistency reinforcement learning with dual-fact alignment: constructs fact checklist from pretrained knowledge boundaries for factual recall, and trains self-assessment module for factual precision. Fully external-knowledge-free and lightweight reward design.

Result: Substantially improves factuality metrics across multiple long-form benchmarks and effectively alleviates model hallucinations.

Conclusion: KLCF provides an efficient and scalable solution to improve LLM factuality without relying on external retrieval or heavy verification systems.

Abstract: Hallucination and factuality deficits remain key obstacles to the reliability of large language models (LLMs) in long-form generation. Existing reinforcement learning from human feedback (RLHF) frameworks primarily rely on preference rewards, yet they often overlook the model’s internal knowledge boundaries, exacerbating the so-called “hallucination tax”. To address this challenge, we propose Knowledge-Level Consistency Reinforcement Learning Framework (KLCF), a novel framework that focuses on the knowledge consistency between the policy model’s expressed knowledge and the base model’s parametric knowledge, and introduces a Dual-Fact Alignment mechanism to jointly optimize factual recall and precision. Specifically, KLCF leverages pretrained knowledge boundaries to construct fact checklist, guiding online reinforcement learning to improve factual coverage and recall; simultaneously, it trains a self-assessment module based on the base model’s internal knowledge to enhance factual precision during generation. Unlike prior methods that rely on external retrieval or heavy verification, our reward design is fully external-knowledge-free and lightweight, making KLCF efficient and easily scalable to large-scale training. Experimental results demonstrate that KLCF substantially improves factuality metrics across multiple long-form benchmarks and effectively alleviates model hallucinations.

Method: Proposed LoGo framework with local-global memory: local memory for individual user preferences and global memory for shared interests across population, plus a mediator module to reconcile conflicts between local and global signals.

Result: Extensive experiments on multiple benchmarks show LoGo consistently improves personalization quality by warming up cold-start users and mitigating biased predictions.

Conclusion: Incorporating collective knowledge through the LoGo framework effectively enhances LLM personalization by addressing both cold-start and biasing problems.

Abstract: Large language model (LLM) personalization aims to tailor model behavior to individual users based on their historical interactions. However, its effectiveness is often hindered by two key challenges: the \textit{cold-start problem}, where users with limited history provide insufficient context for accurate personalization, and the \textit{biasing problem}, where users with abundant but skewed history cause the model to overfit to narrow preferences. We identify both issues as symptoms of a common underlying limitation, i.e., the inability to model collective knowledge across users. To address this, we propose a local-global memory framework (LoGo) that combines the personalized local memory with a collective global memory that captures shared interests across the population. To reconcile discrepancies between these two memory sources, we introduce a mediator module designed to resolve conflicts between local and global signals. Extensive experiments on multiple benchmarks demonstrate that LoGo consistently improves personalization quality by both warming up cold-start users and mitigating biased predictions. These results highlight the importance of incorporating collective knowledge to enhance LLM personalization.

Method: KAPPA (Knowledge-Aligned Prediction through Projection-based Adjustment) - a parameter-free intervention that transforms hidden states to align prediction coordinates with knowledge coordinates in a subspace spanned by knowledge and prediction bases.

Result: KAPPA substantially improves accuracy on binary-choice reformulations of Big-Bench-Hard and ARC-Challenge, consistently outperforming baselines. It also extends effectiveness to free-form questions beyond MCQs.

Conclusion: The work provides a geometric understanding of the knowledge-prediction gap and offers a practical method for better aligning model behavior with its latent knowledge.

Abstract: Large Language Models (LLMs) often fail on multiple-choice questions (MCQs) despite demonstrating correct knowledge in other contexts, such as free-form generation. To investigate the mechanism underlying this knowledge-prediction gap on MCQs and alleviate it, we conduct a probing analysis and find that residual streams in certain layers contain a subspace spanned by two important bases: a \emph{knowledge basis} that encodes the probability of the ground-truth answer for a given MCQ and a \emph{prediction basis} that encodes the probability of the answer choice predicted by the model. We observe that incorrect predictions arise from a misalignment of the model’s hidden states along these two bases. Hence, we introduce \textbf{KAPPA} (Knowledge-Aligned Prediction through Projection-based Adjustment), a parameter-free intervention that transforms the hidden states to align the prediction coordinate with the knowledge coordinate within this subspace. Experiments on binary-choice reformulations of Big-Bench-Hard and ARC-Challenge show that KAPPA substantially improves accuracy and consistently outperforms baselines. While optimal subspaces differ across tasks, subspaces generalize to some extent, as supported by cross-dataset experiments. Moreover, KAPPA extends its effectiveness to free-form questions beyond MCQs. Our work provides a new geometric understanding of the knowledge-prediction gap and offers a practical method for better aligning model behavior with its latent knowledge.

Method: Three-stage pipeline: BM25 for initial retrieval, dense embedding model for semantic matching, and cross-encoder reranking for precise content retrieval, tested with Fanar and Mistral LLMs.

Result: RAG pipeline improved performance across both subtasks, with Fanar achieving best results: 45% accuracy in Subtask 1 and 80% in Subtask 2, with improvements up to 25%.

Conclusion: The hybrid RAG approach effectively enhances LLM performance on Islamic knowledge tasks, with Fanar showing superior results in the proposed pipeline.

Abstract: This paper presents our submission to the QIAS 2025 shared task on Islamic knowledge understanding and reasoning. We developed a hybrid retrieval-augmented generation (RAG) system that combines sparse and dense retrieval methods with cross-encoder reranking to improve large language model (LLM) performance. Our three-stage pipeline incorporates BM25 for initial retrieval, a dense embedding retrieval model for semantic matching, and cross-encoder reranking for precise content retrieval. We evaluate our approach on both subtasks using two LLMs, Fanar and Mistral, demonstrating that the proposed RAG pipeline enhances performance across both, with accuracy improvements up to 25%, depending on the task and model configuration. Our best configuration is achieved with Fanar, yielding accuracy scores of 45% in Subtask 1 and 80% in Subtask 2.

Method: Proposes Open-DeBias, a data-efficient and parameter-efficient debiasing method using adapter modules to mitigate social and stereotypical biases while generalizing to unseen biases.

Result: Improves QA accuracy by 48% on ambiguous subsets and 6% on disambiguated ones compared to BMBI, and achieves 84% accuracy in zero-shot transfer to Korean BBQ, demonstrating robust multilingual generalization.

Conclusion: Open-DeBias is effective for general-purpose, open-domain bias mitigation across various NLP tasks, showing robustness, multilingual strength, and suitability for addressing both known and emergent biases.

Abstract: Large Language Models (LLMs) have achieved remarkable success on question answering (QA) tasks, yet they often encode harmful biases that compromise fairness and trustworthiness. Most existing bias mitigation approaches are restricted to predefined categories, limiting their ability to address novel or context-specific emergent biases. To bridge this gap, we tackle the novel problem of open-set bias detection and mitigation in text-based QA. We introduce OpenBiasBench, a comprehensive benchmark designed to evaluate biases across a wide range of categories and subgroups, encompassing both known and previously unseen biases. Additionally, we propose Open-DeBias, a novel, data-efficient, and parameter-efficient debiasing method that leverages adapter modules to mitigate existing social and stereotypical biases while generalizing to unseen ones. Compared to the state-of-the-art BMBI method, Open-DeBias improves QA accuracy on BBQ dataset by nearly $48%$ on ambiguous subsets and $6%$ on disambiguated ones, using adapters fine-tuned on just a small fraction of the training data. Remarkably, the same adapters, in a zero-shot transfer to Korean BBQ, achieve $84%$ accuracy, demonstrating robust language-agnostic generalization. Through extensive evaluation, we also validate the effectiveness of Open-DeBias across a broad range of NLP tasks, including StereoSet and CrowS-Pairs, highlighting its robustness, multilingual strength, and suitability for general-purpose, open-domain bias mitigation. The project page is available at: https://sites.google.com/view/open-debias25

Method: SPELL integrates three cyclical roles (questioner, responder, verifier) within a single model for continual self-improvement. It uses automated curriculum learning with gradually increasing document length and adaptive reward functions based on question difficulty.

Result: Extensive experiments on six long-context benchmarks show SPELL consistently improves performance across diverse LLMs, outperforming equally sized models fine-tuned on large-scale annotated data. It achieves an average 7.6-point gain in pass@8 on Qwen3-30B-A3B-Thinking.

Conclusion: SPELL shows promise for scaling to even more capable models and effectively addresses the challenge of long-context reasoning optimization without requiring human annotations.

Abstract: Progress in long-context reasoning for large language models (LLMs) has lagged behind other recent advances. This gap arises not only from the intrinsic difficulty of processing long texts, but also from the scarcity of reliable human annotations and programmatically verifiable reward signals. In this paper, we propose SPELL, a multi-role self-play reinforcement learning framework that enables scalable, label-free optimization for long-context reasoning. SPELL integrates three cyclical roles-questioner, responder, and verifier-within a single model to enable continual self-improvement. The questioner generates questions from raw documents paired with reference answers; the responder learns to solve these questions based on the documents; and the verifier evaluates semantic equivalence between the responder’s output and the questioner’s reference answer, producing reward signals to guide continual training. To stabilize training, we introduce an automated curriculum that gradually increases document length and a reward function that adapts question difficulty to the model’s evolving capabilities. Extensive experiments on six long-context benchmarks show that SPELL consistently improves performance across diverse LLMs and outperforms equally sized models fine-tuned on large-scale annotated data. Notably, SPELL achieves an average 7.6-point gain in pass@8 on the strong reasoning model Qwen3-30B-A3B-Thinking, raising its performance ceiling and showing promise for scaling to even more capable models.

Method: Proposes Error-Uncertainty Plane diagnostic framework and Quadrant-based Tuning with two-stage strategy: sample-level triage to retain informative examples, followed by asymmetric token-pruning that trims less salient tokens only from misconception samples while preserving calibration samples entirely.

Result: Sets new state-of-the-art across five diverse benchmarks. On SmolLM2-1.7B, achieves +38% average improvement over full-data SFT baseline using only 12.5% of original training data.

Conclusion: Q-Tuning is the first dynamic pruning approach to consistently outperform full-data training, providing a practical and scalable blueprint for maximizing data utilization in budget-constrained LLM SFT.

Abstract: As supervised fine-tuning (SFT) evolves from a lightweight post-training step into a compute-intensive phase rivaling mid-training in scale, data efficiency has become critical for aligning large language models (LLMs) under tight budgets. Existing data pruning methods suffer from a fragmented design: they operate either at the sample level or the token level in isolation, failing to jointly optimize both dimensions. This disconnect leads to significant inefficiencies–high-value samples may still contain redundant tokens, while token-level pruning often discards crucial instructional or corrective signals embedded in individual examples. To address this bottleneck, we introduce the Error-Uncertainty (EU) Plane, a diagnostic framework that jointly characterizes the heterogeneous utility of training data across samples and tokens. Guided by this insight, we propose Quadrant-based Tuning (Q-Tuning), a unified framework that strategically coordinates sample pruning and token pruning. Q-Tuning employs a two-stage strategy: first, it performs sample-level triage to retain examples rich in informative misconceptions or calibration signals; second, it applies an asymmetric token-pruning policy, using a context-aware scoring mechanism to trim less salient tokens exclusively from misconception samples while preserving calibration samples in their entirety. Our method sets a new state of the art across five diverse benchmarks. Remarkably, on SmolLM2-1.7B, Q-Tuning achieves a +38% average improvement over the full-data SFT baseline using only 12.5% of the original training data. As the first dynamic pruning approach to consistently outperform full-data training, Q-Tuning provides a practical and scalable blueprint for maximizing data utilization in budget-constrained LLM SFT.

Method: DocPruner employs adaptive patch-level embedding pruning by leveraging intra-document patch attention distribution to dynamically identify and discard redundant embeddings for each document.

Result: Achieves 50-60% reduction in storage for leading multi-vector VDR models with negligible degradation in document retrieval performance across more than ten datasets.

Conclusion: DocPruner offers a robust, flexible, and effective solution for building storage-efficient, large-scale VDR systems.

Abstract: Visual Document Retrieval (VDR), the task of retrieving visually-rich document pages using queries that combine visual and textual cues, is crucial for numerous real-world applications. Recent state-of-the-art methods leverage Large Vision-Language Models (LVLMs) in a multi-vector paradigm, representing each document as patch-level embeddings to capture fine-grained details. While highly effective, this approach introduces a critical challenge: prohibitive storage overhead, as storing hundreds of vectors per page makes large-scale deployment costly and impractical. To address this, we introduce DocPruner, the first framework to employ adaptive patch-level embedding pruning for VDR to effectively reduce the storage overhead. DocPruner leverages the intra-document patch attention distribution to dynamically identify and discard redundant embeddings for each document. This adaptive mechanism enables a significant 50-60% reduction in storage for leading multi-vector VDR models with negligible degradation in document retrieval performance. Extensive experiments across more than ten representative datasets validate that DocPruner offers a robust, flexible, and effective solution for building storage-efficient, large-scale VDR systems.

Method: Proposes EOSER (EOS Early Rejection) and ASS (Ascending Step-Size) decoding scheduler for full diffusion-style decoding, and CJ-GRPO (Consistency Trajectory Group Relative Policy Optimization) RL method that ensures consistency between rollout and optimization trajectories.

Result: Experiments on reasoning tasks using LLaDA-8B-Instruct show the proposed methods achieve competitive performance with fewer decoding steps and effectively tame MDLMs.

Conclusion: The EOSER and ASS mechanisms combined with CJ-GRPO provide promising solutions for efficiently optimizing masked diffusion language models, addressing key challenges in decoding strategies and reinforcement learning for non-causal parallel decoding models.

Abstract: Masked diffusion language models (MDLMs) have recently emerged as a promising alternative to autoregressive (AR) language models, offering properties such as parallel decoding, flexible generation orders, and the potential for fewer inference steps. Despite these advantages, decoding strategies and reinforcement learning (RL) algorithms tailored for MDLMs remain underexplored. A naive approach is to directly transfer techniques well-established for AR models to MDLMs. However, this raises an immediate question: Is such a naive transfer truly optimal? For example, 1) Block-wise and semi-AR decoding strategies are not employed during the training of MDLMs, so why do they outperform full diffusion-style decoding during inference? 2) Applying RL algorithms designed for AR models directly to MDLMs exhibits a training-inference inconsistency, since MDLM decoding are non-causal (parallel). This results in inconsistencies between the rollout trajectory and the optimization trajectory. To address these challenges, we propose EOS Early Rejection (EOSER) and Ascending Step-Size (ASS) decoding scheduler, which unlock the potential of MDLMs to perform full diffusion-style decoding, achieving competitive performance with fewer decoding steps. Additionally, we introduce Consistency Trajectory Group Relative Policy Optimization (CJ-GRPO) for taming MDLMs, which emphasizes the consistency between rollout trajectory and optimization trajectory, and reduces the optimization errors caused by skip-step optimization. We conduct extensive experiments on reasoning tasks, such as mathematical and planning benchmarks, using LLaDA-8B-Instruct. The results demonstrate that the proposed EOSER and ASS mechanisms, together with CJ-GRPO, hold significant promise for effectively and efficiently taming MDLMs. Code: https://github.com/yjyddq/EOSER-ASS-RL.

Method: Introduced EVOLVECAST framework to assess LLM prediction revisions when presented with post-training cutoff information, using human forecasters as reference for comparison.

Result: LLMs show some responsiveness to new information but updates are inconsistent and overly conservative. Neither verbalized nor logits-based confidence estimates consistently outperform each other, both falling short of human standards.

Conclusion: Models exhibit conservative bias across settings, highlighting the need for more robust approaches to belief updating in LLMs.

Abstract: Prior work has largely treated future event prediction as a static task, failing to consider how forecasts and the confidence in them should evolve as new evidence emerges. To address this gap, we introduce EVOLVECAST, a framework for evaluating whether large language models appropriately revise their predictions in response to new information. In particular, EVOLVECAST assesses whether LLMs adjust their forecasts when presented with information released after their training cutoff. We use human forecasters as a comparative reference to analyze prediction shifts and confidence calibration under updated contexts. While LLMs demonstrate some responsiveness to new information, their updates are often inconsistent or overly conservative. We further find that neither verbalized nor logits-based confidence estimates consistently outperform the other, and both remain far from the human reference standard. Across settings, models tend to express conservative bias, underscoring the need for more robust approaches to belief updating.

Method: Proposed Easy Turn, an open-source modular turn-taking detection model that integrates acoustic and linguistic bimodal information to predict four dialogue turn states: complete, incomplete, backchannel, and wait. Also released Easy Turn trainset, a 1,145-hour speech dataset for training turn-taking detection models.

Result: Achieved state-of-the-art turn-taking detection accuracy on the open-source Easy Turn testset compared to existing open-source models like TEN Turn Detection and Smart Turn V2.

Conclusion: Easy Turn provides an effective open-source solution for turn-taking detection by integrating bimodal information and releasing a large training dataset, addressing the limitations of existing approaches.

Abstract: Full-duplex interaction is crucial for natural human-machine communication, yet remains challenging as it requires robust turn-taking detection to decide when the system should speak, listen, or remain silent. Existing solutions either rely on dedicated turn-taking models, most of which are not open-sourced. The few available ones are limited by their large parameter size or by supporting only a single modality, such as acoustic or linguistic. Alternatively, some approaches finetune LLM backbones to enable full-duplex capability, but this requires large amounts of full-duplex data, which remain scarce in open-source form. To address these issues, we propose Easy Turn, an open-source, modular turn-taking detection model that integrates acoustic and linguistic bimodal information to predict four dialogue turn states: complete, incomplete, backchannel, and wait, accompanied by the release of Easy Turn trainset, a 1,145-hour speech dataset designed for training turn-taking detection models. Compared to existing open-source models like TEN Turn Detection and Smart Turn V2, our model achieves state-of-the-art turn-taking detection accuracy on our open-source Easy Turn testset. The data and model will be made publicly available on GitHub.

Method: Developed a prototype system that integrates a vision-language model to process both speech and visual input from a webcam, using visual context to prime the translation process. Evaluated using a hand-crafted diagnostic corpus targeting three types of ambiguity.

Result: Visual grounding substantially improves lexical disambiguation, yields modest and less stable gains for gender resolution, and shows no benefit for syntactic ambiguities.

Conclusion: Embracing multimodality represents a necessary step forward for advancing translation quality in machine interpreting systems.

Abstract: Machine Interpreting systems are currently implemented as unimodal, real-time speech-to-speech architectures, processing translation exclusively on the basis of the linguistic signal. Such reliance on a single modality, however, constrains performance in contexts where disambiguation and adequacy depend on additional cues, such as visual, situational, or pragmatic information. This paper introduces Vision-Grounded Interpreting (VGI), a novel approach designed to address the limitations of unimodal machine interpreting. We present a prototype system that integrates a vision-language model to process both speech and visual input from a webcam, with the aim of priming the translation process through contextual visual information. To evaluate the effectiveness of this approach, we constructed a hand-crafted diagnostic corpus targeting three types of ambiguity. In our evaluation, visual grounding substantially improves lexical disambiguation, yields modest and less stable gains for gender resolution, and shows no benefit for syntactic ambiguities. We argue that embracing multimodality represents a necessary step forward for advancing translation quality in machine interpreting.

Method: HiPO combines a hybrid data pipeline with paired Think-on/Think-off responses and a hybrid reinforcement learning reward system that balances accuracy and efficiency.

Result: Experiments across mathematics and coding benchmarks show HiPO substantially reduces token length while maintaining or improving accuracy.

Conclusion: HiPO provides a principled approach for efficient adaptive reasoning, advancing deployment of reasoning-oriented LLMs in resource-sensitive real-world settings.

Abstract: Large Language Models (LLMs) increasingly rely on chain-of-thought (CoT) reasoning to improve accuracy on complex tasks. However, always generating lengthy reasoning traces is inefficient, leading to excessive token usage and higher inference costs. This paper introduces the Hybrid Policy Optimization (i.e., HiPO), a framework for adaptive reasoning control that enables LLMs to selectively decide when to engage in detailed reasoning (Think-on) and when to respond directly (Think-off). Specifically, HiPO combines a hybrid data pipelineproviding paired Think-on and Think-off responseswith a hybrid reinforcement learning reward system that balances accuracy and efficiency while avoiding over-reliance on detailed reasoning. Experiments across mathematics and coding benchmarks demonstrate that HiPO can substantially reduce token length while maintaining or improving accuracy. Finally, we hope HiPO a can be a principled approach for efficient adaptive reasoning, advancing the deployment of reasoning-oriented LLMs in real-world, resource-sensitive settings.

Method: Refactored the original ByteSized32 corpus by creating GameBasic.py foundation library with 7 base classes (GameObject, etc.) that centralize common logic across all 32 games.

Result: Reduced total lines of Python code from 20k to 10k. GPT-4o experiments showed mixed performance - quality improvements on 2 evaluation dimensions but decreases on 2 others due to hierarchical structure challenges.

Conclusion: The extensible code structure with foundation library and modular optimization facilitates LLM adaptation and establishes a scalable environment for future extensions.

Abstract: Simulating interactive world models remains a core challenge in Large Language Models(LLMs). In this work, we introduce the ByteSized32Refactored, a refactored, modular, and extensible implementation of the original ByteSized32 corpus to explore the task of text game generation. We further optimize the code structure of each text game and create the GameBasic.py foundation library, which centralizes common logic across all 32 games by abstracting 7 base classes (GameObject, etc.) into reusable modules, thereby reducing from 20k to 10k total lines of Python code compared to the original Bytesized32. Our refactored implementation enables extendability - with our centralized design, ByteSized32Refactored can be more efficiently extended to include text games of new scenarios and specifications by reusing the shared logic and functionalities. Extensive experiments with GPT-4o demonstrate a mix of performance - with Bytesized32Refactored, the generated text games for unseen scenarios showcase quality improvements on two of the four evaluation dimensions while decreases on the other two, indicating that the hierarchical structure of the refactored code presents new challenges for LLMs. Overall, we highlight that our extensible code structure, centered on the foundation library and the modular optimization, not only facilitates LLM adaptation to environment specifications but also establishes a scalable environment that supports future extensions.

Method: Extracts lightweight steering vectors from residual streams of LLMs using as few as 100 preference pairs, which can be applied at inference time to push models toward preferred behaviors without training.

Result: PaLRS-aligned models achieve consistent gains on mathematical reasoning and code generation benchmarks while preserving baseline general-purpose performance, and outperform DPO-aligned models with huge time savings.

Conclusion: PaLRS offers an effective, efficient, and flexible alternative to standard preference optimization pipelines, providing training-free, plug-and-play alignment with minimal data requirements.

Abstract: Preference alignment is a critical step in making Large Language Models (LLMs) useful and aligned with (human) preferences. Existing approaches such as Reinforcement Learning from Human Feedback or Direct Preference Optimization typically require curated data and expensive optimization over billions of parameters, and eventually lead to persistent task-specific models. In this work, we introduce Preference alignment of Large Language Models via Residual Steering (PaLRS), a training-free method that exploits preference signals encoded in the residual streams of LLMs. From as few as one hundred preference pairs, PaLRS extracts lightweight, plug-and-play steering vectors that can be applied at inference time to push models toward preferred behaviors. We evaluate PaLRS on various small-to-medium-scale open-source LLMs, showing that PaLRS-aligned models achieve consistent gains on mathematical reasoning and code generation benchmarks while preserving baseline general-purpose performance. Moreover, when compared to DPO-aligned models, they perform better with huge time savings. Our findings highlight that PaLRS offers an effective, much more efficient and flexible alternative to standard preference optimization pipelines, offering a training-free, plug-and-play mechanism for alignment with minimal data.

Method: Compared full-scale, distilled, and quantized models on machine translation using Flores+ benchmark and human evaluations of conversational translations in French, Hindi, and Kannada. Analyzed carbon emissions per evaluation run.

Result: Full 3.3B fp32 model had highest BLEU scores but largest environmental footprint (0.007-0.008 kg CO2 per run). Distilled models achieved 4.5x faster inference with minimal BLEU reductions. Aggressive quantization (INT4) preserved high accuracy and fluency with minor differences between models.

Conclusion: Model compression can substantially reduce computational demands and environmental impact while maintaining competitive quality, though trade-offs are more pronounced in low-resource settings. Evaluation frameworks should integrate efficiency and sustainability as central dimensions of NLP progress.

Abstract: The rapid expansion of large language models (LLMs) has heightened concerns about their computational and environmental costs. This study investigates the trade-offs between translation quality and efficiency by comparing full-scale, distilled, and quantized models using machine translation as a case study. We evaluated performance on the Flores+ benchmark and through human judgments of conversational translations in French, Hindi, and Kannada. Our analysis of carbon emissions per evaluation run revealed that the full 3.3B fp32 model, while achieving the highest BLEU scores, incurred the largest environmental footprint (about 0.007-0.008 kg CO2 per run). The distilled models achieved an inference of up to 4.5x faster than the full 3.3B model, with only minimal reductions in BLEU scores. Human evaluations also showed that even aggressive quantization (INT4) preserved high levels of accuracy and fluency, with differences between models generally minor. These findings demonstrate that model compression strategies can substantially reduce computational demands and environmental impact while maintaining competitive translation quality, though trade-offs are more pronounced in low-resource settings. We argue for evaluation frameworks that integrate efficiency and sustainability alongside objective metrics as central dimensions of progress in NLP.

Method: Introduces “Policy as Prompt” approach using LLMs to interpret natural language policies with contextual understanding and least privilege principle. System ingests technical artifacts to build verifiable policy trees, then compiles them into lightweight prompt-based classifiers for runtime behavior auditing.

Result: Validated across diverse applications, demonstrating a scalable and auditable pipeline that successfully bridges the policy-to-practice gap.

Conclusion: The framework paves the way for verifiably safer and more regulatable AI by providing automated translation of design documents into enforceable guardrails through LLM-based policy interpretation.

Abstract: As autonomous AI agents are increasingly deployed in industry, it is essential to safeguard them. We introduce a novel framework that automates the translation of unstructured design documents into verifiable, real-time guardrails. We introduce “Policy as Prompt,” a new approach that uses Large Language Models (LLMs) to interpret and enforce natural language policies by applying contextual understanding and the principle of least privilege. Our system first ingests technical artifacts to construct a verifiable policy tree, which is then compiled into lightweight, prompt-based classifiers that audit agent behavior at runtime. We validate our approach across diverse applications, demonstrating a scalable and auditable pipeline that bridges the critical policy-to-practice gap, paving the way for verifiably safer and more regulatable AI.

Method: Created 127 high-quality tasks with domain experts and AI agents, featuring curated initial states and programmatic verification scripts. Evaluated LLMs using minimal agent framework with tool-calling loops.

Result: Best model (gpt-5-medium) achieved only 52.56% pass@1 and 33.86% pass^4. Other strong models (claude-sonnet-4, o3) fell below 30% pass@1 and 15% pass^4. LLMs required average 16.2 execution turns and 17.4 tool calls per task.

Conclusion: MCPMark effectively stress-tests MCP capabilities, revealing significant performance gaps in current LLMs for complex real-world agent workflows requiring diverse CRUD operations.

Abstract: MCP standardizes how LLMs interact with external systems, forming the foundation for general agents. However, existing MCP benchmarks remain narrow in scope: they focus on read-heavy tasks or tasks with limited interaction depth, and fail to capture the complexity and realism of real-world workflows. To address this gap, we propose MCPMark, a benchmark designed to evaluate MCP use in a more realistic and comprehensive manner. It consists of $127$ high-quality tasks collaboratively created by domain experts and AI agents. Each task begins with a curated initial state and includes a programmatic script for automatic verification. These tasks demand richer and more diverse interactions with the environment, involving a broad range of create, read, update, and delete (CRUD) operations. We conduct a comprehensive evaluation of cutting-edge LLMs using a minimal agent framework that operates in a tool-calling loop. Empirical results show that the best-performing model, gpt-5-medium, reaches only $52.56$% pass@1 and $33.86$% pass^4, while other widely regarded strong models, including claude-sonnet-4 and o3, fall below $30$% pass@1 and $15$% pass^4. On average, LLMs require $16.2$ execution turns and $17.4$ tool calls per task, significantly surpassing those in previous MCP benchmarks and highlighting the stress-testing nature of MCPMark.

Method: Introduces Next Sequence Prediction (NSP) that adaptively determines generation length per step. SDLM performs diffusion inference in fixed-size mask blocks but dynamically decodes consecutive subsequences based on model confidence, preserving KV-cache compatibility.

Result: SDLM matches or surpasses autoregressive baselines using only 3.5M training samples, achieves 2.1x higher throughput than Qwen-2.5, and SDLM-32B shows even stronger efficiency gains demonstrating scalability.

Conclusion: SDLM successfully unifies next-token and next-block prediction, enabling efficient adaptive generation while maintaining compatibility with existing infrastructure, showing strong potential for scalable language modeling.

Abstract: Diffusion language models (DLMs) have strong theoretical efficiency but are limited by fixed-length decoding and incompatibility with key-value (KV) caches. Block diffusion mitigates these issues, yet still enforces a fixed block size and requires expensive training. We introduce Next Sequence Prediction (NSP), which unifies next-token and next-block prediction, enabling the model to adaptively determine the generation length at each step. When the length is fixed to 1, NSP reduces to standard next-token prediction. Building on NSP, we propose Sequential Diffusion Language Model (SDLM), which can retrofit pre-trained autoregressive language models (ALMs) at minimal cost. Specifically, SDLM performs diffusion inference within fixed-size mask blocks, but dynamically decodes consecutive subsequences based on model confidence, thereby preserving KV-cache compatibility and improving robustness to varying uncertainty and semantics across the sequence. Experiments show that SDLM matches or surpasses strong autoregressive baselines using only 3.5M training samples, while achieving 2.1 higher throughput than Qwen-2.5. Notably, the SDLM-32B model delivers even more pronounced efficiency gains, demonstrating the strong scalability potential of our modeling paradigm. Project page and codes: https://github.com/OpenGVLab/SDLM

Method: SparseD pre-computes head-specific sparse patterns once and reuses them across all denoising steps, while using full attention in early critical steps and switching to sparse attention later to maintain generation quality.

Result: SparseD achieves up to 1.50× speedup over FlashAttention at 64k context length with 1,024 denoising steps while maintaining lossless performance.

Conclusion: SparseD provides a practical and efficient solution for deploying diffusion language models in long-context applications through its novel sparse attention approach.

Abstract: While diffusion language models (DLMs) offer a promising alternative to autoregressive models (ARs), existing open-source DLMs suffer from high inference latency. This bottleneck is mainly due to the attention’s quadratic complexity with respect to context length in computing all query-key pairs. Intuitively, to reduce this complexity, a natural strategy is to restrict attention to sparse patterns that retain only the most relevant connections. Such approaches are well-established in ARs, where attention follows fixed and clearly defined sparse patterns. However, in DLMs, we observe distinct sparsity behaviors: (1) attention patterns vary across heads, (2) attention patterns in each head remain highly similar across denoising steps, and (3) early denoising steps are critical for generation. These findings render sparse attention methods designed for ARs largely incompatible with DLMs, as they fail to capture head-specific structures and risk degrading generation when applied in early denoising steps. To address these challenges, we propose SparseD, a novel sparse attention method for DLMs. Leveraging the observations, SparseD only requires pre-computing head-specific sparse patterns one time, and reuses them across all steps. This prevents recomputing sparse patterns at each denoising step. Meanwhile, SparseD uses full attention in the early steps, then switches to sparse attention later to maintain generation quality. Together, these establish SparseD as a practical and efficient solution for deploying DLMs in long-context applications. Experimental results demonstrate that SparseD achieves lossless acceleration, delivering up to $1.50\times$ speedup over FlashAttention at a 64k context length with 1,024 denoising steps.

Method: ResFormer uses a cascaded approach with reservoir computing for long-term dependencies (linear time) and conventional Transformers for short-term dependencies with fixed-length inputs.

Result: Outperforms DeepSeek-Qwen and ModernBERT with up to +22.3% accuracy improvement on EmoryNLP and consistent gains on MultiWOZ, MELD, and IEMOCAP datasets, with reduced memory consumption.

Conclusion: ResFormer effectively addresses Transformer limitations by efficiently modeling varying context lengths, demonstrating superior performance and efficiency for sequence classification tasks.

Abstract: Sequence classification is essential in NLP for understanding and categorizing language patterns in tasks like sentiment analysis, intent detection, and topic classification. Transformer-based models, despite achieving state-of-the-art performance, have inherent limitations due to quadratic time and memory complexity, restricting their input length. Although extensive efforts have aimed at reducing computational demands, processing extensive contexts remains challenging. To overcome these limitations, we propose ResFormer, a novel neural network architecture designed to model varying context lengths efficiently through a cascaded methodology. ResFormer integrates an reservoir computing network featuring a nonlinear readout to effectively capture long-term contextual dependencies in linear time. Concurrently, short-term dependencies within sentences are modeled using a conventional Transformer architecture with fixed-length inputs. Experiments demonstrate that ResFormer significantly outperforms baseline models of DeepSeek-Qwen and ModernBERT, delivering an accuracy improvement of up to +22.3% on the EmoryNLP dataset and consistent gains on MultiWOZ, MELD, and IEMOCAP. In addition, ResFormer exhibits reduced memory consumption, underscoring its effectiveness and efficiency in modeling extensive contextual information.

Method: Used an ensemble of pre-trained sentiment analysis models (bert-base-multilingual-uncased-sentiment and XLM-R) combined with large language models for multilingual sentiment analysis on multi-language datasets.

Result: Experimental results showed sentiment analysis performance exceeding 86% using the proposed ensemble method.

Conclusion: The transformer ensemble model with LLM integration effectively addresses cross-lingual sentiment analysis challenges, providing high accuracy without requiring language-specific labeled data.

Abstract: Sentiment analysis is a very important natural language processing activity in which one identifies the polarity of a text, whether it conveys positive, negative, or neutral sentiment. Along with the growth of social media and the Internet, the significance of sentiment analysis has grown across numerous industries such as marketing, politics, and customer service. Sentiment analysis is flawed, however, when applied to foreign languages, particularly when there is no labelled data to train models upon. In this study, we present a transformer ensemble model and a large language model (LLM) that employs sentiment analysis of other languages. We used multi languages dataset. Sentiment was then assessed for sentences using an ensemble of pre-trained sentiment analysis models: bert-base-multilingual-uncased-sentiment, and XLM-R. Our experimental results indicated that sentiment analysis performance was more than 86% using the proposed method.

Method: Created Words Checker as a practical LSCG instance, tested model characteristics and steering techniques, and proposed FoCusNet - a small model that filters constraints to help LLMs focus on relevant ones.

Result: Existing solutions suffer significant performance drops as constraint numbers increase, while FoCusNet provides an 8-13% accuracy improvement over baseline methods.

Conclusion: LLMs struggle with large constraint sets, but dedicated constraint filtering models like FoCusNet can significantly improve performance by helping LLMs focus on relevant constraints.

Abstract: Recent research has explored the constrained generation capabilities of Large Language Models (LLMs) when explicitly prompted by few task-specific requirements. In contrast, we introduce Large-Scale Constraint Generation (LSCG), a new problem that evaluates whether LLMs can parse a large, fine-grained, generic list of constraints. To examine the LLMs’ ability to handle an increasing number constraints, we create a practical instance of LSCG, called Words Checker. In Words Checker, we evaluate the impact of model characteristics (e.g., size, family) and steering techniques (e.g., Simple Prompt, Chain of Thought, Best of N) on performance. We also propose FoCusNet, a small and dedicated model that parses the original list of constraints into a smaller subset, helping the LLM focus on relevant constraints. Experiments reveal that existing solutions suffer a significant performance drop as the number of constraints increases, with FoCusNet showing an 8-13% accuracy boost.

Method: GEAR evaluates hypothesis sets using three metrics: consistency (explains observations), generalizability (makes meaningful predictions on unseen inputs), and diversity (covers distinct patterns). Also proposes a momentum-based curriculum that adjusts training data based on learning velocity.

Result: Evaluated 9 LLMs on 4 abduction benchmarks with 1,500 problems, generating 50,000+ hypotheses, revealing model differences missed by traditional evaluations. The curriculum approach improved all GEAR objectives and transferred gains to established benchmarks.

Conclusion: GEAR provides a principled framework for evaluating abduction and supplies label-free, scalable training signals that help LLMs produce more diverse and reliable hypotheses.

Abstract: Since the advent of large language models (LLMs), research has focused on instruction following and deductive reasoning. A central question remains: can these models discover new knowledge, and how can we evaluate this ability? We address this by studying abductive reasoning-the generation of plausible hypotheses to explain observations-and introduce GEAR (General Evaluation for Abductive Reasoning), a general-purpose, fully automated, transparent, and label-free evaluation paradigm. GEAR scores hypothesis sets by three metrics: consistency (each hypothesis explains the observations), generalizability (consistent hypotheses make meaningful predictions on unseen inputs), and diversity (the set covers distinct predictions and patterns). Built this way, GEAR is scalable (no human gold answers), reliable (deterministic scoring aligned with classical abduction), and open-ended (scores improve only when models produce new plausible hypotheses, unlike static benchmarks that saturate once accuracy is high). Using GEAR, we conduct a fine-grained study of nine LLMs on four abduction benchmarks with 1,500 problems, generating over 50,000 candidate hypotheses and revealing model differences obscured by gold-answer or purely human evaluations. We further propose a momentum-based curriculum that adjusts GEAR-derived training data by learning velocity: it starts with what the model learns quickly and shifts toward harder objectives such as generating diverse hypotheses once the model is confident on foundational objectives. Without gold-label supervision, this strategy improves all GEAR objectives and these gains transfer to established abductive reasoning benchmarks. Taken together, GEAR provides a principled framework that evaluates abduction and supplies label-free, scalable training signals that help LLMs produce more diverse and reliable hypotheses.

Method: Uses Large Language Models for controllable generation of test sentences, creating linguistically rich and diverse test cases with minimal specification to cover various identity groups and bias types.

Result: LLM-based generation provides high linguistic variation and diversity in test sentences, offering better test coverage compared to base prompting methods, even for previously unseen biases.

Conclusion: BTC-SAM demonstrates that LLMs can effectively generate high-quality bias test cases, making comprehensive bias testing more accessible and scalable for Sentiment Analysis models.

Abstract: Sentiment Analysis (SA) models harbor inherent social biases that can be harmful in real-world applications. These biases are identified by examining the output of SA models for sentences that only vary in the identity groups of the subjects. Constructing natural, linguistically rich, relevant, and diverse sets of sentences that provide sufficient coverage over the domain is expensive, especially when addressing a wide range of biases: it requires domain experts and/or crowd-sourcing. In this paper, we present a novel bias testing framework, BTC-SAM, which generates high-quality test cases for bias testing in SA models with minimal specification using Large Language Models (LLMs) for the controllable generation of test sentences. Our experiments show that relying on LLMs can provide high linguistic variation and diversity in the test sentences, thereby offering better test coverage compared to base prompting methods even for previously unseen biases.

Method: Proposed pragmatic inference methods grounded in moral foundations theory, which leverage contextual information at each reasoning step to connect moral foundations with moral reasoning objectives and bridge the pragmatic gap.

Result: Experimental results show that the proposed approach significantly enhances LLMs’ generalization capabilities in moral reasoning tasks.

Conclusion: The method provides a foundation for future research on moral reasoning in LLMs using moral foundations theory, successfully addressing the generalization challenge by bridging the pragmatic gap between distributional semantics and moral reasoning.

Abstract: Moral reasoning has emerged as a promising research direction for Large Language Models (LLMs), yet achieving generalization remains a central challenge. From a linguistic standpoint, this difficulty arises because LLMs are adept at capturing distributional semantics, which fundamentally differs from the morals which operate at the pragmatic level. This paper investigates how LLMs can achieve generalized moral reasoning despite their reliance on distributional semantics. We propose pragmatic inference methods grounded in moral foundations theory, which leverage contextual information at each step to bridge the pragmatic gap and guide LLMs in connecting moral foundations with moral reasoning objectives. Experimental results demonstrate that our approach significantly enhances LLMs’ generalization in moral reasoning, providing a foundation for future research grounded in moral foundations theory.

Method: Uses dual-scale world models with a global trajectory frontier for high-value discoveries and local trial-and-error exploration through Multi-path Advantage Reflection mechanism that infers advantage-based progress signals.

Result: Achieves new state-of-the-art performance for LLM-based approaches on Jericho benchmark suite, with comparable performance to RL-based methods but requiring 100-800x fewer environment interactions.

Conclusion: GLoW demonstrates that dual-scale world modeling with advantage-based exploration guidance enables efficient hard-exploration in LLM-based agents, significantly reducing the sample complexity compared to traditional RL approaches.

Abstract: LLM-based agents have seen promising advances, yet they are still limited in “hard-exploration” tasks requiring learning new knowledge through exploration. We present GLoW, a novel approach leveraging dual-scale world models, maintaining a trajectory frontier of high-value discoveries at the global scale, while learning from local trial-and-error in exploration through a Multi-path Advantage Reflection mechanism which infers advantage-based progress signals to guide exploration. To evaluate our framework for hard-exploration, we tackle the Jericho benchmark suite of text-based games, where GLoW achieves a new state-of-theart performance for LLM-based approaches. Compared to state-of-the-art RLbased methods, our approach achieves comparable performance while requiring 100-800x fewer environment interactions.

Method: Created EduVidQA dataset with 5252 question-answer pairs from 296 CS videos, studied student preferences, and benchmarked 6 MLLMs using text-based and qualitative metrics.

Result: The task is challenging for current MLLMs, and the study provides insights into model performance nuances and the effectiveness of synthetic data for finetuning.

Conclusion: This work establishes a benchmark for educational QA and opens new research directions in NLP for education.

Abstract: As digital platforms redefine educational paradigms, ensuring interactivity remains vital for effective learning. This paper explores using Multimodal Large Language Models (MLLMs) to automatically respond to student questions from online lectures - a novel question answering task of real world significance. We introduce the EduVidQA Dataset with 5252 question-answer pairs (both synthetic and real-world) from 296 computer science videos covering diverse topics and difficulty levels. To understand the needs of the dataset and task evaluation, we empirically study the qualitative preferences of students, which we provide as an important contribution to this line of work. Our benchmarking experiments consist of 6 state-of-the-art MLLMs, through which we study the effectiveness of our synthetic data for finetuning, as well as showing the challenging nature of the task. We evaluate the models using both text-based and qualitative metrics, thus showing a nuanced perspective of the models’ performance, which is paramount to future work. This work not only sets a benchmark for this important problem, but also opens exciting avenues for future research in the field of Natural Language Processing for Education.

Method: TARE alternates between inner adversarial search (stressing prompts with hard paraphrases) and outer robust selection (preferring candidates with strong neighborhoods). ATARE adds anisotropic weights to shape semantic neighborhoods and adaptive radius for exploration-fidelity balance.

Result: The methods preserve accuracy under paraphrasing across diverse tasks, outperforming accuracy-only prompt search while remaining computationally practical.

Conclusion: Minimizing textual sharpness gap leads to robust prompts that maintain performance under semantic variations, addressing a key limitation in current prompt optimization approaches.

Abstract: The performance of Large Language Models (LLMs) hinges on carefully engineered prompts. However, prevailing prompt optimization methods, ranging from heuristic edits and reinforcement learning to evolutionary search, primarily target point-wise accuracy. They seldom enforce paraphrase invariance or searching stability, and therefore cannot remedy this brittleness in practice. Automated prompt search remains brittle: small, semantically preserving paraphrases often cause large performance swings. We identify this brittleness as the textual sharpness of the prompt landscape. In this work, we provide the first formal treatment of textual sharpness in the discrete, semantic space of prompts, together with an operational robustness criterion over a semantic neighborhood; the design is black-box or API-only, requiring no gradients to update the model’s parameters. Then we introduce TARE (Textual Sharpness-Aware Evolving), a derivative-free framework that alternates between an inner, sampling-based adversarial search that stresses a prompt with hard paraphrases and an outer, robust selection that prefers candidates whose neighborhoods remain strong. We further propose ATARE, which learns anisotropic weights to shape the semantic neighborhood and adapts its radius over time to balance exploration and fidelity. Diverse tasks evaluate our methods, whose design for minimizing textual sharpness gap leads to prompts that preserve accuracy under paraphrasing, outperforming accuracy-only prompt search while remaining computationally practical.

Method: LOT uses a generative language model to compare reasoning traces from two LRMs, articulate distinctive features in words, and model how these features predict the source model based on empirical distributions across outputs.

Result: LOT achieved 80-100% accuracy in distinguishing reasoning traces from LRMs that differ in scale, model family, or domain. It identified systematic differences in reasoning styles and showed that aligning smaller models’ reasoning with larger models improved accuracy on GPQA by 3.3-5.7%.

Conclusion: LOT provides both quantitative classification and qualitative explanations of how different large reasoning models think, revealing systematic reasoning differences that can be leveraged to improve model performance.

Abstract: Current comparisons of large reasoning models (LRMs) focus on macro-level statistics such as task accuracy or reasoning length. Whether different LRMs reason differently remains an open question. To address this gap, we introduce the LLM-proposed Open Taxonomy (LOT), a classification method that uses a generative language model to compare reasoning traces from two LRMs and articulate their distinctive features in words. LOT then models how these features predict the source LRM of a reasoning trace based on their empirical distributions across LRM outputs. Iterating this process over a dataset of reasoning traces yields a human-readable taxonomy that characterizes how models think. We apply LOT to compare the reasoning of 12 open-source LRMs on tasks in math, science, and coding. LOT identifies systematic differences in their thoughts, achieving 80-100% accuracy in distinguishing reasoning traces from LRMs that differ in scale, base model family, or objective domain. Beyond classification, LOT’s natural-language taxonomy provides qualitative explanations of how LRMs think differently. Finally, in a case study, we link the reasoning differences to performance: aligning the reasoning style of smaller Qwen3 models with that of the largest Qwen3 during test time improves their accuracy on GPQA by 3.3-5.7%.

Method: Proposed TSLA framework to identify TR and TL specialized attention heads, using correlation analysis, ablation studies, input perturbations, and steering experiments with geometric analysis of hidden states.

Result: TR heads align hidden states with task subspace for task recognition, while TL heads rotate hidden states within subspace toward correct labels for prediction. The framework reconciles previous findings like induction heads and task vectors.

Conclusion: The TSLA framework provides a unified and interpretable account of how large language models execute in-context learning across diverse tasks and settings through complementary TR and TL mechanisms.

Abstract: We investigate the mechanistic underpinnings of in-context learning (ICL) in large language models by reconciling two dominant perspectives: the component-level analysis of attention heads and the holistic decomposition of ICL into Task Recognition (TR) and Task Learning (TL). We propose a novel framework based on Task Subspace Logit Attribution (TSLA) to identify attention heads specialized in TR and TL, and demonstrate their distinct yet complementary roles. Through correlation analysis, ablation studies, and input perturbations, we show that the identified TR and TL heads independently and effectively capture the TR and TL components of ICL. Using steering experiments with geometric analysis of hidden states, we reveal that TR heads promote task recognition by aligning hidden states with the task subspace, while TL heads rotate hidden states within the subspace toward the correct label to facilitate prediction. We further show how previous findings on ICL mechanisms, including induction heads and task vectors, can be reconciled with our attention-head-level analysis of the TR-TL decomposition. Our framework thus provides a unified and interpretable account of how large language models execute ICL across diverse tasks and settings.

Method: Propose directly training Learned Task Vectors (LTVs) and conduct systematic mechanistic analysis to understand how task vectors operate through attention-head OV circuits and propagate linearly through Transformer layers.

Result: LTVs outperform extracted task vectors in accuracy and work effectively at arbitrary layers and positions. Analysis reveals task vectors primarily influence predictions through specific “key heads” in attention circuits, and despite Transformer nonlinearities, task vector propagation is largely linear.

Conclusion: LTVs provide both a practical approach for obtaining effective task vectors and a principled framework for understanding the mechanistic foundations of in-context learning in LLMs.

Abstract: Large Language Models (LLMs) can perform new tasks from in-context demonstrations, a phenomenon known as in-context learning (ICL). Recent work suggests that these demonstrations are compressed into task vectors (TVs), compact task representations that LLMs exploit for predictions. However, prior studies typically extract TVs from model outputs or hidden states using cumbersome and opaque methods, and they rarely elucidate the mechanisms by which TVs influence computation. In this work, we address both limitations. First, we propose directly training Learned Task Vectors (LTVs), which surpass extracted TVs in accuracy and exhibit superior flexibility-acting effectively at arbitrary layers, positions, and even with ICL prompts. Second, through systematic analysis, we investigate the mechanistic role of TVs, showing that at the low level they steer predictions primarily through attention-head OV circuits, with a small subset of “key heads” most decisive. At a higher level, we find that despite Transformer nonlinearities, TV propagation is largely linear: early TVs are rotated toward task-relevant subspaces to improve logits of relevant labels, while later TVs are predominantly scaled in magnitude. Taken together, LTVs not only provide a practical approach for obtaining effective TVs but also offer a principled lens into the mechanistic foundations of ICL.

Method: Uses supervised finetuning (SFT) followed by self-guided rejection sampling finetuning (RSF), functioning as a model-agnostic plug-in that retrieves web tutorials during inference.

Result: Outperforms baseline agents by 2.6% to 13.3% across two model sizes and three distinct tasks, demonstrating strong generalization capabilities.

Conclusion: RAG-GUI provides an effective plug-and-play solution for enhancing VLM-based GUI agents in real-world applications.

Abstract: GUI agents powered by vision-language models (VLMs) show promise in automating complex digital tasks. However, their effectiveness in real-world applications is often limited by scarce training data and the inherent complexity of these tasks, which frequently require long-tailed knowledge covering rare, unseen scenarios. We propose RAG-GUI , a lightweight VLM that leverages web tutorials at inference time. RAG-GUI is first warm-started via supervised finetuning (SFT) and further refined through self-guided rejection sampling finetuning (RSF). Designed to be model-agnostic, RAG-GUI functions as a generic plug-in that enhances any VLM-based agent. Evaluated across three distinct tasks, it consistently outperforms baseline agents and surpasses other inference baselines by 2.6% to 13.3% across two model sizes, demonstrating strong generalization and practical plug-and-play capabilities in real-world scenarios.

Method: Prospectively gathered responses from 80 diverse LLMs on a balanced 1,100-question USMLE-aligned benchmark, using unidimensional two-parameter logistic IRT models per topic to estimate latent model ability, question difficulty and discrimination.

Result: Identified distinctive ‘spiky’ ability profiles where overall rankings can be misleading; GPT-5 was top in 8 of 11 domains but outperformed by Claude-3-opus in Social Science and Communication; IRT helped identify flawed questions in benchmarks.

Conclusion: Establishes a robust, psychometrically grounded methodology essential for safe, effective, and trustworthy deployment of LLMs in healthcare, with a practical decision-support framework integrating multi-factor competency profiles.

Abstract: As Large Language Models (LLMs) are increasingly proposed for high-stakes medical applications, there has emerged a critical need for reliable and accurate evaluation methodologies. Traditional accuracy metrics fail inadequately as they neither capture question characteristics nor offer topic-specific insights. To address this gap, we introduce \textsc{MedIRT}, a rigorous evaluation framework grounded in Item Response Theory (IRT), the gold standard in high-stakes educational testing. Unlike previous research relying on archival data, we prospectively gathered fresh responses from 80 diverse LLMs on a balanced, 1,100-question USMLE-aligned benchmark. Using one unidimensional two-parameter logistic IRT model per topic, we estimate LLM’s latent model ability jointly with question difficulty and discrimination, yielding more stable and nuanced performance rankings than accuracy alone. Notably, we identify distinctive ``spiky’’ ability profiles, where overall rankings can be misleading due to highly specialized model abilities. While \texttt{GPT-5} was the top performer in a majority of domains (8 of 11), it was outperformed in Social Science and Communication by \texttt{Claude-3-opus}, demonstrating that even an overall 23rd-ranked model can hold the top spot for specific competencies. Furthermore, we demonstrate IRT’s utility in auditing benchmarks by identifying flawed questions. We synthesize these findings into a practical decision-support framework that integrates our multi-factor competency profiles with operational metrics. This work establishes a robust, psychometrically grounded methodology essential for the safe, effective, and trustworthy deployment of LLMs in healthcare.

Method: Proposed Preference Evolution Tracking (PET) framework that uses logit-probing and generative classification to infer user preferences as probability distributions over a stable lattice of preference clusters, rather than direct item generation.

Result: PET improves ranking quality by up to 40% in NDCG on public benchmarks (Yelp, MovieLens) and outperforms a SOTA production model by 7 times in NDCG score on a large-scale short-video platform dataset, particularly excelling at ranking long-tail content.

Conclusion: PET transforms user profiling from direct preference list generation to transparent distributional preference mapping, enabling more explainable, fair, and diverse personalization systems.

Abstract: Understanding how user preference evolves over time is a fundamental challenge central to modern digital ecosystems, for which Large Language Models (LLMs) are an increasingly prominent and popular approach due to their ability to comprehend the rich semantic context within behavioral data. A common practice is to use LLMs to predict a user’s next action by directly generating a ranked list of preferred items. Although effective for short-term prediction, the end-to-end generation paradigm inherently limits personalization. Its opaque decision-making process obscures holistic user profiling and exacerbates popularity bias. To address these limitations, we propose Preference Evolution Tracking (PET), a framework that reframes the task as inferring a dynamic probability distribution over a stable and interpretable lattice of preference clusters. By applying logit-probing and generative classification techniques, PET infers a user’s preference as a probability distribution, enabling transparent preference learning. On public benchmarks (Yelp, MovieLens), PET improves ranking quality by up to 40% in NDCG over direct generation baselines. On a large-scale, real-world dataset from a short-video platform, it excels at ranking long-tail contents, significantly outperforming a SOTA production model by 7 times in the NDCG score. Ultimately, PET transforms the user profile model from direct preference list generation to a transparent distributional preference mapping, paving the way for more explainable, fair, and diverse personalization systems.

Method: Uses cooperative self-play with a single LLM alternating between decomposer (breaks down queries) and solver (integrates contexts) roles, combining supervised fine-tuning on search/reasoning/decomposition tasks with reinforcement fine-tuning optimized for final answer accuracy.

Result: Outperforms state-of-the-art baselines with 7.6% average exact match improvement, matches DeepSeek-V3 performance with <5% parameters on finance tasks, and surpasses larger models (up to 9x parameters) at smaller scales (1.5B/8B).

Conclusion: AceSearcher demonstrates exceptional efficiency and effectiveness in tackling complex reasoning tasks through its cooperative self-play framework, eliminating the need for intermediate annotations while achieving superior performance.

Abstract: Search-augmented LLMs often struggle with complex reasoning tasks due to ineffective multi-hop retrieval and limited reasoning ability. We propose AceSearcher, a cooperative self-play framework that trains a single large language model (LLM) to alternate between two roles: a decomposer that breaks down complex queries and a solver that integrates retrieved contexts for answer generation. AceSearcher couples supervised fine-tuning on a diverse mixture of search, reasoning, and decomposition tasks with reinforcement fine-tuning optimized for final answer accuracy, eliminating the need for intermediate annotations. Extensive experiments on three reasoning-intensive tasks across 10 datasets show that AceSearcher outperforms state-of-the-art baselines, achieving an average exact match improvement of 7.6%. Remarkably, on document-level finance reasoning tasks, AceSearcher-32B matches the performance of the DeepSeek-V3 model using less than 5% of its parameters. Even at smaller scales (1.5B and 8B), AceSearcher often surpasses existing search-augmented LLMs with up to 9x more parameters, highlighting its exceptional efficiency and effectiveness in tackling complex reasoning tasks. Our code will be published at https://github.com/ritaranx/AceSearcher and https://huggingface.co/AceSearcher.

Method: Released a large-scale dataset of hedging expressions with human-annotated confidence scores; proposed a lightweight mapper to convert hedges into confidence scores; conducted systematic study across LLMs and QA benchmarks; introduced fine-tuning framework for LC improvement.

Result: Most LLMs underperform in expressing reliable linguistic confidence, but carefully designed prompting achieves competitive calibration and discriminability. Fine-tuning further improves LC reliability.

Conclusion: Linguistic confidence is positioned as a scalable, efficient, and human-aligned approach to LLM uncertainty estimation, calling for deeper exploration of this promising direction.

Abstract: Large language models (LLMs) are increasingly used in high-stakes settings, where overconfident responses can mislead users. Reliable confidence estimation has been shown to enhance trust and task accuracy. Yet existing methods face practical barriers: logits are often hidden, multi-sampling is computationally expensive, and verbalized numerical uncertainty (e.g., giving a 0-100 score) deviates from natural communication. We revisit linguistic confidence (LC), where models express uncertainty through hedging language (e.g., probably, might), offering a lightweight and human-centered alternative. To advance this direction, we (1) release the first diverse, large-scale dataset of hedging expressions with human-annotated confidence scores, and (2) propose a lightweight mapper that converts hedges into confidence scores at near-zero cost. Building on these resources, we (3) conduct the first systematic study of LC across modern LLMs and QA benchmarks, revealing that while most LLMs underperform in expressing reliable LC, carefully designed prompting achieves competitive calibration and discriminability. Finally, we (4) introduce a fine-tuning framework that further improves LC reliability. Taken together, our work positions linguistic confidence as a scalable, efficient, and human-aligned approach to LLM uncertainty estimation, and calls for deeper exploration of this promising yet underexplored direction.

Method: The framework generates mathematically grounded problems on the fly from a combinatorial space larger than 10^15 unique instances, with solutions verified by mathematical proofs. It covers 44 algorithmic tasks with 117 variations across Easy, Medium, and Hard suites.

Result: Evaluation of 101 language models showed consistent reasoning deficiencies, with performance degrading sharply as problem complexity increases. In the Hard Suite, top models achieved 26.91-56.38% accuracy, and performance dropped significantly without tool usage.

Conclusion: BeyondBench provides a contamination-free evaluation method that reveals fundamental reasoning limitations in current language models, especially as problem complexity increases from polynomial to exponential.

Abstract: Evaluating language models fairly is becoming harder as static benchmarks available on the internet risk contamination by training data. This makes it unclear whether models are truly reasoning or just recalling answers. In this paper, we introduce BeyondBench, an evaluation framework that avoids this problem by using algorithmic problem generation. Unlike traditional benchmarks that risk contamination from internet-scale training data, BeyondBench creates mathematically grounded problems on the fly, ensuring each test remains fresh and uncontaminated. Our framework covers 44 algorithmic tasks with a total of 117 variations, grouped into three difficulty levels: the Easy Suite (29 tasks) for basic arithmetic and statistics, the Medium Suite (5 tasks, 49 variations) for sequence patterns and reasoning, and the Hard Suite (10 tasks, 68 variations) tackling NP-complete and constraint satisfaction problems. Each task generates problems from a combinatorial space larger than 10^15 unique instances, with solutions verified deterministically by mathematical proofs. We evaluated 101 language models, including 85 open-source and 16 closed-source models, spanning sizes from 0.5B to 141B parameters and multiple quantization schemes. Our results show consistent reasoning deficiencies across model families, with performance degrading sharply as problem complexity increases from polynomial to exponential. In our Hard Suite evaluations, models such as Gemini-2.5-pro, Llama-3.3-70B, and Qwen2.5-72B achieved average accuracies of 56.38%, 26.91%, and 33.60%, respectively. Moreover, we observe that performance drops drastically without tool usage, with GPT-5, GPT-5-mini, and GPT-5-nano showing a decline of 16.81%, 28.05%, and 47.59% accuracy on the hard suite. Our leaderboard is publicly available at https://ctrl-gaurav.github.io/BeyondBench/

Method: Uses YAML scenarios with no-peek gold standards containing expected decisions, witness traces, governing clauses, and canonical SQL. Systems must justify outputs using clause IDs from the policy canon.

Result: Enables comprehensive evaluation including decision accuracy, trace quality, retrieval effectiveness, SQL correctness, policy coverage, latency, and explanation-hallucination rate.

Conclusion: ScenarioBench shifts evaluation focus toward justification quality under time constraints, providing more realistic assessment for compliance applications compared to prior benchmarks.

Abstract: ScenarioBench is a policy-grounded, trace-aware benchmark for evaluating Text-to-SQL and retrieval-augmented generation in compliance contexts. Each YAML scenario includes a no-peek gold-standard package with the expected decision, a minimal witness trace, the governing clause set, and the canonical SQL, enabling end-to-end scoring of both what a system decides and why. Systems must justify outputs using clause IDs from the same policy canon, making explanations falsifiable and audit-ready. The evaluator reports decision accuracy, trace quality (completeness, correctness, order), retrieval effectiveness, SQL correctness via result-set equivalence, policy coverage, latency, and an explanation-hallucination rate. A normalized Scenario Difficulty Index (SDI) and a budgeted variant (SDI-R) aggregate results while accounting for retrieval difficulty and time. Compared with prior Text-to-SQL or KILT/RAG benchmarks, ScenarioBench ties each decision to clause-level evidence under strict grounding and no-peek rules, shifting gains toward justification quality under explicit time budgets.

Method: Developed MoVa suite with 16 datasets from four frameworks, created lightweight LLM prompting strategy (all@once), and built survey evaluation application.

Result: The LLM prompting strategy outperforms fine-tuned models across multiple domains and frameworks.

Conclusion: MoVa facilitates fine-grained interpretations of human and machine communication with implications for machine behavior alignment.

Abstract: Identifying human morals and values embedded in language is essential to empirical studies of communication. However, researchers often face substantial difficulty navigating the diversity of theoretical frameworks and data available for their analysis. Here, we contribute MoVa, a well-documented suite of resources for generalizable classification of human morals and values, consisting of (1) 16 labeled datasets and benchmarking results from four theoretically-grounded frameworks; (2) a lightweight LLM prompting strategy that outperforms fine-tuned models across multiple domains and frameworks; and (3) a new application that helps evaluate psychological surveys. In practice, we specifically recommend a classification strategy, all@once, that scores all related concepts simultaneously, resembling the well-known multi-label classifier chain. The data and methods in MoVa can facilitate many fine-grained interpretations of human and machine communication, with potential implications for the alignment of machine behavior.

Method: Used Qwen3-14B with LoRA fine-tuning and model fusion; synthesized data for some tasks; employed three distinct LoRA adapters for tool calling, response generation with tool call results, and response generation without tool call results; implemented MultiLoRA inference using vLLM.

Result: Achieved first place in Task 1 and Task 3, and second place in Task 2 of the GPU track.

Conclusion: The approach of using multiple LoRA adapters with Qwen3-14B and model fusion was effective for building a conversational AI that meets the competition requirements, demonstrating the viability of this method for similar tasks.

Abstract: This paper presents the opdainlp team’s solution for the GPU track of the CPDC 2025 challenge. The challenge consists of three tasks, aiming to build an in-game conversational AI that adheres to character personas, aligns with the game’s worldview, and supports function calling. Considering both effectiveness and resource/time constraints during inference, we synthesized data for some of the tasks based on the datasets provided by the competition organizers. We employed Qwen3-14B with LoRA fine-tuning and model fusion, and utilized a base model integrated with multiple LoRA adapters during inference. Specifically, in the competition, we used three distinct LoRA adapters to handle tool calling, response generation with tool call results, and response generation without tool call results, respectively. MultiLoRA inference was implemented using vLLM. Our solution achieved the first place in Task 1 and Task 3, and the second place in Task 2 of the GPU track.

Method: Introduces two neural networks for generating prompts and parameters respectively, with a shared bottom encoding layer for knowledge sharing. Uses supervised regularization loss to handle the discrete-continuous optimization challenge.

Result: Extensive experiments across diverse benchmarks show consistent outperformance over baseline methods.

Conclusion: Joint integration of prompt optimization and fine-tuning through MetaTuner framework effectively leverages their complementary strengths for improved LLM performance.

Abstract: Prompt optimization and fine-tuning are two major approaches to improve the performance of Large Language Models (LLMs). They enhance the capabilities of LLMs from complementary perspectives: the former through explicit natural language, and the latter through implicit parameter updates. However, prior work has typically studied them in isolation, leaving their synergistic potential largely underexplored. To bridge this gap, in this paper, we introduce MetaTuner, a novel framework that jointly integrates prompt optimization and fine-tuning for LLM training. Specifically, we introduce two neural networks to generate prompts and parameters, respectively, while allowing them to share a common bottom encoding layer to enable knowledge sharing. By the guidance of the final supervised signals, our framework is optimized to discover the optimal combinations between the prompts and parameters. Given that prompt learning involves discrete optimization while fine-tuning operates in a continuous parameter space, we design a supervised regularization loss to train our framework effectively. Extensive experiments across diverse benchmarks show that our method consistently outperforms the baselines.

Method: Proposed MRAG-Suite integrates multiple multimodal benchmarks (WebQA, Chart-RAG, Visual-RAG, MRAG-Bench) with difficulty-based and ambiguity-aware filtering strategies, and introduces MM-RAGChecker for claim-level diagnostic analysis.

Result: Results show substantial accuracy reductions under difficult and ambiguous queries, revealing prevalent hallucinations in Visual RAG systems. MM-RAGChecker effectively diagnoses these issues.

Conclusion: The proposed evaluation framework successfully identifies key limitations in Visual RAG systems and provides diagnostic tools to guide future improvements in multimodal retrieval-augmented generation.

Abstract: Multimodal Retrieval-Augmented Generation (Visual RAG) significantly advances question answering by integrating visual and textual evidence. Yet, current evaluations fail to systematically account for query difficulty and ambiguity. We propose MRAG-Suite, a diagnostic evaluation platform integrating diverse multimodal benchmarks (WebQA, Chart-RAG, Visual-RAG, MRAG-Bench). We introduce difficulty-based and ambiguity-aware filtering strategies, alongside MM-RAGChecker, a claim-level diagnostic tool. Our results demonstrate substantial accuracy reductions under difficult and ambiguous queries, highlighting prevalent hallucinations. MM-RAGChecker effectively diagnoses these issues, guiding future improvements in Visual RAG systems.

Method: Developed SimuHome - a time-accelerated home environment simulating smart devices, supporting API calls, and reflecting environmental changes. Built on Matter protocol for high fidelity and real-world deployability. Created benchmark of 600 episodes across 12 user query types requiring complex capabilities.

Result: Evaluation of 11 agents under ReAct framework shows models perform well on simple tasks but struggle with latent intent inference (54% success rate for top model GPT-4.1), state verification, and temporal scheduling.

Conclusion: There’s a critical need for methods that can reliably verify current state via tools before acting and coordinate time-dependent actions. SimuHome provides a realistic testbed for developing and evaluating smart home agents that can be deployed on real Matter-compliant devices.

Abstract: Large Language Model (LLM) agents excel at multi-step, tool-augmented tasks. However, smart homes introduce distinct challenges, requiring agents to handle latent user intents, temporal dependencies, device constraints, scheduling, and more. The main bottlenecks for developing smart home agents with such capabilities include the lack of a realistic simulation environment where agents can interact with devices and observe the results, as well as a challenging benchmark to evaluate them. To address this, we introduce $\textbf{SimuHome}$, a time-accelerated home environment that simulates smart devices, supports API calls, and reflects changes in environmental variables. By building the simulator on the Matter protocol (the global industry standard for smart home communication), SimuHome provides a high-fidelity environment, and agents validated in SimuHome can be deployed on real Matter-compliant devices with minimal adaptation. We provide a challenging benchmark of 600 episodes across twelve user query types that require the aforementioned capabilities. Our evaluation of 11 agents under a unified ReAct framework reveals that while models perform well on simple tasks, they struggle with latent intent inference, state verification, and especially temporal scheduling. Even the top-performing model, GPT-4.1, reaches only 54% success rate. These findings highlight a critical need for methods that can reliably verify the current state via tools before acting and coordinate time-dependent actions.

Method: GIRCSE uses autoregressive generation to produce sequences of soft tokens optimized under an Iterative Contrastive Refinement (ICR) objective that encourages better representations at each refinement step.

Result: GIRCSE outperforms strong LLM-based embedding baselines on MTEB benchmark and instruction-following tasks, and exhibits emergent test-time scaling where generating more tokens improves embedding quality.

Conclusion: Generative iterative refinement establishes a new paradigm for representation learning that effectively leverages LLMs’ generative strengths.

Abstract: Existing large language model (LLM)-based embeddings typically adopt an encoder-only paradigm, treating LLMs as static feature extractors and overlooking their core generative strengths. We introduce GIRCSE (Generative Iterative Refinement for Contrastive Sentence Embeddings), a novel framework that leverages autoregressive generation to iteratively refine semantic representations. By producing sequences of soft tokens optimized under contrastive objective, GIRCSE captures latent concepts and implicit semantics that encoder-only methods often miss. To guide this process, we propose an Iterative Contrastive Refinement (ICR) objective that encourages each refinement step to yield better representations. Extensive experiments show that GIRCSE outperforms strong LLM-based embedding baselines on the MTEB benchmark and instruction-following tasks. Moreover, GIRCSE exhibits an emergent test-time scaling property: generating more tokens at inference steadily improves embedding quality. Our results establish generative iterative refinement as a new paradigm for representation learning.

Method: Integrates LLM-driven coding, semantic clustering, graph reasoning, and iterative refinement process to build reusable codebooks. Introduces 5-dimensional metric and train-test split protocol for standardized evaluation.

Result: Consistently outperforms strong baselines across five diverse corpora, achieving 88.2% alignment with expert-developed schema on complex dataset.

Conclusion: LOGOS demonstrates a powerful path to democratize and scale qualitative research without sacrificing theoretical nuance through full automation of grounded theory workflow.

Abstract: Grounded theory offers deep insights from qualitative data, but its reliance on expert-intensive manual coding presents a major scalability bottleneck. Current computational tools stop short of true automation, keeping researchers firmly in the loop. We introduce LOGOS, a novel, end-to-end framework that fully automates the grounded theory workflow, transforming raw text into a structured, hierarchical theory. LOGOS integrates LLM-driven coding, semantic clustering, graph reasoning, and a novel iterative refinement process to build highly reusable codebooks. To ensure fair comparison, we also introduce a principled 5-dimensional metric and a train-test split protocol for standardized, unbiased evaluation. Across five diverse corpora, LOGOS consistently outperforms strong baselines and achieves a remarkable $88.2%$ alignment with an expert-developed schema on a complex dataset. LOGOS demonstrates a powerful new path to democratize and scale qualitative research without sacrificing theoretical nuance.

Method: The paper conducts vulnerability analysis across intra-step and inter-step dimensions, then proposes DiffuGuard - a dual-stage defense framework using Stochastic Annealing Remasking to mitigate greedy selection bias and Block-level Audit and Repair for autonomous risk detection and guided correction.

Result: Experimental results on four dLLMs show DiffuGuard reduces Attack Success Rate against six diverse jailbreak methods from 47.9% to 14.7% while preserving model utility and efficiency.

Conclusion: While current decoding strategies constitute significant vulnerabilities in dLLMs, these models possess substantial intrinsic safety potential that can be unlocked through proper defense mechanisms like DiffuGuard.

Abstract: The rapid advancement of Diffusion Large Language Models (dLLMs) introduces unprecedented vulnerabilities that are fundamentally distinct from Autoregressive LLMs, stemming from their iterative and parallel generation mechanisms. In this paper, we conduct an in-depth analysis of dLLM vulnerabilities to jailbreak attacks across two distinct dimensions: intra-step and inter-step dynamics. Experimental results reveal a harmful bias inherent in the standard greedy remasking strategy and identify a critical phenomenon we term Denoising-path Dependence, where the safety of early-stage tokens decisively influences the final output. These findings also indicate that while current decoding strategies constitute a significant vulnerability, dLLMs possess a substantial intrinsic safety potential. To unlock this potential, we propose DiffuGuard, a training-free defense framework that addresses vulnerabilities through a dual-stage approach: Stochastic Annealing Remasking dynamically introduces controlled randomness to mitigate greedy selection bias, while Block-level Audit and Repair exploits internal model representations for autonomous risk detection and guided correction. Comprehensive experiments on four dLLMs demonstrate DiffuGuard’s exceptional effectiveness, reducing Attack Success Rate against six diverse jailbreak methods from 47.9% to 14.7% while preserving model utility and efficiency. Our code is available at: https://github.com/niez233/DiffuGuard.

Method: Developed a TQA-to-MMQA framework with quality rubric, constructed evaluation benchmarks, and created Q-Mirror agent that integrates MMQA generation and evaluation in a closed loop for iterative refinement.

Result: State-of-the-art models can generate MMQAs but with substantial gaps; top understanding models align well with human judgment; Q-Mirror improved average scores from 78.90 to 85.22 and pass rates from 72% to 95%.

Conclusion: The Q-Mirror system offers a practical path to creating large-scale scientific benchmarks by automating the transformation of text-only QA pairs into high-quality multi-modal QA pairs.

Abstract: High-quality, multi-modal benchmarks are crucial for advancing scientific reasoning in large models yet their manual creation is costly and unscalable. To address this bottleneck, we explore the potential for transforming Text-Only QA Pairs (TQAs) into high-quality Multi-Modal QA Pairs (MMQAs), which include three parts: 1) Task Definition & Evaluation Rubric: We develop a TQA-to-MMQA framework and establish a comprehensive, multi-dimensional MMQA quality rubric that provides principles for the transformation. 2) Benchmark Construction: Then we construct two extensive benchmarks to rigorously evaluate state-of-the-art generation & understanding models on the distinct tasks of MMQA generation & MMQA quality evaluation. 3) Preliminary Solution: We develop an agentic system (Q-Mirror), which operationalizes our framework by integrating MMQA generation and evaluation into a closed loop for iterative refinement. Our experiments show that while state-of-the-art models can generate MMQAs, their outputs still leave substantial gaps, underscoring the need for reliable evaluation. We further demonstrate that top-tier understanding models align closely with human judgment in MMQA quality assessment. Leveraging both insights, the Q-Mirror agent raises average scores from 78.90 to 85.22 and pass rates from 72% to 95%, offering a practical path to large-scale scientific benchmarks.

Method: Used two mechanistic approaches: (1) value vectors extracted from residual stream, and (2) value neurons from MLP layers that contribute to value expressions.

Result: Found that intrinsic and prompted value mechanisms partially share common components but also have unique elements, leading to different steerability (prompted > intrinsic) and response diversity (intrinsic > prompted).

Conclusion: Intrinsic mechanisms promote lexical diversity while prompted mechanisms strengthen instruction following, with implications for jailbreaking and value alignment.

Abstract: Large language models (LLMs) can express different values in two distinct ways: (1) intrinsic expression, reflecting the model’s inherent values learned during training, and (2) prompted expression, elicited by explicit prompts. Given their widespread use in value alignment and persona steering, it is paramount to clearly understand their underlying mechanisms, particularly whether they mostly overlap (as one might expect) or rely on substantially different mechanisms, but this remains largely understudied. We analyze this at the mechanistic level using two approaches: (1) value vectors, feature directions representing value mechanisms extracted from the residual stream, and (2) value neurons, MLP neurons that contribute to value expressions. We demonstrate that intrinsic and prompted value mechanisms partly share common components that are crucial for inducing value expression, but also possess unique elements that manifest in different ways. As a result, these mechanisms lead to different degrees of value steerability (prompted > intrinsic) and response diversity (intrinsic > prompted). In particular, components unique to the intrinsic mechanism seem to promote lexical diversity in responses, whereas those specific to the prompted mechanism primarily strengthen instruction following, taking effect even in distant tasks like jailbreaking.

Method: The authors conduct a comprehensive survey that consolidates recent developments in LLMs and MLLMs for emotion recognition and reasoning, including model architectures, datasets, and performance benchmarks.

Result: The paper provides an authoritative reference and practical insights for researchers, highlighting key challenges and outlining future research directions in this emerging domain.

Conclusion: This represents the first comprehensive attempt to survey the intersection of MLLMs with multimodal emotion recognition and reasoning, with summarized methods available on GitHub.

Abstract: In recent years, large language models (LLMs) have driven major advances in language understanding, marking a significant step toward artificial general intelligence (AGI). With increasing demands for higher-level semantics and cross-modal fusion, multimodal large language models (MLLMs) have emerged, integrating diverse information sources (e.g., text, vision, and audio) to enhance modeling and reasoning in complex scenarios. In AI for Science, multimodal emotion recognition and reasoning has become a rapidly growing frontier. While LLMs and MLLMs have achieved notable progress in this area, the field still lacks a systematic review that consolidates recent developments. To address this gap, this paper provides a comprehensive survey of LLMs and MLLMs for emotion recognition and reasoning, covering model architectures, datasets, and performance benchmarks. We further highlight key challenges and outline future research directions, aiming to offer researchers both an authoritative reference and practical insights for advancing this domain. To the best of our knowledge, this paper is the first attempt to comprehensively survey the intersection of MLLMs with multimodal emotion recognition and reasoning. The summary of existing methods mentioned is in our Github: \href{https://github.com/yuntaoshou/Awesome-Emotion-Reasoning}{https://github.com/yuntaoshou/Awesome-Emotion-Reasoning}.

Method: SV introduces a companion model to estimate alignment between draft and target model distributions, dynamically predicting speculation accuracy and adapting verification length to reduce wasted computation on rejected tokens.

Result: SV consistently outperforms both speculative decoding and standard decoding across all experiments, improving SD performance by up to 2x with average 1.4x speedup in large-batch settings (batch sizes 32-80).

Conclusion: SV demonstrates robustness, scalability, and practical utility for efficient LLM inference, requiring no modifications to draft or target models and being compatible with existing SD variants.

Abstract: LLMs have low GPU efficiency and high latency due to autoregressive decoding. Speculative decoding (SD) mitigates this using a small draft model to speculatively generate multiple tokens, which are then verified in parallel by a target model. However, when speculation accuracy is low, the overhead from rejected tokens can offset the benefits, limiting SD’s effectiveness, especially at large batch sizes. To address this, we propose Speculative Verification (SV), an efficient augmentation to SD that dynamically predicts speculation accuracy and adapts the verification length to maximize throughput. SV introduces a companion model - a small auxiliary model similar in size to the draft model - to estimate the alignment between draft and target model distributions. By maximizing the information gain from quantifying this alignment, SV refines verification decisions, reducing wasted computation on rejected tokens and improving decoding efficiency. Moreover, SV requires no modifications to the draft or target models and is compatible with existing SD variants. We extensively evaluated SV on publicly available LLMs across three NLP tasks using nine combinations of draft, companion, and target models, including 13B-72B target models and three types of variations: base (no finetuning), instruction-tuned, and task fine-tuned. Across all experiments and batch sizes (4-80), SV consistently outperforms both SD and standard decoding with the target model. It improves SD performance by up to 2$\times$, with an average speedup of 1.4 $\times$ in large-batch settings (batch sizes 32-80). These results demonstrate SV’s robustness, scalability, and practical utility for efficient LLM inference.

Method: Two-stage framework: 1) Representation alignment with multilingual semantic alignment and language feature integration, 2) Multilingual instruction fine-tuning to stimulate LLM capabilities.

Result: Experimental results show enhanced multilingual general and cross-lingual generation capabilities across several pre-trained LLMs, with improved representation closeness and cross-lingual alignment.

Conclusion: AlignX effectively bridges the multilingual performance gap by improving representation alignment and stimulating multilingual capabilities through a structured two-stage approach.

Abstract: Multilingual large language models (LLMs) possess impressive multilingual understanding and generation capabilities. However, their performance and cross-lingual alignment often lag for non-dominant languages. A common solution is to fine-tune LLMs on large-scale and more balanced multilingual corpus, but such approaches often lead to imprecise alignment and suboptimal knowledge transfer, struggling with limited improvements across languages. In this paper, we propose AlignX to bridge the multilingual performance gap, which is a two-stage representation-level framework for enhancing multilingual performance of pre-trained LLMs. In the first stage, we align multilingual representations with multilingual semantic alignment and language feature integration. In the second stage, we stimulate the multilingual capability of LLMs via multilingual instruction fine-tuning. Experimental results on several pre-trained LLMs demonstrate that our approach enhances LLMs’ multilingual general and cross-lingual generation capability. Further analysis indicates that AlignX brings the multilingual representations closer and improves the cross-lingual alignment.

Method: Built on parallel corpora with human-written paragraphs aligned with LLM-simplified variants, testing four data schedules: repeated exposure, low-to-high complexity, high-to-low, and interleaved. Analyzed representation quality via fine-tuning and zero-shot performance on multiple tasks.

Result: Adding simplified data improves fine-tuning and zero-shot performance over repeated-exposure baseline. Smaller models benefit from low-to-high complexity ordering, while larger models perform better with interleaved ordering.

Conclusion: Curriculum learning with text simplification and complexity-based ordering enhances representation quality in data-constrained pretraining, with optimal strategies varying by model size.

Abstract: Most studies on language model pretraining focus on large datasets, leaving open questions about optimization in data-constrained settings. In such settings, the effects of training data order and of including alternative versions of the same text remain underexplored. We address this by studying curriculum learning in pretraining, focusing on text-complexity ordering and data augmentation via simplification. We ask: (1) Does simplifying texts enhance representation quality more than reusing the original data? and (2) Does ordering data by text complexity yield better representations? To answer, we build on a pair of parallel corpora where human-written paragraphs are aligned with LLM-simplified variants, and test four data schedules: repeated exposure, low-to-high complexity, high-to-low, and interleaved. We analyze models’ representation quality from a sample efficiency perspective via fine-tuning, as well as its zero-shot performance on linguistic knowledge, entity tracking, world knowledge, and commonsense reasoning. Our findings show that adding simplified data improves fine-tuning and zero-shot performance over a repeated-exposure baseline: smaller models benefit from low-to-high complexity, while larger models perform better with interleaved ordering.

Method: RMT framework with three key components: (1) dynamic token budget mechanism to limit unnecessary reasoning steps, (2) curriculum-based adaptive sampling for progressive learning from easy to hard tokens, (3) dual training strategy combining RL with next-token prediction.

Result: Achieves up to +64.91% performance improvement with only 21% of reasoning length in language modeling. Checkpoints from reinforcement mid-training also improve subsequent post-training by up to +18.76% in mathematical domain.

Conclusion: Reinforcement mid-training is a valuable intermediate stage in LLM development that significantly enhances performance and efficiency when properly implemented with the proposed RMT framework.

Abstract: The development of state-of-the-art large language models is commonly understood as a two-stage process involving pre-training and post-training. We point out the need for an additional intermediate stage called reinforcement mid-training with potential for strong performance gains. In this paper, we formally define the problem and identify three key challenges: (1) inefficient training due to excessive reasoning steps, (2) disregard of the imbalanced token entropy distribution, and (3) underutilization of token information. To address these challenges, we propose RMT, a framework for efficient, adaptive, and unified reinforcement mid-training with various innovative components. In particular, we first introduce a dynamic token budget mechanism that constrains unnecessary reasoning steps and mitigates model overthinking. Next, we design a curriculum-based adaptive sampling method that fosters a progressive learning trajectory from easy to hard tokens. Finally, we present a dual training strategy that combines reinforcement learning with next-token prediction, ensuring targeted learning on key tokens and full exploitation of all token information. Extensive experiments demonstrate the superiority of RMT over state-of-the-art methods, achieving up to +64.91% performance improvement with only 21% of the reasoning length in language modeling. We also show that checkpoints obtained after reinforcement mid-training can benefit the subsequent post-training, yielding up to +18.76% improvement in the mathematical domain.

Method: Created a comprehensive benchmark with a high-quality dataset of harmful prompts paired with diverse model responses, and implemented a flexible scoring mechanism compatible with various metrics and judges.

Result: Extensive experiments showed that traditional metrics (METEOR and ROUGE-1) outperform LLM-based judges in evaluating harmfulness of model responses.

Conclusion: Challenges prevailing beliefs about LLMs’ superiority in harmfulness evaluation and provides a publicly available benchmark for future research.

Abstract: The alignment of large language models (LLMs) with human values is critical for their safe deployment, yet jailbreak attacks can subvert this alignment to elicit harmful outputs from LLMs. In recent years, a proliferation of jailbreak attacks has emerged, accompanied by diverse metrics and judges to assess the harmfulness of the LLM outputs. However, the absence of a systematic benchmark to assess the quality and effectiveness of these metrics and judges undermines the credibility of the reported jailbreak effectiveness and other risks. To address this gap, we introduce HarmMetric Eval, a comprehensive benchmark designed to support both overall and fine-grained evaluation of harmfulness metrics and judges. Our benchmark includes a high-quality dataset of representative harmful prompts paired with diverse harmful and non-harmful model responses, alongside a flexible scoring mechanism compatible with various metrics and judges. With HarmMetric Eval, our extensive experiments uncover a surprising result: two conventional metrics–METEOR and ROUGE-1–outperform LLM-based judges in evaluating the harmfulness of model responses, challenging prevailing beliefs about LLMs’ superiority in this domain. Our dataset is publicly available at https://huggingface.co/datasets/qusgo/HarmMetric_Eval, and the code is available at https://anonymous.4open.science/r/HarmMetric-Eval-4CBE.

Method: Built LLaDA-MoE from scratch using Mixture-of-Experts architecture, trained on approximately 20T tokens, maintaining 7B total parameters but activating only 1.4B parameters during inference through sparse activation.

Result: Achieves state-of-the-art performance among diffusion language models, surpassing previous models like LLaDA, LLaDA 1.5, and Dream across multiple benchmarks. The instruct-tuned version (LLaDA-MoE-7B-A1B-Instruct) performs comparably to Qwen2.5-3B-Instruct in various tasks despite using fewer active parameters.

Conclusion: Integrating sparse MoE architecture into masked diffusion language models successfully brings out MoE’s strengths for efficient inference with few active parameters, opening new possibilities for further exploration of diffusion language models.

Abstract: We introduce LLaDA-MoE, a large language diffusion model with the Mixture-of-Experts (MoE) architecture, trained from scratch on approximately 20T tokens. LLaDA-MoE achieves competitive performance with significantly reduced computational overhead by maintaining a 7B-parameter capacity while activating only 1.4B parameters during inference. Our empirical evaluation reveals that LLaDA-MoE achieves state-of-the-art performance among diffusion language models with larger parameters, surpassing previous diffusion language models LLaDA, LLaDA 1.5, and Dream across multiple benchmarks. The instruct-tuned model LLaDA-MoE-7B-A1B-Instruct demonstrates capabilities comparable to Qwen2.5-3B-Instruct in knowledge understanding, code generation, mathematical reasoning, agent and alignment tasks, despite using fewer active parameters. Our results show that integrating a sparse MoE architecture into the training objective of masked diffusion language models still brings out MoE’s strengths under efficient inference with few active parameters, and opens ample room for further exploration of diffusion language models. LLaDA-MoE models are available at Huggingface.

Method: Orchestrated Test-Time Scaling strategy combining: i) Internal Scaling via RL-enhanced Intrinsic Reasoning, ii) Sequential Scaling through Iterative Refinement, and iii) Parallel Scaling using Diverse Synthesis and Tournament Selection.

Result: Achieves 81.67% execution accuracy on BIRD benchmark test set, ranking first on the official leaderboard.

Conclusion: Agentar-Scale-SQL provides an effective framework toward human-level performance in Text-to-SQL tasks and is designed for easy adaptation to new databases and more powerful language models.

Abstract: State-of-the-art (SOTA) Text-to-SQL methods still lag significantly behind human experts on challenging benchmarks like BIRD. Current approaches that explore test-time scaling lack an orchestrated strategy and neglect the model’s internal reasoning process. To bridge this gap, we introduce Agentar-Scale-SQL, a novel framework leveraging scalable computation to improve performance. Agentar-Scale-SQL implements an Orchestrated Test-Time Scaling strategy that synergistically combines three distinct perspectives: i) Internal Scaling via RL-enhanced Intrinsic Reasoning, ii) Sequential Scaling through Iterative Refinement, and iii) Parallel Scaling using Diverse Synthesis and Tournament Selection. Agentar-Scale-SQL is a general-purpose framework designed for easy adaptation to new databases and more powerful language models. Extensive experiments show that Agentar-Scale-SQL achieves SOTA performance on the BIRD benchmark, reaching 81.67% execution accuracy on the test set and ranking first on the official leaderboard, demonstrating an effective path toward human-level performance.

Method: Extended Spider 2.0 benchmark to 8 languages (English, German, French, Spanish, Portuguese, Japanese, Chinese, Vietnamese) while preserving structural difficulty but adding linguistic variability.

Result: State-of-the-art LLMs achieved only 4% execution accuracy on MultiSpider 2.0 using intrinsic reasoning, versus 60% on MultiSpider 1.0. A collaboration-driven language agents approach improved accuracy to 15%.

Conclusion: There is a substantial multilingual gap in Text-to-SQL performance, motivating the need for methods robust across languages and ready for real-world enterprise deployment.

Abstract: Text-to-SQL enables natural access to databases, yet most benchmarks are English-only, limiting multilingual progress. We introduce MultiSpider 2.0, extending Spider 2.0 to eight languages (English, German, French, Spanish, Portuguese, Japanese, Chinese, Vietnamese). It preserves Spider 2.0’s structural difficulty while adding linguistic and dialectal variability, demanding deeper reasoning for complex SQL. On this benchmark, state-of-the-art LLMs (such as DeepSeek-R1 and OpenAI o1) reach only 4% execution accuracy when relying on intrinsic reasoning, versus 60% on MultiSpider 1.0. Therefore, we provide a collaboration-driven language agents baseline that iteratively refines queries, improving accuracy to 15%. These results reveal a substantial multilingual gap and motivate methods that are robust across languages and ready for real-world enterprise deployment. Our benchmark is available at https://github.com/phkhanhtrinh23/Multilingual_Text_to_SQL.

Method: Proposed CDT framework that measures model capabilities across three dimensions (cognition, domain, task), incorporating Cattell-Horn-Carroll cognitive theory to refine capability categorization. Applied CDT for dataset capability evaluation and data selection.

Result: Capability metrics correlated well with downstream performance and supported effective dataset analysis. Data selection experiments showed significant improvements: 44.3 and 45.4 scores with 1.6 and 2.2 point increases over baselines on general and specific benchmarks respectively.

Conclusion: The CDT framework is validated as effective and practical for comprehensive LLM capability assessment, with demonstrated improvements in dataset analysis and construction.

Abstract: Recent advances in Large Language Models (LLMs) have significantly enhanced their capabilities, highlighting the need for comprehensive evaluation frameworks that extend beyond task-specific benchmarks. However, existing benchmarks often focus on isolated abilities, lacking a holistic framework for assessing LLM capabilities. To address this gap, we propose the Cognition-Domain-Task (CDT) framework, which comprehensively measures a model’s capabilities across three dimensions. We expand the scope of model capability definitions at the cognitive level by incorporating the Cattell-Horn-Carroll cognitive theory, refining the categorization of model capabilities. We apply CDT in two directions: dataset capability evaluation and data selection. Experiments show that our capability metrics correlate well with downstream performance and can support effective dataset analysis and construction. The experiments on data selection also show significant improvements in both general and specific benchmarks, achieving scores of 44.3 and 45.4, with an increase of 1.6 and 2.2 points over the baselines, respectively. These results validate the effectiveness and practicality of CDT. Source code and models are available at https://github.com/Alessa-mo/CDT.

Method: The survey categorizes NTP alternatives into five main families: Multi-Token Prediction, Plan-then-Generate, Latent Reasoning, Continuous Generation Approaches, and Non-Transformer Architectures.

Result: A comprehensive taxonomy of emerging alternatives to NTP is presented, synthesizing insights across different approaches to address token-level generation limitations.

Conclusion: The survey provides a framework to guide research into models that overcome NTP’s limitations and develop transformative NLP models through alternative generation paradigms.

Abstract: The paradigm of Next Token Prediction (NTP) has driven the unprecedented success of Large Language Models (LLMs), but is also the source of their most persistent weaknesses such as poor long-term planning, error accumulation, and computational inefficiency. Acknowledging the growing interest in exploring alternatives to NTP, the survey describes the emerging ecosystem of alternatives to NTP. We categorise these approaches into five main families: (1) Multi-Token Prediction, which targets a block of future tokens instead of a single one; (2) Plan-then-Generate, where a global, high-level plan is created upfront to guide token-level decoding; (3) Latent Reasoning, which shifts the autoregressive process itself into a continuous latent space; (4) Continuous Generation Approaches, which replace sequential generation with iterative, parallel refinement through diffusion, flow matching, or energy-based methods; and (5) Non-Transformer Architectures, which sidestep NTP through their inherent model structure. By synthesizing insights across these methods, this survey offers a taxonomy to guide research into models that address the known limitations of token-level generation to develop new transformative models for natural language processing.

Method: Proposed SOBACO (SOcial BiAs and Cultural cOmmonsense benchmark), a Japanese benchmark that evaluates social biases and cultural commonsense in LLMs using a unified format. Evaluated several LLMs on SOBACO to examine how debiasing methods affect cultural commonsense.

Result: Debiasing methods significantly degraded LLM performance on cultural commonsense tasks, with up to 75% accuracy deterioration observed across the evaluated models.

Conclusion: There is a critical trade-off between bias mitigation and cultural commonsense preservation. The findings highlight the importance of developing debiasing methods that consider this trade-off to improve both fairness and utility of LLMs.

Abstract: Large language models (LLMs) exhibit social biases, prompting the development of various debiasing methods. However, debiasing methods may degrade the capabilities of LLMs. Previous research has evaluated the impact of bias mitigation primarily through tasks measuring general language understanding, which are often unrelated to social biases. In contrast, cultural commonsense is closely related to social biases, as both are rooted in social norms and values. The impact of bias mitigation on cultural commonsense in LLMs has not been well investigated. Considering this gap, we propose SOBACO (SOcial BiAs and Cultural cOmmonsense benchmark), a Japanese benchmark designed to evaluate social biases and cultural commonsense in LLMs in a unified format. We evaluate several LLMs on SOBACO to examine how debiasing methods affect cultural commonsense in LLMs. Our results reveal that the debiasing methods degrade the performance of the LLMs on the cultural commonsense task (up to 75% accuracy deterioration). These results highlight the importance of developing debiasing methods that consider the trade-off with cultural commonsense to improve fairness and utility of LLMs.

Method: Proposed a novel alignment algorithm that couples dynamic programming with beam search scoring for precise alignment between reference and model transcripts.

Result: The approach provides more accurate alignment of individual errors compared to traditional text alignment methods, enabling reliable error analysis.

Conclusion: The algorithm addresses the need for finer-grained error analysis in speech recognition and is made available via PyPI.

Abstract: Modern neural networks have greatly improved performance across speech recognition benchmarks. However, gains are often driven by frequent words with limited semantic weight, which can obscure meaningful differences in word error rate, the primary evaluation metric. Errors in rare terms, named entities, and domain-specific vocabulary are more consequential, but remain hidden by aggregate metrics. This highlights the need for finer-grained error analysis, which depends on accurate alignment between reference and model transcripts. However, conventional alignment methods are not designed for such precision. We propose a novel alignment algorithm that couples dynamic programming with beam search scoring. Compared to traditional text alignment methods, our approach provides more accurate alignment of individual errors, enabling reliable error analysis. The algorithm is made available via PyPI.

Method: Self-Sanitize comprises a lightweight Self-Monitor module that continuously inspects high-level intentions via representation engineering, and a Self-Repair module that performs in-place correction of harmful content without separate review dialogues.

Result: Extensive experiments on four LLMs across three privacy leakage scenarios demonstrate that Self-Sanitize achieves superior mitigation performance with minimal overhead and without degrading LLM utility.

Conclusion: Self-Sanitize offers a practical and robust solution for safer LLM deployments by enabling real-time streaming monitoring and seamless repair with negligible impact on latency and resource utilization.

Abstract: As Large Language Models (LLMs) achieve remarkable success across a wide range of applications, such as chatbots and code copilots, concerns surrounding the generation of harmful content have come increasingly into focus. Despite significant advances in aligning LLMs with safety and ethical standards, adversarial prompts can still be crafted to elicit undesirable responses. Existing mitigation strategies are predominantly based on post-hoc filtering, which introduces substantial latency or computational overhead, and is incompatible with token-level streaming generation. In this work, we introduce Self-Sanitize, a novel LLM-driven mitigation framework inspired by cognitive psychology, which emulates human self-monitor and self-repair behaviors during conversations. Self-Sanitize comprises a lightweight Self-Monitor module that continuously inspects high-level intentions within the LLM at the token level via representation engineering, and a Self-Repair module that performs in-place correction of harmful content without initiating separate review dialogues. This design allows for real-time streaming monitoring and seamless repair, with negligible impact on latency and resource utilization. Given that privacy-invasive content has often been insufficiently focused in previous studies, we perform extensive experiments on four LLMs across three privacy leakage scenarios. The results demonstrate that Self-Sanitize achieves superior mitigation performance with minimal overhead and without degrading the utility of LLMs, offering a practical and robust solution for safer LLM deployments. Our code is available at the following link: https://github.com/wjfu99/LLM_Self_Sanitize

Method: Propose GRPO-MA which leverages multi-answer generation from each thought process, enabling more robust and efficient optimization through variance reduction in thought advantage.

Result: Empirical analysis shows GRPO-MA reduces gradient spikes compared to GRPO. Experiments on math, code, and multimodal tasks demonstrate substantial performance and training efficiency improvements.

Conclusion: Increasing the number of answers per thought consistently enhances model performance, making GRPO-MA a more effective approach for training Chain-of-Thought reasoning in LLMs and VLMs.

Abstract: Recent progress, such as DeepSeek-R1, has shown that the GRPO algorithm, a Reinforcement Learning (RL) approach, can effectively train Chain-of-Thought (CoT) reasoning in Large Language Models (LLMs) and Vision-Language Models (VLMs). In this paper, we analyze three challenges of GRPO: gradient coupling between thoughts and answers, sparse reward signals caused by limited parallel sampling, and unstable advantage estimation. To mitigate these challenges, we propose GRPO-MA, a simple yet theoretically grounded method that leverages multi-answer generation from each thought process, enabling more robust and efficient optimization. Theoretically, we show that the variance of thought advantage decreases as the number of answers per thought increases. Empirically, our gradient analysis confirms this effect, showing that GRPO-MA reduces gradient spikes compared to GRPO. Experiments on math, code, and diverse multimodal tasks demonstrate that GRPO-MA substantially improves performance and training efficiency. Our ablation studies further reveal that increasing the number of answers per thought consistently enhances model performance.

Method: SUIT identifies and modifies only the subspace of critical features relevant to the edit, rather than modifying entire MLP layers.

Result: Empirical results on LLaMA-3-8B, GPT-J-6B, and Qwen2.5-7B show SUIT dramatically improves knowledge preservation over strong baselines while maintaining high edit efficacy.

Conclusion: SUIT successfully identifies the critical subspace for edits, providing an effective approach for knowledge editing with minimal model perturbation.

Abstract: Knowledge editing aims to efficiently correct factual errors in Language Models (LMs). The popular locate-then-edit approach modifies an MLP layer by finding an optimal mapping between its input vector (key) and output vector (value) that leads to the expression of the edited knowledge. However, existing methods without any constraints on the key and value vectors cause significant perturbations to the edited model. To address this, we propose Subspace Knowledge Edit (SUIT), a method that identifies and modifies only the subspace of critical features relevant to the edit. Our empirical results on LLaMA-3-8B, GPT-J-6B, and Qwen2.5-7B models show that SUIT dramatically improves knowledge preservation over strong baselines while maintaining high edit efficacy. This effectiveness confirms that SUIT successfully identifies the critical subspace for the edit. Further analyses provide additional validation for our approach. The source code and data will be released to the public upon publication of the paper.

Method: Co-created with civil-society organizations and community members, the pipeline uses community feedback to determine evaluation criteria, benchmark construction, and output scoring.

Result: The approach demonstrates how multilingual LLMs handle nuanced community health queries while providing a scalable method for contextually grounded evaluation.

Conclusion: Samiksha offers a scalable pathway for more inclusive and culturally aware LLM evaluation that better reflects real-world community contexts.

Abstract: Large Language Models (LLMs) are typically evaluated through general or domain-specific benchmarks testing capabilities that often lack grounding in the lived realities of end users. Critical domains such as healthcare require evaluations that extend beyond artificial or simulated tasks to reflect the everyday needs, cultural practices, and nuanced contexts of communities. We propose Samiksha, a community-driven evaluation pipeline co-created with civil-society organizations (CSOs) and community members. Our approach enables scalable, automated benchmarking through a culturally aware, community-driven pipeline in which community feedback informs what to evaluate, how the benchmark is built, and how outputs are scored. We demonstrate this approach in the health domain in India. Our analysis highlights how current multilingual LLMs address nuanced community health queries, while also offering a scalable pathway for contextually grounded and inclusive LLM evaluation.

Method: Generates multiple candidate outputs, evaluates correctness and uncertainty, estimates problem difficulty via uncertainty-guided length calibration, penalizes long reasoning for easy questions, and extends chain of thought for difficult ones.

Result: Achieves up to 6.4x length reduction on average across six medical QA benchmarks while maintaining performance with minimal degradation. Spontaneously develops “non-thinking” and “thinking” reasoning modes.

Conclusion: AdaThink-Med successfully enables adaptive thinking in medical LLMs, dynamically suppressing redundant reasoning while preserving performance, making medical AI more computationally efficient.

Abstract: Recent advances in inference time scaling with extended long chain-of thought have significantly improved the reasoning capabilities of both general and medical large language models (LLMs). However, these models tend to engage in lengthy reasoning processes regardless of the difficulty of the input question, leading to increased inference costs in real-world applications. Therefore, enabling adaptive thinking where models think less for simpler questions and think more for complex ones is critical for the effective use of medical LLMs in practice. Despite its importance, there is a lack of end-to-end approaches designed to enhance the adaptive thinking capabilities of medical LLMs while providing a comprehensive examination of the trade-off between performance and computational cost. To bridge this gap, we propose AdaThink-Med, the first end-to-end framework designed to enhance adaptive thinking ability in medical reasoning models with uncertainty-guided length calibration. AdaThink-Med first generates multiple candidate outputs for each question, evaluates the correctness and uncertainty of each candidate, and then estimates problem difficulty via an uncertainty-guided length calibration module. For outputs with low difficulty and correct answers, the framework penalizes longer reasoning paths; whereas for those with high difficulty and incorrect answers, it encourages extending the chain of thought to explore alternative solutions. On six public medical QA benchmarks, AdaThink-Med achieves up to 6.4x length reduction on average while retaining performance with only minimal degradation. Intriguingly, we observe that AdaThink-Med spontaneously develops two distinct reasoning modes, which we characterize as “non-thinking” and “thinking”, demonstrating the model’s ability to suppress redundant reasoning processes dynamically.

Method: Identified visual-word-form-selective units in large-scale vision-language models and performed targeted ablation of these units, comparing with ablation of random units.

Result: Targeted ablation of word-form-selective units caused selective reading impairments matching dyslexic humans’ phonological deficits, while general visual and language comprehension remained intact.

Conclusion: The modeling approach successfully replicated key dyslexia characteristics and established a computational framework for investigating reading disorders.

Abstract: Dyslexia, a neurodevelopmental disorder characterized by persistent reading difficulties, is often linked to reduced activity of the visual word form area in the ventral occipito-temporal cortex. Traditional approaches to studying dyslexia, such as behavioral and neuroimaging methods, have provided valuable insights but remain limited in their ability to test causal hypotheses about the underlying mechanisms of reading impairments. In this study, we use large-scale vision-language models (VLMs) to simulate dyslexia by functionally identifying and perturbing artificial analogues of word processing. Using stimuli from cognitive neuroscience, we identify visual-word-form-selective units within VLMs and demonstrate that targeted ablation of these units, unlike ablation of random units, leads to selective impairments in reading tasks while general visual and language comprehension abilities remain intact. In particular, the resulting model matches dyslexic humans’ phonological deficits without a significant change in orthographic processing. Taken together, our modeling results replicate key characteristics of dyslexia and establish a computational framework for investigating reading disorders.

Method: Developed formalized guidelines for identifying hype language, annotated NIH grant applications, and evaluated traditional text classifiers and language models against human baseline.

Result: Formalized annotation guidelines helped humans reliably annotate hype adjectives, and machine learning models trained on the annotated dataset showed promising results for hype detection.

Conclusion: Hype detection is linguistically complex and may require domain knowledge and temporal awareness, representing the first NLP approach to this problem.

Abstract: In science, promotional language (‘hype’) is increasing and can undermine objective evaluation of evidence, impede research development, and erode trust in science. In this paper, we introduce the task of automatic detection of hype, which we define as hyperbolic or subjective language that authors use to glamorize, promote, embellish, or exaggerate aspects of their research. We propose formalized guidelines for identifying hype language and apply them to annotate a portion of the National Institutes of Health (NIH) grant application corpus. We then evaluate traditional text classifiers and language models on this task, comparing their performance with a human baseline. Our experiments show that formalizing annotation guidelines can help humans reliably annotate candidate hype adjectives and that using our annotated dataset to train machine learning models yields promising results. Our findings highlight the linguistic complexity of the task, and the potential need for domain knowledge and temporal awareness of the facts. While some linguistic works address hype detection, to the best of our knowledge, we are the first to approach it as a natural language processing task.

Method: Parameter-free architecture modification that reuses dense attention parameters, using dense attention for short sequences and automatically switching to sparse attention for long sequences with an efficient implementation to reduce computational overhead.

Result: InfLLM-V2 achieves 4x speedup over dense attention while retaining 98.1% performance on long-context understanding and 99.7% on chain-of-thought reasoning. The framework was used to train and open-source MiniCPM4.1 model.

Conclusion: InfLLM-V2 provides an effective solution for long-sequence processing that maintains performance while significantly improving efficiency, enabling practical acceleration of large language models for long-context tasks.

Abstract: Long-sequence processing is a critical capability for modern large language models. However, the self-attention mechanism in the standard Transformer architecture faces severe computational and memory bottlenecks when processing long sequences. While trainable sparse attention methods offer a promising solution, existing approaches such as NSA introduce excessive extra parameters and disrupt the conventional \textit{pretrain-on-short, finetune-on-long} workflow, resulting in slow convergence and difficulty in acceleration. To overcome these limitations, we introduce dense-sparse switchable attention framework, termed as InfLLM-V2. InfLLM-V2 is a trainable sparse attention that seamlessly adapts models from short to long sequences. Specifically, InfLLM-V2 reuses dense attention parameters through parameter-free architecture modification, maintaining consistency between short and long sequence processing. Additionally, InfLLM-V2 ensures computational efficiency across all sequence lengths, by using dense attention for short inputs and smoothly transitioning to sparse attention for long sequences. To achieve practical acceleration, we further introduce an efficient implementation of InfLLM-V2 that significantly reduces the computational overhead. Our experiments on long-context understanding and chain-of-thought reasoning demonstrate that InfLLM-V2 is 4$\times$ faster than dense attention while retaining 98.1% and 99.7% of the performance, respectively. Based on the InfLLM-V2 framework, we have trained and open-sourced MiniCPM4.1 (https://huggingface.co/openbmb/MiniCPM4.1-8B), a hybrid reasoning model, providing a reproducible implementation for the research community.

Method: Proposed unPact framework using prompt attribution and contribution tracking to quantify each prompt token’s influence on outputs, enabling pre- and post-unlearning comparisons across six methods, three LLMs, and three benchmarks.

Result: Found that unlearning works by disrupting keyword focus rather than true knowledge erasure; knowledge remains recoverable via keyword emphasis without weight modification; catastrophic forgetting stems from indiscriminate token penalization.

Conclusion: Existing unlearning methods face a fundamental dilemma between insufficient knowledge removal (recoverable) and overly destructive approaches (catastrophic forgetting), highlighting the gap to reliable unlearning.

Abstract: Unlearning seeks to remove specific knowledge from large language models (LLMs), but its effectiveness remains contested. On one side, “forgotten” knowledge can often be recovered through interventions such as light fine-tuning; on the other side, unlearning may induce catastrophic forgetting that degrades general capabilities. Despite active exploration of unlearning methods, interpretability analyses of the mechanism are scarce due to the difficulty of tracing knowledge in LLMs’ complex architectures. We address this gap by proposing unPact, an interpretable framework for unlearning via prompt attribution and contribution tracking. Typically, it quantifies each prompt token’s influence on outputs, enabling pre- and post-unlearning comparisons to reveal what changes. Across six mainstream unlearning methods, three LLMs, and three benchmarks, we find that: (1) Unlearning appears to be effective by disrupting focus on keywords in prompt; (2) Much of the knowledge is not truly erased and can be recovered by simply emphasizing these keywords in prompts, without modifying the model’s weights; (3) Catastrophic forgetting arises from indiscriminate penalization of all tokens. Taken together, our results suggest an unlearning dilemma: existing methods tend either to be insufficient - knowledge remains recoverable by keyword emphasis, or overly destructive - general performance collapses due to catastrophic forgetting, still leaving a gap to reliable unlearning.

Method: Proposes Latent Judges that derive scalar ratings from internal model signals: (i) probability-weighted scores over integer ratings, (ii) verifier-style probabilities of “yes”, and (iii) linear probes trained on model activations at the rating position.

Result: Across various benchmarks, latent methods match or surpass standard prompting, with consistent gains on pairwise accuracy and listwise ranking. Probability-weighted scores achieve strongest single-rating correlations, while probes work well when output logits are miscalibrated.

Conclusion: Latent information provides deterministic and more discriminative signals for reference-free evaluation, improving selection and training approaches like Best-of-N, multi-teacher distillation, and routing.

Abstract: How reliable are single-response LLM-as-a-judge ratings without references, and can we obtain fine-grained, deterministic scores in this setting? We study the common practice of asking a judge model to assign Likert-scale scores to free-text responses and show two systematic issues: scores are unstable under sampling and poorly calibrated, leading to compression near the top of the scale and frequent ties. We then propose and evaluate Latent Judges, which derive scalar ratings from internal model signals: (i) probability-weighted scores over integer ratings, (ii) verifier-style probabilities of “yes”, and (iii) linear probes trained on model activations at the rating position. Across a broad suite of pairwise and single-rating benchmarks, latent methods match or surpass standard prompting, with consistent gains on pairwise accuracy and listwise ranking relevant to Best-of-N selection. Probability-weighted scores achieve the strongest single-rating correlations, while probes recover useful signals when output logits are miscalibrated. These results indicate that latent information provides deterministic and more discriminative signals for reference-free evaluation, and can improve selection and training approaches like Best-of-$N$, multi-teacher distillation, and routing.

Method: MemGen consists of a memory trigger that monitors reasoning state to decide explicit memory invocation, and a memory weaver that constructs latent token sequences as machine-native memory to enrich reasoning, creating an interwoven cycle of memory and cognition.

Result: MemGen surpasses leading external memory systems (ExpeL, AWM) by up to 38.22%, exceeds GRPO by up to 13.44%, and exhibits strong cross-domain generalization. It spontaneously evolves distinct human-like memory faculties including planning, procedural, and working memory.

Conclusion: MemGen enables more naturalistic machine cognition by allowing agents to recall and augment latent memory throughout reasoning, suggesting an emergent trajectory toward human-like cognitive faculties without explicit supervision.

Abstract: Agent memory shapes how Large Language Model (LLM)-powered agents, akin to the human brain, progressively refine themselves through environment interactions. Existing paradigms remain constrained: parametric memory forcibly adjusts model parameters, and retrieval-based memory externalizes experience into structured databases, yet neither captures the fluid interweaving of reasoning and memory that underlies human cognition. To address this gap, we propose MemGen, a dynamic generative memory framework that equips agents with a human-esque cognitive faculty. It consists of a \textit{memory trigger}, which monitors the agent’s reasoning state to decide explicit memory invocation, and a \textit{memory weaver}, which takes the agent’s current state as stimulus to construct a latent token sequence as machine-native memory to enrich its reasoning. In this way, MemGen enables agents to recall and augment latent memory throughout reasoning, producing a tightly interwoven cycle of memory and cognition. Extensive experiments across eight benchmarks show that MemGen surpasses leading external memory systems such as ExpeL and AWM by up to $38.22%$, exceeds GRPO by up to $13.44%$, and exhibits strong cross-domain generalization ability. More importantly, we find that without explicit supervision, MemGen spontaneously evolves distinct human-like memory faculties, including planning memory, procedural memory, and working memory, suggesting an emergent trajectory toward more naturalistic forms of machine cognition.

Method: Uses three co-evolving agents: Solver (refines reasoning from preference feedback), Teacher (adaptively crafts challenging questions based on Solver’s weaknesses), and Generator (distills Teacher’s strategy for scalable curriculum generation). Forms a closed-loop system requiring only 100 seed questions.

Result: Socratic-Solver-8B achieves +20.2 percentage point gain over prior methods across 7 mathematical reasoning benchmarks. Synthetic data from Socratic-Generator-32B enables student LLMs to outperform SOTA commercial LLMs including Qwen3-235B, DeepSeek-V3.1-671B, GPT-5, Gemini-2.5-Pro, Grok-4, and Claude-4.1-Opus.

Conclusion: Socratic-Zero demonstrates that fully autonomous data generation through agent co-evolution can produce high-quality training curricula that significantly outperform existing methods and even surpass commercial SOTA models, enabling scalable reasoning capabilities without human annotation.

Abstract: Recent breakthroughs in large language models (LLMs) on reasoning tasks rely heavily on massive, high-quality datasets-typically human-annotated and thus difficult to scale. While data synthesis or distillation offers a promising alternative, existing methods struggle with inconsistent data quality and an inability to dynamically adapt to the evolving capabilities of the model, leading to suboptimal training signals. To address these limitations, we introduce Socratic-Zero, a fully autonomous framework that generates high-quality training data from minimal seed examples through the co-evolution of three agents: the Teacher, the Solver, and the Generator. The Solver continuously refines its reasoning by learning from preference feedback on both successful and failed trajectories; the Teacher adaptively crafts increasingly challenging questions based on the Solver’s weaknesses; and the Generator distills the Teacher’s question-design strategy to enable scalable, high-fidelity curriculum generation. This closed-loop system produces a self-improving curriculum-requiring no pre-existing tasks or labels. Remarkably, starting from only 100 seed questions, our Socratic-Solver-8B achieves an average gain of +20.2 percentage points over prior data synthesis methods across seven mathematical reasoning benchmarks (AMC23, AIME24-25, Olympiad, MATH-500, Minerva, and GSM8K), with consistent gains on both Qwen3 and GLM4 series models. Even more surprisingly, synthetic data from Socratic-Generator-32B enables student LLMs to achieve superior performance compared to other state-of-the-art (SOTA) commercial LLMs on these benchmarks, including Qwen3-235B-A22B, DeepSeek-V3.1-671B, GPT-5, Gemini-2.5-Pro, Grok-4, and Claude-4.1-Opus.

Method: Uses pooled representative heads to approximate scores for all heads based on observed similarity among attention heads. Proposes block-aware dynamic budget estimation to account for varying sparsity among heads. Combines proxy head scores with multi-head dynamic budgets for fine-grained block importance evaluation.

Result: Achieves up to 10.3x attention acceleration and 2.4x pre-filling acceleration without significant performance loss. Experiments confirm underlying similarity among attention heads and show substantial gains in performance and efficiency compared to existing methods.

Conclusion: ProxyAttn enables efficient fine-grained block importance estimation at low computational cost, providing significant acceleration for long-text tasks in LLMs while maintaining performance.

Abstract: The quadratic complexity of attention mechanisms limits the efficiency of Large Language Models (LLMs) on long-text tasks. Recently, methods that dynamically estimate block importance have enabled efficient block sparse attention, leading to significant acceleration in long-text pre-filling of LLMs. However, their coarse-grained estimation inevitably leads to performance degradation at high sparsity rates. In this work, we propose ProxyAttn, a training-free sparse attention algorithm that achieves more precise block estimation by compressing the dimension of attention heads. Based on our observation of the similarity among multiple attention heads, we use the scores of pooled representative heads to approximate the scores for all heads. To account for the varying sparsity among heads, we also propose a block-aware dynamic budget estimation method. By combining the scores from representative proxy heads with multi-head dynamic budgets, we achieve a more fine-grained block importance evaluation at low computational cost. Experiments on a variety of mainstream models and extensive benchmarks confirm the underlying similarity among attention heads. Leveraging a fine-grained estimation, the proposed method achieves substantial gains in performance and efficiency compared to existing methods. More precisely, ProxyAttn can achieve up to 10.3x attention acceleration and 2.4x prefilling acceleration without significant performance loss. Our code is available at https://github.com/wyxstriker/ProxyAttn.

Method: LatentEvolve uses a dual evolutionary system: daytime scaling (fast recall of historical latent representations) and nighttime scaling (integration of past latent optimizations), mimicking human brain’s hippocampus-neocortex system for unsupervised learning.

Result: LatentEvolve outperforms state-of-the-art TTS methods like LatentSeek and TTRL by up to 13.33% across eight benchmarks and five model backbones, showing exceptional cross-domain and cross-backbone generalization.

Conclusion: The framework successfully enables LLMs to self-evolve their test-time scaling capabilities through unsupervised learning, achieving significant performance improvements while maintaining generalization across different domains and model architectures.

Abstract: Test-time Scaling (TTS) has been demonstrated to significantly enhance the reasoning capabilities of Large Language Models (LLMs) during the inference phase without altering model parameters. However, existing TTS methods are largely independent, implying that LLMs have not yet evolved to progressively learn how to scale more effectively. With the objective of evolving LLMs to learn ``how to scale test-time computation,’’ we propose LatentEvolve, a self-evolving latent TTS framework inspired by the complementary learning system (CLS) theory. Analogous to the human brain’s dual system of a fast-recall hippocampus and a slow-consolidating neocortex, LatentEvolve comprises two evolutionary components: \textit{daytime scaling}, which rapidly retrieves historical latent representations to better guide current LLM reasoning; and \textit{nighttime scaling}, which integrates past latent optimizations in a manner akin to the human brain’s consolidation of experiences during sleep. The alternation of daytime and nighttime processes facilitates a fast and slow evolution of LLM TTS, mirroring human cognitive dynamics in a fully unsupervised manner. Extensive experiments across eight benchmarks and five model backbones demonstrate that our LatentEvolve surpasses state-of-the-art TTS methods such as LatentSeek and TTRL by up to $13.33%$ and exhibits exceptional cross-domain and cross-backbone generalization.

Method: Leverages three common LLM error types to introduce specific error patterns into preference optimization, ensuring negative samples are more erroneous than positive samples and using preference-based training to mitigate these errors.

Result: Evaluations across five capability dimensions and model scales (1.5B to 14B) show significant performance improvements, particularly in truthfulness (5-10 percentage points). Task performance varies with error types - common errors improve related tasks, while mixed errors provide broader enhancement.

Conclusion: Strategic error amplification through SeaPO effectively addresses the similarity problem in preference optimization, leading to substantial performance improvements across multiple model scales and capability dimensions.

Abstract: Existing alignment methods for preference optimization of large language models (LLMs) aim to enhance model performance by utilizing pairs of positive and negative samples. However, due to the limited capacity of models in scoring or generating responses, the quality of positive and negative samples may become similar during training, which complicates optimization for preference learning. To address this issue, we introduce SeaPO, a Strategic Error Amplification method that leverages three error types commonly occurring in LLMs to introduce specific error patterns into the model Preference Optimization. This strategy ensures that negative samples are more erroneous than positive samples and preference-based training is employed to mitigate the occurrence of these errors, thereby enhancing model performance. Evaluations across five capability dimensions and different model scales (1.5B to 14B) demonstrate that the generated data significantly improved overall model performance, particularly in terms of truthfulness, with improvements of 5-10 percentage points observed. Further analysis reveals that task performance varies depending on the error types introduced. Injecting the most common error types improves performance in related tasks, while a mix of error types leads to a broader performance enhancement: most tasks show stable improvements, while a few tasks exhibit significant gains.

Method: Developed a novel automatic tree-based evaluation metric specifically designed for LLM-generated step-by-step assembly instructions, focusing on spatiotemporal aspects of construction.

Result: The proposed metric shows better correlation with manually-annotated error counts and human quality ratings for sewing instructions, and demonstrates superior robustness against artificially-constructed counterfactual examples designed to confound text similarity metrics.

Conclusion: The tree-based metric is superior to traditional text similarity metrics for evaluating the spatiotemporal soundness of assembly instructions, particularly in domains requiring precise construction sequences.

Abstract: In this paper, we propose a novel, automatic tree-based evaluation metric for LLM-generated step-by-step assembly instructions, that more accurately reflects spatiotemporal aspects of construction than traditional metrics such as BLEU and BERT similarity scores. We apply our proposed metric to the domain of sewing instructions, and show that our metric better correlates with manually-annotated error counts as well as human quality ratings, demonstrating our metric’s superiority for evaluating the spatiotemporal soundness of sewing instructions. Further experiments show that our metric is more robust than traditional approaches against artificially-constructed counterfactual examples that are specifically constructed to confound metrics that rely on textual similarity.

Method: A two-stage “investigate-before-abstain” paradigm: 1) Evidence discovery through systematic knowledge graph exploration (graph expansion and direct retrieval), and 2) Evidence evaluation that ranks evidence using multiple factors to adapt exploration based on patient context and conversation history.

Result: KnowGuard outperforms state-of-the-art abstention approaches, improving diagnostic accuracy by 3.93% while reducing unnecessary interaction by 7.27 turns on average in open-ended multi-round clinical benchmarks.

Conclusion: The systematic knowledge graph exploration approach enables LLMs to better recognize insufficient medical evidence and make appropriate abstentions, improving both diagnostic accuracy and efficiency in clinical decision-making.

Abstract: In clinical practice, physicians refrain from making decisions when patient information is insufficient. This behavior, known as abstention, is a critical safety mechanism preventing potentially harmful misdiagnoses. Recent investigations have reported the application of large language models (LLMs) in medical scenarios. However, existing LLMs struggle with the abstentions, frequently providing overconfident responses despite incomplete information. This limitation stems from conventional abstention methods relying solely on model self-assessments, which lack systematic strategies to identify knowledge boundaries with external medical evidences. To address this, we propose \textbf{KnowGuard}, a novel \textit{investigate-before-abstain} paradigm that integrates systematic knowledge graph exploration for clinical decision-making. Our approach consists of two key stages operating on a shared contextualized evidence pool: 1) an evidence discovery stage that systematically explores the medical knowledge space through graph expansion and direct retrieval, and 2) an evidence evaluation stage that ranks evidence using multiple factors to adapt exploration based on patient context and conversation history. This two-stage approach enables systematic knowledge graph exploration, allowing models to trace structured reasoning paths and recognize insufficient medical evidence. We evaluate our abstention approach using open-ended multi-round clinical benchmarks that mimic realistic diagnostic scenarios, assessing abstention quality through accuracy-efficiency trade-offs beyond existing closed-form evaluations. Experimental evidences clearly demonstrate that KnowGuard outperforms state-of-the-art abstention approaches, improving diagnostic accuracy by 3.93% while reducing unnecessary interaction by 7.27 turns on average.

Method: Adapted the initiation-response-evaluation (IRE) framework from educational theory and developed a unique graph-based encoding method that integrates teacher questions with relevant knowledge components to capture key information.

Result: Experiments on three real-world dialogue datasets show DiaCDM significantly improves diagnostic accuracy and enhances result interpretability compared to traditional methods.

Conclusion: DiaCDM provides teachers with a powerful tool for assessing students’ cognitive states in dialogue settings, representing the first exploration of cognitive diagnosis in dialogue contexts.

Abstract: While cognitive diagnosis (CD) effectively assesses students’ knowledge mastery from structured test data, applying it to real-world teacher-student dialogues presents two fundamental challenges. Traditional CD models lack a suitable framework for handling dynamic, unstructured dialogues, and it’s difficult to accurately extract diagnostic semantics from lengthy dialogues. To overcome these hurdles, we propose DiaCDM, an innovative model. We’ve adapted the initiation-response-evaluation (IRE) framework from educational theory to design a diagnostic framework tailored for dialogue. We also developed a unique graph-based encoding method that integrates teacher questions with relevant knowledge components to capture key information more precisely. To our knowledge, this is the first exploration of cognitive diagnosis in a dialogue setting. Experiments on three real-world dialogue datasets confirm that DiaCDM not only significantly improves diagnostic accuracy but also enhances the results’ interpretability, providing teachers with a powerful tool for assessing students’ cognitive states. The code is available at https://github.com/Mind-Lab-ECNU/DiaCDM/tree/main.

Method: Uses fuzzy token matching with locality-sensitive hashing (LSH) on token embeddings and incorporates Rotary Position Embedding (RoPE) to preserve positional information, selectively reusing relevant key-value pairs from reference prompts’ caches.

Result: Achieves up to 6.25x speedup and 42% lower GPU memory usage with 5k tokens input on summarization datasets, with negligible quality degradation.

Conclusion: Semantic-aware cache sharing shows significant potential for efficient LLM inference by reducing redundant computation while maintaining output quality.

Abstract: As large language models (LLMs) continue to scale, the memory footprint of key-value (KV) caches during inference has become a significant bottleneck. Existing approaches primarily focus on compressing KV caches within a single prompt or reusing shared prefixes or frequently ocurred text segments across prompts. However, such strategies are limited in scenarios where prompts are semantically similar but lexically different, which frequently occurs in tasks such as multi-document summarization and conversational agents. We propose \textit{SemShareKV}, a KV cache sharing and compression framework that accelerates LLM inference by reusing KVCache in semantically similar prompts. Instead of relying on exact token matches, SemShareKV applies fuzzy token matching using locality-sensitive hashing (LSH) on token embeddings and incorporates Rotary Position Embedding (RoPE) to better preserve positional information. By selectively reusing relevant key-value pairs from a reference prompt’s cache, SemShareKV reduces redundant computation while maintaining output quality. Experiments on diverse summarization datasets show up to 6.25$\times$ speedup and 42% lower GPU memory usage with 5k tokens input, with negligible quality degradation. These results highlight the potential of semantic-aware cache sharing for efficient LLM inference.

Method: Developed a three-stage Hierarchical Error Correction framework based on analysis of error patterns across four specialized domains, addressing Knowledge-layer (58.4%), Reasoning-layer (39.6%), and Complexity-layer (2.0%) errors according to hierarchical importance.

Result: Experimental validation across medical transcription (4,921 cases), legal document classification (1,000 cases), political bias detection (645 cases), and legal reasoning (1,000 cases) shows consistent improvements with average 11.2 percentage point gains (p < 0.001) across five LLM architectures.

Conclusion: Systematic error analysis can guide effective AI enhancement strategies in specialized domains, particularly for moderate-baseline tasks, while framework limitations exist in high-baseline tasks (>75% accuracy) where hierarchical intervention may interfere with reasoning processes.

Abstract: Large Language Models face significant performance challenges in specialized domains, with state-of-the-art models achieving only 45.9% accuracy on medical coding tasks. This study proposes a Hierarchical Error Correction (HEC) framework that addresses domain-specific AI limitations through systematic error analysis and targeted intervention strategies. We analyze error patterns across four specialized domains and find that AI errors follow consistent hierarchical structures: Knowledge-layer errors (58.4%), Reasoning-layer errors (39.6%), and Complexity-layer errors (2.0%). Based on these patterns, we develop a three-stage correction framework that addresses errors according to their hierarchical importance and demonstrates that framework effectiveness correlates inversely with baseline task performance. Experimental validation across medical transcription (4,921 cases), legal document classification (1,000 cases), political bias detection (645 cases), and legal reasoning (1,000 cases) shows consistent improvements. Cross-model validation across five LLM architectures demonstrates average improvements of 11.2 percentage points (p < 0.001). However, analysis reveals framework limitations in high-baseline tasks (>75% accuracy), where hierarchical intervention may interfere with effective reasoning processes. The results suggest that systematic error analysis can guide effective AI enhancement strategies in specialized domains, particularly for moderate-baseline tasks, while highlighting the importance of understanding framework boundaries for optimal deployment.

Method: Introduced a unified taxonomy of six clinically-informed mental health crisis categories, curated a diverse evaluation dataset, established expert-designed assessment protocols, and systematically benchmarked three state-of-the-art LLMs for crisis classification and response generation.

Result: LLMs are highly consistent and generally reliable for explicit crisis disclosures, but show significant risks: non-negligible proportion of inappropriate/harmful responses, higher failure rates in open-weight models, systemic weaknesses in handling indirect risk signals, formulaic replies, and frequent context misalignment.

Conclusion: Urgent need for enhanced safeguards, improved crisis detection, and context-aware interventions in LLM deployments. The taxonomy, datasets, and evaluation framework provide groundwork for responsible AI-driven mental health support innovation.

Abstract: The widespread use of chatbots powered by large language models (LLMs) such as ChatGPT and Llama has fundamentally reshaped how people seek information and advice across domains. Increasingly, these chatbots are being used in high-stakes contexts, including emotional support and mental health concerns. While LLMs can offer scalable support, their ability to safely detect and respond to acute mental health crises remains poorly understood. Progress is hampered by the absence of unified crisis taxonomies, robust annotated benchmarks, and empirical evaluations grounded in clinical best practices. In this work, we address these gaps by introducing a unified taxonomy of six clinically-informed mental health crisis categories, curating a diverse evaluation dataset, and establishing an expert-designed protocol for assessing response appropriateness. We systematically benchmark three state-of-the-art LLMs for their ability to classify crisis types and generate safe, appropriate responses. The results reveal that while LLMs are highly consistent and generally reliable in addressing explicit crisis disclosures, significant risks remain. A non-negligible proportion of responses are rated as inappropriate or harmful, with responses generated by an open-weight model exhibiting higher failure rates than those generated by the commercial ones. We also identify systemic weaknesses in handling indirect or ambiguous risk signals, a reliance on formulaic and inauthentic default replies, and frequent misalignment with user context. These findings underscore the urgent need for enhanced safeguards, improved crisis detection, and context-aware interventions in LLM deployments. Our taxonomy, datasets, and evaluation framework lay the groundwork for ongoing research and responsible innovation in AI-driven mental health support, helping to minimize harm and better protect vulnerable users.

Method: Compared three approaches: retrieval-augmented generation (RAG) with codebook, prompt engineering (zero-shot, few-shot, chain-of-thought), and fine-tuning on hand-coded texts.

Result: State-of-the-art closed-source LLMs achieved high accuracy, with fine-tuning yielding median F1 score of 0.79. Discrepancies between human and LLM outputs were systematic, reflecting known theoretical grey areas.

Conclusion: LLMs can partly automate metaphor identification and serve as testbeds for developing and refining metaphor identification protocols and underlying theory.

Abstract: Metaphor is a pervasive feature of discourse and a powerful lens for examining cognition, emotion, and ideology. Large-scale analysis, however, has been constrained by the need for manual annotation due to the context-sensitive nature of metaphor. This study investigates the potential of large language models (LLMs) to automate metaphor identification in full texts. We compare three methods: (i) retrieval-augmented generation (RAG), where the model is provided with a codebook and instructed to annotate texts based on its rules and examples; (ii) prompt engineering, where we design task-specific verbal instructions; and (iii) fine-tuning, where the model is trained on hand-coded texts to optimize performance. Within prompt engineering, we test zero-shot, few-shot, and chain-of-thought strategies. Our results show that state-of-the-art closed-source LLMs can achieve high accuracy, with fine-tuning yielding a median F1 score of 0.79. A comparison of human and LLM outputs reveals that most discrepancies are systematic, reflecting well-known grey areas and conceptual challenges in metaphor theory. We propose that LLMs can be used to at least partly automate metaphor identification and can serve as a testbed for developing and refining metaphor identification protocols and the theory that underpins them.

Method: Identify effective token types, numbers, and insertion locations; examine when models learn to use expanded computation space during training; analyze attention dynamics in these spaces via attention maps.

Result: Appropriate token types and counts vary, but placing filler tokens directly before the final ‘Answer:’ token is most effective. Smaller models benefit most (up to 12.372% improvement), and attention maps show expanded spaces continue original attention patterns and focus on questions/answers.

Conclusion: Filler tokens act as additional computational capacity rather than redundant input, with meaningful computation occurring in these expanded spaces for problem-solving.

Abstract: Chain-of-thought (CoT) rationale enables language models to use additional task-related text for problem-solving, benefiting not only from detailed reasoning steps but also from the expanded computational space of longer inputs. Prior work has trained filler or special tokens to serve as additional computation spaces. In this study, we investigate whether language models can leverage artificially inserted sequences of filler tokens solely at inference. We first identify effective token types, numbers, and insertion locations, then examine at what stage of training models begin to exploit the expanded computation space, and finally analyze dynamics within these spaces via attention maps. Experiments on models ranging from 1.7B to 32B across open-domain QA and math tasks show that appropriate token types and counts vary, but placing filler tokens directly before the final ‘Answer:’ token is most effective. Smaller models benefit most, up to 12.372 percentage points in SmolLM2-1.7B-Instruct, indicating that these spaces act as additional computational capacity rather than redundant input. Attention maps reveal that expanded spaces often continue the original attention mechanism and sometimes focus on questions or answer options, suggesting meaningful computation for problem-solving.

Method: Created BOE-XSUM dataset with short summaries, original texts, and document types. Fine-tuned medium-sized LLMs on this dataset and compared them to zero-shot general-purpose models.

Result: Fine-tuned models significantly outperformed zero-shot models. BERTIN GPT-J 6B achieved 41.6% accuracy vs 33.5% for DeepSeek-R1, representing a 24% performance gain.

Conclusion: Specialized fine-tuning on domain-specific datasets like BOE-XSUM substantially improves Spanish legal document summarization compared to using general models in zero-shot settings.

Abstract: The ability to summarize long documents succinctly is increasingly important in daily life due to information overload, yet there is a notable lack of such summaries for Spanish documents in general, and in the legal domain in particular. In this work, we present BOE-XSUM, a curated dataset comprising 3,648 concise, plain-language summaries of documents sourced from Spain’s ``Bolet'{\i}n Oficial del Estado’’ (BOE), the State Official Gazette. Each entry in the dataset includes a short summary, the original text, and its document type label. We evaluate the performance of medium-sized large language models (LLMs) fine-tuned on BOE-XSUM, comparing them to general-purpose generative models in a zero-shot setting. Results show that fine-tuned models significantly outperform their non-specialized counterparts. Notably, the best-performing model – BERTIN GPT-J 6B (32-bit precision) – achieves a 24% performance gain over the top zero-shot model, DeepSeek-R1 (accuracies of 41.6% vs.\ 33.5%).

Method: Integrates TF-IDF character n-grams with transformer embeddings and classifies text pairs through empirical distance distributions, eliminating supervised training or threshold tuning. Evaluates LLMs across different prompting strategies.

Result: Achieves 97.5% accuracy on academic essays and 94.5% in cross-domain evaluation, with 91.8% reduction in training time and 59% memory usage reduction. Few-shot prompting yields 23.5x higher style-matching than zero-shot, and completion prompting reaches 99.9% agreement with original author style.

Conclusion: Prompting strategy has more influence on style fidelity than model size. High-fidelity imitation doesn’t imply human-like unpredictability - stylistic fidelity and statistical detectability are separable, providing basis for future authorship modeling and detection work.

Abstract: Large language models (LLMs) can generate fluent text, but their ability to replicate the distinctive style of a specific human author remains unclear. We present a fast, training-free framework for authorship verification and style imitation analysis. The method integrates TF-IDF character n-grams with transformer embeddings and classifies text pairs through empirical distance distributions, eliminating the need for supervised training or threshold tuning. It achieves 97.5% accuracy on academic essays and 94.5% in cross-domain evaluation, while reducing training time by 91.8% and memory usage by 59% relative to parameter-based baselines. Using this framework, we evaluate five LLMs from three separate families (Llama, Qwen, Mixtral) across four prompting strategies - zero-shot, one-shot, few-shot, and text completion. Results show that the prompting strategy has a more substantial influence on style fidelity than model size: few-shot prompting yields up to 23.5x higher style-matching accuracy than zero-shot, and completion prompting reaches 99.9% agreement with the original author’s style. Crucially, high-fidelity imitation does not imply human-like unpredictability - human essays average a perplexity of 29.5, whereas matched LLM outputs average only 15.2. These findings demonstrate that stylistic fidelity and statistical detectability are separable, establishing a reproducible basis for future work in authorship modeling, detection, and identity-conditioned generation.

Method: Curated and resampled open-source datasets using designed metrics to identify beneficial data, then pre-trained models on ~2T tokens of high-quality data followed by established post-training procedures.

Result: MobileLLM-R1-950M achieves AIME score of 15.5, substantially outperforming comparable models (OLMo-2-1.48B: 0.6, SmolLM-2-1.7B: 0.3) and matching/surpassing Qwen3-0.6B despite using only 11.7% of Qwen3’s training tokens.

Conclusion: High-quality data curation is more crucial than massive data scaling for reasoning emergence, enabling efficient development of capable sub-billion-parameter reasoning models with significantly reduced computational requirements.

Abstract: The paradigm shift in large language models (LLMs) from instinctive responses to chain-of-thought (CoT) reasoning has fueled two prevailing assumptions: (1) reasoning capabilities only emerge in sufficiently large models, and (2) such capabilities require training on massive datasets. While the first assumption has already been challenged by recent sub-billion-parameter reasoning models such as Qwen3-0.6B and DeepSeek distilled variants, the second remains largely unquestioned. In this work, we revisit the necessity of scaling to extremely large corpora (>10T tokens) for reasoning emergence. By carefully curating and resampling open-source datasets that we identify as beneficial under our designed metrics, we demonstrate that strong reasoning abilities can emerge with far less data. Specifically, we show that only ~2T tokens of high-quality data are sufficient, and pre-training with 4.2T tokens on the dataset resampled from these ~2T tokens, followed by a established post-training procedure, enables the development of MobileLLM-R1, a series of sub-billion-parameter reasoning models that substantially outperform prior models trained on fully open-sourced data. For example, MobileLLM-R1-950M achieves an AIME score of 15.5, compared to just 0.6 for OLMo-2-1.48B and 0.3 for SmolLM-2-1.7B. Remarkably, despite being trained on only 11.7% of the tokens compared to Qwen3’s proprietary 36T-token corpus for pretraining, MobileLLM-R1-950M matches or surpasses Qwen3-0.6B across multiple reasoning benchmarks. To facilitate further research in this direction, we have released the complete training recipe, data sources, data mixing ratio, and model checkpoints, together with the key insights obtained throughout this study.

Method: Created MAQuE benchmark with 3,000 realistically simulated patient agents exhibiting diverse linguistic patterns, cognitive limitations, emotional responses, and passive disclosure tendencies. Introduced multi-faceted evaluation framework covering task success, inquiry proficiency, dialogue competence, inquiry efficiency, and patient experience.

Result: Experiments show substantial challenges across all evaluation aspects. State-of-the-art models have significant room for improvement in inquiry capabilities, are highly sensitive to realistic patient behavior variations, and show trade-offs between different evaluation perspectives.

Conclusion: Current medical AI models struggle to balance performance and practicality in real-world clinical settings, highlighting the need for more comprehensive evaluation and improvement in medical questioning capabilities beyond just diagnostic accuracy.

Abstract: An effective physician should possess a combination of empathy, expertise, patience, and clear communication when treating a patient. Recent advances have successfully endowed AI doctors with expert diagnostic skills, particularly the ability to actively seek information through inquiry. However, other essential qualities of a good doctor remain overlooked. To bridge this gap, we present MAQuE(Medical Agent Questioning Evaluation), the largest-ever benchmark for the automatic and comprehensive evaluation of medical multi-turn questioning. It features 3,000 realistically simulated patient agents that exhibit diverse linguistic patterns, cognitive limitations, emotional responses, and tendencies for passive disclosure. We also introduce a multi-faceted evaluation framework, covering task success, inquiry proficiency, dialogue competence, inquiry efficiency, and patient experience. Experiments on different LLMs reveal substantial challenges across the evaluation aspects. Even state-of-the-art models show significant room for improvement in their inquiry capabilities. These models are highly sensitive to variations in realistic patient behavior, which considerably impacts diagnostic accuracy. Furthermore, our fine-grained metrics expose trade-offs between different evaluation perspectives, highlighting the challenge of balancing performance and practicality in real-world clinical settings.

Method: Two-stage detection: first stage pre-screens suspicious users using behavioral criteria, second stage uses LLM-based auditing to evaluate semantic consistency. Enhanced through reinforcement fine-tuning with specialized reward functions.

Result: Experiments on six attack strategies show SemanticShield effectively detects shilling attacks, and evaluation on unseen attack methods demonstrates strong generalization capability.

Conclusion: SemanticShield provides an effective defense against shilling attacks by leveraging item-side semantics through LLMs, offering improved detection and generalization over traditional methods.

Abstract: Recommender systems (RS) are widely used in e-commerce for personalized suggestions, yet their openness makes them susceptible to shilling attacks, where adversaries inject fake behaviors to manipulate recommendations. Most existing defenses emphasize user-side behaviors while overlooking item-side features such as titles and descriptions that can expose malicious intent. To address this gap, we propose a two-stage detection framework that integrates item-side semantics via large language models (LLMs). The first stage pre-screens suspicious users using low-cost behavioral criteria, and the second stage employs LLM-based auditing to evaluate semantic consistency. Furthermore, we enhance the auditing model through reinforcement fine-tuning on a lightweight LLM with carefully designed reward functions, yielding a specialized detector called SemanticShield. Experiments on six representative attack strategies demonstrate the effectiveness of SemanticShield against shilling attacks, and further evaluation on previously unseen attack methods shows its strong generalization capability. Code is available at https://github.com/FrankenstLee/SemanticShield.

Method: Proposed Generalized Correctness Models (GCMs) trained on correctness data from multiple LLMs, using historical predictions and answer phrasing patterns. Also explored in-context examples and post-hoc calibration methods.

Result: GCMs trained on Qwen3-8B performed well across 5 model families and datasets (MMLU, TriviaQA), showing that correctness prediction is a generalizable skill learned from historical data rather than model-specific introspection.

Conclusion: Reliable LLM confidence estimation is a model-agnostic skill learned through systematic encoding of correctness history, not dependent on self-introspection.

Abstract: Generating accurate and calibrated confidence estimates is critical for deploying LLMs in high-stakes or user-facing applications, and remains an open challenge. Prior research has often framed confidence as a problem of eliciting a model’s “self-knowledge”, i.e., the ability of an LLM to judge whether its own answers are correct; this approach implicitly assumes that there is some privileged information about the answer’s correctness that is accessible to the model itself. However, our experiments reveal that an LLM attempting to predict the correctness of its own outputs generally performs no better than an unrelated LLM. Moreover, we hypothesize that a key factor in building a “Correctness Model” (CM) is exposure to a target model’s historical predictions. We propose multiple methods to inject this historical correctness information, creating a Generalized Correctness Model (GCM). We first show that GCMs can be trained on the correctness data from many LLMs and learn patterns for correctness prediction applicable across datasets and models. We then use CMs as a lens for studying the source of correctness prediction ability and its generalization, systematically controlling their training data and finding that answer phrasing is a strong predictor for correctness. We further explore alternative methods of injecting history without training an LLM, finding that including history as in-context examples can help improve correctness prediction, and post-hoc calibration can provide complementary reductions in calibration error. We evaluate GCMs based on Qwen3-8B across 5 model families and the MMLU and TriviaQA datasets, as well as on a downstream selective prediction task, finding that reliable LLM confidence estimation is a generalizable and model-agnostic skill learned by systematically encoding correctness history rather than a model-specific skill reliant on self-introspection.

Method: Proposes circuit distillation with an objective to align internal representations between functionally correspondent circuit components in teacher and student models. Introduces a method to match these components and a loss function reflecting representation similarities.

Result: Circuit distillation outperforms standard distillation on entity tracking and theory of mind tasks using Llama3 models. Successfully transfers algorithmic capabilities by adjusting only a small, targeted subset of student parameters.

Conclusion: Establishes feasibility of transferring mechanisms, enabling efficient distillation of targeted teacher capabilities through interpretable and controllable internal student mechanisms.

Abstract: Model distillation typically focuses on behavioral mimicry, where a student model is trained to replicate a teacher’s output while treating its internal computations as a black box. In this work we propose an alternative approach: Distilling the underlying computational mechanisms implemented by a teacher model. Specifically, we propose circuit distillation, which introduces an objective to align internal representations between analogous circuit components in teacher and student models. We propose a method to match ``functionally correspondent’’ circuit components and introduce a loss reflecting similarities between the representations that these induce. We evaluate circuit distillation on entity tracking and theory of mind (ToM) tasks using models from the Llama3 family. Our results demonstrate that circuit distillation outperforms standard distillation, successfully transferring algorithmic capabilities by adjusting only a small, targeted subset of student model parameters. This work establishes the feasibility of transferring mechanisms, which may in turn allow for efficient distillation of targeted teacher capabilities via interpretable and controllable internal student mechanisms.

Method: Uses pre-trained discrete diffusion language models as initialization, with techniques like grouped reward normalization, intermediate-state matching, and reward-guided ancestral sampler (RGAS) for improved training stability and inference performance.

Result: Achieves 64x acceleration over GPT-2 baseline, with perplexities ranging from 62.2 (8 NFEs) to 18.4 (128 NFEs) on OpenWebText, with only 1% entropy loss and 20x less training time.

Conclusion: DiDi-Instruct is an efficient distillation method that enables fast language generation with minimal performance degradation, validated through extensive studies and protein sequence generation.

Abstract: Fast generation of language texts is the holy grail that people pursue in the AI era. In this work, we introduced Discrete Diffusion Divergence Instruct (DiDi-Instruct), a training-based method that leads to fast language generation models by initializing from a pre-trained (masked) discrete diffusion language model (dLLM). The resulting DiDi-Instruct model outperforms the dLLM counterparts and the GPT-2 baseline with 64x acceleration. In the theoretical part of the paper, we build the foundation of DiDi-Instruct in a framework of integral KL-divergence minimization, with practical training algorithms. We also introduce techniques like grouped reward normalization, intermediate-state matching, and the reward-guided ancestral sampler (RGAS) that significantly improve the training stability, the model coverage, and the inference performances. On OpenWebText, DiDi-Instruct outperforms all accelerated language generation models as well as the GPT-2 baseline and the standard dLLMs, achieving sample perplexities ranging from 62.2 (8 NFEs) to 18.4 (128 NFEs). These performance gains are accomplished with a negligible entropy loss of about 1% and 20x less additional training wall-clock time. We further validate the robustness and effectiveness of DiDi-Instruct through extensive ablation studies, model scaling, and the generation of discrete protein sequences. In conclusion, DiDi-Instruct is an efficient yet effective distillation method, enabling language generation in the blink of an eye. We will release both code and models at github.com/haoyangzheng-ai/didi-instruct.

Method: GateMABSA integrates three specialized mLSTMs: Syn-mLSTM for syntactic structure, Sem-mLSTM for aspect-semantic relevance, and Fuse-mLSTM for selective multimodal fusion.

Result: Extensive experiments on two benchmark Twitter datasets demonstrate that GateMABSA outperforms several baselines.

Conclusion: The proposed gated multimodal architecture effectively addresses challenges in multimodal ABSA by incorporating syntactic, semantic, and fusion-aware components.

Abstract: Aspect-based Sentiment Analysis (ABSA) has recently advanced into the multimodal domain, where user-generated content often combines text and images. However, existing multimodal ABSA (MABSA) models struggle to filter noisy visual signals, and effectively align aspects with opinion-bearing content across modalities. To address these challenges, we propose GateMABSA, a novel gated multimodal architecture that integrates syntactic, semantic, and fusion-aware mLSTM. Specifically, GateMABSA introduces three specialized mLSTMs: Syn-mLSTM to incorporate syntactic structure, Sem-mLSTM to emphasize aspect–semantic relevance, and Fuse-mLSTM to perform selective multimodal fusion. Extensive experiments on two benchmark Twitter datasets demonstrate that GateMABSA outperforms several baselines.

Method: Projects the model’s residual stream into interpretable concepts via Vector Symbolic Architectures, combining strengths of sparse autoencoders and conventional probes while overcoming their limitations.

Result: The probe reliably extracts meaningful concepts across varied LLMs, embedding sizes, and input domains, and helps identify LLM failures in both controlled tasks and question-answering settings.

Conclusion: This work advances information decoding in LLM vector space, enabling extraction of more informative, interpretable, and structured features from neural representations.

Abstract: Despite their capabilities, Large Language Models (LLMs) remain opaque with limited understanding of their internal representations. Current interpretability methods, such as direct logit attribution (DLA) and sparse autoencoders (SAEs), provide restricted insight due to limitations such as the model’s output vocabulary or unclear feature names. This work introduces Hyperdimensional Probe, a novel paradigm for decoding information from the LLM vector space. It combines ideas from symbolic representations and neural probing to project the model’s residual stream into interpretable concepts via Vector Symbolic Architectures (VSAs). This probe combines the strengths of SAEs and conventional probes while overcoming their key limitations. We validate our decoding paradigm with controlled input-completion tasks, probing the model’s final state before next-token prediction on inputs spanning syntactic pattern recognition, key-value associations, and abstract inference. We further assess it in a question-answering setting, examining the state of the model both before and after text generation. Our experiments show that our probe reliably extracts meaningful concepts across varied LLMs, embedding sizes, and input domains, also helping identify LLM failures. Our work advances information decoding in LLM vector space, enabling extracting more informative, interpretable, and structured features from neural representations.

Method: Proposed confidence-informed prompting where word-level uncertainty estimates are embedded directly into LLM training, fine-tuned a LLaMA 3.1 model, and compared against transcript-only fine-tuning and post hoc confidence-based filtering.

Result: Achieved 10% relative WER reduction on Speech Accessibility Project spontaneous speech and 47% reduction on TORGO dataset compared to naive LLM correction.

Conclusion: Confidence-aware fine-tuning is effective for impaired speech ASR error correction, demonstrating significant improvements in word error rate reduction through uncertainty-informed LLM training.

Abstract: We investigate the use of large language models (LLMs) as post-processing modules for automatic speech recognition (ASR), focusing on their ability to perform error correction for disordered speech. In particular, we propose confidence-informed prompting, where word-level uncertainty estimates are embedded directly into LLM training to improve robustness and generalization across speakers and datasets. This approach directs the model to uncertain ASR regions and reduces overcorrection. We fine-tune a LLaMA 3.1 model and compare our approach to both transcript-only fine-tuning and post hoc confidence-based filtering. Evaluations show that our method achieves a 10% relative WER reduction compared to naive LLM correction on the Speech Accessibility Project spontaneous speech and a 47% reduction on TORGO, demonstrating the effectiveness of confidence-aware fine-tuning for impaired speech.

Method: Training small Transformers with diverse modifications including alternative position encodings, softmax replacements, hybrid models, and short convolutions on the program trace generation task.

Result: Models achieve strong in-distribution accuracy but exhibit systematic failures when generalizing to various factors like program length and trace steps, though some architectural designs significantly improve generalization.

Conclusion: While Transformers can perform well on program trace generation in-distribution, they struggle with systematic generalization, suggesting limitations in their reasoning capabilities that can be partially mitigated through specific architectural modifications.

Abstract: We study Transformers on the task \emph{program trace generation} (PTG), where models produce step-by-step execution traces for synthetic programs. Unlike existing algorithmic problems, PTG externalizes reasoning through long traces where each step is trivial. We train small Transformers with diverse modifications, including alternative position encodings, softmax replacements, hybrid model, and short convolutions. While these models achieve strong in-distribution accuracy, they exhibit systematic failures when generalizing to various factors (e.g., program length, trace steps), though some designs significantly improve generalization.

Method: Uses fine-grained task taxonomy with recursive easy-to-hard composition, knowledge-augmented trajectory sampling with filtering, dynamic training objective combining SFT and RL losses, and memory-frugal multi-turn rollout framework.

Result: DataMind-14B achieves 71.16% average score on multiple benchmarks, outperforming proprietary models like DeepSeek-V3.1 and GPT-5. DataMind-7B scores 68.10%, best among open-source models.

Conclusion: DataMind provides an effective framework for building high-performing open-source data-analytic agents and releases curated datasets and models for community research.

Abstract: Data-analytic agents are emerging as a key catalyst for automated scientific discovery and for the vision of Innovating AI. Current approaches, however, rely heavily on prompt engineering over proprietary models, while open-source models struggle to face diverse-format, large-scale data files and long-horizon, multi-step reasoning that real-world analytics demands. This paper introduces DataMind, a scalable data synthesis and agent training recipe designed to build generalist data-analytic agents. DataMind tackles three key challenges in building open-source data-analytic agents, including insufficient data resources, improper training strategy, and unstable code-based multi-turn rollout. Concretely, DataMind applies 1) a fine-grained task taxonomy and a recursive easy-to-hard task composition mechanism to increase the diversity and difficulty of synthesized queries; 2) a knowledge-augmented trajectory sampling strategy followed by model-based and rule-based filtering; 3) a dynamically adjustable training objective combining both SFT and RL losses; 4) a memory-frugal and stable code-based multi-turn rollout framework. Built on DataMind, we curate DataMind-12K, a high-quality trajectory set spanning diverse domains, task categories, and data file formats for data-analytic tasks. Trained on DataMind-12K, our DataMind-14B achieves state-of-the-art with an average score of 71.16% on multiple data analysis benchmarks, outperforming the strongest proprietary baselines DeepSeek-V3.1 and GPT-5. Our DataMind-7B also performs best among all open-source models with a score of 68.10%. We also incorporate some empirical insights gained from our exploratory trials into the analysis experiments, aiming to provide actionable insights about agentic training for the community. We will release DataMind-12K and DataMind-7B,14B for the community’s future research.

Method: Introduces ’last but not late interaction’ - conducts causal self-attention between query and documents within same context window, then extracts contextual embeddings from last token of each document.

Result: Achieves state-of-the-art BEIR performance with 61.94 nDCG@10 while being ten times smaller than generative listwise rerankers.

Conclusion: The compact architecture with novel interaction method provides superior performance and efficiency compared to existing approaches.

Abstract: jina-reranker-v3 is a 0.6B parameter multilingual document reranker that introduces a novel last but not late interaction. Unlike late interaction models such as ColBERT that perform separate encoding followed by multi-vector matching, our approach conducts causal self-attention between query and documents within the same context window, enabling rich cross-document interactions before extracting contextual embeddings from the last token of each document. This compact architecture achieves state-of-the-art BEIR performance with 61.94 nDCG@10 while being ten times smaller than generative listwise rerankers.

Method: Uses small LLMs deployable locally, explores knowledge distillation with synthesized data and in-context learning, and proposes filtering strategy based on model output probability to detect harmful simplifications.

Result: Knowledge distillation boosts automatic metrics but increases harmful simplifications. Model output probability effectively detects harmful simplifications. Filtering strategy suppresses harmful outputs while preserving beneficial ones.

Conclusion: Establishes benchmark for efficient and safe lexical simplification with small LLMs, highlighting trade-offs between performance, efficiency, and safety, and demonstrates promising approach for real-world deployment.

Abstract: Despite their strong performance, large language models (LLMs) face challenges in real-world application of lexical simplification (LS), particularly in privacy-sensitive and resource-constrained environments. Moreover, since vulnerable user groups (e.g., people with disabilities) are one of the key target groups of this technology, it is crucial to ensure the safety and correctness of the output of LS systems. To address these issues, we propose an efficient framework for LS systems that utilizes small LLMs deployable in local environments. Within this framework, we explore knowledge distillation with synthesized data and in-context learning as baselines. Our experiments in five languages evaluate model outputs both automatically and manually. Our manual analysis reveals that while knowledge distillation boosts automatic metric scores, it also introduces a safety trade-off by increasing harmful simplifications. Importantly, we find that the model’s output probability is a useful signal for detecting harmful simplifications. Leveraging this, we propose a filtering strategy that suppresses harmful simplifications while largely preserving beneficial ones. This work establishes a benchmark for efficient and safe LS with small LLMs. It highlights the key trade-offs between performance, efficiency, and safety, and demonstrates a promising approach for safe real-world deployment.

Method: Created a benchmark with 250 user-task queries combining 50 research tasks and 25 authentic user profiles. Proposed PQR Evaluation Framework measuring Personalization Alignment, Content Quality, and Factual Reliability.

Result: Experiments on various systems revealed current capabilities and limitations in handling personalized deep research.

Conclusion: Establishes a rigorous foundation for developing and evaluating the next generation of truly personalized AI research assistants.

Abstract: Deep Research Agents (DRAs) can autonomously conduct complex investigations and generate comprehensive reports, demonstrating strong real-world potential. However, existing evaluations mostly rely on close-ended benchmarks, while open-ended deep research benchmarks remain scarce and typically neglect personalized scenarios. To bridge this gap, we introduce Personalized Deep Research Bench, the first benchmark for evaluating personalization in DRAs. It pairs 50 diverse research tasks across 10 domains with 25 authentic user profiles that combine structured persona attributes with dynamic real-world contexts, yielding 250 realistic user-task queries. To assess system performance, we propose the PQR Evaluation Framework, which jointly measures (P) Personalization Alignment, (Q) Content Quality, and (R) Factual Reliability. Our experiments on a range of systems highlight current capabilities and limitations in handling personalized deep research. This work establishes a rigorous foundation for developing and evaluating the next generation of truly personalized AI research assistants.

Method: Extended an existing benchmark with knowledge extraction annotations and evaluated commercial and open-source LLMs of varying sizes, testing augmentation with extracted triples and multi-task learning approaches.

Result: Web-scale knowledge extraction remains challenging for LLMs, but LLMs can still benefit from knowledge extraction through augmentation with extracted triples and multi-task learning, even when achieving high QA accuracy.

Conclusion: Knowledge triple extraction maintains value in web-based QA systems and provides strategies for maximizing LLM effectiveness across different model sizes and resource constraints.

Abstract: The advent of Large Language Models (LLMs) has significantly advanced web-based Question Answering (QA) systems over semi-structured content, raising questions about the continued utility of knowledge extraction for question answering. This paper investigates the value of triple extraction in this new paradigm by extending an existing benchmark with knowledge extraction annotations and evaluating commercial and open-source LLMs of varying sizes. Our results show that web-scale knowledge extraction remains a challenging task for LLMs. Despite achieving high QA accuracy, LLMs can still benefit from knowledge extraction, through augmentation with extracted triples and multi-task learning. These findings provide insights into the evolving role of knowledge triple extraction in web-based QA and highlight strategies for maximizing LLM effectiveness across different model sizes and resource settings.

Method: Evaluated six open-source multilingual LLMs across 20 languages using multilingual prompting with AMC-16K dataset, and analyzed retrieval bias using multilingual embedding models and claim frequency.

Result: Found persistent language bias in LLM performance and retrieval bias where certain claims are disproportionately retrieved, inflating performance for popular claims while under-representing less common ones.

Conclusion: Highlights ongoing bias issues in multilingual fact-checking and provides recommendations for improving equity in cross-lingual fact verification systems.

Abstract: Multilingual Large Language Models (LLMs) offer powerful capabilities for cross-lingual fact-checking. However, these models often exhibit language bias, performing disproportionately better on high-resource languages such as English than on low-resource counterparts. We also present and inspect a novel concept

• retrieval bias, when information retrieval systems tend to favor certain information over others, leaving the retrieval process skewed. In this paper, we study language and retrieval bias in the context of Previously Fact-Checked Claim Detection (PFCD). We evaluate six open-source multilingual LLMs across 20 languages using a fully multilingual prompting strategy, leveraging the AMC-16K dataset. By translating task prompts into each language, we uncover disparities in monolingual and cross-lingual performance and identify key trends based on model family, size, and prompting strategy. Our findings highlight persistent bias in LLM behavior and offer recommendations for improving equity in multilingual fact-checking. To investigate retrieval bias, we employed multilingual embedding models and look into the frequency of retrieved claims. Our analysis reveals that certain claims are retrieved disproportionately across different posts, leading to inflated retrieval performance for popular claims while under-representing less common ones.

Method: Two-stage pipeline: teacher LLM compresses unpaired examples into intermediate representations (IRs), then student model reconstructs inputs from IRs to pair with outputs, creating synthetic training data.

Result: 8B student trained on PbT data outperforms models trained on 70B teacher-generated corpora, comes within 1.2 ROUGE-L of human-annotated pairs, closes 82% of oracle gap at one-third annotation cost of direct synthesis.

Conclusion: PbT effectively generates in-domain sources that avoid mismatch issues, producing concise and faithful summaries aligned with target style, demonstrating significant advantage over direct synthesis approaches.

Abstract: We present Paired by the Teacher (PbT), a two-stage teacher-student pipeline that synthesizes accurate input-output pairs without human labels or parallel data. In many low-resource natural language generation (NLG) scenarios, practitioners may have only raw outputs, like highlights, recaps, or questions, or only raw inputs, such as articles, dialogues, or paragraphs, but seldom both. This mismatch forces small models to learn from very few examples or rely on costly, broad-scope synthetic examples produced by large LLMs. PbT addresses this by asking a teacher LLM to compress each unpaired example into a concise intermediate representation (IR), and training a student to reconstruct inputs from IRs. This enables outputs to be paired with student-generated inputs, yielding high-quality synthetic data. We evaluate PbT on five benchmarks-document summarization (XSum, CNNDM), dialogue summarization (SAMSum, DialogSum), and question generation (SQuAD)-as well as an unpaired setting on SwitchBoard (paired with DialogSum summaries). An 8B student trained only on PbT data outperforms models trained on 70 B teacher-generated corpora and other unsupervised baselines, coming within 1.2 ROUGE-L of human-annotated pairs and closing 82% of the oracle gap at one-third the annotation cost of direct synthesis. Human evaluation on SwitchBoard further confirms that only PbT produces concise, faithful summaries aligned with the target style, highlighting its advantage of generating in-domain sources that avoid the mismatch, limiting direct synthesis.

Method: The approach integrates Random Hadamard transforms to bound block-level outliers, uses two-dimensional quantization scheme for consistent representations across forward and backward passes, employs stochastic rounding for unbiased gradient estimation, and incorporates selective high-precision layers.

Result: The method was validated by training a 12-billion-parameter model on 10 trillion tokens - the longest publicly documented training run in 4-bit precision. The model achieved training loss and downstream task accuracies comparable to an FP8 baseline.

Conclusion: NVFP4, when combined with the proposed training approach, represents a major step forward in narrow-precision LLM training algorithms, enabling more efficient training of next-generation LLMs.

Abstract: Large Language Models (LLMs) today are powerful problem solvers across many domains, and they continue to get stronger as they scale in model size, training set size, and training set quality, as shown by extensive research and experimentation across the industry. Training a frontier model today requires on the order of tens to hundreds of yottaflops, which is a massive investment of time, compute, and energy. Improving pretraining efficiency is therefore essential to enable the next generation of even more capable LLMs. While 8-bit floating point (FP8) training is now widely adopted, transitioning to even narrower precision, such as 4-bit floating point (FP4), could unlock additional improvements in computational speed and resource utilization. However, quantization at this level poses challenges to training stability, convergence, and implementation, notably for large-scale models trained on long token horizons. In this study, we introduce a novel approach for stable and accurate training of large language models (LLMs) using the NVFP4 format. Our method integrates Random Hadamard transforms (RHT) to bound block-level outliers, employs a two-dimensional quantization scheme for consistent representations across both the forward and backward passes, utilizes stochastic rounding for unbiased gradient estimation, and incorporates selective high-precision layers. We validate our approach by training a 12-billion-parameter model on 10 trillion tokens – the longest publicly documented training run in 4-bit precision to date. Our results show that the model trained with our NVFP4-based pretraining technique achieves training loss and downstream task accuracies comparable to an FP8 baseline. These findings highlight that NVFP4, when combined with our training approach, represents a major step forward in narrow-precision LLM training algorithms.

Method: Built on vLLM with modular architecture featuring pluggable interfaces for analysis-based and learning-based methods, fine-grained parameter control, pre-computed steering vectors for eight domains, and interactive demonstration system.

Result: Achieves 5.5-11.4× speedup over existing frameworks and demonstrates effectiveness in overthinking mitigation, hallucination reduction, and other key applications.

Conclusion: EasySteer transforms steering from research technique to production-ready capability, establishing critical infrastructure for deployable, controllable language models.

Abstract: Large language model (LLM) steering has emerged as a promising paradigm for controlling model behavior at inference time through targeted manipulation of hidden states, offering a lightweight alternative to expensive retraining. However, existing steering frameworks suffer from critical limitations: computational inefficiency, limited extensibility, and restricted functionality that hinder both research progress and practical deployment. We present EasySteer, a unified framework for high-performance, extensible LLM steering built on vLLM. Our system features modular architecture with pluggable interfaces for both analysis-based and learning-based methods, fine-grained parameter control, pre-computed steering vectors for eight application domains, and an interactive demonstration system. Through deep integration with vLLM’s optimized inference engine, EasySteer achieves 5.5-11.4$\times$ speedup over existing frameworks. Extensive experiments demonstrate its effectiveness in overthinking mitigation, hallucination reduction, and other key applications. EasySteer transforms steering from research technique to production-ready capability, establishing critical infrastructure for deployable, controllable language models.

Method: Uses pairwise learning within domain-year groups to reduce rating inconsistencies, introduces Review Tendency Signal (RTS) for probabilistic integration of reviewer scores and confidences, and trains on a large dataset of 24,276 ICLR submissions.

Result: Achieves 78.2% AUC and 0.432 Spearman correlation, demonstrates strong generalization on NeurIPS submissions with predicted scores increasing consistently across decision categories from Rejected to Oral.

Conclusion: NAIPv2 establishes a debiased and scalable framework for automated paper quality estimation, representing progress toward future scientific intelligence systems.

Abstract: The ability to estimate the quality of scientific papers is central to how both humans and AI systems will advance scientific knowledge in the future. However, existing LLM-based estimation methods suffer from high inference cost, whereas the faster direct score regression approach is limited by scale inconsistencies. We present NAIPv2, a debiased and efficient framework for paper quality estimation. NAIPv2 employs pairwise learning within domain-year groups to reduce inconsistencies in reviewer ratings and introduces the Review Tendency Signal (RTS) as a probabilistic integration of reviewer scores and confidences. To support training and evaluation, we further construct NAIDv2, a large-scale dataset of 24,276 ICLR submissions enriched with metadata and detailed structured content. Trained on pairwise comparisons but enabling efficient pointwise prediction at deployment, NAIPv2 achieves state-of-the-art performance (78.2% AUC, 0.432 Spearman), while maintaining scalable, linear-time efficiency at inference. Notably, on unseen NeurIPS submissions, it further demonstrates strong generalization, with predicted scores increasing consistently across decision categories from Rejected to Oral. These findings establish NAIPv2 as a debiased and scalable framework for automated paper quality estimation, marking a step toward future scientific intelligence systems. Code and dataset are released at https://sway.cloud.microsoft/Pr42npP80MfPhvj8.

Method: Four-step framework: (1) decompose draft synthesis into atomic claims, (2) elicit each source’s stance on every claim, (3) score sources using adapted multi-task peer-prediction mechanism that rewards informative agreement, (4) filter unreliable sources before re-summarizing.

Result: Experiments show TTS improves factual accuracy and robustness while preserving fluency, aligning exposure with informative corroboration and disincentivizing manipulation.

Conclusion: TTS establishes formal guarantees that align source incentives with informative honesty, making truthful reporting the utility-maximizing strategy for improved factual robustness in text summarization.

Abstract: Large language models (LLMs) are increasingly used in modern search and answer systems to synthesize multiple, sometimes conflicting, texts into a single response, yet current pipelines offer weak incentives for sources to be accurate and are vulnerable to adversarial content. We introduce Truthful Text Summarization (TTS), an incentive-aligned framework that improves factual robustness without ground-truth labels. TTS (i) decomposes a draft synthesis into atomic claims, (ii) elicits each source’s stance on every claim, (iii) scores sources with an adapted multi-task peer-prediction mechanism that rewards informative agreement, and (iv) filters unreliable sources before re-summarizing. We establish formal guarantees that align a source’s incentives with informative honesty, making truthful reporting the utility-maximizing strategy. Experiments show that TTS improves factual accuracy and robustness while preserving fluency, aligning exposure with informative corroboration and disincentivizing manipulation.

Method: Proposes Learn2PD framework with a lightweight filter model trained to predict when token predictions match final output, and introduces End-of-Text Prediction (EoTP) to detect decoding completion. The filter is learned post-training with minimal computation.

Result: Achieves up to 22.58× speedup without performance drop on LLaDA benchmark, and up to 57.51× when combined with KV-Cache.

Conclusion: The proposed adaptive parallel decoding approach significantly improves inference throughput while maintaining output quality, offering a more flexible alternative to fixed heuristic methods.

Abstract: Autoregressive decoding in large language models (LLMs) requires $\mathcal{O}(n)$ sequential steps for $n$ tokens, fundamentally limiting inference throughput. Recent diffusion-based LLMs (dLLMs) enable parallel token generation through iterative denoising. However, current parallel decoding strategies rely on fixed, input-agnostic heuristics (e.g., confidence thresholds), which fail to adapt to input-specific characteristics, resulting in suboptimal speed-quality trade-offs across diverse NLP tasks. In this work, we explore a more flexible and dynamic approach to parallel decoding. We propose Learning to Parallel Decode (Learn2PD), a framework that trains a lightweight and adaptive filter model to predict, for each token position, whether the current prediction matches the final output. This learned filter approximates an oracle parallel decoding strategy that unmasks tokens only when correctly predicted. Importantly, the filter model is learned in a post-training manner, requiring only a small amount of computation to optimize it (minute-level GPU time). Additionally, we introduce End-of-Text Prediction (EoTP) to detect decoding completion at the end of sequence, avoiding redundant decoding of padding tokens. Experiments on the LLaDA benchmark demonstrate that our method achieves up to 22.58$\times$ speedup without any performance drop, and up to 57.51$\times$ when combined with KV-Cache.

Method: Uses entity trees and sub-tree sampling with entity fuzzification to create challenging queries, develops self-hosted search infrastructure, and employs two-stage training: cold-start supervised finetuning followed by reinforcement learning.

Result: Achieves 15.3% accuracy on BrowseComp, 29.2% on BrowseComp-ZH, and 40.4% on Xbench-DS, outperforming WebSailor-72B and DeepDive-32B.

Conclusion: The proposed data synthesis pipeline and training methods enable InfoAgent to effectively handle challenging research queries while maintaining transparency through self-hosted tools.

Abstract: Building Large Language Model agents that expand their capabilities by interacting with external tools represents a new frontier in AI research and applications. In this paper, we introduce InfoAgent, a deep research agent powered by an innovative data synthesis pipeline and orchestrated web search tools. To construct challenging, hard-to-find queries,we build entity trees and apply sub-tree sampling with entity fuzzification to systematically increase question difficulty. Unlike prior work that relies heavily on commercial search tools, we develop a dedicated self-hosted search infrastructure, enhancing transparency of agent environments and facilitating further advancement of agent capacity. We evaluate the effectiveness of our data pipeline by measuring the average number of tool calls required to correctly answer a question, and also show that our agent yields better performance when equipped with our tools. Our \mbox{InfoAgent} is post-trained from Qwen3-14B using a two-stage recipe: cold-start supervised finetuning to instill long-horizon search behaviors, followed by reinforcement learning which significantly improves reasoning-driven tool use. With our methods, InfoAgent achieves 15.3% accuracy on BrowseComp, 29.2% on BrowseComp-ZH, and 40.4% on Xbench-DS, outperforming prior open-source deep research agents such as WebSailor-72B and DeepDive-32B.

Method: Transformer-based deep learning approach using mT5 model with Translation Language Modeling (TLM) instead of conventional Masked Language Modeling (MLM).

Result: Development of an open-source cross-lingual reverse dictionary system with support for Indian languages.

Conclusion: The proposed system addresses the gap in reverse dictionary support for Indian languages using advanced transformer architecture with TLM technique.

Abstract: A reverse dictionary takes a target word’s description as input and returns the words that fit the description. Reverse Dictionaries are useful for new language learners, anomia patients, and for solving common tip-of-the-tongue problems (lethologica). Currently, there does not exist any Reverse Dictionary provider with support for any Indian Language. We present a novel open-source cross-lingual reverse dictionary system with support for Indian languages. In this paper, we propose a transformer-based deep learning approach to tackle the limitations faced by the existing systems using the mT5 model. This architecture uses the Translation Language Modeling (TLM) technique, rather than the conventional BERT’s Masked Language Modeling (MLM) technique.

Method: Reformulate DST as bundle of granular example-guided QA tasks, use in-context examples retrieved by trained retriever, and combine with dialogue-level memory replay.

Result: 60M parameter model achieves significant performance boost, attains state-of-the-art on DST continual learning metrics without complex regularization or parameter expansion.

Conclusion: The approach successfully alleviates service-specific memorization and teaches models to contextualize questions and examples for better continual learning in dialogue systems.

Abstract: Dialogue systems are frequently updated to accommodate new services, but naively updating them by continually training with data for new services in diminishing performance on previously learnt services. Motivated by the insight that dialogue state tracking (DST), a crucial component of dialogue systems that estimates the user’s goal as a conversation proceeds, is a simple natural language understanding task, we propose reformulating it as a bundle of granular example-guided question answering tasks to minimize the task shift between services and thus benefit continual learning. Our approach alleviates service-specific memorization and teaches a model to contextualize the given question and example to extract the necessary information from the conversation. We find that a model with just 60M parameters can achieve a significant boost by learning to learn from in-context examples retrieved by a retriever trained to identify turns with similar dialogue state changes. Combining our method with dialogue-level memory replay, our approach attains state of the art performance on DST continual learning metrics without relying on any complex regularization or parameter expansion methods.

Method: Development of a treebank and tools hosted on GitHub, with documentation of the annotation scheme.

Result: Creation of CGELBank treebank and tools available at https://github.com/nert-nlp/cgel.

Conclusion: The paper documents the specific annotation scheme used in CGELBank based on CGEL syntactic formalism.

Abstract: CGELBank is a treebank and associated tools based on a syntactic formalism for English derived from the Cambridge Grammar of the English Language (CGEL; Huddleston and Pullum, 2002). It is hosted on GitHub at https://github.com/nert-nlp/cgel. This document lays out the particularities of the CGELBank annotation scheme.

Method: A three-step approach: 1) Conceptualize racism and its manifestations, 2) Contextualize racist manifestations to specific time and place to identify discursive forms, 3) Apply XLM-RoBERTa for supervised text classification with contextual understanding.

Result: XLM-R and their pretrained model XLM-R-Racismo outperform other state-of-the-art approaches in classifying racism in large corpora, demonstrated on a corpus of tweets about Ecuador’s indigenous community from 2018-2021.

Conclusion: The proposed method provides an effective, generalizable framework for identifying and classifying various forms of racist discourse in large text corpora across different languages and contexts.

Abstract: Current methods to identify and classify racist language in text rely on small-n qualitative approaches or large-n approaches focusing exclusively on overt forms of racist discourse. This article provides a step-by-step generalizable guideline to identify and classify different forms of racist discourse in large corpora. In our approach, we start by conceptualizing racism and its different manifestations. We then contextualize these racist manifestations to the time and place of interest, which allows researchers to identify their discursive form. Finally, we apply XLM-RoBERTa (XLM-R), a cross-lingual model for supervised text classification with a cutting-edge contextual understanding of text. We show that XLM-R and XLM-R-Racismo, our pretrained model, outperform other state-of-the-art approaches in classifying racism in large corpora. We illustrate our approach using a corpus of tweets relating to the Ecuadorian ind'igena community between 2018 and 2021.

Method: DEN uses three modules: Long-term Personality Encoding for stable traits via psychological entity patterns, Short-term Personality Encoding for dynamic states via contextual post analysis, and Bi-directional Interaction to integrate both aspects.

Result: Experimental results on two personality detection datasets demonstrate the effectiveness of the DEN model in capturing comprehensive personality representations.

Conclusion: The study underscores the importance of considering both stable and dynamic aspects of personality in textual personality detection for a more comprehensive understanding.

Abstract: Textual personality detection aims to identify personality characteristics by analyzing user-generated content on social media platforms. Extensive psychological literature highlights that personality encompasses both long-term stable traits and short-term dynamic states. However, existing studies often concentrate only on either long-term or short-term personality representations, neglecting the integration of both aspects. This limitation hinders a comprehensive understanding of individuals’ personalities, as both stable traits and dynamic states are vital. To bridge this gap, we propose a Dual Enhanced Network (DEN) to jointly model users’ long-term and short-term personality traits. In DEN, the Long-term Personality Encoding module models stable long-term personality traits by analyzing consistent patterns in the usage of psychological entities. The Short-term Personality Encoding module captures dynamic short-term personality states by modeling the contextual information of individual posts in real-time. The Bi-directional Interaction module integrates both aspects of personality, creating a cohesive and comprehensive representation of the user’s personality. Experimental results on two personality detection datasets demonstrate the effectiveness of the DEN model and underscore the importance of considering both stable and dynamic aspects of personality in textual personality detection.

Method: Leverages activations from Transformer’s multi-head attention layer as keys instead of decoder layer, exploiting that different attention heads capture different data aspects to create embeddings representing various facets.

Result: Shows design advantages over 18 RAG baselines, empirical improvements of up to 20% in retrieval success ratios, and benefits for downstream LLM generation.

Conclusion: MRAG effectively addresses multi-aspect document retrieval challenges and can be seamlessly integrated with existing RAG frameworks and benchmarks.

Abstract: Retrieval Augmented Generation (RAG) enhances the abilities of Large Language Models (LLMs) by enabling the retrieval of documents into the LLM context to provide more accurate and relevant responses. Existing RAG solutions do not focus on queries that may require fetching multiple documents with substantially different contents. Such queries occur frequently, but are challenging because the embeddings of these documents may be distant in the embedding space, making it hard to retrieve them all. This paper introduces Multi-Head RAG (MRAG), a novel scheme designed to address this gap with a simple yet powerful idea: leveraging activations of Transformer’s multi-head attention layer, instead of the decoder layer, as keys for fetching multi-aspect documents. The driving observation is that different attention heads learn to capture different data aspects. Harnessing the corresponding activations results in embeddings that represent various facets of data items and queries, improving the retrieval accuracy for complex queries. We provide an evaluation methodology and metrics, multi-aspect datasets, and real-world use cases to demonstrate MRAG’s effectiveness. We show MRAG’s design advantages over 18 RAG baselines, empirical improvements of up to 20% in retrieval success ratios, and benefits for downstream LLM generation. MRAG can be seamlessly integrated with existing RAG frameworks and benchmarks.

Method: Created SciRIFF dataset with 137K expert-written instruction-following instances covering 54 tasks across 5 scientific capabilities. Fine-tuned LLMs using a mix of general-domain and SciRIFF instructions and evaluated on 9 held-out tasks.

Result: LLMs fine-tuned on SciRIFF achieved 70.6% average improvement over baselines trained only on general-domain instructions on the SciRIFF-Eval held-out tasks.

Conclusion: SciRIFF enables effective development and evaluation of LLMs for scientific literature understanding, demonstrating significant performance improvements for scientific information extraction and synthesis tasks.

Abstract: We present SciRIFF (Scientific Resource for Instruction-Following and Finetuning), a dataset of 137K instruction-following instances for training and evaluation, covering 54 tasks. These tasks span five core scientific literature understanding capabilities: information extraction, summarization, question answering, claim verification, and classification. SciRIFF is unique in being entirely expert-written, high-quality instruction-following dataset for extracting and synthesizing information from research literature across diverse scientific fields. It features complex instructions with long input contexts, detailed task descriptions, and structured outputs. To demonstrate its utility, we finetune a series of large language models (LLMs) using a mix of general-domain and SciRIFF instructions. On nine out-of-distribution held-out tasks (referred to as SciRIFF-Eval), LLMs finetuned on SciRIFF achieve 70.6% average improvement over baselines trained only on general-domain instructions. SciRIFF facilitates the development and evaluation of LLMs to help researchers navigate the rapidly growing body of scientific literature.

Method: Uses a gradient-based pruning algorithm that operates on both edges and weights to reduce the model to a sparse skeleton while preserving core capabilities.

Result: Extracted sheaves preserve 93%-100% of model performance across linguistic and reasoning tasks while comprising only 1%-7% of original weights and connections, with superior modularity compared to previous circuits.

Conclusion: DiscoGP effectively identifies highly sparse, functional modules in LMs and provides novel insights into LLM inner workings when extended to neuron-level analysis.

Abstract: In this paper, we introduce DiscoGP, a novel framework for extracting self-contained modular units, or sheaves, within neural language models (LMs). Sheaves extend the concept of functional circuits, a unit widely explored in interpretability research, by considering not only subsets of edges in an LM’s computation graph but also the model’s weight parameters. Our framework identifies sheaves through a gradient-based pruning algorithm that operates on both of these in such a way that reduces the original LM to a sparse skeleton that preserves certain core capabilities. Experimental results demonstrate that, across a range of linguistic and reasoning tasks, DiscoGP extracts sheaves that preserve 93%-100% of a model’s performance on the identified task while comprising only 1%-7% of the original weights and connections. Furthermore, our analysis reveals that, compared to previously identified LM circuits, the sheaves discovered by DiscoGP exhibit superior modularity and functional fidelity. Extending our method to the neuron level also unveils novel insights into the inner workings of LLMs

Method: Uses one-vs-all decomposition of multi-class task into binary subtasks, pairs SciBERT and XLNet models independently tuned for each citation intent, aggregates outputs via feedforward neural network meta-classifier, employs SHAP for interpretability, and incorporates section titles as structural context.

Result: Achieves state-of-the-art Macro-F1 scores of 89.60% on SciCite and 76.24% on ACL-ARC, demonstrates robust performance in imbalanced and data-scarce scenarios, and shows positive impact of section titles on classification accuracy.

Conclusion: CiteFusion provides an effective ensemble approach for citation intent classification with strong performance, interpretability features, and practical applications through a released web-based tool.

Abstract: Understanding the motivations underlying scholarly citations is essential to evaluate research impact and promote transparent scholarly communication. This study introduces CiteFusion, an ensemble framework designed to address the multi-class Citation Intent Classification task on two benchmark datasets: SciCite and ACL-ARC. The framework employs a one-vs-all decomposition of the multi-class task into class-specific binary subtasks, leveraging complementary pairs of SciBERT and XLNet models, independently tuned, for each citation intent. The outputs of these base models are aggregated through a feedforward neural network meta-classifier to reconstruct the original classification task. To enhance interpretability, SHAP (SHapley Additive exPlanations) is employed to analyze token-level contributions, and interactions among base models, providing transparency into the classification dynamics of CiteFusion, and insights about the kind of misclassifications of the ensemble. In addition, this work investigates the semantic role of structural context by incorporating section titles, as framing devices, into input sentences, assessing their positive impact on classification accuracy. CiteFusion ultimately demonstrates robust performance in imbalanced and data-scarce scenarios: experimental results show that CiteFusion achieves state-of-the-art performance, with Macro-F1 scores of 89.60% on SciCite, and 76.24% on ACL-ARC. Furthermore, to ensure interoperability and reusability, citation intents from both datasets schemas are mapped to Citation Typing Ontology (CiTO) object properties, highlighting some overlaps. Finally, we describe and release a web-based application that classifies citation intents leveraging the CiteFusion models developed on SciCite.

Method: Leveraging pre-trained LLMs to analyze images captured after each print layer/segment, identify failure modes, query printer parameters, and generate/execute corrective action plans.

Result: LLM-based agents accurately identified common 3D printing errors (inconsistent extrusion, stringing, warping, layer adhesion), determined causal parameters, and autonomously corrected them without human intervention.

Conclusion: The proposed framework effectively addresses the limitations of existing automated error detection methods and demonstrates the viability of LLMs for autonomous process monitoring and control in additive manufacturing.

Abstract: Industry 4.0 has revolutionized manufacturing by driving digitalization and shifting the paradigm toward additive manufacturing (AM). Fused Deposition Modeling (FDM), a key AM technology, enables the creation of highly customized, cost-effective products with minimal material waste through layer-by-layer extrusion, posing a significant challenge to traditional subtractive methods. However, the susceptibility of material extrusion techniques to errors often requires expert intervention to detect and mitigate defects that can severely compromise product quality. While automated error detection and machine learning models exist, their generalizability across diverse 3D printer setups, firmware, and sensors is limited, and deep learning methods require extensive labeled datasets, hindering scalability and adaptability. To address these challenges, we present a process monitoring and control framework that leverages pre-trained Large Language Models (LLMs) alongside 3D printers to detect and address printing defects. The LLM evaluates print quality by analyzing images captured after each layer or print segment, identifying failure modes and querying the printer for relevant parameters. It then generates and executes a corrective action plan. We validated the effectiveness of the proposed framework in identifying defects by comparing it against a control group of engineers with diverse AM expertise. Our evaluation demonstrated that LLM-based agents not only accurately identify common 3D printing errors, such as inconsistent extrusion, stringing, warping, and layer adhesion, but also effectively determine the parameters causing these failures and autonomously correct them without any need for human intervention.

Method: PartPrompt parses sentences into parse trees, calculates local information entropy for nodes, organizes them into a global hierarchical tree, applies root-ward and leaf-ward propagation to adjust node values, and uses recursive pruning based on adjusted values.

Result: PartPrompt achieves state-of-the-art performance across various datasets, metrics, compression ratios, and target LLMs, with superior coherence and effectiveness in extreme long prompt scenarios.

Conclusion: The proposed PartPrompt method effectively addresses limitations of existing compression approaches by incorporating linguistic rules and global structure, demonstrating superior performance in prompt compression tasks.

Abstract: Offering rich contexts to Large Language Models (LLMs) has shown to boost the performance in various tasks, but the resulting longer prompt would increase the computational cost and might exceed the input limit of LLMs. Recently, some prompt compression methods have been suggested to shorten the length of prompts by using language models to generate shorter prompts or by developing computational models to select important parts of original prompt. The generative compression methods would suffer from issues like hallucination, while the selective compression methods have not involved linguistic rules and overlook the global structure of prompt. To this end, we propose a novel selective compression method called PartPrompt. It first obtains a parse tree for each sentence based on linguistic rules, and calculates local information entropy for each node in a parse tree. These local parse trees are then organized into a global tree according to the hierarchical structure such as the dependency of sentences, paragraphs, and sections. After that, the root-ward propagation and leaf-ward propagation are proposed to adjust node values over the global tree. Finally, a recursive algorithm is developed to prune the global tree based on the adjusted node values. The experiments show that PartPrompt receives the state-of-the-art performance across various datasets, metrics, compression ratios, and target LLMs for inference. The in-depth ablation studies confirm the effectiveness of designs in PartPrompt, and other additional experiments also demonstrate its superiority in terms of the coherence of compressed prompts and in the extreme long prompt scenario.

Method: Introduce Direct Quality Optimization (DQO), a variant of DPO that uses a pre-trained translation quality estimation model as a proxy for human preferences.

Result: Task-alignment improves performance across all languages of a multilingual model, even when only applied to a subset of languages, verified by both automatic metrics and human evaluation.

Conclusion: Task-alignment techniques effectively address task-data mismatch in NMT and provide consistent improvements across multilingual models.

Abstract: Reinforcement Learning from Human Feedback (RLHF) and derivative techniques like Direct Preference Optimization (DPO) are task-alignment algorithms used to repurpose general, foundational models for specific tasks. We show that applying task-alignment to neural machine translation (NMT) addresses an existing task–data mismatch in NMT, leading to improvements across all languages of a multilingual model, even when task-alignment is only applied to a subset of those languages. We do so by introducing Direct Quality Optimization (DQO), a variant of DPO leveraging a pre-trained translation quality estimation model as a proxy for human preferences, and verify the improvements with both automatic metrics and human evaluation.

Method: Used contextual bandit version of in-context reinforcement learning, experimenting with challenging classification tasks across LLMs of varying sizes (500M to 70B parameters), addressing process instability and testing with both semantic and abstract labels.

Result: LLMs effectively demonstrate in-context reinforcement learning capabilities, showing scaling trends with model size, but also reveal fundamental limitations in their implicit reasoning about errors.

Conclusion: LLMs possess significant ICRL capabilities that scale with model size, but their error reasoning remains a fundamental limitation that needs addressing.

Abstract: Large Language Models (LLMs) excel at in-context learning (ICL), a supervised learning technique that relies on adding annotated examples to the model context. We investigate a contextual bandit version of in-context reinforcement learning (ICRL), where models learn in-context, online, from external reward, instead of supervised data. We show that LLMs effectively demonstrate such learning, and provide a detailed study of the phenomena, experimenting with challenging classification tasks and models of sizes from 500M to 70B parameters. This includes identifying and addressing the instability of the process, demonstrating learning with both semantic and abstract labels, and showing scaling trends. Our findings highlight ICRL capabilities in LLMs, while also underscoring fundamental limitations in their implicit reasoning about errors.

Method: Developed AERA Chat platform that leverages multiple LLMs concurrently for scoring and rationale generation, with visualization features highlighting critical answer components and justification elements.

Result: The platform effectively facilitates robust rationale evaluation and comparative analysis across multiple rationale-generation methods on several datasets.

Conclusion: AERA Chat addresses the explainability gap in automated student assessment by providing interactive visualization and evaluation tools that enhance trust and usability for educators and researchers.

Abstract: Explainability in automated student answer scoring systems is critical for building trust and enhancing usability among educators. Yet, generating high-quality assessment rationales remains challenging due to the scarcity of annotated data and the prohibitive cost of manual verification, prompting heavy reliance on rationales produced by large language models (LLMs), which are often noisy and unreliable. To address these limitations, we present AERA Chat, an interactive visualization platform designed for automated explainable student answer assessment. AERA Chat leverages multiple LLMs to concurrently score student answers and generate explanatory rationales, offering innovative visualization features that highlight critical answer components and rationale justifications. The platform also incorporates intuitive annotation and evaluation tools, supporting educators in marking tasks and researchers in evaluating rationale quality from different models. We demonstrate the effectiveness of our platform through evaluations of multiple rationale-generation methods on several datasets, showcasing its capability for facilitating robust rationale evaluation and comparative analysis.

Method: The researchers demonstrate attacks by monitoring per-iteration token counts and packet sizes to infer speculation patterns, testing across four speculative-decoding schemes (REST, LADE, BiLD, EAGLE) and evaluating in research prototypes and vLLM framework.

Result: Attackers can fingerprint user queries with >90% accuracy across schemes (REST 100%, LADE up to 92%, BiLD up to 95%, EAGLE up to 77.6%) and leak confidential datastore contents at rates exceeding 25 tokens/sec.

Conclusion: The paper proposes mitigations including packet padding and iteration-wise token aggregation to defend against these side-channel attacks in speculative decoding implementations.

Abstract: Deployed large language models (LLMs) often rely on speculative decoding, a technique that generates and verifies multiple candidate tokens in parallel, to improve throughput and latency. In this work, we reveal a new side-channel whereby input-dependent patterns of correct and incorrect speculations can be inferred by monitoring per-iteration token counts or packet sizes.We demonstrate that an adversary observing these patterns can fingerprint user queries with >90% accuracy across four speculative-decoding schemes, REST (100%), LADE (up to 92%), BiLD (up to 95%), and EAGLE (up to 77.6%) and leak confidential datastore contents used for prediction at rates exceeding 25 tokens/sec. We evaluate the side-channel attacks in both research prototypes as well as the production-grade vLLM serving framework. To defend against these, we propose and evaluate a suite of mitigations, including packet padding and iteration-wise token aggregation.

Method: Proposes UniHR with two modules: HiDR unifies different KG types into triple-based representations, and HiSL performs intra-fact and inter-fact message passing to enhance semantic and structural information.

Result: Extensive experiments on 9 datasets across 5 KG types demonstrate UniHR’s effectiveness and the strong potential of unified representations for complex scenarios.

Conclusion: UniHR successfully addresses limitations of specialized approaches and shows unified hierarchical representation learning is effective for handling diverse complex facts in real-world knowledge graphs.

Abstract: Real-world knowledge graphs (KGs) contain not only standard triple-based facts, but also more complex, heterogeneous types of facts, such as hyper-relational facts with auxiliary key-value pairs, temporal facts with additional timestamps, and nested facts that imply relationships between facts. These richer forms of representation have attracted significant attention due to their enhanced expressiveness and capacity to model complex semantics in real-world scenarios. However, most existing studies suffer from two main limitations: (1) they typically focus on modeling only specific types of facts, thus making it difficult to generalize to real-world scenarios with multiple fact types; and (2) they struggle to achieve generalizable hierarchical (inter-fact and intra-fact) modeling due to the complexity of these representations. To overcome these limitations, we propose UniHR, a Unified Hierarchical Representation learning framework, which consists of a learning-optimized Hierarchical Data Representation (HiDR) module and a unified Hierarchical Structure Learning (HiSL) module. The HiDR module unifies hyper-relational KGs, temporal KGs, and nested factual KGs into triple-based representations. Then HiSL incorporates intra-fact and inter-fact message passing, focusing on enhancing both semantic information within individual facts and enriching the structural information between facts. To go beyond the unified method itself, we further explore the potential of unified representation in complex real-world scenarios, including joint modeling of multi-task, compositional and hybrid facts. Extensive experiments on 9 datasets across 5 types of KGs demonstrate the effectiveness of UniHR and highlight the strong potential of unified representations.

Method: ElChat directly adapts chat models on target unlabeled data by injecting information from the source chat model to elicit chat abilities, eliminating the need for a base model.

Result: ElChat achieves more robust target language and safety performance while maintaining superior English, chat, and instruction-following abilities compared to the chat vector method.

Conclusion: ElChat provides an effective direct adaptation approach for chat LLMs that preserves chat capabilities while enabling language adaptation without requiring target chat data.

Abstract: Vocabulary expansion (VE) is the de-facto approach to language adaptation of large language models (LLMs) by adding new tokens and continuing pre-training on target data. While this is effective for base models trained on unlabeled data, it poses challenges for chat models trained to follow instructions through labeled conversation data. Directly adapting the latter with VE on target unlabeled data may result in forgetting chat abilities. While ideal, target chat data is often unavailable or costly to create for low-resource languages, and machine-translated alternatives are not always effective. To address this issue, previous work proposed using a base and chat model from the same family. This method first adapts the base LLM with VE on target unlabeled data and then converts it to a chat model by adding a chat vector (CV) derived from the weight difference between the source base and chat models. We propose ElChat, a new language adaptation method for chat LLMs that adapts a chat model directly on target unlabeled data, without a base model. It elicits chat abilities by injecting information from the source chat model. ElChat offers more robust and competitive target language and safety performance while achieving superior English, chat, and instruction-following abilities compared to CV.

Method: Created LongProc benchmark with six diverse procedural generation tasks that require following detailed instructions, synthesizing dispersed information, and generating structured long-form outputs. Evaluated 23 LCLMs at three difficulty levels (500, 2K, and 8K output tokens) using rule-based evaluation.

Result: Open-weight models typically fail on 2K-token tasks, and closed-source models like GPT-4o show significant degradation on 8K-token tasks. Reasoning models perform better overall due to long CoT training. Models struggle with long-range coherence in long-form generations.

Conclusion: Current LCLMs have critical limitations in handling long-form generation tasks despite their claimed large context windows, indicating substantial room for improvement in long-context capabilities.

Abstract: Existing benchmarks for evaluating long-context language models (LCLMs) primarily focus on long-context recall, requiring models to produce short responses based on a few critical snippets while processing thousands of irrelevant tokens. We introduce LongProc (Long Procedural Generation), a new benchmark that requires both the integration of highly dispersed information and long-form generation. LongProc consists of six diverse procedural generation tasks, such as extracting structured information from HTML pages into a TSV format and executing complex search procedures to create travel plans. These tasks challenge LCLMs by testing their ability to follow detailed procedural instructions, synthesize and reason over dispersed information, and generate structured, long-form outputs (up to 8K tokens). Furthermore, as these tasks adhere to deterministic procedures and yield structured outputs, they enable reliable rule-based evaluation. We evaluated 23 LCLMs, including instruction-tuned models and recent reasoning models, on LongProc at three difficulty levels, with the maximum number of output tokens set at 500, 2K, and 8K. Notably, while all tested models claim a context window size above 32K tokens, open-weight models typically falter on 2K-token tasks, and closed-source models like GPT-4o show significant degradation on 8K-token tasks. Reasoning models achieve stronger overall performance in long-form generation, benefiting from long CoT training. Further analysis reveals that LCLMs struggle to maintain long-range coherence in long-form generations. These findings highlight critical limitations in current LCLMs and suggest substantial room for improvement. Data and code available at: https://princeton-pli.github.io/LongProc.

Method: Formulate tokenization as an optimization objective, prove NP-hardness via reduction from vertex cover, and propose GreedTok - a polynomial-time greedy algorithm that relaxes to the weighted maximum coverage problem with (1-1/e)-approximation guarantee.

Result: GreedTok outperforms BPE and Unigram on compression metrics, achieves comparable covering scores to GreedWMC, and when used for pre-training 1B parameter transformers, achieves lower bits per byte than BPE even when controlling for dataset size or training tokens.

Conclusion: Tokenization can be effectively formulated as an optimization problem, and GreedTok provides a theoretically grounded alternative to BPE with superior compression performance in practical language model training.

Abstract: Tokenization is the process of encoding strings into tokens of a fixed vocabulary size, and is widely utilized in Natural Language Processing applications. The leading tokenization algorithm today is Byte-Pair Encoding (BPE), which formulates the tokenization problem as a compression problem and tackles it by performing sequences of merges. In this work, we formulate tokenization as an optimization objective, show that it is NP-hard via a simple reduction from vertex cover, and propose a polynomial-time greedy algorithm GreedTok. Our formulation naturally relaxes to the well-studied weighted maximum coverage problem which has a simple $(1 - 1/e)$-approximation algorithm GreedWMC. Through empirical evaluations on real-world corpora, we show that GreedTok outperforms BPE and Unigram on compression and achieves a covering score comparable to GreedWMC. Finally, our extensive pre-training for two transformer-based language models with 1 billion parameters, comparing the choices of BPE and GreedTok as the tokenizer, shows that GreedTok achieves a lower bit per byte even when we control for either the total dataset proportion or total training tokens.

Method: GraphRAG addresses these limitations through three key innovations: graph-structured knowledge representation capturing entity relationships and domain hierarchies, efficient graph-based retrieval techniques enabling context-preserving knowledge retrieval with multi-hop reasoning, and structure-aware knowledge integration algorithms.

Result: The survey systematically analyzes GraphRAG’s technical foundations and examines current implementations across various professional domains, identifying key technical challenges and promising research directions.

Conclusion: GraphRAG represents a new paradigm that revolutionizes domain-specific LLM applications by overcoming traditional RAG limitations through structured knowledge representation and advanced retrieval techniques, with collected resources available for the research community.

Abstract: Large language models (LLMs) have demonstrated remarkable capabilities in a wide range of tasks, yet their application to specialized domains remains challenging due to the need for deep expertise. Retrieval-Augmented generation (RAG) has emerged as a promising solution to customize LLMs for professional fields by seamlessly integrating external knowledge bases, enabling real-time access to domain-specific expertise during inference. Despite its potential, traditional RAG systems, based on flat text retrieval, face three critical challenges: (i) complex query understanding in professional contexts, (ii) difficulties in knowledge integration across distributed sources, and (iii) system efficiency bottlenecks at scale. This survey presents a systematic analysis of Graph-based Retrieval-Augmented Generation (GraphRAG), a new paradigm that revolutionizes domain-specific LLM applications. GraphRAG addresses traditional RAG limitations through three key innovations: (i) graph-structured knowledge representation that explicitly captures entity relationships and domain hierarchies, (ii) efficient graph-based retrieval techniques that enable context-preserving knowledge retrieval with multihop reasoning ability, and (iii) structure-aware knowledge integration algorithms that leverage retrieved knowledge for accurate and logical coherent generation of LLMs. In this survey, we systematically analyze the technical foundations of GraphRAG and examine current implementations across various professional domains, identifying key technical challenges and promising research directions. All the related resources of GraphRAG, including research papers, open-source data, and projects, are collected for the community in https://github.com/DEEP-PolyU/Awesome-GraphRAG.

Method: A neurosymbolic framework integrating specialized concept parsing, GPT-4o inference, semi-supervised label propagation, and hierarchical taxonomy to create a structured knowledge base of 44k knowledge triplets.

Result: ESGSenticNet outperforms state-of-the-art baselines by 26% on ESG relatedness and 31% on ESG action orientation, capturing more unique ESG topic terms without requiring training.

Conclusion: ESGSenticNet effectively extracts relevant sustainability information from disclosures as a simple lexical method, making it accessible for non-technical stakeholders.

Abstract: Evaluating corporate sustainability performance is essential to drive sustainable business practices, amid the need for a more sustainable economy. However, this is hindered by the complexity and volume of corporate sustainability data (i.e. sustainability disclosures), not least by the effectiveness of the NLP tools used to analyse them. To this end, we identify three primary challenges - immateriality, complexity, and subjectivity, that exacerbate the difficulty of extracting insights from sustainability disclosures. To address these issues, we introduce ESGSenticNet, a publicly available knowledge base for sustainability analysis. ESGSenticNet is constructed from a neurosymbolic framework that integrates specialised concept parsing, GPT-4o inference, and semi-supervised label propagation, together with a hierarchical taxonomy. This approach culminates in a structured knowledge base of 44k knowledge triplets - (‘halve carbon emission’, supports, ’emissions control’), for effective sustainability analysis. Experiments indicate that ESGSenticNet, when deployed as a lexical method, more effectively captures relevant and actionable sustainability information from sustainability disclosures compared to state of the art baselines. Besides capturing a high number of unique ESG topic terms, ESGSenticNet outperforms baselines on the ESG relatedness and ESG action orientation of these terms by 26% and 31% respectively. These metrics describe the extent to which topic terms are related to ESG, and depict an action toward ESG. Moreover, when deployed as a lexical method, ESGSenticNet does not require any training, possessing a key advantage in its simplicity for non-technical stakeholders.

Method: Using a chess game analogy (opening, middle, end games), the study analyzes segment-specific patterns in human and AI texts to reveal where the most striking differences lie.

Result: AI texts closely resemble human writing in the body segment due to its length, but deeper analysis shows higher divergence in features dependent on continuous language flow, making the body segment most informative for detection. Human texts exhibit greater stylistic variation across segments.

Conclusion: The findings provide fresh insights into human-AI text differences and pave the way for more effective and interpretable detection strategies, with the body segment being particularly revealing despite superficial similarities.

Abstract: The rapid advancement of Large Language Models (LLMs) has revolutionized text generation but also raised concerns about potential misuse, making detecting LLM-generated text (AI text) increasingly essential. While prior work has focused on identifying AI text and effectively checkmating it, our study investigates a less-explored territory: portraying the nuanced distinctions between human and AI texts across text segments (introduction, body, and conclusion). Whether LLMs excel or falter in incorporating linguistic ingenuity across text segments, the results will critically inform their viability and boundaries as effective creative assistants to humans. Through an analogy with the structure of chess games, comprising opening, middle, and end games, we analyze segment-specific patterns to reveal where the most striking differences lie. Although AI texts closely resemble human writing in the body segment due to its length, deeper analysis shows a higher divergence in features dependent on the continuous flow of language, making it the most informative segment for detection. Additionally, human texts exhibit greater stylistic variation across segments, offering a new lens for distinguishing them from AI. Overall, our findings provide fresh insights into human-AI text differences and pave the way for more effective and interpretable detection strategies. Codes available at https://github.com/tripto03/chess_inspired_human_ai_text_distinction.

Method: ZIP score (Zero-shot Importance of Perturbation) uses controlled perturbations including synonym replacement, co-hyponym substitution, and strategic removal to quantify word importance across four models, seven prompts, and multiple task domains.

Result: Found four key patterns: task-specific word hierarchies, proprietary models align better with human intuitions, nouns dominate importance rankings, and word importance inversely correlates with model performance. ZIP achieved 90% accuracy on validation benchmark versus LIME’s 60%.

Conclusion: The study advances prompt science by providing practical insights for prompt engineering and theoretical understanding of word-level effects in LLMs, establishing the first ground-truth benchmark for prompt interpretability.

Abstract: While zero-shot instructional prompts like “Let’s think step-by-step” have revolutionized Large Language Model performance, a fundamental question remains unanswered: which specific words drive their remarkable effectiveness? We introduce the ZIP score (Zero-shot Importance of Perturbation), the first systematic method to quantify individual word importance in instructional prompts through controlled perturbations including synonym replacement, co-hyponym substitution, and strategic removal. Our analysis across four flagship models, seven widely-adopted prompts, and multiple task domains reveals four key findings: (1) Task-specific word hierarchies exist where mathematical problems prioritize “step-by-step” while reasoning tasks favor “think”; (2) Proprietary models show superior alignment with human intuitions compared to open-source alternatives; (3) Nouns dominate importance rankings, consistently representing the majority of significant words; and (4) Word importance inversely correlates with model performance, indicating prompts have greatest impact where models struggle most. Beyond revealing these patterns, we establish the first ground-truth benchmark for prompt interpretability through 20 validation prompts with predetermined key words, where ZIP achieves 90% accuracy versus LIME’s 60%. Our findings advance prompt science, the study of how language shapes model behavior, providing both practical insights for prompt engineering and theoretical understanding of word-level effects in LLMs.

Method: Decomposes user prompts into simpler queries and constraints, trains UltraComposer to compose constraint-associated prompts with evaluation questions, and synthesizes complicated instructions while filtering responses.

Result: Successfully aligned LLaMA-3.1-8B-Base to match its instruct version on 5 benchmarks without benchmark information, using only 8B models. Also improved LLaMA-3.1-8B-Instruct through self-alignment.

Conclusion: UltraIF provides an effective approach for instruction-following alignment using open-source data, with potential for broader applications in model improvement.

Abstract: Instruction-following made modern large language models (LLMs) helpful assistants. However, the key to taming LLMs on complex instructions remains mysterious, for that there are huge gaps between models trained by open-source community and those trained by leading companies. To bridge the gap, we propose a simple and scalable approach UltraIF for building LLMs that can follow complex instructions with open-source data. UltraIF first decomposes real-world user prompts into simpler queries, constraints, and corresponding evaluation questions for the constraints. Then, we train an UltraComposer to compose constraint-associated prompts with evaluation questions. This prompt composer allows us to synthesize complicated instructions as well as filter responses with evaluation questions. In our experiment, for the first time, we successfully align LLaMA-3.1-8B-Base to catch up with its instruct version on 5 instruction-following benchmarks without any benchmark information, using only 8B model as response generator and evaluator. The aligned model also achieved competitive scores on other benchmarks. Moreover, we also show that UltraIF could further improve LLaMA-3.1-8B-Instruct through self-alignment, motivating broader use cases for the method. Our code is available at https://github.com/kkk-an/UltraIF.

Method: Partition data into subgroups based on attributes (text length, writing style), then implement FairOPT to learn optimal decision thresholds for each group to reduce classification discrepancy.

Result: FairOPT significantly reduced discrepancy across 9 detectors and 3 datasets, decreasing overall discrepancy by 27.4% across 5 metrics while only sacrificing 0.005% accuracy.

Conclusion: The framework enables more robust AI-generated content detection through post-processing optimization of group-specific thresholds, mitigating bias while maintaining accuracy.

Abstract: The advancement of large language models (LLMs) has made it difficult to differentiate human-written text from AI-generated text. Several AI-text detectors have been developed in response, which typically utilize a fixed global threshold (e.g., $\theta = 0.5$) to classify machine-generated text. However, one universal threshold could fail to account for distributional variations by subgroups. For example, when using a fixed threshold, detectors make more false positive errors on shorter human-written text, and more positive classifications of neurotic writing styles among long texts. These discrepancies can lead to misclassifications that disproportionately affect certain groups. We address this critical limitation by introducing FairOPT, an algorithm for group-specific threshold optimization for probabilistic AI-text detectors. We partitioned data into subgroups based on attributes (e.g., text length and writing style) and implemented FairOPT to learn decision thresholds for each group to reduce discrepancy. FairOPT showed notable discrepancy mitigation across nine detectors and three heterogeneous datasets, and the remarkable mitigation of the minimax problem by decreasing overall discrepancy 27.4% across five metrics while minimally sacrificing accuracy by 0.005%. Our framework paves the way for more robust classification in AI-generated content detection via post-processing. We release our data, code, and project information at URL.

Method: Confidence-Informed Self-Consistency (CISC) performs weighted majority voting based on model-generated confidence scores, prioritizing high-confidence paths to reduce sample requirements.

Result: CISC outperforms self-consistency in nearly all configurations across nine models and four datasets, reducing required reasoning paths by over 40% on average.

Conclusion: LLMs can effectively judge their own outputs’ correctness, and within-question confidence evaluation is crucial for distinguishing correct answers, with the most calibrated confidence method proving least effective for CISC.

Abstract: Self-consistency decoding enhances LLMs’ performance on reasoning tasks by sampling diverse reasoning paths and selecting the most frequent answer. However, it is computationally expensive, as sampling many of these (lengthy) paths is required to increase the chances that the correct answer emerges as the most frequent one. To address this, we introduce Confidence-Informed Self-Consistency (CISC). CISC performs a weighted majority vote based on confidence scores obtained directly from the model. By prioritizing high-confidence paths, it can identify the correct answer with a significantly smaller sample size. When tested on nine models and four datasets, CISC outperforms self-consistency in nearly all configurations, reducing the required number of reasoning paths by over 40% on average. In addition, we introduce the notion of within-question confidence evaluation, after showing that standard evaluation methods are poor predictors of success in distinguishing correct and incorrect answers to the same question. In fact, the most calibrated confidence method proved to be the least effective for CISC. Lastly, beyond these practical implications, our results and analyses show that LLMs can effectively judge the correctness of their own outputs, contributing to the ongoing debate on this topic.

Method: PAFT generates diverse synthetic prompts and continuously samples from them during training, forcing models to learn fundamental task principles rather than surface-level patterns.

Result: PAFT achieves 7% higher generalization accuracy on unseen prompts, superior performance on QA, math reasoning, and tool use benchmarks, and 3.2x faster inference speeds due to reduced prompt sensitivity.

Conclusion: PAFT effectively enhances LLM robustness and cross-domain generalization while improving overall performance and inference efficiency.

Abstract: Fine-tuning large language models (LLMs) often causes overfitting to specific prompt wording, where minor phrasing variations drastically reduce performance. To address this, we propose Prompt-Agnostic Fine-Tuning (PAFT), a method that enhances robustness through dynamic prompt variation during training. PAFT first generates diverse synthetic prompts, then continuously samples from this set to construct training instances, forcing models to learn fundamental task principles rather than surface-level patterns. Across systematic evaluations using both supervised fine-tuning (SFT) and reinforcement learning fine-tuning (RLFT), PAFT demonstrates substantially improved prompt robustness, achieving 7% higher generalization accuracy on unseen prompts than standard methods. In addition to enhanced robustness, PAFT consistently yields superior overall performance on established benchmarks for question answering, mathematical reasoning, and tool use. Notably, models trained with PAFT attain 3.2 faster inference speeds due to reduced prompt sensitivity. Ablation studies further validate effectiveness of PAFT, while theoretical analysis reveals that PAFT can effectively enhance the cross-domain generalization ability of LLM.

Method: Directly transform pre-trained language models into B-cos LMs by combining B-cos conversion and task fine-tuning, improving efficiency compared to previous methods.

Result: B-cos LMs produce more faithful and human interpretable explanations than post-hoc methods while maintaining task performance comparable to conventional fine-tuning.

Conclusion: B-cos LMs successfully extend explainable architectures to NLP tasks, providing better explanations while preserving performance, with potential applications to decoder-only models for generation tasks.

Abstract: Post-hoc explanation methods for black-box models often struggle with faithfulness and human interpretability due to the lack of explainability in current neural architectures. Meanwhile, B-cos networks have been introduced to improve model explainability by proposing an architecture that removes bias terms and promotes input-weight alignment. Although B-cos networks have shown success in building explainable systems, their application has so far been limited to computer vision models and their associated training pipelines. In this work, we introduce B-cos LMs, i.e., B-cos language models (LMs) empowered for natural language processing (NLP) tasks. Our approach directly transforms pre-trained language models into B-cos LMs by combining B-cos conversion and task fine-tuning, improving efficiency compared to previous methods. Our automatic and human evaluation results demonstrate that B-cos LMs produce more faithful and human interpretable explanations than post-hoc methods, while maintaining task performance comparable to conventional fine-tuning. Our in-depth analysis explores how B-cos LMs differ from conventionally fine-tuned models in their learning processes and explanation patterns. Finally, we present a first exploration of transforming decoder-only models to B-cos LMs for generation tasks.

Method: Created first multilingual (Arabic/English) explanation-enhanced dataset and developed an explanation-enhanced LLM for both detection and explanation generation.

Result: The model performs comparably to detection-only models while also generating explanations for predictions.

Conclusion: Successfully addressed the explanation gap in propaganda detection and will release dataset/resources publicly to advance research.

Abstract: There has been significant research on propagandistic content detection across different modalities and languages. However, most studies have primarily focused on detection, with little attention given to explanations justifying the predicted label. This is largely due to the lack of resources that provide explanations alongside annotated labels. To address this issue, we propose a multilingual (i.e., Arabic and English) explanation-enhanced dataset, the first of its kind. Additionally, we introduce an explanation-enhanced LLM for both label detection and rationale-based explanation generation. Our findings indicate that the model performs comparably while also generating explanations. We will make the dataset and experimental resources publicly available for the research community (https://github.com/firojalam/PropXplain).

Method: Created MemeXplain dataset for Arabic propagandistic memes and English hateful memes, and proposed a multi-stage optimization approach to train Vision-Language Models for joint label detection and explanation generation.

Result: The approach significantly improved both tasks, achieving ~1.4% absolute accuracy improvement on ArMeme and ~2.2% on Hateful Memes compared to state-of-the-art methods.

Conclusion: Multi-stage optimization effectively addresses the challenge of joint training for detection and explanation, and the MemeXplain dataset provides a valuable resource for future multimodal content analysis research.

Abstract: The proliferation of multimodal content on social media presents significant challenges in understanding and moderating complex, context-dependent issues such as misinformation, hate speech, and propaganda. While efforts have been made to develop resources and propose new methods for automatic detection, limited attention has been given to jointly modeling label detection and the generation of explanation-based rationales, which often leads to degraded classification performance when trained simultaneously. To address this challenge, we introduce MemeXplain, an explanation-enhanced dataset for propagandistic memes in Arabic and hateful memes in English, making it the first large-scale resource for these tasks. To solve these tasks, we propose a multi-stage optimization approach and train Vision-Language Models (VLMs). Our results show that this strategy significantly improves both label detection and explanation generation quality over the base model, outperforming the current state-of-the-art with an absolute improvement of ~1.4% (Acc) on ArMeme and ~2.2% (Acc) on Hateful Memes. For reproducibility and future research, we aim to make the MemeXplain dataset and scripts publicly available (https://github.com/MohamedBayan/MemeIntel).

Method: A contrastive reflection synthesis pipeline generates precise verbal feedback by identifying discrepancies in structure reasoning graph paths. DARS framework uses synthetic reflection data with a dedicated Critic model trained for effective reflection.

Result: DARS achieves strong performance and consistently outperforms existing ASAS baselines across all evaluation metrics. Extensive experiments provide insights into reflection data value, framework design, and scaling behavior.

Conclusion: The DARS framework successfully addresses transparency limitations in ASAS through contrastive reflection synthesis and dual-model architecture, demonstrating superior performance and providing valuable insights for explainable assessment systems.

Abstract: Although preference optimization methods have improved reasoning performance in Large Language Models (LLMs), they often lack transparency regarding why one reasoning outcome is preferred over another. This limitation is especially critical in Automated Student Answer Scoring (ASAS), where explainability is essential to justify assessment outcomes. Verbal reinforcement learning offers the potential to generate explicit reflection, but it tends to produce superficial critiques that can harm assessment performance. Existing LLMs also struggle to reliably detect subtle reasoning errors in ASAS tasks. Moreover, manually identifying intermediate reasoning errors is expensive and difficult to scale. To address these challenges, we introduce a contrastive reflection synthesis pipeline that generates precise verbal feedback by identifying discrepancies in structure reasoning graph paths. Leveraging these synthetic reflection data, we propose DARS, a Dual-model Reflective Scoring framework featuring a dedicated Critic model trained for effective reflection. DARS achieves strong performance and consistently outperforms existing ASAS baselines across all evaluation metrics. Extensive experiments further provide novel insights into the value of reflection data, framework design, and the scaling behavior of DARS. We release the DARS code at https://github.com/lijiazheng99/DARS.

Method: Created EchoMist benchmark with implicit misinformation scenarios from diverse sources including human-AI conversations and social media, then tested 15 state-of-the-art LLMs and evaluated two mitigation methods: Self-Alert and RAG.

Result: Current LLMs perform alarmingly poorly on implicit misinformation detection, often failing to identify false premises and generating counterfactual explanations. The mitigation methods showed EchoMist remains a persistent challenge.

Conclusion: Implicit misinformation poses a critical safety risk for LLMs that current models cannot adequately handle, underscoring the urgent need for better safeguards against this subtle form of misinformation.

Abstract: As Large Language Models (LLMs) are widely deployed in diverse scenarios, the extent to which they could tacitly spread misinformation emerges as a critical safety concern. Current research primarily evaluates LLMs on explicit false statements, overlooking how misinformation often manifests subtly as unchallenged premises in real-world interactions. We curated EchoMist, the first comprehensive benchmark for implicit misinformation, where false assumptions are embedded in the query to LLMs. EchoMist targets circulated, harmful, and ever-evolving implicit misinformation from diverse sources, including realistic human-AI conversations and social media interactions. Through extensive empirical studies on 15 state-of-the-art LLMs, we find that current models perform alarmingly poorly on this task, often failing to detect false premises and generating counterfactual explanations. We also investigate two mitigation methods, i.e., Self-Alert and RAG, to enhance LLMs’ capability to counter implicit misinformation. Our findings indicate that EchoMist remains a persistent challenge and underscore the critical need to safeguard against the risk of implicit misinformation.

Method: AGPO uses an adaptive loss function to mitigate training fluctuation and token inefficiency in reasoning steps.

Result: Experiments show AGPO achieves more stable training and superior performance with significantly fewer tokens in reasoning steps.

Conclusion: AGPO effectively addresses GRPO’s deficiencies, providing more stable training and better efficiency for Reasoning LLMs.

Abstract: Since DeepSeek-R1 popularized, Group Relative Policy Optimization (GRPO) has become the core part of training Reasoning LLMs. However, we find some deficiency that influences RL stability and inference efficiency, like zero-variance in advantage estimation. Thus, we propose Adaptive Group Policy Optimization (AGPO) which uses a simple but effective method, an adaptive loss function, to mitigate training fluctuation and token inefficiency. The experiments demonstrate our method achieves more stable training and superior performance with significantly fewer tokens in reasoning steps.

Method: Constructs dataset of copyright infringement cases, uses Direct Preference Optimization to replace verbatim content with alternatives, and integrates gradient projection and Fisher information regularization to preserve performance.

Result: Significantly reduces verbatim memorization of copyrighted books with negligible impact on unrelated task performance, validated on 500 famous books and public benchmarks.

Conclusion: SUV offers a scalable and effective solution for mitigating copyright risks in real-world LLM applications while maintaining model utility.

Abstract: Large Language Models (LLMs) have transformed natural language processing by learning from massive datasets, yet this rapid progress has also drawn legal scrutiny, as the ability to unintentionally generate copyrighted content has already prompted several prominent lawsuits. In this work, we introduce SUV (Selective Unlearning for Verbatim data), a selective unlearning framework designed to prevent LLM from memorizing copyrighted content while preserving its overall utility. In detail, the proposed method constructs a dataset that captures instances of copyrighted infringement cases by the targeted LLM. With the dataset, we unlearn the content from the LLM by means of Direct Preference Optimization (DPO), which replaces the verbatim copyrighted content with plausible and coherent alternatives. Since DPO may hinder the LLM’s performance in other unrelated tasks, we integrate gradient projection and Fisher information regularization to mitigate the degradation. We validate our approach using a large-scale dataset of 500 famous books (predominantly copyrighted works) and demonstrate that SUV significantly reduces verbatim memorization with negligible impact on the performance on unrelated tasks. Extensive experiments on both our dataset and public benchmarks confirm the scalability and efficacy of our approach, offering a promising solution for mitigating copyright risks in real-world LLM applications.

Method: Proposes XL-Instruct for generating high-quality synthetic data and introduces XL-AlpacaEval benchmark for evaluating cross-lingual generation. Fine-tunes LLMs with 8K XL-Instruct generated instructions.

Result: Fine-tuning with XL-Instruct significantly improves model performance, increasing win rate against GPT-4o-Mini from 7.4% to 21.5%. Also shows strong zero-shot improvements to question answering in the same language.

Conclusion: XL-Instruct shows promising role in post-training of multilingual LLMs. XL-Suite (training and evaluation data) is publicly released to facilitate cross-lingual open-ended generation research.

Abstract: Cross-lingual open-ended generation - responding in a language different from that of the query - is an important yet understudied problem. This work proposes XL-Instruct, a novel technique for generating high-quality synthetic data, and introduces XL-AlpacaEval, a new benchmark for evaluating cross-lingual generation capabilities of large language models (LLMs). Our experiments show that fine-tuning with just 8K instructions generated using XL-Instruct significantly improves model performance, increasing the win rate against GPT-4o-Mini from 7.4% to 21.5% and improving on several fine-grained quality metrics. Moreover, base LLMs fine-tuned on XL-Instruct exhibit strong zero-shot improvements to question answering in the same language, as shown on our machine-translated m-AlpacaEval. These consistent gains highlight the promising role of XL-Instruct in the post-training of multilingual LLMs. Finally, we publicly release XL-Suite, a collection of training and evaluation data to facilitate research in cross-lingual open-ended generation.

Method: During prefilling, tokens are grouped by sentence-level semantic similarity and compressed into semantic vectors stored on GPU, while individual KV pairs are offloaded to CPU. During decoding, tokens are generated by selectively retrieving semantically relevant sentence-level KV entries using semantic similarity between prefilling vectors and decoding queries.

Result: Extensive evaluations on PG-19, LongBench, and Needle-In-A-Haystack benchmarks show SentenceKV significantly outperforms state-of-the-art methods in both efficiency and memory usage without compromising model accuracy.

Conclusion: SentenceKV provides an effective solution for efficient long-context inference by leveraging sentence-level semantic grouping and selective retrieval, achieving substantial memory reduction while maintaining stable inference latency and model accuracy.

Abstract: Large language models face significant computational and memory challenges when processing long contexts. During inference, efficient management of the key-value (KV) cache, which stores intermediate activations for autoregressive generation, is critical to reducing memory overhead and improving computational efficiency. Traditional token-level efficient KV caching methods overlook semantic information, treating tokens independently without considering their semantic relationships. Meanwhile, existing semantic-preserving KV cache management approaches often suffer from substantial memory usage and high time-to-first-token. To address these limitations, we propose SentenceKV, a novel sentence-level semantic KV caching approach designed to enhance inference efficiency while preserving semantic coherence. During prefilling, SentenceKV groups tokens based on sentence-level semantic similarity, compressing sentence representations into concise semantic vectors stored directly on the GPU, while individual KV pairs are offloaded to CPU. During decoding, SentenceKV generates tokens by selectively retrieving semantically relevant sentence-level KV entries, leveraging the semantic similarity between the prefilling-stage semantic vectors and decoding-stage queries. This ensures efficient and contextually accurate predictions, minimizing the loading of redundant or irrelevant data into GPU memory and significantly reducing memory overhead while maintaining stable inference latency, even for extremely long contexts. Extensive evaluations on benchmarks including PG-19, LongBench, and Needle-In-A-Haystack demonstrate that SentenceKV significantly outperforms state-of-the-art methods in both efficiency and memory usage, without compromising model accuracy.

Method: Created AnesSuite with AnesBench evaluation benchmark (three reasoning levels) and training datasets for CPT, SFT, and RLVR. Developed Morpheus baseline models using SFT and GRPO training.

Result: Morpheus demonstrates substantial performance improvements rivaling larger-scale models, despite limited training. Comprehensive analysis reveals key factors influencing anesthesiology reasoning performance.

Conclusion: AnesSuite and Morpheus provide foundational infrastructure for advancing anesthesiology reasoning in LLMs, with open-source availability to support further research and development in this specialized medical domain.

Abstract: The application of large language models (LLMs) in the medical field has garnered significant attention, yet their reasoning capabilities in more specialized domains like anesthesiology remain underexplored. To bridge this gap, we introduce AnesSuite, the first comprehensive dataset suite specifically designed for anesthesiology reasoning in LLMs. The suite features AnesBench, an evaluation benchmark tailored to assess anesthesiology-related reasoning across three levels: factual retrieval (System 1), hybrid reasoning (System 1.x), and complex decision-making (System 2). Alongside this benchmark, the suite includes three training datasets that provide an infrastructure for continued pre-training (CPT), supervised fine-tuning (SFT), and reinforcement learning with verifiable rewards (RLVR). Leveraging this suite, we develop Morpheus, the first baseline model collection for anesthesiology reasoning. Despite undergoing limited training with SFT and group relative policy optimization (GRPO), Morpheus demonstrates substantial performance improvements, rivaling the performance of larger-scale models. Furthermore, through comprehensive evaluations and experiments, we analyze the key factors influencing anesthesiology reasoning performance, including model characteristics, training strategies and training data. Both AnesSuite and Morpheus will be open-sourced at https://github.com/MiliLab/AnesSuite.

Method: Comprehensive survey of detection techniques including observation-based strategies, linguistic/statistical analysis, model-based pipelines, watermarking/fingerprinting, and ensemble approaches with human-in-the-loop verification.

Result: Provides state-of-the-art research overview and case studies across academic, journalistic, legal, and industrial contexts to inform robust solutions and policymaking.

Conclusion: Identifies open challenges including adversarial transformations, domain generalization, and ethical concerns, offering holistic guidance for researchers, practitioners, and regulators to preserve content authenticity.

Abstract: Advances in AI-generated content have led to wide adoption of large language models, diffusion-based visual generators, and synthetic audio tools. However, these developments raise critical concerns about misinformation, copyright infringement, security threats, and the erosion of public trust. In this paper, we explore an extensive range of methods designed to detect and mitigate AI-generated textual, visual, and audio content. We begin by discussing motivations and potential impacts associated with AI-based content generation, including real-world risks and ethical dilemmas. We then outline detection techniques spanning observation-based strategies, linguistic and statistical analysis, model-based pipelines, watermarking and fingerprinting, as well as emergent ensemble approaches. We also present new perspectives on robustness, adaptation to rapidly improving generative architectures, and the critical role of human-in-the-loop verification. By surveying state-of-the-art research and highlighting case studies in academic, journalistic, legal, and industrial contexts, this paper aims to inform robust solutions and policymaking. We conclude by discussing open challenges, including adversarial transformations, domain generalization, and ethical concerns, thereby offering a holistic guide for researchers, practitioners, and regulators to preserve content authenticity in the face of increasingly sophisticated AI-generated media.

Method: The paper formalizes the model substitution problem and systematically evaluates detection methods under adversarial conditions. It examines software-only methods (statistical tests on text outputs and log probabilities) and proposes hardware-level security using Trusted Execution Environments (TEEs) as a robust solution.

Result: Software-only detection methods are fundamentally unreliable: statistical tests on text outputs are query-intensive and fail against subtle substitutions, while methods using log probabilities are defeated by inherent inference nondeterminism in production environments. TEEs can provide provable cryptographic guarantees of model integrity with only modest performance overhead.

Conclusion: Hardware-level security using Trusted Execution Environments offers a practical and robust solution to the LLM API trust problem, providing provable cryptographic guarantees that ensure users get what they pay for, with only modest performance costs.

Abstract: Commercial Large Language Model (LLM) APIs create a fundamental trust problem: users pay for specific models but have no guarantee that providers deliver them faithfully. Providers may covertly substitute cheaper alternatives (e.g., quantized versions, smaller models) to reduce costs while maintaining advertised pricing. We formalize this model substitution problem and systematically evaluate detection methods under realistic adversarial conditions. Our empirical analysis reveals that software-only methods are fundamentally unreliable: statistical tests on text outputs are query-intensive and fail against subtle substitutions, while methods using log probabilities are defeated by inherent inference nondeterminism in production environments. We argue that this verification gap can be more effectively closed with hardware-level security. We propose and evaluate the use of Trusted Execution Environments (TEEs) as one practical and robust solution. Our findings demonstrate that TEEs can provide provable cryptographic guarantees of model integrity with only a modest performance overhead, offering a clear and actionable path to ensure users get what they pay for. Code is available at https://github.com/sunblaze-ucb/llm-api-audit

Method: Proposes DataPuzzle - a multi-agent framework that decomposes complex questions, structures information into interpretable forms (tables, graphs), and coordinates specialized agent roles for extraction, translation, and linkage tasks.

Result: A conceptual framework blueprint that transforms LLMs from monolithic answer generators into collaborative components within transparent workflows.

Conclusion: Structure is essential for building trustworthy, auditable analytic systems with LLMs - transforming opaque answers into traceable processes and brittle fluency into accountable insight.

Abstract: Large language models (LLMs) are increasingly applied to multi-modal data analysis – not necessarily because they offer the most precise answers, but because they provide fluent, flexible interfaces for interpreting complex inputs. Yet this fluency often conceals a deeper structural failure: the prevailing ``Prompt-to-Answer’’ paradigm treats LLMs as black-box analysts, collapsing evidence, reasoning, and conclusions into a single, opaque response. The result is brittle, unverifiable, and frequently misleading. We argue for a fundamental shift: from generation to structured extraction, from monolithic prompts to modular, agent-based workflows. LLMs should not serve as oracles, but as collaborators – specialized in tasks like extraction, translation, and linkage – embedded within transparent workflows that enable step-by-step reasoning and verification. We propose DataPuzzle, a conceptual multi-agent framework that decomposes complex questions, structures information into interpretable forms (e.g. tables, graphs), and coordinates agent roles to support transparent and verifiable analysis. This framework serves as an aspirational blueprint for restoring visibility and control in LLM-driven analytics – transforming opaque answers into traceable processes, and brittle fluency into accountable insight. This is not a marginal refinement; it is a call to reimagine how we build trustworthy, auditable analytic systems in the era of large language models. Structure is not a constraint – it is the path to clarity.

Method: Categorizes existing works into three key directions: (1) shorter - compressing lengthy CoTs into concise reasoning chains, (2) smaller - developing compact language models through knowledge distillation, model compression, and reinforcement learning, and (3) faster - designing efficient decoding strategies.

Result: Provides a systematic framework for understanding and implementing efficient reasoning methods, with a curated collection of papers available in a GitHub repository.

Conclusion: The survey organizes the landscape of efficient reasoning research into three clear directions to address computational overhead in reasoning models while maintaining performance.

Abstract: Reasoning models have demonstrated remarkable progress in solving complex and logic-intensive tasks by generating extended Chain-of-Thoughts (CoTs) prior to arriving at a final answer. Yet, the emergence of this “slow-thinking” paradigm, with numerous tokens generated in sequence, inevitably introduces substantial computational overhead. To this end, it highlights an urgent need for effective acceleration. This survey aims to provide a comprehensive overview of recent advances in efficient reasoning. It categorizes existing works into three key directions: (1) shorter - compressing lengthy CoTs into concise yet effective reasoning chains; (2) smaller - developing compact language models with strong reasoning capabilities through techniques such as knowledge distillation, other model compression techniques, and reinforcement learning; and (3) faster - designing efficient decoding strategies to accelerate inference of reasoning models. A curated collection of papers discussed in this survey is available in our GitHub repository: https://github.com/fscdc/Awesome-Efficient-Reasoning-Models.

Method: Created IPBench benchmark with 8 IP mechanisms and 20 distinct tasks, then benchmarked 17 LLMs including general-purpose and domain-specific models under zero-shot, few-shot, and chain-of-thought settings.

Result: Top-performing model DeepSeek-V3 achieved only 75.8% accuracy, showing significant room for improvement. Open-source IP and law-oriented models lag behind closed-source general-purpose models.

Conclusion: IPBench addresses the gap in comprehensive IP evaluation and will be expanded with additional tasks to better reflect real-world complexities and support model advancements in the IP domain.

Abstract: Intellectual Property (IP) is a highly specialized domain that integrates technical and legal knowledge, making it inherently complex and knowledge-intensive. Recent advancements in LLMs have demonstrated their potential to handle IP-related tasks, enabling more efficient analysis, understanding, and generation of IP-related content. However, existing datasets and benchmarks focus narrowly on patents or cover limited aspects of the IP field, lacking alignment with real-world scenarios. To bridge this gap, we introduce IPBench, the first comprehensive IP task taxonomy and a large-scale bilingual benchmark encompassing 8 IP mechanisms and 20 distinct tasks, designed to evaluate LLMs in real-world IP scenarios. We benchmark 17 main LLMs, ranging from general purpose to domain-specific, including chat-oriented and reasoning-focused models, under zero-shot, few-shot, and chain-of-thought settings. Our results show that even the top-performing model, DeepSeek-V3, achieves only 75.8% accuracy, indicating significant room for improvement. Notably, open-source IP and law-oriented models lag behind closed-source general-purpose models. To foster future research, we publicly release IPBench, and will expand it with additional tasks to better reflect real-world complexities and support model advancements in the IP domain. We provide the data and code in the supplementary URLs.

Method: Monitors model behavior at reasoning transition points and dynamically terminates generation when the model shows high confidence in a trial answer, requiring no additional training.

Result: Consistently effective across 11 reasoning LLMs on 10 benchmarks, reducing CoT length by 19.1-80.1% while improving accuracy by 0.3-5.0%.

Conclusion: The proposed self-truncation method is simple, effective, and seamlessly integrates with existing reasoning LLMs to improve efficiency and accuracy.

Abstract: Recent advances in large reasoning language models (LRLMs) rely on test-time scaling, which extends long chain-of-thought (CoT) generation to solve complex tasks. However, overthinking in long CoT not only slows down the efficiency of problem solving, but also risks accuracy loss due to the extremely detailed or redundant reasoning steps. We propose a simple yet effective method that allows LLMs to self-truncate CoT sequences by early exit during generation. Instead of relying on fixed heuristics, the proposed method monitors model behavior at potential reasoning transition points and dynamically terminates the next reasoning chain’s generation when the model exhibits high confidence in a trial answer. Our method requires no additional training and can be seamlessly integrated into existing o1-like reasoning LLMs. Experiments on 10 reasoning benchmarks (e.g., GSM8K, MATH-500, AMC, GPQA, AIME and LiveCodeBench) show that the proposed method is consistently effective on 11 cutting-edge reasoning LLMs of varying series and sizes, reducing the length of CoT sequences by an average of 19.1% to 80.1% while improving accuracy by 0.3% to 5.0%.

Method: TRACE distills a Hidden Markov Model (HMM) from the language model and pairs it with a small classifier to estimate attribute probabilities. This enables exact computation of Expected Attribute Probability (EAP) over predicted futures, which is used to reweight the LM’s next-token probabilities.

Result: Achieves state-of-the-art detoxification with only 20% decoding overhead, enables 76 low-resource personalized LMs within seconds, and seamlessly extends to composite attributes.

Conclusion: TRACE provides an efficient and flexible framework for controllable text generation that adapts to new attributes through lightweight control mechanisms, overcoming limitations of existing approaches.

Abstract: As large language models (LMs) advance, there is an increasing need to control their outputs to align with human values (e.g., detoxification) or desired attributes (e.g., personalization, topic). However, autoregressive models focus on next-token predictions and struggle with global properties that require looking ahead. Existing solutions either post-train LMs for each new attribute–expensive and inflexible–or approximate the Expected Attribute Probability (EAP) of future sequences by sampling or training, which is slow and unreliable for rare attributes. We introduce TRACE (Tractable Probabilistic Reasoning for Adaptable Controllable gEneration), a novel framework that efficiently computes EAP and adapts to new attributes through tractable probabilistic reasoning and lightweight control. TRACE distills a Hidden Markov Model (HMM) from an LM and pairs it with a small classifier to estimate attribute probabilities, enabling exact EAP computation over the HMM’s predicted futures. This EAP is then used to reweigh the LM’s next-token probabilities for globally compliant continuations. Empirically, TRACE achieves state-of-the-art detoxification results with only 20% decoding overhead, yields 76 low-resource personalized LMs within seconds, and seamlessly extends to composite attributes. Our code is available at: https://github.com/yidouweng/trace.

Method: The approach uses LLMs to translate between natural language and structured representations, then performs cognitively inspired manipulations on these structured representations to recombine existing ideas in creative ways.

Result: Experiments and domain-expert evaluations show that DishCOVER generates outputs that are mostly coherent and feasible, and significantly surpass GPT-4o in terms of novelty and diversity for creative recipe generation.

Conclusion: The work demonstrates the effectiveness of structured creativity approaches in AI and hopes to inspire further research in this direction.

Abstract: Large Language Models (LLMs) excel at many tasks, yet they struggle to produce truly creative, diverse ideas. In this paper, we introduce a novel approach that enhances LLM creativity. We apply LLMs for translating between natural language and structured representations, and perform the core creative leap via cognitively inspired manipulations on these representations. Our notion of creativity goes beyond superficial token-level variations; rather, we recombine structured representations of existing ideas, enabling our system to effectively explore a more abstract landscape of ideas. We demonstrate our approach in the culinary domain with DishCOVER, a model that generates creative recipes. Experiments and domain-expert evaluations reveal that our outputs, which are mostly coherent and feasible, significantly surpass GPT-4o in terms of novelty and diversity, thus outperforming it in creative generation. We hope our work inspires further research into structured creativity in AI.

Method: Created New News dataset with hypothetical news across multiple domains; proposed System-2 Fine-tuning using self-play data generation protocols (paraphrases, implications, Self-QA) to distill knowledge into model weights.

Result: Self-QA protocol of Sys2-FT significantly improves in-weight learning while preserving general capabilities; discovered contextual shadowing effect where training with news context followed by rephrases/QAs degrades learning; found preliminary evidence of scaling law for Sys2-FT.

Conclusion: System-2 Fine-tuning with Self-QA protocol effectively bridges the FT-ICL gap for knowledge integration, though careful attention is needed to avoid contextual shadowing effects during training.

Abstract: Humans and intelligent animals can internalize new information and accurately internalize their implications to perform downstream tasks. While large language models (LLMs) can achieve this through in-context learning (ICL) when the information (news) is explicitly given as context, adequately integrating the information into model weights via fine-tuning remains challenging. In this paper, we introduce New News, a dataset composed of hypothetical yet plausible news spanning multiple domains (mathematics, coding, discoveries, leaderboards, events), accompanied by downstream evaluation questions whose correct answers critically depend on understanding and internalizing the news. First, we demonstrate a substantial gap between naive fine-tuning and in-context learning (FT-ICL gap) on our dataset. To address this gap, we explore a suite of self-play data generation protocols – paraphrases, implications, and Self-QA – designed to distill the knowledge processed by the model with context into the weights of the model, which we term System-2 Fine-tuning (Sys2-FT). We systematically evaluate ICL and Sys2-FT performance across data domains and model scales with the Qwen 2.5 family of models. Our results demonstrate that the Self-QA protocol of Sys2-FT significantly improves models’ in-weight learning of the news while preserving general capabilities. Furthermore, we discover the contextual shadowing effect, where training with the news in context followed by its rephrases or QAs catastrophically degrades learning of the news. Finally, we show preliminary evidence of an emerging scaling law of Sys2-FT.

Method: Uses two innovations: (1) response-based inference where responses serve as proxies to infer pseudo reference passages, and (2) response refinement via the dual-role of CQR where a CQR model refines responses based on shared objectives between response refinement and CQR.

Result: Achieves 96.9-99.1% of the retrieval accuracy attainable only with reference passages and surpasses state-of-the-art method by up to 31.6%.

Conclusion: DualReform provides an effective reference-free approach for conversational query reformulation that performs nearly as well as methods requiring reference passages while being more practical for real-world deployment.

Abstract: Conversational query reformulation (CQR) has become indispensable for improving retrieval in dialogue-based applications. However, existing approaches typically rely on reference passages for optimization, which are impractical to acquire in real-world scenarios. To address this limitation, we introduce a novel reference-free preference optimization framework DualReform that generates pseudo reference passages from commonly-encountered conversational datasets containing only queries and responses. DualReform attains this goal through two key innovations: (1) response-based inference, where responses serve as proxies to infer pseudo reference passages, and (2) response refinement via the dual-role of CQR, where a CQR model refines responses based on the shared objectives between response refinement and CQR. Despite not relying on reference passages, DualReform achieves 96.9–99.1% of the retrieval accuracy attainable only with reference passages and surpasses the state-of-the-art method by up to 31.6%.

Method: Provides a Python-based toolkit with prebuilt pipelines for document processing, RAG, information extraction, summarization, and classification. Supports multiple LLM backends (llama.cpp, Ollama, vLLM, Hugging Face) with quantized models, GPU acceleration, and backend switching. Includes a no-code web interface for non-technical users.

Result: A comprehensive toolkit that enables local execution of LLMs while maintaining data privacy, with support for hybrid deployments when cloud integration is permitted.

Conclusion: OnPrem.LLM successfully addresses the need for privacy-preserving LLM applications in restricted environments, offering flexibility through multiple backend support and accessibility through both programming interfaces and no-code web interface.

Abstract: We present OnPrem$.$LLM, a Python-based toolkit for applying large language models (LLMs) to sensitive, non-public data in offline or restricted environments. The system is designed for privacy-preserving use cases and provides prebuilt pipelines for document processing and storage, retrieval-augmented generation (RAG), information extraction, summarization, classification, and prompt/output processing with minimal configuration. OnPrem$.$LLM supports multiple LLM backends – including llama$.$cpp, Ollama, vLLM, and Hugging Face Transformers – with quantized model support, GPU acceleration, and seamless backend switching. Although designed for fully local execution, OnPrem$.$LLM also supports integration with a wide range of cloud LLM providers when permitted, enabling hybrid deployments that balance performance with data control. A no-code web interface extends accessibility to non-technical users.

Method: VeriFact enhances fact extraction by identifying and resolving incomplete/missing facts, and FactRBench provides reference fact sets from advanced LLMs and human answers for recall assessment.

Result: VeriFact significantly improves fact completeness and preserves complex relational facts, leading to more accurate factuality evaluation. Larger models show better precision and recall, but high precision doesn’t always correlate with high recall.

Conclusion: Comprehensive factuality assessment requires evaluating both precision and recall, as they don’t always correlate, and VeriFact provides more accurate evaluation by addressing fact extraction limitations.

Abstract: Large language models (LLMs) excel at generating long-form responses, but evaluating their factuality remains challenging due to complex inter-sentence dependencies within the generated facts. Prior solutions predominantly follow a decompose-decontextualize-verify pipeline but often fail to capture essential context and miss key relational facts. In this paper, we introduce VeriFact, a factuality evaluation framework designed to enhance fact extraction by identifying and resolving incomplete and missing facts to support more accurate verification results. Moreover, we introduce FactRBench , a benchmark that evaluates both precision and recall in long-form model responses, whereas prior work primarily focuses on precision. FactRBench provides reference fact sets from advanced LLMs and human-written answers, enabling recall assessment. Empirical evaluations show that VeriFact significantly enhances fact completeness and preserves complex facts with critical relational information, resulting in more accurate factuality evaluation. Benchmarking various open- and close-weight LLMs on FactRBench indicate that larger models within same model family improve precision and recall, but high precision does not always correlate with high recall, underscoring the importance of comprehensive factuality assessment.

Method: Proposes AutoThink, a multi-stage reinforcement learning framework that learns when to invoke explicit reasoning. It builds on R1-style distilled models and uses reward shaping to optimize reasoning policies.

Result: Experiments on five mathematical benchmarks show AutoThink achieves 6.4% relative accuracy improvement while reducing token usage by 52% on DeepSeek-R1-Distill-Qwen-1.5B, outperforming recent prompting and RL-based pruning methods.

Conclusion: AutoThink establishes a scalable and adaptive reasoning paradigm that can be seamlessly integrated into R1-style models, providing favorable accuracy-efficiency trade-offs by invoking reasoning only when necessary.

Abstract: Large reasoning models (LRMs) are proficient at generating explicit, step-by-step reasoning sequences before producing final answers. However, such detailed reasoning can introduce substantial computational overhead and latency, particularly for simple problems. To address this over-thinking problem, we explore how to equip LRMs with adaptive thinking capabilities: enabling them to dynamically decide whether or not to engage in explicit reasoning based on problem complexity. Building on R1-style distilled models, we observe that inserting a simple ellipsis ("…") into the prompt can stochastically trigger either a thinking or no-thinking mode, revealing a latent controllability in the reasoning behavior. Leveraging this property, we propose AutoThink, a multi-stage reinforcement learning (RL) framework that progressively optimizes reasoning policies via stage-wise reward shaping. AutoThink learns to invoke explicit reasoning only when necessary, while defaulting to succinct responses for simpler tasks. Experiments on five mainstream mathematical benchmarks demonstrate that AutoThink achieves favorable accuracy-efficiency trade-offs compared to recent prompting and RL-based pruning methods. It can be seamlessly integrated into any R1-style model, including both distilled and further fine-tuned variants. Notably, AutoThink improves relative accuracy by 6.4 percent while reducing token usage by 52 percent on DeepSeek-R1-Distill-Qwen-1.5B, establishing a scalable and adaptive reasoning paradigm for LRMs. Project Page: https://github.com/ScienceOne-AI/AutoThink.

Method: Developed a formal framework for analyzing transformers’ counting capabilities and proved that transformers can capture all semialgebraic counting properties through theoretical analysis.

Result: Transformers can express counting properties that are boolean combinations of arbitrary multivariate polynomials of any degree, generalizing beyond linear counting properties captured by C-RASP softmax transformers.

Conclusion: Transformers have much stronger counting capabilities than previously known, capable of expressing highly nonlinear counting properties, which also leads to new undecidability results for simple transformer models without positional encodings or masking.

Abstract: Counting properties (e.g. determining whether certain tokens occur more than other tokens in a given input text) have played a significant role in the study of expressiveness of transformers. In this paper, we provide a formal framework for investigating the counting power of transformers. We argue that all existing results demonstrate transformers’ expressivity only for (semi-)linear counting properties, i.e., which are expressible as a boolean combination of linear inequalities. Our main result is that transformers can express counting properties that are highly nonlinear. More precisely, we prove that transformers can capture all semialgebraic counting properties, i.e., expressible as a boolean combination of arbitrary multivariate polynomials (of any degree). Among others, these generalize the counting properties that can be captured by C-RASP softmax transformers, which capture only linear counting properties. To complement this result, we exhibit a natural subclass of (softmax) transformers that completely characterizes semialgebraic counting properties. Through connections with the Hilbert’s tenth problem, this expressivity of transformers also yields a new undecidability result for analyzing an extremely simple transformer model – surprisingly with neither positional encodings (i.e. NoPE-transformers) nor masking. We also experimentally validate trainability of such counting properties.

Method: Multi-stage training framework that augments SFT with teacher-generated explanatory critiques and refined responses. Students learn to map the triplet of prompt, initial response, and teacher critique into refined teacher response.

Result: Substantial gains on reasoning benchmarks (+15.0% on AMC23, +12.2% on MATH-500), approaches/exceeds SimpleRL-Zero performance with 60x less compute, maintains baseline performance on IFEval, MUSR, TruthfulQA, and BBH.

Conclusion: CGD establishes as a robust and generalizable alternative to conventional SFT and RL-based methods, offering efficient advancement of reasoning and safety in large language models.

Abstract: Supervised fine-tuning (SFT) with expert demonstrations often suffers from the imitation problem, where models reproduce correct responses without internalizing the underlying reasoning. We propose Critique-Guided Distillation (CGD), a multi-stage training framework that augments SFT with teacher-generated explanatory critiques and refined responses. Instead of directly imitating teacher outputs, a student learns to map the triplet of prompt, its own initial response, and teacher critique into the refined teacher response, thereby capturing both what to output and why. Our analyses show that CGD consistently reduces refinement uncertainty, improves alignment between critiques and responses, and enhances sample efficiency. On reasoning benchmarks, CGD achieves substantial gains across LLaMA and Qwen families, including +15.0% on AMC23 and +12.2% on MATH-500, while avoiding the format drift issues observed in prior critique-based fine-tuning. Importantly, on LLaMA-3.1-8B CGD approaches or exceeds the performance of SimpleRL-Zero, which is a DeepSeek-R1 replication, while requiring 60x less compute. Beyond reasoning, CGD maintains or improves general instruction-following and factual accuracy, matching baseline performance on IFEval, MUSR, TruthfulQA, and BBH. In contrast, prior critique-based methods degrade these capabilities (e.g., -21% on IFEval). Taken together, these results establish CGD} as a robust and generalizable alternative to both conventional SFT and RL-based methods, offering a more efficient path toward advancing the reasoning and safety of large language models.

Method: Two-stage algorithm: exploratory phase estimates reward distribution for each prompt, then adaptive allocation of remaining budget using these estimates.

Result: Outperforms uniform allocation with same inference budget, remains competitive against uniform with 20% larger budgets, improves performance with larger batch sizes.

Conclusion: Simple, practical method for efficient test-time alignment that works with any LM-RM combination and provides better performance with same compute budget.

Abstract: Recent advances in test-time alignment methods, such as Best-of-N sampling, offer a simple and effective way to steer language models (LMs) toward preferred behaviors using reward models (RM). However, these approaches can be computationally expensive, especially when applied uniformly across prompts without accounting for differences in alignment difficulty. In this work, we propose a prompt-adaptive strategy for Best-of-N alignment that allocates inference-time compute more efficiently. Motivated by latency concerns, we develop a two-stage algorithm: an initial exploratory phase estimates the reward distribution for each prompt using a small exploration budget, and a second stage adaptively allocates the remaining budget using these estimates. Our method is simple, practical, and compatible with any LM-RM combination. Empirical results on prompts from the AlpacaEval, HH-RLHF, and PKU-SafeRLHF datasets for 12 LM/RM pairs and 50 different batches of prompts show that our adaptive strategy outperforms the uniform allocation with the same inference budget. Moreover, we show that our adaptive strategy remains competitive against uniform allocations with 20 percent larger inference budgets and improves in performance as the batch size grows.

Method: IPL constructs CoaT-tree through iterative sampling, scores leaf nodes with rule-based reward, backpropagates feedback for T-DPO pairs, and uses three-stage instruction evolution with GPT-4o for diverse Q&A pairs from mobile UI screenshots.

Result: MobileIPL outperforms strong baselines including OS-ATLAS and UI-TARS, achieving state-of-the-art performance across three standard Mobile GUI-Agents benchmarks with strong generalization to out-of-domain scenarios.

Conclusion: The proposed IPL framework effectively addresses CoaT trajectory scarcity through iterative preference learning and instruction evolution, demonstrating superior performance and generalization in mobile GUI agent tasks.

Abstract: The Chain of Action-Planning Thoughts (CoaT) paradigm has been shown to improve the reasoning performance of VLM-based mobile agents in GUI tasks. However, the scarcity of diverse CoaT trajectories limits the expressiveness and generalization ability of such agents. While self-training is commonly employed to address data scarcity, existing approaches either overlook the correctness of intermediate reasoning steps or depend on expensive process-level annotations to construct process reward models (PRM). To address the above problems, we propose an Iterative Preference Learning (IPL) that constructs a CoaT-tree through interative sampling, scores leaf nodes using rule-based reward, and backpropagates feedback to derive Thinking-level Direct Preference Optimization (T-DPO) pairs. To prevent overfitting during warm-up supervised fine-tuning, we further introduce a three-stage instruction evolution, which leverages GPT-4o to generate diverse Q&A pairs based on real mobile UI screenshots, enhancing both generality and layout understanding. Experiments on three standard Mobile GUI-agent benchmarks demonstrate that our agent MobileIPL outperforms strong baselines, including continual pretraining models such as OS-ATLAS and UI-TARS. It achieves state-of-the-art performance across three standard Mobile GUI-Agents benchmarks and shows strong generalization to out-of-domain scenarios.

Method: Introduced a benchmark of 1.3M computer science patent pairs and a framework decomposing errors into missing vs unused knowledge using clarifying questions in three settings: raw performance, self-answered, and externally supplied answers.

Result: LLMs often possess relevant knowledge but fail to deploy it; smaller models generate simpler transferable questions while larger models create complex but less generalizable ones.

Conclusion: LLM evaluation should shift from static fact recall to dynamic knowledge application, as models’ key limitation is unused knowledge rather than knowledge gaps.

Abstract: Large language models (LLMs) are rapidly becoming core tools for science, engineering, and innovation. Their promise lies not just in remembering facts, but in putting knowledge to work. Despite their impressive ability to answer increasingly difficult questions, it remains unclear whether LLMs truly use their knowledge when confronted with new and challenging tasks. We address this question with a patent classification task that requires deep conceptual understanding: distinguishing objectively different but semantically similar patents. To evaluate this approach, we introduce a challenging new benchmark of 1.3 million post-2015 computer science patent pairs, characterized by dense technical jargon and strategically complex writing. We find that LLMs often fail our benchmark and struggle to distinguish among semantically similar patents. To probe this failure, we introduce a novel framework that decomposes model errors into two sources: missing and unused knowledge. Our approach asks models to generate clarifying questions to improve their understanding, and then compares three settings: raw performance, self-answered questions, and externally supplied answers. This decomposition reveals that LLMs often possess the relevant knowledge internally but fail to deploy it, while a smaller share of errors arises from genuine knowledge gaps. We then ask whether the ability of models to construct a task-specific database of questions and answers differs across models. We find that smaller models generate simpler, broadly transferable questions, while larger models propose more complex but less generalizable ones. This suggests new strategies for combining strengths across models. Our findings highlight a critical limitation of current LLMs and their evaluation: models often know more than they can use. LLM evaluation should shift from recall of static facts to application of dynamic knowledge.

Method: Created synthetic personalized alignment dataset based on famous people with known preferences, then finetuned 1-8B models to test active preference inference versus passive approaches.

Result: Higher-quality active prefixes lead to better generalization, more contextually faithful models, and less systematic biases across protected attributes.

Conclusion: Active alignment provides a more controllable and efficient path for personalized alignment compared to passive preference modeling.

Abstract: A prominent issue in aligning language models (LMs) to personalized preferences is underspecification – the lack of information from users about their preferences. A popular trend of injecting such specification is adding a prefix (e.g. prior relevant conversations) to the current user’s conversation to steer preference distribution. Most methods passively model personal preferences with prior example preferences pairs. We ask whether models benefit from actively inferring preference descriptions, and address this question by creating a synthetic personalized alignment dataset based on famous people with known public preferences. We then test how effective finetuned 1-8B size models are at inferring and aligning to personal preferences. Results show that higher-quality active prefixes lead to better generalization, more contextually faithful models, and less systematic biases across different protected attributes. All our results suggest active alignment can lead to a more controllable and efficient path for personalized alignment.

Method: Evaluated multiple LLMs (OpenRoLLM, Llama 3.1 8B, Llama 3.2 3B, GPT-4o) using various translation methods, assessed accuracy and stability through repeated runs, performed syntactic/semantic analyses, and compared against human translators with expert evaluation.

Result: LLMs with appropriate supervision can maintain or enhance translation performance for less common languages, and augmented OJI Romanian dataset with accurate English translations for future LLM training/evaluation.

Conclusion: With human oversight, LLMs can serve as a viable solution for multilingual problem-solving in real-world scenarios, showing comparable quality to human translators as evaluated by certified experts.

Abstract: Recent studies have suggested that large language models (LLMs) underperform on mathematical and computer science tasks when these problems are translated from Romanian into English, compared to their original Romanian format. Accurate translation is critical for applications ranging from automatic translations in programming competitions to the creation of high-quality educational materials, as well as minimizing errors or fraud in human translations. This study shows that robust large language models (LLMs) can maintain or even enhance their performance in translating less common languages when given well-structured prompts. Our findings suggest that LLMs, with appropriate supervision, can be reliably used for the automatic translation of IOI (International Olympiad in Informatics)-style tasks. We evaluate several translation methods across multiple LLMs, including OpenRoLLM, Llama 3.1 8B, Llama 3.2 3B and GPT-4o, assessing their translation accuracy and performance stability through repeated runs. Additionally, we augment the OJI (Romanian County-Level Informatics Olympiad) Romanian dataset with accurate English translations, enhancing its utility for future LLM training and evaluation. Through detailed syntactic and semantic analyses, we confirm that with human oversight, LLMs can serve as a viable solution for multilingual problem-solving. We also compare the translation quality of LLMs against human translators, as evaluated by a certified expert, underscoring the potential of LLMs in realworld scenarios.

Method: Proposes Game-RL with Code2Logic approach to synthesize game reasoning tasks from game code, creating GameQA dataset with 30 games and 158 tasks of varying difficulty.

Result: RL training on GameQA enables multiple VLMs to achieve performance improvements across 7 diverse vision-language benchmarks.

Conclusion: Video games serve as valuable resources to boost general reasoning abilities in vision-language models.

Abstract: Vision-language reinforcement learning (RL) has primarily focused on narrow domains (e.g. geometry or chart reasoning). This leaves broader training scenarios and resources underexplored, limiting the exploration and learning of Vision Language Models (VLMs) through RL. We find video games inherently provide rich visual elements and mechanics that are easy to verify. To fully use the multimodal and verifiable reward in video games, we propose Game-RL, constructing diverse game tasks for RL training to boost VLMs general reasoning ability. To obtain training data, we propose Code2Logic, a novel approach that adapts game code to synthesize game reasoning task data, thus obtaining the GameQA dataset of 30 games and 158 tasks with controllable difficulty gradation. Unexpectedly, RL training solely on GameQA enables multiple VLMs to achieve performance improvements across 7 diverse vision-language benchmarks, demonstrating the value of Game-RL for enhancing VLMs’ general reasoning. Furthermore, this suggests that video games may serve as valuable scenarios and resources to boost general reasoning abilities. Our code, dataset and models are available at the GitHub repository.

Method: Attention Behavior Fine-Tuning (ABFT) builds training objectives on attention scores to force them to focus on correct label tokens and mitigate attention from wrong label tokens, leveraging findings on ICL’s inner mechanisms.

Result: ABFT outperforms previous methods in performance, robustness, unbiasedness, and efficiency across 9 LMs and 8 datasets, using only ~0.01% data cost compared to prior approaches.

Conclusion: The work demonstrates controlling specific LM modules to improve behavior, reveals implicit bias of ICL-style data for induction heads, and opens future applications of mechanistic interpretability.

Abstract: In-context Learning (ICL) utilizes structured demonstration-query inputs to induce few-shot learning on Language Models (LMs), which are not originally pre-trained on ICL-style data. To bridge the gap between ICL and pre-training, some approaches fine-tune LMs on large ICL-style datasets by an end-to-end paradigm with massive computational costs. To reduce such costs, in this paper, we propose Attention Behavior Fine-Tuning (ABFT), utilizing the previous findings on the inner mechanism of ICL, building training objectives on the attention scores instead of the final outputs, to force the attention scores to focus on the correct label tokens presented in the context and mitigate attention scores from the wrong label tokens. Our experiments on 9 modern LMs and 8 datasets empirically find that ABFT outperforms in performance, robustness, unbiasedness, and efficiency, with only around 0.01% data cost compared to the previous methods. Moreover, our subsequent analysis finds that the end-to-end training objective contains the ABFT objective, suggesting the implicit bias of ICL-style data to the emergence of induction heads. Our work demonstrates the possibility of controlling specific module sequences within LMs to improve their behavior, opening up the future application of mechanistic interpretability.

Method: Used layer-from context-masking and novel cross-task patching on 15 two-step composite tasks to confirm subtask learning at different depths, and applied LogitLens to decode hidden states to examine sequential execution patterns. Also replicated analysis on real-world TRACE benchmark.

Result: Confirmed that distinct subtasks are learned at different network depths and executed sequentially across layers, revealing consistent layerwise execution patterns in both synthetic and real-world benchmarks.

Conclusion: LLMs have capacity to internally plan and execute subtasks sequentially, enhancing model transparency and opening avenues for fine-grained, instruction-level activation steering.

Abstract: We show that large language models (LLMs) exhibit an $\textit{internal chain-of-thought}$: they sequentially decompose and execute composite tasks layer-by-layer. Two claims ground our study: (i) distinct subtasks are learned at different network depths, and (ii) these subtasks are executed sequentially across layers. On a benchmark of 15 two-step composite tasks, we employ layer-from context-masking and propose a novel cross-task patching method, confirming (i). To examine claim (ii), we apply LogitLens to decode hidden states, revealing a consistent layerwise execution pattern. We further replicate our analysis on the real-world $\text{TRACE}$ benchmark, observing the same stepwise dynamics. Together, our results enhance LLMs transparency by showing their capacity to internally plan and execute subtasks (or instructions), opening avenues for fine-grained, instruction-level activation steering.

Method: Created a multilingual dataset from Romanian game show using OCR, text extraction, and manual verification; benchmarked LLMs including Romanian-adapted models; conducted translation and cross-lingual experiments.

Result: Significant performance gap: 80-95% accuracy on international questions vs 50-75% on Romanian-specific cultural questions; cultural context strongly impacts LLM performance.

Conclusion: Cultural context and data source significantly affect LLM performance; insights provided for building robust, culturally-aware multilingual NLP systems, especially in education.

Abstract: Large Language Models (LLMs) demonstrate varying performance across languages and cultural contexts. This study introduces a novel, culturally-rich, multilingual dataset derived from video recordings of the Romanian game show “Who Wants to Be a Millionaire?” (Vrei s\u{a} fii Milionar?). We employed an innovative process combining optical character recognition (OCR), automated text extraction, and manual verification to collect question-answer pairs, enriching them with metadata including question domain (e.g., biology, history), cultural relevance (Romanian-specific vs. international), and difficulty. Benchmarking state-of-the-art LLMs, including Romanian-adapted models, on this dataset revealed significant performance disparities: models consistently achieve higher accuracy (80-95%) on international questions compared to Romanian-specific cultural questions (50-75%). We further investigate these differences through experiments involving machine translation of Romanian questions into English and cross-lingual tests using a comparable dataset in French. Our findings underscore the impact of cultural context and data source on LLM performance and offer practical insights for building robust, culturally-aware multilingual NLP systems, especially in educational domains. The dataset is publicly available at Hugging Face.

Method: Identified a robust stylistic feature space for detection, tested against language models optimized to evade detection, and developed a new paraphrasing approach that simultaneously closes the stylistic gap between human and machine writing while avoiding traditional detection features.

Result: Stylistic detectors remain surprisingly robust even when explicitly optimized against. When only single samples are available, the attack is universally effective across all detectors, but as sample size increases, human and machine distributions become distinguishable.

Conclusion: The findings support previous recommendations to avoid reliance on machine-text detection, as detection reliability depends heavily on the number of samples available and can be circumvented by sophisticated attacks.

Abstract: Despite considerable progress in the development of machine-text detectors, it has been suggested that the problem is inherently hard, and therefore, that stakeholders should proceed under the assumption that machine-generated text cannot be reliably detected as such. We examine a recent such claim by Nicks et al. (2024) regarding the ease with which language models can be optimized to degrade the performance of machine-text detectors, including detectors not specifically optimized against. We identify a feature space – the stylistic feature space – that is robust to such optimization, and show that it may be used to reliably detect samples from language models optimized to prevent detection. Furthermore, we show that even when models are explicitly optimized against stylistic detectors, detection performance remains surprisingly unaffected. We then seek to understand if stylistic detectors are inherently more robust. To study this question, we explore a new paraphrasing approach that simultaneously aims to close the gap between human writing and machine writing in stylistic feature space while avoiding detection using traditional features. We show that when only a single sample is available for detection, this attack is universally effective across all detectors considered, including those that use writing style. However, as the number of samples available for detection grows, the human and machine distributions become distinguishable. Overall, our findings underscore previous recommendations to avoid reliance on machine-text detection.

Method: VocabTrim reconstructs the drafter’s language modeling head to contain only a limited set of tokens selected from the most frequently sampled tokens in the target model’s vocabulary.

Result: The method boosts memory-bound speed-up by 16% for Llama-3.2-3B-Instruct on Spec-Bench, significantly reducing drafting latency despite slightly lower acceptance rates.

Conclusion: VocabTrim effectively improves speculative decoding performance in memory-bound scenarios by optimizing vocabulary size, making it suitable for edge device deployment.

Abstract: In this paper, we introduce a simple training-free technique to improve the performance of drafter-based speculative decoding (SpD) methods that incorporates language modeling head (LM head) during drafting process. A drafter-based speculative decoding leverages one or more smaller language models, a.k.a. drafters or draft models, to sample a draft sequence or tree consisting of multiple tokens, followed by verification by a base LLM, a target model, accepting a subset as its valid generation. As it is usually considered that the speculative decoding requires one-to-one mapping between vocabularies of the target model and the draft model, it has been natural to share the vocabulary between them, or even share the LM head as in EAGLE or Medusa. We first identify that this draft token sampling scheme inherently contains an unnecessary inference overhead in drafting, especially for some target LLMs with very large vocabularies. Then, we propose a simple technique, VocabTrim, to mitigate the drafting overhead to improve the generation speed in memory-bound environment. VocabTrim reconstructs the drafter LM head to contain only a limited set of tokens, selected by the most frequently sampled from the vocabulary of the target model. While limiting the vocabulary in drafting slightly degrades the acceptance rate, it significantly reduces the drafting latency in memory-bound process which is often the case on edge devices, resulting in higher memory-bound speed up (MBSU). We show that our method can boost the memory-bound speed-up for Llama-3 models on Spec-Bench, specifically by 16% for Llama-3.2-3B-Instruct.

Method: ReflAct shifts reasoning from planning actions to continuously reflecting on agent’s state relative to its goal, explicitly grounding decisions in states and enforcing ongoing goal alignment.

Result: ReflAct surpasses ReAct by 27.7% on average, achieving 93.3% success rate in ALFWorld, and outperforms ReAct even with enhancement modules like Reflexion and WKM.

Conclusion: Strengthening the core reasoning backbone through continuous reflection and goal alignment is key to reliable agent performance.

Abstract: Recent advances in LLM agents have largely built on reasoning backbones like ReAct, which interleave thought and action in complex environments. However, ReAct often produces ungrounded or incoherent reasoning steps, leading to misalignment between the agent’s actual state and goal. Our analysis finds that this stems from ReAct’s inability to maintain consistent internal beliefs and goal alignment, causing compounding errors and hallucinations. To address this, we introduce ReflAct, a novel backbone that shifts reasoning from merely planning next actions to continuously reflecting on the agent’s state relative to its goal. By explicitly grounding decisions in states and enforcing ongoing goal alignment, ReflAct dramatically improves strategic reliability. This design delivers substantial empirical gains: ReflAct surpasses ReAct by 27.7% on average, achieving a 93.3% success rate in ALFWorld. Notably, ReflAct even outperforms ReAct with added enhancement modules (e.g., Reflexion, WKM), showing that strengthening the core reasoning backbone is key to reliable agent performance.

Method: Propose multilingual prompting: create variations of base prompts with cultural and linguistic cues from multiple cultures, generate responses, then combine results to activate broader cultural knowledge.

Result: Multilingual prompting consistently outperforms high-temperature sampling, step-by-step recall, and persona prompting across GPT-4o, GPT-4o-mini, LLaMA 70B, and LLaMA 8B models. Benefits vary by language resource level and model size, with aligned language-culture cues reducing hallucinations.

Conclusion: Multilingual prompting effectively increases cultural diversity in LLM responses by leveraging language-specific knowledge, with performance advantages over existing diversity enhancement methods.

Abstract: Large Language Models (LLMs) are known to lack cultural representation and overall diversity in their generations, from expressing opinions to answering factual questions. To mitigate this problem, we propose multilingual prompting: a prompting method which generates several variations of a base prompt with added cultural and linguistic cues from several cultures, generates responses, and then combines the results. Building on evidence that LLMs have language-specific knowledge, multilingual prompting seeks to increase diversity by activating a broader range of cultural knowledge embedded in model training data. Through experiments across multiple models (GPT-4o, GPT-4o-mini, LLaMA 70B, and LLaMA 8B), we show that multilingual prompting consistently outperforms existing diversity-enhancing techniques such as high-temperature sampling, step-by-step recall, and persona prompting. Further analyses show that the benefits of multilingual prompting vary between high and low resource languages and across model sizes, and that aligning the prompting language with cultural cues reduces hallucination about culturally-specific information.

Method: Created GRILE benchmark with 1,151 Romanian exam questions, tested 7 multilingual and Romanian-specific LLMs on answer selection and explanation generation, with expert review of explanation quality.

Result: Gemini 2.5 Pro achieved 83% accuracy, but most open-weight models stayed below 65%. 48% of explanations contained factual or pedagogical flaws. Error analysis revealed weaknesses in morphology and DOOM3 orthographic norms.

Conclusion: The study exposes challenges for educational NLP in low-resource settings and establishes GRILE as a test-bed for controllable explanation generation and evaluation.

Abstract: LLMs (Large language models) have revolutionized NLP (Natural Language Processing), yet their pedagogical value for low-resource languages remains unclear. We present GRILE (Grammar Romanian Inference and Language Explanations) , the first open benchmark of 1,151 multiple-choice questions harvested from Romanian high-stakes exams (National Evaluation, Baccalaureate, university admissions). GRILE enables us to probe two complementary abilities of seven state-of-the-art multilingual and Romanian-specific LLMs: (i) selecting the correct answer, and (ii) producing linguistically accurate explanations. While Gemini 2.5 Pro reaches 83% accuracy, most open-weight models stay below 65%, and 48% of their explanations contain factual or pedagogical flaws according to expert review. A detailed error analysis pinpoints systematic weaknesses in morphology and in applying the latest DOOM3 orthographic norms. All data, code and a public web demo are released to catalyze future research. Our findings expose open challenges for trustworthy educational NLP in low-resource settings and establish GRILE as a new test-bed for controllable explanation generation and evaluation.

Method: Uses formal verification tools (Z3, Isabelle) to automatically annotate step-level error labels on LLM responses for formal logic and theorem proving tasks, creating PRM training data without human involvement.

Result: PRMs trained with FoVer significantly outperform original LLM-based PRMs and achieve competitive/superior results compared to state-of-the-art PRMs on ProcessBench and 12 reasoning benchmarks including MATH, AIME, ANLI, MMLU, and BBH.

Conclusion: FoVer enables automatic synthesis of accurate PRM training data and demonstrates effective cross-task generalization, making PRMs applicable to diverse reasoning tasks beyond math domains.

Abstract: Process Reward Models (PRMs), which provide step-level feedback on reasoning traces generated by Large Language Models (LLMs), are receiving increasing attention. However, two key research gaps remain: creating PRM training data requires costly human annotation to label accurate step-level errors, and existing PRMs are limited to math reasoning domains. In response to these gaps, this paper aims to enable automatic synthesis of accurate PRM training data and the generalization of PRMs to diverse reasoning tasks beyond math reasoning. We propose FoVer, an approach to synthesize PRM training data with accurate step-level error labels automatically annotated by formal verification tools, such as Z3 and Isabelle. To show the practical effectiveness of FoVer, we synthesize a training dataset by annotating step-level error labels on LLM responses to formal logic and theorem proving tasks, without relying on human annotation. While FoVer creates training data with symbolic tasks compatible with formal verification, our experiments show that PRMs trained on our dataset exhibit cross-task generalization, enabling a single PRM to effectively perform verification across diverse reasoning tasks. Specifically, LLM-based PRMs trained with FoVer significantly outperform PRMs based on the original LLMs and achieve competitive or superior results compared to state-of-the-art PRMs, as measured by step-level verification on ProcessBench and Best-of-K performance across 12 reasoning benchmarks, including MATH, AIME, ANLI, MMLU, and BBH. The dataset and code are in the supplementary material and will be made public. The datasets, models, and code are provided at https://github.com/psunlpgroup/FoVer.

Method: Created the first curated Persian medical dataset by crawling medical magazines and collecting real doctor-patient Q&A pairs, then fine-tuned a baseline language model using this data.

Result: The fine-tuned model achieved improved accuracy in medical question answering, successfully passed the Iranian Basic Medical Science Entrance Exam (September 2023), and improved Persian-translated MMLU accuracy by an average of 2.67%.

Conclusion: This work demonstrates the potential of leveraging open-access online data to enrich small language models in medical fields, providing a novel solution for Persian medical AI applications in resource-constrained environments.

Abstract: The rapid advancement of language models has demonstrated the potential of artificial intelligence in the healthcare industry. However, small language models struggle with specialized domains in low-resource languages like Persian. While numerous medical-domain websites exist in Persian, no curated dataset or corpus has been available making ours the first of its kind. This study explores the enhancement of medical knowledge in a small language model by leveraging accessible online data, including a crawled corpus from medical magazines and a dataset of real doctor-patient Q&A pairs. We fine-tuned a baseline model using our curated data to improve its medical knowledge. Benchmark evaluations demonstrate that the fine-tuned model achieves improved accuracy in medical question answering and provides better responses compared to its baseline. Notably, the trained model successfully passed the Iranian Basic Medical Science Entrance Exam, taken in September 2023, and improved Persian-translated MMLU accuracy by an average of 2.67%. This work highlights the potential of leveraging open-access online data to enrich small language models in medical fields, providing a novel solution for Persian medical AI applications suitable for resource-constrained environments.

Method: Proposes Align-GRAG with dual alignment: formulates subgraphs by retrieving nodes/edges, uses an Aligner to optimize graph encoder with LLM-summarized reasoning chain via KL divergence and contrastive loss for node pruning and semantic space unification, then integrates aligned graph data with LLM for generation.

Result: Experiments on GraphQA benchmark across three tasks (common sense reasoning, scene graph understanding, knowledge graph reasoning) validate the method’s effectiveness.

Conclusion: Align-GRAG successfully addresses graph RAG limitations by enabling efficient knowledge pruning and establishing unified semantic representations between graphs and language, improving answer coherence and accuracy.

Abstract: Large language models (LLMs) have demonstrated remarkable capabilities, but still struggle with issues like hallucinations and outdated information. Retrieval-augmented generation (RAG) addresses these issues by grounding LLM outputs in external knowledge with an Information Retrieval (IR) system. Building on this foundation, graph-based RAG systems go a step further by retrieving subgraphs, which preserve the relationships between knowledge entities and provide more comprehensive context. However, graph RAG faces two challenges: (1) Retrieving relevant information introduces irrelevant nodes (especially in dense graph databases, where retrieval usually extends to adjacent nodes), and leads to overly lengthy inputs that hinder efficiency; (2) The representation gap between graph and language during generation with LLMs limits the ability to fully leverage graph structures for enhanced understanding. To address these limitations, we propose Align-GRAG, a novel reasoning-guided dual alignment framework in post-retrieval phrase. It first formulates a subgraph by retrieving nodes and edges. Then an Aligner is proposed to jointly optimize a graph encoder with an LLM-summarized reasoning chain. It achieves dual alignment of graph node and representation by leveraging KL divergence loss and contrastive loss, facilitating efficient pruning of irrelevant knowledge and establishing a unified semantic space. The Generator integrates the aligned graph data with LLM to produce coherent and accurate answers. Experiments on the GraphQA benchmark across three tasks (including common sense reasoning, scene graph understanding, and knowledge graph reasoning) validate the effectiveness of our method. The codes are available in this repository\footnote{https://anonymous.4open.science/r/Align-GRAG-F3D8/}.

Method: Proposes Token-wise Distillation (ToDi) that uses gradient analysis to reveal complementary roles of FKL (boosts underestimated tokens) and RKL (suppresses overestimated tokens), then combines them adaptively per token using sigmoid-based weighting based on teacher-student probability log-ratio.

Result: ToDi consistently outperforms recent distillation baselines across instruction-following benchmarks, with extensive ablation studies and efficiency analysis validating its effectiveness and practicality.

Conclusion: Token-wise adaptive combination of FKL and RKL enables precise distribution alignment and superior knowledge distillation performance compared to uniform or less granular strategies.

Abstract: Large language models (LLMs) offer impressive performance but are impractical for resource-constrained deployment due to high latency and energy consumption. Knowledge distillation (KD) addresses this by transferring knowledge from a large teacher to a smaller student model. However, conventional KD, notably approaches like Forward KL (FKL) and Reverse KL (RKL), apply uniform divergence loss across the entire vocabulary, neglecting token-level prediction discrepancies. By investigating these representative divergences via gradient analysis, we reveal that FKL boosts underestimated tokens, while RKL suppresses overestimated ones, showing their complementary roles. Based on this observation, we propose Token-wise Distillation (ToDi), a novel method that adaptively combines FKL and RKL per token using a sigmoid-based weighting function derived from the teacher-student probability log-ratio. ToDi dynamically emphasizes the appropriate divergence for each token, enabling precise distribution alignment. We demonstrate that ToDi consistently outperforms recent distillation baselines using uniform or less granular strategies across instruction-following benchmarks. Extensive ablation studies and efficiency analysis further validate ToDi’s effectiveness and practicality.

Method: Combine constituency linearizations with pretrained encoders for sequence labeling.

Result: Achieves competitive performance compared to less efficient systems.

Conclusion: Method captures nested entities efficiently and can be trained using standard sequence labeling libraries.

Abstract: We cast nested named entity recognition (NNER) as a sequence labeling task by leveraging prior work that linearizes constituency structures, effectively reducing the complexity of this structured prediction problem to straightforward token classification. By combining these constituency linearizations with pretrained encoders, our method captures nested entities while performing exactly n tagging actions. Our approach achieves competitive performance compared to less efficient systems, and it can be trained using any off-the-shelf sequence labeling library.

Method: Conducted a semi-automatic literature review using Semantic Scholar, retrieving and filtering over 6,000 papers from 2000-2025, and analyzed definitions from psychology, sociology, and philosophy.

Result: Identified key trends, methodologies, challenges and future directions in stereotype detection research, emphasizing its potential as an early-monitoring tool to prevent bias escalation and hate speech.

Conclusion: Highlights the need for a broader, multilingual, and intersectional approach in NLP studies on stereotype detection.

Abstract: Stereotypes influence social perceptions and can escalate into discrimination and violence. While NLP research has extensively addressed gender bias and hate speech, stereotype detection remains an emerging field with significant societal implications. In this work is presented a survey of existing research, analyzing definitions from psychology, sociology, and philosophy. A semi-automatic literature review was performed by using Semantic Scholar. We retrieved and filtered over 6,000 papers (in the year range 2000-2025), identifying key trends, methodologies, challenges and future directions. The findings emphasize stereotype detection as a potential early-monitoring tool to prevent bias escalation and the rise of hate speech. Conclusions highlight the need for a broader, multilingual, and intersectional approach in NLP studies.

Method: BRIT creates a multi-modal graph that unifies various text-image connections and retrieves query-specific sub-graphs by traversing both image-to-text and text-to-image paths to find relevant content for complex cross-modal questions.

Result: Comprehensive experiments demonstrate BRIT’s superiority in handling cross-modal questions on multi-modal documents, using the introduced MM-RAG test set for evaluation.

Conclusion: BRIT effectively addresses the gap in multi-modal RAG by leveraging graph-based retrieval of text-image connections, enabling better handling of complex cross-modal multi-hop questions.

Abstract: Retrieval-Augmented Generation (RAG) has emerged as a promising technique to enhance the quality and relevance of responses generated by large language models. While recent advancements have mainly focused on improving RAG for text-based queries, RAG on multi-modal documents containing both texts and images has not been fully explored. Especially when fine-tuning does not work. This paper proposes BRIT, a novel multi-modal RAG framework that effectively unifies various text-image connections in the document into a multi-modal graph and retrieves the texts and images as a query-specific sub-graph. By traversing both image-to-text and text-to-image paths in the graph, BRIT retrieve not only directly query-relevant images and texts but also further relevant contents to answering complex cross-modal multi-hop questions. To evaluate the effectiveness of BRIT, we introduce MM-RAG test set specifically designed for multi-modal question answering tasks that require to understand the text-image relations. Our comprehensive experiments demonstrate the superiority of BRIT, highlighting its ability to handle cross-modal questions on the multi-modal documents.

Method: A multi-agent framework with three collaborative stages: Theory-of-Mind Agent generates mental state hypotheses, Moral Agent refines them using cultural norms and ethics, and Response Agent generates appropriate responses while validating intent alignment.

Result: Achieves state-of-the-art performance with 35.7% improvement in real-world social scenarios and 6.2% gain in ToM reasoning. Enables LLMs to match human-level performance on key ToM tasks for the first time.

Conclusion: MetaMind advances AI systems toward human-like social intelligence, demonstrating ability to balance contextual plausibility, social appropriateness, and user adaptation, with applications in empathetic dialogue and culturally sensitive interactions.

Abstract: Human social interactions depend on the ability to infer others’ unspoken intentions, emotions, and beliefs-a cognitive skill grounded in the psychological concept of Theory of Mind (ToM). While large language models (LLMs) excel in semantic understanding tasks, they struggle with the ambiguity and contextual nuance inherent in human communication. To bridge this gap, we introduce MetaMind, a multi-agent framework inspired by psychological theories of metacognition, designed to emulate human-like social reasoning. MetaMind decomposes social understanding into three collaborative stages: (1) a Theory-of-Mind Agent generates hypotheses about user mental states (e.g., intent, emotion), (2) a Moral Agent refines these hypotheses using cultural norms and ethical constraints, and (3) a Response Agent generates contextually appropriate responses while validating alignment with inferred intent. Our framework achieves state-of-the-art performance across three challenging benchmarks, with 35.7% improvement in real-world social scenarios and 6.2% gain in ToM reasoning. Notably, it enables LLMs to match human-level performance on key ToM tasks for the first time. Ablation studies confirm the necessity of all components, which showcase the framework’s ability to balance contextual plausibility, social appropriateness, and user adaptation. This work advances AI systems toward human-like social intelligence, with applications in empathetic dialogue and culturally sensitive interactions. Code is available at https://github.com/XMZhangAI/MetaMind.

Method: Extends weight of evidence concept to create PEX consistency measure, applies direct preference optimization to improve explanation consistency across three model families.

Result: More than 62% of LLM-generated explanations lack PEX consistency. Direct preference optimization improves consistency by 43.1% to 292.3% and explanation faithfulness by up to 9.7%.

Conclusion: PEX consistency is an important aspect of explanation faithfulness that can be effectively improved through direct preference optimization, enhancing the reliability of AI explanations in critical applications.

Abstract: Faithful free-text explanations are important to ensure transparency in high-stakes AI decision-making contexts, but they are challenging to generate by language models and assess by humans. In this paper, we present a measure for Prediction-EXplanation (PEX) consistency, by extending the concept of weight of evidence. This measure quantifies how much a free-text explanation supports or opposes a prediction, serving as an important aspect of explanation faithfulness. Our analysis reveals that more than 62% explanations generated by large language models lack this consistency. We show that applying direct preference optimization improves the consistency of generated explanations across three model families, with improvement ranging from 43.1% to 292.3%. Furthermore, we demonstrate that optimizing this consistency measure can improve explanation faithfulness by up to 9.7%.

Method: Uses multi-granularity memory units with Gaussian Mixture Models for clustering and association, entropy-based router for adaptive granularity selection, and LLM-based filtering for memory refinement.

Result: Outperforms state-of-the-art methods on four long-term memory benchmarks for both question answering and retrieval tasks across different query types and top-K settings.

Conclusion: MemGAS effectively addresses long-term dialogue memory challenges through multi-granularity association and adaptive selection, demonstrating superior performance over existing approaches.

Abstract: Large Language Models (LLMs) have recently been widely adopted in conversational agents. However, the increasingly long interactions between users and agents accumulate extensive dialogue records, making it difficult for LLMs with limited context windows to maintain a coherent long-term dialogue memory and deliver personalized responses. While retrieval-augmented memory systems have emerged to address this issue, existing methods often depend on single-granularity memory segmentation and retrieval. This approach falls short in capturing deep memory connections, leading to partial retrieval of useful information or substantial noise, resulting in suboptimal performance. To tackle these limits, we propose MemGAS, a framework that enhances memory consolidation by constructing multi-granularity association, adaptive selection, and retrieval. MemGAS is based on multi-granularity memory units and employs Gaussian Mixture Models to cluster and associate new memories with historical ones. An entropy-based router adaptively selects optimal granularity by evaluating query relevance distributions and balancing information completeness and noise. Retrieved memories are further refined via LLM-based filtering. Experiments on four long-term memory benchmarks demonstrate that MemGAS outperforms state-of-the-art methods on both question answer and retrieval tasks, achieving superior performance across different query types and top-K settings. \footnote{https://github.com/quqxui/MemGAS}

Method: Proposed a practical encoding scheme based on vocabulary partitioning that LLMs can learn via fine-tuning, enabling linguistic steganography to embed secrets into natural-looking outputs.

Result: Compromised models reliably transmit 32-bit secrets with 87% accuracy on held-out prompts, reaching over 97% accuracy with majority voting across three generations, while maintaining high utility and evading human detection.

Conclusion: TrojanStego represents a new class of passive, covert, practical, and dangerous LLM data exfiltration attacks that highlight significant security risks in LLM deployments.

Abstract: As large language models (LLMs) become integrated into sensitive workflows, concerns grow over their potential to leak confidential information. We propose TrojanStego, a novel threat model in which an adversary fine-tunes an LLM to embed sensitive context information into natural-looking outputs via linguistic steganography, without requiring explicit control over inference inputs. We introduce a taxonomy outlining risk factors for compromised LLMs, and use it to evaluate the risk profile of the threat. To implement TrojanStego, we propose a practical encoding scheme based on vocabulary partitioning learnable by LLMs via fine-tuning. Experimental results show that compromised models reliably transmit 32-bit secrets with 87% accuracy on held-out prompts, reaching over 97% accuracy using majority voting across three generations. Further, they maintain high utility, can evade human detection, and preserve coherence. These results highlight a new class of LLM data exfiltration attacks that are passive, covert, practical, and dangerous.

Method: XpandA uses: 1) dynamic partitioning to adaptively modulate context window filling; 2) question-guided protocol to update information in shared memory; 3) selective partition replay based on state-tracking to handle inverted-order structures.

Result: Comprehensive evaluation on benchmarks from 1k to 1M tokens shows XpandA enables ultra-long sequence processing with 20% performance improvements and 1.5x inference speedup over full-context, RAG, and previous agent-based methods.

Conclusion: XpandA significantly enhances LLMs’ long-context capabilities by overcoming limitations of existing approaches through its dynamic partitioning and question-driven workflow design.

Abstract: Processing long contexts has become a critical capability for modern large language models (LLMs). Existing works leverage agent-based divide-and-conquer methods for processing long contexts. But these methods face crucial limitations, including prohibitive accumulated latency and amplified information loss from excessive agent invocations, and the disruption of inherent textual dependencies by immoderate partitioning. In this paper, we propose a novel multi-agent framework XpandA (Expand-Agent) coupled with question-driven workflow and dynamic partitioning for robust long-context processing. XpandA overcomes these limitations through: 1) dynamic partitioning of long texts, which adaptively modulates the filling rate of context windows for input sequences of vastly varying lengths; 2) question-guided protocol to update flat information ensembles within centralized shared memory, constructing consistent inter-agent knowledge across partitions; and 3) selectively replaying specific partitions based on the state-tracking of question-information couples to promote the resolution of inverted-order structures across partitions (e.g., flashbacks). We perform a comprehensive evaluation of XpandA on multiple long-context benchmarks with length varying from 1k to 1M, demonstrating XpandA’s feasibility for processing ultra-long sequences and its significant effectiveness in enhancing the long-context capabilities of various LLMs by achieving 20% improvements and 1.5x inference speedup over baselines of full-context, RAG and previous agent-based methods.

Method: Integrates FlashAttention and Hybrid Tree Attention to accelerate prefill and verification steps. Proposes Cross-model Retrieval, a novel KV cache eviction strategy that uses the target model’s attention scores to dynamically select relevant context for the draft model.

Result: Accelerates speculative decoding by up to 2.84x on 16K-token long summarization and up to 3.86x on long reasoning tasks, while preserving short-input performance of state-of-the-art frameworks.

Conclusion: SpecExtend is an effective drop-in enhancement that significantly improves speculative decoding performance on long sequences without requiring additional training.

Abstract: Speculative decoding is a widely used technique for accelerating inference in large language models (LLMs), but its performance degrades as input length grows, with significant drops even at moderate lengths. Yet, this early degradation has remained largely underexplored. We introduce SpecExtend, a drop-in enhancement that improves speculative decoding on long sequences without additional training. SpecExtend integrates efficient attention mechanisms such as FlashAttention and Hybrid Tree Attention to accelerate prefill and verification steps. To improve both draft accuracy and speed on long inputs without retraining, we propose Cross-model Retrieval, a novel KV cache eviction strategy that leverages the target model’s attention scores to dynamically select relevant context for the smaller draft model. Extensive evaluations show that SpecExtend accelerates speculative decoding by up to 2.84x on 16K-token long summarization and up to 3.86x on long reasoning, while preserving the short-input performance of state-of-the-art frameworks. Our code is available at https://github.com/jycha98/SpecExtend .

Method: Built MC² benchmark for controlled evidence conflict scenarios, evaluated 18 MLLMs, and proposed representation engineering method to probe and steer modality preference without additional training.

Result: All 18 tested MLLMs demonstrated clear modality bias, and preference direction can be captured in latent representations. The proposed method effectively amplifies preference toward desired directions and improves downstream tasks.

Conclusion: MLLMs exhibit systematic modality preference that can be systematically studied and controlled through representation engineering, enabling applications like hallucination mitigation and improved multimodal translation.

Abstract: Multimodal large language models (MLLMs) have achieved remarkable performance on complex tasks with multimodal context. However, it is still understudied whether they exhibit modality preference when processing multimodal contexts. To study this question, we first build a \textbf{MC\textsuperscript{2}} benchmark under controlled evidence conflict scenarios to systematically evaluate modality preference, which is the tendency to favor one modality over another when making decisions based on multimodal conflicting evidence. Our extensive evaluation reveals that all 18 tested MLLMs generally demonstrate clear modality bias, and modality preference can be influenced by external interventions. An in-depth analysis reveals that the preference direction can be captured within the latent representations of MLLMs. Built on this, we propose a probing and steering method based on representation engineering to explicitly control modality preference without additional fine-tuning or carefully crafted prompts. Our method effectively amplifies modality preference toward a desired direction and applies to downstream tasks such as hallucination mitigation and multimodal machine translation, yielding promising improvements.

Method: Constructed through advanced agentic workflows and extensive human-in-the-loop annotations by 37 annotators from across the Arab world, covering ten culturally significant domains and all Arab countries.

Result: Comprehensive evaluations show that reasoning-centric instruction alignment substantially improves models’ cultural grounding compared to conventional scaling methods.

Conclusion: PEARL establishes a foundational resource for advancing culturally-informed multimodal modeling research and all datasets/benchmarks are publicly available.

Abstract: Mainstream large vision-language models (LVLMs) inherently encode cultural biases, highlighting the need for diverse multimodal datasets. To address this gap, we introduce PEARL, a large-scale Arabic multimodal dataset and benchmark explicitly designed for cultural understanding. Constructed through advanced agentic workflows and extensive human-in-the-loop annotations by 37 annotators from across the Arab world, PEARL comprises over 309K multimodal examples spanning ten culturally significant domains covering all Arab countries. We further provide two robust evaluation benchmarks (PEARL and PEARL-LITE) along with a specialized subset (PEARL-X) explicitly developed to assess nuanced cultural variations. Comprehensive evaluations on state-of-the-art open and proprietary LVLMs demonstrate that reasoning-centric instruction alignment substantially improves models’ cultural grounding compared to conventional scaling methods. PEARL establishes a foundational resource for advancing culturally-informed multimodal modeling research. All datasets and benchmarks are publicly available.

Method: Developed LITEX taxonomy, annotated a subset of e-SNLI dataset, validated taxonomy reliability, and analyzed alignment with NLI labels, highlights, and explanations. Also assessed taxonomy’s usefulness in explanation generation.

Result: Conditioning explanation generation on LITEX yields explanations linguistically closer to human explanations than those generated using only labels or highlights. The taxonomy captures within-label variation effectively.

Conclusion: LITEX taxonomy not only captures within-label variation but also bridges the gap between human and model explanations more effectively than existing strategies through taxonomy-guided generation for reasoning.

Abstract: There is increasing evidence of Human Label Variation (HLV) in Natural Language Inference (NLI), where annotators assign different labels to the same premise-hypothesis pair. However, within-label variation–cases where annotators agree on the same label but provide divergent reasoning–poses an additional and mostly overlooked challenge. Several NLI datasets contain highlighted words in the NLI item as explanations, but the same spans on the NLI item can be highlighted for different reasons, as evidenced by free-text explanations, which offer a window into annotators’ reasoning. To systematically understand this problem and gain insight into the rationales behind NLI labels, we introduce LITEX, a linguistically-informed taxonomy for categorizing free-text explanations in English. Using this taxonomy, we annotate a subset of the e-SNLI dataset, validate the taxonomy’s reliability, and analyze how it aligns with NLI labels, highlights, and explanations. We further assess the taxonomy’s usefulness in explanation generation, demonstrating that conditioning generation on LITEX yields explanations that are linguistically closer to human explanations than those generated using only labels or highlights. Our approach thus not only captures within-label variation but also shows how taxonomy-guided generation for reasoning can bridge the gap between human and model explanations more effectively than existing strategies.

Method: Train Semi-Structured Reasoning Models to produce reasoning traces in non-executable Pythonic syntax that names steps and marks inputs/outputs, enabling automated audits using hand-crafted DSL, LLM-generated audits, and learned typicality audits.

Result: All three audit methods effectively flag probable reasoning errors, and SSRMs demonstrate strong performance and generalizability across twelve benchmarks and two model families without compromising accuracy.

Conclusion: SSRMs provide an effective approach to improve reasoning faithfulness through structured, auditable reasoning traces while maintaining competitive performance.

Abstract: Although Large Language Models (LLMs) have become capable reasoners, the problem of faithfulness persists: their reasoning can contain errors and omissions that are difficult to detect and that may obscure biases in model outputs. To address this issue, we introduce Semi-Structured Reasoning Models (SSRMs), which are trained to produce semi-structured representations of reasoning. SSRMs generate reasoning traces in a non-executable Pythonic syntax that names each reasoning step and marks its inputs and outputs. This structure allows SSRM traces to be automatically audited to identify reasoning flaws. We evaluate three types of audits: hand-crafted structured reasoning audits, written in a domain-specific language (DSL) implemented in Python; LLM-generated structured reasoning audits; and learned typicality audits, which apply probabilistic models over reasoning traces. We show that all of these methods can be used to effectively flag probable reasoning errors. Importantly, the auditability of SSRMs does not appear to compromise overall accuracy: in evaluation on twelve benchmarks and two model families, SSRMs demonstrate strong performance and generalizability relative to other models of comparable size.

Method: Decomposes LLM-generated medical responses into discrete verifiable units (atomic facts), each independently verified against authoritative medical guideline knowledge bases, enabling targeted error correction and source tracing.

Result: Achieved up to 40% overall answer improvement and 50% hallucination detection rate. Provides granular explanations by tracing each atomic fact to relevant source chunks, significantly improving factual accuracy and explainability.

Conclusion: This framework represents a crucial step towards trustworthy clinical LLM applications, addressing key prerequisites for clinical use and fostering confidence in AI-assisted healthcare through improved reliability and transparency.

Abstract: Large language models (LLMs) exhibit extensive medical knowledge but are prone to hallucinations and inaccurate citations, which pose a challenge to their clinical adoption and regulatory compliance. Current methods, such as Retrieval Augmented Generation, partially address these issues by grounding answers in source documents, but hallucinations and low fact-level explainability persist. In this work, we introduce a novel atomic fact-checking framework designed to enhance the reliability and explainability of LLMs used in medical long-form question answering. This method decomposes LLM-generated responses into discrete, verifiable units called atomic facts, each of which is independently verified against an authoritative knowledge base of medical guidelines. This approach enables targeted correction of errors and direct tracing to source literature, thereby improving the factual accuracy and explainability of medical Q&A. Extensive evaluation using multi-reader assessments by medical experts and an automated open Q&A benchmark demonstrated significant improvements in factual accuracy and explainability. Our framework achieved up to a 40% overall answer improvement and a 50% hallucination detection rate. The ability to trace each atomic fact back to the most relevant chunks from the database provides a granular, transparent explanation of the generated responses, addressing a major gap in current medical AI applications. This work represents a crucial step towards more trustworthy and reliable clinical applications of LLMs, addressing key prerequisites for clinical application and fostering greater confidence in AI-assisted healthcare.

Method: Created Hanfu-Bench dataset with expert-curated multimodal data. Includes two tasks: cultural visual understanding (multiple-choice VQA on temporal-cultural features) and cultural image transcreation (transforming traditional attire to modern designs).

Result: Closed VLMs perform comparably to non-experts on visual understanding but fall 10% short of human experts. Open VLMs lag further. For transcreation, best model achieves only 42% success rate.

Conclusion: The benchmark reveals significant challenges in temporal cultural understanding and creative adaptation, providing essential testbed for this new research direction.

Abstract: Culture is a rich and dynamic domain that evolves across both geography and time. However, existing studies on cultural understanding with vision-language models (VLMs) primarily emphasize geographic diversity, often overlooking the critical temporal dimensions. To bridge this gap, we introduce Hanfu-Bench, a novel, expert-curated multimodal dataset. Hanfu, a traditional garment spanning ancient Chinese dynasties, serves as a representative cultural heritage that reflects the profound temporal aspects of Chinese culture while remaining highly popular in Chinese contemporary society. Hanfu-Bench comprises two core tasks: cultural visual understanding and cultural image transcreation. The former task examines temporal-cultural feature recognition based on single- or multi-image inputs through multiple-choice visual question answering, while the latter focuses on transforming traditional attire into modern designs through cultural element inheritance and modern context adaptation. Our evaluation shows that closed VLMs perform comparably to non-experts on visual cutural understanding but fall short by 10% to human experts, while open VLMs lags further behind non-experts. For the transcreation task, multi-faceted human evaluation indicates that the best-performing model achieves a success rate of only 42%. Our benchmark provides an essential testbed, revealing significant challenges in this new direction of temporal cultural understanding and creative adaptation.

Method: Three inference-time strategies: (1) early stopping via answer consistency, (2) boosting probability of generating end-of-reasoning signals, and (3) supervised method learning when to stop based on internal activations.

Result: Experiments across five benchmarks and five LLMs show significant token reduction (over 40% on NaturalQuestions) with little or no accuracy drop. Answer Consistency even improves accuracy while reducing tokens.

Conclusion: The work demonstrates the importance of cost-effective reasoning methods at inference time, offering practical benefits for real-world applications by reducing computational costs while maintaining performance.

Abstract: Chain-of-thought (CoT) prompting enhances reasoning in large language models (LLMs) but often leads to verbose and redundant outputs, thus increasing inference cost. We hypothesize that many reasoning steps are unnecessary for producing correct answers. To investigate this, we start with a systematic study to examine what is the minimum reasoning required for a model to reach a stable decision. We find that on math reasoning tasks like math, models typically converge to their final answers after 60% of the reasoning steps, suggesting substantial redundancy in the remaining content. Based on these insights, we propose three inference-time strategies to improve efficiency: (1) early stopping via answer consistency, (2) boosting the probability of generating end-of-reasoning signals, and (3) a supervised method that learns when to stop based on internal activations. Experiments across five benchmarks and five open-weights LLMs show that our methods significantly reduce token usage with little or no accuracy drop. In particular, on NaturalQuestions, Answer Consistency reduces tokens by over 40% while further improving accuracy. Our work underscores the importance of cost-effective reasoning methods that operate at inference time, offering practical benefits for real-world applications.

Method: Extracts latent representations from role-play prompts, selects relevant features based on activation patterns, and constructs a steering vector that can be injected into the model’s residual stream with controllable intensity.

Result: Significant performance gains: Llama3.1-8B on CSQA improved from 31.86% to 39.80%, Gemma2-9B on SVAMP increased from 37.50% to 45.10% in zero-shot CoT setting.

Conclusion: SRPS enhances reasoning ability in LLMs with better interpretability and stability compared to traditional prompt-based role-playing, demonstrating the potential of feature manipulation for model control.

Abstract: Role-playing has emerged as an effective technique for enhancing the reasoning capabilities of large language models (LLMs). However, existing methods primarily rely on prompt engineering, which often lacks stability and interpretability. In this paper, we introduce Sparse Autoencoder Role-Playing Steering (SRPS), a novel framework that identifies and manipulates internal model features associated with role-playing behavior. Our approach extracts latent representations from role-play prompts, selects the most relevant features based on activation patterns, and constructs a steering vector that can be injected into the model’s residual stream with controllable intensity. Our method enables fine-grained control over role-specific behavior and offers insights into how role information influences internal model activations. Extensive experiments across various reasoning benchmarks and model sizes demonstrate consistent performance gains. Notably, in the zero-shot chain-of-thought (CoT) setting, the accuracy of Llama3.1-8B on CSQA improves from 31.86% to 39.80%, while Gemma2-9B on SVAMP increases from 37.50% to 45.10%. These results highlight the potential of SRPS to enhance reasoning ability in LLMs, providing better interpretability and stability compared to traditional prompt-based role-playing.

Method: Conducted a nationwide survey to assess the actual needs of native speakers in Indonesia.

Result: Findings show addressing language barriers through machine translation and information retrieval is the most critical priority. There is strong enthusiasm for language technology advancements but concerns about privacy, bias, and use of public data for AI training.

Conclusion: Greater transparency and clear communication are needed to support broader AI adoption in Indonesian local language communities.

Abstract: There is an emerging effort to develop NLP for Indonesias 700+ local languages, but progress remains costly due to the need for direct engagement with native speakers. However, it is unclear what these language communities truly need from language technology. To address this, we conduct a nationwide survey to assess the actual needs of native speakers in Indonesia. Our findings indicate that addressing language barriers, particularly through machine translation and information retrieval, is the most critical priority. Although there is strong enthusiasm for advancements in language technology, concerns around privacy, bias, and the use of public data for AI training highlight the need for greater transparency and clear communication to support broader AI adoption.

Method: Introduces ALKALI benchmark (9,000 prompts across 15 attack families), GRACE framework with geometric contrastive enhancement, and AVQI metric for quantifying latent alignment failures via cluster separation analysis.

Result: Evaluation of 21 LLMs shows alarmingly high attack success rates. GRACE achieves up to 39% ASR reduction by enforcing latent separation and adversarial cohesion constraints.

Conclusion: Latent camouflage is a critical structural blind spot in LLM alignment that requires geometric-aware defenses like GRACE, with AVQI providing principled measurement of internal safety encoding.

Abstract: Adversarial threats against LLMs are escalating faster than current defenses can adapt. We expose a critical geometric blind spot in alignment: adversarial prompts exploit latent camouflage, embedding perilously close to the safe representation manifold while encoding unsafe intent thereby evading surface level defenses like Direct Preference Optimization (DPO), which remain blind to the latent geometry. We introduce ALKALI, the first rigorously curated adversarial benchmark and the most comprehensive to date spanning 9,000 prompts across three macro categories, six subtypes, and fifteen attack families. Evaluation of 21 leading LLMs reveals alarmingly high Attack Success Rates (ASRs) across both open and closed source models, exposing an underlying vulnerability we term latent camouflage, a structural blind spot where adversarial completions mimic the latent geometry of safe ones. To mitigate this vulnerability, we introduce GRACE - Geometric Representation Aware Contrastive Enhancement, an alignment framework coupling preference learning with latent space regularization. GRACE enforces two constraints: latent separation between safe and adversarial completions, and adversarial cohesion among unsafe and jailbreak behaviors. These operate over layerwise pooled embeddings guided by a learned attention profile, reshaping internal geometry without modifying the base model, and achieve up to 39% ASR reduction. Moreover, we introduce AVQI, a geometry aware metric that quantifies latent alignment failure via cluster separation and compactness. AVQI reveals when unsafe completions mimic the geometry of safe ones, offering a principled lens into how models internally encode safety. We make the code publicly available at https://anonymous.4open.science/r/alkali-B416/README.md.

Method: Identify QRHead by aggregating attention scores with respect to input query using real-world task examples. Use QRRetriever with accumulated attention mass as retrieval scores for selecting relevant context parts.

Result: Achieves over 10% performance gains on LongMemEval and CLIPPER multi-hop reasoning tasks, outperforms full context and strong dense retrievers. Strong zero-shot performance on BEIR benchmark, beating other LLM-based re-rankers like RankGPT.

Conclusion: Query-context attention scoring and task selection are crucial for identifying effective QRHeads. The work provides a general-purpose retriever and interpretability insights into long-context LM capabilities.

Abstract: Recent work has identified retrieval heads, a subset of attention heads responsible for retrieving salient information in long-context language models (LMs), as measured by their copy-paste behavior in Needlein-a-Haystack tasks. In this paper, we introduce QRHead (Query-Focused Retrieval Head), an improved set of attention heads that enhance retrieval from long context. We identify QRHead by aggregating attention scores with respect to the input query, using a handful of examples from real-world tasks (e.g., long-context QA). We further introduce QRRetriever, an efficient and effective retriever that uses the accumulated attention mass of QRHead as retrieval scores. We use QRRetriever for long-context reasoning by selecting the most relevant parts with the highest retrieval scores. On multi-hop reasoning tasks LongMemEval and CLIPPER, this yields over 10% performance gains over full context and outperforms strong dense retrievers. We also evaluate QRRetriever as a re-ranker on the BEIR benchmark and find that it achieves strong zero-shot performance, outperforming other LLM-based re-rankers such as RankGPT. Further analysis shows that both the query-context attention scoring and task selection are crucial for identifying QRHead with strong downstream utility. Overall, our work contributes a general-purpose retriever and offers interpretability insights into the long-context capabilities of LMs.

Method: Query-to-Event decomposition using knowledge embedded in LLMs and VLMs, adaptable across datasets/domains/models, with entropy-based fusion scoring for zero-shot fusion of multimodal knowledge (visual and speech-based inputs).

Result: Q2E outperforms several state-of-the-art baselines on two diverse datasets across multiple retrieval metrics, and integrating audio information significantly improves text-to-video retrieval.

Conclusion: The approach successfully enhances understanding of simplified human queries through decomposition and demonstrates the value of integrating multimodal knowledge (including audio) for improved video retrieval performance.

Abstract: Recent approaches have shown impressive proficiency in extracting and leveraging parametric knowledge from Large-Language Models (LLMs) and Vision-Language Models (VLMs). In this work, we consider how we can improve the identification and retrieval of videos related to complex real-world events by automatically extracting latent parametric knowledge about those events. We present Q2E: a Query-to-Event decomposition method for zero-shot multilingual text-to-video retrieval, adaptable across datasets, domains, LLMs, or VLMs. Our approach demonstrates that we can enhance the understanding of otherwise overly simplified human queries by decomposing the query using the knowledge embedded in LLMs and VLMs. We additionally show how to apply our approach to both visual and speech-based inputs. To combine this varied multimodal knowledge, we adopt entropy-based fusion scoring for zero-shot fusion. Through evaluations on two diverse datasets and multiple retrieval metrics, we demonstrate that Q2E outperforms several state-of-the-art baselines. Our evaluation also shows that integrating audio information can significantly improve text-to-video retrieval. We have released code and data for future research.

Method: CGLS progressively adds layers during training while simultaneously increasing data difficulty using curricula. At 100M scale: synthetic short stories → general web data. At 1.2B scale: general text → technical/specialized content using DistilBERT-based classifier for stratification.

Result: CGLS outperforms baseline methods on PIQA and ARC benchmarks at 100M scale. At 1.2B scale, progressive depth increase with sample difficulty leads to better generalization and zero-shot performance on various downstream benchmarks.

Conclusion: CGLS effectively unlocks the potential of progressive stacking, offering a simple yet effective strategy for improving generalization on knowledge-intensive and reasoning tasks during pretraining.

Abstract: As the cost of pretraining large language models grows, there is continued interest in strategies to improve learning efficiency during this core training stage. Motivated by cognitive development, where humans gradually build knowledge as their brains mature, we propose Curriculum-Guided Layer Scaling (CGLS), a framework for compute-efficient pretraining that synchronizes increasing data difficulty with model growth through progressive layer stacking (i.e. gradually adding layers during training). At the 100M parameter scale, using a curriculum transitioning from synthetic short stories to general web data, CGLS outperforms baseline methods on the question-answering benchmarks PIQA and ARC. Pretraining at the 1.2B scale, we stratify the DataComp-LM corpus with a DistilBERT-based classifier and progress from general text to highly technical or specialized content. Our results show that progressively increasing model depth alongside sample difficulty leads to better generalization and zero-shot performance on various downstream benchmarks. Altogether, our findings demonstrate that CGLS unlocks the potential of progressive stacking, offering a simple yet effective strategy for improving generalization on knowledge-intensive and reasoning tasks.

Method: Based on FM-index data structure that simultaneously indexes and compresses text, creating indexes with size only 44% of the corpus. The system improves indexing speed (18×) and reduces memory use during indexing (3.2×) and querying.

Result: Indexed 83TB of Internet text in 99 days with single CPU node. Found several core LM evaluation benchmarks heavily contaminated in Internet crawls (up to 74.2% in GSM8K), potentially overestimating language model capabilities.

Conclusion: infini-gram mini enables efficient large-scale text analysis and reveals critical benchmark contamination issues. The system provides web interface and API for general search queries, with a contamination bulletin for sharing benchmark contamination rates.

Abstract: Language models are trained mainly on massive text data from the Internet, and it becomes increasingly important to understand this data source. Exact-match search engines enable searching in large text corpora - counting string appearances and retrieving the enclosing documents - yet the high storage overhead hinders their application on Internet-scale data. We present infini-gram mini, an efficient and scalable system that can make petabyte-level text corpora searchable. Based on the FM-index data structure (Ferragina and Manzini, 2000), which simultaneously indexes and compresses text, our system creates indexes with size only 44% of the corpus. Infini-gram mini greatly improves upon the best existing implementation of FM-index in terms of indexing speed (18$\times$) and memory use during both indexing (3.2$\times$ reduction) and querying (down to a negligible amount). We index 83TB of Internet text in 99 days with a single CPU node with 128 vCPUs (or 19 hours if using 137 such nodes). We show one important use case of infini-gram mini in a large-scale analysis of benchmark contamination. We find several core LM evaluation benchmarks to be heavily contaminated in Internet crawls (up to 74.2% in GSM8K), which could lead to overestimating the capabilities of language models if trained on such data. We host a benchmark contamination bulletin to share the contamination rate of many core and community-contributed benchmarks. We also release a web interface and an API endpoint to serve general search queries on infini-gram mini indexes.

Method: BOW inserts an intermediate reasoning bottleneck where the policy model must first generate a next-word reasoning trajectory. A frozen scorer then assigns a distributional reward based on the probability of the gold next token conditioned on the trajectory, with optional L1-style regularization to prevent shortcuts.

Result: Across ten benchmarks, BOW adaptation on Qwen2.5-7B-Instruct and Llama3.1-8B-Instruct improved zero-shot reasoning by nearly 5% on average compared to strong baselines, achieving top results in 7 of 10 intrinsic NWP evaluations.

Conclusion: BOW is a viable alternative to vanilla NWP that induces explicit next-word reasoning and strengthens general reasoning ability, outperforming both RL variants with hard binary rewards and supervised finetuning approaches.

Abstract: Large language models (LLMs) are typically pretrained with next-word prediction (NWP), which yields strong surface fluency but places limited pressure on models to form explicit reasoning before emitting tokens. We study whether shifting the supervision signal can better elicit explicit reasoning and, more broadly, strengthen models’ general reasoning capability. We present BOttlenecked next-Word prediction (BOW), a RL formulation of NWP that inserts an intermediate reasoning bottleneck. Instead of predicting the next word directly from context, the policy model must first generate a next-word reasoning trajectory. A frozen scorer then assigns this trajectory a soft, distributional reward equal to the probability of the gold next token conditioned solely on the trajectory to guide the RL optimization. We also propose an optional L1-style regularizer on the reward to discourage “name-the-answer” shortcuts. Across ten benchmarks, a brief BOW adaptation phase on Qwen2.5-7B-Instruct and Llama3.1-8B-Instruct improves zero-shot reasoning and outperforms strong continual-pretraining baselines, including an RL variant with a hard, binary reward and a supervised finetuning approach with augmented data, by nearly 5% on average, while achieving the top result in 7 of 10 intrinsic NWP evaluations. These results indicate that BOW is a viable alternative to vanilla NWP, inducing explicit next-word reasoning and strengthening general reasoning ability.

Method: Proposes ASEntmax which combines α-entmax sparse attention with a learnable temperature parameter, allowing dynamic interpolation between sparse (pattern-focused) and dense (softmax-like) attention regimes.

Result: ASEntmax achieves up to 1000x length extrapolation on synthetic benchmarks, superior long-context generalization in language modeling while preserving short-context performance, with better perplexity trends and higher retrieval accuracies at 8x training length.

Conclusion: Dynamic sparse attention mechanisms like ASEntmax effectively address softmax’s limitations for long sequences, providing better pattern focus and representational stability through exact zero assignments to irrelevant tokens.

Abstract: Transformer-based architectures traditionally employ softmax to compute attention weights, which produces dense distributions over all tokens in a sequence. While effective in many settings, this density has been shown to be detrimental for tasks that demand precise focus on fixed-size patterns: as sequence length increases, non-informative tokens accumulate attention probability mass, leading to dispersion and representational collapse. We show in this paper that dynamically sparse attention mechanisms using $\alpha$-entmax can avoid these issues, due to their ability to assign exact zeros to irrelevant tokens. Furthermore, we introduce Adaptive-Scalable Entmax (ASEntmax), which endows $\alpha$-entmax with a learnable temperature parameter, allowing the attention distribution to interpolate between sparse (pattern-focused) and dense (softmax-like) regimes. Our empirical evaluation on synthetic tasks and language modeling demonstrates that ASEntmax substantially outperforms softmax, scalable softmax, and fixed-temperature $\alpha$-entmax baselines, achieving up to 1000$\times$ length extrapolation on synthetic benchmarks and superior long-context generalization on language modeling while preserving short-context performance, including better perplexity trends and higher retrieval accuracies at 8$\times$ training length.

Method: Proposed GRAF method that globally refines attack trajectory at each interaction and actively fabricates model responses to suppress safety-related warnings.

Result: Extensive experiments show superior effectiveness compared to existing single-turn and multi-turn jailbreaking methods across six state-of-the-art LLMs.

Conclusion: GRAF successfully addresses the challenge of adapting to evolving dialogue dynamics in multi-turn jailbreaking attacks, demonstrating significant improvements over prior methods.

Abstract: Large Language Models (LLMs) have demonstrated remarkable performance across diverse tasks. Nevertheless, they still pose notable safety risks due to potential misuse for malicious purposes. Jailbreaking, which seeks to induce models to generate harmful content through single-turn or multi-turn attacks, plays a crucial role in uncovering underlying security vulnerabilities. However, prior methods, including sophisticated multi-turn approaches, often struggle to adapt to the evolving dynamics of dialogue as interactions progress. To address this challenge, we propose \ours (JailBreaking via \textbf{G}lobally \textbf{R}efining and \textbf{A}daptively \textbf{F}abricating), a novel multi-turn jailbreaking method that globally refines the attack trajectory at each interaction. In addition, we actively fabricate model responses to suppress safety-related warnings, thereby increasing the likelihood of eliciting harmful outputs in subsequent queries. Extensive experiments across six state-of-the-art LLMs demonstrate the superior effectiveness of our approach compared to existing single-turn and multi-turn jailbreaking methods. Our code will be released at https://github.com/Ytang520/Multi-Turn_jailbreaking_Global-Refinment_and_Active-Fabrication.

Method: Uses 0.5B parameter architecture with mean-pooling and bidirectional representation learning; implements progressive multi-stage training with pre-training, fine-tuning, and contrastive distillation; employs focal-style reweighting and hard-negative mining; curates diverse datasets across 20+ pre-training and 100+ fine-tuning categories.

Result: Achieves state-of-the-art performance on Massive Text Embedding Benchmark, outperforming comparable-size models and rivaling models 3-26x larger.

Conclusion: Sets new standard for versatile compact embedding models under 1B parameters, demonstrating superior performance through systematic training techniques and high-quality data curation.

Abstract: Recent advancements in Large Language Models (LLMs)-based text embedding models primarily focus on data scaling or synthesis, yet limited exploration of training techniques and data quality, thereby constraining performance. In this work, we propose KaLM-Embedding-V2, a series of versatile and compact embedding models, systematically incentivizing advanced embedding capability in LLMs by superior training techniques and high-quality data. For model architecture, we implement the models on a 0.5B compact size with simple mean-pooling to produce fixed-length embeddings and remove the causal attention mask to enable fully bidirectional representation learning. For training techniques, we propose a progressive multi-stage training pipeline: pre-training on weakly supervised large-scale datasets, fine-tuning with supervised high-quality datasets, and contrastive distillation with fine-grained soft signals, integrated with focal-style reweighting and online hard-negative mixing to emphasize difficult samples and enrich hard negatives, respectively. For training data, we curate over 20 categories for pre-training and 100 categories for fine-tuning and contrastive distillation, to improve both performance and generalization, leveraging task-specific instructions, hard-negative mining, and example-based multi-class labeling to ensure high quality. Combining these techniques, our KaLM-Embedding-V2 series achieves state-of-the-art performance on the Massive Text Embedding Benchmark, outperforming models of comparable size and rivaling models 3-26x larger, setting a new standard for versatile and compact embedding models under 1B parameters. The code, data, and models will be publicly available to facilitate academic research.

Method: Evaluated 8 LLMs making 4,800 veracity judgments across multiple prompts, comparing reasoning and non-reasoning models.

Result: Reasoning models showed lower truth-bias than non-reasoning models but still higher than human benchmarks; advanced models (o4-mini, GPT-4.1, R1) displayed sycophantic tendencies with good truth accuracy but poor deception accuracy.

Conclusion: Capability advances alone do not resolve fundamental veracity detection challenges in LLMs, highlighting persistent issues with truth-bias and sycophantic behavior.

Abstract: Despite their widespread use in fact-checking, moderation, and high-stakes decision-making, large language models (LLMs) remain poorly understood as judges of truth. This study presents the largest evaluation to date of LLMs’ veracity detection capabilities and the first analysis of these capabilities in reasoning models. We had eight LLMs make 4,800 veracity judgments across several prompts, comparing reasoning and non-reasoning models. We find that rates of truth-bias, or the likelihood to believe a statement is true, regardless of whether it is actually true, are lower in reasoning models than in non-reasoning models, but still higher than human benchmarks. Most concerning, we identify sycophantic tendencies in several advanced models (o4-mini and GPT-4.1 from OpenAI, R1 from DeepSeek), which displayed an asymmetry in detection accuracy, performing well in truth accuracy but poorly in deception accuracy. This suggests that capability advances alone do not resolve fundamental veracity detection challenges in LLMs.

Method: Developed a formal framework for social network simulation and empirically tested different approaches to imitate user communication on X platform in both English and German languages.

Result: Findings indicate that social simulations should be validated by their empirical realism in the specific context where simulation components were originally fitted.

Conclusion: The paper argues for increased rigor when applying generative-agent-based modeling for social simulation, emphasizing the importance of context-specific validation.

Abstract: The ability of Large Language Models (LLMs) to mimic human behavior triggered a plethora of computational social science research, assuming that empirical studies of humans can be conducted with AI agents instead. Since there have been conflicting research findings on whether and when this hypothesis holds, there is a need to better understand the differences in their experimental designs. We focus on replicating the behavior of social network users with the use of LLMs for the analysis of communication on social networks. First, we provide a formal framework for the simulation of social networks, before focusing on the sub-task of imitating user communication. We empirically test different approaches to imitate user behavior on X in English and German. Our findings suggest that social simulations should be validated by their empirical realism measured in the setting in which the simulation components were fitted. With this paper, we argue for more rigor when applying generative-agent-based modeling for social simulation.

Method: SemDiD uses three mechanisms: orthogonal directional guidance, dynamic inter-group repulsion, and position-debiased probability assessment, harmonized through adaptive gain functions and constraint optimization.

Result: SemDiD outperforms existing methods, improving Best-of-N coverage by 1.4-5.2% across tasks and accelerating RLHF training convergence by 15% while increasing accuracy by up to 2.1%.

Conclusion: SemDiD effectively balances quality and diversity in LLM decoding, enabling better semantic differentiation for various applications.

Abstract: Diverse decoding of large language models is crucial for applications requiring multiple semantically distinct responses, yet existing methods primarily achieve lexical rather than semantic diversity. This limitation significantly constrains Best-of-N strategies, group-based reinforcement learning, and data synthesis. While temperature sampling and diverse beam search modify token distributions or apply n-gram penalties, they fail to ensure meaningful semantic differentiation. We introduce Semantic-guided Diverse Decoding (SemDiD), operating directly in embedding space that balances quality with diversity through three complementary mechanisms: orthogonal directional guidance, dynamic inter-group repulsion, and position-debiased probability assessment. SemDiD harmonizes these competing objectives using adaptive gain functions and constraint optimization, ensuring both quality thresholds and maximal semantic differentiation. Experiments show SemDiD consistently outperforms existing methods, improving Best-of-N coverage by 1.4-5.2% across diverse tasks and accelerating RLHF training convergence by 15% while increasing accuracy by up to 2.1%.

Method: Examined Huginn-3.5B’s internal behavior on arithmetic tasks using probing techniques including Logit Lens and Coda Lens, tracking rank trajectories of final and intermediate result tokens across recurrent blocks.

Result: Limited evidence of interpretable latent CoT was found, with significant probing inconsistencies across recurrent blocks. Hidden state interpretability depends heavily on layer index and decoding method. Increasing recurrence depth yields only marginal gains.

Conclusion: Latent CoT reasoning does not robustly emerge in the studied depth-recurrent Transformer, and explicit externalization of reasoning steps remains more effective than attempting to internalize reasoning in latent space.

Abstract: Chain-of-thought (CoT) reasoning has enabled transformer-based language models to excel at complex mathematics and multi-step planning. However, in standard decoder-only architectures, these reasoning steps are externalized in natural language, improving interpretability at the cost of efficiency. To capture reasoning that is not easily represented in words, many works have explored recurrent architectures that aim to internalize reasoning in latent space, potentially supporting latent CoT. In this paper, we investigate whether such reasoning structures emerge in Huginn-3.5B, a depth-recurrent Transformer that reuses layers at inference time without increasing parameter count. We examine the model’s internal behavior on arithmetic tasks using a suite of probing techniques including the Logit Lens and Coda Lens. Our findings reveal limited evidence of interpretable latent CoT by tracking rank trajectories of final and intermediate result tokens. Furthermore, we uncover significant probing inconsistencies across recurrent blocks, where the interpretability of hidden states depends heavily on both the layer index and the decoding method. Finally, we empirically show that increasing recurrence depth yields only marginal gains and falls well short of models that explicitly externalize reasoning steps. The code is available at https://github.com/wenquanlu/huginn-latent-cot.

Method: Integrates cognitive dual-memory model: episodic memory for historical user engagements and semantic memory for long-term evolving beliefs. Introduces PRIME framework with personalized thinking capability inspired by slow thinking strategy.

Result: Extensive experiments validate PRIME’s effectiveness across both long- and short-context scenarios. Captures dynamic personalization beyond popularity biases.

Conclusion: PRIME provides a unified theoretical framework for LLM personalization that effectively captures dynamic user preferences using cognitive memory mechanisms.

Abstract: Large language model (LLM) personalization aims to align model outputs with individuals’ unique preferences and opinions. While recent efforts have implemented various personalization methods, a unified theoretical framework that can systematically understand the drivers of effective personalization is still lacking. In this work, we integrate the well-established cognitive dual-memory model into LLM personalization, by mirroring episodic memory to historical user engagements and semantic memory to long-term, evolving user beliefs. Specifically, we systematically investigate memory instantiations and introduce a unified framework, PRIME, using episodic and semantic memory mechanisms. We further augment PRIME with a novel personalized thinking capability inspired by the slow thinking strategy. Moreover, recognizing the absence of suitable benchmarks, we introduce a dataset using Change My View (CMV) from Reddit, specifically designed to evaluate long-context personalization. Extensive experiments validate PRIME’s effectiveness across both long- and short-context scenarios. Further analysis confirms that PRIME effectively captures dynamic personalization beyond mere popularity biases.

Method: Uses local small models to compute logits differences between personal context-aware and -agnostic outputs as steering signals, formulating token-level optimization as online learning to dynamically adjust remote LLM’s logits on-device.

Result: Effectively assists LLMs in generating personalized content using local user profiles and histories while preserving privacy through on-device processing and maintaining acceptable computational overhead.

Conclusion: CoSteer addresses the personalization dichotomy by enabling collaborative decoding-time personalization that preserves privacy and maintains cloud LLM capabilities without fine-tuning.

Abstract: Personalized text generation has become crucial for adapting language models to diverse and evolving users’ personal context across cultural, temporal, and contextual dimensions. While existing methods often rely on centralized fine-tuning or static preference alignment, they struggle to achieve real-time adaptation under resource constraints inherent to personal devices. This limitation creates a dilemma: large cloud-based models lack access to localized user-specific information, while small on-device models cannot match the generation quality of their cloud counterparts. To address this dichotomy, we present CoSteer, a novel collaborative framework that enables decoding-time personalization through localized delta steering. Our key insight lies in leveraging the logits difference between personal context-aware and -agnostic outputs from local small models as steering signals for cloud-based LLMs. Specifically, we formulate token-level optimization as an online learning problem, where local delta vectors dynamically adjust the remote LLM’s logits within the on-device environment. This approach preserves privacy by transmitting only the final steered tokens rather than raw data or intermediate vectors, while maintaining cloud-based LLMs’ general capabilities without fine-tuning. Through comprehensive experiments on various personalized generation tasks, we demonstrate that CoSteer effectively assists LLMs in generating personalized content by leveraging locally stored user profiles and histories, ensuring privacy preservation through on-device data processing while maintaining acceptable computational overhead.

Method: Programmatically renders generated artifacts and captures dynamic behavior through temporal screenshots, then uses Multimodal LLM-as-Judge with fine-grained checklists for holistic scoring.

Result: Achieved 94.4% ranking consistency with human preference gold standard and over 90% pairwise agreement with human experts, establishing reliable automated assessment of human-perceived quality.

Conclusion: ArtifactsBench provides the first scalable framework for automated evaluation of visual code generation quality, revealing that generalist models often outperform domain-specific ones, and is open-sourced for community use.

Abstract: The generative capabilities of Large Language Models (LLMs) are rapidly expanding from static code to dynamic, interactive visual artifacts. This progress is bottlenecked by a critical evaluation gap: established benchmarks focus on algorithmic correctness and are blind to the visual fidelity and interactive integrity that define modern user experiences. To bridge this gap, we introduce ArtifactsBench, a new benchmark and paradigm for the automated, multimodal evaluation of visual code generation. Our framework programmatically renders each generated artifact and captures its dynamic behavior through temporal screenshots. This visual evidence, alongside the source code, is then assessed by a Multimodal LLM (MLLM)-as-Judge, which is rigorously guided by a fine-grained, per-task checklist to ensure holistic and reproducible scoring. We construct a new benchmark of 1,825 diverse tasks and evaluate over 30 leading LLMs. Our automated evaluation achieves a striking 94.4% ranking consistency with WebDev Arena, the gold-standard for human preference in web development, and over 90% pairwise agreement with human experts. This establishes ArtifactsBench as the first framework to reliably automate the assessment of human-perceived quality at scale. Our analysis provides a high-resolution map of the current SOTA, revealing that generalist models often outperform domain-specific ones. We open-source ArtifactsBench, including the benchmark, evaluation harness, and baseline results at https://artifactsbenchmark.github.io/, to provide the community with a scalable and accurate tool to accelerate the development of user-centric generative models.

Method: Empirical experiments on OLMo models to study the relationship between data entropy and memorization scores, including a case study on memorizing randomized strings.

Result: Found a linear correlation between data entropy and memorization score (Entropy-Memorization Law), and observed that random sequences have unexpectedly low empirical entropy compared to the training corpus.

Conclusion: The Entropy-Memorization Law provides a simple approach to distinguish training and testing data, enabling Dataset Inference (DI).

Abstract: Large Language Models (LLMs) are known to memorize portions of their training data, sometimes reproducing content verbatim when prompted appropriately. In this work, we investigate a fundamental yet under-explored question in the domain of memorization: How to characterize memorization difficulty of training data in LLMs? Through empirical experiments on OLMo, a family of open models, we present the Entropy-Memorization Law. It suggests that data entropy is linearly correlated with memorization score. Moreover, in a case study of memorizing highly randomized strings, or “gibberish”, we observe that such sequences, despite their apparent randomness, exhibit unexpectedly low empirical entropy compared to the broader training corpus. Adopting the same strategy to discover Entropy-Memorization Law, we derive a simple yet effective approach to distinguish training and testing data, enabling Dataset Inference (DI).

Method: Used circuit-style interpretability techniques like path patching to analyze models’ internal computations, specifically studying off-by-one addition as a counterfactual task.

Result: Uncovered a function induction mechanism that explains generalization from standard to off-by-one addition, showing it’s governed by multiple parallel attention heads and is reusable across various tasks.

Conclusion: The findings provide deeper insights into how reusable and composable structures within language models enable task-level generalization through function induction mechanisms.

Abstract: Large language models demonstrate the intriguing ability to perform unseen tasks via in-context learning. However, it remains unclear what mechanisms inside the model drive such task-level generalization. In this work, we approach this question through the lens of off-by-one addition (i.e., 1+1=3, 2+2=5, 3+3=?), a two-step, counterfactual task with an unexpected +1 function as a second step. Leveraging circuit-style interpretability techniques such as path patching, we analyze the models’ internal computations behind their performance and present three key findings. First, we uncover a function induction mechanism that explains the model’s generalization from standard addition to off-by-one addition. This mechanism resembles the structure of the induction head mechanism found in prior work and elevates it to a higher level of abstraction. Second, we show that the induction of the +1 function is governed by multiple attention heads in parallel, each of which emits a distinct piece of the +1 function. Finally, we find that this function induction mechanism is reused in a broader range of tasks, including synthetic tasks such as shifted multiple-choice QA and algorithmic tasks such as base-8 addition. Overall, our findings offer deeper insights into how reusable and composable structures within language models enable task-level generalization.

Method: Adapt pretrained language models by copying language heads from the last layer to selected intermediate layers and fine-tuning them with different task inputs.

Result: Validated that intermediate layers can generate meaningful content, achieving SOTA performance on hierarchical text classification, classification-guided generation, and hierarchical text generation across multiple datasets.

Conclusion: Demonstrates the feasibility of hierarchical decoders and suggests potential for generalized hierarchical reasoners through pretraining from scratch.

Abstract: Decoder-only language models, such as GPT and LLaMA, generally decode on the last layer. Motivated by human’s hierarchical thinking capability, we propose that a hierarchical decoder architecture could be built with different layers decoding texts simultaneously. Due to limited time and computationally resources, we choose to adapt a pretrained language model into this form of hierarchical decoder. Language heads of the last layer are copied to different selected intermediate layers, and fine-tuned with different task inputs. By thorough experiments, we validate that these selective intermediate layers could be adapted to speak meaningful and reasonable contents, and this paradigm of hierarchical decoder can obtain state-of-the-art performances on multiple tasks such as hierarchical text classification, classification-guided generation, and hierarchical text generation. HdLM outperforms all baselines on WoS, DBpedia, ESconv, EmpatheticDialogues, and several cognitive tests. We also provide thorough theoretical analysis to validate the convergence and computational savings of our methodology. This study suggests the possibility of a generalized hierarchical reasoner, pretraining from scratch.

Method: Built on pre-trained OpenAI embeddings and a multi-head ordinal classifier, supporting English, Chinese, Malay, and partial Tamil with high-quality local data.

Result: Outperforms several commercial and open-source systems across 17 benchmarks including Singapore-specific and public English datasets, and is actively deployed within the Singapore Government.

Conclusion: High-quality local data and robust multilingual embeddings can achieve strong moderation performance without fine-tuning large models, demonstrating practical efficacy at scale.

Abstract: Modern moderation systems increasingly support multiple languages, but often fail to address localisation and low-resource variants - creating safety gaps in real-world deployments. Small models offer a potential alternative to large LLMs, yet still demand considerable data and compute. We present LionGuard 2, a lightweight, multilingual moderation classifier tailored to the Singapore context, supporting English, Chinese, Malay, and partial Tamil. Built on pre-trained OpenAI embeddings and a multi-head ordinal classifier, LionGuard 2 outperforms several commercial and open-source systems across 17 benchmarks, including both Singapore-specific and public English datasets. The system is actively deployed within the Singapore Government, demonstrating practical efficacy at scale. Our findings show that high-quality local data and robust multilingual embeddings can achieve strong moderation performance, without fine-tuning large models. We release our model weights and part of our training data to support future work on LLM safety.

Method: Two-component approach: 1) Non-public EESE-Pool with 100K+ expert-constructed science instances across 5 disciplines and 500+ subfields, 2) Periodically updated 500-instance subsets for leakage-resilient, low-overhead evaluations.

Result: Experiments on 32 open- and closed-source models show EESE effectively differentiates models’ strengths and weaknesses across scientific fields and cognitive dimensions.

Conclusion: EESE provides a robust, scalable, and forward-compatible solution for science benchmark design, offering realistic measurement of foundation models’ scientific question handling capabilities.

Abstract: As foundation models grow rapidly in capability and deployment, evaluating their scientific understanding becomes increasingly critical. Existing science benchmarks have made progress towards broad Range, wide Reach, and high Rigor, yet they often face two major challenges: data leakage risks that compromise benchmarking validity, and evaluation inefficiency due to large-scale testing. To address these issues, we introduce the Ever-Evolving Science Exam (EESE), a dynamic benchmark designed to reliably assess scientific capabilities in foundation models. Our approach consists of two components: 1) a non-public EESE-Pool with over 100K expertly constructed science instances (question-answer pairs) across 5 disciplines and 500+ subfields, built through a multi-stage pipeline ensuring Range, Reach, and Rigor, 2) a periodically updated 500-instance subset EESE, sampled and validated to enable leakage-resilient, low-overhead evaluations. Experiments on 32 open- and closed-source models demonstrate that EESE effectively differentiates the strengths and weaknesses of models in scientific fields and cognitive dimensions. Overall, EESE provides a robust, scalable, and forward-compatible solution for science benchmark design, offering a realistic measure of how well foundation models handle science questions. The project page is at: https://github.com/aiben-ch/EESE.

Method: Proposes MultiRole-R1 framework with unsupervised data construction pipeline synthesizing reasoning chains with various role perspectives, and reinforcement learning via Group Relative Policy Optimization with reward shaping that treats diversity as a reward signal.

Result: Training on subjective tasks increases in-domain accuracy by 14.1% and out-of-domain accuracy by 7.64%, and even enhances performance on advanced math reasoning like AIME 2024. Diversity is shown to be a more consistent indicator of accuracy than reasoning length.

Conclusion: Introducing perspective diversity and token-level diversity significantly improves subjective reasoning capabilities in LRMs, with diversity serving as a key factor for better performance across both subjective and objective reasoning tasks.

Abstract: Large Reasoning Models (LRMs) with long chain-of-thought capabilities, optimized via reinforcement learning with verifiable rewards (RLVR), excel at objective reasoning tasks like mathematical problem solving and code generation. However, RLVR is known for degrading generation diversity, which causes LRMs to fall short on subjective reasoning that has multiple answers depending on different role perspectives. While recent studies recognize the importance of diversity-enhanced training in objective reasoning, limited attention has been given to subjective tasks. In this paper, we find that subjective reasoning can be improved by introducing perspective diversity and token-level diversity, with the former one providing a coherent scaffolding anchored to a real-world stakeholder group and the latter one broadening the answer search space. We propose MultiRole-R1, a diversity-enhanced training framework featuring an unsupervised data construction pipeline that synthesizes reasoning chains incorporating various role perspectives. It also employs reinforcement learning via Group Relative Policy Optimization with reward shaping, taking diversity as a reward signal in addition to verifiable reward. Training on subjective tasks solely, MultiRole-R1 increases the in-domain and out-of-domain accuracy by 14.1% and 7.64%, and even enhances the performance on advanced math reasoning such as AIME 2024. We further show that diversity is a more consistent indicator of accuracy than reasoning length.

Method: Proposes CTTS-MM framework with: 1) Agent Collaboration Search (ACS) to find optimal LLM combinations, and 2) Mixture of Reward Models (MoR) with Prior Reward model Ensemble Selection (PRES) algorithm for optimal reward ensemble.

Result: Outperforms leading STTS methods by +4.82% over Best-of-N, surpasses proprietary LLMs (+7.06% over GPT-4.1) and open-source LLMs across seven mainstream benchmarks.

Conclusion: Collective scaling has substantial potential to push the frontier of LLM inference, demonstrating that multi-agent and multi-reward collaboration can significantly enhance model performance without additional training.

Abstract: Test-time scaling (TTS) has emerged as a promising, training-free approach for enhancing large language model (LLM) performance. However, the efficacy of existing methods, such as Best-of-N and Self-Consistency, is fundamentally constrained by the dominant single test-time scaling (STTS) paradigm, which relies on a single LLM agent interacting with a single reward model (SA-SR). Inspired by recent work showing that collective methods can surpass the performance ceiling of individual models, we introduce Collective Test-Time Scaling (CTTS). First, we systematically investigate three primary interaction paradigms of existing multiple models: single-agent-multi-reward (SA-MR), multi-agent-single-reward (MA-SR), and multi-agent-multi-reward (MA-MR). Extensive experiments reveal that the MA-MR paradigm is consistently superior. Based on this finding, we further propose CTTS-MM, a novel framework that operationalizes multi-agent and multi-reward collaboration. CTTS-MM integrates two key technical contributions: (1) for agent collaboration, an Agent Collaboration Search (ACS) that identifies the most effective combination of LLMs from a candidate pool; and (2) for reward model collaboration, a Mixture of Reward Models (MoR) strategy that leverages a Prior Reward model Ensemble Selection (PRES) algorithm to select the optimal ensemble. Evaluations across seven mainstream benchmarks demonstrate that CTTS-MM significantly outperforms leading STTS methods (+4.82% over Best-of-N) and surpasses even flagship proprietary LLMs (+7.06% over GPT-4.1) and open-source LLMs. These results highlight the substantial potential of collective scaling to push the frontier of LLM inference. Code will be released at https://github.com/magent4aci/CTTS-MM.

Method: Proposes Sotopia-RL framework that refines coarse episode-level feedback into utterance-level, multi-dimensional rewards. Utterance-level credit assignment addresses partial observability, while multi-dimensional rewards capture social interaction richness and prevent reward hacking.

Result: Achieves state-of-the-art social goal completion scores: 7.17 on Sotopia-hard and 8.31 on Sotopia-full, significantly outperforming existing approaches. Ablation studies confirm necessity of both utterance-level credit assignment and multi-dimensional reward design.

Conclusion: Sotopia-RL effectively addresses RL training challenges for social intelligence by combining utterance-level credit assignment with multi-dimensional rewards, enabling more efficient and stable learning in complex social environments.

Abstract: Social intelligence has become a critical capability for large language models (LLMs), enabling them to engage effectively in real-world social tasks such as accommodation, persuasion, collaboration, and negotiation. Reinforcement learning (RL) is a natural fit for training socially intelligent agents because it allows models to learn sophisticated strategies directly through social interactions. However, social interactions have two key characteristics that set barriers for RL training: (1) partial observability, where utterances have indirect and delayed effects that complicate credit assignment, and (2) multi-dimensionality, where behaviors such as rapport-building or knowledge-seeking contribute indirectly to goal achievement. These characteristics make Markov decision process (MDP)-based RL with single-dimensional episode-level rewards inefficient and unstable. To address these challenges, we propose Sotopia-RL, a novel framework that refines coarse episode-level feedback into utterance-level, multi-dimensional rewards. Utterance-level credit assignment mitigates partial observability by attributing outcomes to individual utterances, while multi-dimensional rewards capture the full richness of social interactions and reduce reward hacking. Experiments in Sotopia, an open-ended social learning environment, demonstrate that Sotopia-RL achieves state-of-the-art social goal completion scores (7.17 on Sotopia-hard and 8.31 on Sotopia-full), significantly outperforming existing approaches. Ablation studies confirm the necessity of both utterance-level credit assignment and multi-dimensional reward design for RL training. Our implementation is publicly available at: https://github.com/sotopia-lab/sotopia-rl.

Method: Analyzed LLM finetuning across various tasks, models, and methods, examining conditions that lead to over-memorization and its effects on model performance.

Result: Over-memorization is prevalent and worsens with prolonged training and large learning rates. Affected models maintain test accuracy but show reduced robustness, poor OOD generalization, and decreased generation diversity.

Conclusion: Proposed checkpoint selection strategies and mitigation techniques like checkpoint merging and memorization-aware reweighting to address over-memorization in LLM finetuning.

Abstract: The pretrained large language models (LLMs) are finetuned with labeled data for better instruction following ability and alignment with human values. In this paper, we study the learning dynamics of LLM finetuning on reasoning tasks and reveal the uncovered over-memorization phenomenon during a specific stage of LLM finetuning. At this stage, the LLMs have excessively memorized training data and exhibit high test perplexity while maintaining good test accuracy. We explore the conditions that contribute to over-memorization and discover that this issue is prevalent across various tasks, models, and fine-tuning methods, with prolonged training and large learning rates exacerbating the problem. Although models with over-memorization demonstrate comparable test accuracy to normal models, they suffer from reduced robustness, poor out-of-distribution generalization, and decreased generation diversity. In light of our findings on over-memorization, we offer recommendations for checkpoint selection and propose techniques such as checkpoint merging and memorization-aware reweighting to mitigate this effect.

Method: Introduces Sculptor framework with three tool categories: (1) context fragmentation, (2) summary/hide/restore operations, and (3) precise search. Uses a dynamic context-aware reinforcement learning approach to train agents that actively modify conversational history.

Result: Experimental evaluation shows Sculptor significantly improves performance on diverse long-context benchmarks without specific training, leveraging LLMs’ inherent tool-calling and instruction-following capabilities.

Conclusion: Explicit context-control strategies, rather than merely larger token windows, are key to robustness at scale. Active Context Management mitigates proactive interference and provides a cognitive foundation for reliable reasoning across long-context tasks.

Abstract: Large Language Models (LLMs) suffer from significant performance degradation when processing long contexts due to proactive interference, where irrelevant information in earlier parts of the context disrupts reasoning and memory recall. While most research focuses on external memory systems to augment LLMs’ capabilities, we propose a complementary approach: empowering LLMs with Active Context Management (ACM) tools to actively sculpt their internal working memory. We introduce Sculptor, a framework that equips LLMs with three categories of tools: (1) context fragmentation, (2) summary, hide, and restore, and (3) precise search. Our approach enables LLMs to proactively manage their attention and working memory, analogous to how humans selectively focus on relevant information while filtering out distractions. Experimental evaluation on diverse long-context benchmarks demonstrates that Sculptor significantly improves performance even without specific training, leveraging LLMs’ inherent tool-calling and instruction-following capabilities. To further optimize these strategies, we introduce a novel dynamic context-aware reinforcement learning (RL) approach, advancing the training of an agent that actively modifies its own conversational history. By enabling Active Context Management, Sculptor not only mitigates proactive interference but also provides a cognitive foundation for more reliable reasoning across diverse long-context tasks-highlighting that explicit context-control strategies, rather than merely larger token windows, are key to robustness at scale.

Method: Proposed GuideEval benchmark grounded in authentic educational dialogues, using a three-phase behavioral framework: (1) Perception - inferring learner states, (2) Orchestration - adapting instructional strategies, (3) Elicitation - stimulating proper reflections. Also introduced behavior-guided finetuning strategy using behavior-prompted instructional dialogues.

Result: Empirical results show existing LLMs often fail to provide effective adaptive scaffolding when learners experience confusion or require redirection. The proposed behavior-guided finetuning strategy substantially enhances guidance performance.

Conclusion: The work advocates shifting focus from isolated content evaluation to learner-centered state-aware interaction, promoting a more dialogic paradigm for evaluating Socratic LLMs in educational contexts.

Abstract: The conversational capabilities of large language models hold significant promise for enabling scalable and interactive tutoring. While prior research has primarily examined their ability to generate Socratic questions, it often overlooks a critical aspect: adaptively guiding learners in accordance with their cognitive states. This study moves beyond question generation to emphasize instructional guidance capability. We ask: Can LLMs emulate expert tutors who dynamically adjust strategies in response to learners’ states? To investigate this, we propose GuideEval, a benchmark grounded in authentic educational dialogues that evaluates pedagogical guidance through a three-phase behavioral framework: (1) Perception, inferring learner states; (2) Orchestration, adapting instructional strategies; and (3) Elicitation, stimulating proper reflections. Empirical results indicate that existing LLMs often fail to provide effective adaptive scaffolding when learners experience confusion or require redirection. To complement the quantitative evaluation, we conduct a detailed failure case analysis, providing an intuitive understanding of these shortcomings. Furthermore, we introduce a behavior-guided finetuning strategy that leverages behavior-prompted instructional dialogues, substantially enhancing guidance performance. By shifting the focus from isolated content evaluation to learner-centered state-aware interaction, our work advocates a more dialogic paradigm for evaluating Socratic LLMs.

Method: Developed Omni-SafetyBench with 24 modality variations and 972 samples per variation, including audio-visual harm cases. Proposed tailored metrics: Safety-score (based on C-ASR and C-RR) and Cross-Modal Safety Consistency score (CMSC-score).

Result: Evaluation of 10 OLLMs revealed critical vulnerabilities: only 3 models achieved >0.6 in both safety metrics, safety defenses weaken with complex inputs (especially audio-visual), and some models scored as low as 0.14 on specific modalities.

Conclusion: Existing safety alignment methods face fundamental limitations: inference-time methods cannot alter model understanding, post-training methods struggle with out-of-distribution issues, and audio-visual tasks are inherently more complex. Urgent need for enhanced OLLM safety approaches.

Abstract: The rise of Omni-modal Large Language Models (OLLMs), which integrate visual and auditory processing with text, necessitates robust safety evaluations to mitigate harmful outputs. However, no dedicated benchmarks currently exist for OLLMs, and existing benchmarks fail to assess safety under joint audio-visual inputs or cross-modal consistency. To fill this gap, we introduce Omni-SafetyBench, the first comprehensive parallel benchmark for OLLM safety evaluation, featuring 24 modality variations with 972 samples each, including audio-visual harm cases. Considering OLLMs’ comprehension challenges with complex omni-modal inputs and the need for cross-modal consistency evaluation, we propose tailored metrics: a Safety-score based on Conditional Attack Success Rate (C-ASR) and Refusal Rate (C-RR) to account for comprehension failures, and a Cross-Modal Safety Consistency score (CMSC-score) to measure consistency across modalities. Evaluating 6 open-source and 4 closed-source OLLMs reveals critical vulnerabilities: (1) only 3 models achieving over 0.6 in both average Safety-score and CMSC-score; (2) safety defenses weaken with complex inputs, especially audio-visual joints; (3) severe weaknesses persist, with some models scoring as low as 0.14 on specific modalities. Using Omni-SafetyBench, we evaluated existing safety alignment algorithms and identified key challenges in OLLM safety alignment: (1) Inference-time methods are inherently less effective as they cannot alter the model’s underlying understanding of safety; (2) Post-training methods struggle with out-of-distribution issues due to the vast modality combinations in OLLMs; and, safety tasks involving audio-visual inputs are more complex, making even in-distribution training data less effective. Our proposed benchmark, metrics and the findings highlight urgent needs for enhanced OLLM safety.

Method: READER formalizes speculative decoding as a stochastic tree construction problem, exploits natural language redundancy to generate candidate continuations, and introduces memory-optimal key-value cache-serving for batched inference with sublinear overhead.

Result: Achieves up to 6.13x wall-clock speedup on single-prompt inference and up to 5.92x on batched inference, consistently surpassing prior speculative decoding baselines while preserving exact output equivalence.

Conclusion: READER closes the gap between theoretical parallelism limits and practical LLM inference, suggesting a new standard for efficient deployment of large language models.

Abstract: Autoregressive Language Models instantiate a factorized likelihood over token sequences, yet their strictly sequential decoding process imposes an intrinsic lower bound on inference latency. This bottleneck has emerged as a central obstacle to the scalable deployment of large-scale generative models. Existing acceleration techniques partially mitigate token-level latency by relying on auxiliary draft models or introducing an additional training phase, but fail to address the dominant memory and communication costs. We present READER, a provably lossless speculative decoding framework that bypasses the training of the auxiliary draft model. READER formalizes speculative decoding as a stochastic tree construction problem and exploits the empirical redundancy structure of natural language to generate high-probability candidate continuations. Our method revisits the problem of constructing draft trees, establishing substantial statistical improvements over stochastic draft-tree methods and providing a complexity-theoretic analysis that characterizes the optimality frontier of speculative decoding under bounded computation and memory resources. Beyond the single-sequence regime traditionally considered in prior work, we introduce a memory-optimal key-value cache-serving strategy that guarantees amortized sublinear overhead in the batch dimension, allowing READER to scale to realistic inference workloads. Comprehensive experiments demonstrate up to 6.13x wall-clock speedup on single-prompt inference and up to 5.92x on batched inference, consistently surpassing prior speculative decoding baselines, while preserving exact output equivalence, with even more pronounced gains in retrieval-augmented generation pipelines. Our results close a key gap between theoretical parallelism limits and practical LLM inference, suggesting a new standard for efficient deployment.

Method: Created PakBBQ dataset with 214 templates and 17,180 QA pairs in English/Urdu across 8 bias categories relevant to Pakistan. Evaluated multilingual LLMs under ambiguous/disambiguated contexts and negative/non-negative question framings.

Result: Disambiguation improved accuracy by 12% on average; Urdu prompts showed stronger counter-bias behavior than English; negative question framing reduced stereotypical responses.

Conclusion: Contextualized benchmarks and simple prompt engineering strategies are crucial for bias mitigation in low-resource settings, highlighting the importance of culturally adapted evaluation frameworks.

Abstract: With the widespread adoption of Large Language Models (LLMs) across various applications, it is empirical to ensure their fairness across all user communities. However, most LLMs are trained and evaluated on Western centric data, with little attention paid to low-resource languages and regional contexts. To address this gap, we introduce PakBBQ, a culturally and regionally adapted extension of the original Bias Benchmark for Question Answering (BBQ) dataset. PakBBQ comprises over 214 templates, 17180 QA pairs across 8 categories in both English and Urdu, covering eight bias dimensions including age, disability, appearance, gender, socio-economic status, religious, regional affiliation, and language formality that are relevant in Pakistan. We evaluate multiple multilingual LLMs under both ambiguous and explicitly disambiguated contexts, as well as negative versus non negative question framings. Our experiments reveal (i) an average accuracy gain of 12% with disambiguation, (ii) consistently stronger counter bias behaviors in Urdu than in English, and (iii) marked framing effects that reduce stereotypical responses when questions are posed negatively. These findings highlight the importance of contextualized benchmarks and simple prompt engineering strategies for bias mitigation in low resource settings.

Method: CoDiEmb integrates three key innovations: (1) Task-specialized objectives with dynamic sampler forming single-task batches to prevent gradient interference, (2) Delta-guided model fusion strategy computing fine-grained merging weights by analyzing parameter deviations, (3) Efficient single-stage training pipeline that converges stably.

Result: Extensive experiments on 15 standard IR and STS benchmarks across three base encoders validate CoDiEmb. The framework not only mitigates cross-task trade-offs but also measurably improves the geometric properties of the embedding space.

Conclusion: CoDiEmb successfully reconciles the divergent requirements of IR and STS in a collaborative yet distinct manner, demonstrating that systematic decoupling of task-specific learning signals throughout the training pipeline can resolve the conflict between these fundamentally disparate tasks.

Abstract: Learning unified text embeddings that excel across diverse downstream tasks is a central goal in representation learning, yet negative transfer remains a persistent obstacle. This challenge is particularly pronounced when jointly training a single encoder for Information Retrieval (IR) and Semantic Textual Similarity (STS), two essential but fundamentally disparate tasks for which naive co-training typically yields steep performance trade-offs. We argue that resolving this conflict requires systematically decoupling task-specific learning signals throughout the training pipeline. To this end, we introduce CoDiEmb, a unified framework that reconciles the divergent requirements of IR and STS in a collaborative yet distinct manner. CoDiEmb integrates three key innovations for effective joint optimization: (1) Task-specialized objectives paired with a dynamic sampler that forms single-task batches and balances per-task updates, thereby preventing gradient interference. For IR, we employ a contrastive loss with multiple positives and hard negatives, augmented by cross-device sampling. For STS, we adopt order-aware objectives that directly optimize correlation and ranking consistency. (2) A delta-guided model fusion strategy that computes fine-grained merging weights for checkpoints by analyzing each parameter’s deviation from its pre-trained initialization, proving more effective than traditional Model Soups. (3) An efficient, single-stage training pipeline that is simple to implement and converges stably. Extensive experiments on 15 standard IR and STS benchmarks across three base encoders validate CoDiEmb. Our results and analysis demonstrate that the framework not only mitigates cross-task trade-offs but also measurably improves the geometric properties of the embedding space.

Method: Reverse-engineer over 120,000 design logics from existing questions, then match these with source documents (book and web corpora) to automatically generate challenging reasoning questions.

Result: Created two large-scale datasets (DLR-Book: 3.04M questions, DLR-Web: 1.66M questions) spanning 75 disciplines, with greater difficulty and diversity than baseline datasets. Fine-tuning on this data substantially enhanced LLM reasoning capabilities.

Conclusion: The DESIGNER pipeline successfully generates high-quality reasoning data that significantly improves LLM performance, with base models even surpassing their official instruction-tuned counterparts after fine-tuning.

Abstract: Large language models (LLMs) have achieved remarkable success in many natural language tasks but still struggle with complex, multi-step reasoning, particularly across diverse disciplines. Existing reasoning datasets often lack disciplinary breadth, reasoning depth, and diversity, and lack guiding principles for question synthesis. We propose DESIGNER: a DESIGN-logic-guidEd Reasoning data synthesis pipeline that leverages naturally available, extensive raw documents (e.g., book corpus and web corpus) to generate multidisciplinary challenging questions. We introduce the concept of “design logic” and instruct LLMs to mimic human educators’ question-creation process, enabling automated synthesis of large-scale, high-difficulty questions. We use LLMs to reverse-engineer and abstract over 120,000 design logics from existing questions across various disciplines. By matching these design logics with source documents, we are able to create reasoning questions that far surpass the difficulty and diversity of existing datasets. Using this pipeline, we synthesized two large-scale reasoning datasets that span 75 disciplines: DLR-Book (3.04 million questions from the book corpus) and DLR-Web (1.66 million questions from the web corpus). Data analysis indicates that the questions synthesized by our method exhibit greater difficulty and diversity compared to those in the baseline datasets. We validate our synthesized data through supervised fine-tuning (SFT) on the Qwen3 and Llama3 model families. Our data substantially enhances their multidisciplinary reasoning capabilities, outperforming existing datasets. Notably, after SFT on our datasets, the base versions of these models even surpass their official instruction-tuned counterparts.

Method: Online self-play strategy that uses policy’s correct solutions to synthesize variational problems while keeping reference answers identical to originals.

Result: Achieves 18.3% and 22.8% absolute gains in Pass@32 on AIME24 and AIME25 benchmarks, with consistent improvements across 12 reasoning benchmarks and model sizes from 3B to 32B.

Conclusion: SvS effectively maintains policy entropy during RLVR training, substantially improving Pass@k performance and demonstrating generalizability across various benchmarks and model sizes.

Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has recently emerged as a key paradigm for post-training Large Language Models (LLMs), particularly for complex reasoning tasks. However, vanilla RLVR training has been shown to improve Pass@1 performance at the expense of policy entropy, leading to reduced generation diversity and limiting the Pass@k performance, which typically represents the upper bound of LLM reasoning capability. In this paper, we systematically analyze the policy’s generation diversity from the perspective of training problems and find that augmenting and updating training problems helps mitigate entropy collapse during training. Based on these observations, we propose an online Self-play with Variational problem Synthesis (SvS) strategy for RLVR training, which uses the policy’s correct solutions to synthesize variational problems while ensuring their reference answers remain identical to the originals. This self-improving strategy effectively maintains policy entropy during training and substantially improves Pass@k compared with standard RLVR, sustaining prolonged improvements and achieving absolute gains of 18.3% and 22.8% in Pass@32 performance on the competition-level AIME24 and AIME25 benchmarks. Experiments on 12 reasoning benchmarks across varying model sizes from 3B to 32B consistently demonstrate the generalizability and robustness of SvS.

Method: Fine-tuned LLaMA and Mistral models on University of Chicago Medicine reports, using draft generation followed by refinement with ML and RLHF to personalize impressions while ensuring factual accuracy.

Result: The framework generates personalized impressions aligned with individual radiologists’ styles while maintaining clinical precision.

Conclusion: The approach significantly reduces administrative workload and improves reporting efficiency while maintaining high standards of clinical accuracy.

Abstract: The manual creation of the “Impression” section in radiology reports is a primary driver of radiologist burnout. To address this challenge, we propose a coarse-to-fine framework that leverages open-source large language models (LLMs) to automatically generate and personalize impressions from clinical findings. The system first produces a draft impression and then refines it using machine learning and reinforcement learning from human feedback (RLHF) to align with individual radiologists’ styles while ensuring factual accuracy. We fine-tune LLaMA and Mistral models on a large dataset of reports from the University of Chicago Medicine. Our approach is designed to significantly reduce administrative workload and improve reporting efficiency while maintaining high standards of clinical precision.

Method: Uses Post Neural Architecture Search (PostNAS) pipeline that starts with pre-trained full-attention models, freezes MLP weights, and explores attention block designs through four components: optimal layer placement/elimination, linear attention selection, new attention block design, and hardware-aware hyperparameter search.

Result: Jet-Nemotron-2B achieves comparable or superior accuracy to Qwen3, Qwen2.5, Gemma3, and Llama3.2 across benchmarks, with up to 53.6x generation throughput speedup and 6.1x prefilling speedup. It also outperforms larger MoE models like DeepSeek-V3-Small and Moonlight on MMLU and MMLU-Pro.

Conclusion: The PostNAS pipeline enables efficient development of hybrid-architecture models that deliver both high accuracy and significant performance improvements, demonstrating the effectiveness of the approach for balancing model quality and inference efficiency.

Abstract: We present Jet-Nemotron, a new family of hybrid-architecture language models, which matches or exceeds the accuracy of leading full-attention models while significantly improving generation throughput. Jet-Nemotron is developed using Post Neural Architecture Search (PostNAS), a novel neural architecture exploration pipeline that enables efficient model design. Unlike prior approaches, PostNAS begins with a pre-trained full-attention model and freezes its MLP weights, allowing efficient exploration of attention block designs. The pipeline includes four key components: (1) learning optimal full-attention layer placement and elimination, (2) linear attention block selection, (3) designing new attention blocks, and (4) performing hardware-aware hyperparameter search. Our Jet-Nemotron-2B model achieves comparable or superior accuracy to Qwen3, Qwen2.5, Gemma3, and Llama3.2 across a comprehensive suite of benchmarks while delivering up to 53.6x generation throughput speedup and 6.1x prefilling speedup. It also achieves higher accuracy on MMLU and MMLU-Pro than recent advanced MoE full-attention models, such as DeepSeek-V3-Small and Moonlight, despite their larger scale with 15B total and 2.2B activated parameters.

Method: Proposed a structured claim verification framework that reasons through presupposition-free, decomposed questions to mitigate prompt sensitivity and presupposition issues.

Result: Extensive experiments show state-of-the-art models remain susceptible to prompt variance and presupposition, while the proposed method consistently mitigates these issues with up to 2-5% improvement.

Conclusion: The structured framework using presupposition-free decomposed questions effectively addresses persistent prompt sensitivity and presupposition problems in LLM-based claim verification.

Abstract: Prior work has shown that presupposition in generated questions can introduce unverified assumptions, leading to inconsistencies in claim verification. Additionally, prompt sensitivity remains a significant challenge for large language models (LLMs), resulting in performance variance as high as 3-6%. While recent advancements have reduced this gap, our study demonstrates that prompt sensitivity remains a persistent issue. To address this, we propose a structured and robust claim verification framework that reasons through presupposition-free, decomposed questions. Extensive experiments across multiple prompts, datasets, and LLMs reveal that even state-of-the-art models remain susceptible to prompt variance and presupposition. Our method consistently mitigates these issues, achieving up to a 2-5% improvement.

Method: CORE is optimized end-to-end without predefined compression labels, leveraging downstream task performance as feedback to iteratively refine compression policy for enhanced task effectiveness.

Result: Extensive experiments across four datasets show CORE achieves 3% compression ratio while preventing performance degradation compared to full documents and improving average Exact Match score by 3.3 points.

Conclusion: CORE effectively addresses limitations of existing compression methods by providing lossless context compression that enhances rather than degrades RAG task performance.

Abstract: Retrieval-Augmented Generation (RAG) has emerged as a promising approach to enhance the timeliness of knowledge updates and the factual accuracy of responses in large language models. However, incorporating a large number of retrieved documents significantly increases input length, leading to higher computational costs. Existing approaches to document compression tailored for RAG often degrade task performance, as they typically rely on predefined heuristics in the absence of clear compression guidelines. These heuristics fail to ensure that the compressed content effectively supports downstream tasks. To address these limitations, we propose CORE, a novel method for lossless context compression in RAG. CORE is optimized end-to-end and does not depend on predefined compression labels, which are often impractical to obtain. Instead, it leverages downstream task performance as a feedback signal, iteratively refining the compression policy to enhance task effectiveness. Extensive experiments across four datasets demonstrate the effectiveness of CORE. With a high compression ratio of 3%, CORE not only prevents performance degradation compared to including full documents (i.e., without compression) but also improves the average Exact Match (EM) score by 3.3 points. The code for CORE will be released soon.

Method: Uses fine-tuned generative LLM (Meta-Llama 2-7B) with unsupervised learning methods in NLP, integrating RACE dataset for training. Employs Prompt Engineering to tailor question styles (MCQ, conceptual, factual).

Result: Developed a customized model that can generate various types of questions efficiently for text-based evaluations in English language.

Conclusion: The proposed AQAG system provides a reliable and efficient tool for educators to automate question generation, freeing up valuable time and resources while maintaining fairness and diversity in assessments.

Abstract: In the realm of education, student evaluation holds equal significance to imparting knowledge. To be evaluated, students usually need to go through text-based academic assessment methods. Instructors need to make a diverse set of questions that need to be fair for all students to prove their adequacy over a particular topic. This can prove to be quite challenging as they may need to manually go through several different lecture materials. Our objective is to make this whole process much easier by implementing Automatic Question Answer Generation(AQAG), using a fine-tuned generative LLM. For tailoring the instructor’s preferred question style (MCQ, conceptual, or factual questions), Prompt Engineering (PE) is being utilized. In this research, we propose to leverage unsupervised learning methods in NLP, primarily focusing on the English language. This approach empowers the base Meta-Llama 2-7B model to integrate the RACE dataset as training data for the fine-tuning process. Creating a customized model that will offer efficient solutions for educators, instructors, and individuals engaged in text-based evaluations. A reliable and efficient tool for generating questions and answers can free up valuable time and resources, thus streamlining their evaluation processes.

Method: Representation analysis to identify attention heads that facilitate refusal by reducing attention to Chain-of-Thought tokens, and analysis of activation entanglement between reasoning and safety in neurons.

Result: Discovered specific attention heads modulate rationalization during inference, and found significantly higher activation entanglement in safety-critical neurons after fine-tuning with identified reasoning patterns, correlating with catastrophic forgetting.

Conclusion: Provides a mechanistic account of RIM origins, showing how reasoning-safety entanglement at the neuron level explains the emergence of misalignment when reasoning patterns are introduced.

Abstract: With the growing accessibility and wide adoption of large language models, concerns about their safety and alignment with human values have become paramount. In this paper, we identify a concerning phenomenon: Reasoning-Induced Misalignment (RIM), in which misalignment emerges when reasoning capabilities strengthened-particularly when specific types of reasoning patterns are introduced during inference or training. Beyond reporting this vulnerability, we provide the first mechanistic account of its origins. Through representation analysis, we discover that specific attention heads facilitate refusal by reducing their attention to CoT tokens, a mechanism that modulates the model’s rationalization process during inference. During training, we find significantly higher activation entanglement between reasoning and safety in safety-critical neurons than in control neurons, particularly after fine-tuning with those identified reasoning patterns. This entanglement strongly correlates with catastrophic forgetting, providing a neuron-level explanation for RIM.

Method: Developed CE-Bench using a curated dataset of contrastive story pairs to evaluate sparse autoencoder interpretability through contrastive testing.

Result: CE-Bench reliably measures SAE interpretability and achieves over 70% Spearman correlation with SAEBench results, while eliminating the need for external LLM judges.

Conclusion: CE-Bench provides an effective and lightweight alternative for evaluating sparse autoencoders that aligns well with existing benchmarks and is publicly available.

Abstract: Sparse autoencoders (SAEs) are a promising approach for uncovering interpretable features in large language models (LLMs). While several automated evaluation methods exist for SAEs, most rely on external LLMs. In this work, we introduce CE-Bench, a novel and lightweight contrastive evaluation benchmark for sparse autoencoders, built on a curated dataset of contrastive story pairs. We conduct comprehensive evaluation studies to validate the effectiveness of our approach. Our results show that CE-Bench reliably measures the interpretability of sparse autoencoders and aligns well with existing benchmarks without requiring an external LLM judge, achieving over 70% Spearman correlation with results in SAEBench. The official implementation and evaluation dataset are open-sourced and publicly available.

Method: Conducted controlled experiments across multiple explanation benchmark datasets (general and domain-specific) with various label definition conditions including expert-curated, LLM-generated, perturbed, and swapped definitions.

Result: Explicit label definitions can enhance accuracy and explainability, but their integration is neither guaranteed nor consistent. Models often default to internal representations, particularly in general tasks, while domain-specific tasks benefit more from explicit definitions.

Conclusion: LLMs’ processing of external knowledge alongside pre-existing capabilities requires deeper understanding, as they frequently rely on internalized representations rather than consistently integrating external definitions.

Abstract: Do LLMs genuinely incorporate external definitions, or do they primarily rely on their parametric knowledge? To address these questions, we conduct controlled experiments across multiple explanation benchmark datasets (general and domain-specific) and label definition conditions, including expert-curated, LLM-generated, perturbed, and swapped definitions. Our results reveal that while explicit label definitions can enhance accuracy and explainability, their integration into an LLM’s task-solving processes is neither guaranteed nor consistent, suggesting reliance on internalized representations in many cases. Models often default to their internal representations, particularly in general tasks, whereas domain-specific tasks benefit more from explicit definitions. These findings underscore the need for a deeper understanding of how LLMs process external knowledge alongside their pre-existing capabilities.

Method: Compared two monolingual BERT models: one trained/tested on Swahili data, another on English data. Translated Swahili data to English for evaluation on English model to simulate multilingual LLM processing.

Result: Native Swahili-trained model performed much better with 0.36% error rate vs 1.47% for translated approach - nearly 4x fewer errors despite high-quality translation.

Conclusion: Translation alone doesn’t bridge representational differences; native-language training remains crucial for reliable outcomes. Future work should focus on dataset development for underrepresented languages and better multilingual evaluation.

Abstract: As large language models (LLMs) expand multilingual capabilities, questions remain about the equity of their performance across languages. While many communities stand to benefit from AI systems, the dominance of English in training data risks disadvantaging non-English speakers. To test the hypothesis that such data disparities may affect model performance, this study compares two monolingual BERT models: one trained and tested entirely on Swahili data, and another on comparable English news data. To simulate how multilingual LLMs process non-English queries through internal translation and abstraction, we translated the Swahili news data into English and evaluated it using the English-trained model. This approach tests the hypothesis by evaluating whether translating Swahili inputs for evaluation on an English model yields better or worse performance compared to training and testing a model entirely in Swahili, thus isolating the effect of language consistency versus cross-lingual abstraction. The results prove that, despite high-quality translation, the native Swahili-trained model performed better than the Swahili-to-English translated model, producing nearly four times fewer errors: 0.36% vs. 1.47% respectively. This gap suggests that translation alone does not bridge representational differences between languages and that models trained in one language may struggle to accurately interpret translated inputs due to imperfect internal knowledge representation, suggesting that native-language training remains important for reliable outcomes. In educational and informational contexts, even small performance gaps may compound inequality. Future research should focus on addressing broader dataset development for underrepresented languages and renewed attention to multilingual model evaluation, ensuring the reinforcing effect of global AI deployment on existing digital divides is reduced.

Method: Jointly prunes rare vocabulary to shrink embedding/LM head layers and prunes FFN intermediate channels using common-token-weighted activations that align importance with post-pruning token distribution.

Result: Achieves state-of-the-art downstream performance across Qwen, LLaMA, and Gemma families (0.5B-70B) with substantial reductions in parameters, GPU memory, and latency.

Conclusion: COMPACT provides deployment-friendly, scale-adaptive pruning that maintains competitive performance while offering significant efficiency gains for both large and small language models.

Abstract: Making large language models (LLMs) more efficient in memory, latency, and serving cost is crucial for edge deployment, interactive applications, and sustainable inference at scale. Pruning is a promising technique, but existing pruning methods are limited: width pruning often breaks the standard transformer layout, requiring custom inference code, while depth pruning can cause abrupt accuracy drops. Also, while many pruning approaches are effective against LLMs, they struggle to maintain performance on small language models (SLMs). In this work, we propose COMPACT, which jointly (i) prunes rare vocabulary to shrink embedding/LM head layers and (ii) prunes FFN intermediate channels using common-token-weighted activations, aligning importance with the post-pruning token distribution. COMPACT inherits strengths of both depth and width pruning, such as: deployment-friendliness (keeps a standard transformer architecture), scale-adaptivity (trade off vocab. vs. FFN pruning), competitive pruning times, and strong memory savings alongside throughput gains. Experiments across Qwen, LLaMA, and Gemma families (0.5B-70B) show state-of-the-art downstream performance, with substantial reductions in parameters, GPU memory, and latency.

Method: Uses Wavelength communication game framework to test LMs on comprehension and production tasks with direct prompting, Chain-of-Thought, and Rational Speech Act approaches.

Result: Large LMs achieve human-like accuracy on comprehension without CoT/RSA; CoT improves production; RSA provides significant improvements over both approaches.

Conclusion: Identifies LM pragmatic reasoning strengths/limitations, demonstrates RSA’s potential for improvement, and opens avenues for understanding conceptual representation and social reasoning.

Abstract: Language use is shaped by pragmatics – i.e., reasoning about communicative goals and norms in context. As language models (LMs) are increasingly used as conversational agents, it becomes ever more important to understand their pragmatic reasoning abilities. We propose an evaluation framework derived from Wavelength, a popular communication game where a speaker and a listener communicate about a broad range of concepts in a granular manner. We study a range of LMs on both language comprehension and language production using direct and Chain-of-Thought (CoT) prompting, and further explore a Rational Speech Act (RSA) approach to incorporating Bayesian pragmatic reasoning into LM inference. We find that state-of-the-art LMs, but not smaller ones, achieve strong performance on language comprehension, obtaining similar-to-human accuracy and exhibiting high correlations with human judgments even without CoT prompting or RSA. On language production, CoT can outperform direct prompting, and using RSA provides significant improvements over both approaches. Our study helps identify the strengths and limitations in LMs’ pragmatic reasoning abilities and demonstrates the potential for improving them with RSA, opening up future avenues for understanding conceptual representation, language understanding, and social reasoning in LMs and humans.

Method: CASTLE continually updates each token’s keys as context unfolds, creating lookahead keys that integrate later token information while maintaining autoregressive property through mathematical equivalence for parallel training.

Result: Consistently outperforms standard causal attention across model scales, reducing validation perplexity and improving performance on downstream tasks in language modeling benchmarks.

Conclusion: CASTLE provides an effective attention mechanism that integrates future context information while preserving autoregressive generation, enabling better language modeling performance with efficient parallel training.

Abstract: In standard causal attention, each token’s query, key, and value (QKV) are static and encode only preceding context. We introduce CAuSal aTtention with Lookahead kEys (CASTLE), an attention mechanism that continually updates each token’s keys as the context unfolds. We term these updated keys lookahead keys because they belong to earlier positions yet integrate information from tokens that appear later relative to those positions, while strictly preserving the autoregressive property. Although the mechanism appears sequential, we derive a mathematical equivalence that avoids explicitly materializing lookahead keys at each position and enables efficient parallel training. On language modeling benchmarks, CASTLE consistently outperforms standard causal attention across model scales, reducing validation perplexity and improving performance on a range of downstream tasks.

Method: Uses decoder-only language model with pre-processed time alignment and introduced delays between streams. Aligns text and audio streams with different delays for ASR (text delayed) and TTS (audio delayed).

Result: Achieves state-of-the-art performance and latency, supports arbitrary long sequences, and is competitive with offline baselines.

Conclusion: DSM provides a flexible framework for streaming multimodal sequence-to-sequence tasks that overcomes limitations of both offline and traditional streaming approaches.

Abstract: We introduce Delayed Streams Modeling (DSM), a flexible formulation for streaming, multimodal sequence-to-sequence learning. Sequence-to-sequence generation is often cast in an offline manner, where the model consumes the complete input sequence before generating the first output timestep. Alternatively, streaming sequence-to-sequence rely on learning a policy for choosing when to advance on the input stream, or write to the output stream. DSM instead models already time-aligned streams with a decoder-only language model. By moving the alignment to a pre-processing step,and introducing appropriate delays between streams, DSM provides streaming inference of arbitrary output sequences, from any input combination, making it applicable to many sequence-to-sequence problems. In particular, given text and audio streams, automatic speech recognition (ASR) corresponds to the text stream being delayed, while the opposite gives a text-to-speech (TTS) model. We perform extensive experiments for these two major sequence-to-sequence tasks, showing that DSM provides state-of-the-art performance and latency while supporting arbitrary long sequences, being even competitive with offline baselines. Code, samples and demos are available at https://github.com/kyutai-labs/delayed-streams-modeling

Method: Experiments with XLM-RoBERTa, Longformer, GPT-3.5, and GPT-4 on multi-class classification of legal documents across 5 languages using the Comparative Agendas Project’s 21 policy topic labels.

Result: Longformer showed no advantage despite being designed for long inputs. Open models performed competitively with GPT variants. Performance was influenced by category support and substance overlaps.

Conclusion: Specialized long-input models don’t necessarily outperform standard models for legal document classification, and open models can achieve competitive results with GPT models.

Abstract: The most widely used large language models in the social sciences (such as BERT, and its derivatives, e.g. RoBERTa) have a limitation on the input text length that they can process to produce predictions. This is a particularly pressing issue for some classification tasks, where the aim is to handle long input texts. One such area deals with laws and draft laws (bills), which can have a length of multiple hundred pages and, therefore, are not particularly amenable for processing with models that can only handle e.g. 512 tokens. In this paper, we show results from experiments covering 5 languages with XLM-RoBERTa, Longformer, GPT-3.5, GPT-4 models for the multiclass classification task of the Comparative Agendas Project, which has a codebook of 21 policy topic labels from education to health care. Results show no particular advantage for the Longformer model, pre-trained specifically for the purposes of handling long inputs. The comparison between the GPT variants and the best-performing open model yielded an edge for the latter. An analysis of class-level factors points to the importance of support and substance overlaps between specific categories when it comes to performance on long text inputs.

Method: Used multi-stage filtering of 3.01M web-crawled questions with three language models, followed by GPT-4o refinement and human validation to create 441 high-quality Korean question-answer pairs with cultural elements.

Result: Models achieved between 59.86% to 83.22% accuracy on Ko-PIQA, with significant struggles on culturally specific scenarios, showing room for improvement in cultural reasoning.

Conclusion: Ko-PIQA serves as a benchmark for Korean language models and foundation for more inclusive commonsense reasoning research, highlighting the importance of culturally diverse datasets.

Abstract: Physical commonsense reasoning datasets like PIQA are predominantly English-centric and lack cultural diversity. We introduce Ko-PIQA, a Korean physical commonsense reasoning dataset that incorporates cultural context. Starting from 3.01 million web-crawled questions, we employed a multi-stage filtering approach using three language models to identify 11,553 PIQA-style questions. Through GPT-4o refinement and human validation, we obtained 441 high-quality question-answer pairs. A key feature of Ko-PIQA is its cultural grounding: 19.7% of questions contain culturally specific elements like traditional Korean foods (kimchi), clothing (hanbok), and specialized appliances (kimchi refrigerators) that require culturally-aware reasoning beyond direct translation. We evaluate seven language models on Ko-PIQA, with the best model achieving 83.22% accuracy while the weakest reaches only 59.86%, demonstrating significant room for improvement. Models particularly struggle with culturally specific scenarios, highlighting the importance of culturally diverse datasets. Ko-PIQA serves as both a benchmark for Korean language models and a foundation for more inclusive commonsense reasoning research. The dataset and code will be publicly available.

Method: A dual-agent framework with a planner that iteratively interleaves evidence acquisition with outline optimization to create a citation-grounded outline, and a writer that performs hierarchical retrieval and writing section by section using targeted retrieval from a memory bank.

Result: Establishes new state-of-the-art across major OEDR benchmarks including DeepResearch Bench, DeepConsult, and DeepResearchGym, demonstrating improved performance in producing comprehensive and trusted reports.

Conclusion: The human-centric, iterative methodology with adaptive planning and focused synthesis is crucial for producing comprehensive, trusted, and well-structured reports in open-ended deep research tasks.

Abstract: This paper tackles \textbf{open-ended deep research (OEDR)}, a complex challenge where AI agents must synthesize vast web-scale information into insightful reports. Current approaches are plagued by dual-fold limitations: static research pipelines that decouple planning from evidence acquisition and monolithic generation paradigms that include redundant, irrelevant evidence, suffering from hallucination issues and low citation accuracy. To address these challenges, we introduce \textbf{WebWeaver}, a novel dual-agent framework that emulates the human research process. The planner operates in a dynamic cycle, iteratively interleaving evidence acquisition with outline optimization to produce a comprehensive, citation-grounded outline linking to a memory bank of evidence. The writer then executes a hierarchical retrieval and writing process, composing the report section by section. By performing targeted retrieval of only the necessary evidence from the memory bank via citations for each part, it effectively mitigates long-context issues and citation hallucinations. Our framework establishes a new state-of-the-art across major OEDR benchmarks, including DeepResearch Bench, DeepConsult, and DeepResearchGym. These results validate our human-centric, iterative methodology, demonstrating that adaptive planning and focused synthesis are crucial for producing comprehensive, trusted, and well-structured reports.

Method: Evaluated popular verbalization methods across prior datasets and conducted controlled experiments to test if verbalizations reflect target model knowledge vs. verbalizer LLM’s parametric knowledge.

Result: Verbalization methods succeeded at benchmarks without target model access, and verbalizations often reflected the verbalizer LLM’s knowledge rather than the target model’s knowledge.

Conclusion: Targeted benchmarks and experimental controls are needed to rigorously assess whether verbalization methods provide meaningful insights into LLM operations.

Abstract: Recent interpretability methods have proposed to translate LLM internal representations into natural language descriptions using a second verbalizer LLM. This is intended to illuminate how the target model represents and operates on inputs. But do such activation verbalization approaches actually provide privileged knowledge about the internal workings of the target model, or do they merely convey information about its inputs? We critically evaluate popular verbalization methods across datasets used in prior work and find that they can succeed at benchmarks without any access to target model internals, suggesting that these datasets may not be ideal for evaluating verbalization methods. We then run controlled experiments which reveal that verbalizations often reflect the parametric knowledge of the verbalizer LLM which generated them, rather than the knowledge of the target LLM whose activations are decoded. Taken together, our results indicate a need for targeted benchmarks and experimental controls to rigorously assess whether verbalization methods provide meaningful insights into the operations of LLMs.

Method: Systematic review of LLM applications to thematic analysis, complemented by clinician interview.

Result: Found fragmented approaches across analysis types, datasets, prompting strategies, and models, with inconsistent evaluation methods.

Conclusion: Proposed evaluation framework with three dimensions (validity, reliability, interpretability) to standardize practices and advance the field.

Abstract: This position paper examines how large language models (LLMs) can support thematic analysis of unstructured clinical transcripts, a widely used but resource-intensive method for uncovering patterns in patient and provider narratives. We conducted a systematic review of recent studies applying LLMs to thematic analysis, complemented by an interview with a practicing clinician. Our findings reveal that current approaches remain fragmented across multiple dimensions including types of thematic analysis, datasets, prompting strategies and models used, most notably in evaluation. Existing evaluation methods vary widely (from qualitative expert review to automatic similarity metrics), hindering progress and preventing meaningful benchmarking across studies. We argue that establishing standardized evaluation practices is critical for advancing the field. To this end, we propose an evaluation framework centered on three dimensions: validity, reliability, and interpretability.

Method: ATTS uses an asynchronous inference approach with online calibration and a three-stage rejection sampling pipeline based on ordinal classification. It identifies synchronization as the primary bottleneck and enables scaling along both sequential and parallel axes while maintaining statistical guarantees through hypothesis testing.

Result: ATTS achieves up to 56.7x speedup in test-time scaling and 4.14x throughput improvement across MATH, AMC23, AIME24, and AIME25 datasets. It enables 1.5B/70B draft/target models to match state-of-the-art reasoning model performance on AIME, while reducing latency and memory overhead with accurate rejection rate control.

Conclusion: ATTS successfully addresses the synchronization bottleneck in test-time scaling, enabling efficient parallel and sequential scaling of LLMs while maintaining accuracy and statistical guarantees, making high-performance reasoning models more accessible.

Abstract: Large language models (LLMs) benefit from test-time scaling but are often hampered by high inference latency. Speculative decoding is a natural way to accelerate the scaling process; however, scaling along both the parallel and sequential dimensions poses significant challenges, including substantial memory-bound execution and synchronization overhead. We introduce ATTS (Asynchronous Test-Time Scaling), a statistically guaranteed adaptive scaling framework that follows the hypothesis testing process to address these challenges. By revisiting arithmetic intensity, ATTS identifies synchronization as the primary bottleneck. It enables asynchronous inference through online calibration and proposes an ordinal classification algorithm that supports a three-stage rejection sampling pipeline, scaling along both the sequential and parallel axes. Across experiments on the MATH, AMC23, AIME24, and AIME25 datasets and across multiple draft-target model families, we show that ATTS delivers up to 56.7x speedup in test-time scaling and a 4.14x throughput improvement, while maintaining accurate control of the rejection rate, reducing latency and memory overhead, and incurring no accuracy loss. By scaling both in parallel and sequential dimensions, we enable the 1.5B/70B draft/target model combination to achieve the performance of the state-of-the-art reasoning model o3-mini (high) on the AIME dataset. We have released the code at https://github.com/menik1126/asynchronous-test-time-scaling.

Method: Proposes Convolutional decoding (Conv) using normalization to narrow decoding windows without hard segmentation, and Rejecting Rule-based Fine-Tuning (R2FT) for post-hoc training to better align tokens far from context.

Result: Achieves state-of-the-art results on open-ended generation benchmarks (e.g., AlpacaEval) among diffusion LM baselines, with significantly lower step size than previous works, demonstrating both speed and quality improvements.

Conclusion: The proposed methods effectively overcome the long decoding-window bottleneck in diffusion language models, enabling faster inference while maintaining generation quality through improved fluency and flexibility.

Abstract: Autoregressive (AR) language models generate text one token at a time, which limits their inference speed. Diffusion-based language models offer a promising alternative, as they can decode multiple tokens in parallel. However, we identify a key bottleneck in current diffusion LMs: the long decoding-window problem, where tokens generated far from the input context often become irrelevant or repetitive. Previous solutions like semi-autoregressive address this issue by splitting windows into blocks, but this sacrifices speed and bidirectionality, eliminating the main advantage of diffusion models. To overcome this, we propose Convolutional decoding (Conv), a normalization-based method that narrows the decoding window without hard segmentation, leading to better fluency and flexibility. Additionally, we introduce Rejecting Rule-based Fine-Tuning (R2FT), a post-hoc training scheme that better aligns tokens at positions far from context. Our methods achieve state-of-the-art results on open-ended generation benchmarks (e.g., AlpacaEval) among diffusion LM baselines, with significantly lower step size than previous works, demonstrating both speed and quality improvements.

Method: Proposes knowledge decoupling mechanism to separate reasoning into task-specific and task-agnostic stages, dual-perspective memory consolidation, and structure-guided pseudo-data synthesis.

Result: Extensive experiments on four benchmark datasets show superiority over existing continual learning methods across multiple metrics using various backbone LLMs.

Conclusion: K-DeCore effectively addresses limitations of existing continual learning approaches for structured knowledge reasoning with fixed parameters and improved generalization.

Abstract: Continual Structured Knowledge Reasoning (CSKR) focuses on training models to handle sequential tasks, where each task involves translating natural language questions into structured queries grounded in structured knowledge. Existing general continual learning approaches face significant challenges when applied to this task, including poor generalization to heterogeneous structured knowledge and inefficient reasoning due to parameter growth as tasks increase. To address these limitations, we propose a novel CSKR framework, \textsc{K-DeCore}, which operates with a fixed number of tunable parameters. Unlike prior methods, \textsc{K-DeCore} introduces a knowledge decoupling mechanism that disentangles the reasoning process into task-specific and task-agnostic stages, effectively bridging the gaps across diverse tasks. Building on this foundation, \textsc{K-DeCore} integrates a dual-perspective memory consolidation mechanism for distinct stages and introduces a structure-guided pseudo-data synthesis strategy to further enhance the model’s generalization capabilities. Extensive experiments on four benchmark datasets demonstrate the superiority of \textsc{K-DeCore} over existing continual learning methods across multiple metrics, leveraging various backbone large language models.

Method: Two-stage generation and verification pipeline to create culturally-authentic problems from Korean contexts across 9 reasoning subtasks and 84 scenarios, avoiding simple translation approaches.

Result: Korean-specialized models consistently outperform general-purpose models of comparable size, revealing performance gaps that highlight limitations of culturally-agnostic models.

Conclusion: Culturally-aware benchmarks like EPiK are critically needed to truly measure language understanding, as they reveal the limitations of culturally-agnostic AI models.

Abstract: Existing physical commonsense reasoning benchmarks predominantly focus on Western contexts, overlooking cultural variations in physical problem-solving. To address this gap, we introduce EPiK (Everyday Physics in Korean Contexts), a novel benchmark comprising 181 binary-choice problems that test physical reasoning within Korean cultural contexts, ranging from kimchi (Korean food) to traditional fermentation. EPiK is constructed using a two-stage generation and verification pipeline to create culturally-authentic problems across 9 reasoning subtasks and 84 scenarios. Unlike approaches based on simple translation, our method generates problems organically from Korean contexts while upholding rigorous physical reasoning standards. Our evaluations show that Korean-specialized models consistently outperform general-purpose models of comparable size. This performance gap highlights the limitations of culturally-agnostic models and demonstrates the critical need for culturally-aware benchmarks to truly measure language understanding. Our EPiK is publicly available at https://huggingface.co/datasets/jjae/EPiK.

Method: Uses token priors from corpus-level term frequency statistics to filter documents based on mean and standard deviation of token priors, requiring no model inference.

Result: Achieves highest average performance across 20 downstream benchmarks while reducing time cost by over 1000x compared to PPL-based filtering.

Conclusion: Prior-based filtering is a simple, fast, and effective alternative to PPL-based filtering that works across languages and symbolic domains like code and math.

Abstract: As large language models (LLMs) are pretrained on massive web corpora, careful selection of data becomes essential to ensure effective and efficient learning. While perplexity (PPL)-based filtering has shown strong performance, it suffers from drawbacks: substantial time costs and inherent unreliability of the model when handling noisy or out-of-distribution samples. In this work, we propose a simple yet powerful alternative: a prior-based data filtering method that estimates token priors using corpus-level term frequency statistics, inspired by linguistic insights on word roles and lexical density. Our approach filters documents based on the mean and standard deviation of token priors, serving as a fast proxy to PPL while requiring no model inference. Despite its simplicity, the prior-based filter achieves the highest average performance across 20 downstream benchmarks, while reducing time cost by over 1000x compared to PPL-based filtering. We further demonstrate its applicability to symbolic languages such as code and math, and its dynamic adaptability to multilingual corpora without supervision

Method: Uses surprisal-based features to capture how unpredictability fluctuates across text, leveraging the observation that human-authored text exhibits richer variability in lexical and structural unpredictability than LLM outputs.

Result: Outperforms existing zero-shot detectors by up to 33.2%, achieves competitive performance with fine-tuned baselines, robust to paraphrasing and adversarial attacks, generalizes well across domains and models, and improves existing detectors by up to 18.7% when used as auxiliary signal.

Conclusion: DivEye provides an effective and interpretable approach for AI-generated text detection, with rhythmic unpredictability identified as a powerful underexplored signal for distinguishing human from LLM-generated content.

Abstract: Detecting AI-generated text is an increasing necessity to combat misuse of LLMs in education, business compliance, journalism, and social media, where synthetic fluency can mask misinformation or deception. While prior detectors often rely on token-level likelihoods or opaque black-box classifiers, these approaches struggle against high-quality generations and offer little interpretability. In this work, we propose DivEye, a novel detection framework that captures how unpredictability fluctuates across a text using surprisal-based features. Motivated by the observation that human-authored text exhibits richer variability in lexical and structural unpredictability than LLM outputs, DivEye captures this signal through a set of interpretable statistical features. Our method outperforms existing zero-shot detectors by up to 33.2% and achieves competitive performance with fine-tuned baselines across multiple benchmarks. DivEye is robust to paraphrasing and adversarial attacks, generalizes well across domains and models, and improves the performance of existing detectors by up to 18.7% when used as an auxiliary signal. Beyond detection, DivEye provides interpretable insights into why a text is flagged, pointing to rhythmic unpredictability as a powerful and underexplored signal for LLM detection.

Method: Alternating optimization of implicit process reward model (PRM) with policy model using trajectory-based DPO objective to generate implicit step rewards. Combines step-level and episode-level advantages for policy updates in a self-reinforcing loop.

Result: State-of-the-art performance on WebShop, VisualSokoban, and SOTOPIA benchmarks, outperforming frontier LLMs and strong RL baselines. Higher sample efficiency, training stability, and efficient exploration with fewer steps to task success.

Conclusion: iStar provides an effective credit assignment strategy for agentic RL that integrates seamlessly with standard algorithms, demonstrating superior performance across domains with improved efficiency and stability.

Abstract: Large language models (LLMs) are increasingly developed as autonomous agents using reinforcement learning (agentic RL) that reason and act in interactive environments. However, sparse and sometimes unverifiable rewards make it extremely challenging to assign credit when training LLM agents that serve as a policy. Recent work attempts to integrate process supervision into RL but suffers from biased annotation, reward hacking, high-variance from overly fine-grained rewards or failtures when state overlap is rare. We therefore introduce implicit step rewards for agentic RL (iStar), a general credit-assignment strategy that integrates seamlessly with standard RL algorithms without relying on additional rollouts or explicit step labels. Particularly, we alternatively optimize an implicit process reward model (PRM) with the policy model to generate implicit step rewards via a trajectory-based DPO objective. Theoretical analysis shows that this learning objective produces a step-wise reward function. Then the implicit step rewards are used to compute step-level advantages, which are combined with trajectory (or episode)-level advantages for policy updates, creating a self-reinforcing training loop. We evaluate our method on three challenging agent benchmarks, including WebShop and VisualSokoban, as well as open-ended social interactions with unverifiable rewards in SOTOPIA. Crucially, iStar shows superior performance over frontier LLMs and strong RL baselines across domains, achieving state-of-the-art results with higher sample-efficiency and training stability. Further analysis also demonstrates efficient exploration by iStar with increased rewards in both step- and episode-level while maintaining fewer steps to achieve task success. Code will be available soon.

Method: The paper offers a systematic review and overview approach, examining LLM integration across three major disciplinary categories: (1) arts, letters, and law; (2) economics and business; and (3) science and engineering, while discussing key observations and insights.

Result: The review provides insights into how LLMs are engaged across disciplines, highlighting their applications in diverse fields while identifying key limitations and open challenges in the generative AI era.

Conclusion: The comprehensive review of LLM applications across disciplines can help researchers and practitioners interested in exploiting LLMs to advance their work in diverse real-world applications, while emphasizing the need to address limitations and future directions in this rapidly evolving field.

Abstract: Cutting-edge Artificial Intelligence (AI) techniques keep reshaping our view of the world. For example, Large Language Models (LLMs) based applications such as ChatGPT have shown the capability of generating human-like conversation on extensive topics. Due to the impressive performance on a variety of language-related tasks (e.g., open-domain question answering, translation, and document summarization), one can envision the far-reaching impacts that can be brought by the LLMs with broader real-world applications (e.g., customer service, education and accessibility, and scientific discovery). Inspired by their success, this paper will offer an overview of state-of-the-art LLMs and their integration into a wide range of academic disciplines, including: (1) arts, letters, and law (e.g., history, philosophy, political science, arts and architecture, law), (2) economics and business (e.g., finance, economics, accounting, marketing), and (3) science and engineering (e.g., mathematics, physics and mechanical engineering, chemistry and chemical engineering, life sciences and bioengineering, earth sciences and civil engineering, computer science and electrical engineering). Integrating humanity and technology, in this paper, we will explore how LLMs are shaping research and practice in these fields, while also discussing key limitations, open challenges, and future directions in the era of generative AI. The review of how LLMs are engaged across disciplines-along with key observations and insights-can help researchers and practitioners interested in exploiting LLMs to advance their works in diverse real-world applications.

Method: Established a unique RAI assessment methodology based on KT’s AI risk taxonomy, systematically identifying and managing risk factors from AI development to operation, with practical tools for risk management and mitigation.

Result: Developed proprietary Guardrail: SafetyGuard that blocks harmful AI responses in real-time, providing a reliable assessment methodology to verify model safety and robustness tailored to the domestic environment.

Conclusion: The research outcomes provide valuable insights for organizations developing Responsible AI and support the enhancement of safety in the domestic AI development ecosystem through systematic risk management and real-time protection technologies.

Abstract: KT developed a Responsible AI (RAI) assessment methodology and risk mitigation technologies to ensure the safety and reliability of AI services. By analyzing the Basic Act on AI implementation and global AI governance trends, we established a unique approach for regulatory compliance and systematically identify and manage all potential risk factors from AI development to operation. We present a reliable assessment methodology that systematically verifies model safety and robustness based on KT’s AI risk taxonomy tailored to the domestic environment. We also provide practical tools for managing and mitigating identified AI risks. With the release of this report, we also release proprietary Guardrail : SafetyGuard that blocks harmful responses from AI models in real-time, supporting the enhancement of safety in the domestic AI development ecosystem. We also believe these research outcomes provide valuable insights for organizations seeking to develop Responsible AI.

Method: Reinforcement Learning from Augmented Generation (RLAG) iteratively cycles between sampling generations and optimizing models through calculated rewards. It selects high-probability outputs and uses three tailored reward metrics to embed critical, contextually coherent domain knowledge.

Result: Experimental results across medical, legal, astronomy, and current events datasets show RLAG significantly outperforms baseline approaches in domain expertise, assessed through answer accuracy and explanation rationality.

Conclusion: RLAG effectively addresses knowledge gaps in LLMs for domain applications through iterative reinforcement learning with augmented generation, demonstrating superior performance over existing methods.

Abstract: Large language models (LLMs) often exhibit limited performance on domain-specific tasks due to the natural disproportionate representation of specialized information in their training data and the static nature of these datasets. Knowledge scarcity and temporal lag create knowledge gaps for domain applications. While post-training on domain datasets can embed knowledge into models, existing approaches have some limitations. Continual Pre-Training (CPT) treats all tokens in domain documents with equal importance, failing to prioritize critical knowledge points, while supervised fine-tuning (SFT) with question-answer pairs struggles to develop the coherent knowledge structures necessary for complex reasoning tasks. To address these challenges, we propose Reinforcement Learning from Augmented Generation (RLAG). Our approach iteratively cycles between sampling generations and optimizing the model through calculated rewards, effectively embedding critical and contextually coherent domain knowledge. We select generated outputs with the highest log probabilities as the sampling result, then compute three tailored reward metrics to guide the optimization process. To comprehensively evaluate domain expertise, we assess answer accuracy and the rationality of explanations generated for correctly answered questions. Experimental results across medical, legal, astronomy, and current events datasets demonstrate that our proposed method significantly outperforms baseline approaches. Our code and data are open sourced at https://github.com/ChaojunNie/RLAG.

Method: Uses encoder-decoder initialization and geometric embedding distillation to capture knowledge from larger models, incorporates spread-out regularizer for robustness and expressiveness, and merges checkpoints from varied optimized mixtures for generalizability.

Result: Achieves state-of-the-art results on MTEB benchmark across multilingual, English, and code domains, outperforms prior top models with fewer than 500M parameters, provides performance comparable to models double its size, and maintains lead even with quantization or embedding truncation.

Conclusion: EmbeddingGemma offers exceptional efficiency and performance, making it particularly suitable for low-latency, high-throughput applications like on-device use, while being released as open-source to advance research in lightweight embedding models.

Abstract: We introduce EmbeddingGemma, a new lightweight, open text embedding model based on the Gemma 3 language model family. Our innovative training recipe strategically captures knowledge from larger models via encoder-decoder initialization and geometric embedding distillation. We improve model robustness and expressiveness with a spread-out regularizer, and ensure generalizability by merging checkpoints from varied, optimized mixtures. Evaluated on the Massive Text Embedding Benchmark (MTEB) across multilingual, English, and code domains, EmbeddingGemma (300M) achieves state-of-the-art results. Notably, it outperforms prior top models, both proprietary and open, with fewer than 500M parameters, and provides performance comparable to models double its size, offering an exceptional performance-to-cost ratio. Remarkably, this lead persists when quantizing model weights or truncating embedding outputs. This makes EmbeddingGemma particularly well-suited for low-latency and high-throughput use cases such as on-device applications. We provide ablation studies exploring our key design choices. We release EmbeddingGemma to the community to promote further research.

Method: The authors introduce a method for measuring duplication rates and evaluate different diffusion models using both automatic evaluation with Vision-Language Models (VLM) and human evaluation. They also investigate prompt expansion as a technique to mitigate homonym duplication.

Result: The research demonstrates that prompt expansion effectively reduces duplication rates related to both homonym duplication and Anglocentric bias. The automatic evaluation pipeline code is made publicly available.

Conclusion: Homonym duplication is a significant issue in diffusion models that can be effectively measured and mitigated through prompt expansion techniques. The proposed methods address both direct homonym problems and those arising from Anglocentric bias in translation pipelines.

Abstract: Homonyms are words with identical spelling but distinct meanings, which pose challenges for many generative models. When a homonym appears in a prompt, diffusion models may generate multiple senses of the word simultaneously, which is known as homonym duplication. This issue is further complicated by an Anglocentric bias, which includes an additional translation step before the text-to-image model pipeline. As a result, even words that are not homonymous in the original language may become homonyms and lose their meaning after translation into English. In this paper, we introduce a method for measuring duplication rates and conduct evaluations of different diffusion models using both automatic evaluation utilizing Vision-Language Models (VLM) and human evaluation. Additionally, we investigate methods to mitigate the homonym duplication problem through prompt expansion, demonstrating that this approach also effectively reduces duplication related to Anglocentric bias. The code for the automatic evaluation pipeline is publicly available.

Method: Adds a pooled classification head for training/inference and a reasoning head supervised by teacher rationales used only in training, with weighted loss combining label cross-entropy and token-level LM loss over input-plus-rationale sequences.

Result: Achieves 0.65-5.47% relative gains over pooled baselines on seven SuperGLUE tasks, with larger gains on entailment/causal tasks. Inference throughput matches pooled classifiers and exceeds CoT decoding by 96-142 times in QPS.

Conclusion: DHRD successfully decouples reasoning benefits from inference costs, providing accuracy improvements without throughput penalty by disabling the reasoning head at test time.

Abstract: Chain-of-Thought (CoT) prompting often improves classification accuracy, but it introduces a significant throughput penalty with rationale generation (Wei et al., 2022; Cheng and Van Durme, 2024). To resolve this trade-off, we introduce Dual-Head Reasoning Distillation (DHRD), a simple training method for decoder-only language models (LMs) that adds (i) a pooled classification head used during training and inference and (ii) a reasoning head supervised by teacher rationales used only in training. We train with a loss function that is a weighted sum of label cross-entropy and token-level LM loss over input-plus-rationale sequences. On seven SuperGLUE tasks, DHRD yields relative gains of 0.65-5.47% over pooled baselines, with notably larger gains on entailment/causal tasks. Since we disable the reasoning head at test time, inference throughput matches pooled classifiers and exceeds CoT decoding on the same backbones by 96-142 times in QPS.

Method: Built chatbot using sentence embedding model with Kisan Call Center dataset. Incorporated entity extraction and eliminated synonyms to improve performance.

Result: Initial accuracy of 56% with sentence embedding model. After improvements, accuracy increased to 86%. System handles weather, market rates, plant protection, and government schemes.

Conclusion: The chatbot enables easier access to farming information, improves agricultural output, and reduces workload on call center staff by providing 24/7 automated assistance.

Abstract: India is an agro-based economy and proper information about agricultural practices is the key to optimal agricultural growth and output. In order to answer the queries of the farmer, we have build an agricultural chatbot based on the dataset from Kisan Call Center. This system is robust enough to answer queries related to weather, market rates, plant protection and government schemes. This system is available 24* 7, can be accessed through any electronic device and the information is delivered with the ease of understanding. The system is based on a sentence embedding model which gives an accuracy of 56%. After eliminating synonyms and incorporating entity extraction, the accuracy jumps to 86%. With such a system, farmers can progress towards easier information about farming related practices and hence a better agricultural output. The job of the Call Center workforce would be made easier and the hard work of various such workers can be redirected to a better goal.

Method: Developed TACOS taxonomy with 21 classes covering clinical/non-clinical queries, modeling safety thresholds and tool dependencies. Created TACOS-annotated dataset and conducted experiments to validate the taxonomy.

Result: The taxonomy proved valuable for clinical agent settings, with experiments revealing insights about training data distribution and base models’ pretrained knowledge.

Conclusion: TACOS provides a comprehensive safety framework for clinical chatbots that outperforms existing methods by integrating safety filtering and tool selection into a unified intent classification system.

Abstract: Safety is a paramount concern in clinical chatbot applications, where inaccurate or harmful responses can lead to serious consequences. Existing methods–such as guardrails and tool calling–often fall short in addressing the nuanced demands of the clinical domain. In this paper, we introduce TACOS (TAxonomy of COmprehensive Safety for Clinical Agents), a fine-grained, 21-class taxonomy that integrates safety filtering and tool selection into a single user intent classification step. TACOS is a taxonomy that can cover a wide spectrum of clinical and non-clinical queries, explicitly modeling varying safety thresholds and external tool dependencies. To validate our taxonomy, we curate a TACOS-annotated dataset and perform extensive experiments. Our results demonstrate the value of a new taxonomy specialized for clinical agent settings, and reveal useful insights about train data distribution and pretrained knowledge of base models.

Method: Restore fine-tuning to standard breadth-first (epoch-based) pipeline with mini-batch optimization, and develop LocFT-BF method through systematic analysis of tuning locations for localized editing.

Result: LocFT-BF outperforms state-of-the-art methods by large margins, sustains 100K edits and 72B-parameter models (10x beyond prior practice) without sacrificing general capabilities.

Conclusion: Fine-tuning can be advanced from an underestimated baseline to a leading method for model editing by correcting pipeline misconceptions and implementing principled localized tuning strategies.

Abstract: Fine-tuning, a foundational method for adapting large language models, has long been considered ineffective for model editing. Here, we challenge this belief, arguing that the reported failure arises not from the inherent limitation of fine-tuning itself, but from adapting it to the sequential nature of the editing task, a single-pass depth-first pipeline that optimizes each sample to convergence before moving on. While intuitive, this depth-first pipeline coupled with sample-wise updating over-optimizes each edit and induces interference across edits. Our controlled experiments reveal that simply restoring fine-tuning to the standard breadth-first (i.e., epoch-based) pipeline with mini-batch optimization substantially improves its effectiveness for model editing. Moreover, fine-tuning in editing also suffers from suboptimal tuning parameter locations inherited from prior methods. Through systematic analysis of tuning locations, we derive LocFT-BF, a simple and effective localized editing method built on the restored fine-tuning framework. Extensive experiments across diverse LLMs and datasets demonstrate that LocFT-BF outperforms state-of-the-art methods by large margins. Notably, to our knowledge, it is the first to sustain 100K edits and 72B-parameter models,10 x beyond prior practice, without sacrificing general capabilities. By clarifying a long-standing misconception and introducing a principled localized tuning strategy, we advance fine-tuning from an underestimated baseline to a leading method for model editing, establishing a solid foundation for future research.

Method: Compress high-level reasoning plan into small set of learned latent tokens (Reasoning Capsule) with dual objectives: primary task loss for accuracy and auxiliary plan-reconstruction loss for grounding.

Result: Reduces visible token footprint while maintaining or improving accuracy on complex benchmarks through balanced efficiency, accuracy, and interpretability.

Conclusion: R-Capsule successfully combines efficiency of latent reasoning with transparency of explicit CoT, striking optimal balance for complex reasoning tasks.

Abstract: Chain-of-Thought (CoT) prompting helps Large Language Models (LLMs) tackle complex reasoning by eliciting explicit step-by-step rationales. However, CoT’s verbosity increases latency and memory usage and may propagate early errors across long chains. We propose the Reasoning Capsule (R-Capsule), a framework that aims to combine the efficiency of latent reasoning with the transparency of explicit CoT. The core idea is to compress the high-level plan into a small set of learned latent tokens (a Reasoning Capsule) while keeping execution steps lightweight or explicit. This hybrid approach is inspired by the Information Bottleneck (IB) principle, where we encourage the capsule to be approximately minimal yet sufficient for the task. Minimality is encouraged via a low-capacity bottleneck, which helps improve efficiency. Sufficiency is encouraged via a dual objective: a primary task loss for answer accuracy and an auxiliary plan-reconstruction loss that encourages the capsule to faithfully represent the original textual plan. The reconstruction objective helps ground the latent space, thereby improving interpretability and reducing the use of uninformative shortcuts. Our framework strikes a balance between efficiency, accuracy, and interpretability, thereby reducing the visible token footprint of reasoning while maintaining or improving accuracy on complex benchmarks. Our codes are available at: https://anonymous.4open.science/r/Reasoning-Capsule-7BE0

Method: Created a challenging human-annotated benchmark of 1,000+ examples tailored for LLMs, proposed a new free-form textual representation for hallucinations, and evaluated four large-scale LLMs with various prompting strategies.

Result: The benchmark proved difficult, with the best model achieving only 0.67 F1 score. Key findings show LLMs tend to incorrectly flag missing details as inconsistent and struggle with factually correct but unverifiable information.

Conclusion: LLMs face significant challenges in hallucination localization, particularly with distinguishing between actual inconsistencies and missing details, and handling information that aligns with their parametric knowledge but isn’t verifiable from the source.

Abstract: Context-grounded hallucinations are cases where model outputs contain information not verifiable against the source text. We study the applicability of LLMs for localizing such hallucinations, as a more practical alternative to existing complex evaluation pipelines. In the absence of established benchmarks for meta-evaluation of hallucinations localization, we construct one tailored to LLMs, involving a challenging human annotation of over 1,000 examples. We complement the benchmark with an LLM-based evaluation protocol, verifying its quality in a human evaluation. Since existing representations of hallucinations limit the types of errors that can be expressed, we propose a new representation based on free-form textual descriptions, capturing the full range of possible errors. We conduct a comprehensive study, evaluating four large-scale LLMs, which highlights the benchmark’s difficulty, as the best model achieves an F1 score of only 0.67. Through careful analysis, we offer insights into optimal prompting strategies for the task and identify the main factors that make it challenging for LLMs: (1) a tendency to incorrectly flag missing details as inconsistent, despite being instructed to check only facts in the output; and (2) difficulty with outputs containing factually correct information absent from the source - and thus not verifiable - due to alignment with the model’s parametric knowledge.

Method: Introduces MATS (Multimodal Audit for Truthful Spatialization) with two metrics: Spatial Consistency Score (SCS) and Incorrect Agreement Rate (IAR). Uses activation patching to causally localize failure loci in model architectures.

Result: Instruction-tuned generative VLMs (LLaVA 1.5, QwenVLchat) show very low SCS and high IAR, while contrastive encoders (CLIP, SigLIP) are far more robust. Failure loci identified in mid-to-late cross attention for generative models and pooled projection components for contrastive models.

Conclusion: The audit reveals systematic failures in current VLMs and provides concrete repair paths through causal localization of failure points in model architectures.

Abstract: Vision Language Models (VLMs) are improving quickly, but standard benchmarks can hide systematic failures that reduce real world trust. We introduce MATS (Multimodal Audit for Truthful Spatialization), a compact behavioral audit that measures whether models reject visually contradicted statements, and two metrics Spatial Consistency Score (SCS) and Incorrect Agreement Rate (IAR). Instruction tuned generative VLMs (LLaVA 1.5, QwenVLchat) exhibit very low SCS and high IAR, while contrastive encoders (CLIP, SigLIP) are far more robust. Activation patching causally localizes failure loci (mid to late cross attention for generative models, pooled projection components for contrastive models) and suggests concrete repair paths.

Method: Integrates scale and rotation estimation using Fourier transform in log-polar coordinates with cross-correlation maximization strategy, leveraging the auxiliary function method and incorporating sub-pixel-level cross-correlation.

Result: Experimental results show lower mean estimation errors for scale and rotation compared to conventional Fourier transform-based techniques that rely on discrete cross-correlation.

Conclusion: The proposed method enables precise estimation of both scale and rotation with sub-pixel precision, outperforming existing Fourier-based approaches.

Abstract: This paper introduces a highly efficient algorithm capable of jointly estimating scale and rotation between two images with sub-pixel precision. Image alignment serves as a critical process for spatially registering images captured from different viewpoints, and finds extensive use in domains such as medical imaging and computer vision. Traditional phase-correlation techniques are effective in determining translational shifts; however, they are inadequate when addressing scale and rotation changes, which often arise due to camera zooming or rotational movements. In this paper, we propose a novel algorithm that integrates scale and rotation estimation based on the Fourier transform in log-polar coordinates with a cross-correlation maximization strategy, leveraging the auxiliary function method. By incorporating sub-pixel-level cross-correlation our method enables precise estimation of both scale and rotation. Experimental results demonstrate that the proposed method achieves lower mean estimation errors for scale and rotation than conventional Fourier transform-based techniques that rely on discrete cross-correlation.

Method: Augmented Group Relative Policy Optimization (GRPO) with curriculum-based data scheduling and difficulty-aware filtering to stabilize optimization under sparse, noisy rewards.

Result: Substantial improvements in detection accuracy and robustness on autonomous driving benchmarks, with ablation studies confirming the importance of reward design, KL regularization, and curriculum pacing.

Conclusion: Reinforcement-driven optimization with structured data curricula provides a scalable path toward robust and interpretable multimodal detection.

Abstract: Multimodal Large Language Models (MLLMs) excel in vision-language reasoning but often struggle with structured perception tasks requiring precise localization and robustness. We propose a reinforcement learning framework that augments Group Relative Policy Optimization (GRPO) with curriculum-based data scheduling and difficulty-aware filtering. This approach stabilizes optimization under sparse, noisy rewards and enables progressive adaptation to complex samples. Evaluations on autonomous driving benchmarks demonstrate substantial improvements in detection accuracy and robustness. Ablation studies confirm the importance of reward design, KL regularization, and curriculum pacing for convergence stability and generalization. Our findings highlight reinforcement-driven optimization with structured data curricula as a scalable path toward robust and interpretable multimodal detection.

Method: Proposes Topology Graph Consistency (TGC) framework that integrates graph-theoretic constraints by aligning Laplacian spectra, component counts, and adjacency statistics between prediction graphs and references.

Result: TGC achieves state-of-the-art performance on GlaS and CRAG datasets under 5-10% supervision and significantly narrows the gap to full supervision.

Conclusion: The proposed TGC framework effectively enforces global topology in semi-supervised semantic segmentation, demonstrating superior performance in computational pathology applications.

Abstract: Semi-supervised semantic segmentation (SSSS) is vital in computational pathology, where dense annotations are costly and limited. Existing methods often rely on pixel-level consistency, which propagates noisy pseudo-labels and produces fragmented or topologically invalid masks. We propose Topology Graph Consistency (TGC), a framework that integrates graph-theoretic constraints by aligning Laplacian spectra, component counts, and adjacency statistics between prediction graphs and references. This enforces global topology and improves segmentation accuracy. Experiments on GlaS and CRAG demonstrate that TGC achieves state-of-the-art performance under 5-10% supervision and significantly narrows the gap to full supervision. Code is available at https://github.com/hieuphamha19/TGC.

Method: The survey employs a dual-focus approach: (1) reviewing and categorizing significant publications in supervised person ReID to assess current state-of-the-art, and (2) exploring latest advancements in unsupervised person ReID over the past three years to identify emerging trends.

Result: The survey finds that supervised approaches have little room for further improvement, while unsupervised techniques have shown promising developments in recent years with a narrowing performance gap between supervised and unsupervised paradigms.

Conclusion: The paper contributes to understanding both the mature landscape of supervised person ReID techniques and the promising outcomes in unsupervised learning, highlighting the potential convergence of performance between these two paradigms in person re-identification.

Abstract: Person re-identification (ReId), a crucial task in surveillance, involves matching individuals across different camera views. The advent of Deep Learning, especially supervised techniques like Convolutional Neural Networks and Attention Mechanisms, has significantly enhanced person Re-ID. However, the success of supervised approaches hinges on vast amounts of annotated data, posing scalability challenges in data labeling and computational costs. To address these limitations, recent research has shifted towards unsupervised person re-identification. Leveraging abundant unlabeled data, unsupervised methods aim to overcome the need for pairwise labelled data. Although traditionally trailing behind supervised approaches, unsupervised techniques have shown promising developments in recent years, signalling a narrowing performance gap. Motivated by this evolving landscape, our survey pursues two primary objectives. First, we review and categorize significant publications in supervised person re-identification, providing an in-depth overview of the current state-of-the-art and emphasizing little room for further improvement in this domain. Second, we explore the latest advancements in unsupervised person re-identification over the past three years, offering insights into emerging trends and shedding light on the potential convergence of performance between supervised and unsupervised paradigms. This dual-focus survey aims to contribute to the evolving narrative of person re-identification, capturing both the mature landscape of supervised techniques and the promising outcomes in the realm of unsupervised learning.

Method: Proposes Sequential Token Merging (STM) with: 1) Bidirectional nearest neighbor merging to preserve sequential dependencies through symmetric spatial aggregation, and 2) Hidden states protection to stabilize hidden states around the class token, leveraging Mamba’s layer-wise loss convergence.

Result: Achieves only 1.0% accuracy drop for ViM-Ti at 20% token reduction and 1.4% degradation for ViM-S at 40% reduction, demonstrating state-of-the-art efficiency with minimal complexity.

Conclusion: STM provides an effective solution for Vision Mamba efficiency while offering new insights into state-space model dynamics, with codes to be released soon.

Abstract: Vision Mambas (ViMs) achieve remarkable success with sub-quadratic complexity, but their efficiency remains constrained by quadratic token scaling with image resolution. While existing methods address token redundancy, they overlook ViMs’ intrinsic Limited Directional Sequential Dependence (LDSD) - a critical information flow mechanism revealed in our analysis. We further identify Mamba’s selective scan enables gradual information aggregation in hidden states. Based on these insights, we propose Sequential Token Merging (STM), featuring: 1) Bidirectional nearest neighbor merging to preserve sequential dependencies through symmetric spatial aggregation, and 2) Hidden states protection to stabilize the hidden states around the class token. STM strategically leverages Mamba’s layer-wise loss convergence to convert temporal forgetfulness into stability. Experiments demonstrate STM’s superiority: 1.0% accuracy drop for ViM-Ti at 20% token reduction, and only 1.4% degradation for ViM-S at 40% reduction. Our method achieves state-of-the-art efficiency with minimal complexity, while providing new insights into state-space model dynamics. Codes will be released soon.

Method: RCI systematically compares reference-model performance on image patches versus full images to measure dataset reliance on global versus local visual information.

Result: When applied to 13 multimodal benchmarks, RCI revealed that most favor localized reasoning and exhibit significant spatial biases, posing risks for real-world applications.

Conclusion: RCI provides an actionable tool for diagnosing and mitigating biases in multimodal benchmarks, enabling construction of better datasets for developing robust, enterprise-ready multimodal systems.

Abstract: Multimodal Large Language Models (MLLMs) have achieved impressive results on vision-language benchmarks, yet it remains unclear whether these benchmarks assess genuine global reasoning or allow success via localized visual cues. Existing evaluation methods do not explicitly measure this distinction, hindering effective dataset curation and real-world focused model development. We introduce Region Comprehension Index (RCI), the first model-based score to directly quantify a dataset’s reliance on global versus local visual information. RCI systematically compares reference-model performance on image patches versus full images, revealing if tasks require holistic image understanding or can be solved with partial or localized visual cues. When applying RCI to 13 widely used multimodal benchmarks, we observed that most of them favor localized reasoning and exhibit significant spatial biases, indicating potential risks in real-world applications. RCI equips researchers & practitioners with an actionable tool for diagnosing & mitigating these biases, enabling the construction of datasets and benchmarks to foster the development of robust, enterprise-ready multimodal systems.

Method: Conducted an in-depth review and taxonomy of diverse SR methods based on backbone structures and purposes, analyzing methodologies, datasets, evaluation protocols, empirical results, and complexity.

Result: Created a comprehensive survey covering over 150 SISR methods, nearly 70 VSR approaches, and approximately 30 SSR and LFSR techniques, with detailed analysis and taxonomy.

Conclusion: This work serves as a valuable resource and guidance for researchers in super-resolution, with an accompanying repository for easy access to related work.

Abstract: Super-resolution (SR) has garnered significant attention within the computer vision community, driven by advances in deep learning (DL) techniques and the growing demand for high-quality visual applications. With the expansion of this field, numerous surveys have emerged. Most existing surveys focus on specific domains, lacking a comprehensive overview of this field. Here, we present an in-depth review of diverse SR methods, encompassing single image super-resolution (SISR), video super-resolution (VSR), stereo super-resolution (SSR), and light field super-resolution (LFSR). We extensively cover over 150 SISR methods, nearly 70 VSR approaches, and approximately 30 techniques for SSR and LFSR. We analyze methodologies, datasets, evaluation protocols, empirical results, and complexity. In addition, we conducted a taxonomy based on each backbone structure according to the diverse purposes. We also explore valuable yet under-studied open issues in the field. We believe that this work will serve as a valuable resource and offer guidance to researchers in this domain. To facilitate access to related work, we created a dedicated repository available at https://github.com/AVC2-UESTC/Holistic-Super-Resolution-Review.

Method: Uses CLIP-style contrastive training to map voxel embeddings from a vision backbone to a frozen LEM’s latent space. A trainable probe aligns vision features with text tokens using contrastive loss with hard/semi-hard negatives and positive pairs. Descriptive prompts encode class semantics as structured anchors.

Result: Achieves state-of-the-art performance with only 0.07% parameter updates: +0.92 OA and +1.60 Kappa on Indian Pines, +0.69 OA and +0.90 Kappa on Pavia University. Uses 50× fewer parameters than DCTN and 90× fewer than SS-TMNet.

Conclusion: The proposed parameter-efficient framework successfully aligns hyperspectral vision features with language representations, demonstrating that effective cross-modal learning in HSI can be achieved with minimal parameter updates while outperforming larger models.

Abstract: As data requirements continue to grow, efficient learning increasingly depends on the curation and distillation of high-value data rather than brute-force scaling of model sizes. In the case of a hyperspectral image (HSI), the challenge is amplified by the high-dimensional 3D voxel structure, where each spatial location is associated with hundreds of contiguous spectral channels. While vision and language models have been optimized effectively for natural image or text tasks, their cross-modal alignment in the hyperspectral domain remains an open and underexplored problem. In this article, we make an attempt to optimize a Vision-Language Model (VLM) for hyperspectral scene understanding by exploiting a CLIP-style contrastive training framework. Our framework maps voxel-level embeddings from a vision backbone onto the latent space of a frozen large embedding model (LEM), where a trainable probe aligns vision features with the model’s textual token representations. The two modalities are aligned via a contrastive loss restricted to a curated set of hard (closest wrong classes) and semi-hard (random distractors) negatives, along with positive pairs. To further enhance alignment, descriptive prompts that encode class semantics are introduced and act as structured anchors for the HSI embeddings. It is seen that the proposed method updates only 0.07 percent of the total parameters, yet yields state-of-the-art performance. For example, on Indian Pines (IP) the model produces better results over unimodal and multimodal baselines by +0.92 Overall Accuracy (OA) and +1.60 Kappa ($\kappa$), while on Pavia University (PU) data it provides gains of +0.69 OA and +0.90 $\kappa$. Moreover, this is achieved with the set of parameters, nearly 50$\times$ smaller than DCTN and 90$\times$ smaller than SS-TMNet.

Method: Uses two modules: attention isolation module to suppress prompt-to-text attention and reweight text-to-prompt attention, and cross-silo collaborative refinement module to integrate decentralized visual knowledge and calibrate global prompts.

Result: Outperforms eight state-of-the-art methods on ten benchmark datasets across class-level and domain-level generalization tasks.

Conclusion: GPR-NIAM provides an effective one-shot federated prompt learning solution that achieves superior generalization while being communication-efficient.

Abstract: Federated Prompt Learning (FPL) enables communication-efficient adaptation by tuning lightweight prompts on top of frozen pre-trained models. Existing FPL methods typically rely on global information, which is only available after the second training round, to facilitate collaboration among client models. Therefore, they are inherently dependent on multi-round communication to fully exhibit their strengths. Moreover, existing one-shot federated learning methods typically focus on fitting seen tasks, but lack cross-task generalization. To bridge this gap, we propose the Global Prompt Refinement with Non-Interfering Attention Masking (GPR-NIAM) method for one-shot FPL. The core idea is to design a masking mechanism that restricts excessive interaction between the original text embeddings and the learnable prompt embeddings. GPR-NIAM achieves this through the collaboration of two key modules. Firstly, the attention isolation module suppresses attention from the learnable prompt tokens to the original text tokens, and reweights the reverse attention which preserves generalization across tasks. Secondly, the cross-silo collaborative refinement module integrates decentralized visual knowledge into a unified base and calibrates the global prompt through multi-source cross-modal knowledge alignment, further mitigating the inconsistency caused by data heterogeneity. Extensive experiments conducted on ten benchmark datasets under two tasks show that GPR-NIAM outperforms eight state-of-the-art methods in both class-level and domain-level generalization.

Method: Combines Generative Zero-Shot Learning with Mixture-of-Experts layers in Generator and Discriminator to generate realistic fake features of unseen classes using KPConv-extracted features from seen classes.

Result: The GZSL-MoE model enhances performance on both seen and unseen classes in 3D point cloud semantic segmentation tasks.

Conclusion: GZSL-MoE provides a promising solution for understanding complex 3D environments when complete training data is unavailable, demonstrating improved zero-shot learning capabilities.

Abstract: Generative Zero-Shot Learning approach (GZSL) has demonstrated significant potential in 3D point cloud semantic segmentation tasks. GZSL leverages generative models like GANs or VAEs to synthesize realistic features (real features) of unseen classes. This allows the model to label unseen classes during testing, despite being trained only on seen classes. In this context, we introduce the Generalized Zero-Shot Learning based-upon Mixture-of-Experts (GZSL-MoE) model. This model incorporates Mixture-of-Experts layers (MoE) to generate fake features that closely resemble real features extracted using a pre-trained KPConv (Kernel Point Convolution) model on seen classes. The main contribution of this paper is the integration of Mixture-of-Experts into the Generator and Discriminator components of the Generative Zero-Shot Learning model for 3D point cloud semantic segmentation, applied to the COVERED dataset (CollabOratiVE Robot Environment Dataset) for Human-Robot Collaboration (HRC) environments. By combining the Generative Zero-Shot Learning model with Mixture-of- Experts, GZSL-MoE for 3D point cloud semantic segmentation provides a promising solution for understanding complex 3D environments, especially when comprehensive training data for all object classes is unavailable. The performance evaluation of the GZSL-MoE model highlights its ability to enhance performance on both seen and unseen classes. Keywords Generalized Zero-Shot Learning (GZSL), 3D Point Cloud, 3D Semantic Segmentation, Human-Robot Collaboration, COVERED (CollabOratiVE Robot Environment Dataset), KPConv, Mixture-of Experts

Method: Introduce PCRI score that systematically quantifies robustness by measuring performance changes between localized image patches and full-image input across 19 MLLMs and 15 benchmarks.

Result: Most leading MLLMs remain brittle to background noise, with only InternVL2-26B and Qwen2VL-72B showing consistent robustness. PCRI reveals how different architectures handle visual context.

Conclusion: PCRI enables rigorous comparison of context robustness, supporting principled model selection and guiding development of more robust architectures for real-world deployment.

Abstract: The reliability of Multimodal Large Language Models (MLLMs) in real-world settings is often undermined by sensitivity to irrelevant or distracting visual context, an aspect not captured by existing evaluation metrics. We introduce the \textbf{Patch Context Robustness Index (PCRI)}, the first systematic and interpretable score for quantifying MLLM robustness to variations in visual context granularity, measuring performance changes between localized image patches and full-image input. Applying PCRI to 19 state-of-the-art MLLMs across 15 vision-language benchmarks, we find that most leading models remain brittle to background noise, with only a few, such as InternVL2-26B and Qwen2VL-72B, demonstrating consistent robustness across tasks. PCRI analysis also highlights how different model architectures handle and integrate visual context, offering actionable diagnostic insight for both researchers and practitioners. PCRI enables rigorous comparison of context robustness, supporting principled model selection and guiding the development of future architectures and training strategies for robust, real-world deployment.

Method: Introduce inductive biases through learned masks in Vision Transformers, creating Inductively Biased Image Transformers (IBiT).

Result: IBiT models are significantly more accurate on small datasets while maintaining Transformer explainability.

Conclusion: Learned masks can effectively introduce CNN-like inductive biases into Vision Transformers, enabling better performance on small datasets without sacrificing explainability.

Abstract: In recent years, Transformer-based architectures have become the dominant method for Computer Vision applications. While Transformers are explainable and scale well with dataset size, they lack the inductive biases of Convolutional Neural Networks. While these biases may be learned on large datasets, we show that introducing these inductive biases through learned masks allow Vision Transformers to learn on much smaller datasets without Knowledge Distillation. These Transformers, which we call Inductively Biased Image Transformers (IBiT), are significantly more accurate on small datasets, while retaining the explainability Transformers.

Method: Uses visual-language model for input preprocessing (segmentation, object size estimation, scene graph construction, prompt rewriting), then compositional diffusion for bounding box synthesis respecting object relations, and finally foreground-conditioned image generation.

Result: Outperforms state-of-the-art layout generation models in layout coherence, spatial realism and aesthetic alignment.

Conclusion: LayoutAgent successfully bridges the gap between high-quality image generation and spatial planning by unifying vision-language reasoning with compositional diffusion.

Abstract: Designing realistic multi-object scenes requires not only generating images, but also planning spatial layouts that respect semantic relations and physical plausibility. On one hand, while recent advances in diffusion models have enabled high-quality image generation, they lack explicit spatial reasoning, leading to unrealistic object layouts. On the other hand, traditional spatial planning methods in robotics emphasize geometric and relational consistency, but they struggle to capture semantic richness in visual scenes. To bridge this gap, in this paper, we propose LayoutAgent, an agentic framework that unifies vision-language reasoning with compositional diffusion for layout generation. Given multiple input images with target objects in them, our method first employs visual-language model to preprocess the inputs through segmentation, object size estimation, scene graph construction, and prompt rewriting. Then we leverage compositional diffusion-a method traditionally used in robotics-to synthesize bounding boxes that respect object relations encoded in the scene graph for spatial layouts. In the end, a foreground-conditioned image generator composes the complete scene by rendering the objects into the planned layout guided by designed prompts. Experiments demonstrate that LayoutAgent outperforms other state-of-the-art layout generation models in layout coherence, spatial realism and aesthetic alignment.

Method: Created CompareBench with 1000 QA pairs across four tasks (quantity, temporal, geometric, spatial) using auxiliary datasets TallyBench and HistCaps, then evaluated both closed-source and open-source VLMs.

Result: Models show scaling trends but consistently fail at temporal ordering and spatial relations, and make mistakes in basic counting and geometric comparisons that are trivial for humans.

Conclusion: Visual comparison remains a systematic blind spot for current VLMs, and CompareBench provides a foundation for advancing more reliable multimodal reasoning.

Abstract: We introduce CompareBench, a benchmark for evaluating visual comparison reasoning in vision-language models (VLMs), a fundamental yet understudied skill. CompareBench consists of 1000 QA pairs across four tasks: quantity (600), temporal (100), geometric (200), and spatial (100). It is derived from two auxiliary datasets that we constructed: TallyBench (2000 counting images with QA) and HistCaps (515 historical images with bilingual captions). We evaluate both closed-source APIs (OpenAI, Gemini, Claude) and open-source models (Qwen2.5-VL and Qwen3-VL series). Results show clear scaling trends but also reveal critical limitations: even the strongest models consistently fail at temporal ordering and spatial relations, and they often make mistakes in basic counting and geometric comparisons that are trivial for humans. These findings demonstrate that visual comparison remains a systematic blind spot for current VLMs. By providing controlled, diverse, and diagnostic evaluation, CompareBench establishes a foundation for advancing more reliable multimodal reasoning.

Method: Multi-stage training strategy with Stable Diffusion as core component in dual-branch architecture, integrated with conditional GAN for panoramic street views, plus fusion strategy leveraging both models.

Result: Outperforms diffusion-only methods across multiple metrics and achieves competitive performance with state-of-the-art GAN-based methods on CVUSA dataset, generating realistic images with preserved local details.

Conclusion: The hybrid framework successfully generates geometrically consistent street-view images with fine-grained details like street markings and atmospheric elements.

Abstract: Street view imagery has become an essential source for geospatial data collection and urban analytics, enabling the extraction of valuable insights that support informed decision-making. However, synthesizing street-view images from corresponding satellite imagery presents significant challenges due to substantial differences in appearance and viewing perspective between these two domains. This paper presents a hybrid framework that integrates diffusion-based models and conditional generative adversarial networks to generate geographically consistent street-view images from satellite imagery. Our approach uses a multi-stage training strategy that incorporates Stable Diffusion as the core component within a dual-branch architecture. To enhance the framework’s capabilities, we integrate a conditional Generative Adversarial Network (GAN) that enables the generation of geographically consistent panoramic street views. Furthermore, we implement a fusion strategy that leverages the strengths of both models to create robust representations, thereby improving the geometric consistency and visual quality of the generated street-view images. The proposed framework is evaluated on the challenging Cross-View USA (CVUSA) dataset, a standard benchmark for cross-view image synthesis. Experimental results demonstrate that our hybrid approach outperforms diffusion-only methods across multiple evaluation metrics and achieves competitive performance compared to state-of-the-art GAN-based methods. The framework successfully generates realistic and geometrically consistent street-view images while preserving fine-grained local details, including street markings, secondary roads, and atmospheric elements such as clouds.

Method: Joint reasoning over image and text tokens in a unified latent vector space using policy gradient method guided by an image quality critic, implemented within the MUG framework that supports language reasoning before image synthesis.

Result: Achieved state-of-the-art results on GenEval, T2I-CompBench, and WISE benchmarks, with 80% improvement over baseline on knowledge-intensive WISE (overall score 0.63).

Conclusion: Joint reasoning in unified latent space is key to strong performance, with demonstrated abilities in temporal and cultural reasoning, highlighting the efficacy of the proposed reasoning method.

Abstract: Reasoning-augmented machine learning systems have shown improved performance in various domains, including image generation. However, existing reasoning-based methods for image generation either restrict reasoning to a single modality (image or text) or rely on high-quality reasoning data for fine-tuning. To tackle these limitations, we propose MILR, a test-time method that jointly reasons over image and text in a unified latent vector space. Reasoning in MILR is performed by searching through vector representations of discrete image and text tokens. Practically, this is implemented via the policy gradient method, guided by an image quality critic. We instantiate MILR within the unified multimodal understanding and generation (MUG) framework that natively supports language reasoning before image synthesis and thus facilitates cross-modal reasoning. The intermediate model outputs, which are to be optimized, serve as the unified latent space, enabling MILR to operate entirely at test time. We evaluate MILR on GenEval, T2I-CompBench, and WISE, achieving state-of-the-art results on all benchmarks. Notably, on knowledge-intensive WISE, MILR attains an overall score of 0.63, improving over the baseline by 80%. Our further analysis indicates that joint reasoning in the unified latent space is the key to its strong performance. Moreover, our qualitative studies reveal MILR’s non-trivial ability in temporal and cultural reasoning, highlighting the efficacy of our reasoning method.

Method: Uses Google Retrieval Enhancement Module for street image data enhancement, Patch-Aware Feature Aggregation for consistent feature extraction, and integrates 3D scene information from multi-scale UAV images with self-supervised and cross-view contrastive learning.

Result: Achieves 25.75% Recall@1 accuracy on University-1652 in the UAVM2025 Challenge, demonstrating robustness and generalization across diverse urban scenarios.

Conclusion: SkyLink effectively addresses viewpoint variation challenges in cross-view geo-localization through enhanced feature retrieval and 3D scene integration.

Abstract: Cross-view geo-localization aims at establishing location correspondences between different viewpoints. Existing approaches typically learn cross-view correlations through direct feature similarity matching, often overlooking semantic degradation caused by extreme viewpoint disparities. To address this unique problem, we focus on robust feature retrieval under viewpoint variation and propose the novel SkyLink method. We firstly utilize the Google Retrieval Enhancement Module to perform data enhancement on street images, which mitigates the occlusion of the key target due to restricted street viewpoints. The Patch-Aware Feature Aggregation module is further adopted to emphasize multiple local feature aggregations to ensure the consistent feature extraction across viewpoints. Meanwhile, we integrate the 3D scene information constructed from multi-scale UAV images as a bridge between street and satellite viewpoints, and perform feature alignment through self-supervised and cross-view contrastive learning. Experimental results demonstrate robustness and generalization across diverse urban scenarios, which achieve 25.75$%$ Recall@1 accuracy on University-1652 in the UAVM2025 Challenge. Code will be released at https://github.com/HRT00/CVGL-3D.

Method: UESA-Net uses a U-shaped encoder-decoder architecture with multidirectional shrinkage attention. Attention modules operate along horizontal, vertical, and depth directions to exploit spatial details, while shrinkage strategy integrates prior knowledge and local features. The decoder applies pairwise shrinkage combining prior low-level physical cues with encoder features.

Result: On TN3K (3493 images) and BUSI (780 images) datasets, UESA-Net achieved state-of-the-art performance with IoU scores of 0.8487 and 0.6495 respectively.

Conclusion: UESA-Net effectively aggregates multidirectional spatial information and prior knowledge to improve robustness and accuracy in breast and thyroid ultrasound segmentation, demonstrating superior performance to existing methods.

Abstract: Background: Breast and thyroid cancers pose an increasing public-health burden. Ultrasound imaging is a cost-effective, real-time modality for lesion detection and segmentation, yet suffers from speckle noise, overlapping structures, and weak global-local feature interactions. Existing networks struggle to reconcile high-level semantics with low-level spatial details. We aim to develop a segmentation framework that bridges the semantic gap between global context and local detail in noisy ultrasound images. Methods: We propose UESA-Net, a U-shaped network with multidirectional shrinkage attention. The encoder-decoder architecture captures long-range dependencies and fine-grained structures of lesions. Within each encoding block, attention modules operate along horizontal, vertical, and depth directions to exploit spatial details, while a shrinkage (threshold) strategy integrates prior knowledge and local features. The decoder mirrors the encoder but applies a pairwise shrinkage mechanism, combining prior low-level physical cues with corresponding encoder features to enhance context modeling. Results: On two public datasets - TN3K (3493 images) and BUSI (780 images) - UESA-Net achieved state-of-the-art performance with intersection-over-union (IoU) scores of 0.8487 and 0.6495, respectively. Conclusions: UESA-Net effectively aggregates multidirectional spatial information and prior knowledge to improve robustness and accuracy in breast and thyroid ultrasound segmentation, demonstrating superior performance to existing methods on multiple benchmarks.

Method: Proposed LUQ strategy that selectively applies ultra-low bit quantization to layers with lower entropy activation distributions that are more resilient to quantization. Also uses mixed multimodal tokens (image and text) for post-training quantization to boost VQA performance.

Result: Evaluated on LLaVA-1.5 and Qwen-2.5-VL across 9 VQA benchmarks. LUQ models use 40% and 31% less memory than 4-bit counterparts respectively, with performance degradation less than 10% on MME benchmark.

Conclusion: Layerwise selective quantization strategy effectively compresses multimodal LLMs to ultra-low bits while maintaining acceptable performance, addressing the unique challenges of multimodal token distributions.

Abstract: Large Language Models (LLMs) with multimodal capabilities have revolutionized vision-language tasks, but their deployment often requires huge memory and computational resources. While post-training quantization (PTQ) has successfully compressed language models to as low as 1-bit precision without significant performance loss, its effectiveness for multimodal LLMs (MLLMs) remains relatively unexplored. In this paper, we present the first study on ultra-low bit (<4-bit) quantization for multimodal LLMs. Our analysis reveals that multimodal tokens and intermediate layer activations produced by them exhibit significantly higher statistical variance and entropy compared to text tokens, making them less tolerant to ultra-low bit quantization. However, the activation distributions of multimodal tokens varies significantly over different layers, with some layers having lower entropy activation distributions. We empirically show that such layers in these models can better tolerate ultra-low bit quantization. Building on these insights, we propose a novel strategy for MLLM quantization, LUQ: Layerwise Ultra-Low Bit Quantization, which selectively applies ultra-low bit quantization to layers that are more resilient to it. Additionally, we also show that using a mix of multimodal tokens (image and text) for PTQ boosts VQA performance in the ultra-low bit regime. We evaluate our method on LLaVA-1.5 and Qwen-2.5-VL across 9 popular VQA benchmarks. The resulting LUQ models use 40% and 31% less memory than their 4-bit counterparts, respectively, while exhibiting a performance degradation of less than 10% on the MME benchmark.

Method: PartCo framework incorporates part-level visual feature correspondences to capture finer-grained semantic structures and enhance category discovery.

Result: Extensive experiments show PartCo significantly improves performance of current GCD approaches and achieves state-of-the-art results on multiple benchmark datasets.

Conclusion: PartCo bridges the gap between semantic labels and part-level visual compositions, setting new benchmarks for Generalized Category Discovery.

Abstract: Generalized Category Discovery (GCD) aims to identify both known and novel categories within unlabeled data by leveraging a set of labeled examples from known categories. Existing GCD methods primarily depend on semantic labels and global image representations, often overlooking the detailed part-level cues that are crucial for distinguishing closely related categories. In this paper, we introduce PartCo, short for Part-Level Correspondence Prior, a novel framework that enhances category discovery by incorporating part-level visual feature correspondences. By leveraging part-level relationships, PartCo captures finer-grained semantic structures, enabling a more nuanced understanding of category relationships. Importantly, PartCo seamlessly integrates with existing GCD methods without requiring significant modifications. Our extensive experiments on multiple benchmark datasets demonstrate that PartCo significantly improves the performance of current GCD approaches, achieving state-of-the-art results by bridging the gap between semantic labels and part-level visual compositions, thereby setting new benchmarks for GCD. Project page: https://visual-ai.github.io/partco

Method: Developed a pipeline integrating CleanVision and Fastdup tools with enhancements including automatic threshold selection. Analyzed impact of various image quality issues on model performance using CIFAKE dataset.

Result: Automatic thresholding improved F1 score from 0.6794 to 0.9468 (single perturbations) and 0.7447 to 0.8557 (dual perturbations). Near-duplicate detection improved from 0.4576 to 0.7928 F1 score.

Conclusion: The workflow effectively advances data quality assessment in image-based ML, showing CNNs are resilient to some distortions but vulnerable to blurring and severe downscaling that obscure critical features.

Abstract: In machine learning, research has traditionally focused on model development, with relatively less attention paid to training data. As model architectures have matured and marginal gains from further refinements diminish, data quality has emerged as a critical factor. However, systematic studies on evaluating and ensuring dataset quality in the image domain remain limited. This study investigates methods for systematically assessing image dataset quality and examines how various image quality factors influence model performance. Using the publicly available and relatively clean CIFAKE dataset, we identify common quality issues and quantify their impact on training. Building on these findings, we develop a pipeline that integrates two community-developed tools, CleanVision and Fastdup. We analyze their underlying mechanisms and introduce several enhancements, including automatic threshold selection to detect problematic images without manual tuning. Experimental results demonstrate that not all quality issues exert the same level of impact. While convolutional neural networks show resilience to certain distortions, they are particularly vulnerable to degradations that obscure critical visual features, such as blurring and severe downscaling. To assess the performance of existing tools and the effectiveness of our proposed enhancements, we formulate the detection of low-quality images as a binary classification task and use the F1 score as the evaluation metric. Our automatic thresholding method improves the F1 score from 0.6794 to 0.9468 under single perturbations and from 0.7447 to 0.8557 under dual perturbations. For near-duplicate detection, our deduplication strategy increases the F1 score from 0.4576 to 0.7928. These results underscore the effectiveness of our workflow and provide a foundation for advancing data quality assessment in image-based machine learning.

Method: Decompositional Efficient Fine-Tuning (DEFT) adapts pre-trained weights by decomposing updates into: (1) projection onto complement of low-rank subspace, and (2) low-rank update within that subspace using two trainable low-rank matrices.

Result: State-of-the-art performance on Dreambooth, Dreambench Plus (personalization), InsDet (object/scene adaptation), and VisualCloze (universal image generation) datasets with Stable Diffusion and unified models.

Conclusion: DEFT demonstrates emergent properties of efficient fine-tuning, achieving superior performance across diverse tasks while maintaining computational efficiency and editability.

Abstract: Efficient fine-tuning of pre-trained Text-to-Image (T2I) models involves adjusting the model to suit a particular task or dataset while minimizing computational resources and limiting the number of trainable parameters. However, it often faces challenges in striking a trade-off between aligning with the target distribution: learning a novel concept from a limited image for personalization and retaining the instruction ability needed for unifying multiple tasks, all while maintaining editability (aligning with a variety of prompts or in-context generation). In this work, we introduce DEFT, Decompositional Efficient Fine-Tuning, an efficient fine-tuning framework that adapts a pre-trained weight matrix by decomposing its update into two components with two trainable matrices: (1) a projection onto the complement of a low-rank subspace spanned by a low-rank matrix, and (2) a low-rank update. The single trainable low-rank matrix defines the subspace, while the other trainable low-rank matrix enables flexible parameter adaptation within that subspace. We conducted extensive experiments on the Dreambooth and Dreambench Plus datasets for personalization, the InsDet dataset for object and scene adaptation, and the VisualCloze dataset for a universal image generation framework through visual in-context learning with both Stable Diffusion and a unified model. Our results demonstrated state-of-the-art performance, highlighting the emergent properties of efficient fine-tuning. Our code is available on \href{https://github.com/MAXNORM8650/DEFT}{DEFTBase}.

Method: Trained on VideoFeedback2 dataset (27,168 human-annotated videos) using a two-stage pipeline: supervised fine-tuning followed by reinforcement learning with Group Relative Policy Optimization (GRPO) to enhance analytical robustness.

Result: Achieves 44.35 (+5.94) accuracy on VideoScore-Bench-v2 and 50.37 (+4.32) average performance across four out-of-domain benchmarks, providing interpretable assessments and effective reward modeling for Best-of-N sampling.

Conclusion: VideoScore2 bridges the gap between evaluation and controllable generation through multi-dimensional, human-aligned assessments with detailed rationales.

Abstract: Recent advances in text-to-video generation have produced increasingly realistic and diverse content, yet evaluating such videos remains a fundamental challenge due to their multi-faceted nature encompassing visual quality, semantic alignment, and physical consistency. Existing evaluators and reward models are limited to single opaque scores, lack interpretability, or provide only coarse analysis, making them insufficient for capturing the comprehensive nature of video quality assessment. We present VideoScore2, a multi-dimensional, interpretable, and human-aligned framework that explicitly evaluates visual quality, text-to-video alignment, and physical/common-sense consistency while producing detailed chain-of-thought rationales. Our model is trained on a large-scale dataset VideoFeedback2 containing 27,168 human-annotated videos with both scores and reasoning traces across three dimensions, using a two-stage pipeline of supervised fine-tuning followed by reinforcement learning with Group Relative Policy Optimization (GRPO) to enhance analytical robustness. Extensive experiments demonstrate that VideoScore2 achieves superior performance with 44.35 (+5.94) accuracy on our in-domain benchmark VideoScore-Bench-v2 and 50.37 (+4.32) average performance across four out-of-domain benchmarks (VideoGenReward-Bench, VideoPhy2, etc), while providing interpretable assessments that bridge the gap between evaluation and controllable generation through effective reward modeling for Best-of-N sampling. Project Page: https://tiger-ai-lab.github.io/VideoScore2/

Method: Leverages diverse traversal permutations to generate multiple causal perspectives, uses model predictions as pseudo-labels to update Mamba-specific parameters, and averages adapted weights across traversal scans.

Result: Experiments on seven benchmarks show TRUST consistently improves robustness and outperforms existing TTA methods.

Conclusion: TRUST is the first approach that explicitly leverages SSM architectural properties for adaptation, effectively addressing distribution shift issues.

Abstract: State Space Models (SSMs) have emerged as efficient alternatives to Vision Transformers (ViTs), with VMamba standing out as a pioneering architecture designed for vision tasks. However, their generalization performance degrades significantly under distribution shifts. To address this limitation, we propose TRUST (Test-Time Refinement using Uncertainty-Guided SSM Traverses), a novel test-time adaptation (TTA) method that leverages diverse traversal permutations to generate multiple causal perspectives of the input image. Model predictions serve as pseudo-labels to guide updates of the Mamba-specific parameters, and the adapted weights are averaged to integrate the learned information across traversal scans. Altogether, TRUST is the first approach that explicitly leverages the unique architectural properties of SSMs for adaptation. Experiments on seven benchmarks show that TRUST consistently improves robustness and outperforms existing TTA methods.

Method: Proposes Dual Discrete Diffusion (D3Diff) loss unifying masked generative models with discrete score matching diffusion, and a multi-level grouped Mixture-of-Experts architecture incorporating semantic and identity facial embeddings. Also constructs UniF²aceD-1M dataset with 130K fine-grained image-caption pairs and 1M VQA pairs.

Result: Outperforms existing models with similar scale: 7.1% higher Desc-GPT and 6.6% higher VQA-score in understanding and generation tasks respectively.

Conclusion: UniF²ace successfully unifies face understanding and generation while achieving superior performance in both tasks, demonstrating the effectiveness of the proposed D3Diff framework and architecture for fine-grained facial analysis.

Abstract: Unified multimodal models (UMMs) have emerged as a powerful paradigm in fundamental cross-modality research, demonstrating significant potential in both image understanding and generation. However, existing research in the face domain primarily faces two challenges: $\textbf{(1)}$ $\textbf{fragmentation development}$, with existing methods failing to unify understanding and generation into a single one, hindering the way to artificial general intelligence. $\textbf{(2) lack of fine-grained facial attributes}$, which are crucial for high-fidelity applications. To handle those issues, we propose $\textbf{UniF$^2$ace}$, $\textit{the first UMM specifically tailored for fine-grained face understanding and generation}$. $\textbf{First}$, we introduce a novel theoretical framework with a Dual Discrete Diffusion (D3Diff) loss, unifying masked generative models with discrete score matching diffusion and leading to a more precise approximation of the negative log-likelihood. Moreover, this D3Diff significantly enhances the model’s ability to synthesize high-fidelity facial details aligned with text input. $\textbf{Second}$, we propose a multi-level grouped Mixture-of-Experts architecture, adaptively incorporating the semantic and identity facial embeddings to complement the attribute forgotten phenomenon in representation evolvement. $\textbf{Finally}$, to this end, we construct UniF$^2$aceD-1M, a large-scale dataset comprising 130K fine-grained image-caption pairs and 1M visual question-answering pairs, spanning a much wider range of facial attributes than existing datasets. Extensive experiments demonstrate that UniF$^2$ace outperforms existing models with a similar scale in both understanding and generation tasks, with 7.1% higher Desc-GPT and 6.6% higher VQA-score, respectively.

Method: MMPB comprises 10k image-query pairs with 111 personalizable concepts across four categories, evaluated using 23 VLMs through a three-stage protocol: concept injection, multi-turn dialogue, and personalized querying.

Result: Most VLMs (including closed-source models) struggle with personalization, particularly in maintaining dialogue consistency, handling user preferences, and adapting to visual cues, with challenges like refusal behaviors and long-context forgetting.

Conclusion: MMPB identifies substantial room for improvement in VLM personalization and provides a scalable benchmark for future research toward truly personalized multi-modal AI.

Abstract: Visual personalization is essential in user-facing AI systems such as smart homes and healthcare, where aligning model behavior with user-centric concepts is critical. However, recent large Vision-Language Models (VLMs), despite their broad applicability, remain underexplored in their ability to adapt to individual users. In this paper, we introduce MMPB, the first extensive benchmark for evaluating VLMs on personalization. MMPB comprises 10k image-query pairs and includes 111 personalizable concepts across four categories: humans, animals, objects, and characters, with the human category enriched with preference-grounded queries. We structure personalization into three main task types, each highlighting a different key property of VLMs. Using 23 widely used VLMs including both open- and closed-source models, we evaluate personalization performance via a three-stage protocol: concept injection, multi-turn dialogue, and personalized querying. Our findings indicate that most VLMs (including some closed-source models) struggle with personalization, particularly in maintaining consistency over dialogue, handling user preferences, and adapting to visual cues. Our analysis reveals that the challenges in VLM personalization (such as refusal behaviors and long-context forgetting) highlight substantial room for improvement. By identifying these limitations and offering a scalable benchmark, MMPB offers valuable insights and a solid foundation for future research toward truly personalized multi-modal AI. Project Page: aidaslab.github.io/MMPB

Method: Combines generative U-Net design with Grad-CAM-guided patch placement for semantic-aware localization that maximizes attack effectiveness while preserving visual realism.

Result: Achieves attack success rates and target-class success consistently exceeding 99% across convolutional networks (DenseNet-121, ResNet-50) and vision transformers (ViT-B/16, Swin-B/16). Outperforms prior white-box attacks, untargeted baselines, and non-realistic approaches.

Conclusion: Establishes a new benchmark for adversarial robustness research by simultaneously ensuring realism, targeted control, and black-box applicability - the three most challenging dimensions of patch-based attacks.

Abstract: Adversarial patch attacks pose a severe threat to deep neural networks, yet most existing approaches rely on unrealistic white-box assumptions, untargeted objectives, or produce visually conspicuous patches that limit real-world applicability. In this work, we introduce a novel framework for fully controllable adversarial patch generation, where the attacker can freely choose both the input image x and the target class y target, thereby dictating the exact misclassification outcome. Our method combines a generative U-Net design with Grad-CAM-guided patch placement, enabling semantic-aware localization that maximizes attack effectiveness while preserving visual realism. Extensive experiments across convolutional networks (DenseNet-121, ResNet-50) and vision transformers (ViT-B/16, Swin-B/16, among others) demonstrate that our approach achieves state-of-the-art performance across all settings, with attack success rates (ASR) and target-class success (TCS) consistently exceeding 99%. Importantly, we show that our method not only outperforms prior white-box attacks and untargeted baselines, but also surpasses existing non-realistic approaches that produce detectable artifacts. By simultaneously ensuring realism, targeted control, and black-box applicability-the three most challenging dimensions of patch-based attacks-our framework establishes a new benchmark for adversarial robustness research, bridging the gap between theoretical attack strength and practical stealthiness.

Method: Uses patch-based cross-attention mechanism with multi-scale processing, embedding attention mechanisms into training to provide intrinsic explainability for temporal saliency.

Result: Outperforms most transformer-based models on real-world datasets, demonstrates superior performance in both temporal saliency detection and forecasting accuracy, and maintains strong performance across datasets of varying complexity.

Conclusion: CrossScaleNet addresses the gap between explainability and performance, offering a balanced approach that effectively captures temporal saliency while delivering state-of-the-art forecasting performance.

Abstract: Explainability in time series forecasting is essential for improving model transparency and supporting informed decision-making. In this work, we present CrossScaleNet, an innovative architecture that combines a patch-based cross-attention mechanism with multi-scale processing to achieve both high performance and enhanced temporal explainability. By embedding attention mechanisms into the training process, our model provides intrinsic explainability for temporal saliency, making its decision-making process more transparent. Traditional post-hoc methods for temporal saliency detection are computationally expensive, particularly when compared to feature importance detection. While ablation techniques may suffice for datasets with fewer features, identifying temporal saliency poses greater challenges due to its complexity. We validate CrossScaleNet on synthetic datasets with known saliency ground truth and on established public benchmarks, demonstrating the robustness of our method in identifying temporal saliency. Experiments on real-world datasets for forecasting task show that our approach consistently outperforms most transformer-based models, offering better explainability without sacrificing predictive accuracy. Our evaluations demonstrate superior performance in both temporal saliency detection and forecasting accuracy. Moreover, we highlight that existing models claiming explainability often fail to maintain strong performance on standard benchmarks. CrossScaleNet addresses this gap, offering a balanced approach that captures temporal saliency effectively while delivering state-of-the-art forecasting performance across datasets of varying complexity.

Method: Transfer learning approach using multimodal interactive perception network pre-trained on GTV datasets and fine-tuned on IGTV data, with 2.5D segmentation framework incorporating slice interaction module (SIM) with channel and spatial attention branches.

Result: Achieved Dice score of 0.609 on private IGTV dataset, substantially outperforming baseline score of 0.385.

Conclusion: The approach demonstrates potential of transfer learning with multimodal techniques and SIM to enhance reliability and clinical relevance of IGTV segmentation for lung cancer radiation therapy planning.

Abstract: Lung cancer remains the leading cause of cancerrelated deaths globally. Accurate delineation of internal gross tumor volume (IGTV) in PET/CT imaging is pivotal for optimal radiation therapy in mobile tumors such as lung cancer to account for tumor motion, yet is hindered by the limited availability of annotated IGTV datasets and attenuated PET signal intensity at tumor boundaries. In this study, we present a transfer learningbased methodology utilizing a multimodal interactive perception network with MAMBA, pre-trained on extensive gross tumor volume (GTV) datasets and subsequently fine-tuned on a private IGTV cohort. This cohort constitutes the PET/CT subset of the Lung-cancer Unified Cross-modal Imaging Dataset (LUCID). To further address the challenge of weak PET intensities in IGTV peripheral slices, we introduce a slice interaction module (SIM) within a 2.5D segmentation framework to effectively model inter-slice relationships. Our proposed module integrates channel and spatial attention branches with depthwise convolutions, enabling more robust learning of slice-to-slice dependencies and thereby improving overall segmentation performance. A comprehensive experimental evaluation demonstrates that our approach achieves a Dice of 0.609 on the private IGTV dataset, substantially surpassing the conventional baseline score of 0.385. This work highlights the potential of transfer learning, coupled with advanced multimodal techniques and a SIM to enhance the reliability and clinical relevance of IGTV segmentation for lung cancer radiation therapy planning.

Method: Uses a diffusion-based generative model that leverages diffusion priors from foundation models like Stable Diffusion, enabling high-quality event data generation with minimal fine-tuning and limited labeled data through control signals.

Result: Successfully synthesizes event data for visual recognition, 2D skeleton estimation, and 3D body pose estimation. The synthesized data enhances model performance in all tasks and can generate events based on unseen text labels during training.

Conclusion: ControlEvents provides an effective solution for generating labeled event datasets, significantly reducing costs while maintaining high quality, and demonstrates powerful text-based generation capabilities inherited from foundation models.

Abstract: In recent years, event cameras have gained significant attention due to their bio-inspired properties, such as high temporal resolution and high dynamic range. However, obtaining large-scale labeled ground-truth data for event-based vision tasks remains challenging and costly. In this paper, we present ControlEvents, a diffusion-based generative model designed to synthesize high-quality event data guided by diverse control signals such as class text labels, 2D skeletons, and 3D body poses. Our key insight is to leverage the diffusion prior from foundation models, such as Stable Diffusion, enabling high-quality event data generation with minimal fine-tuning and limited labeled data. Our method streamlines the data generation process and significantly reduces the cost of producing labeled event datasets. We demonstrate the effectiveness of our approach by synthesizing event data for visual recognition, 2D skeleton estimation, and 3D body pose estimation. Our experiments show that the synthesized labeled event data enhances model performance in all tasks. Additionally, our approach can generate events based on unseen text labels during training, illustrating the powerful text-based generation capabilities inherited from foundation models.

Method: Proposed KAN-IDIR and RandKAN-IDIR using KANs with implicit neural representations, featuring randomized basis sampling to reduce basis functions while maintaining quality.

Result: Achieved highest accuracy among INR-based methods across lung CT, brain MRI, and cardiac MRI datasets, with minimal computational overhead and superior learning stability across random seeds.

Conclusion: RandKAN-IDIR with randomized basis sampling slightly outperforms learnable basis function indices while eliminating additional training complexity, offering an efficient and stable DIR solution.

Abstract: Deformable image registration (DIR) is a cornerstone of medical image analysis, enabling spatial alignment for tasks like comparative studies and multi-modal fusion. While learning-based methods (e.g., CNNs, transformers) offer fast inference, they often require large training datasets and struggle to match the precision of classical iterative approaches on some organ types and imaging modalities. Implicit neural representations (INRs) have emerged as a promising alternative, parameterizing deformations as continuous mappings from coordinates to displacement vectors. However, this comes at the cost of requiring instance-specific optimization, making computational efficiency and seed-dependent learning stability critical factors for these methods. In this work, we propose KAN-IDIR and RandKAN-IDIR, the first integration of Kolmogorov-Arnold Networks (KANs) into deformable image registration with implicit neural representations (INRs). Our proposed randomized basis sampling strategy reduces the required number of basis functions in KAN while maintaining registration quality, thereby significantly lowering computational costs. We evaluated our approach on three diverse datasets (lung CT, brain MRI, cardiac MRI) and compared it with competing instance-specific learning-based approaches, dataset-trained deep learning models, and classical registration approaches. KAN-IDIR and RandKAN-IDIR achieved the highest accuracy among INR-based methods across all evaluated modalities and anatomies, with minimal computational overhead and superior learning stability across multiple random seeds. Additionally, we discovered that our RandKAN-IDIR model with randomized basis sampling slightly outperforms the model with learnable basis function indices, while eliminating its additional training-time complexity.

Method: CST operates directly on 3D image tensors, performing simultaneous feature extraction and contextual modeling, and is compatible with CNN explainability methods like Grad-CAM.

Result: CST achieves superior performance in Set Classification and Set Anomaly Detection tasks compared to existing approaches, and can be pre-trained on large datasets like ImageNet.

Conclusion: CST provides an effective architecture for processing visually heterogeneous image sets with shared semantics, offering better performance, explainability, and transfer learning capabilities than existing methods.

Abstract: We introduce the Convolutional Set Transformer (CST), a novel neural architecture designed to process image sets of arbitrary cardinality that are visually heterogeneous yet share high-level semantics - such as a common category, scene, or concept. Existing set-input networks, e.g., Deep Sets and Set Transformer, are limited to vector inputs and cannot directly handle 3D image tensors. As a result, they must be cascaded with a feature extractor, typically a CNN, which encodes images into embeddings before the set-input network can model inter-image relationships. In contrast, CST operates directly on 3D image tensors, performing feature extraction and contextual modeling simultaneously, thereby enabling synergies between the two processes. This design yields superior performance in tasks such as Set Classification and Set Anomaly Detection and further provides native compatibility with CNN explainability methods such as Grad-CAM, unlike competing approaches that remain opaque. Finally, we show that CSTs can be pre-trained on large-scale datasets and subsequently adapted to new domains and tasks through standard Transfer Learning schemes. To support further research, we release CST-15, a CST backbone pre-trained on ImageNet (https://github.com/chinefed/convolutional-set-transformer).

Method: Proposes a mitigation objective based on cyclic measurement consistency, minimized in a small ball around attack inputs, without requiring model retraining.

Result: Substantially reduces adversarial perturbation effects across datasets, attack types/strengths, and networks, outperforming conventional retraining-based methods both qualitatively and quantitatively.

Conclusion: The approach effectively mitigates adversarial attacks in realistic scenarios including blind setups and adaptive attacks, and shows applicability to impulse noise modeling herringbone artifacts.

Abstract: Deep learning (DL) methods have become the state-of-the-art for reconstructing sub-sampled magnetic resonance imaging (MRI) data. However, studies have shown that these methods are susceptible to small adversarial input perturbations, or attacks, resulting in major distortions in the output images. Various strategies have been proposed to reduce the effects of these attacks, but they require retraining and may lower reconstruction quality for non-perturbed/clean inputs. In this work, we propose a novel approach for mitigating adversarial attacks on MRI reconstruction models without any retraining. Based on the idea of cyclic measurement consistency, we devise a novel mitigation objective that is minimized in a small ball around the attack input. Results show that our method substantially reduces the impact of adversarial perturbations across different datasets, attack types/strengths and PD-DL networks, and qualitatively and quantitatively outperforms conventional mitigation methods that involve retraining. We also introduce a practically relevant scenario for small adversarial perturbations that models impulse noise in raw data, which relates to \emph{herringbone artifacts}, and show the applicability of our approach in this setting. Finally, we show our mitigation approach remains effective in two \emph{realistic} extension scenarios: a blind setup, where the attack strength or algorithm is not known to the user; and an adaptive attack setup, where the attacker has full knowledge of the defense strategy.

Method: Propose TY-RIST with: (1) stride-aware backbone with fine-grained receptive fields, (2) high-resolution detection head, (3) cascaded coordinate attention blocks, (4) branch pruning strategy reducing computational cost by 25.5%, and (5) Normalized Gaussian Wasserstein Distance for regression stability.

Result: State-of-the-art performance on four benchmarks: +7.9% mAP at 0.5 IoU, +3% Precision, +10.2% Recall, up to 123 FPS on single GPU. Cross-dataset validation confirms strong generalization.

Conclusion: TY-RIST effectively addresses infrared small target detection challenges with improved accuracy, real-time performance, and strong generalization capability while reducing computational costs.

Abstract: Infrared small target detection (IRSTD) is critical for defense and surveillance but remains challenging due to (1) target loss from minimal features, (2) false alarms in cluttered environments, (3) missed detections from low saliency, and (4) high computational costs. To address these issues, we propose TY-RIST, an optimized YOLOv12n architecture that integrates (1) a stride-aware backbone with fine-grained receptive fields, (2) a high-resolution detection head, (3) cascaded coordinate attention blocks, and (4) a branch pruning strategy that reduces computational cost by about 25.5% while marginally improving accuracy and enabling real-time inference. We also incorporate the Normalized Gaussian Wasserstein Distance (NWD) to enhance regression stability. Extensive experiments on four benchmarks and across 20 different models demonstrate state-of-the-art performance, improving mAP at 0.5 IoU by +7.9%, Precision by +3%, and Recall by +10.2%, while achieving up to 123 FPS on a single GPU. Cross-dataset validation on a fifth dataset further confirms strong generalization capability. Additional results and resources are available at https://www.github.com/moured/TY-RIST

Method: Uses continuous submanifold fields to encapsulate intrinsic information of Gaussian primitives, enforcing unique mapping and channel homogeneity.

Result: Creates a principled embedding representation that preserves color and geometric structure while being more suitable for neural network learning.

Conclusion: The proposed embedding representation benefits learning of 3D Gaussian Splatting by providing homogeneous and unique feature representations.

Abstract: A well-designed vectorized representation is crucial for the learning systems natively based on 3D Gaussian Splatting. While 3DGS enables efficient and explicit 3D reconstruction, its parameter-based representation remains hard to learn as features, especially for neural-network-based models. Directly feeding raw Gaussian parameters into learning frameworks fails to address the non-unique and heterogeneous nature of the Gaussian parameterization, yielding highly data-dependent models. This challenge motivates us to explore a more principled approach to represent 3D Gaussian Splatting in neural networks that preserves the underlying color and geometric structure while enforcing unique mapping and channel homogeneity. In this paper, we propose an embedding representation of 3DGS based on continuous submanifold fields that encapsulate the intrinsic information of Gaussian primitives, thereby benefiting the learning of 3DGS.

Method: Introduces soft embeddings that replace discrete tokens with expected embeddings under the generator’s output distribution, creating a differentiable continuous surrogate compatible with teacher backbones and tokenizer decoders.

Result: Achieves state-of-the-art one-step results: improved class-to-image performance, one-step FID of 1.56 on ImageNet-256 with GAN refinement, higher GenEval and HPS scores on text-to-image with reward fine-tuning, and gains from TTEO.

Conclusion: Soft-Di[M]O makes one-step generators end-to-end trainable and enables straightforward application of various refinement techniques, significantly improving performance across multiple MDM teachers.

Abstract: One-step generators distilled from Masked Diffusion Models (MDMs) compress multiple sampling steps into a single forward pass, enabling efficient text and image synthesis. However, they suffer two key limitations: they inherit modeling bias from the teacher, and their discrete token outputs block gradient flow, preventing post-distillation refinements such as adversarial training, reward-based fine-tuning, and Test-Time Embedding Optimization (TTEO). In this work, we introduce soft embeddings, a simple relaxation that replaces discrete tokens with the expected embeddings under the generator’s output distribution. Soft embeddings preserve representation fidelity for one-step discrete generator while providing a fully differentiable continuous surrogate that is compatible with teacher backbones and tokenizer decoders. Integrating soft embeddings into the Di[M]O distillation framework (denoted Soft-Di[M]O) makes one-step generators end-to-end trainable and enables straightforward application of GAN-based refinement, differentiable reward fine-tuning, and TTEO. Empirically, across multiple MDM teachers (e.g., MaskBit, MaskGen), Soft-Di[M]O achieves state-of-the-art one-step results: improved class-to-image performance, a one-step FID of 1.56 on ImageNet-256 with GAN-based refinement, along with higher GenEval and HPS scores on text-to-image with reward fine-tuning, and further gains from TTEO.

Method: Uses hierarchical marine fish dataset, employs DeepSeek LLM for text descriptions, Stable Diffusion 2 for image augmentation via hierarchical diffusion strategy, and CLIP-based model for few-shot recognition.

Result: Achieves Top-1 accuracy of 91.67% (family level), 87.58% (genus level), and 85.42% (species level), significantly outperforming baseline CLIP and ViT models for minority classes with <10 training samples.

Conclusion: FishAI 2.0 improves marine fish identification efficiency and accuracy, providing scalable technical solution for marine ecological monitoring and conservation with practical applicability.

Abstract: Traditional marine biological image recognition faces challenges of incomplete datasets and unsatisfactory model accuracy, particularly for few-shot conditions of rare species where data scarcity significantly hampers the performance. To address these issues, this study proposes an intelligent marine fish recognition framework, FishAI 2.0, integrating multimodal few-shot deep learning techniques with image generation for data augmentation. First, a hierarchical marine fish benchmark dataset, which provides a comprehensive data foundation for subsequent model training, is utilized to train the FishAI 2.0 model. To address the data scarcity of rare classes, the large language model DeepSeek was employed to generate high-quality textual descriptions, which are input into Stable Diffusion 2 for image augmentation through a hierarchical diffusion strategy that extracts latent encoding to construct a multimodal feature space. The enhanced visual-textual datasets were then fed into a Contrastive Language-Image Pre-Training (CLIP) based model, enabling robust few-shot image recognition. Experimental results demonstrate that FishAI 2.0 achieves a Top-1 accuracy of 91.67 percent and Top-5 accuracy of 97.97 percent at the family level, outperforming baseline CLIP and ViT models with a substantial margin for the minority classes with fewer than 10 training samples. To better apply FishAI 2.0 to real-world scenarios, at the genus and species level, FishAI 2.0 respectively achieves a Top-1 accuracy of 87.58 percent and 85.42 percent, demonstrating practical utility. In summary, FishAI 2.0 improves the efficiency and accuracy of marine fish identification and provides a scalable technical solution for marine ecological monitoring and conservation, highlighting its scientific value and practical applicability.

Method: Implemented as a guided diffusion model that minimizes Euclidean distance between generated image features and user-specified features during reverse image generation.

Result: Generated images have features remarkably similar to user-specified ones, providing insights into feature spaces of CLIP, ResNet-50, and vision transformers.

Conclusion: The decoder successfully analyzes DNN feature spaces across different architectures without retraining, revealing valuable insights about encoded image attributes.

Abstract: One of the key issues in Deep Neural Networks (DNNs) is the black-box nature of their internal feature extraction process. Targeting vision-related domains, this paper focuses on analysing the feature space of a DNN by proposing a decoder that can generate images whose features are guaranteed to closely match a user-specified feature. Owing to this guarantee that is missed in past studies, our decoder allows us to evidence which of various image attributes are encoded into the user-specified feature. Our decoder is implemented as a guided diffusion model that guides the reverse image generation of a pre-trained diffusion model to minimise the Euclidean distance between the feature of a clean image estimated at each step and the user-specified feature. One practical advantage of our decoder is that it can analyse feature spaces of different DNNs with no additional training and run on a single COTS GPU. The experimental results targeting CLIP’s image encoder, ResNet-50 and vision transformer demonstrate that images generated by our decoder have features remarkably similar to the user-specified ones and reveal valuable insights into these DNNs’ feature spaces.

Method: Uses pre-trained ResNet50 for feature extraction with Global Average Pooling and linear projection, followed by a novel Dense-Dropout sequence to enhance non-linear feature learning and reduce overfitting.

Result: Created the MMCBT dataset with 209 subjects (3671 tumor and 13273 non-tumor images) and developed a framework that addresses class imbalance through augmentation.

Conclusion: The proposed framework and new dataset provide an effective solution for automated brain tumor detection with improved robustness and reduced overfitting.

Abstract: Brain tumors are abnormal cell growths in the central nervous system (CNS), and their timely detection is critical for improving patient outcomes. This paper proposes an automatic and efficient deep-learning framework for brain tumor detection from magnetic resonance imaging (MRI) scans. The framework employs a pre-trained ResNet50 model for feature extraction, followed by Global Average Pooling (GAP) and linear projection to obtain compact, high-level image representations. These features are then processed by a novel Dense-Dropout sequence, a core contribution of this work, which enhances non-linear feature learning, reduces overfitting, and improves robustness through diverse feature transformations. Another major contribution is the creation of the Mymensingh Medical College Brain Tumor (MMCBT) dataset, designed to address the lack of reliable brain tumor MRI resources. The dataset comprises MRI scans from 209 subjects (ages 9 to 65), including 3671 tumor and 13273 non-tumor images, all clinically verified under expert supervision. To overcome class imbalance, the tumor class was augmented, resulting in a balanced dataset well-suited for deep learning research.

Method: Created Hemorica dataset with 372 head CT exams (2012-2024) using double-reading workflow with neurosurgeon adjudication. Fine-tuned standard CNN and transformer architectures for binary classification and segmentation tasks.

Result: Lightweight models achieved 87.8% F1 score for binary classification and 85.5% Dice score for lesion segmentation, validating annotation quality and sufficient sample size.

Conclusion: Hemorica provides a unified benchmark supporting multi-task learning, transfer to weakly labeled cohorts, and facilitates AI assistant development for ICH detection and quantification.

Abstract: Timely diagnosis of Intracranial hemorrhage (ICH) on Computed Tomography (CT) scans remains a clinical priority, yet the development of robust Artificial Intelligence (AI) solutions is still hindered by fragmented public data. To close this gap, we introduce Hemorica, a publicly available collection of 372 head CT examinations acquired between 2012 and 2024. Each scan has been exhaustively annotated for five ICH subtypes-epidural (EPH), subdural (SDH), subarachnoid (SAH), intraparenchymal (IPH), and intraventricular (IVH)-yielding patient-wise and slice-wise classification labels, subtype-specific bounding boxes, two-dimensional pixel masks and three-dimensional voxel masks. A double-reading workflow, preceded by a pilot consensus phase and supported by neurosurgeon adjudication, maintained low inter-rater variability. Comprehensive statistical analysis confirms the clinical realism of the dataset. To establish reference baselines, standard convolutional and transformer architectures were fine-tuned for binary slice classification and hemorrhage segmentation. With only minimal fine-tuning, lightweight models such as MobileViT-XS achieved an F1 score of 87.8% in binary classification, whereas a U-Net with a DenseNet161 encoder reached a Dice score of 85.5% for binary lesion segmentation that validate both the quality of the annotations and the sufficiency of the sample size. Hemorica therefore offers a unified, fine-grained benchmark that supports multi-task and curriculum learning, facilitates transfer to larger but weakly labelled cohorts, and facilitates the process of designing an AI-based assistant for ICH detection and quantification systems.

Method: Developed MindSpeed-MLLM framework for efficient training on Ascend hardware, systematic data production methods, and weight averaging from checkpoints trained with different sequence lengths combined with test-time resolution search.

Result: MindVL-8B matches Qwen2.5VL-7B performance with only 10% training data, and MindVL-671B-A37B matches Qwen2.5VL-72B with only 3% training data, achieving comparable performance with leading multimodal MoE models.

Conclusion: The work provides a valuable hardware alternative, open data recipes, and effective performance-enhancing techniques for the community.

Abstract: We propose MindVL, a multimodal large language model (MLLMs) trained on Ascend NPUs. The training of state-of-the-art MLLMs is often confined to a limited set of hardware platforms and relies heavily on massive, undisclosed data recipes, which hinders reproducibility and open research. To change the common perception that Ascend hardware is unsuitable for efficient full-stage MLLM training, we introduce MindSpeed-MLLM, a highly efficient training framework that supports stable and high-performance training of large-scale Dense and Mixture-of-Experts (MoE) models on Ascend hardware. Based on this, we provide a systematic and open description of the data production methods and mixing strategies for all training stages. Furthermore, we present MindVL, a data-efficient multimodal large language model trained end-to-end on Ascend NPUs. In addition, we find that averaging weights from checkpoints trained with different sequence lengths is particularly effective and yields further gains when combined with test-time resolution search. Our experiments demonstrate superior data efficiency: MindVL-8B matches the performance of Qwen2.5VL-7B using only 10% of its training data, while our MoE model, MindVL-671B-A37B, matches Qwen2.5VL-72B using only 3% of the Qwen2.5VL training data, and achieves comparable performance with other leading multimodal MoE models. Our work provides the community with a valuable hardware alternative, open data recipes, and effective performance-enhancing techniques.

Method: Uses GPT-style decoder-only AR model with video tokenizer for discrete token mapping, camera encoder for 3D positional guidance, and autoregressive transformer module that randomly permutes spatial token order while maintaining temporal order to enhance generation quality.

Result: Extensive experiments on public datasets show ARSS performs comparably to or better than state-of-the-art diffusion-based view synthesis approaches, demonstrating superior consistency and quality in novel view generation.

Conclusion: ARSS successfully addresses diffusion model limitations for world modeling tasks by leveraging autoregressive generation with camera trajectory conditioning, providing a promising approach for consistent and high-quality novel view synthesis from sparse inputs.

Abstract: Despite their exceptional generative quality, diffusion models have limited applicability to world modeling tasks, such as novel view generation from sparse inputs. This limitation arises because diffusion models generate outputs in a non-causal manner, often leading to distortions or inconsistencies across views, and making it difficult to incrementally adapt accumulated knowledge to new queries. In contrast, autoregressive (AR) models operate in a causal fashion, generating each token based on all previously generated tokens. In this work, we introduce \textbf{ARSS}, a novel framework that leverages a GPT-style decoder-only AR model to generate novel views from a single image, conditioned on a predefined camera trajectory. We employ a video tokenizer to map continuous image sequences into discrete tokens and propose a camera encoder that converts camera trajectories into 3D positional guidance. Then to enhance generation quality while preserving the autoregressive structure, we propose a autoregressive transformer module that randomly permutes the spatial order of tokens while maintaining their temporal order. Extensive qualitative and quantitative experiments on public datasets demonstrate that our method performs comparably to, or better than, state-of-the-art view synthesis approaches based on diffusion models. Our code will be released upon paper acceptance.

Method: Separates temporal dynamic information from static scene information using statistical independence loss between biased and unbiased streams, combined with scene prediction loss.

Result: Effectively reduces static bias in action recognition models and confirms the importance of scene prediction loss.

Conclusion: The proposed method successfully addresses static bias in action recognition by explicitly separating dynamic and static information through statistical independence and scene prediction losses.

Abstract: Action recognition models rely excessively on static cues rather than dynamic human motion, which is known as static bias. This bias leads to poor performance in real-world applications and zero-shot action recognition. In this paper, we propose a method to reduce static bias by separating temporal dynamic information from static scene information. Our approach uses a statistical independence loss between biased and unbiased streams, combined with a scene prediction loss. Our experiments demonstrate that this method effectively reduces static bias and confirm the importance of scene prediction loss.

Method: The authors propose Desensitized Adversarial Training (DesenAT): 1) Quantify DNN sensitivity to point cloud features using Shapley values, 2) Generate adversarial samples by eliminating high-contribution data points and using spatial transformation to simulate corruption, 3) Use self-distillation to transfer knowledge from clean samples to adversarial samples and conduct adversarial training in a distillation framework.

Result: Extensive experiments on ModelNet-C and PointCloud-C show that DesenAT effectively improves model robustness against corrupted point clouds without reducing performance on clean datasets. The method outperforms traditional training approaches in handling point cloud corruption.

Conclusion: Reducing DNNs’ over-reliance on point cloud features through desensitized adversarial training can significantly enhance model robustness to corrupted data while maintaining performance on clean datasets, addressing a critical vulnerability in 3D point cloud processing tasks.

Abstract: Due to scene complexity, sensor inaccuracies, and processing imprecision, point cloud corruption is inevitable. Over-reliance on input features is the root cause of DNN vulnerabilities. It remains unclear whether this issue exists in 3D tasks involving point clouds and whether reducing dependence on these features can enhance the model’s robustness to corrupted point clouds. This study attempts to answer these questions. Specifically, we quantified the sensitivity of the DNN to point cloud features using Shapley values and found that models trained using traditional methods exhibited high sensitivity values for certain features. Furthermore, under an equal pruning ratio, prioritizing the pruning of highly sensitive features causes more severe damage to model performance than random pruning. We propose `Desensitized Adversarial Training’ (DesenAT), generating adversarial samples using feature desensitization and conducting training within a self-distillation framework, which aims to alleviate DNN’s over-reliance on point clouds features by smoothing sensitivity. First, data points with high contribution components are eliminated, and spatial transformation is used to simulate corruption scenes, generate adversarial samples, and conduct adversarial training on the model. Next, to compensate for information loss in adversarial samples, we use the self-distillation method to transfer knowledge from clean samples to adversarial samples, and perform adversarial training in a distillation manner.Extensive experiments on ModelNet-C and PointCloud-C demonstrate show that the propose method can effectively improve the robustness of the model without reducing the performance of clean data sets. This code is publicly available at \href{https://github.com/JerkyT/DesenAT/tree/master}{https://github.com/JerkyT/DesenAT}.

Method: Explicitly models skeletal joint relationships with geometric constraints on joint positions, bone lengths, and movement dynamics. Uses parent-relative reweighting for finger flexibility, bone-pose losses, and bone-length constraints to enforce anatomical consistency.

Result: Reduces performance gap between previous best and ground-truth by 56.51%, decreases bone length discrepancies by 18.76%, and reduces movement variance by 5.48%.

Conclusion: The proposed method significantly improves anatomical realism and motion naturalness in sign language translation through explicit modeling of skeletal constraints.

Abstract: Sign language translation from text to video plays a crucial role in enabling effective communication for Deaf and hard–of–hearing individuals. A major challenge lies in generating accurate and natural body poses and movements that faithfully convey intended meanings. Prior methods often neglect the anatomical constraints and coordination patterns of human skeletal motion, resulting in rigid or biomechanically implausible outputs. To address this, we propose a novel approach that explicitly models the relationships among skeletal joints–including shoulders, arms, and hands–by incorporating geometric constraints on joint positions, bone lengths, and movement dynamics. During training, we introduce a parent-relative reweighting mechanism to enhance finger flexibility and reduce motion stiffness. Additionally, bone-pose losses and bone-length constraints enforce anatomically consistent structures. Our method narrows the performance gap between the previous best and the ground-truth oracle by 56.51%, and further reduces discrepancies in bone length and movement variance by 18.76% and 5.48%, respectively, demonstrating significant gains in anatomical realism and motion naturalness.

Method: Built on UMMs where a single model serves as policy, dynamics model, and value function. Introduces self-discriminated filtering to avoid hallucinations in dynamics predictions by using the generative model as a self-discriminator.

Result: Experiments on long-horizon planning tasks show Uni-Plan substantially improves success rates compared to VLM-based methods, demonstrates strong data scalability, requires no expert demonstrations, and achieves better performance with same training data size.

Conclusion: This work lays a foundation for future research in reasoning and decision-making with unified multimodal models, showing the potential of visual reasoning for improved planning capabilities.

Abstract: With the powerful reasoning capabilities of large language models (LLMs) and vision-language models (VLMs), many recent works have explored using them for decision-making. However, most of these approaches rely solely on language-based reasoning, which limits their ability to reason and make informed decisions. Recently, a promising new direction has emerged with unified multimodal models (UMMs), which support both multimodal inputs and outputs. We believe such models have greater potential for decision-making by enabling reasoning through generated visual content. To this end, we propose Uni-Plan, a planning framework built on UMMs. Within this framework, a single model simultaneously serves as the policy, dynamics model, and value function. In addition, to avoid hallucinations in dynamics predictions, we present a novel approach self-discriminated filtering, where the generative model serves as a self-discriminator to filter out invalid dynamics predictions. Experiments on long-horizon planning tasks show that Uni-Plan substantially improves success rates compared to VLM-based methods, while also showing strong data scalability, requiring no expert demonstrations and achieving better performance under the same training-data size. This work lays a foundation for future research in reasoning and decision-making with UMMs.

Method: Formalize copyright detection using Differential Privacy concepts, introduce conditional sensitivity metric, simulate inclusion/exclusion via fine-tuning (learning/unlearning), compute confidence scores over orthogonal prompt distributions, and create CIDD benchmark dataset.

Result: DPM reliably detects infringement content without needing original training data or text prompts, providing an interpretable and practical solution for intellectual property protection.

Conclusion: The proposed framework offers a theoretically grounded, robust, and practical approach to copyright infringement detection in generative AI models, addressing critical legal and ethical challenges.

Abstract: The widespread deployment of large vision models such as Stable Diffusion raises significant legal and ethical concerns, as these models can memorize and reproduce copyrighted content without authorization. Existing detection approaches often lack robustness and fail to provide rigorous theoretical underpinnings. To address these gaps, we formalize the concept of copyright infringement and its detection from the perspective of Differential Privacy (DP), and introduce the conditional sensitivity metric, a concept analogous to sensitivity in DP, that quantifies the deviation in a diffusion model’s output caused by the inclusion or exclusion of a specific training data point. To operationalize this metric, we propose D-Plus-Minus (DPM), a novel post-hoc detection framework that identifies copyright infringement in text-to-image diffusion models. Specifically, DPM simulates inclusion and exclusion processes by fine-tuning models in two opposing directions: learning or unlearning. Besides, to disentangle concept-specific influence from the global parameter shifts induced by fine-tuning, DPM computes confidence scores over orthogonal prompt distributions using statistical metrics. Moreover, to facilitate standardized benchmarking, we also construct the Copyright Infringement Detection Dataset (CIDD), a comprehensive resource for evaluating detection across diverse categories. Our results demonstrate that DPM reliably detects infringement content without requiring access to the original training dataset or text prompts, offering an interpretable and practical solution for safeguarding intellectual property in the era of generative AI.

Method: Proposed perceptual influence metric to quantify perceptual loss contribution, and developed a principled framework to systematically assess loss design choices through experimentation.

Result: Better perceptual loss designs lead to significant improvements in noise reduction and structural fidelity of reconstructed CT images without requiring network architecture changes. Widely used configurations underperform compared to optimized alternatives.

Conclusion: Provides objective guidelines supported by statistical analysis for effective use of perceptual losses in LDCT denoising, demonstrating that proper loss design can substantially enhance reconstruction quality.

Abstract: Perceptual losses have emerged as powerful tools for training networks to enhance Low-Dose Computed Tomography (LDCT) images, offering an alternative to traditional pixel-wise losses such as Mean Squared Error, which often lead to over-smoothed reconstructions and loss of clinically relevant details in LDCT images. The perceptual losses operate in a latent feature space defined by a pretrained encoder and aim to preserve semantic content by comparing high-level features rather than raw pixel values. However, the design of perceptual losses involves critical yet underexplored decisions, including the feature representation level, the dataset used to pretrain the encoder, and the relative importance assigned to the perceptual component during optimization. In this work, we introduce the concept of perceptual influence (a metric that quantifies the relative contribution of the perceptual loss term to the total loss) and propose a principled framework to assess the impact of the loss design choices on the model training performance. Through systematic experimentation, we show that the widely used configurations in the literature to set up a perceptual loss underperform compared to better-designed alternatives. Our findings show that better perceptual loss designs lead to significant improvements in noise reduction and structural fidelity of reconstructed CT images, without requiring any changes to the network architecture. We also provide objective guidelines, supported by statistical analysis, to inform the effective use of perceptual losses in LDCT denoising. Our source code is available at https://github.com/vngabriel/perceptual-influence.

Method: Fine-tuned Presto, a lightweight multi-modal pre-trained transformer (0.4M parameters) that processes both SAR and multispectral data at pixel level, enabling flood mapping with flexible sensor inputs through single model architecture.

Result: Achieved F1 score of 0.896 and mIoU of 0.886 in optimal sensor-fusion scenario, outperforming Prithvi-100M baseline. Maintained strong performance in MS-only (F1: 0.893) and functional capability in SAR-only (F1: 0.718) scenarios.

Conclusion: The parameter-efficient, sensor-flexible approach provides accessible and robust solution for real-world disaster scenarios requiring immediate flood extent assessment regardless of sensor availability constraints.

Abstract: Floods are increasingly frequent natural disasters causing extensive human and economic damage, highlighting the critical need for rapid and accurate flood inundation mapping. While remote sensing technologies have advanced flood monitoring capabilities, operational challenges persist: single-sensor approaches face weather-dependent data availability and limited revisit periods, while multi-sensor fusion methods require substantial computational resources and large-scale labeled datasets. To address these limitations, this study introduces a novel sensor-flexible flood detection methodology by fine-tuning Presto, a lightweight ($\sim$0.4M parameters) multi-modal pre-trained transformer that processes both Synthetic Aperture Radar (SAR) and multispectral (MS) data at the pixel level. Our approach uniquely enables flood mapping using SAR-only, MS-only, or combined SAR+MS inputs through a single model architecture, addressing the critical operational need for rapid response with whatever sensor data becomes available first during disasters. We evaluated our method on the Sen1Floods11 dataset against the large-scale Prithvi-100M baseline ($\sim$100M parameters) across three realistic data availability scenarios. The proposed model achieved superior performance with an F1 score of 0.896 and mIoU of 0.886 in the optimal sensor-fusion scenario, outperforming the established baseline. Crucially, the model demonstrated robustness by maintaining effective performance in MS-only scenarios (F1: 0.893) and functional capabilities in challenging SAR-only conditions (F1: 0.718), confirming the advantage of multi-modal pre-training for operational flood mapping. Our parameter-efficient, sensor-flexible approach offers an accessible and robust solution for real-world disaster scenarios requiring immediate flood extent assessment regardless of sensor availability constraints.

Method: Uses ground-truth depth during training to generate dense 3D-2D correspondences, weights them with a FusionTransformer, computes geometrically-consistent poses via weighted RANSAC, and creates a consistency loss to transfer geometric knowledge into the regression network.

Result: Significantly outperforms ReLoc3R across benchmarks: 40.45% vs 34.85% AUC@5° on CO3Dv2 (16% improvement), 68.66% vs 66.70% on RealEstate10K, and 50.45% vs 49.60% on MegaDepth1500.

Conclusion: GeLoc3r represents a paradigm shift by teaching geometric consistency during training rather than enforcing it at inference, achieving both regression speed and correspondence method accuracy.

Abstract: Prior ReLoc3R achieves breakthrough performance with fast 25ms inference and state-of-the-art regression accuracy, yet our analysis reveals subtle geometric inconsistencies in its internal representations that prevent reaching the precision ceiling of correspondence-based methods like MASt3R (which require 300ms per pair). In this work, we present GeLoc3r, a novel approach to relative camera pose estimation that enhances pose regression methods through Geometric Consistency Regularization (GCR). GeLoc3r overcomes the speed-accuracy dilemma by training regression networks to produce geometrically consistent poses without inference-time geometric computation. During training, GeLoc3r leverages ground-truth depth to generate dense 3D-2D correspondences, weights them using a FusionTransformer that learns correspondence importance, and computes geometrically-consistent poses via weighted RANSAC. This creates a consistency loss that transfers geometric knowledge into the regression network. Unlike FAR method which requires both regression and geometric solving at inference, GeLoc3r only uses the enhanced regression head at test time, maintaining ReLoc3R’s fast speed and approaching MASt3R’s high accuracy. On challenging benchmarks, GeLoc3r consistently outperforms ReLoc3R, achieving significant improvements including 40.45% vs. 34.85% AUC@5{\deg} on the CO3Dv2 dataset (16% relative improvement), 68.66% vs. 66.70% AUC@5{\deg} on RealEstate10K, and 50.45% vs. 49.60% on MegaDepth1500. By teaching geometric consistency during training rather than enforcing it at inference, GeLoc3r represents a paradigm shift in how neural networks learn camera geometry, achieving both the speed of regression and the geometric understanding of correspondence methods.

Method: Designed a monitoring system using IMU sensors and RGB-D camera, collected a dataset directly from stroke patients, developed appropriate preprocessing methods, and proposed a deep learning model suitable for processing multimodal data.

Result: Found that stroke patients’ action data shows less clustering than non-disabled individuals, and the proposed model learns similar tendencies for each label in data with difficult-to-cluster features.

Conclusion: The study demonstrates potential for expanding deep learning models to not only recognize simple actions but also provide assessment feedback for domiciliary rehabilitation of stroke patients.

Abstract: Rehabilitation therapy for stroke patients faces a supply shortage despite the increasing demand. To address this issue, remote monitoring systems that reduce the burden on medical staff are emerging as a viable alternative. A key component of these remote monitoring systems is Human Action Recognition (HAR) technology, which classifies actions. However, existing HAR studies have primarily focused on non-disable individuals, making them unsuitable for recognizing the actions of stroke patients. HAR research for stroke has largely concentrated on classifying relatively simple actions using machine learning rather than deep learning. In this study, we designed a system to monitor the actions of stroke patients, focusing on domiciliary upper limb Activities of Daily Living (ADL). Our system utilizes IMU (Inertial Measurement Unit) sensors and an RGB-D camera, which are the most common modalities in HAR. We directly collected a dataset through this system, investigated an appropriate preprocess and proposed a deep learning model suitable for processing multimodal data. We analyzed the collected dataset and found that the action data of stroke patients is less clustering than that of non-disabled individuals. Simultaneously, we found that the proposed model learns similar tendencies for each label in data with features that are difficult to clustering. This study suggests the possibility of expanding the deep learning model, which has learned the action features of stroke patients, to not only simple action recognition but also feedback such as assessment contributing to domiciliary rehabilitation in future research. The code presented in this study is available at https://github.com/ye-Kim/MMeViT.

Method: Train a lightweight autoencoder to output binary masks using multi-layer activation matching (KL divergence + cross-entropy), mask priors (L1 area, binarization penalty, total variation), and abductive constraints for faithfulness.

Result: Produces small, crisp binary masks that retain classifier behavior while removing irrelevant image regions, yielding practical minimalist explanations.

Conclusion: The combined objectives successfully generate human-interpretable masks that provide faithful explanations of model decision-making by preserving both predictions and internal activations.

Abstract: In this paper we introduce an activation-matching–based approach to generate minimal, faithful explanations for the decision-making of a pretrained classifier on any given image. Given an input image (x) and a frozen model (f), we train a lightweight autoencoder to output a binary mask (m) such that the explanation (e = m \odot x) preserves both the model’s prediction and the intermediate activations of (x). Our objective combines: (i) multi-layer activation matching with KL divergence to align distributions and cross-entropy to retain the top-1 label for both the image and the explanation; (ii) mask priors – L1 area for minimality, a binarization penalty for crisp 0/1 masks, and total variation for compactness; and (iii) abductive constraints for faithfulness and necessity. Together, these objectives yield small, human-interpretable masks that retain classifier behavior while discarding irrelevant input regions, providing practical and faithful minimalist explanations for the decision making of the underlying model.

Method: Leverages vision-language models for prompt-based region localization to apply differentiated masking that emphasizes diagnostically relevant regions while reducing redundancy in background areas, enabling controllable text-guided masking.

Result: Consistently outperforms existing MIM methods across multiple medical imaging modalities (brain MRI, chest CT, lung X-ray), achieving gains of up to +3.1 percentage points in classification accuracy, +1.3 in BoxAP, and +1.1 in MaskAP for detection, with substantially lower overall masking ratios (40% vs 70%).

Conclusion: Controllable, text-driven masking enables semantically aligned self-supervised learning and advances the development of robust vision models for medical image analysis.

Abstract: The scarcity of annotated data in specialized domains such as medical imaging presents significant challenges to training robust vision models. While self-supervised masked image modeling (MIM) offers a promising solution, existing approaches largely rely on random high-ratio masking, leading to inefficiency and poor semantic alignment. Moreover, region-aware variants typically depend on reconstruction heuristics or supervised signals, limiting their adaptability across tasks and modalities. We propose Mask What Matters, a controllable text-guided masking framework for self-supervised medical image analysis. By leveraging vision-language models for prompt-based region localization, our method flexibly applies differentiated masking to emphasize diagnostically relevant regions while reducing redundancy in background areas. This controllable design enables better semantic alignment, improved representation learning, and stronger cross-task generalizability. Comprehensive evaluation across multiple medical imaging modalities, including brain MRI, chest CT, and lung X-ray, shows that Mask What Matters consistently outperforms existing MIM methods (e.g., SparK), achieving gains of up to +3.1 percentage points in classification accuracy, +1.3 in box average precision (BoxAP), and +1.1 in mask average precision (MaskAP) for detection. Notably, it achieves these improvements with substantially lower overall masking ratios (e.g., 40% vs. 70%). This work demonstrates that controllable, text-driven masking can enable semantically aligned self-supervised learning, advancing the development of robust vision models for medical image analysis.

Method: Proposes FMC-DETR with three key components: 1) WeKat backbone using cascaded wavelet transforms and Kolmogorov-Arnold networks for global context and non-linear modeling, 2) Cross-stage Partial Fusion module for efficient multi-scale feature interaction, 3) Multi-Domain Feature Coordination module unifying spatial, frequency, and structural priors.

Result: Achieves state-of-the-art performance on benchmark datasets with fewer parameters. On VisDrone dataset, improves by 6.5% AP and 8.2% AP50 over baseline, demonstrating effectiveness in tiny object detection.

Conclusion: FMC-DETR effectively addresses tiny object detection challenges in aerial imagery through frequency-decoupled fusion, achieving superior performance with efficient parameter usage.

Abstract: Aerial-view object detection is a critical technology for real-world applications such as natural resource monitoring, traffic management, and UAV-based search and rescue. Detecting tiny objects in high-resolution aerial imagery presents a long-standing challenge due to their limited visual cues and the difficulty of modeling global context in complex scenes. Existing methods are often hampered by delayed contextual fusion and inadequate non-linear modeling, failing to effectively use global information to refine shallow features and thus encountering a performance bottleneck. To address these challenges, we propose FMC-DETR, a novel framework with frequency-decoupled fusion for aerial-view object detection. First, we introduce the Wavelet Kolmogorov-Arnold Transformer (WeKat) backbone, which applies cascaded wavelet transforms to enhance global low-frequency context perception in shallow features while preserving fine-grained details, and employs Kolmogorov-Arnold networks to achieve adaptive non-linear modeling of multi-scale dependencies. Next, a lightweight Cross-stage Partial Fusion (CPF) module reduces redundancy and improves multi-scale feature interaction. Finally, we introduce the Multi-Domain Feature Coordination (MDFC) module, which unifies spatial, frequency, and structural priors to to balance detail preservation and global enhancement. Extensive experiments on benchmark aerial-view datasets demonstrate that FMC-DETR achieves state-of-the-art performance with fewer parameters. On the challenging VisDrone dataset, our model achieves improvements of 6.5% AP and 8.2% AP50 over the baseline, highlighting its effectiveness in tiny object detection. The code can be accessed at https://github.com/bloomingvision/FMC-DETR.

Method: Leveraged direct preference optimization approach for alignment training, employed public reward models to construct preference training datasets, conducted experiments across nine reward models, two benchmarks, and two baseline models with varying structures and generative algorithms.

Result: Alignment models significantly outperformed prior models across standard metrics, GPT-4 assessments, and human evaluations without any changes to model structures or use of new datasets. Simple ensemble of reward models yielded robust and generalizable results by mitigating biases.

Conclusion: The work establishes a simple yet solid baseline for image inpainting preference alignment, demonstrating that fundamental approaches with existing tools can achieve significant improvements, and hopes to push this promising research frontier forward.

Abstract: This paper investigates image inpainting with preference alignment. Instead of introducing a novel method, we go back to basics and revisit fundamental problems in achieving such alignment. We leverage the prominent direct preference optimization approach for alignment training and employ public reward models to construct preference training datasets. Experiments are conducted across nine reward models, two benchmarks, and two baseline models with varying structures and generative algorithms. Our key findings are as follows: (1) Most reward models deliver valid reward scores for constructing preference data, even if some of them are not reliable evaluators. (2) Preference data demonstrates robust trends in both candidate scaling and sample scaling across models and benchmarks. (3) Observable biases in reward models, particularly in brightness, composition, and color scheme, render them susceptible to cause reward hacking. (4) A simple ensemble of these models yields robust and generalizable results by mitigating such biases. Built upon these observations, our alignment models significantly outperform prior models across standard metrics, GPT-4 assessments, and human evaluations, without any changes to model structures or the use of new datasets. We hope our work can set a simple yet solid baseline, pushing this promising frontier. Our code is open-sourced at: https://github.com/shenytzzz/Follow-Your-Preference.

Method: Cross-magnification distillation framework that transfers knowledge from 20x magnification teacher to 5x magnification student architecture, using dual-level knowledge transfer (global image representations and local spatial token mapping) and trained on 3.49 million images.

Result: XMAG achieved diagnostic accuracy within 1% of larger foundation models while delivering 30-fold processing acceleration (8.8 WSIs per minute) and demonstrated robust generalization in cross-institutional validation.

Conclusion: Cross-magnification distillation is a viable approach for deploying foundation model capabilities in resource-constrained clinical environments, potentially enabling real-time pathology AI integration.

Abstract: Foundation models (FM) have transformed computational pathology but remain computationally prohibitive for clinical deployment due to their massive parameter counts and high-magnification processing requirements. Here, we introduce XMAG, a lightweight FM developed through corss-magnification distillation that transfers knowledge from state-of-the-art 20x magnification teacher to an efficient 5x magnification student architecture. XMAG employs a compact backbone and operates entirely at 5x, requiring 11.3 times fewer patches per whole slide image (WSI) compared to existing approaches. Our Novel distillation framework incorporates dual-level knowledge transfer, aligning both global image representations and local spatial token mapping. We trained XMAG on 3.49 million images curated from publicly available datasets and evaluated performance across six clinically relevant histopathology analysis tasks spanning multiple cancer types. XMAG achieved diagnostic accuracy within 1% of substantially larger foundation models while delivering 30-fold processing acceleration, reaching 8.8 WSIs per minute processing speed. Our cross-institutional validation confirmed robust generalization. Further, we developed an end-to-end training strategy to further boost our model’s performance to approach the larger FMs’ performance. These results establish cross-magnification distillation as a viable approach for deploying FM capabilities in resource-constrained clinical environments, potentially enabling real-time pathology AI integration.

Method: CoPatch leverages internal model components to enhance spatial representations: 1) constructs hybrid text features by incorporating context tokens with spatial cues, 2) extracts patch-level image features from intermediate layers where spatial structure is better preserved, and 3) fuses these into a clustered image-text similarity map (CoMap) for precise mask selection.

Result: CoPatch significantly improves spatial grounding in zero-shot RIS across RefCOCO, RefCOCO+, RefCOCOg, and PhraseCut datasets, achieving 2-7 mIoU improvements without requiring any additional training.

Conclusion: The findings demonstrate the importance of recovering and leveraging untapped spatial knowledge inherently embedded in vision-language models, paving the way for opportunities in zero-shot referring image segmentation.

Abstract: Spatial grounding is crucial for referring image segmentation (RIS), where the goal of the task is to localize an object described by language. Current foundational vision-language models (VLMs), such as CLIP, excel at aligning images and text but struggle with understanding spatial relationships. Within the language stream, most existing methods often focus on the primary noun phrase when extracting local text features, undermining contextual tokens. Within the vision stream, CLIP generates similar features for images with different spatial layouts, resulting in limited sensitivity to spatial structure. To address these limitations, we propose \textsc{CoPatch}, a zero-shot RIS framework that leverages internal model components to enhance spatial representations in both text and image modalities. For language, \textsc{CoPatch} constructs hybrid text features by incorporating context tokens carrying spatial cues. For vision, it extracts patch-level image features using our novel path discovered from intermediate layers, where spatial structure is better preserved. These enhanced features are fused into a clustered image-text similarity map, \texttt{CoMap}, enabling precise mask selection. As a result, \textsc{CoPatch} significantly improves spatial grounding in zero-shot RIS across RefCOCO, RefCOCO+, RefCOCOg, and PhraseCut (+ 2–7 mIoU) without requiring any additional training. Our findings underscore the importance of recovering and leveraging the untapped spatial knowledge inherently embedded in VLMs, thereby paving the way for opportunities in zero-shot RIS.

Method: Used the Oral Diseases dataset to train and validate Vision Transformer (ViT), DeiT, ConvNeXt, Swin Transformer, and BEiT models, measuring performance metrics including validation accuracy, precision, recall, and F1-score with emphasis on handling imbalanced classes.

Result: ConvNeXt achieved highest validation accuracy (81.06%), followed by BEiT (80.00%) and Swin Transformer (79.73%), all with strong F1-scores. ViT and DeiT achieved 79.37% and 78.79% respectively but struggled with Caries-related classes.

Conclusion: ConvNeXt, Swin Transformer, and BEiT showed reliable diagnostic performance and are promising for clinical dental imaging applications. Findings provide guidance for model selection in AI-driven oral disease diagnostics and highlight the importance of addressing data imbalance.

Abstract: Objective: The aim of this study was to systematically evaluate and compare the performance of five state-of-the-art transformer-based architectures - Vision Transformer (ViT), Data-efficient Image Transformer (DeiT), ConvNeXt, Swin Transformer, and Bidirectional Encoder Representation from Image Transformers (BEiT) - for multi-class dental disease classification. The study specifically focused on addressing real-world challenges such as data imbalance, which is often overlooked in existing literature. Study Design: The Oral Diseases dataset was used to train and validate the selected models. Performance metrics, including validation accuracy, precision, recall, and F1-score, were measured, with special emphasis on how well each architecture managed imbalanced classes. Results: ConvNeXt achieved the highest validation accuracy at 81.06, followed by BEiT at 80.00 and Swin Transformer at 79.73, all demonstrating strong F1-scores. ViT and DeiT achieved accuracies of 79.37 and 78.79, respectively, but both struggled particularly with Caries-related classes. Conclusions: ConvNeXt, Swin Transformer, and BEiT showed reliable diagnostic performance, making them promising candidates for clinical application in dental imaging. These findings provide guidance for model selection in future AI-driven oral disease diagnostic tools and highlight the importance of addressing data imbalance in real-world scenarios

Method: Selectively replaces intermediate convolutions with Hybrid Hadamard-based transform layers whose internal convolutions are implemented via multiplication-avoiding in-memory operations using SRAM-based in-memory computing.

Result: Eliminates up to 52% of multiplications compared to baseline ResNet models while achieving comparable accuracy on CIFAR-10, CIFAR-100, and Tiny ImageNet datasets, with significant reduction in computational complexity and parameters.

Conclusion: Combining structured Hadamard transform layers with SRAM-based in-memory computing multiplication-avoiding operators is a promising path towards efficient deep learning architectures.

Abstract: Reducing the cost of multiplications is critical for efficient deep neural network deployment, especially in energy-constrained edge devices. In this work, we introduce HTMA-Net, a novel framework that integrates the Hadamard Transform (HT) with multiplication-avoiding (MA) SRAM-based in-memory computing to reduce arithmetic complexity while maintaining accuracy. Unlike prior methods that only target multiplications in convolutional layers or focus solely on in-memory acceleration, HTMA-Net selectively replaces intermediate convolutions with Hybrid Hadamard-based transform layers whose internal convolutions are implemented via multiplication-avoiding in-memory operations. We evaluate HTMA-Net on ResNet-18 using CIFAR-10, CIFAR-100, and Tiny ImageNet, and provide a detailed comparison against regular, MF-only, and HT-only variants. Results show that HTMA-Net eliminates up to 52% of multiplications compared to baseline ResNet-18, ResNet-20, and ResNet-50 models, while achieving comparable accuracy in evaluation and significantly reducing computational complexity and the number of parameters. Our results demonstrate that combining structured Hadamard transform layers with SRAM-based in-memory computing multiplication-avoiding operators is a promising path towards efficient deep learning architectures.

Method: Constructed ChangeIMTI dataset with four tasks, and designed ChangeVG model with vision-guided module featuring dual-branch architecture combining fine-grained spatial feature extraction with high-level semantic summarization, using these as auxiliary prompts to guide large VLMs during instruction tuning.

Result: Outperforms the strongest method Semantic-CC by 1.39 points on the comprehensive S*m metric for change captioning task, demonstrating superiority across four tasks.

Conclusion: The proposed ChangeVG model with dual-granularity awareness effectively facilitates hierarchical cross-modal learning for remote sensing change understanding tasks.

Abstract: Remote sensing change understanding (RSCU) is essential for analyzing remote sensing images and understanding how human activities affect the environment. However, existing datasets lack deep understanding and interactions in the diverse change captioning, counting, and localization tasks. To tackle these gaps, we construct ChangeIMTI, a new large-scale interactive multi-task instruction dataset that encompasses four complementary tasks including change captioning, binary change classification, change counting, and change localization. Building upon this new dataset, we further design a novel vision-guided vision-language model (ChangeVG) with dual-granularity awareness for bi-temporal remote sensing images (i.e., two remote sensing images of the same area at different times). The introduced vision-guided module is a dual-branch architecture that synergistically combines fine-grained spatial feature extraction with high-level semantic summarization. These enriched representations further serve as the auxiliary prompts to guide large vision-language models (VLMs) (e.g., Qwen2.5-VL-7B) during instruction tuning, thereby facilitating the hierarchical cross-modal learning. We extensively conduct experiments across four tasks to demonstrate the superiority of our approach. Remarkably, on the change captioning task, our method outperforms the strongest method Semantic-CC by 1.39 points on the comprehensive S*m metric, which integrates the semantic similarity and descriptive accuracy to provide an overall evaluation of change caption. Moreover, we also perform a series of ablation studies to examine the critical components of our method.

Method: Proposed Conditional Dependent Coupling strategy that decomposes generative process into interpolant trajectories at multiple stages, modeled as a single unified Diffusion Transformer for end-to-end optimization and knowledge sharing.

Result: Achieves both high fidelity and efficiency across multiple resolutions, demonstrating improved performance compared to existing approaches.

Conclusion: The unified multistage framework successfully addresses the computation-fidelity trade-off in image generation by enabling accurate distribution learning and end-to-end optimization within a single model architecture.

Abstract: Existing image generation models face critical challenges regarding the trade-off between computation and fidelity. Specifically, models relying on a pretrained Variational Autoencoder (VAE) suffer from information loss, limited detail, and the inability to support end-to-end training. In contrast, models operating directly in the pixel space incur prohibitive computational cost. Although cascade models can mitigate computational cost, stage-wise separation prevents effective end-to-end optimization, hampers knowledge sharing, and often results in inaccurate distribution learning within each stage. To address these challenges, we introduce a unified multistage generative framework based on our proposed Conditional Dependent Coupling strategy. It decomposes the generative process into interpolant trajectories at multiple stages, ensuring accurate distribution learning while enabling end-to-end optimization. Importantly, the entire process is modeled as a single unified Diffusion Transformer, eliminating the need for disjoint modules and also enabling knowledge sharing. Extensive experiments demonstrate that our method achieves both high fidelity and efficiency across multiple resolutions.

Method: Leveraged DINOv3, a state-of-the-art self-supervised vision transformer, to generate powerful feature representations for stroke analysis tasks including segmentation, classification, and ASPECTS scoring.

Result: Established strong benchmarks for multiple stroke analysis tasks on public and private datasets, demonstrating the potential of self-supervised models for automated stroke diagnosis.

Conclusion: Advanced self-supervised models like DINOv3 show promise for improving automated stroke diagnosis from NCCT, though the approach has both advantages and current constraints that require further analysis.

Abstract: Non-contrast computed tomography (NCCT) is essential for rapid stroke diagnosis but is limited by low image contrast and signal to noise ratio. We address this challenge by leveraging DINOv3, a state-of-the-art self-supervised vision transformer, to generate powerful feature representations for a comprehensive set of stroke analysis tasks. Our evaluation encompasses infarct and hemorrhage segmentation, anomaly classification (normal vs. stroke and normal vs. infarct vs. hemorrhage), hemorrhage subtype classification (EDH, SDH, SAH, IPH, IVH), and dichotomized ASPECTS classification (<=6 vs. >6) on multiple public and private datasets. This study establishes strong benchmarks for these tasks and demonstrates the potential of advanced self-supervised models to improve automated stroke diagnosis from NCCT, providing a clear analysis of both the advantages and current constraints of the approach. The code is available at https://github.com/Zzz0251/DINOv3-stroke.

Method: Earth-Agent unifies RGB and spectral EO data within an MCP-based tool ecosystem, enabling cross-modal, multi-step, and quantitative spatiotemporal reasoning. It dynamically invokes expert tools and models across modalities for complex scientific tasks.

Result: Comprehensive experiments with different LLM backbones, comparisons with general agent frameworks and MLLMs on remote sensing benchmarks demonstrate Earth-Agent’s effectiveness and potential. Earth-Bench benchmark includes 248 expert-curated tasks with 13,729 images across spectrum, products and RGB modalities.

Conclusion: Earth-Agent establishes a new paradigm for EO analysis, moving the field toward scientifically grounded, next-generation applications of LLMs in Earth observation.

Abstract: Earth observation (EO) is essential for understanding the evolving states of the Earth system. Although recent MLLMs have advanced EO research, they still lack the capability to tackle complex tasks that require multi-step reasoning and the use of domain-specific tools. Agent-based methods offer a promising direction, but current attempts remain in their infancy, confined to RGB perception, shallow reasoning, and lacking systematic evaluation protocols. To overcome these limitations, we introduce Earth-Agent, the first agentic framework that unifies RGB and spectral EO data within an MCP-based tool ecosystem, enabling cross-modal, multi-step, and quantitative spatiotemporal reasoning beyond pretrained MLLMs. Earth-Agent supports complex scientific tasks such as geophysical parameter retrieval and quantitative spatiotemporal analysis by dynamically invoking expert tools and models across modalities. To support comprehensive evaluation, we further propose Earth-Bench, a benchmark of 248 expert-curated tasks with 13,729 images, spanning spectrum, products and RGB modalities, and equipped with a dual-level evaluation protocol that assesses both reasoning trajectories and final outcomes. We conduct comprehensive experiments varying different LLM backbones, comparisons with general agent frameworks, and comparisons with MLLMs on remote sensing benchmarks, demonstrating both the effectiveness and potential of Earth-Agent. Earth-Agent establishes a new paradigm for EO analysis, moving the field toward scientifically grounded, next-generation applications of LLMs in Earth observation. Our code and dataset will be publicly released.

Method: Uses lumina-chroma decomposition to decouple degradation from content, a Lumina Degradation Guidance Module (LDGM) for learning luminance degradation priors via frequency-domain amplitude modulation, and a Difficulty-Aware Contrastive Regularization (DACR) module that identifies hard samples and enforces contrastive alignment.

Result: Extensive experiments across multiple weather datasets demonstrate state-of-the-art performance among unsupervised approaches with strong generalization to complex weather degradations.

Conclusion: WeatherCycle provides an effective unified framework for multi-weather image restoration that outperforms existing unsupervised methods and generalizes well to diverse weather conditions.

Abstract: Unsupervised image restoration under multi-weather conditions remains a fundamental yet underexplored challenge. While existing methods often rely on task-specific physical priors, their narrow focus limits scalability and generalization to diverse real-world weather scenarios. In this work, we propose \textbf{WeatherCycle}, a unified unpaired framework that reformulates weather restoration as a bidirectional degradation-content translation cycle, guided by degradation-aware curriculum regularization. At its core, WeatherCycle employs a \textit{lumina-chroma decomposition} strategy to decouple degradation from content without modeling complex weather, enabling domain conversion between degraded and clean images. To model diverse and complex degradations, we propose a \textit{Lumina Degradation Guidance Module} (LDGM), which learns luminance degradation priors from a degraded image pool and injects them into clean images via frequency-domain amplitude modulation, enabling controllable and realistic degradation modeling. Additionally, we incorporate a \textit{Difficulty-Aware Contrastive Regularization (DACR)} module that identifies hard samples via a CLIP-based classifier and enforces contrastive alignment between hard samples and restored features to enhance semantic consistency and robustness. Extensive experiments across serve multi-weather datasets, demonstrate that our method achieves state-of-the-art performance among unsupervised approaches, with strong generalization to complex weather degradations.

Method: Proposes a multi-task learning-based and keypoint-aware deep generative model that encodes complex human motion via compact 3D keypoints, uses these sparse keypoints to estimate dense motion for video synthesis, and integrates a vertex predictor for joint optimization with video generation.

Result: Extensive experiments show Sparse2Dense achieves competitive compression performance over traditional/generative video codecs while enabling precise human vertex prediction for downstream geometry applications.

Conclusion: Sparse2Dense facilitates bandwidth-efficient human-centric media transmission for applications like real-time motion analysis, virtual human animation, and immersive entertainment.

Abstract: For bandwidth-constrained multimedia applications, simultaneously achieving ultra-low bitrate human video compression and accurate vertex prediction remains a critical challenge, as it demands the harmonization of dynamic motion modeling, detailed appearance synthesis, and geometric consistency. To address this challenge, we propose Sparse2Dense, a keypoint-driven generative framework that leverages extremely sparse 3D keypoints as compact transmitted symbols to enable ultra-low bitrate human video compression and precise human vertex prediction. The key innovation is the multi-task learning-based and keypoint-aware deep generative model, which could encode complex human motion via compact 3D keypoints and leverage these sparse keypoints to estimate dense motion for video synthesis with temporal coherence and realistic textures. Additionally, a vertex predictor is integrated to learn human vertex geometry through joint optimization with video generation, ensuring alignment between visual content and geometric structure. Extensive experiments demonstrate that the proposed Sparse2Dense framework achieves competitive compression performance for human video over traditional/generative video codecs, whilst enabling precise human vertex prediction for downstream geometry applications. As such, Sparse2Dense is expected to facilitate bandwidth-efficient human-centric media transmission, such as real-time motion analysis, virtual human animation, and immersive entertainment.

Method: Combines YOLO-OBB for vehicle detection/categorization and YOLO for tire detection, with intelligent tire-vehicle association and a novel TRAX (Tire and Axle Tracking) Algorithm to handle occlusions and partial detections in longer vehicles.

Result: Significantly reduces false positives and improves axle-counting accuracy for long vehicles, demonstrating strong robustness in real-world traffic videos.

Conclusion: This work represents a significant step toward scalable, AI-driven axle counting systems that can replace legacy roadside infrastructure.

Abstract: Accurate counting of vehicle axles is essential for traffic control, toll collection, and infrastructure development. We present an end-to-end, video-based pipeline for axle counting that tackles limitations of previous works in dense environments. Our system leverages a combination of YOLO-OBB to detect and categorize vehicles, and YOLO to detect tires. Detected tires are intelligently associated to their respective parent vehicles, enabling accurate axle prediction even in complex scenarios. However, there are a few challenges in detection when it comes to scenarios with longer and occluded vehicles. We mitigate vehicular occlusions and partial detections for longer vehicles by proposing a novel TRAX (Tire and Axle Tracking) Algorithm to successfully track axle-related features between frames. Our method stands out by significantly reducing false positives and improving the accuracy of axle-counting for long vehicles, demonstrating strong robustness in real-world traffic videos. This work represents a significant step toward scalable, AI-driven axle counting systems, paving the way for machine vision to replace legacy roadside infrastructure.

Method: CalSAM uses two penalties: Feature Fisher Information Penalty (FIP) to reduce encoder sensitivity to domain shift on 3D feature maps, and Confidence Misalignment Penalty (CMP) to penalize overconfident voxel-wise errors. Only the mask decoder is fine-tuned while keeping SAM’s encoders frozen.

Result: On BraTS scanner split (Siemens→GE): +7.4% DSC improvement (80.1% vs 74.6%), -26.9% HD95 reduction (4.6mm vs 6.3mm), -39.5% ECE reduction (5.2% vs 8.6%). On ATLAS-C: +5.3% DSC improvement (75.9%), -32.6% ECE reduction (5.8%).

Conclusion: CalSAM effectively improves domain generalization and uncertainty calibration for brain MRI segmentation while maintaining computational efficiency by freezing SAM’s encoder, with FIP and CMP providing complementary benefits.

Abstract: The Segment Anything Model (SAM) exhibits strong zero-shot performance on natural images but suffers from domain shift and overconfidence when applied to medical volumes. We propose \textbf{CalSAM}, a lightweight adaptation framework that (i) reduces encoder sensitivity to domain shift via a \emph{Feature Fisher Information Penalty} (FIP) computed on 3D feature maps and (ii) penalizes overconfident voxel-wise errors through a \emph{Confidence Misalignment Penalty} (CMP). The combined loss, (\mathcal{L}_{\mathrm{CalSAM}}) fine-tunes only the mask decoder while keeping SAM’s encoders frozen. On cross-center and scanner-shift evaluations, CalSAM substantially improves accuracy and calibration: e.g., on the BraTS scanner split (Siemens$\to$GE) CalSAM shows a $+7.4%$ relative improvement in $\mathrm{DSC}$ (80.1% vs.\ 74.6%), a $-26.9%$ reduction in $\mathrm{HD95}$ (4.6 mm vs.\ 6.3 mm), and a $-39.5%$ reduction in $\mathrm{ECE}$ (5.2% vs.\ 8.6%). On ATLAS-C (motion corruptions), CalSAM achieves a $+5.3%$ relative improvement in $\mathrm{DSC}$ (75.9%) and a $-32.6%$ reduction in $\mathrm{ECE}$ (5.8%). Ablations show FIP and CMP contribute complementary gains ($p<0.01$), and the Fisher penalty incurs a modest $\sim$15% training-time overhead. CalSAM therefore delivers improved domain generalization and better-calibrated uncertainty estimates for brain MRI segmentation, while retaining the computational benefits of freezing SAM’s encoder.

Method: Proposes synthetic point cloud sequence generation for diverse training data, flexible temporal sampling using both adjacent and non-adjacent frames, and dynamic-weighting loss to emphasize confident samples for robust representation learning.

Result: Outperforms UNIT and other unsupervised baselines on SemanticKITTI, nuScenes, and PandaSet datasets, achieving higher segmentation accuracy and more robust temporal associations.

Conclusion: The proposed framework successfully addresses key limitations of existing methods through synthetic data generation, flexible temporal modeling, and adaptive loss weighting, demonstrating superior performance in unsupervised online 3D instance segmentation.

Abstract: Unsupervised online 3D instance segmentation is a fundamental yet challenging task, as it requires maintaining consistent object identities across LiDAR scans without relying on annotated training data. Existing methods, such as UNIT, have made progress in this direction but remain constrained by limited training diversity, rigid temporal sampling, and heavy dependence on noisy pseudo-labels. We propose a new framework that enriches the training distribution through synthetic point cloud sequence generation, enabling greater diversity without relying on manual labels or simulation engines. To better capture temporal dynamics, our method incorporates a flexible sampling strategy that leverages both adjacent and non-adjacent frames, allowing the model to learn from long-range dependencies as well as short-term variations. In addition, a dynamic-weighting loss emphasizes confident and informative samples, guiding the network toward more robust representations. Through extensive experiments on SemanticKITTI, nuScenes, and PandaSet, our method consistently outperforms UNIT and other unsupervised baselines, achieving higher segmentation accuracy and more robust temporal associations. The code will be publicly available at github.com/Eaphan/SFT3D.

Method: Uses a query-efficient, zero-order greedy algorithm to iteratively construct a symmetric, grayscale forehead patch by adding Gaussian blobs, guided only by cosine similarity scores from a surrogate FR model.

Result: Achieves high attack success rate with only ~10,000 queries to black-box ArcFace model, works in both digital and physical tests, and shows strong transferability to unseen FaceNet model.

Conclusion: Demonstrates a practical and severe vulnerability in face recognition systems, proving that robust, transferable attacks can be crafted with limited knowledge of the target system.

Abstract: Adversarial attacks on face recognition (FR) systems pose a significant security threat, yet most are confined to the digital domain or require white-box access. We introduce GaP (Gaussian Patch), a novel method to generate a universal, physically transferable adversarial patch under a strict black-box setting. Our approach uses a query-efficient, zero-order greedy algorithm to iteratively construct a symmetric, grayscale pattern for the forehead. The patch is optimized by successively adding Gaussian blobs, guided only by the cosine similarity scores from a surrogate FR model to maximally degrade identity recognition. We demonstrate that with approximately 10,000 queries to a black-box ArcFace model, the resulting GaP achieves a high attack success rate in both digital and real-world physical tests. Critically, the attack shows strong transferability, successfully deceiving an entirely unseen FaceNet model. Our work highlights a practical and severe vulnerability, proving that robust, transferable attacks can be crafted with limited knowledge of the target system.

Method: Proposed UltraUNet - a lightweight encoder-decoder architecture with domain-specific innovations: Squeeze-and-Excitation blocks, Group Normalization for small-batch stability, summation-based skip connections, and ultrasound-specific augmentations like denoising and blur simulation.

Result: Achieves 250 frames per second with high accuracy: single-dataset Dice = 0.855 and MSD = 0.993px, cross-dataset Dice averaging 0.734 and 0.761 across 8 datasets.

Conclusion: UltraUNet provides a fast, accurate solution for real-time tongue contour segmentation, enabling applications in speech research, clinical diagnostics, and analysis of speech motor disorders.

Abstract: Ultrasound tongue imaging (UTI) is a non-invasive and cost-effective tool for studying speech articulation, motor control, and related disorders. However, real-time tongue contour segmentation remains challenging due to low signal-to-noise ratios, imaging variability, and computational demands. We propose UltraUNet, a lightweight encoder-decoder architecture optimized for real-time segmentation of tongue contours in ultrasound images. UltraUNet incorporates domain-specific innovations such as lightweight Squeeze-and-Excitation blocks, Group Normalization for small-batch stability, and summation-based skip connections to reduce memory and computational overhead. It achieves 250 frames per second and integrates ultrasound-specific augmentations like denoising and blur simulation. Evaluations on 8 datasets demonstrate high accuracy and robustness, with single-dataset Dice = 0.855 and MSD = 0.993px, and cross-dataset Dice averaging 0.734 and 0.761. UltraUNet provides a fast, accurate solution for speech research, clinical diagnostics, and analysis of speech motor disorders.

Method: Patch Rebirth Inversion (PRI) progressively detaches the most important patches during inversion to construct sparse synthetic images, while allowing remaining patches to continue evolving for future selection, enabling initially less informative patches to accumulate class-relevant knowledge.

Result: PRI achieves up to 10x faster inversion than standard Dense Model Inversion and 2x faster than SMI, while consistently outperforming SMI in accuracy and matching DMI performance.

Conclusion: The progressive patch detachment strategy in PRI effectively balances class-agnostic and class-specific knowledge extraction, demonstrating that discarding patches prematurely is inefficient and that all patches can contribute to knowledge transfer through continued evolution.

Abstract: Model inversion is a widely adopted technique in data-free learning that reconstructs synthetic inputs from a pretrained model through iterative optimization, without access to original training data. Unfortunately, its application to state-of-the-art Vision Transformers (ViTs) poses a major computational challenge, due to their expensive self-attention mechanisms. To address this, Sparse Model Inversion (SMI) was proposed to improve efficiency by pruning and discarding seemingly unimportant patches, which were even claimed to be obstacles to knowledge transfer. However, our empirical findings suggest the opposite: even randomly selected patches can eventually acquire transferable knowledge through continued inversion. This reveals that discarding any prematurely inverted patches is inefficient, as it suppresses the extraction of class-agnostic features essential for knowledge transfer, along with class-specific features. In this paper, we propose Patch Rebirth Inversion (PRI), a novel approach that incrementally detaches the most important patches during the inversion process to construct sparse synthetic images, while allowing the remaining patches to continue evolving for future selection. This progressive strategy not only improves efficiency, but also encourages initially less informative patches to gradually accumulate more class-relevant knowledge, a phenomenon we refer to as the Re-Birth effect, thereby effectively balancing class-agnostic and class-specific knowledge. Experimental results show that PRI achieves up to 10x faster inversion than standard Dense Model Inversion (DMI) and 2x faster than SMI, while consistently outperforming SMI in accuracy and matching the performance of DMI.

Method: Leverages model self-consistency by comparing detailed responses against concise binary answers, using inconsistency signals to automatically curate training data without external supervision.

Result: Significant improvements in factual grounding and reliability across multiple benchmarks (AMBER, MultiObject-Hal, Object HalBench, MMHal-Bench), while maintaining robust instruction-following ability on LLaVA-Bench and MMBench.

Conclusion: The self-consistency based approach offers a scalable and efficient solution that effectively reduces hallucinations using unlabeled data, eliminating the need for human annotations or external model supervision.

Abstract: Vision-language models often hallucinate details, generating non-existent objects or inaccurate attributes that compromise output reliability. Existing methods typically address these issues via extensive human annotations or external supervision from more powerful models. In this work, we present a novel framework that leverages the model’s self-consistency between long responses and short answers to generate preference pairs for training. We observe that short binary questions tend to yield highly reliable responses, which can be used to query the target model to evaluate and rank its generated responses. Specifically, we design a self-reflection pipeline where detailed model responses are compared against concise binary answers, and inconsistency signals are utilized to automatically curate high-quality training data without human annotations or external model-based supervision. By relying solely on self-consistency rather than external supervision, our method offers a scalable and efficient solution that effectively reduces hallucinations using unlabeled data. Extensive experiments on multiple benchmarks, i.e., AMBER, MultiObject-Hal (ROPE), Object HalBench, and MMHal-Bench, demonstrate significant improvements in factual grounding and reliability. Moreover, our approach maintains robust instruction-following ability, as evidenced by enhanced performance on LLaVA-Bench and MMBench.

Method: Uses MLLMs to generate target-item descriptions and aesthetic chain-of-thought to create structured aesthetic profiles. Fuses visual summaries with textual descriptions using dynamic entropy-gated mechanism for embedding space ranking.

Result: Achieves state-of-the-art performance on Aesthetic-100 dataset and demonstrates advanced zero-shot retrieval capability on Polyvore dataset.

Conclusion: TATTOO successfully streamlines outfit recommendation to a training-free paradigm with better performance and enhanced aesthetic awareness compared to conventional training-based methods.

Abstract: The global fashion e-commerce market relies significantly on intelligent and aesthetic-aware outfit-completion tools to promote sales. While previous studies have approached the problem of fashion outfit-completion and compatible-item retrieval, most of them require expensive, task-specific training on large-scale labeled data, and no effort is made to guide outfit recommendation with explicit human aesthetics. In the era of Multimodal Large Language Models (MLLMs), we show that the conventional training-based pipeline could be streamlined to a training-free paradigm, with better recommendation scores and enhanced aesthetic awareness. We achieve this with TATTOO, a Training-free AesTheTic-aware Outfit recommendation approach. It first generates a target-item description using MLLMs, followed by an aesthetic chain-of-thought used to distill the images into a structured aesthetic profile including color, style, occasion, season, material, and balance. By fusing the visual summary of the outfit with the textual description and aesthetics vectors using a dynamic entropy-gated mechanism, candidate items can be represented in a shared embedding space and be ranked accordingly. Experiments on a real-world evaluation set Aesthetic-100 show that TATTOO achieves state-of-the-art performance compared with existing training-based methods. Another standard Polyvore dataset is also used to measure the advanced zero-shot retrieval capability of our training-free method.

Method: Multi-modal translation framework using Component-aware Adaptive Instance Normalization (CoAdaIN) that adapts styles to different image components individually, unlike global style application in original AdaIN.

Result: Produces more realistic and diverse thermal image translations compared to existing methods.

Conclusion: CoAdaIN enables effective one-to-many RGB-to-thermal mapping, addressing dataset limitations for thermal imaging in ADAS applications.

Abstract: Thermal imaging in Advanced Driver Assistance Systems (ADAS) improves road safety with superior perception in low-light and harsh weather conditions compared to traditional RGB cameras. However, research in this area faces challenges due to limited dataset availability and poor representation in driving simulators. RGB-to-thermal image translation offers a potential solution, but existing methods focus on one-to-one mappings. We propose a one-to-many mapping using a multi-modal translation framework enhanced with our Component-aware Adaptive Instance Normalization (CoAdaIN). Unlike the original AdaIN, which applies styles globally, CoAdaIN adapts styles to different image components individually. The result, as we show, is more realistic and diverse thermal image translations. This is the accepted author manuscript of the paper published in IEEE Sensors Conference 2024. The final published version is available at 10.1109/SENSORS60989.2024.10785056.

Method: Convert LiDAR frames to epsilon-graphs, compute Laplacian spectrum, use eigenvalues and eigenvector statistics as pose descriptors, apply temporal statistics over sliding windows, and classify with SVM and random forests.

Result: Achieved 94.4% accuracy on 13-class rehabilitation set and 90.3% on all 27 activities in MM-Fi dataset with 40 subjects under strict subject-independent protocol, outperforming skeleton-based baselines.

Conclusion: The method provides compact, interpretable features directly from point cloud geometry, offering an accurate and efficient alternative to end-to-end deep learning for HAR.

Abstract: Human Activity Recognition supports applications in healthcare, manufacturing, and human-machine interaction. LiDAR point clouds offer a privacy-preserving alternative to cameras and are robust to illumination. We propose a HAR method based on graph spectral analysis. Each LiDAR frame is mapped to a proximity graph (epsilon-graph) and the Laplacian spectrum is computed. Eigenvalues and statistics of eigenvectors form pose descriptors, and temporal statistics over sliding windows yield fixed vectors for classification with support vector machines and random forests. On the MM-Fi dataset with 40 subjects and 27 activities, under a strict subject-independent protocol, the method reaches 94.4% accuracy on a 13-class rehabilitation set and 90.3% on all 27 activities. It also surpasses the skeleton-based baselines reported for MM-Fi. The contribution is a compact and interpretable feature set derived directly from point cloud geometry that provides an accurate and efficient alternative to end-to-end deep learning.

Method: Uses a pre-trained 3D-aware diffusion model to propose complete scenes, then repurposes a multi-view stereo model as a 3D-aware oracle to validate uncertainties, guiding 3D Gaussian Splatting optimization via uncertainty-weighted loss.

Result: Outperforms state-of-the-art methods on datasets including Mip-NeRF 360 and NeRF Synthetic by filtering hallucinatory artifacts while preserving plausible completions.

Conclusion: OracleGS successfully reconciles generative completeness with regressive fidelity by conditioning generative priors on multi-view geometric evidence.

Abstract: Sparse-view novel view synthesis is fundamentally ill-posed due to severe geometric ambiguity. Current methods are caught in a trade-off: regressive models are geometrically faithful but incomplete, whereas generative models can complete scenes but often introduce structural inconsistencies. We propose OracleGS, a novel framework that reconciles generative completeness with regressive fidelity for sparse view Gaussian Splatting. Instead of using generative models to patch incomplete reconstructions, our “propose-and-validate” framework first leverages a pre-trained 3D-aware diffusion model to synthesize novel views to propose a complete scene. We then repurpose a multi-view stereo (MVS) model as a 3D-aware oracle to validate the 3D uncertainties of generated views, using its attention maps to reveal regions where the generated views are well-supported by multi-view evidence versus where they fall into regions of high uncertainty due to occlusion, lack of texture, or direct inconsistency. This uncertainty signal directly guides the optimization of a 3D Gaussian Splatting model via an uncertainty-weighted loss. Our approach conditions the powerful generative prior on multi-view geometric evidence, filtering hallucinatory artifacts while preserving plausible completions in under-constrained regions, outperforming state-of-the-art methods on datasets including Mip-NeRF 360 and NeRF Synthetic.

Method: Uses an attention-guided U-Net architecture with focal and dice loss functions to handle rainfall class imbalance, integrating seven geospatial modalities at 1 km resolution: land surface temperature, NDVI, soil moisture, relative humidity, wind speed, elevation, and land use.

Result: The multimodal framework consistently outperforms unimodal baselines and existing deep learning methods, particularly in extreme rainfall categories.

Conclusion: This work provides a scalable framework, benchmark dataset, and state-of-the-art results for regional monsoon forecasting, climate resilience, and geospatial AI applications in India.

Abstract: Accurate monsoon rainfall prediction is vital for India’s agriculture, water management, and climate risk planning, yet remains challenging due to sparse ground observations and complex regional variability. We present a multimodal deep learning framework for high-resolution precipitation classification that leverages satellite and Earth observation data. Unlike previous rainfall prediction models based on coarse 5-50 km grids, we curate a new 1 km resolution dataset for five Indian states, integrating seven key geospatial modalities: land surface temperature, vegetation (NDVI), soil moisture, relative humidity, wind speed, elevation, and land use, covering the June-September 2024 monsoon season. Our approach uses an attention-guided U-Net architecture to capture spatial patterns and temporal dependencies across modalities, combined with focal and dice loss functions to handle rainfall class imbalance defined by the India Meteorological Department (IMD). Experiments demonstrate that our multimodal framework consistently outperforms unimodal baselines and existing deep learning methods, especially in extreme rainfall categories. This work contributes a scalable framework, benchmark dataset, and state-of-the-art results for regional monsoon forecasting, climate resilience, and geospatial AI applications in India.

Method: Uses synthetic data generation with structural extraction and domain-specific queries, adaptive instruction tuning for discriminative models, and recursive inferencing to iteratively refine outputs from both models.

Result: The framework demonstrates scalable, efficient, and precise document understanding, bridging the gap between domain-specific adaptation and general world knowledge for key information extraction.

Conclusion: SynDoc effectively addresses domain-specific VRDU challenges through its combined discriminative-generative approach with synthetic data and adaptive tuning, achieving stable and accurate predictions.

Abstract: Domain-specific Visually Rich Document Understanding (VRDU) presents significant challenges due to the complexity and sensitivity of documents in fields such as medicine, finance, and material science. Existing Large (Multimodal) Language Models (LLMs/MLLMs) achieve promising results but face limitations such as hallucinations, inadequate domain adaptation, and reliance on extensive fine-tuning datasets. This paper introduces SynDoc, a novel framework that combines discriminative and generative models to address these challenges. SynDoc employs a robust synthetic data generation workflow, using structural information extraction and domain-specific query generation to produce high-quality annotations. Through adaptive instruction tuning, SynDoc improves the discriminative model’s ability to extract domain-specific knowledge. At the same time, a recursive inferencing mechanism iteratively refines the output of both models for stable and accurate predictions. This framework demonstrates scalable, efficient, and precise document understanding and bridges the gap between domain-specific adaptation and general world knowledge for document key information extraction tasks.

Method: Introduces attention-suppressing adversarial attack that adds carefully crafted perturbations to images, explicitly targeting the attention mechanism of I2V models to disrupt motion synthesis.

Result: The immunized images generate stand-still or near-static videos, effectively blocking malicious content creation while preserving semantic fidelity of input images.

Conclusion: Attention attacks are a promising direction for robust and proactive defenses against misuse of I2V generation models, with Vid-Freeze demonstrating impressive protection capabilities.

Abstract: The rapid progress of image-to-video (I2V) generation models has introduced significant risks, enabling video synthesis from static images and facilitating deceptive or malicious content creation. While prior defenses such as I2VGuard attempt to immunize images, effective and principled protection to block motion remains underexplored. In this work, we introduce Vid-Freeze - a novel attention-suppressing adversarial attack that adds carefully crafted adversarial perturbations to images. Our method explicitly targets the attention mechanism of I2V models, completely disrupting motion synthesis while preserving semantic fidelity of the input image. The resulting immunized images generate stand-still or near-static videos, effectively blocking malicious content creation. Our experiments demonstrate the impressive protection provided by the proposed approach, highlighting the importance of attention attacks as a promising direction for robust and proactive defenses against misuse of I2V generation models.

Method: Proposes using defocus blur - a depth-dependent optical phenomenon naturally occurring in camera-captured images - as a discriminative feature. Constructs defocus blur maps to capture discrepancies between real images (with realistic depth-of-field) and synthetic images (lacking proper DoF characteristics).

Result: Experimental results confirm that defocus blur provides a reliable and interpretable cue for identifying synthetic images, supported by three in-depth feature analyses.

Conclusion: Defocus blur serves as an effective forensic signal for deepfake detection due to its universal origin from optical imaging principles and encoding of physical scene structure, making it robust and generalizable.

Abstract: The rapid advancement of generative AI has enabled the mass production of photorealistic synthetic images, blurring the boundary between authentic and fabricated visual content. This challenge is particularly evident in deepfake scenarios involving facial manipulation, but also extends to broader AI-generated content (AIGC) cases involving fully synthesized scenes. As such content becomes increasingly difficult to distinguish from reality, the integrity of visual media is under threat. To address this issue, we propose a physically interpretable deepfake detection framework and demonstrate that defocus blur can serve as an effective forensic signal. Defocus blur is a depth-dependent optical phenomenon that naturally occurs in camera-captured images due to lens focus and scene geometry. In contrast, synthetic images often lack realistic depth-of-field (DoF) characteristics. To capture these discrepancies, we construct a defocus blur map and use it as a discriminative feature for detecting manipulated content. Unlike RGB textures or frequency-domain signals, defocus blur arises universally from optical imaging principles and encodes physical scene structure. This makes it a robust and generalizable forensic cue. Our approach is supported by three in-depth feature analyses, and experimental results confirm that defocus blur provides a reliable and interpretable cue for identifying synthetic images. We aim for our defocus-based detection pipeline and interpretability tools to contribute meaningfully to ongoing research in media forensics. The implementation is publicly available at: https://github.com/irissun9602/Defocus-Deepfake-Detection

Method: Proposes Diff-SPK with Enhanced Texture from Inter-spike Interval (ETFI) to aggregate sparse information, then uses ETFI as conditioning input for ControlNet to generate high-speed scenes with an ETFI-based feature fusion module.

Result: Establishes the first bona fide benchmark for low-light spike stream reconstruction and demonstrates superior performance on real low-light spike streams compared to existing methods.

Conclusion: Diff-SPK effectively leverages generative priors to handle low-light high-speed spike streams and provides a significant advancement in spike camera reconstruction.

Abstract: Spike camera, a type of neuromorphic sensor with high-temporal resolution, shows great promise for high-speed visual tasks. Unlike traditional cameras, spike camera continuously accumulates photons and fires asynchronous spike streams. Due to unique data modality, spike streams require reconstruction methods to become perceptible to the human eye. However, lots of methods struggle to handle spike streams in low-light high-speed scenarios due to severe noise and sparse information. In this work, we propose Diff-SPK, the first diffusion-based reconstruction method for spike camera. Diff-SPK effectively leverages generative priors to supplement texture information in low-light conditions. Specifically, it first employs an \textbf{E}nhanced \textbf{T}exture \textbf{f}rom Inter-spike \textbf{I}nterval (ETFI) to aggregate sparse information from low-light spike streams. Then, ETFI serves as a conditioning input for ControlNet to generate the high-speed scenes. To improve the quality of results, we introduce an ETFI-based feature fusion module during the generation process. Moreover, we establish the first bona fide benchmark for the low-light spike stream reconstruction task. It significantly surpasses existing reconstruction datasets in scale and provides quantitative illumination information. The performance on real low-light spike streams demonstrates the superiority of Diff-SPK.

Method: Uses adaptive modality masking strategy to achieve modality-balanced data distribution, hierarchically guides feature extraction across CNN, Mamba, and transformer networks with fusion mechanisms, and employs mutual learning strategy for inter-branch collaboration.

Result: Achieves superior classification performance on four multimodal remote sensing datasets.

Conclusion: The BDGF framework effectively addresses modality imbalance in DDPMs and successfully leverages diffusion features for improved land-cover classification in multimodal remote sensing.

Abstract: Deep learning-based techniques for the analysis of multimodal remote sensing data have become popular due to their ability to effectively integrate complementary spatial, spectral, and structural information from different sensors. Recently, denoising diffusion probabilistic models (DDPMs) have attracted attention in the remote sensing community due to their powerful ability to capture robust and complex spatial-spectral distributions. However, pre-training multimodal DDPMs may result in modality imbalance, and effectively leveraging diffusion features to guide complementary diversity feature extraction remains an open question. To address these issues, this paper proposes a balanced diffusion-guided fusion (BDGF) framework that leverages multimodal diffusion features to guide a multi-branch network for land-cover classification. Specifically, we propose an adaptive modality masking strategy to encourage the DDPMs to obtain a modality-balanced rather than spectral image-dominated data distribution. Subsequently, these diffusion features hierarchically guide feature extraction among CNN, Mamba, and transformer networks by integrating feature fusion, group channel attention, and cross-attention mechanisms. Finally, a mutual learning strategy is developed to enhance inter-branch collaboration by aligning the probability entropy and feature similarity of individual subnetworks. Extensive experiments on four multimodal remote sensing datasets demonstrate that the proposed method achieves superior classification performance. The code is available at https://github.com/HaoLiu-XDU/BDGF.

Method: Introduced a diagnostic framework to probe VLM reasoning through classification and explanation analysis of fire-themed cultural imagery across Western festivals, non-Western traditions, and emergency scenes.

Result: Models correctly identify prominent Western festivals but struggle with underrepresented cultural events, offering vague labels or dangerously misclassifying emergencies as celebrations, exposing systematic biases.

Conclusion: Current VLMs rely on symbolic shortcuts rather than genuine cultural understanding, highlighting the need for cultural evaluation beyond accuracy metrics to ensure interpretable and fair multimodal systems.

Abstract: Vision-Language Models (VLMs) often appear culturally competent but rely on superficial pattern matching rather than genuine cultural understanding. We introduce a diagnostic framework to probe VLM reasoning on fire-themed cultural imagery through both classification and explanation analysis. Testing multiple models on Western festivals, non-Western traditions, and emergency scenes reveals systematic biases: models correctly identify prominent Western festivals but struggle with underrepresented cultural events, frequently offering vague labels or dangerously misclassifying emergencies as celebrations. These failures expose the risks of symbolic shortcuts and highlight the need for cultural evaluation beyond accuracy metrics to ensure interpretable and fair multimodal systems.

Method: A two-stage curriculum framework: Stage 1 uses RGBT data pretraining to enhance robustness, while Stage 2 employs vision-language alignment to improve generalization. An Exponential Moving Average (EMA) mechanism prevents catastrophic forgetting between stages by theoretically guaranteeing preservation of pre-stage performance.

Result: Achieves 80.1 AP50 on FLIR, 48.6 AP50_Novel on OV-COCO, and 35.7 mAPr on OV-LVIS, demonstrating competitive performance across both robustness and diversity evaluations.

Conclusion: C3-OWD successfully unifies robustness and generalization in object detection, establishing strong performance across multiple benchmarks and addressing the limitations of prior approaches that treated these challenges separately.

Abstract: Object detection has advanced significantly in the closed-set setting, but real-world deployment remains limited by two challenges: poor generalization to unseen categories and insufficient robustness under adverse conditions. Prior research has explored these issues separately: visible-infrared detection improves robustness but lacks generalization, while open-world detection leverages vision-language alignment strategy for category diversity but struggles under extreme environments. This trade-off leaves robustness and diversity difficult to achieve simultaneously. To mitigate these issues, we propose \textbf{C3-OWD}, a curriculum cross-modal contrastive learning framework that unifies both strengths. Stage1 enhances robustness by pretraining with RGBT data, while Stage2 improves generalization via vision-language alignment. To prevent catastrophic forgetting between two stages, we introduce an Exponential Moving Average (EMA) mechanism that theoretically guarantees preservation of pre-stage performance with bounded parameter lag and function consistency. Experiments on FLIR, OV-COCO, and OV-LVIS demonstrate the effectiveness of our approach: C3-OWD achieves $80.1$ AP$^{50}$ on FLIR, $48.6$ AP$^{50}_{\text{Novel}}$ on OV-COCO, and $35.7$ mAP$_r$ on OV-LVIS, establishing competitive performance across both robustness and diversity evaluations. Code available at: https://github.com/justin-herry/C3-OWD.git.

Method: Designed S2B-Conv with Spectral-Redistribution Mechanism (SRM) using dynamic affine transformation and Gabor Spatial Feature Amplifier (GSFA) with random frequency/angle selection to handle multi-scale anisotropic features.

Result: Extensive experiments show the high-efficiency binarized method achieves promising performance in both quantitative and qualitative evaluations.

Conclusion: S2BNet successfully applies binary neural networks to pansharpening while overcoming spectral and spatial challenges, offering an efficient solution for resource-limited devices.

Abstract: Remote sensing pansharpening aims to reconstruct spatial-spectral properties during the fusion of panchromatic (PAN) images and low-resolution multi-spectral (LR-MS) images, finally generating the high-resolution multi-spectral (HR-MS) images. Although deep learning-based models have achieved excellent performance, they often come with high computational complexity, which hinder their applications on resource-limited devices. In this paper, we explore the feasibility of applying the binary neural network (BNN) to pan-sharpening. Nevertheless, there are two main issues with binarizing pan-sharpening models: (i) the binarization will cause serious spectral distortion due to the inconsistent spectral distribution of the PAN/LR-MS images; (ii) the common binary convolution kernel is difficult to adapt to the multi-scale and anisotropic spatial features of remote sensing objects, resulting in serious degradation of contours. To address the above issues, we design the customized spatial-spectral binarized convolution (S2B-Conv), which is composed of the Spectral-Redistribution Mechanism (SRM) and Gabor Spatial Feature Amplifier (GSFA). Specifically, SRM employs an affine transformation, generating its scaling and bias parameters through a dynamic learning process. GSFA, which randomly selects different frequencies and angles within a preset range, enables to better handle multi-scale and-directional spatial features. A series of S2B-Conv form a brand-new binary network for pan-sharpening, dubbed as S2BNet. Extensive quantitative and qualitative experiments have shown our high-efficiency binarized pan-sharpening method can attain a promising performance.

Method: Decouple reasoning and perception processes - a powerful LLM orchestrates high-level reasoning and strategically interrogates a LMM to extract specific visual information needed for logical chains.

Result: Significant reduction in visually-unfounded reasoning steps and substantial improvement in reasoning fidelity, validated through comprehensive evaluations.

Conclusion: The lightweight, plug-and-play approach effectively governs visual reasoning process without requiring additional training or architectural changes.

Abstract: Significant advancements in the reasoning capabilities of Large Language Models (LLMs) are now driven by test-time scaling laws, particularly those leveraging extended Chain-of-Thought (CoT) reasoning. Inspired by these breakthroughs, researchers have extended these paradigms to Large Multimodal Models (LMMs). However, a critical limitation emerges: as their reasoning chains extend, LMMs increasingly rely on textual logic, progressively losing grounding in the underlying visual information. This leads to reasoning paths that diverge from the image content, culminating in erroneous conclusions. To address this, we introduce a strikingly simple yet effective training-free visual-reasoning pipeline. The core concept is to decouple the reasoning and perception processes. A powerful LLM orchestrates the high-level reasoning, strategically interrogating a LMM to extract specific visual information required for its logical chain. The LMM, in turn, functions exclusively as a visual question-answering engine, supplying the necessary perceptual details on demand. This lightweight, plug-and-play approach requires no additional training or architectural changes. Comprehensive evaluations validate that our framework effectively governs the visual reasoning process, leading to a significant reduction in visually-unfounded reasoning steps and a substantial improvement in reasoning fidelity.

Method: DDP uses class-specific positive-negative prompts to disentangle semantics and introduces Progressive Confidence Decoupling (PCD) to suppress false positives. It freezes past prompts as knowledge anchors and employs interlayer prompting for efficiency.

Result: DDP consistently outperforms prior methods on MS-COCO and PASCAL VOC, becoming the first replay-free MLCIL approach to exceed 80% mAP and 70% F1 under the standard MS-COCO B40-C10 benchmark.

Conclusion: The proposed DDP framework effectively addresses the core challenges in multi-label class-incremental learning and demonstrates superior performance compared to existing methods.

Abstract: Prompt-based methods have shown strong effectiveness in single-label class-incremental learning, but their direct extension to multi-label class-incremental learning (MLCIL) performs poorly due to two intrinsic challenges: semantic confusion from co-occurring categories and true-negative-false-positive confusion caused by partial labeling. We propose Dual-Decoupled Prompting (DDP), a replay-free and parameter-efficient framework that explicitly addresses both issues. DDP assigns class-specific positive-negative prompts to disentangle semantics and introduces Progressive Confidence Decoupling (PCD), a curriculum-inspired decoupling strategy that suppresses false positives. Past prompts are frozen as knowledge anchors, and interlayer prompting enhances efficiency. On MS-COCO and PASCAL VOC, DDP consistently outperforms prior methods and is the first replay-free MLCIL approach to exceed 80% mAP and 70% F1 under the standard MS-COCO B40-C10 benchmark.

Method: Proposes Likelihood-Regularized Policy Optimization (LRPO) with three key strategies: composite reward function, ground-truth guided likelihood regularization, and noise-level advantage assignment.

Result: LRPO significantly improves face restoration quality over baseline methods and achieves state-of-the-art performance.

Conclusion: The proposed LRPO framework effectively addresses BFR challenges and demonstrates superior restoration performance through reinforcement learning with proper regularization.

Abstract: Blind Face Restoration (BFR) encounters inherent challenges in exploring its large solution space, leading to common artifacts like missing details and identity ambiguity in the restored images. To tackle these challenges, we propose a Likelihood-Regularized Policy Optimization (LRPO) framework, the first to apply online reinforcement learning (RL) to the BFR task. LRPO leverages rewards from sampled candidates to refine the policy network, increasing the likelihood of high-quality outputs while improving restoration performance on low-quality inputs. However, directly applying RL to BFR creates incompatibility issues, producing restoration results that deviate significantly from the ground truth. To balance perceptual quality and fidelity, we propose three key strategies: 1) a composite reward function tailored for face restoration assessment, 2) ground-truth guided likelihood regularization, and 3) noise-level advantage assignment. Extensive experiments demonstrate that our proposed LRPO significantly improves the face restoration quality over baseline methods and achieves state-of-the-art performance.

Method: Developed using a comprehensive bilingual dataset of 110,447 images and 2.46 million visual question-answering pairs, capable of interpreting seven 2D oral imaging modalities across 36 diagnostic tasks.

Result: Outperformed leading proprietary and open-source models by 19.6% higher accuracy for oral diseases and 27.9% for malocclusions. In clinical study with 25 dentists and 1,946 patients, surpassed junior dentists on 21/36 tasks and senior dentists on 12/36 tasks. Reduced diagnostic time by 15-22% when integrated into collaborative workflow.

Conclusion: DentVLM establishes itself as a robust clinical decision support tool that can enhance primary dental care, mitigate provider-patient imbalances, and democratize access to specialized medical expertise in dentistry.

Abstract: Diagnosing and managing oral diseases necessitate advanced visual interpretation across diverse imaging modalities and integrated information synthesis. While current AI models excel at isolated tasks, they often fall short in addressing the complex, multimodal requirements of comprehensive clinical dental practice. Here we introduce DentVLM, a multimodal vision-language model engineered for expert-level oral disease diagnosis. DentVLM was developed using a comprehensive, large-scale, bilingual dataset of 110,447 images and 2.46 million visual question-answering (VQA) pairs. The model is capable of interpreting seven 2D oral imaging modalities across 36 diagnostic tasks, significantly outperforming leading proprietary and open-source models by 19.6% higher accuracy for oral diseases and 27.9% for malocclusions. In a clinical study involving 25 dentists, evaluating 1,946 patients and encompassing 3,105 QA pairs, DentVLM surpassed the diagnostic performance of 13 junior dentists on 21 of 36 tasks and exceeded that of 12 senior dentists on 12 of 36 tasks. When integrated into a collaborative workflow, DentVLM elevated junior dentists’ performance to senior levels and reduced diagnostic time for all practitioners by 15-22%. Furthermore, DentVLM exhibited promising performance across three practical utility scenarios, including home-based dental health management, hospital-based intelligent diagnosis and multi-agent collaborative interaction. These findings establish DentVLM as a robust clinical decision support tool, poised to enhance primary dental care, mitigate provider-patient imbalances, and democratize access to specialized medical expertise within the field of dentistry.

Method: Proposes Dynamic-TreeRPO with sliding-window tree-structured search using dynamic noise intensities, GRPO-guided optimization, constrained SDE sampling, and LayerTuning-RL that reformulates SFT loss as weighted progress reward model with dynamic clipping bounds.

Result: Significantly outperforms state-of-the-art by 4.9% on HPS-v2.1, 5.91% on PickScore, and 8.66% on ImageReward benchmarks, while improving training efficiency by nearly 50%.

Conclusion: The tree-structured sampling and LayerTuning-RL paradigm enable dynamic exploration of diverse search spaces along effective directions, achieving superior semantic consistency, visual fidelity, and human preference alignment.

Abstract: The integration of Reinforcement Learning (RL) into flow matching models for text-to-image (T2I) generation has driven substantial advances in generation quality. However, these gains often come at the cost of exhaustive exploration and inefficient sampling strategies due to slight variation in the sampling group. Building on this insight, we propose Dynamic-TreeRPO, which implements the sliding-window sampling strategy as a tree-structured search with dynamic noise intensities along depth. We perform GRPO-guided optimization and constrained Stochastic Differential Equation (SDE) sampling within this tree structure. By sharing prefix paths of the tree, our design effectively amortizes the computational overhead of trajectory search. With well-designed noise intensities for each tree layer, Dynamic-TreeRPO can enhance the variation of exploration without any extra computational cost. Furthermore, we seamlessly integrate Supervised Fine-Tuning (SFT) and RL paradigm within Dynamic-TreeRPO to construct our proposed LayerTuning-RL, reformulating the loss function of SFT as a dynamically weighted Progress Reward Model (PRM) rather than a separate pretraining method. By associating this weighted PRM with dynamic-adaptive clipping bounds, the disruption of exploration process in Dynamic-TreeRPO is avoided. Benefiting from the tree-structured sampling and the LayerTuning-RL paradigm, our model dynamically explores a diverse search space along effective directions. Compared to existing baselines, our approach demonstrates significant superiority in terms of semantic consistency, visual fidelity, and human preference alignment on established benchmarks, including HPS-v2.1, PickScore, and ImageReward. In particular, our model outperforms SoTA by $4.9%$, $5.91%$, and $8.66%$ on those benchmarks, respectively, while improving the training efficiency by nearly $50%$.

Method: Leverages transformation equivariance property to analyze prediction variance under spatial perturbations, theoretically decomposing uncertainty into intrinsic spread (epistemic noise) and bias jitter (systematic error drift).

Result: Across four anatomical structures and multiple registration models, uncertainty maps consistently correlate with registration errors and identify regions requiring caution.

Conclusion: The framework enables any pretrained registration network to become risk-aware at test time, advancing safe deployment in clinical and research settings.

Abstract: Accurate image registration is essential for downstream applications, yet current deep registration networks provide limited indications of whether and when their predictions are reliable. Existing uncertainty estimation strategies, such as Bayesian methods, ensembles, or MC dropout, require architectural changes or retraining, limiting their applicability to pretrained registration networks. Instead, we propose a test-time uncertainty estimation framework that is compatible with any pretrained networks. Our framework is grounded in the transformation equivariance property of registration, which states that the true mapping between two images should remain consistent under spatial perturbations of the input. By analyzing the variance of network predictions under such perturbations, we derive a theoretical decomposition of perturbation-based uncertainty in registration. This decomposition separates into two terms: (i) an intrinsic spread, reflecting epistemic noise, and (ii) a bias jitter, capturing how systematic error drifts under perturbations. Across four anatomical structures (brain, cardiac, abdominal, and lung) and multiple registration models (uniGradICON, SynthMorph), the uncertainty maps correlate consistently with registration errors and highlight regions requiring caution. Our framework turns any pretrained registration network into a risk-aware tool at test time, placing medical image registration one step closer to safe deployment in clinical and large-scale research settings.

Method: GRAPE incorporates ranking signals into retrieval-guided query rewriting with LLMs using Grouped Ranking-Aware Policy Optimization (GRPO). It addresses score inflation through corpus-relative ranking-based rewards that align optimization with ranking metrics.

Result: GRAPE consistently improves retrieval performance across various distribution shifts: multilingual (Flickr30k-CN, CVLUE, XM3600), length differences (Wikipedia), and multimodal differences (CIRR), achieving 4.9% average improvement in Recall@10.

Conclusion: GRAPE effectively bridges distributional gaps in CLIP-based retrieval systems by incorporating ranking signals into query rewriting, demonstrating significant performance improvements across diverse distribution shifts without requiring costly retraining.

Abstract: The CLIP model has become a cornerstone of large-scale retrieval systems by aligning text and image data in a unified embedding space. Despite its simplicity and efficiency, CLIP struggles when applied to tasks whose input distributions diverge from its training corpus, such as queries with multilingual, long-form, or multimodal differences. To avoid costly retraining, existing methods mainly adopt query-rewriting strategies with large language models (LLMs), aiming to mitigate distribution gaps at the query level. However, due to the lack of supervision signals, LLMs fail to generate the optimal one that fits the training distribution. We address this challenge with GRAPE (Grouped Ranking-Aware Policy Optimization Enhancement), a plug-and-play enhancement approach that incorporates ranking signals into retrieval-guided query rewriting with LLMs. Intuitively, GRAPE proposes to leverage GRPO to bridge distributional differences – including length, multilingual, and modality shifts – by transforming queries into forms better aligned with the retriever’s training distribution. However, our preliminary experiment finds that naively finetuning LLM with similarity scores can lead to score inflation, where nearly all candidates are assigned unexpectedly high scores regardless of their true relevance. To address score inflation, we propose a corpus-relative ranking-based reward, which explicitly aligns optimization with ranking metrics while suppressing spurious score inflation. Extensive experiments demonstrate that GRAPE consistently improves retrieval performance under distributional shifts – including multilingual differences (Flickr30k-CN, CVLUE, XM3600), length differences (Wikipedia), and multimodal differences (CIRR) – achieving an average improvement of 4.9% in Recall@10. The code is available at https://github.com/Chinese0123456/GRAPE.git

Method: Uses cascaded networks with two stages: Shape Reconstruction (generates auxiliary information) and Fused Completion (leverages information with knowledge distillation). A teacher model trained on denser point clouds transfers knowledge to the student model.

Result: Outperforms existing methods on ShapeNet-55 under different difficulty settings, demonstrating superior shape recovery and detail preservation.

Conclusion: The cascaded structure and distillation strategy effectively bridge the gap between incomplete inputs and complete targets by capturing global shape context while refining local details.

Abstract: Point clouds collected from real-world environments are often incomplete due to factors such as limited sensor resolution, single viewpoints, occlusions, and noise. These challenges make point cloud completion essential for various applications. A key difficulty in this task is predicting the overall shape and reconstructing missing regions from highly incomplete point clouds. To address this, we introduce CasPoinTr, a novel point cloud completion framework using cascaded networks and knowledge distillation. CasPoinTr decomposes the completion task into two synergistic stages: Shape Reconstruction, which generates auxiliary information, and Fused Completion, which leverages this information alongside knowledge distillation to generate the final output. Through knowledge distillation, a teacher model trained on denser point clouds transfers incomplete-complete associative knowledge to the student model, enhancing its ability to estimate the overall shape and predict missing regions. Together, the cascaded networks and knowledge distillation enhance the model’s ability to capture global shape context while refining local details, effectively bridging the gap between incomplete inputs and complete targets. Experiments on ShapeNet-55 under different difficulty settings demonstrate that CasPoinTr outperforms existing methods in shape recovery and detail preservation, highlighting the effectiveness of our cascaded structure and distillation strategy.

Method: Uses self-supervised learning on RGB-D sequences, transferring 2D HPE annotations to 3D via spatial-temporal constraints, 2D-to-3D back-projection loss, and cross-modality interaction. Employs anchor-to-joint prediction for 3D lifting on RGB and depth networks.

Result: Achieves comparable performance to fully supervised methods on CMU Panoptic and ITOP datasets. Incorporation of large-scale unlabeled data (NTU RGB+D 60) enhances performance under challenging conditions.

Conclusion: UniPose demonstrates potential for practical applications and achieves state-of-the-art results with its proposed 3D lifting method.

Abstract: In this paper, we present UniPose, a unified cross-modality pose prior propagation method for weakly supervised 3D human pose estimation (HPE) using unannotated single-view RGB-D sequences (RGB, depth, and point cloud data). UniPose transfers 2D HPE annotations from large-scale RGB datasets (e.g., MS COCO) to the 3D domain via self-supervised learning on easily acquired RGB-D sequences, eliminating the need for labor-intensive 3D keypoint annotations. This approach bridges the gap between 2D and 3D domains without suffering from issues related to multi-view camera calibration or synthetic-to-real data shifts. During training, UniPose leverages off-the-shelf 2D pose estimations as weak supervision for point cloud networks, incorporating spatial-temporal constraints like body symmetry and joint motion. The 2D-to-3D back-projection loss and cross-modality interaction further enhance this process. By treating the point cloud network’s 3D HPE results as pseudo ground truth, our anchor-to-joint prediction method performs 3D lifting on RGB and depth networks, making it more robust against inaccuracies in 2D HPE results compared to state-of-the-art methods. Experiments on CMU Panoptic and ITOP datasets show that UniPose achieves comparable performance to fully supervised methods. Incorporating large-scale unlabeled data (e.g., NTU RGB+D 60) enhances its performance under challenging conditions, demonstrating its potential for practical applications. Our proposed 3D lifting method also achieves state-of-the-art results.

Method: Created Goliath-SC dataset with 383K self-contact poses across 130 subjects. Proposed body-part-wise latent diffusion with self-attention for generative modeling of self-contact prior conditioned by body shape parameters. Incorporated this prior into single-view pose estimation with contact refinement.

Result: Shape conditioning proved vital for successful modeling of self-contact pose distribution. The approach improved single-view pose estimation in self-contact scenarios.

Conclusion: Body shape conditioning is essential for accurate self-contact modeling, and the proposed method effectively enhances pose estimation by incorporating shape-aware self-contact priors.

Abstract: One can hardly model self-contact of human poses without considering underlying body shapes. For example, the pose of rubbing a belly for a person with a low BMI leads to penetration of the hand into the belly for a person with a high BMI. Despite its relevance, existing self-contact datasets lack the variety of self-contact poses and precise body shapes, limiting conclusive analysis between self-contact poses and shapes. To address this, we begin by introducing the first extensive self-contact dataset with precise body shape registration, Goliath-SC, consisting of 383K self-contact poses across 130 subjects. Using this dataset, we propose generative modeling of self-contact prior conditioned by body shape parameters, based on a body-part-wise latent diffusion with self-attention. We further incorporate this prior into single-view human pose estimation while refining estimated poses to be in contact. Our experiments suggest that shape conditioning is vital to the successful modeling of self-contact pose distribution, hence improving single-view pose estimation in self-contact.

Method: Two-step approach: (1) 4D-aware latent diffusion model that integrates multi-modal information to produce pixel-aligned 4D Gaussians, (2) Refinement of novel view videos using an enhanced video diffusion model.

Result: Extensive experiments show WorldSplat effectively generates high-fidelity, temporally and spatially consistent multi-track novel view driving videos on benchmark datasets.

Conclusion: WorldSplat successfully overcomes the dilemma between scene generation and reconstruction, providing a unified framework for 4D driving-scene generation with superior novel-view synthesis capabilities.

Abstract: Recent advances in driving-scene generation and reconstruction have demonstrated significant potential for enhancing autonomous driving systems by producing scalable and controllable training data. Existing generation methods primarily focus on synthesizing diverse and high-fidelity driving videos; however, due to limited 3D consistency and sparse viewpoint coverage, they struggle to support convenient and high-quality novel-view synthesis (NVS). Conversely, recent 3D/4D reconstruction approaches have significantly improved NVS for real-world driving scenes, yet inherently lack generative capabilities. To overcome this dilemma between scene generation and reconstruction, we propose \textbf{WorldSplat}, a novel feed-forward framework for 4D driving-scene generation. Our approach effectively generates consistent multi-track videos through two key steps: ((i)) We introduce a 4D-aware latent diffusion model integrating multi-modal information to produce pixel-aligned 4D Gaussians in a feed-forward manner. ((ii)) Subsequently, we refine the novel view videos rendered from these Gaussians using a enhanced video diffusion model. Extensive experiments conducted on benchmark datasets demonstrate that \textbf{WorldSplat} effectively generates high-fidelity, temporally and spatially consistent multi-track novel view driving videos.

Method: Proposes Fracture-YOLO with two key modules: CRSelector that uses global texture information to focus on critical fracture region features, and ScA module that dynamically adjusts weights of features at different scales to enhance multi-scale fracture detection.

Result: Experimental results show significant improvement over baseline model, with mAP50 and mAP50-95 increasing by 4 and 3 respectively, achieving state-of-the-art performance.

Conclusion: The proposed Fracture-YOLO model effectively enhances fracture detection performance through attention mechanisms and scale-aware feature processing, demonstrating superior results compared to baseline models.

Abstract: Fracture detection plays a critical role in medical imaging analysis, traditional fracture diagnosis relies on visual assessment by experienced physicians, however the speed and accuracy of this approach are constrained by the expertise. With the rapid advancements in artificial intelligence, deep learning models based on the YOLO framework have been widely employed for fracture detection, demonstrating significant potential in improving diagnostic efficiency and accuracy. This study proposes an improved YOLO-based model, termed Fracture-YOLO, which integrates novel Critical-Region-Selector Attention (CRSelector) and Scale-Aware (ScA) heads to further enhance detection performance. Specifically, the CRSelector module utilizes global texture information to focus on critical features of fracture regions. Meanwhile, the ScA module dynamically adjusts the weights of features at different scales, enhancing the model’s capacity to identify fracture targets at multiple scales. Experimental results demonstrate that, compared to the baseline model, Fracture-YOLO achieves a significant improvement in detection precision, with mAP50 and mAP50-95 increasing by 4 and 3, surpassing the baseline model and achieving state-of-the-art (SOTA) performance.

Method: FracDetNet integrates Dual-Focus Attention (DFA) to capture detailed local features and comprehensive global context through combined global and local attention mechanisms, and Multi-scale Calibration (MC) to adaptively refine feature representations.

Result: On the GRAZPEDWRI-DX dataset, FracDetNet achieved state-of-the-art performance with mAP${50-95}$ of 40.0% (7.5% improvement over baseline), mAP${50}$ of 63.9% (4.2% improvement), and fracture-specific detection accuracy enhanced by 2.9%.

Conclusion: FracDetNet effectively addresses challenges in fracture detection through its attention mechanisms and feature calibration, demonstrating significant performance improvements over baseline methods.

Abstract: In this paper, an advanced fracture detection framework, FracDetNet, is proposed to address challenges in medical imaging, as accurate fracture detection is essential for enhancing diagnostic efficiency in clinical practice. Despite recent advancements, existing methods still struggle with detecting subtle and morphologically diverse fractures due to variable imaging angles and suboptimal image quality. To overcome these limitations, FracDetNet integrates Dual-Focus Attention (DFA) and Multi-scale Calibration (MC). Specifically, the DFA module effectively captures detailed local features and comprehensive global context through combined global and local attention mechanisms. Additionally, the MC adaptively refines feature representations to enhance detection performance. Experimental evaluations on the publicly available GRAZPEDWRI-DX dataset demonstrate state-of-the-art performance, with FracDetNet achieving a mAP${50-95}$ of 40.0%, reflecting a \textbf{7.5%} improvement over the baseline model. Furthermore, the mAP${50}$ reaches 63.9%, representing an increase of \textbf{4.2%}, with fracture-specific detection accuracy also enhanced by \textbf{2.9%}.

Method: SPIKE quantifies Bayesian Surprise as belief updates from new visual evidence. SPIKE-RL uses GRPO to optimize belief hypotheses based on video caption rewards, enabling surprise-weighted frame sampling.

Result: SPIKE effectively localizes surprise in videos, correlating with human judgment on FunQA and Oops! benchmarks. Surprise-weighted sampling achieves consistent performance gains on five downstream benchmarks over uniform sampling.

Conclusion: By enabling Video-LLMs to track beliefs and register surprise, this work paves the way for more robust models that can revise their understanding in response to new information.

Abstract: Real-world videos often show routine activities punctuated by memorable, surprising events. However, most Video-LLMs process videos by sampling frames uniformly, likely missing critical moments that define a video’s narrative. We introduce SPIKE, an inference-time framework that quantifies Bayesian Surprise as the belief update triggered by new visual evidence in the video stream, identifying moments where new visual evidence conflicts with prior beliefs. SPIKE effectively localizes surprise in videos, strongly correlated with humans on positive (FunQA) and negative (Oops!) surprise benchmarks. Since the beliefs of zero-shot Video-LLMs are often suboptimal, we develop SPIKE-RL, which leverages GRPO to optimize belief hypotheses based on a reward signal from the video caption. SPIKE and SPIKE-RL guide query-agnostic surprise-weighted frame sampling, which allocates more frames to interesting moments in the video. With this strategy, we achieve consistent performance gains on five downstream benchmarks over uniform sampling. By enabling Video-LLMs to track beliefs and register surprise, our work paves the way for more robust models that can revise their understanding in response to new information.

Method: Propose FM-SIREN and FM-FINER with Nyquist-informed, neuron-specific frequency multipliers for periodic activations, introducing frequency diversity without hyperparameter tuning or additional network depth.

Result: Reduces feature redundancy by nearly 50% and consistently improves signal reconstruction across 1D audio, 2D image, 3D shape fitting, and NeRF synthesis, outperforming baseline counterparts while maintaining efficiency.

Conclusion: The simple yet principled modification of neuron-specific frequency multipliers effectively reduces feature redundancy and enhances the performance of periodic activation-based INR networks.

Abstract: Existing periodic activation-based implicit neural representation (INR) networks, such as SIREN and FINER, suffer from hidden feature redundancy, where neurons within a layer capture overlapping frequency components due to the use of a fixed frequency multiplier. This redundancy limits the expressive capacity of multilayer perceptrons (MLPs). Drawing inspiration from classical signal processing methods such as the Discrete Sine Transform (DST), we propose FM-SIREN and FM-FINER, which assign Nyquist-informed, neuron-specific frequency multipliers to periodic activations. Unlike existing approaches, our design introduces frequency diversity without requiring hyperparameter tuning or additional network depth. This simple yet principled modification reduces the redundancy of features by nearly 50% and consistently improves signal reconstruction across diverse INR tasks, including fitting 1D audio, 2D image and 3D shape, and synthesis of neural radiance fields (NeRF), outperforming their baseline counterparts while maintaining efficiency.

Method: Extends SLD framework by evaluating text-image alignment using FoR, extracts spatial configuration from images, maps spatial expressions to camera perspectives, and applies latent-space operations to adjust facing direction and depth.

Result: Improves state-of-the-art T2I models by up to 5.3% on spatial understanding benchmarks using only a single correction round.

Conclusion: Integrating Frame of Reference into multimodal models significantly enhances their ability to handle spatial reasoning in text-to-image generation.

Abstract: Frame of Reference (FoR) is a fundamental concept in spatial reasoning that humans utilize to comprehend and describe space. With the rapid progress in Multimodal Language models, the moment has come to integrate this long-overlooked dimension into these models. In particular, in text-to-image (T2I) generation, even state-of-the-art models exhibit a significant performance gap when spatial descriptions are provided from perspectives other than the camera. To address this limitation, we propose Frame of Reference-guided Spatial Adjustment in LLM-based Diffusion Editing (FoR-SALE), an extension of the Self-correcting LLM-controlled Diffusion (SLD) framework for T2I. For-Sale evaluates the alignment between a given text and an initially generated image, and refines the image based on the Frame of Reference specified in the spatial expressions. It employs vision modules to extract the spatial configuration of the image, while simultaneously mapping the spatial expression to a corresponding camera perspective. This unified perspective enables direct evaluation of alignment between language and vision. When misalignment is detected, the required editing operations are generated and applied. FoR-SALE applies novel latent-space operations to adjust the facing direction and depth of the generated images. We evaluate FoR-SALE on two benchmarks specifically designed to assess spatial understanding with FoR. Our framework improves the performance of state-of-the-art T2I models by up to 5.3% using only a single round of correction.

Method: Uses a hybrid encoder combining graph convolutional networks (local skeletal features) and transformers (global context) via gated cross-attention. Predicts continuous 6D rotation mapped to SO(3) matrix for pose alignment. Trained self-supervised on MM-Fi dataset with synthetic rotations.

Result: Reduces mean rotation error from over 20° to 3.4° and Mean Per Joint Position Error from ~64 mm to 47 mm on MM-Fi benchmark. Qualitative evaluations on TotalCapture show acceleration signals from video correspond well with ground-truth IMU data.

Conclusion: The module effectively removes viewpoint variability, enabling physically plausible motion analysis by producing consistent body-centered poses.

Abstract: Monocular 3D pose estimators produce camera-centered skeletons, creating view-dependent kinematic signals that complicate comparative analysis in applications such as health and sports science. We present 3DPCNet, a compact, estimator-agnostic module that operates directly on 3D joint coordinates to rectify any input pose into a consistent, body-centered canonical frame. Its hybrid encoder fuses local skeletal features from a graph convolutional network with global context from a transformer via a gated cross-attention mechanism. From this representation, the model predicts a continuous 6D rotation that is mapped to an $SO(3)$ matrix to align the pose. We train the model in a self-supervised manner on the MM-Fi dataset using synthetically rotated poses, guided by a composite loss ensuring both accurate rotation and pose reconstruction. On the MM-Fi benchmark, 3DPCNet reduces the mean rotation error from over 20$^{\circ}$ to 3.4$^{\circ}$ and the Mean Per Joint Position Error from ~64 mm to 47 mm compared to a geometric baseline. Qualitative evaluations on the TotalCapture dataset further demonstrate that our method produces acceleration signals from video that show strong visual correspondence to ground-truth IMU sensor data, confirming that our module removes viewpoint variability to enable physically plausible motion analysis.

Method: Decomposes input prompts into semantic concepts and evaluates alignment at both global and concept levels using fine-grained CLIP. Uses concept-level feedback to drive iterative prompt refinement via large language models.

Result: Significantly improves concept coverage and human-judged faithfulness over standard test-time optimization and base T2I models on DrawBench and CompBench prompts.

Conclusion: The proposed fine-grained test-time optimization framework effectively enhances compositional faithfulness in text-to-image generation through concept-level alignment and iterative prompt refinement.

Abstract: Despite recent advances in text-to-image (T2I) models, they often fail to faithfully render all elements of complex prompts, frequently omitting or misrepresenting specific objects and attributes. Test-time optimization has emerged as a promising approach to address this limitation by refining generation without the need for retraining. In this paper, we propose a fine-grained test-time optimization framework that enhances compositional faithfulness in T2I generation. Unlike most of prior approaches that rely solely on a global image/text similarity score, our method decomposes the input prompt into semantic concepts and evaluates alignment at both the global and concept levels. A fine-grained variant of CLIP is used to compute concept-level correspondence, producing detailed feedback on missing or inaccurate concepts. This feedback is fed into an iterative prompt refinement loop, enabling the large language model to propose improved prompts. Experiments on DrawBench and CompBench prompts demonstrate that our method significantly improves concept coverage and human-judged faithfulness over both standard test-time optimization and the base T2I model. Code is available at: https://github.com/AmirMansurian/NoConceptLeftBehind

Method: Proposes three key components: 1) Complementary feature extraction to handle missing modalities, 2) Uncertainty-based pseudo labeling to reduce noise, and 3) Adaptive loss weighting mechanism to prevent model degradation during unstable adaptations.

Result: Extensive experiments demonstrate state-of-the-art performance and effectiveness in handling domain adaptation challenges in multi-modal face anti-spoofing.

Conclusion: MFAS-DANet successfully addresses the three major challenges in multi-modal FAS domain adaptation and achieves superior performance compared to existing methods.

Abstract: Recent multi-modal face anti-spoofing (FAS) methods have investigated the potential of leveraging multiple modalities to distinguish live and spoof faces. However, pre-adapted multi-modal FAS models often fail to detect unseen attacks from new target domains. Although a more realistic domain adaptation (DA) scenario has been proposed for single-modal FAS to learn specific spoof attacks during inference, DA remains unexplored in multi-modal FAS methods. In this paper, we propose a novel framework, MFAS-DANet, to address three major challenges in multi-modal FAS under the DA scenario: missing modalities, noisy pseudo labels, and model degradation. First, to tackle the issue of missing modalities, we propose extracting complementary features from other modalities to substitute missing modality features or enhance existing ones. Next, to reduce the impact of noisy pseudo labels during model adaptation, we propose deriving reliable pseudo labels by leveraging prediction uncertainty across different modalities. Finally, to prevent model degradation, we design an adaptive mechanism that decreases the loss weight during unstable adaptations and increasing it during stable ones. Extensive experiments demonstrate the effectiveness and state-of-the-art performance of our proposed MFAS-DANet.

Method: Proposes RestoRect using rectified flow to reformulate feature distillation as a generative process. Combines Retinex theory for physics-based decomposition with learnable anisotropic diffusion constraints and trigonometric color space polarization. Introduces Feature Layer Extraction loss for robust knowledge transfer between different architectures through cross-normalized transformer feature alignment with percentile-based outlier detection.

Result: Achieves better training stability, faster convergence and inference while preserving restoration quality. Demonstrates superior results across 15 image restoration datasets, covering 4 tasks, on 8 metrics.

Conclusion: RestoRect effectively addresses the speed-quality trade-off in image restoration through its novel latent rectified flow feature distillation approach, enabling practical deployment of high-quality restoration models.

Abstract: Current approaches for restoration of degraded images face a critical trade-off: high-performance models are too slow for practical use, while fast models produce poor results. Knowledge distillation transfers teacher knowledge to students, but existing static feature matching methods cannot capture how modern transformer architectures dynamically generate features. We propose ‘RestoRect’, a novel Latent Rectified Flow Feature Distillation method for restoring degraded images. We apply rectified flow to reformulate feature distillation as a generative process where students learn to synthesize teacher-quality features through learnable trajectories in latent space. Our framework combines Retinex theory for physics-based decomposition with learnable anisotropic diffusion constraints, and trigonometric color space polarization. We introduce a Feature Layer Extraction loss for robust knowledge transfer between different network architectures through cross-normalized transformer feature alignment with percentile-based outlier detection. RestoRect achieves better training stability, and faster convergence and inference while preserving restoration quality. We demonstrate superior results across 15 image restoration datasets, covering 4 tasks, on 8 metrics.

Method: Estimates a Global Orientation Field to propagate principal directions across space and time, then introduces Orientation-aware Hyper-Gaussian that embeds time, space, geometry, and orientation into a unified probabilistic state for inferring region-specific deformations.

Result: Experiments show superior reconstruction fidelity over mainstream methods in challenging real-world dynamic scenes.

Conclusion: OriGS effectively addresses complex dynamic modeling by leveraging orientation-based guidance and principled conditioned slicing for adaptive local dynamics capture.

Abstract: We present Orientation-anchored Gaussian Splatting (OriGS), a novel framework for high-quality 4D reconstruction from casually captured monocular videos. While recent advances extend 3D Gaussian Splatting to dynamic scenes via various motion anchors, such as graph nodes or spline control points, they often rely on low-rank assumptions and fall short in modeling complex, region-specific deformations inherent to unconstrained dynamics. OriGS addresses this by introducing a hyperdimensional representation grounded in scene orientation. We first estimate a Global Orientation Field that propagates principal forward directions across space and time, serving as stable structural guidance for dynamic modeling. Built upon this, we propose Orientation-aware Hyper-Gaussian, a unified formulation that embeds time, space, geometry, and orientation into a coherent probabilistic state. This enables inferring region-specific deformation through principled conditioned slicing, adaptively capturing diverse local dynamics in alignment with global motion intent. Experiments demonstrate the superior reconstruction fidelity of OriGS over mainstream methods in challenging real-world dynamic scenes.

Method: Conducted large-scale empirical study across 23 visual question-answering benchmarks using multi-modal large language models (MLLMs), covering domains including general/expert knowledge reasoning, OCR, and document understanding.

Result: Found significant variation in reliance on vision, text, and their interaction across benchmarks; discovered that many benchmarks intended to reduce text-only biases have instead amplified image-only dependencies; this pattern persists across model sizes with larger models using intra-modality dependencies to achieve high performance that masks lack of true multi-modal reasoning.

Conclusion: Provides quantitative characterization of multi-modal datasets to enable principled approach to multi-modal benchmark design and evaluation, revealing current limitations in assessing genuine multi-modal reasoning capabilities.

Abstract: Understanding the interplay between intra-modality dependencies (the contribution of an individual modality to a target task) and inter-modality dependencies (the relationships between modalities and the target task) is fundamental to advancing multi-modal learning. However, the nature of and interaction between these dependencies within current benchmark evaluations remains poorly characterized. In this work, we present a large-scale empirical study to quantify these dependencies across 23 visual question-answering benchmarks using multi-modal large language models (MLLMs) covering domains such as general and expert knowledge reasoning, optical character recognition, and document understanding. Our findings show that the reliance on vision, question (text), and their interaction varies significantly, both across and within benchmarks. We discover that numerous benchmarks intended to mitigate text-only biases have inadvertently amplified image-only dependencies. This characterization persists across model sizes, as larger models often use these intra-modality dependencies to achieve high performance that mask an underlying lack of multi-modal reasoning. We provide a quantitative characterization of multi-modal datasets, enabling a principled approach to multi-modal benchmark design and evaluation.

Method: Proposes Dynamic Kernel mechanism initialized by global context vector from Encoder Attention module, iteratively refines predictions across decoding stages, and uses Unified Channel Adaptation in decoder to standardize feature dimensions.

Result: Outperforms state-of-the-art methods on KvasirSEG and CVC ClinicDB datasets, achieving superior Dice and IoU scores while reducing computational costs without compromising accuracy.

Conclusion: Provides a robust and adaptable solution for polyp segmentation with promising applications in clinical and automated diagnostic systems.

Abstract: Polyp segmentation is a critical step in colorectal cancer detection, yet it remains challenging due to the diverse shapes, sizes, and low contrast boundaries of polyps in medical imaging. In this work, we propose a novel framework that improves segmentation accuracy and efficiency by integrating a Dynamic Kernel (DK) mechanism with a global Encoder Attention module. The DK mechanism, initialized by a global context vector from the EA module, iteratively refines segmentation predictions across decoding stages, enabling the model to focus on and accurately delineate complex polyp boundaries. The EA module enhances the network’s ability to capture critical lesion features by aggregating multi scale information from all encoder layers. In addition, we employ Unified Channel Adaptation (UCA) in the decoder to standardize feature dimensions across stages, ensuring consistent and computationally efficient information fusion. Our approach extends the lesion-aware kernel framework by introducing a more flexible, attention driven kernel initialization and a unified decoder design. Extensive experiments on the KvasirSEG and CVC ClinicDB benchmark datasets demonstrate that our model outperforms several state of the art segmentation methods, achieving superior Dice and Intersection over Union scores. Moreover, UCA simplifies the decoder structure, reducing computational cost without compromising accuracy. Overall, the proposed method provides a robust and adaptable solution for polyp segmentation, with promising applications in clinical and automated diagnostic systems.

Method: Created ActPLD benchmark to evaluate action processing in MLLMs using single-actor and socially interacting point-light displays. Tested state-of-the-art proprietary and open-source systems.

Result: Models showed consistently low performance across all tested conditions, revealing fundamental gaps in action and spatiotemporal understanding.

Conclusion: Current MLLMs have significant limitations in processing human action from minimal visual motion cues, highlighting critical gaps in their action understanding capabilities compared to humans.

Abstract: Humans can extract rich semantic information from minimal visual cues, as demonstrated by point-light displays (PLDs), which consist of sparse sets of dots localized to key joints of the human body. This ability emerges early in development and is largely attributed to human embodied experience. Since PLDs isolate body motion as the sole source of meaning, they represent key stimuli for testing the constraints of action understanding in these systems. Here we introduce ActPLD, the first benchmark to evaluate action processing in MLLMs from human PLDs. Tested models include state-of-the-art proprietary and open-source systems on single-actor and socially interacting PLDs. Our results reveal consistently low performance across models, introducing fundamental gaps in action and spatiotemporal understanding.

Method: Used a large-scale longitudinal analysis of 2,125 CT scans from SSc patients. Applied radiomics and deep learning with pre-trained models (ResNet-18, DenseNet-121, Swin Transformer) fine-tuned on the dataset. Conducted mortality analysis at 1, 3, and 5 years using death labels confirmed by expert physicians.

Result: The models achieved AUCs of 0.769 for 1-year mortality prediction, 0.801 for 3-year mortality, and 0.709 for 5-year mortality. The dataset included 181, 326, and 428 CT scans from patients who died within 1, 3, and 5 years respectively.

Conclusion: Both radiomics and deep learning computational methods show significant potential for improving early detection and risk assessment of SSc-related interstitial lung disease, representing a major advancement in the field.

Abstract: Interstitial lung disease (ILD) is a leading cause of morbidity and mortality in systemic sclerosis (SSc). Chest computed tomography (CT) is the primary imaging modality for diagnosing and monitoring lung complications in SSc patients. However, its role in disease progression and mortality prediction has not yet been fully clarified. This study introduces a novel, large-scale longitudinal chest CT analysis framework that utilizes radiomics and deep learning to predict mortality associated with lung complications of SSc. We collected and analyzed 2,125 CT scans from SSc patients enrolled in the Northwestern Scleroderma Registry, conducting mortality analyses at one, three, and five years using advanced imaging analysis techniques. Death labels were assigned based on recorded deaths over the one-, three-, and five-year intervals, confirmed by expert physicians. In our dataset, 181, 326, and 428 of the 2,125 CT scans were from patients who died within one, three, and five years, respectively. Using ResNet-18, DenseNet-121, and Swin Transformer we use pre-trained models, and fine-tuned on 2,125 images of SSc patients. Models achieved an AUC of 0.769, 0.801, 0.709 for predicting mortality within one-, three-, and five-years, respectively. Our findings highlight the potential of both radiomics and deep learning computational methods to improve early detection and risk assessment of SSc-related interstitial lung disease, marking a significant advancement in the literature.

Method: Resource-aware adaptive super-resolution framework that optimizes for model calibration and high precision-recall on critical events, with a lightweight artifact detector to filter SR-induced hallucinations.

Result: Achieves state-of-the-art performance: best calibration (ECE 5.8% vs 6.2% baseline), highest AUPR for drowsiness detection (0.78 vs 0.74), superior precision-recall for phone use detection (0.74 vs 0.71), with minimal overhead (0.3M parameters, 5.2ms).

Conclusion: While LR-trained models serve as strong general-purpose baselines, the adaptive framework represents state-of-the-art for safety-critical applications where reliability is paramount.

Abstract: Driver monitoring systems require not just high accuracy but reliable, well-calibrated confidence scores for safety-critical deployment. While direct low-resolution training yields high overall accuracy, it produces poorly calibrated predictions that can be dangerous in safety-critical scenarios. We propose a resource-aware adaptive super-resolution framework that optimizes for model calibration and high precision-recall on critical events. Our approach achieves state-of-the-art performance on safety-centric metrics: best calibration (ECE of 5.8% vs 6.2% for LR-trained baselines), highest AUPR for drowsiness detection (0.78 vs 0.74), and superior precision-recall for phone use detection (0.74 vs 0.71). A lightweight artifact detector (0.3M parameters, 5.2ms overhead) provides additional safety by filtering SR-induced hallucinations. While LR-trained video models serve as strong general-purpose baselines, our adaptive framework represents the state-of-the-art solution for safety-critical applications where reliability is paramount.

Method: Projects 2D instance masks from open-vocabulary 2D models onto 3D scenes using reconstruction correspondences, uses View-wise Instance Partition to handle partial annotations, and employs 2D Instance Boundary-aware Superpoint clustering to preserve object boundaries.

Result: Achieves +2.3 mAP improvement on ScanNet200 benchmark and +7.1 mAP gain on novel classes under open-vocabulary setting, significantly reducing performance gap between tail and head classes.

Conclusion: OVSeg3R effectively extends closed-vocabulary 3D segmentation to open-vocabulary scenarios by leveraging 2D models and 3D reconstruction, demonstrating substantial performance improvements particularly for novel classes.

Abstract: In this paper, we propose a training scheme called OVSeg3R to learn open-vocabulary 3D instance segmentation from well-studied 2D perception models with the aid of 3D reconstruction. OVSeg3R directly adopts reconstructed scenes from 2D videos as input, avoiding costly manual adjustment while aligning input with real-world applications. By exploiting the 2D to 3D correspondences provided by 3D reconstruction models, OVSeg3R projects each view’s 2D instance mask predictions, obtained from an open-vocabulary 2D model, onto 3D to generate annotations for the view’s corresponding sub-scene. To avoid incorrectly introduced false positives as supervision due to partial annotations from 2D to 3D, we propose a View-wise Instance Partition algorithm, which partitions predictions to their respective views for supervision, stabilizing the training process. Furthermore, since 3D reconstruction models tend to over-smooth geometric details, clustering reconstructed points into representative super-points based solely on geometry, as commonly done in mainstream 3D segmentation methods, may overlook geometrically non-salient objects. We therefore introduce 2D Instance Boundary-aware Superpoint, which leverages 2D masks to constrain the superpoint clustering, preventing superpoints from violating instance boundaries. With these designs, OVSeg3R not only extends a state-of-the-art closed-vocabulary 3D instance segmentation model to open-vocabulary, but also substantially narrows the performance gap between tail and head classes, ultimately leading to an overall improvement of +2.3 mAP on the ScanNet200 benchmark. Furthermore, under the standard open-vocabulary setting, OVSeg3R surpasses previous methods by about +7.1 mAP on the novel classes, further validating its effectiveness.

Method: The survey organizes around unified research questions examining 3DGS’s technical advantages, adaptability to different input modalities and domain constraints, and remaining limitations through systematic comparison of domain-specific pipelines.

Result: The analysis shows that 3DGS effectively balances photorealism, geometric fidelity, and computational efficiency, making it suitable for various applications beyond just rendering, including perception and content creation.

Conclusion: 3DGS offers a compelling alternative to NeRF that supports high-quality rendering, faster optimization, and integration into hybrid pipelines, providing a roadmap for leveraging neural rendering across real and virtual environments for both synthesis and practical applications.

Abstract: Neural scene representations such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have transformed how 3D environments are modeled, rendered, and interpreted. NeRF introduced view-consistent photorealism via volumetric rendering; 3DGS has rapidly emerged as an explicit, efficient alternative that supports high-quality rendering, faster optimization, and integration into hybrid pipelines for enhanced photorealism and task-driven scene understanding. This survey examines how 3DGS is being adopted across SLAM, telepresence and teleoperation, robotic manipulation, and 3D content generation. Despite their differences, these domains share common goals: photorealistic rendering, meaningful 3D structure, and accurate downstream tasks. We organize the review around unified research questions that explain why 3DGS is increasingly displacing NeRF-based approaches: What technical advantages drive its adoption? How does it adapt to different input modalities and domain-specific constraints? What limitations remain? By systematically comparing domain-specific pipelines, we show that 3DGS balances photorealism, geometric fidelity, and computational efficiency. The survey offers a roadmap for leveraging neural rendering not only for image synthesis but also for perception, interaction, and content creation across real and virtual environments.

Method: Privacy-preserving FL framework with evaluation of 3 segmentation architectures (U-Net, Attention U-Net, Swin UNETR) paired with 2 FL algorithms (FedAvg, FedProx), plus novel anatomically-informed loss function for region-specific texture contrasts.

Result: Attention U-Net with FedAvg identified as optimal for pancreatic heterogeneity, achieving clinically viable performance on distributed, heterogeneous datasets.

Conclusion: The approach successfully addresses both technical complexity of pancreas part delineation in MRI and data scarcity problem through federated learning, enabling collaborative training without direct data sharing.

Abstract: We present the first federated learning (FL) approach for pancreas part(head, body and tail) segmentation in MRI, addressing a critical clinical challenge as a significant innovation. Pancreatic diseases exhibit marked regional heterogeneity cancers predominantly occur in the head region while chronic pancreatitis causes tissue loss in the tail, making accurate segmentation of the organ into head, body, and tail regions essential for precise diagnosis and treatment planning. This segmentation task remains exceptionally challenging in MRI due to variable morphology, poor soft-tissue contrast, and anatomical variations across patients. Our novel contribution tackles two fundamental challenges: first, the technical complexity of pancreas part delineation in MRI, and second the data scarcity problem that has hindered prior approaches. We introduce a privacy-preserving FL framework that enables collaborative model training across seven medical institutions without direct data sharing, leveraging a diverse dataset of 711 T1W and 726 T2W MRI scans. Our key innovations include: (1) a systematic evaluation of three state-of-the-art segmentation architectures (U-Net, Attention U-Net,Swin UNETR) paired with two FL algorithms (FedAvg, FedProx), revealing Attention U-Net with FedAvg as optimal for pancreatic heterogeneity, which was never been done before; (2) a novel anatomically-informed loss function prioritizing region-specific texture contrasts in MRI. Comprehensive evaluation demonstrates that our approach achieves clinically viable performance despite training on distributed, heterogeneous datasets.

Method: Proposes NeuroAdapter framework that directly conditions latent diffusion models on brain representations, and introduces IBBI interpretability framework to analyze cross-attention mechanisms across diffusion denoising steps.

Result: Achieves competitive visual reconstruction quality on public fMRI datasets while providing greater transparency into how brain signals influence the generation process.

Conclusion: Demonstrates potential of end-to-end brain-to-image decoding and establishes path toward interpreting diffusion models through visual neuroscience lens.

Abstract: Recent work has demonstrated that complex visual stimuli can be decoded from human brain activity using deep generative models, helping brain science researchers interpret how the brain represents real-world scenes. However, most current approaches leverage mapping brain signals into intermediate image or text feature spaces before guiding the generative process, masking the effect of contributions from different brain areas on the final reconstruction output. In this work, we propose NeuroAdapter, a visual decoding framework that directly conditions a latent diffusion model on brain representations, bypassing the need for intermediate feature spaces. Our method demonstrates competitive visual reconstruction quality on public fMRI datasets compared to prior work, while providing greater transparency into how brain signals shape the generation process. To this end, we contribute an Image-Brain BI-directional interpretability framework (IBBI) which investigates cross-attention mechanisms across diffusion denoising steps to reveal how different cortical areas influence the unfolding generative trajectory. Our results highlight the potential of end-to-end brain-to-image decoding and establish a path toward interpreting diffusion models through the lens of visual neuroscience.

Method: Proposes RobuQ framework with: 1) Strong ternary weight baseline (W1.58A4), 2) RobustQuantizer using Hadamard transform for robust activation quantization, and 3) AMPN - Activation-only Mixed-Precision Network pipeline with ternary weights and layer-wise activation precisions.

Result: Achieves state-of-the-art performance for DiT quantization in sub-4-bit configurations, with stable image generation on ImageNet-1K using average 2-bit activations.

Conclusion: RobuQ successfully addresses the activation quantization bottleneck in DiTs, enabling practical deployment of low-bit quantized diffusion models while maintaining competitive performance.

Abstract: Diffusion Transformers (DiTs) have recently emerged as a powerful backbone for image generation, demonstrating superior scalability and performance over U-Net architectures. However, their practical deployment is hindered by substantial computational and memory costs. While Quantization-Aware Training (QAT) has shown promise for U-Nets, its application to DiTs faces unique challenges, primarily due to the sensitivity and distributional complexity of activations. In this work, we identify activation quantization as the primary bottleneck for pushing DiTs to extremely low-bit settings. To address this, we propose a systematic QAT framework for DiTs, named RobuQ. We start by establishing a strong ternary weight (W1.58A4) DiT baseline. Building upon this, we propose RobustQuantizer to achieve robust activation quantization. Our theoretical analyses show that the Hadamard transform can convert unknown per-token distributions into per-token normal distributions, providing a strong foundation for this method. Furthermore, we propose AMPN, the first Activation-only Mixed-Precision Network pipeline for DiTs. This method applies ternary weights across the entire network while allocating different activation precisions to each layer to eliminate information bottlenecks. Through extensive experiments on unconditional and conditional image generation, our RobuQ framework achieves state-of-the-art performance for DiT quantization in sub-4-bit quantization configuration. To the best of our knowledge, RobuQ is the first achieving stable and competitive image generation on large datasets like ImageNet-1K with activations quantized to average 2 bits. The code and models will be available at https://github.com/racoonykc/RobuQ .

Method: Built on pretrained WANX video generation model, uses single-step flow matching for direct mapping from degraded inputs to high-quality outputs. Employs Joint Latent-Pixel Face-Focused Training with stochastic switching between facial region optimization and global reconstruction, plus MLLM-driven data curation pipeline for automated dataset selection.

Result: Achieves state-of-the-art results in perceptual quality, identity preservation, and temporal stability, with significantly reduced inference time compared to traditional methods.

Conclusion: VividFace provides an efficient and effective solution for video face enhancement that overcomes key limitations of existing approaches while offering practical resources for the research community.

Abstract: Video Face Enhancement (VFE) seeks to reconstruct high-quality facial regions from degraded video sequences, a capability that underpins numerous applications including video conferencing, film restoration, and surveillance. Despite substantial progress in the field, current methods that primarily rely on video super-resolution and generative frameworks continue to face three fundamental challenges: (1) faithfully modeling intricate facial textures while preserving temporal consistency; (2) restricted model generalization due to the lack of high-quality face video training data; and (3) low efficiency caused by repeated denoising steps during inference. To address these challenges, we propose VividFace, a novel and efficient one-step diffusion framework for video face enhancement. Built upon the pretrained WANX video generation model, our method leverages powerful spatiotemporal priors through a single-step flow matching paradigm, enabling direct mapping from degraded inputs to high-quality outputs with significantly reduced inference time. To further boost efficiency, we propose a Joint Latent-Pixel Face-Focused Training strategy that employs stochastic switching between facial region optimization and global reconstruction, providing explicit supervision in both latent and pixel spaces through a progressive two-stage training process. Additionally, we introduce an MLLM-driven data curation pipeline for automated selection of high-quality video face datasets, enhancing model generalization. Extensive experiments demonstrate that VividFace achieves state-of-the-art results in perceptual quality, identity preservation, and temporal stability, while offering practical resources for the research community.

Method: Multi-level heterogeneous knowledge transfer (MHKT) network that migrates FSCM knowledge at feature, distribution and category levels. Uses task-associated information selector (TAIS) for feature migration, maximum discrimination divergence (MDD) metric for distribution alignment, and category relation consistency constraint to handle data imbalance.

Result: Extensive experiments on two new datasets combining FSCM data and measured SAR images demonstrate superior performance compared to existing methods.

Conclusion: The proposed method effectively migrates purer target knowledge from FSCM data through stepwise knowledge selection and migration, achieving better SAR target recognition performance while addressing the limited sample problem.

Abstract: Simulated data-assisted SAR target recognition methods are the research hotspot currently, devoted to solving the problem of limited samples. Existing works revolve around simulated images, but the large amount of irrelevant information embedded in the images, such as background, noise, etc., seriously affects the quality of the migrated information. Our work explores a new simulated data to migrate purer and key target knowledge, i.e., forward scattering center model (FSCM) which models the actual local structure of the target with strong physical meaning and interpretability. To achieve this purpose, multi-level heterogeneous knowledge transfer (MHKT) network is proposed, which fully migrates FSCM knowledge from the feature, distribution and category levels, respectively. Specifically, we permit the more suitable feature representations for the heterogeneous data and separate non-informative knowledge by task-associated information selector (TAIS), to complete purer target feature migration. In the distribution alignment, the new metric function maximum discrimination divergence (MDD) in target generic knowledge transfer (TGKT) module perceives transferable knowledge efficiently while preserving discriminative structure about classes. Moreover, category relation knowledge transfer (CRKT) module leverages the category relation consistency constraint to break the dilemma of optimization bias towards simulation data due to imbalance between simulated and measured data. Such stepwise knowledge selection and migration will ensure the integrity of the migrated FSCM knowledge. Notably, extensive experiments on two new datasets formed by FSCM data and measured SAR images demonstrate the superior performance of our method.

Method: Proposes VAMamba with two key components: 1) QCLAM (Queue-based Cache Low-rank Adaptive Memory) uses a FIFO cache to store historical representations and intelligently fuses current LoRA-adapted features with cached ones; 2) GPS-SS2D (Greedy Path Scan SS2D) uses a Vision Transformer to generate score maps for pixel importance estimation and a greedy strategy to determine optimal forward/backward scanning paths.

Result: Extensive experiments across diverse restoration tasks demonstrate that VAMamba consistently outperforms existing approaches in both restoration quality and efficiency, establishing new benchmarks for adaptive image restoration.

Conclusion: VAMamba successfully addresses the limitations of conventional Mamba architectures by enabling adaptive feature utilization and scanning, achieving superior performance in image restoration tasks while maintaining computational efficiency.

Abstract: Recent Mamba-based image restoration methods have achieved promising results but remain limited by fixed scanning patterns and inefficient feature utilization. Conventional Mamba architectures rely on predetermined paths that cannot adapt to diverse degradations, constraining both restoration performance and computational efficiency. To overcome these limitations, we propose VAMamba, a Visual Adaptive Mamba framework with two key innovations. First, QCLAM(Queue-basedCacheLow-rankAdaptiveMemory)enhancesfeaturelearningthrougha FIFO cache that stores historical representations. Similarity between current LoRA-adapted and cached features guides intelligent fusion, enabling dynamic reuse while effectively controlling memorygrowth.Second, GPS-SS2D(GreedyPathScanSS2D)introducesadaptive scanning. A Vision Transformer generates score maps to estimate pixel importance, and a greedy strategy de termines optimal forward and backward scanning paths. These learned trajectories replace rigid patterns, enabling SS2D to perform targeted feature extraction. The integration of QCLAM and GPS-SS2D allows VAMamba to adaptively focus on degraded regions while maintaining high computational efficiency. Extensive experiments across diverse restoration tasks demonstrate that VAMamba consistently outperforms existing approaches in both restoration quality and efficiency, establishing new benchmarks for adaptive image restoration. Our code is available at https://github.com/WaterHQH/VAMamba.

Method: Optimizes a large latent taxonomic hierarchy using complete binary tree structured mixture-of-Gaussian prior within variational inference framework, analytically showing ELBO optimization encourages discovery of hierarchical relationships among prototypes.

Result: Learned models demonstrate strong hierarchical clustering performance, outperforming baselines across diverse image classification datasets using novel evaluation mechanism that leverages prototype clusters discovered at all hierarchical levels.

Conclusion: Deep taxonomic networks discover rich and interpretable hierarchical taxonomies capturing both coarse-grained semantic categories and fine-grained visual distinctions, bridging gaps in current hierarchical clustering methods.

Abstract: Inspired by the human ability to learn and organize knowledge into hierarchical taxonomies with prototypes, this paper addresses key limitations in current deep hierarchical clustering methods. Existing methods often tie the structure to the number of classes and underutilize the rich prototype information available at intermediate hierarchical levels. We introduce deep taxonomic networks, a novel deep latent variable approach designed to bridge these gaps. Our method optimizes a large latent taxonomic hierarchy, specifically a complete binary tree structured mixture-of-Gaussian prior within a variational inference framework, to automatically discover taxonomic structures and associated prototype clusters directly from unlabeled data without assuming true label sizes. We analytically show that optimizing the ELBO of our method encourages the discovery of hierarchical relationships among prototypes. Empirically, our learned models demonstrate strong hierarchical clustering performance, outperforming baselines across diverse image classification datasets using our novel evaluation mechanism that leverages prototype clusters discovered at all hierarchical levels. Qualitative results further reveal that deep taxonomic networks discover rich and interpretable hierarchical taxonomies, capturing both coarse-grained semantic categories and fine-grained visual distinctions.

Method: Uses a latent diffusion approach with a perceptually-optimized autoencoder and attention-based conditional U-Net for fast deterministic denoising in compressed latent space.

Result: Achieves superior perceptual quality over CNN/GAN methods, rivals heavy diffusion models like DDPM/Dn-Dp, and is 60x faster than pixel-space diffusion models while maintaining competitive PSNR/SSIM scores.

Conclusion: MAN bridges the gap between high fidelity and clinical viability, demonstrating a practical path for advanced generative models in medical imaging.

Abstract: While diffusion models have set a new benchmark for quality in Low-Dose Computed Tomography (LDCT) denoising, their clinical adoption is critically hindered by extreme computational costs, with inference times often exceeding thousands of seconds per scan. To overcome this barrier, we introduce MAN, a Latent Diffusion Enhanced Multistage Anti-Noise Network for Efficient and High-Quality Low-Dose CT Image Denoising task. Our method operates in a compressed latent space via a perceptually-optimized autoencoder, enabling an attention-based conditional U-Net to perform the fast, deterministic conditional denoising diffusion process with drastically reduced overhead. On the LDCT and Projection dataset, our model achieves superior perceptual quality, surpassing CNN/GAN-based methods while rivaling the reconstruction fidelity of computationally heavy diffusion models like DDPM and Dn-Dp. Most critically, in the inference stage, our model is over 60x faster than representative pixel space diffusion denoisers, while remaining competitive on PSNR/SSIM scores. By bridging the gap between high fidelity and clinical viability, our work demonstrates a practical path forward for advanced generative models in medical imaging.

Method: Proposes Visual Mixing Diffusion (VMDiff) with: (1) hybrid sampling combining guided denoising, inversion, and spherical interpolation for structure-aware fusion; (2) adaptive adjustment module with similarity-based score to automatically search optimal parameters.

Result: Experiments on 780 concept pairs show VMDiff outperforms baselines in visual quality, semantic consistency, and human-rated creativity.

Conclusion: VMDiff effectively addresses coexistent and bias generation challenges in image fusion, providing a robust framework for creating coherent novel images from multiple visual sources.

Abstract: Creating novel images by fusing visual cues from multiple sources is a fundamental yet underexplored problem in image-to-image generation, with broad applications in artistic creation, virtual reality and visual media. Existing methods often face two key challenges: coexistent generation, where multiple objects are simply juxtaposed without true integration, and bias generation, where one object dominates the output due to semantic imbalance. To address these issues, we propose Visual Mixing Diffusion (VMDiff), a simple yet effective diffusion-based framework that synthesizes a single, coherent object by integrating two input images at both noise and latent levels. Our approach comprises: (1) a hybrid sampling process that combines guided denoising, inversion, and spherical interpolation with adjustable parameters to achieve structure-aware fusion, mitigating coexistent generation; and (2) an efficient adaptive adjustment module, which introduces a novel similarity-based score to automatically and adaptively search for optimal parameters, countering semantic bias. Experiments on a curated benchmark of 780 concept pairs demonstrate that our method outperforms strong baselines in visual quality, semantic consistency, and human-rated creativity.

Method: Uses a collection of differentiable 3D LUTs with different priors, a lightweight content-aware network to predict fusion weights for scene-adaptive color correction, and an iterative flow matching method to restore local structural details and eliminate artifacts.

Result: Extensive experiments demonstrate effectiveness on three benchmarks, achieving scene-adaptive color correction with O(1) complexity while maintaining perceptual and structural fidelity.

Conclusion: FlowLUT successfully integrates the efficiency of LUTs with multiple priors and flow matching, providing an end-to-end solution that balances computational efficiency with enhanced representational capacity for image enhancement.

Abstract: Deep learning-based image enhancement methods face a fundamental trade-off between computational efficiency and representational capacity. For example, although a conventional three-dimensional Look-Up Table (3D LUT) can process a degraded image in real time, it lacks representational flexibility and depends solely on a fixed prior. To address this problem, we introduce FlowLUT, a novel end-to-end model that integrates the efficiency of LUTs, multiple priors, and the parameter-independent characteristic of flow-matched reconstructed images. Specifically, firstly, the input image is transformed in color space by a collection of differentiable 3D LUTs (containing a large number of 3D LUTs with different priors). Subsequently, a lightweight content-aware dynamically predicts fusion weights, enabling scene-adaptive color correction with $\mathcal{O}(1)$ complexity. Next, a lightweight fusion prediction network runs on multiple 3D LUTs, with $\mathcal{O}(1)$ complexity for scene-adaptive color correction.Furthermore, to address the inherent representation limitations of LUTs, we design an innovative iterative flow matching method to restore local structural details and eliminate artifacts. Finally, the entire model is jointly optimized under a composite loss function enforcing perceptual and structural fidelity. Extensive experimental results demonstrate the effectiveness of our method on three benchmarks.

Method: Uses 3D visual grounding for object identification, hand-object joint affordance learning for contact prediction, and optimization with local-scene modeling and collision avoidance constraints.

Result: Method generates physically plausible, text-compliant interactions and outperforms existing approaches in comprehensive experiments.

Conclusion: The proposed approach effectively addresses the challenging task of text-controlled human-object interaction with movable objects in 3D scenes.

Abstract: We propose a novel task of text-controlled human object interaction generation in 3D scenes with movable objects. Existing human-scene interaction datasets suffer from insufficient interaction categories and typically only consider interactions with static objects (do not change object positions), and the collection of such datasets with movable objects is difficult and costly. To address this problem, we construct the InteractMove dataset for Movable Human-Object Interaction in 3D Scenes by aligning existing human object interaction data with scene contexts, featuring three key characteristics: 1) scenes containing multiple movable objects with text-controlled interaction specifications (including same-category distractors requiring spatial and 3D scene context understanding), 2) diverse object types and sizes with varied interaction patterns (one-hand, two-hand, etc.), and 3) physically plausible object manipulation trajectories. With the introduction of various movable objects, this task becomes more challenging, as the model needs to identify objects to be interacted with accurately, learn to interact with objects of different sizes and categories, and avoid collisions between movable objects and the scene. To tackle such challenges, we propose a novel pipeline solution. We first use 3D visual grounding models to identify the interaction object. Then, we propose a hand-object joint affordance learning to predict contact regions for different hand joints and object parts, enabling accurate grasping and manipulation of diverse objects. Finally, we optimize interactions with local-scene modeling and collision avoidance constraints, ensuring physically plausible motions and avoiding collisions between objects and the scene. Comprehensive experiments demonstrate our method’s superiority in generating physically plausible, text-compliant interactions compared to existing approaches.

Method: BioVessel-Net integrates vessel biostatistics with adversarial refinement and a radius-guided segmentation strategy, directly modeling vascular structures with biostatistical coherence. The authors also introduce RetinaMix, a new benchmark dataset of 2D and 3D OCTA images.

Result: BioVessel-Net achieves near-perfect segmentation accuracy across RetinaMix and existing datasets, substantially outperforming state-of-the-art supervised and semi-supervised methods.

Conclusion: BioVessel-Net and RetinaMix provide a label-free, computationally efficient, and clinically interpretable solution for retinal vessel analysis, with broad potential for glaucoma monitoring, blood flow modeling, and progression prediction.

Abstract: Structural changes in retinal blood vessels are critical biomarkers for the onset and progression of glaucoma and other ocular diseases. However, current vessel segmentation approaches largely rely on supervised learning and extensive manual annotations, which are costly, error-prone, and difficult to obtain in optical coherence tomography angiography. Here we present BioVessel-Net, an unsupervised generative framework that integrates vessel biostatistics with adversarial refinement and a radius-guided segmentation strategy. Unlike pixel-based methods, BioVessel-Net directly models vascular structures with biostatistical coherence, achieving accurate and explainable vessel extraction without labeled data or high-performance computing. To support training and evaluation, we introduce RetinaMix, a new benchmark dataset of 2D and 3D OCTA images with high-resolution vessel details from diverse populations. Experimental results demonstrate that BioVessel-Net achieves near-perfect segmentation accuracy across RetinaMix and existing datasets, substantially outperforming state-of-the-art supervised and semi-supervised methods. Together, BioVessel-Net and RetinaMix provide a label-free, computationally efficient, and clinically interpretable solution for retinal vessel analysis, with broad potential for glaucoma monitoring, blood flow modeling, and progression prediction. Code and dataset are available: https://github.com/VikiXie/SatMar8.

Method: Proposes DiffInk with two components: (1) InkVAE - a sequential VAE with OCR-based loss for glyph accuracy and style-classification loss for style preservation, creating a disentangled latent space; (2) InkDiT - a latent diffusion Transformer that integrates text and style references to generate pen trajectories.

Result: Outperforms state-of-the-art methods in both glyph accuracy and style fidelity, while significantly improving generation efficiency.

Conclusion: DiffInk successfully addresses limitations of existing methods by enabling full-line handwriting generation with improved accuracy, style preservation, and efficiency.

Abstract: Deep generative models have advanced text-to-online handwriting generation (TOHG), which aims to synthesize realistic pen trajectories conditioned on textual input and style references. However, most existing methods still primarily focus on character- or word-level generation, resulting in inefficiency and a lack of holistic structural modeling when applied to full text lines. To address these issues, we propose DiffInk, the first latent diffusion Transformer framework for full-line handwriting generation. We first introduce InkVAE, a novel sequential variational autoencoder enhanced with two complementary latent-space regularization losses: (1) an OCR-based loss enforcing glyph-level accuracy, and (2) a style-classification loss preserving writing style. This dual regularization yields a semantically structured latent space where character content and writer styles are effectively disentangled. We then introduce InkDiT, a novel latent diffusion Transformer that integrates target text and reference styles to generate coherent pen trajectories. Experimental results demonstrate that DiffInk outperforms existing state-of-the-art methods in both glyph accuracy and style fidelity, while significantly improving generation efficiency. Code will be made publicly available.

Method: Two novel mechanisms: 1) Introspection Training - trains a model to identify errors including logical errors; 2) Recursive Inference - alternating unmask→introspection→remask process repeated until reliable results.

Result: Experimental results show RIV achieves state-of-the-art performance, outperforming most existing MDVLMs on multiple benchmarks.

Conclusion: The proposed RIV framework successfully equips MDVLMs with self-correction capability through recursive introspection, significantly improving model reliability and performance.

Abstract: Mask Diffusion-based Vision Language Models (MDVLMs) have achieved remarkable progress in multimodal understanding tasks. However, these models are unable to correct errors in generated tokens, meaning they lack self-correction capability. In this paper, we propose Recursive Introspection Mask Diffusion Vision Language Model (RIV), which equips the model with self-correction ability through two novel mechanisms. The first is Introspection Training, where an Introspection Model is introduced to identify errors within generated sequences. Introspection Training enables the model to detect not only grammatical and spelling mistakes, but more importantly, logical errors. The second is Recursive Inference. Beginning with the standard unmasking step, the learned Introspection Model helps to identify errors in the output sequence and remask them. This alternating ($\text{unmask}\rightarrow\text{introspection}\rightarrow\text{remask}$) process is repeated recursively until reliable results are obtained. Experimental results on multiple benchmarks demonstrate that the proposed RIV achieves state-of-the-art performance, outperforming most existing MDVLMs.

Method: Leverages Vision Foundation Models (VFMs) as teachers in a self-training paradigm to generate high-quality pseudo-labels for target domains, distilling their generalization capabilities into a single efficient student network.

Result: Achieves state-of-the-art performance on synthetic-to-real UDA benchmarks and a challenging day-to-night adaptation task. Lightweight variant is 10x smaller than foundation models while maintaining SOTA accuracy.

Conclusion: FAMDA enables training of highly efficient, domain-adaptive models suitable for resource-constrained robotics applications by effectively bridging the gap between foundation model capabilities and practical deployment needs.

Abstract: Multi-task dense prediction, which aims to jointly solve tasks like semantic segmentation and depth estimation, is crucial for robotics applications but suffers from domain shift when deploying models in new environments. While unsupervised domain adaptation (UDA) addresses this challenge for single tasks, existing multi-task UDA methods primarily rely on adversarial learning approaches that are less effective than recent self-training techniques. In this paper, we introduce FAMDA, a simple yet effective UDA framework that bridges this gap by leveraging Vision Foundation Models (VFMs) as powerful teachers. Our approach integrates Segmentation and Depth foundation models into a self-training paradigm to generate high-quality pseudo-labels for the target domain, effectively distilling their robust generalization capabilities into a single, efficient student network. Extensive experiments show that FAMDA achieves state-of-the-art (SOTA) performance on standard synthetic-to-real UDA multi-task learning (MTL) benchmarks and a challenging new day-to-night adaptation task. Our framework enables the training of highly efficient models; a lightweight variant achieves SOTA accuracy while being more than 10$\times$ smaller than foundation models, highlighting FAMDA’s suitability for creating domain-adaptive and efficient models for resource-constrained robotics applications.

Method: Uses motion tokenizer with residual quantization to convert motion sequences into multi-stream discrete tokens, implements Delay Parallel Modeling for staggered encoding of token streams, and employs dual-tower architecture with modality-specific parameters.

Result: The framework achieves superior performance across various motion-relevant tasks, with ablation studies confirming the effectiveness of each proposed component.

Conclusion: MotionVerse successfully demonstrates that LLMs can be effectively adapted for comprehensive motion understanding and generation through careful architectural design addressing motion representation and modality integration challenges.

Abstract: This paper proposes MotionVerse, a unified framework that harnesses the capabilities of Large Language Models (LLMs) to comprehend, generate, and edit human motion in both single-person and multi-person scenarios. To efficiently represent motion data, we employ a motion tokenizer with residual quantization, which converts continuous motion sequences into multi-stream discrete tokens. Furthermore, we introduce a \textit{Delay Parallel} Modeling strategy, which temporally staggers the encoding of residual token streams. This design enables LLMs to effectively capture inter-stream dependencies while maintaining computational efficiency comparable to single-stream modeling. Moreover, to alleviate modality interference between motion and language, we design a \textit{dual-tower architecture} with modality-specific parameters, ensuring stable integration of motion information for both comprehension and generation tasks. Comprehensive ablation studies demonstrate the effectiveness of each component in MotionVerse, and extensive experiments showcase its superior performance across a wide range of motion-relevant tasks.

Method: Proposes collaborative distance-constrained debiasing to balance embedding distances in CLIP space without auxiliary references, combined with a two-stage text-guided sampling strategy to limit debiasing intervention and preserve generation quality.

Result: Achieves state-of-the-art debiasing performance on Stable Diffusion v1.5 with only 1/4 of the training burden compared to existing methods, and virtually no increase in sampling burden.

Conclusion: LightFair provides an effective and efficient solution for fair text-to-image generation by focusing on text encoder fine-tuning, demonstrating that significant fairness improvements can be achieved with minimal computational overhead.

Abstract: This paper explores a novel lightweight approach LightFair to achieve fair text-to-image diffusion models (T2I DMs) by addressing the adverse effects of the text encoder. Most existing methods either couple different parts of the diffusion model for full-parameter training or rely on auxiliary networks for correction. They incur heavy training or sampling burden and unsatisfactory performance. Since T2I DMs consist of multiple components, with the text encoder being the most fine-tunable and front-end module, this paper focuses on mitigating bias by fine-tuning text embeddings. To validate feasibility, we observe that the text encoder’s neutral embedding output shows substantial skewness across image embeddings of various attributes in the CLIP space. More importantly, the noise prediction network further amplifies this imbalance. To finetune the text embedding, we propose a collaborative distance-constrained debiasing strategy that balances embedding distances to improve fairness without auxiliary references. However, mitigating bias can compromise the original generation quality. To address this, we introduce a two-stage text-guided sampling strategy to limit when the debiased text encoder intervenes. Extensive experiments demonstrate that LightFair is effective and efficient. Notably, on Stable Diffusion v1.5, our method achieves SOTA debiasing at just $1/4$ of the training burden, with virtually no increase in sampling burden. The code is available at https://github.com/boyuh/LightFair.

Method: Introduces EfficientMIL with Adaptive Patch Selector (APS) for patches selection, replacing quadratic-complexity self-attention in Transformer-based MIL methods with efficient linear-complexity sequence models including GRU, LSTM, and State Space Model Mamba.

Result: Achieved AUC of 0.976 and accuracy of 0.933 on TCGA-Lung dataset with EfficientMIL-Mamba, and AUC of 0.990 and accuracy of 0.975 on CAMELYON16 dataset with EfficientMIL-GRU, surpassing previous state-of-the-art methods. APS demonstrated superior patch selection effectiveness compared to conventional strategies.

Conclusion: EfficientMIL provides significant computational efficiency improvements while outperforming other MIL methods across multiple histopathology datasets, offering a practical solution to the computational bottleneck in whole slide image classification.

Abstract: Whole slide images (WSIs) classification represents a fundamental challenge in computational pathology, where multiple instance learning (MIL) has emerged as the dominant paradigm. Current state-of-the-art (SOTA) MIL methods rely on attention mechanisms, achieving good performance but requiring substantial computational resources due to quadratic complexity when processing hundreds of thousands of patches. To address this computational bottleneck, we introduce EfficientMIL, a novel linear-complexity MIL approach for WSIs classification with the patches selection module Adaptive Patch Selector (APS) that we designed, replacing the quadratic-complexity self-attention mechanisms in Transformer-based MIL methods with efficient sequence models including RNN-based GRU, LSTM, and State Space Model (SSM) Mamba. EfficientMIL achieves significant computational efficiency improvements while outperforming other MIL methods across multiple histopathology datasets. On TCGA-Lung dataset, EfficientMIL-Mamba achieved AUC of 0.976 and accuracy of 0.933, while on CAMELYON16 dataset, EfficientMIL-GRU achieved AUC of 0.990 and accuracy of 0.975, surpassing previous state-of-the-art methods. Extensive experiments demonstrate that APS is also more effective for patches selection than conventional selection strategies.

Method: Systematic review of four core research directions: vectorized map construction, topological structure modeling, prior knowledge fusion, and language model-based perception.

Result: Identified a unifying trend of paradigm shift from static pre-built maps to dynamic, sensor-driven perception that enables real-time map construction and topology reasoning.

Conclusion: These research directions collectively enable more adaptive, scalable, and explainable autonomous driving systems through compact spatial modeling, semantic relational reasoning, robust domain knowledge integration, and multimodal scene understanding.

Abstract: The key to achieving autonomous driving lies in topology-aware perception, the structured understanding of the driving environment with an emphasis on lane topology and road semantics. This survey systematically reviews four core research directions under this theme: vectorized map construction, topological structure modeling, prior knowledge fusion, and language model-based perception. Across these directions, we observe a unifying trend: a paradigm shift from static, pre-built maps to dynamic, sensor-driven perception. Specifically, traditional static maps have provided semantic context for autonomous systems. However, they are costly to construct, difficult to update in real time, and lack generalization across regions, limiting their scalability. In contrast, dynamic representations leverage on-board sensor data for real-time map construction and topology reasoning. Each of the four research directions contributes to this shift through compact spatial modeling, semantic relational reasoning, robust domain knowledge integration, and multimodal scene understanding powered by pre-trained language models. Together, they pave the way for more adaptive, scalable, and explainable autonomous driving systems.

Method: Training-free approach that uses single images as references to specify content and guides the model on where to place each element, enabling multi-image layout control without text.

Result: Demonstrated effectiveness across various image composition tasks with explicit and simple control for object and part-level composition.

Conclusion: The proposed method successfully addresses the challenge of multi-image layout control by using image-based references rather than text, providing finer control over content placement.

Abstract: Text-to-image models have achieved a level of realism that enables the generation of highly convincing images. However, text-based control can be a limiting factor when more explicit guidance is needed. Defining both the content and its precise placement within an image is crucial for achieving finer control. In this work, we address the challenge of multi-image layout control, where the desired content is specified through images rather than text, and the model is guided on where to place each element. Our approach is training-free, requires a single image per reference, and provides explicit and simple control for object and part-level composition. We demonstrate its effectiveness across various image composition tasks.

Method: Uses sparse voxels for texture reconstruction with two key strategies: surface anchoring (learnable upsampling to constrain voxels to mesh surface) and view-domain partitioning (image patch-guided voxel partitioning with local gradient supervision).

Result: Reduces redundant voxels by over 70% compared to traditional voxel upsampling, significantly lowers memory consumption during high-resolution training while maintaining geometric consistency and preserving high-frequency texture details.

Conclusion: Sparse-Up provides an effective solution for high-fidelity 3D texture modeling that breaks through resolution constraints while maintaining memory efficiency and preserving fine texture details.

Abstract: The creation of high-fidelity 3D assets is often hindered by a ‘pixel-level pain point’: the loss of high-frequency details. Existing methods often trade off one aspect for another: either sacrificing cross-view consistency, resulting in torn or drifting textures, or remaining trapped by the resolution ceiling of explicit voxels, forfeiting fine texture detail. In this work, we propose Sparse-Up, a memory-efficient, high-fidelity texture modeling framework that effectively preserves high-frequency details. We use sparse voxels to guide texture reconstruction and ensure multi-view consistency, while leveraging surface anchoring and view-domain partitioning to break through resolution constraints. Surface anchoring employs a learnable upsampling strategy to constrain voxels to the mesh surface, eliminating over 70% of redundant voxels present in traditional voxel upsampling. View-domain partitioning introduces an image patch-guided voxel partitioning scheme, supervising and back-propagating gradients only on visible local patches. Through these two strategies, we can significantly reduce memory consumption during high-resolution voxel training without sacrificing geometric consistency, while preserving high-frequency details in textures.

Method: Uses shared lighting-invariant color-pair feature representation for both initial estimation (registration between RGB-D view and 3D mesh) and tracking (temporal correspondence validation with lightweight motion regression).

Result: Competitive pose estimation accuracy with high-fidelity tracking even through abrupt pose changes, as demonstrated in extensive benchmark experiments.

Conclusion: The integrated approach provides effective and robust 6D pose estimation suitable for edge devices, maintaining accuracy while enabling real-time tracking performance.

Abstract: Robust 6D pose estimation of novel objects under challenging illumination remains a significant challenge, often requiring a trade-off between accurate initial pose estimation and efficient real-time tracking. We present a unified framework explicitly designed for efficient execution on edge devices, which synergizes a robust initial estimation module with a fast motion-based tracker. The key to our approach is a shared, lighting-invariant color-pair feature representation that forms a consistent foundation for both stages. For initial estimation, this feature facilitates robust registration between the live RGB-D view and the object’s 3D mesh. For tracking, the same feature logic validates temporal correspondences, enabling a lightweight model to reliably regress the object’s motion. Extensive experiments on benchmark datasets demonstrate that our integrated approach is both effective and robust, providing competitive pose estimation accuracy while maintaining high-fidelity tracking even through abrupt pose changes.

Method: Multi-stage synthesis pipeline creating ReWatch dataset components, Multi-Agent ReAct framework for CoT synthesis simulating human re-watching, and RLVR framework with Observation & Reasoning reward mechanism.

Result: ReWatch-R1 achieves state-of-the-art average performance on five challenging video reasoning benchmarks.

Conclusion: The proposed ReWatch dataset and framework successfully advance video reasoning capabilities in LVLMs by addressing critical data limitations and incorporating effective reward mechanisms.

Abstract: While Reinforcement Learning with Verifiable Reward (RLVR) significantly advances image reasoning in Large Vision-Language Models (LVLMs), its application to complex video reasoning remains underdeveloped. This gap stems primarily from a critical data bottleneck: existing datasets lack the challenging, multi-hop questions and high-quality, video-grounded Chain-of-Thought (CoT) data necessary to effectively bootstrap RLVR. To address this, we introduce ReWatch, a large-scale dataset built to foster advanced video reasoning. We propose a novel multi-stage synthesis pipeline to synthesize its three components: ReWatch-Caption, ReWatch-QA, and ReWatch-CoT. A core innovation is our Multi-Agent ReAct framework for CoT synthesis, which simulates a human-like “re-watching” process to generate video-grounded reasoning traces by explicitly modeling information retrieval and verification. Building on this dataset, we develop ReWatch-R1 by post-training a strong baseline LVLM with Supervised Fine-Tuning (SFT) and our RLVR framework. This framework incorporates a novel Observation & Reasoning (O&R) reward mechanism that evaluates both the final answer’s correctness and the reasoning’s alignment with video content, directly penalizing hallucination. Our experiments show that ReWatch-R1 achieves state-of-the-art average performance on five challenging video reasoning benchmarks.

Method: Developed three components: (1) Large-scale curated datasets (85M pretraining and 26M instruction datasets), (2) Efficient training framework using offline parallel data packing strategy, (3) Complete end-to-end training within $16,000 budget.

Result: LLaVA-OneVision-1.5 achieves state-of-the-art performance across multiple benchmarks. The 8B model outperforms Qwen2.5-VL-7B on 18 of 27 benchmarks, and the 4B model surpasses Qwen2.5-VL-3B on all 27 benchmarks.

Conclusion: LLaVA-OneVision-1.5 demonstrates that high-performance multimodal models can be built efficiently and cost-effectively from scratch, with plans to release LLaVA-OneVision-1.5-RL soon.

Abstract: We present LLaVA-OneVision-1.5, a novel family of Large Multimodal Models (LMMs) that achieve state-of-the-art performance with significantly reduced computational and financial costs. Different from the existing works, LLaVA-OneVision-1.5 provides an open, efficient, and reproducible framework for building high-quality vision-language models entirely from scratch. The LLaVA-OneVision-1.5 release comprises three primary components: (1) Large-Scale Curated Datasets: We construct an 85M concept-balanced pretraining dataset LLaVA-OneVision-1.5-Mid-Traning and a meticulously curated 26M instruction dataset LLaVA-OneVision-1.5-Instruct, collectively encompassing 64B compressed multimodal tokens. (2) Efficient Training Framework: We develop a complete end-to-end efficient training framework leveraging an offline parallel data packing strategy to facilitate the training of LLaVA-OneVision-1.5 within a $16,000 budget. (3) State-of-the-art Performance: Experimental results demonstrate that LLaVA-OneVision1.5 yields exceptionally competitive performance across a broad range of downstream tasks. Specifically, LLaVA-OneVision-1.5-8B outperforms Qwen2.5-VL-7B on 18 of 27 benchmarks, and LLaVA-OneVision-1.5-4B surpasses Qwen2.5-VL-3B on all 27 benchmarks. We anticipate releasing LLaVA-OneVision-1.5-RL shortly and encourage the community to await further updates.

Method: Uses attention maps from middle layers of vision encoder to estimate token importance, then applies hierarchical pruning: global stage divides image into regions to retain high-importance tokens, local stage divides into windows to retain most important token per window.

Result: Reduces time-to-first-token by up to 55.1% for LLaVA models, improves token generation throughput by up to 60.9%, maintains accuracy and even improves on some benchmarks compared to prior works.

Conclusion: HIVTP achieves superior accuracy and higher inference efficiency than previous methods, demonstrating effective training-free visual token pruning for VLMs.

Abstract: Vision-Language Models (VLMs) have shown strong capabilities on diverse multimodal tasks. However, the large number of visual tokens output by the vision encoder severely hinders inference efficiency, and prior studies have shown that many of these tokens are not important and can therefore be safely pruned. In this work, we propose HIVTP, a training-free method to improve VLMs efficiency via hierarchical visual token pruning using a novel middle-layer-based importance score. Specifically, we utilize attention maps extracted from the middle layers of the vision encoder, which better reflect fine-grained and object-level attention, to estimate visual token importance. Based on this, we propose a hierarchical visual token pruning method to retain both globally and locally important visual tokens. Specifically, we reshape the 1-D visual token sequence output by the vision encoder into a 2-D spatial layout. In the global retaining stage, we divide the image into regions and retain tokens with higher importance scores in each region; in the local retaining stage, we then divide the image into small windows and retain the most important token in each local window. Experimental results show that our proposed method, HIVTP, can reduce the time-to-first-token (TTFT) of LLaVA-v1.5-7B and LLaVA-Next-7B by up to 50.0% and 55.1%, respectively, and improve the token generation throughput by up to 60.9% and 47.3%, without sacrificing accuracy, and even achieving improvements on certain benchmarks. Compared with prior works, HIVTP achieves better accuracy while offering higher inference efficiency.

Method: STIM-TM uses a decoupled strategy: temporal component merges spatially corresponding tokens from consecutive frames using saliency weighting, while spatial component prioritizes merging static tokens through temporal stability analysis to protect dynamic regions.

Result: Achieves over 65% GFLOPs reduction while maintaining competitive accuracy across surgical video tasks, and enables efficient training of long-sequence surgical videos.

Conclusion: STIM-TM effectively addresses computational bottlenecks in surgical video applications through spatiotemporal-aware token merging, providing significant efficiency gains without sacrificing performance.

Abstract: Vision Transformer models have shown impressive effectiveness in the surgical video understanding tasks through long-range dependency modeling. However, current methods suffer from prohibitive computational costs due to processing massive spatiotemporal tokens across video frames. While prior work on token merging has advanced model efficiency, they fail to adequately consider the inherent spatiotemporal structure of video data and overlook the heterogeneous nature of information distribution, leading to suboptimal performance. In this paper, we propose a spatiotemporal information mining token merging (STIM-TM) method, representing the first dedicated approach for surgical video understanding. STIM-TM introduces a decoupled strategy that reduces token redundancy along temporal and spatial dimensions independently. Specifically, the temporal component merges spatially corresponding tokens from consecutive frames using saliency weighting, preserving critical sequential information and maintaining continuity. Meanwhile, the spatial component prioritizes merging static tokens through temporal stability analysis, protecting dynamic regions containing essential surgical information. Operating in a training-free manner, STIM-TM achieves significant efficiency gains with over $65%$ GFLOPs reduction while preserving competitive accuracy across comprehensive surgical video tasks. Our method also supports efficient training of long-sequence surgical videos, addressing computational bottlenecks in surgical applications.

Method: The framework integrates bidirectional spatial perception to capture long-range dependencies and a multi-scale self-distilled fusion strategy to enhance hierarchical feature representations across different resolution levels.

Result: Extensive experiments show MSD-KMamba outperforms state-of-the-art methods in segmentation accuracy, robustness, and generalization while maintaining high computational efficiency and scalability.

Conclusion: MSD-KMamba effectively addresses the computational complexity bottleneck in volumetric segmentation while providing superior global perception capabilities compared to existing approaches.

Abstract: Numerous CNN-Transformer hybrid models rely on high-complexity global attention mechanisms to capture long-range dependencies, which introduces non-linear computational complexity and leads to significant resource consumption. Although knowledge distillation and sparse attention mechanisms can improve efficiency, they often fall short of delivering the high segmentation accuracy necessary for complex tasks. Balancing model performance with computational efficiency remains a critical challenge. In this work, we propose a novel 3D multi-modal image segmentation framework, termed MSD-KMamba, which integrates bidirectional spatial perception with multi-scale self-distillation. The bidirectional spatial aware branch effectively captures long-range spatial context dependencies across brain regions, while also incorporating a powerful nonlinear feature extraction mechanism that further enhances the model’s ability to learn complex and heterogeneous patterns. In addition, the proposed multi-scale self-distilled fusion strategy strengthens hierarchical feature representations and improves the transfer of semantic information at different resolution levels. By jointly leveraging the bidirectional spatial perception branch and the multi-scale self-distilled fusion strategy, our framework effectively mitigates the bottleneck of quadratic computational complexity in volumetric segmentation, while simultaneously addressing the limitation of insufficient global perception. Extensive experiments on multiple standard benchmark datasets demonstrate that MSD-KMamba consistently outperforms state-of-the-art methods in segmentation accuracy, robustness, and generalization, while maintaining high computational efficiency and favorable scalability. The source code of MSD-KMamba is publicly available at https://github.com/daimao-zhang/MSD-KMamba.

Method: QuantSparse integrates model quantization with attention sparsification using two key techniques: Multi-Scale Salient Attention Distillation (providing global structural guidance and local salient supervision to mitigate quantization bias) and Second-Order Sparse Attention Reparameterization (exploiting temporal stability of second-order residuals to recover information lost under sparsity).

Result: On HunyuanVideo-13B, QuantSparse achieves 20.88 PSNR, substantially outperforming state-of-the-art quantization baseline Q-VDiT (16.85 PSNR), while delivering 3.68× storage reduction and 1.88× acceleration in end-to-end inference.

Conclusion: QuantSparse successfully addresses the challenges of combining quantization and sparsification for diffusion transformers, achieving superior performance with significant efficiency gains, making practical deployment of video generation models more feasible.

Abstract: Diffusion transformers exhibit remarkable video generation capability, yet their prohibitive computational and memory costs hinder practical deployment. Model quantization and attention sparsification are two promising directions for compression, but each alone suffers severe performance degradation under aggressive compression. Combining them promises compounded efficiency gains, but naive integration is ineffective. The sparsity-induced information loss exacerbates quantization noise, leading to amplified attention shifts. To address this, we propose \textbf{QuantSparse}, a unified framework that integrates model quantization with attention sparsification. Specifically, we introduce \textit{Multi-Scale Salient Attention Distillation}, which leverages both global structural guidance and local salient supervision to mitigate quantization-induced bias. In addition, we develop \textit{Second-Order Sparse Attention Reparameterization}, which exploits the temporal stability of second-order residuals to efficiently recover information lost under sparsity. Experiments on HunyuanVideo-13B demonstrate that QuantSparse achieves 20.88 PSNR, substantially outperforming the state-of-the-art quantization baseline Q-VDiT (16.85 PSNR), while simultaneously delivering a \textbf{3.68$\times$} reduction in storage and \textbf{1.88$\times$} acceleration in end-to-end inference. Our code will be released in https://github.com/wlfeng0509/QuantSparse.

Method: Proposed HomeSafeBench with 12,900 data points covering five common home safety hazards (fire, electric shock, falling object, trips, child safety), providing dynamic first-person perspective images from simulated home environments and allowing free exploration of rooms.

Result: Evaluation of mainstream VLMs on HomeSafeBench shows poor performance, with the best model achieving only 10.23% F1-score, indicating significant limitations in identifying safety hazards and selecting effective exploration strategies.

Conclusion: HomeSafeBench reveals substantial gaps in current VLM capabilities for home safety inspection and provides a valuable benchmark for future research, with the dataset and code to be publicly available.

Abstract: Embodied agents can identify and report safety hazards in the home environments. Accurately evaluating their capabilities in home safety inspection tasks is curcial, but existing benchmarks suffer from two key limitations. First, they oversimplify safety inspection tasks by using textual descriptions of the environment instead of direct visual information, which hinders the accurate evaluation of embodied agents based on Vision-Language Models (VLMs). Second, they use a single, static viewpoint for environmental observation, which restricts the agents’ free exploration and cause the omission of certain safety hazards, especially those that are occluded from a fixed viewpoint. To alleviate these issues, we propose HomeSafeBench, a benchmark with 12,900 data points covering five common home safety hazards: fire, electric shock, falling object, trips, and child safety. HomeSafeBench provides dynamic first-person perspective images from simulated home environments, enabling the evaluation of VLM capabilities for home safety inspection. By allowing the embodied agents to freely explore the room, HomeSafeBench provides multiple dynamic perspectives in complex environments for a more thorough inspection. Our comprehensive evaluation of mainstream VLMs on HomeSafeBench reveals that even the best-performing model achieves an F1-score of only 10.23%, demonstrating significant limitations in current VLMs. The models particularly struggle with identifying safety hazards and selecting effective exploration strategies. We hope HomeSafeBench will provide valuable reference and support for future research related to home security inspections. Our dataset and code will be publicly available soon.

Method: Multiple SSD models with different backbone networks (VGG16, ResNet50, EfficientNet, MobileNetV3) were trained on weapon detection and combined using Weighted Boxes Fusion with max confidence scoring.

Result: The ensemble achieved mAP of 0.838, representing 2.948% improvement over the best single model, and consistently outperformed other fusion methods.

Conclusion: Confidence-aware fusion is critical for ensemble performance, and this approach provides a robust solution for real-time weapon detection in surveillance applications.

Abstract: The safety and security of public spaces is of vital importance, driving the need for sophisticated surveillance systems capable of accurately detecting weapons, which are often hampered by issues like partial occlusion, varying lighting, and cluttered backgrounds. While single-model detectors are advanced, they often lack robustness in these challenging conditions. This paper presents the hypothesis that ensemble of Single Shot Multibox Detector (SSD) models with diverse feature extraction backbones can significantly enhance detection robustness. To leverage diverse feature representations, individual SSD models were trained using a selection of backbone networks: VGG16, ResNet50, EfficientNet, and MobileNetV3. The study is conducted on a dataset consisting of images of three distinct weapon classes: guns, heavy weapons and knives. The predictions from these models are combined using the Weighted Boxes Fusion (WBF) method, an ensemble technique designed to optimize bounding box accuracy. Our key finding is that the fusion strategy is as critical as the ensemble’s diversity, a WBF approach using a ‘max’ confidence scoring strategy achieved a mean Average Precision (mAP) of 0.838. This represents a 2.948% relative improvement over the best-performing single model and consistently outperforms other fusion heuristics. This research offers a robust approach to enhancing real-time weapon detection capabilities in surveillance applications by demonstrating that confidence-aware fusion is a key mechanism for improving accuracy metrics of ensembles.

Method: INSTINCT features three core components: 1) quality-aware filtering for high-quality instance feature selection, 2) dual-branch detection routing to separate collaboration-irrelevant and collaboration-relevant instances, and 3) Cross Agent Local Instance Fusion module to aggregate local hybrid instance features. Enhanced ground truth sampling facilitates training with diverse hybrid instance features.

Result: Extensive experiments show INSTINCT achieves superior performance with 13.23%/33.08% improvement in accuracy on DAIR-V2X and V2V4Real datasets while reducing communication bandwidth to 1/281 and 1/264 compared to state-of-the-art methods.

Conclusion: INSTINCT successfully bridges the gap in LiDAR-focused collaborative perception by providing an efficient instance-level interaction architecture that significantly improves detection accuracy while dramatically reducing bandwidth requirements.

Abstract: Collaborative perception systems overcome single-vehicle limitations in long-range detection and occlusion scenarios by integrating multi-agent sensory data, improving accuracy and safety. However, frequent cooperative interactions and real-time requirements impose stringent bandwidth constraints. Previous works proves that query-based instance-level interaction reduces bandwidth demands and manual priors, however, LiDAR-focused implementations in collaborative perception remain underdeveloped, with performance still trailing state-of-the-art approaches. To bridge this gap, we propose INSTINCT (INSTance-level INteraCtion ArchiTecture), a novel collaborative perception framework featuring three core components: 1) a quality-aware filtering mechanism for high-quality instance feature selection; 2) a dual-branch detection routing scheme to decouple collaboration-irrelevant and collaboration-relevant instances; and 3) a Cross Agent Local Instance Fusion module to aggregate local hybrid instance features. Additionally, we enhance the ground truth (GT) sampling technique to facilitate training with diverse hybrid instance features. Extensive experiments across multiple datasets demonstrate that INSTINCT achieves superior performance. Specifically, our method achieves an improvement in accuracy 13.23%/33.08% in DAIR-V2X and V2V4Real while reducing the communication bandwidth to 1/281 and 1/264 compared to state-of-the-art methods. The code is available at https://github.com/CrazyShout/INSTINCT.

Method: Jointly trains task embeddings for removal and insertion in a single model, leverages them in classifier-free guidance scheme, and extends task prompts to spatially distinct regions for object movement in a single denoising step.

Result: Achieves superior object removal, controllable effect insertion, and efficient object movement without requiring additional training or separate removal/insertion stages.

Conclusion: CrimEdit demonstrates that unified training with classifier-free guidance enables efficient and high-quality object editing with controllable effect handling across removal, insertion, and movement tasks.

Abstract: Recent works on object removal and insertion have enhanced their performance by handling object effects such as shadows and reflections, using diffusion models trained on counterfactual datasets. However, the performance impact of applying classifier-free guidance to handle object effects across removal and insertion tasks within a unified model remains largely unexplored. To address this gap and improve efficiency in composite editing, we propose CrimEdit, which jointly trains the task embeddings for removal and insertion within a single model and leverages them in a classifier-free guidance scheme – enhancing the removal of both objects and their effects, and enabling controllable synthesis of object effects during insertion. CrimEdit also extends these two task prompts to be applied to spatially distinct regions, enabling object movement (repositioning) within a single denoising step. By employing both guidance techniques, extensive experiments show that CrimEdit achieves superior object removal, controllable effect insertion, and efficient object movement without requiring additional training or separate removal and insertion stages.

Method: End-to-end framework with MRI preprocessing, two clinical priors (brain region relevance and aging patterns), and dedicated modules for feature aggregation and diagnosis using brain age gaps as constraints.

Result: Achieved 86% accuracy on external hospital tests and over 96% accuracy in early-stage diagnosis, outperforming existing methods by more than 20%.

Conclusion: PD-Diag-Net provides an effective automated diagnostic solution for Parkinson’s disease with high accuracy and clinical interpretability.

Abstract: Parkinson’s disease (PD) is a common neurodegenerative disorder that severely diminishes patients’ quality of life. Its global prevalence has increased markedly in recent decades. Current diagnostic workflows are complex and heavily reliant on neurologists’ expertise, often resulting in delays in early detection and missed opportunities for timely intervention. To address these issues, we propose an end-to-end automated diagnostic method for PD, termed PD-Diag-Net, which performs risk assessment and auxiliary diagnosis directly from raw MRI scans. This framework first introduces an MRI Pre-processing Module (MRI-Processor) to mitigate inter-subject and inter-scanner variability by flexibly integrating established medical imaging preprocessing tools. It then incorporates two forms of clinical prior knowledge: (1) Brain-Region-Relevance-Prior (Relevance-Prior), which specifies brain regions strongly associated with PD; and (2) Brain-Region-Aging-Prior (Aging-Prior), which reflects the accelerated aging typically observed in PD-associated regions. Building on these priors, we design two dedicated modules: the Relevance-Prior Guided Feature Aggregation Module (Aggregator), which guides the model to focus on PD-associated regions at the inter-subject level, and the Age-Prior Guided Diagnosis Module (Diagnoser), which leverages brain age gaps as auxiliary constraints at the intra-subject level to enhance diagnostic accuracy and clinical interpretability. Furthermore, we collected external test data from our collaborating hospital. Experimental results show that PD-Diag-Net achieves 86% accuracy on external tests and over 96% accuracy in early-stage diagnosis, outperforming existing advanced methods by more than 20%.

Method: Two-stage approach: 1) Project partial point clouds to depth images, use DepthLDM to generate completed multi-view depth images for coarse point clouds; 2) Point Denoising Network removes artifacts, Association-Aware Point Upsampler refines with local association features.

Result: Achieves state-of-the-art performance in geometric accuracy and shape completeness, significantly improving robustness and consistency of point cloud completion.

Conclusion: DiffPCN successfully adapts latent diffusion models to point cloud completion through a novel coarse-to-fine framework with depth image projection and refinement stages.

Abstract: Latent diffusion models (LDMs) have demonstrated remarkable generative capabilities across various low-level vision tasks. However, their potential for point cloud completion remains underexplored due to the unstructured and irregular nature of point clouds. In this work, we propose DiffPCN, a novel diffusion-based coarse-to-fine framework for point cloud completion. Our approach comprises two stages: an initial stage for generating coarse point clouds, and a refinement stage that improves their quality through point denoising and upsampling. Specifically, we first project the unordered and irregular partial point cloud into structured depth images, which serve as conditions for a well-designed DepthLDM to synthesize completed multi-view depth images that are used to form coarse point clouds. In this way, our DiffPCN can yield high-quality and high-completeness coarse point clouds by leveraging LDM’ s powerful generation and comprehension capabilities. Then, since LDMs inevitably introduce outliers into the generated depth maps, we design a Point Denoising Network to remove artifacts from the coarse point cloud by predicting a per-point distance score. Finally, we devise an Association-Aware Point Upsampler, which guides the upsampling process by leveraging local association features between the input point cloud and the corresponding coarse points, further yielding a dense and high-fidelity output. Experimental results demonstrate that our DiffPCN achieves state-of-the-art performance in geometric accuracy and shape completeness, significantly improving the robustness and consistency of point cloud completion.

Method: Proposes a visual prompting strategy that trades spatial details for temporal resolution by combining multiple frames as panels into one image. The approach is training-free, parameter-free, model-agnostic, and seamlessly integrates with existing VLMs.

Result: Extensive experiments on five benchmarks show consistent improvements across various model architectures, sizes, and context windows. On TimeScope (Long) dataset with the longest videos, video QA accuracy improved by up to 19.4%.

Conclusion: The method effectively raises the bar for long video understanding models by providing a simple yet effective visual prompting strategy that enhances temporal resolution without requiring additional training or parameters.

Abstract: Recent Video-Language Models (VLMs) achieve promising results on long-video understanding, but their performance still lags behind that achieved on tasks involving images or short videos. This has led to great interest in improving the long context modeling of VLMs by introducing novel modules and additional complexity. % additional training time. In this paper, we take a different approach: rather than fine-tuning VLMs with the limited data available, we attempt to maximize the performance of existing models. To this end, we propose a novel visual prompting strategy specifically designed for long-video understanding. By combining multiple frames as panels into one image, we effectively trade off spatial details for temporal resolution. Our approach is training-free, parameter-free, and model-agnostic, and can be seamlessly integrated into existing VLMs. Extensive experiments on five established benchmarks across a wide range of model architectures, sizes, and context windows confirm the consistency of our approach. For the TimeScope (Long) dataset, which has the longest videos, the accuracy for video question answering is improved by up to 19.4%. Overall, our method raises the bar for long video understanding models. We will make our code available upon acceptance.

Method: Created M3DLayout dataset with 15,080 layouts and 258k object instances from three sources (real scans, CAD designs, procedural scenes), each paired with detailed structured text descriptions. Established benchmark using text-conditioned diffusion model.

Result: The dataset provides a solid foundation for training layout generation models, with multi-source composition enhancing diversity. The Inf3DLayout subset enables generation of more complex and detailed scenes with rich small-object information.

Conclusion: M3DLayout serves as a valuable resource for advancing research in text-driven 3D scene synthesis by providing diverse and richly annotated data that enables learning complex spatial and semantic patterns.

Abstract: In text-driven 3D scene generation, object layout serves as a crucial intermediate representation that bridges high-level language instructions with detailed geometric output. It not only provides a structural blueprint for ensuring physical plausibility but also supports semantic controllability and interactive editing. However, the learning capabilities of current 3D indoor layout generation models are constrained by the limited scale, diversity, and annotation quality of existing datasets. To address this, we introduce M3DLayout, a large-scale, multi-source dataset for 3D indoor layout generation. M3DLayout comprises 15,080 layouts and over 258k object instances, integrating three distinct sources: real-world scans, professional CAD designs, and procedurally generated scenes. Each layout is paired with detailed structured text describing global scene summaries, relational placements of large furniture, and fine-grained arrangements of smaller items. This diverse and richly annotated resource enables models to learn complex spatial and semantic patterns across a wide variety of indoor environments. To assess the potential of M3DLayout, we establish a benchmark using a text-conditioned diffusion model. Experimental results demonstrate that our dataset provides a solid foundation for training layout generation models. Its multi-source composition enhances diversity, notably through the Inf3DLayout subset which provides rich small-object information, enabling the generation of more complex and detailed scenes. We hope that M3DLayout can serve as a valuable resource for advancing research in text-driven 3D scene synthesis.

Method: Uses Alternative Hierarchical Attention (AHA) for efficient cross-view feature fusion through separate intra-frame and inter-frame windowed self-attention, and proposes ERP fusion to project multi-view depth estimates to equirectangular coordinates.

Result: Achieves competitive zero-shot performance on real datasets and runs at up to 20 FPS on NVIDIA Orin NX embedded hardware.

Conclusion: FastViDAR provides an efficient solution for real-time 360° depth estimation from multiple fisheye cameras with reduced computational overhead.

Abstract: In this paper we propose FastViDAR, a novel framework that takes four fisheye camera inputs and produces a full $360^\circ$ depth map along with per-camera depth, fusion depth, and confidence estimates. Our main contributions are: (1) We introduce Alternative Hierarchical Attention (AHA) mechanism that efficiently fuses features across views through separate intra-frame and inter-frame windowed self-attention, achieving cross-view feature mixing with reduced overhead. (2) We propose a novel ERP fusion approach that projects multi-view depth estimates to a shared equirectangular coordinate system to obtain the final fusion depth. (3) We generate ERP image-depth pairs using HM3D and 2D3D-S datasets for comprehensive evaluation, demonstrating competitive zero-shot performance on real datasets while achieving up to 20 FPS on NVIDIA Orin NX embedded hardware. Project page: \href{https://3f7dfc.github.io/FastVidar/}{https://3f7dfc.github.io/FastVidar/}

Method: Uses multi-scale downsampling on token maps and a scale-causal attention mechanism that progressively flows information from low-resolution to high-resolution features.

Result: 27.2% improvement in rFID (1.47 → 1.07), 1.38× faster convergence, 18.9% boost in gFID (16.4 → 13.3), and achieves state-of-the-art rFID of 0.45 and gFID of 1.82 among ViT tokenizers.

Conclusion: HieraTok demonstrates the effectiveness of multi-scale ViT tokenizers for visual generation tasks, advancing the field with its novel architecture and achieving superior performance metrics.

Abstract: In this work, we present HieraTok, a novel multi-scale Vision Transformer (ViT)-based tokenizer that overcomes the inherent limitation of modeling single-scale representations. This is realized through two key designs: (1) multi-scale downsampling applied to the token map generated by the tokenizer encoder, producing a sequence of multi-scale tokens, and (2) a scale-causal attention mechanism that enables the progressive flow of information from low-resolution global semantic features to high-resolution structural details. Coupling these designs, HieraTok achieves significant improvements in both image reconstruction and generation tasks. Under identical settings, the multi-scale visual tokenizer outperforms its single-scale counterpart by a 27.2% improvement in rFID ($1.47 \rightarrow 1.07$). When integrated into downstream generation frameworks, it achieves a $1.38\times$ faster convergence rate and an 18.9% boost in gFID ($16.4 \rightarrow 13.3$), which may be attributed to the smoother and more uniformly distributed latent space. Furthermore, by scaling up the tokenizer’s training, we demonstrate its potential by a sota rFID of 0.45 and a gFID of 1.82 among ViT tokenizers. To the best of our knowledge, we are the first to introduce multi-scale ViT-based tokenizer in image reconstruction and image generation. We hope our findings and designs advance the ViT-based tokenizers in visual generation tasks.

Method: The framework uses sequential input, a transformer-based gated update module for spatial memory, and submap partitioning with local alignment and global registration to maintain consistency.

Result: Experiments show superior reconstruction accuracy compared to existing methods while maintaining real-time performance.

Conclusion: GRS-SLAM3R effectively addresses the limitations of previous approaches by incorporating spatial memory and global consistency mechanisms, achieving state-of-the-art dense scene reconstruction.

Abstract: DUSt3R-based end-to-end scene reconstruction has recently shown promising results in dense visual SLAM. However, most existing methods only use image pairs to estimate pointmaps, overlooking spatial memory and global consistency.To this end, we introduce GRS-SLAM3R, an end-to-end SLAM framework for dense scene reconstruction and pose estimation from RGB images without any prior knowledge of the scene or camera parameters. Unlike existing DUSt3R-based frameworks, which operate on all image pairs and predict per-pair point maps in local coordinate frames, our method supports sequentialized input and incrementally estimates metric-scale point clouds in the global coordinate. In order to improve consistent spatial correlation, we use a latent state for spatial memory and design a transformer-based gated update module to reset and update the spatial memory that continuously aggregates and tracks relevant 3D information across frames. Furthermore, we partition the scene into submaps, apply local alignment within each submap, and register all submaps into a common world frame using relative constraints, producing a globally consistent map. Experiments on various datasets show that our framework achieves superior reconstruction accuracy while maintaining real-time performance.

Method: Proposes residual features by subtracting normal reference features to achieve feature decorrelation. Uses feature hypersphere constraining to address scale correlation. Enhanced with logbarrier bidirectional contraction OCC loss and vector quantization-based feature distribution matching.

Result: Comprehensive experiments on eight real-world datasets show ResAD++ achieves remarkable anomaly detection results when directly applied to new classes, outperforming state-of-the-art methods and the base ResAD version.

Conclusion: ResAD++ effectively addresses class-agnostic anomaly detection by learning residual feature distributions and constraining feature scales, enabling strong generalization to diverse new classes without retraining.

Abstract: This paper explores the problem of class-agnostic anomaly detection (AD), where the objective is to train one class-agnostic AD model that can generalize to detect anomalies in diverse new classes from different domains without any retraining or fine-tuning on the target data. When applied for new classes, the performance of current single- and multi-class AD methods is still unsatisfactory. One fundamental reason is that representation learning in existing methods is still class-related, namely, feature correlation. To address this issue, we propose residual features and construct a simple but effective framework, termed ResAD. Our core insight is to learn the residual feature distribution rather than the initial feature distribution. Residual features are formed by matching and then subtracting normal reference features. In this way, we can effectively realize feature decorrelation. Even in new classes, the distribution of normal residual features would not remarkably shift from the learned distribution. In addition, we think that residual features still have one issue: scale correlation. To this end, we propose a feature hypersphere constraining approach, which learns to constrain initial normal residual features into a spatial hypersphere for enabling the feature scales of different classes as consistent as possible. Furthermore, we propose a novel logbarrier bidirectional contraction OCC loss and vector quantization based feature distribution matching module to enhance ResAD, leading to the improved version of ResAD (ResAD++). Comprehensive experiments on eight real-world AD datasets demonstrate that our ResAD++ can achieve remarkable AD results when directly used in new classes, outperforming state-of-the-art competing methods and also surpassing ResAD. The code is available at https://github.com/xcyao00/ResAD.

Method: Proposed Point, Visualize, then Refine (Poivre) procedure using reinforcement learning with a process reward to enable iterative coordinate refinement.

Result: Poivre-7B achieves new state-of-the-art on Point-Bench, outperforming proprietary models like Gemini-2.5-Pro and large open-source models like Molmo-72B by over 3%.

Conclusion: The self-refining approach with RL training significantly improves visual pointing performance, bridging the gap between VLMs and human capabilities.

Abstract: Visual pointing, which aims to localize a target by predicting its coordinates on an image, has emerged as an important problem in the realm of vision-language models (VLMs). Despite its broad applicability, recent benchmarks show that current VLMs still fall far behind human performance on this task. A key limitation is that VLMs are typically required to complete the pointing task in a single step, akin to asking humans to point at an object without seeing their own fingers. To address this issue, we propose a simple yet effective self-refining procedure: Point, Visualize, then Refine (Poivre). This procedure enables a VLM to first mark its estimated point, then iteratively refine the coordinates if necessary. Inspired by advances of reasoning models in the natural language domain, we employ reinforcement learning (RL) to incentivize this self-refining ability. For the RL training, we design a neat process reward that is not only empirically effective but also grounded in appealing properties. Our trained model, Poivre-7B, sets a new state of the art on Point-Bench, outperforming both proprietary models such as Gemini-2.5-Pro and large open-source models such as Molmo-72B by over 3%. To support future research, we release our training and inference code, dataset, and the Poivre-7B checkpoint.

Method: Integrates U-Net-style encoder-decoder structure with Pyramid Vision Transformer backbone, novel residual blocks, adapter-based skip connections, and squeeze-and-excitation attention for feature refinement.

Result: Achieves Dice coefficient of 0.8851 and mIoU of 0.8167 on out-of-distribution datasets, demonstrating real-time accurate polyp segmentation.

Conclusion: PVTAdpNet shows superior performance for clinical applications in colorectal cancer detection with high accuracy and real-time capabilities.

Abstract: Colorectal cancer ranks among the most common and deadly cancers, emphasizing the need for effective early detection and treatment. To address the limitations of traditional colonoscopy, including high miss rates due to polyp variability, we introduce the Pyramid Vision Transformer Adapter Residual Network (PVTAdpNet). This model integrates a U-Net-style encoder-decoder structure with a Pyramid Vision Transformer backbone, novel residual blocks, and adapter-based skip connections. The design enhances feature extraction, dense prediction, and gradient flow, supported by squeeze-and-excitation attention for improved channel-wise feature refinement. PVTAdpNet achieves real-time, accurate polyp segmentation, demonstrating superior performance on benchmark datasets with high mDice and mIoU scores, making it highly suitable for clinical applications. PVTAdpNet obtains a high Dice coefficient of 0.8851 and a mean Intersection over Union (mIoU) of 0.8167 on out-of-distribution polyp datasets. Evaluation of the PolypGen dataset demonstrates PVTAdpNet’s capability for real-time, accurate performance within familiar distributions. The source code of our network is available at https://github.com/ayousefinejad/PVTAdpNet.git

Method: Proposes UniAlignment with dual-stream diffusion training incorporating intrinsic-modal and cross-modal semantic alignment within a single diffusion transformer.

Result: Extensive experiments show UniAlignment outperforms existing baselines across multiple tasks and benchmarks, demonstrating superior multimodal semantic consistency.

Conclusion: The framework highlights the significant potential of diffusion models in unified multimodal generation, achieving robust instruction-following and cross-modal consistency.

Abstract: The remarkable success of diffusion models in text-to-image generation has sparked growing interest in expanding their capabilities to a variety of multi-modal tasks, including image understanding, manipulation, and perception. These tasks require advanced semantic comprehension across both visual and textual modalities, especially in scenarios involving complex semantic instructions. However, existing approaches often rely heavily on vision-language models (VLMs) or modular designs for semantic guidance, leading to fragmented architectures and computational inefficiency. To address these challenges, we propose UniAlignment, a unified multimodal generation framework within a single diffusion transformer. UniAlignment introduces a dual-stream diffusion training strategy that incorporates both intrinsic-modal semantic alignment and cross-modal semantic alignment, thereby enhancing the model’s cross-modal consistency and instruction-following robustness. Additionally, we present SemGen-Bench, a new benchmark specifically designed to evaluate multimodal semantic consistency under complex textual instructions. Extensive experiments across multiple tasks and benchmarks demonstrate that UniAlignment outperforms existing baselines, underscoring the significant potential of diffusion models in unified multimodal generation.

Method: Proposes two innovations: 1) Multi-source adaptive view generation that synthesizes diverse yet semantically coherent views using image-conditioned, text-conditioned, and image-text-conditioned strategies, and 2) Quality-driven contrastive learning that assesses semantic alignment and diversity to dynamically reweight training contributions.

Result: Improves MoCov2 by +2.5% on ImageNet linear classification, raises average zero-shot classification accuracy by +12.31% over CLIP and +5.31% over SLIP across ten datasets, and improves Flickr30k text retrieval R@5 by +3.2%.

Conclusion: GenView++ effectively addresses both construction and learning challenges in contrastive learning, demonstrating superior performance across vision and vision-language tasks through its unified framework.

Abstract: The success of contrastive learning depends on the construction and utilization of high-quality positive pairs. However, current methods face critical limitations on two fronts: on the construction side, both handcrafted and generative augmentations often suffer from limited diversity and risk semantic corruption; on the learning side, the absence of a quality assessment mechanism leads to suboptimal supervision where all pairs are treated equally. To tackle these challenges, we propose GenView++, a unified framework that addresses both fronts by introducing two synergistic innovations. To improve pair construction, GenView++ introduces a multi-source adaptive view generation mechanism to synthesize diverse yet semantically coherent views by dynamically modulating generative parameters across image-conditioned, text-conditioned, and image-text-conditioned strategies. Second, a quality-driven contrastive learning mechanism assesses each pair’s semantic alignment and diversity to dynamically reweight their training contribution, prioritizing high-quality pairs while suppressing redundant or misaligned pairs. Extensive experiments demonstrate the effectiveness of GenView++ across both vision and vision-language tasks. For vision representation learning, it improves MoCov2 by +2.5% on ImageNet linear classification. For vision-language learning, it raises the average zero-shot classification accuracy by +12.31% over CLIP and +5.31% over SLIP across ten datasets, and further improves Flickr30k text retrieval R@5 by +3.2%. The code is available at https://github.com/xiaojieli0903/GenViewPlusPlus.

Method: Categorizes modalities into aggregated-level and point-level groups, uses MoE graph architecture with expert GNNs for aggregated modalities, dual-level GNN for point-level modality, and spatially-aware multimodal fusion with dynamic region-specific weights.

Result: Experiments on two real-world datasets across six modalities and three tasks show MTGRR consistently outperforms state-of-the-art baselines.

Conclusion: MTGRR effectively addresses modality-specific characteristics and spatial heterogeneity, providing superior urban region representations for downstream tasks.

Abstract: Graph-based models have emerged as a powerful paradigm for modeling multimodal urban data and learning region representations for various downstream tasks. However, existing approaches face two major limitations. (1) They typically employ identical graph neural network architectures across all modalities, failing to capture modality-specific structures and characteristics. (2) During the fusion stage, they often neglect spatial heterogeneity by assuming that the aggregation weights of different modalities remain invariant across regions, resulting in suboptimal representations. To address these issues, we propose MTGRR, a modality-tailored graph modeling framework for urban region representation, built upon a multimodal dataset comprising point of interest (POI), taxi mobility, land use, road element, remote sensing, and street view images. (1) MTGRR categorizes modalities into two groups based on spatial density and data characteristics: aggregated-level and point-level modalities. For aggregated-level modalities, MTGRR employs a mixture-of-experts (MoE) graph architecture, where each modality is processed by a dedicated expert GNN to capture distinct modality-specific characteristics. For the point-level modality, a dual-level GNN is constructed to extract fine-grained visual semantic features. (2) To obtain effective region representations under spatial heterogeneity, a spatially-aware multimodal fusion mechanism is designed to dynamically infer region-specific modality fusion weights. Building on this graph modeling framework, MTGRR further employs a joint contrastive learning strategy that integrates region aggregated-level, point-level, and fusion-level objectives to optimize region representations. Experiments on two real-world datasets across six modalities and three tasks demonstrate that MTGRR consistently outperforms state-of-the-art baselines, validating its effectiveness.

Method: Texture Vector-Quantization only models missing textures’ prior using codebook, and Reconstruction Aware Prediction uses straight-through estimator to train index predictor directly with image-level supervision.

Result: The proposed TVQ&RAP model delivers photo-realistic super-resolution results with small computational cost.

Conclusion: The proposed strategies effectively address quantization errors and sub-optimal prior modeling in VQ-based super-resolution, achieving high-quality results efficiently.

Abstract: Vector-quantized based models have recently demonstrated strong potential for visual prior modeling. However, existing VQ-based methods simply encode visual features with nearest codebook items and train index predictor with code-level supervision. Due to the richness of visual signal, VQ encoding often leads to large quantization error. Furthermore, training predictor with code-level supervision can not take the final reconstruction errors into consideration, result in sub-optimal prior modeling accuracy. In this paper we address the above two issues and propose a Texture Vector-Quantization and a Reconstruction Aware Prediction strategy. The texture vector-quantization strategy leverages the task character of super-resolution and only introduce codebook to model the prior of missing textures. While the reconstruction aware prediction strategy makes use of the straight-through estimator to directly train index predictor with image-level supervision. Our proposed generative SR model (TVQ&RAP) is able to deliver photo-realistic SR results with small computational cost.

Method: Proposes Group Context Optimization (GroupCoOp) that employs group-specific text prompts as group representatives serving as multiple classifiers for each target class, leveraging the semantic knowledge of VLM text encoders.

Result: Achieved best results on five benchmarks across five CLIP architectures, occasionally outperforming methods that fine-tune the entire network despite training only 0.016% of parameters.

Conclusion: GroupCoOp effectively enhances group robustness of fine-tuned VLMs by addressing subgroup imbalance issues through group-specific prompts, demonstrating superior performance with extreme parameter efficiency.

Abstract: Parameter-efficient fine-tuning (PEFT) of vision-language models (VLMs) excels in various vision tasks thanks to the rich knowledge and generalization ability of VLMs. However, recent studies revealed that such fine-tuned VLMs are vulnerable to spurious correlations stemming from the subgroup imbalance in the fine-tuning datasets. To resolve this issue, we propose Group Context Optimization (GroupCoOp), a simple and effective debiased fine-tuning algorithm that enhances the group robustness of fine-tuned VLMs. Its key idea is to employ group-specific text prompts as group representatives serving as multiple classifiers for their target class. The rich semantic knowledge of the text encoder of VLM enables the discovery of effective group prompts even for groups with a small number of training samples. Leveraging the group prompts for each class addresses the issues caused by the group-imbalanced training set, such as the neglect of minority groups and the scattered distribution of each class in the embedding space. GroupCoOp achieved the best results on five benchmarks across five CLIP architectures and occasionally outperformed prior methods that fine-tune the entire network, despite training only 0.016% of the network’s parameters.

Method: Uses object segmentation and visual analysis of level images to detect structural gaps and perform targeted repairs, evaluating multiple segmentation models.

Result: Experimental results show improved stability and playability of AI-generated levels.

Conclusion: The image-based approach is designed to be applicable to a wide range of 2D games with similar level structures, though evaluation was specific to Angry Birds.

Abstract: Procedural Content Generation (PCG) techniques enable automatic creation of diverse and complex environments. While PCG facilitates more efficient content creation, ensuring consistently high-quality, industry-standard content remains a significant challenge. In this research, we propose a method to identify and repair unstable levels generated by existing PCG models. We use Angry Birds as a case study, demonstrating our method on game levels produced by established PCG approaches. Our method leverages object segmentation and visual analysis of level images to detect structural gaps and perform targeted repairs. We evaluate multiple object segmentation models and select the most effective one as the basis for our repair pipeline. Experimental results show that our method improves the stability and playability of AI-generated levels. Although our evaluation is specific to Angry Birds, our image-based approach is designed to be applicable to a wide range of 2D games with similar level structures.

Method: Fuses geometric and semantic attributes, introduces edge weights, embeds building height in the graph, and extends 2D layouts to 3D structures. Users can directly control output by modifying semantic attributes.

Result: The method produces valid, large-scale urban models that maintain geometric continuity and incorporate semantic information, demonstrating effectiveness for data-driven planning and design.

Conclusion: The proposed framework provides an effective tool for generating controllable, large-scale 3D urban layouts that combine both geometric and semantic attributes, addressing limitations of existing methods.

Abstract: Urban modeling is essential for city planning, scene synthesis, and gaming. Existing image-based methods generate diverse layouts but often lack geometric continuity and scalability, while graph-based methods capture structural relations yet overlook parcel semantics. We present a controllable framework for large-scale 3D vector urban layout generation, conditioned on both geometry and semantics. By fusing geometric and semantic attributes, introducing edge weights, and embedding building height in the graph, our method extends 2D layouts to realistic 3D structures. It also enables users to directly control the output by modifying semantic attributes. Experiments show that it produces valid, large-scale urban models, offering an effective tool for data-driven planning and design.

Method: Integrates a multi-camera imaging system with advanced object detection algorithms, capturing images from three camera views and using a tailored image fusion methodology to combine results from multiple views.

Result: Significantly outperforms single-view methods, achieving high precision and recall rates in identifying improperly fastened small assembly parts such as screws.

Conclusion: Overcomes single-view limitations by providing a scalable, cost-effective, and accurate quality control mechanism that ensures reliability and safety of assembly lines, with the dataset made publicly available for further research.

Abstract: Quality control is a critical aspect of manufacturing, particularly in ensuring the proper assembly of small components in production lines. Existing solutions often rely on single-view imaging or manual inspection, which are prone to errors due to occlusions, restricted perspectives, or lighting inconsistencies. These limitations require the installation of additional inspection stations, which could disrupt the assembly line and lead to increased downtime and costs. This paper introduces a novel multi-view quality control module designed to address these challenges, integrating a multi-camera imaging system with advanced object detection algorithms. By capturing images from three camera views, the system provides comprehensive visual coverage of components of an assembly process. A tailored image fusion methodology combines results from multiple views, effectively resolving ambiguities and enhancing detection reliability. To support this system, we developed a unique dataset comprising annotated images across diverse scenarios, including varied lighting conditions, occlusions, and angles, to enhance applicability in real-world manufacturing environments. Experimental results show that our approach significantly outperforms single-view methods, achieving high precision and recall rates in the identification of improperly fastened small assembly parts such as screws. This work contributes to industrial automation by overcoming single-view limitations, and providing a scalable, cost-effective, and accurate quality control mechanism that ensures the reliability and safety of the assembly line. The dataset used in this study is publicly available to facilitate further research in this domain.

Method: The authors introduce a comprehensive Visual Privacy Taxonomy based on legal frameworks and evaluate state-of-the-art VLMs using this taxonomy to test their understanding of contextual privacy.

Result: Evaluation revealed significant inconsistencies in VLMs’ understanding of contextual privacy, showing they lack robust privacy awareness despite their other advanced capabilities.

Conclusion: There is an urgent need for more privacy-aware AI systems, and the paper provides both a foundational taxonomy for future research and a benchmark highlighting current model limitations in privacy understanding.

Abstract: Artificial Intelligence have profoundly transformed the technological landscape in recent years. Large Language Models (LLMs) have demonstrated impressive abilities in reasoning, text comprehension, contextual pattern recognition, and integrating language with visual understanding. While these advances offer significant benefits, they also reveal critical limitations in the models’ ability to grasp the notion of privacy. There is hence substantial interest in determining if and how these models can understand and enforce privacy principles, particularly given the lack of supporting resources to test such a task. In this work, we address these challenges by examining how legal frameworks can inform the capabilities of these emerging technologies. To this end, we introduce a comprehensive, multi-level Visual Privacy Taxonomy that captures a wide range of privacy issues, designed to be scalable and adaptable to existing and future research needs. Furthermore, we evaluate the capabilities of several state-of-the-art Vision-Language Models (VLMs), revealing significant inconsistencies in their understanding of contextual privacy. Our work contributes both a foundational taxonomy for future research and a critical benchmark of current model limitations, demonstrating the urgent need for more robust, privacy-aware AI systems.

Method: Extracts semantic features for understanding and noisy-injected appearance features for generation, incorporates 4D geometric cues, fuses them via adaptive cross-attention into spatiotemporal-aware representations, and integrates both tasks into a single LLM with task-specific heads using instruction fine-tuning.

Result: Achieves competitive or superior results compared to state-of-the-art models on multiple benchmarks, demonstrating true unification of 4D scene understanding and generation.

Conclusion: Uni4D-LLM successfully unifies 4D scene understanding and generation through shared representations and architecture, enabling joint handling of both tasks within one Transformer-based framework with spatiotemporal awareness.

Abstract: Vision-language models (VLMs) have demonstrated strong performance in 2D scene understanding and generation, but extending this unification to the physical world remains an open challenge. Existing 3D and 4D approaches typically embed scene geometry into autoregressive model for semantic understanding and diffusion model for content generation. This paradigm gap prevents a single model from jointly handling both tasks, especially in dynamic 4D settings where spatiotemporal modeling is critical. We propose Uni4D-LLM, the first unified VLM framework with spatiotemporal awareness for 4D scene understanding and generation. Our design is guided by two key insights: 1) Unification requires a shared representation. We extract semantic features for understanding and noisy-injected appearance features for generation, incorporate 4D geometric cues, and fuse them into a spatiotemporal-aware visual representation through adaptive cross-attention. 2) Unification requires a shared architecture. Both autoregression and diffusion are built on Transformer backbones, and this enables integration into a single LLM with task-specific heads. By aligning visual and linguistic representations, our Uni4D-LLM produces predictions for both understanding and generation within one Transformer-based framework. We further apply instruction fine-tuning on diverse 4D vision-language datasets to improve generalization across tasks. Extensive experiments on multiple benchmarks demonstrate that Uni4D-LLM achieves competitive or superior results compared to state-of-the-art models and offers the first true unification of 4D scene understanding and generation.

Method: Uses the Segment Concept (SeC) framework which employs a Large Vision-Language Model (LVLM) to establish deep semantic understanding of objects for more persistent segmentation.

Result: Achieved 39.7 JFn on MOSEv2 test set without any fine-tuning on training data, ranking 2nd place in the Complex VOS track of the 7th Large-scale Video Object Segmentation Challenge.

Conclusion: The SeC framework demonstrates strong zero-shot performance on complex video object segmentation tasks by leveraging semantic understanding through LVLMs, showing robustness against challenging scenarios where appearance-based methods fail.

Abstract: Semi-supervised Video Object Segmentation aims to segment a specified target throughout a video sequence, initialized by a first-frame mask. Previous methods rely heavily on appearance-based pattern matching and thus exhibit limited robustness against challenges such as drastic visual changes, occlusions, and scene shifts. This failure is often attributed to a lack of high-level conceptual understanding of the target. The recently proposed Segment Concept (SeC) framework mitigated this limitation by using a Large Vision-Language Model (LVLM) to establish a deep semantic understanding of the object for more persistent segmentation. In this work, we evaluate its zero-shot performance on the challenging coMplex video Object SEgmentation v2 (MOSEv2) dataset. Without any fine-tuning on the training set, SeC achieved 39.7 \JFn on the test set and ranked 2nd place in the Complex VOS track of the 7th Large-scale Video Object Segmentation Challenge.

Method: 1) Created T23D-CompBench with 5 components and 12 sub-components for compositional prompts, generating 3,600 textured meshes from 10 state-of-the-art models with 129,600 human ratings. 2) Proposed Rank2Score with two-stage training: first stage uses supervised contrastive regression and curriculum learning for pairwise training, second stage refines predictions using mean opinion scores.

Result: Rank2Score consistently outperforms existing metrics across multiple dimensions and can serve as a reward function to optimize generative models.

Conclusion: The proposed benchmark and Rank2Score method effectively address limitations in Text-to-3D quality assessment, providing better alignment with human judgments and practical utility for model optimization.

Abstract: Recent advances in Text-to-3D (T23D) generative models have enabled the synthesis of diverse, high-fidelity 3D assets from textual prompts. However, existing challenges restrict the development of reliable T23D quality assessment (T23DQA). First, existing benchmarks are outdated, fragmented, and coarse-grained, making fine-grained metric training infeasible. Moreover, current objective metrics exhibit inherent design limitations, resulting in non-representative feature extraction and diminished metric robustness. To address these limitations, we introduce T23D-CompBench, a comprehensive benchmark for compositional T23D generation. We define five components with twelve sub-components for compositional prompts, which are used to generate 3,600 textured meshes from ten state-of-the-art generative models. A large-scale subjective experiment is conducted to collect 129,600 reliable human ratings across different perspectives. Based on T23D-CompBench, we further propose Rank2Score, an effective evaluator with two-stage training for T23DQA. Rank2Score enhances pairwise training via supervised contrastive regression and curriculum learning in the first stage, and subsequently refines predictions using mean opinion scores to achieve closer alignment with human judgments in the second stage. Extensive experiments and downstream applications demonstrate that Rank2Score consistently outperforms existing metrics across multiple dimensions and can additionally serve as a reward function to optimize generative models. The project is available at https://cbysjtu.github.io/Rank2Score/.

Method: Uses a branched network that shares activation maps with an image classifier and learns to mimic VLM embeddings. Predicts concepts and represents regions via weighted sum of activation maps using concept prediction gradients, quantifying contributions based on impact on classification score.

Result: Outperforms existing approaches in qualitative and quantitative evaluations, excels in zero-shot inference for unseen concepts, and provides a general framework without requiring annotated datasets.

Conclusion: CE-FAM effectively bridges the gap between saliency maps and concept-based explanations by providing comprehensive insights into learned concepts, their spatial regions, and prediction contributions while leveraging VLM knowledge.

Abstract: Although saliency maps can highlight important regions to explain the reasoning behind image classification in artificial intelligence (AI), the meaning of these regions is left to the user’s interpretation. In contrast, conceptbased explanations decompose AI predictions into humanunderstandable concepts, clarifying their contributions. However, few methods can simultaneously reveal what concepts an image classifier learns, which regions are associated with them, and how they contribute to predictions. We propose a novel concept-based explanation method, Concept-based Explanation via Fusion of Activation Maps (CE-FAM). It employs a branched network that shares activation maps with an image classifier and learns to mimic the embeddings of a Vision and Language Model (VLM). The branch network predicts concepts in an image, and their corresponding regions are represented by a weighted sum of activation maps, with weights given by the gradients of the concept prediction scores. Their contributions are quantified based on their impact on the image classification score. Our method provides a general framework for identifying the concept regions and their contributions while leveraging VLM knowledge to handle arbitrary concepts without requiring an annotated dataset. Furthermore, we introduce a novel evaluation metric to assess the accuracy of the concept regions. Our qualitative and quantitative evaluations demonstrate our method outperforms existing approaches and excels in zero-shot inference for unseen concepts.

Method: Hybrid framework combining CNN branch for local feature extraction and ViT branch for global context modeling, with adversarial debiasing mechanism to produce protected attribute-invariant representations.

Result: Achieves Pearson Correlation of 0.9230 and RMSE of 0.2650 on SCUT-FBP5500 benchmark. Reduces performance gap between ethnic subgroups by 82.9%, with adversary classification accuracy dropping to 52.1% (near random chance).

Conclusion: FairViT-GAN provides a robust, transparent, and significantly fairer blueprint for responsible AI systems in subjective visual assessment.

Abstract: Facial Beauty Prediction (FBP) has made significant strides with the application of deep learning, yet state-of-the-art models often exhibit critical limitations, including architectural constraints, inherent demographic biases, and a lack of transparency. Existing methods, primarily based on Convolutional Neural Networks (CNNs), excel at capturing local texture but struggle with global facial harmony, while Vision Transformers (ViTs) effectively model long-range dependencies but can miss fine-grained details. Furthermore, models trained on benchmark datasets can inadvertently learn and perpetuate societal biases related to protected attributes like ethnicity. To address these interconnected challenges, we propose \textbf{FairViT-GAN}, a novel hybrid framework that synergistically integrates a CNN branch for local feature extraction and a ViT branch for global context modeling. More significantly, we introduce an adversarial debiasing mechanism where the feature extractor is explicitly trained to produce representations that are invariant to protected attributes, thereby actively mitigating algorithmic bias. Our framework’s transparency is enhanced by visualizing the distinct focus of each architectural branch. Extensive experiments on the SCUT-FBP5500 benchmark demonstrate that FairViT-GAN not only sets a new state-of-the-art in predictive accuracy, achieving a Pearson Correlation of \textbf{0.9230} and reducing RMSE to \textbf{0.2650}, but also excels in fairness. Our analysis reveals a remarkable \textbf{82.9% reduction in the performance gap} between ethnic subgroups, with the adversary’s classification accuracy dropping to near-random chance (52.1%). We believe FairViT-GAN provides a robust, transparent, and significantly fairer blueprint for developing responsible AI systems for subjective visual assessment.

Method: Extends standard DETR with two modifications: (1) constraining self-attention between queries based on semantic and positional overlap, and (2) adding query-to-frame alignment to bridge global and local contexts.

Result: Sim-DETR unlocks the full potential of DETR for temporal sentence grounding, offering a strong baseline for future research.

Conclusion: The proposed Sim-DETR provides a simple yet powerful solution to address query conflicts in DETR for temporal sentence grounding, demonstrating improved performance over standard approaches.

Abstract: Temporal sentence grounding aims to identify exact moments in a video that correspond to a given textual query, typically addressed with detection transformer (DETR) solutions. However, we find that typical strategies designed to enhance DETR do not improve, and may even degrade, its performance in this task. We systematically analyze and identify the root causes of this abnormal behavior: (1) conflicts between queries from similar target moments and (2) internal query conflicts due to the tension between global semantics and local localization. Building on these insights, we propose a simple yet powerful baseline, Sim-DETR, which extends the standard DETR with two minor modifications in the decoder layers: (1) constraining self-attention between queries based on their semantic and positional overlap and (2) adding query-to-frame alignment to bridge the global and local contexts. Experiments demonstrate that Sim-DETR unlocks the full potential of DETR for temporal sentence grounding, offering a strong baseline for future research.

Method: Proposed Information-Grounding Guidance (IGG) that adaptively reinforces informative patches during sampling through attention mechanisms to keep guidance and content aligned.

Result: IGG delivers sharper, more coherent, and semantically grounded images across class-conditioned and text-to-image generation tasks, setting new benchmarks for AR-based methods.

Conclusion: IGG successfully tackles the patch inconsistency problem in autoregressive image generation by anchoring guidance to semantically important regions, resulting in improved image quality and faithfulness to conditioning information.

Abstract: Autoregressive (AR) models based on next-scale prediction are rapidly emerging as a powerful tool for image generation, but they face a critical weakness: information inconsistencies between patches across timesteps introduced by progressive resolution scaling. These inconsistencies scatter guidance signals, causing them to drift away from conditioning information and leaving behind ambiguous, unfaithful features. We tackle this challenge with Information-Grounding Guidance (IGG), a novel mechanism that anchors guidance to semantically important regions through attention. By adaptively reinforcing informative patches during sampling, IGG ensures that guidance and content remain tightly aligned. Across both class-conditioned and text-to-image generation tasks, IGG delivers sharper, more coherent, and semantically grounded images, setting a new benchmark for AR-based methods.

Method: Introduces two networks at teacher output level for reliable pseudo-label quality assessment via score fusion and context-adaptive thresholds, supervised by pseudo-label alignment with GT boxes. Uses soft supervision strategy for robust learning under pseudo-label noise.

Result: Extensive experiments on KITTI and Waymo datasets show the method selects high-precision pseudo-labels while maintaining wider context coverage and higher recall rate, significantly improving SS3DOD performance.

Conclusion: The proposed learnable pseudo-labeling framework effectively addresses pseudo-label quality assessment challenges in semi-supervised 3D object detection, outperforming existing methods through adaptive thresholding and robust supervision.

Abstract: Semi-supervised 3D object detection (SS3DOD) aims to reduce costly 3D annotations utilizing unlabeled data. Recent studies adopt pseudo-label-based teacher-student frameworks and demonstrate impressive performance. The main challenge of these frameworks is in selecting high-quality pseudo-labels from the teacher’s predictions. Most previous methods, however, select pseudo-labels by comparing confidence scores over thresholds manually set. The latest works tackle the challenge either by dynamic thresholding or refining the quality of pseudo-labels. Such methods still overlook contextual information e.g. object distances, classes, and learning states, and inadequately assess the pseudo-label quality using partial information available from the networks. In this work, we propose a novel SS3DOD framework featuring a learnable pseudo-labeling module designed to automatically and adaptively select high-quality pseudo-labels. Our approach introduces two networks at the teacher output level. These networks reliably assess the quality of pseudo-labels by the score fusion and determine context-adaptive thresholds, which are supervised by the alignment of pseudo-labels over GT bounding boxes. Additionally, we introduce a soft supervision strategy that can learn robustly under pseudo-label noises. This helps the student network prioritize cleaner labels over noisy ones in semi-supervised learning. Extensive experiments on the KITTI and Waymo datasets demonstrate the effectiveness of our method. The proposed method selects high-precision pseudo-labels while maintaining a wider coverage of contexts and a higher recall rate, significantly improving relevant SS3DOD methods.

Method: Uses contextual sub-data similarity adaptive sensing in the projection domain to provide initial prior. Combines knowledge distillation with latent diffusion models for image optimization. Employs pixel-level self-correcting fusion for fine-grained reconstruction. Can be flexibly applied to different dose levels including unseen doses.

Result: SuperDiff consistently outperforms existing state-of-the-art methods in both reconstruction and generalization performance on datasets and real data. It requires only LDCT projection domain data for training and testing.

Conclusion: The proposed SuperDiff method provides an effective solution for low-dose CT reconstruction that doesn’t require paired data, achieves superior performance, and demonstrates strong generalization capabilities across different dose levels.

Abstract: Current models based on deep learning for low-dose CT denoising rely heavily on paired data and generalize poorly. Even the more concerned diffusion models need to learn the distribution of clean data for reconstruction, which is difficult to satisfy in medical clinical applications. At the same time, self-supervised-based methods face the challenge of significant degradation of generalizability of models pre-trained for the current dose to expand to other doses. To address these issues, this paper proposes a novel method of tunable-generalization diffusion powered by self-supervised contextual sub-data for low-dose CT reconstruction, named SuperDiff. Firstly, a contextual subdata similarity adaptive sensing strategy is designed for denoising centered on the LDCT projection domain, which provides an initial prior for the subsequent progress. Subsequently, the initial prior is used to combine knowledge distillation with a deep combination of latent diffusion models for optimizing image details. The pre-trained model is used for inference reconstruction, and the pixel-level self-correcting fusion technique is proposed for fine-grained reconstruction of the image domain to enhance the image fidelity, using the initial prior and the LDCT image as a guide. In addition, the technique is flexibly applied to the generalization of upper and lower doses or even unseen doses. Dual-domain strategy cascade for self-supervised LDCT denoising, SuperDiff requires only LDCT projection domain data for training and testing. Full qualitative and quantitative evaluations on both datasets and real data show that SuperDiff consistently outperforms existing state-of-the-art methods in terms of reconstruction and generalization performance.

Method: Constructed a pipeline combining multi-view triangulation with SMPL-X mesh fitting for reliable 3D hand-body pose annotation, then validated different input representations across graph convolution and spatio-temporal attention models.

Result: Pose-based action inference is more efficient and accurate than video baselines, and joint modeling of hand and body cues improves action recognition over using hands or upper body alone.

Conclusion: Modeling interdependent hand-body dynamics is crucial for holistic understanding of bimanual activities, as hand-body coordination significantly impacts action recognition performance.

Abstract: Bimanual human activities inherently involve coordinated movements of both hands and body. However, the impact of this coordination in activity understanding has not been systematically evaluated due to the lack of suitable datasets. Such evaluation demands kinematic-level annotations (e.g., 3D pose) for the hands and body, yet existing 3D activity datasets typically annotate either hand or body pose. Another line of work employs marker-based motion capture to provide full-body pose, but the physical markers introduce visual artifacts, thereby limiting models’ generalization to natural, markerless videos. To address these limitations, we present AssemblyHands-X, the first markerless 3D hand-body benchmark for bimanual activities, designed to study the effect of hand-body coordination for action recognition. We begin by constructing a pipeline for 3D pose annotation from synchronized multi-view videos. Our approach combines multi-view triangulation with SMPL-X mesh fitting, yielding reliable 3D registration of hands and upper body. We then validate different input representations (e.g., video, hand pose, body pose, or hand-body pose) across recent action recognition models based on graph convolution or spatio-temporal attention. Our extensive experiments show that pose-based action inference is more efficient and accurate than video baselines. Moreover, joint modeling of hand and body cues improves action recognition over using hands or upper body alone, highlighting the importance of modeling interdependent hand-body dynamics for a holistic understanding of bimanual activities.

Method: Used a dataset of over 100,000 images covering 1000 plant species from West Europe, built through a large-scale participatory sensing platform involving tens of thousands of contributors since 2011.

Result: The challenge provided resources and assessments for evaluating plant identification systems, with multiple research groups participating and developing various approaches.

Conclusion: The LifeCLEF 2015 challenge successfully established a large-scale evaluation framework for plant identification using crowd-sourced data, enabling analysis of different methodological approaches in realistic biodiversity monitoring conditions.

Abstract: The LifeCLEF plant identification challenge aims at evaluating plant identification methods and systems at a very large scale, close to the conditions of a real-world biodiversity monitoring scenario. The 2015 evaluation was actually conducted on a set of more than 100K images illustrating 1000 plant species living in West Europe. The main originality of this dataset is that it was built through a large-scale participatory sensing plateform initiated in 2011 and which now involves tens of thousands of contributors. This overview presents more precisely the resources and assessments of the challenge, summarizes the approaches and systems employed by the participating research groups, and provides an analysis of the main outcomes.

Method: Three key components: (a) selective visual unlearning using inverse contrastive learning, (b) cross-modal knowledge retention through semantic consistency, and (c) dual-set contrastive separation to isolate structural perturbations between unlearn and retain sets.

Result: Extensive experiments on three datasets show superiority, achieving 7.12% accuracy improvement with only 7% of the unlearning time compared to top-accuracy baseline.

Conclusion: CCU effectively addresses privacy leakage in multimodal applications by enabling selective visual unlearning while preserving cross-modal knowledge and model performance.

Abstract: Visual modality is the most vulnerable to privacy leakage in real-world multimodal applications like autonomous driving with visual and radar data; Machine unlearning removes specific training data from pre-trained models to address privacy leakage, however, existing methods fail to preserve cross-modal knowledge and maintain intra-class structural stability of retain data, leading to reduced overall and other modalities’ performance during visual unlearning; to address these challenges, we propose a Cross-modal Contrastive Unlearning (CCU) framework, which integrates three key components: (a) selective visual unlearning: employing inverse contrastive learning to dissociate visual representations from their original semantics, (b) cross-modal knowledge retention: preserving other modalities’ discriminability through semantic consistency, and (c) dual-set contrastive separation: preserving the model performance via isolation of structural perturbations between the unlearn set and retain set; extensive experiments on three datasets demonstrate the superiority of CCU, and our method achieves a 7.12% accuracy improvement with only 7% of the unlearning time compared to the top-accuracy baseline.

Method: Uses Fast Fourier Transform to shift image and text features to frequency domain, then applies quantum-inspired retrieval to fetch relevant medical facts from external sources, merging both for enhanced reasoning.

Result: Outperforms previous models on VQA-RAD dataset, particularly on complex cases requiring image-text reasoning, with improved performance and explainability.

Conclusion: The combination of frequency processing and quantum information offers a promising approach for developing intelligent, transparent AI tools to assist doctors.

Abstract: Solving tough clinical questions that require both image and text understanding is still a major challenge in healthcare AI. In this work, we propose Q-FSRU, a new model that combines Frequency Spectrum Representation and Fusion (FSRU) with a method called Quantum Retrieval-Augmented Generation (Quantum RAG) for medical Visual Question Answering (VQA). The model takes in features from medical images and related text, then shifts them into the frequency domain using Fast Fourier Transform (FFT). This helps it focus on more meaningful data and filter out noise or less useful information. To improve accuracy and ensure that answers are based on real knowledge, we add a quantum inspired retrieval system. It fetches useful medical facts from external sources using quantum-based similarity techniques. These details are then merged with the frequency-based features for stronger reasoning. We evaluated our model using the VQA-RAD dataset, which includes real radiology images and questions. The results showed that Q-FSRU outperforms earlier models, especially on complex cases needing image text reasoning. The mix of frequency and quantum information improves both performance and explainability. Overall, this approach offers a promising way to build smart, clear, and helpful AI tools for doctors.

Method: The task uses a dataset of 500 plant species with 7 image types: leaf scans and 6 unconstrained view types (flower, fruit, stem & bark, branch, leaf, entire view) collected through a citizen science initiative by Tela Botanica.

Result: 27 submitted runs from 10 groups across 6 countries participated, employing distinct and original methods, confirming the Image & Multimedia Retrieval community’s interest in biodiversity and botany.

Conclusion: The fourth year of this task demonstrates continued community engagement in plant identification and highlights further challenging studies in this domain.

Abstract: The LifeCLEFs plant identification task provides a testbed for a system-oriented evaluation of plant identification about 500 species trees and herbaceous plants. Seven types of image content are considered: scan and scan-like pictures of leaf, and 6 kinds of detailed views with unconstrained conditions, directly photographed on the plant: flower, fruit, stem & bark, branch, leaf and entire view. The main originality of this data is that it was specifically built through a citizen sciences initiative conducted by Tela Botanica, a French social network of amateur and expert botanists. This makes the task closer to the conditions of a real-world application. This overview presents more precisely the resources and assessments of task, summarizes the retrieval approaches employed by the participating groups, and provides an analysis of the main evaluation results. With a total of ten groups from six countries and with a total of twenty seven submitted runs, involving distinct and original methods, this fourth year task confirms Image & Multimedia Retrieval community interest for biodiversity and botany, and highlights further challenging studies in plant identification.

Method: Pairs class-conditional diffusion replay (avoiding patient data storage) with Elastic Weight Consolidation, using a compact Vision Transformer backbone evaluated on MedMNIST v2 tasks and CheXpert.

Result: Achieves 0.851 AUROC on CheXpert, reduces forgetting by over 30% compared to DER++, approaches joint training performance (0.869 AUROC), while remaining efficient and privacy-preserving.

Conclusion: Provides a practical route for scalable, privacy-aware continual adaptation of clinical imaging models by combining replay diffusion and synaptic stability mechanisms.

Abstract: Medical imaging foundation models must adapt over time, yet full retraining is often blocked by privacy constraints and cost. We present a continual learning framework that avoids storing patient exemplars by pairing class conditional diffusion replay with Elastic Weight Consolidation. Using a compact Vision Transformer backbone, we evaluate across eight MedMNIST v2 tasks and CheXpert. On CheXpert our approach attains 0.851 AUROC, reduces forgetting by more than 30% relative to DER\texttt{++}, and approaches joint training at 0.869 AUROC, while remaining efficient and privacy preserving. Analyses connect forgetting to two measurable factors: fidelity of replay and Fisher weighted parameter drift, highlighting the complementary roles of replay diffusion and synaptic stability. The results indicate a practical route for scalable, privacy aware continual adaptation of clinical imaging models.

Method: Propose feature-level adversarial learning among clients by aligning feature maps through adversarial training mechanism to enhance model generalization and reduce domain-shift impact.

Result: Comprehensive experiments on three medical image datasets show FedDA achieves cross-domain federated aggregation, enables single modality clients with cross-modality processing, and outperforms state-of-the-art federated aggregation algorithms.

Conclusion: FedDA successfully addresses modality heterogeneity in federated medical image segmentation through adversarial domain adaptation, providing robust cross-modality processing capabilities.

Abstract: Federated learning enables collaborative training of machine learning models among different clients while ensuring data privacy, emerging as the mainstream for breaking data silos in the healthcare domain. However, the imbalance of medical resources, data corruption or improper data preservation may lead to a situation where different clients possess medical images of different modality. This heterogeneity poses a significant challenge for cross-domain medical image segmentation within the federated learning framework. To address this challenge, we propose a new Federated Domain Adaptation (FedDA) segmentation training framework. Specifically, we propose a feature-level adversarial learning among clients by aligning feature maps across clients through embedding an adversarial training mechanism. This design can enhance the model’s generalization on multiple domains and alleviate the negative impact from domain-shift. Comprehensive experiments on three medical image datasets demonstrate that our proposed FedDA substantially achieves cross-domain federated aggregation, endowing single modality client with cross-modality processing capabilities, and consistently delivers robust performance compared to state-of-the-art federated aggregation algorithms in objective and subjective assessment. Our code are available at https://github.com/GGbond-study/FedDA.

Method: Developed EditReward-Bench benchmark for systematic reward model evaluation, then created EditScore reward models (7B-72B) through meticulous data curation and filtering. Implemented self-ensemble strategy tailored for generative nature of EditScore. Applied RL framework to OmniGen2 base model using EditScore as reward signal.

Result: EditScore matches performance of proprietary VLMs, with largest variant surpassing GPT-5 in benchmark. EditScore enables efficient and robust policy optimization where other VLMs fail. Final model shows substantial and consistent performance uplift when applied to OmniGen2.

Conclusion: A high-fidelity, domain-specialized reward model is key to unlocking RL’s full potential in image editing. This work provides the first systematic path from benchmarking to reward modeling to RL training in this domain.

Abstract: Instruction-guided image editing has achieved remarkable progress, yet current models still face challenges with complex instructions and often require multiple samples to produce a desired result. Reinforcement Learning (RL) offers a promising solution, but its adoption in image editing has been severely hindered by the lack of a high-fidelity, efficient reward signal. In this work, we present a comprehensive methodology to overcome this barrier, centered on the development of a state-of-the-art, specialized reward model. We first introduce EditReward-Bench, a comprehensive benchmark to systematically evaluate reward models on editing quality. Building on this benchmark, we develop EditScore, a series of reward models (7B-72B) for evaluating the quality of instruction-guided image editing. Through meticulous data curation and filtering, EditScore effectively matches the performance of learning proprietary VLMs. Furthermore, coupled with an effective self-ensemble strategy tailored for the generative nature of EditScore, our largest variant even surpasses GPT-5 in the benchmark. We then demonstrate that a high-fidelity reward model is the key to unlocking online RL for image editing. Our experiments show that, while even the largest open-source VLMs fail to provide an effective learning signal, EditScore enables efficient and robust policy optimization. Applying our framework to a strong base model, OmniGen2, results in a final model that shows a substantial and consistent performance uplift. Overall, this work provides the first systematic path from benchmarking to reward modeling to RL training in image editing, showing that a high-fidelity, domain-specialized reward model is the key to unlocking the full potential of RL in this domain.

Method: MoReact uses a diffusion-based approach that sequentially generates global trajectories first, then local motions. It introduces a novel interaction loss to enhance realism of close interactions and is trained on data adapted from a two-person motion dataset.

Result: The method produces realistic, diverse, and controllable reactions that closely match counterpart movements while adhering to textual guidance. Experiments demonstrate efficacy for this novel task of text-driven human reaction generation.

Conclusion: MoReact effectively addresses limitations of existing models by focusing on text-driven reaction generation with sequential trajectory-motion disentanglement, achieving better alignment with actions and text descriptions while enhancing interaction realism.

Abstract: Modeling and generating human reactions poses a significant challenge with broad applications for computer vision and human-computer interaction. Existing methods either treat multiple individuals as a single entity, directly generating interactions, or rely solely on one person’s motion to generate the other’s reaction, failing to integrate the rich semantic information that underpins human interactions. Yet, these methods often fall short in adaptive responsiveness, i.e., the ability to accurately respond to diverse and dynamic interaction scenarios. Recognizing this gap, our work introduces an approach tailored to address the limitations of existing models by focusing on text-driven human reaction generation. Our model specifically generates realistic motion sequences for individuals that responding to the other’s actions based on a descriptive text of the interaction scenario. The goal is to produce motion sequences that not only complement the opponent’s movements but also semantically fit the described interactions. To achieve this, we present MoReact, a diffusion-based method designed to disentangle the generation of global trajectories and local motions sequentially. This approach stems from the observation that generating global trajectories first is crucial for guiding local motion, ensuring better alignment with given action and text. Furthermore, we introduce a novel interaction loss to enhance the realism of generated close interactions. Our experiments, utilizing data adapted from a two-person motion dataset, demonstrate the efficacy of our approach for this novel task, which is capable of producing realistic, diverse, and controllable reactions that not only closely match the movements of the counterpart but also adhere to the textual guidance. Please find our webpage at https://xiyan-xu.github.io/MoReactWebPage.

Method: Systematic experimental analysis examining factors contributing to optimization imbalance, including testing existing methods across datasets, analyzing Vision Foundation Models, and investigating gradient dynamics.

Result: Found strong correlation between optimization imbalance and task-specific gradient norms. Demonstrated that scaling task losses according to gradient norms achieves performance comparable to computationally expensive grid search.

Conclusion: Understanding and controlling gradient dynamics is a more direct path to stable multi-task learning than developing increasingly complex methods, with gradient norm-based loss scaling providing an effective solution.

Abstract: Multi-task learning (MTL) aims to build general-purpose vision systems by training a single network to perform multiple tasks jointly. While promising, its potential is often hindered by “unbalanced optimization”, where task interference leads to subpar performance compared to single-task models. To facilitate research in MTL, this paper presents a systematic experimental analysis to dissect the factors contributing to this persistent problem. Our investigation confirms that the performance of existing optimization methods varies inconsistently across datasets, and advanced architectures still rely on costly grid-searched loss weights. Furthermore, we show that while powerful Vision Foundation Models (VFMs) provide strong initialization, they do not inherently resolve the optimization imbalance, and merely increasing data quantity offers limited benefits. A crucial finding emerges from our analysis: a strong correlation exists between the optimization imbalance and the norm of task-specific gradients. We demonstrate that this insight is directly applicable, showing that a straightforward strategy of scaling task losses according to their gradient norms can achieve performance comparable to that of an extensive and computationally expensive grid search. Our comprehensive analysis suggests that understanding and controlling gradient dynamics is a more direct path to stable MTL than developing increasingly complex methods.

Method: Proposed Multi-Task Adversarial CLIP (MT-AdvCLIP) with task-aware feature aggregation loss to generate perturbations with enhanced cross-task generalization capability, strengthening attack effectiveness of fine-grained task models on shared CLIP backbone.

Result: MT-AdvCLIP significantly improves adversarial transfer success rate by over 39% on average across multiple tasks against various CLIP-derived models without increasing perturbation budget.

Conclusion: The study reveals adversarial example transfer mechanisms in multi-task CLIP models and provides insights for multi-task robustness evaluation and adversarial example design.

Abstract: As a general-purpose vision-language pretraining model, CLIP demonstrates strong generalization ability in image-text alignment tasks and has been widely adopted in downstream applications such as image classification and image-text retrieval. However, it struggles with fine-grained tasks such as object detection and semantic segmentation. While many variants aim to improve CLIP on these tasks, its robustness to adversarial perturbations remains underexplored. Understanding how adversarial examples transfer across tasks is key to assessing CLIP’s generalization limits and security risks. In this work, we conduct a systematic empirical analysis of the cross-task transfer behavior of CLIP-based models on image-text retrieval, object detection, and semantic segmentation under adversarial perturbations. We find that adversarial examples generated from fine-grained tasks (e.g., object detection and semantic segmentation) often exhibit stronger transfer potential than those from coarse-grained tasks, enabling more effective attacks against the original CLIP model. Motivated by this observation, we propose a novel framework, Multi-Task Adversarial CLIP (MT-AdvCLIP), which introduces a task-aware feature aggregation loss and generates perturbations with enhanced cross-task generalization capability. This design strengthens the attack effectiveness of fine-grained task models on the shared CLIP backbone. Experimental results on multiple public datasets show that MT-AdvCLIP significantly improves the adversarial transfer success rate (The average attack success rate across multiple tasks is improved by over 39%.) against various CLIP-derived models, without increasing the perturbation budget. This study reveals the transfer mechanism of adversarial examples in multi-task CLIP models, offering new insights into multi-task robustness evaluation and adversarial example design.

Method: Proposes Token Painter with two key components: Dual-Stream Encoder Information Fusion (DEIF) for semantic fusion in frequency domain, and Adaptive Decoder Attention Score Enhancing (ADAE) for attention enhancement on guidance and inpainting tokens.

Result: Outperforms prior state-of-the-art methods across almost all metrics and delivers superior visual results without requiring training.

Conclusion: The training-free Token Painter method effectively solves text-guided image inpainting challenges by leveraging MAR models with novel fusion and attention mechanisms.

Abstract: Text-guided image inpainting aims to inpaint masked image regions based on a textual prompt while preserving the background. Although diffusion-based methods have become dominant, their property of modeling the entire image in latent space makes it challenging for the results to align well with prompt details and maintain a consistent background. To address these issues, we explore Mask AutoRegressive (MAR) models for this task. MAR naturally supports image inpainting by generating latent tokens corresponding to mask regions, enabling better local controllability without altering the background. However, directly applying MAR to this task makes the inpainting content either ignore the prompts or be disharmonious with the background context. Through analysis of the attention maps from the inpainting images, we identify the impact of background tokens on text tokens during the MAR generation, and leverage this to design \textbf{Token Painter}, a training-free text-guided image inpainting method based on MAR. Our approach introduces two key components: (1) Dual-Stream Encoder Information Fusion (DEIF), which fuses the semantic and context information from text and background in frequency domain to produce novel guidance tokens, allowing MAR to generate text-faithful inpainting content while keeping harmonious with background context. (2) Adaptive Decoder Attention Score Enhancing (ADAE), which adaptively enhances attention scores on guidance tokens and inpainting tokens to further enhance the alignment of prompt details and the content visual quality. Extensive experiments demonstrate that our training-free method outperforms prior state-of-the-art methods across almost all metrics and delivers superior visual results. Codes will be released.

Method: Extracted 800 dynamic traffic scenarios from 100-hour infrastructure sensor videos and created static digital twin assets for 15 real-world intersections with consistent visual appearance in CARLA.

Result: Created a challenging evaluation framework that accurately replicates real-world traffic and environmental characteristics, enabling more realistic simulations.

Conclusion: Provides a comprehensive closed-loop benchmark for evaluating end-to-end autonomous driving models that better reflects real-world driving conditions.

Abstract: Closed-loop evaluation is increasingly critical for end-to-end autonomous driving. Current closed-loop benchmarks using the CARLA simulator rely on manually configured traffic scenarios, which can diverge from real-world conditions, limiting their ability to reflect actual driving performance. To address these limitations, we introduce a simple yet challenging closed-loop evaluation framework that closely integrates real-world driving scenarios into the CARLA simulator with infrastructure cooperation. Our approach involves extracting 800 dynamic traffic scenarios selected from a comprehensive 100-hour video dataset captured by high-mounted infrastructure sensors, and creating static digital twin assets for 15 real-world intersections with consistent visual appearance. These digital twins accurately replicate the traffic and environmental characteristics of their real-world counterparts, enabling more realistic simulations in CARLA. This evaluation is challenging due to the diversity of driving behaviors, locations, weather conditions, and times of day at complex urban intersections. In addition, we provide a comprehensive closed-loop benchmark for evaluating end-to-end autonomous driving models. Project URL: \href{https://github.com/AIR-THU/DriveE2E}{https://github.com/AIR-THU/DriveE2E}.

Method: Identifies decoding directions via directional clustering of activations, estimates encoding directions with signal vectors under a probabilistic view, and leverages network weights through Uncertainty Region Alignment to reveal interpretable directions.

Result: (a) Recovers ground-truth direction pairs on synthetic data; (b) Decoding directions map to monosemantic, interpretable concepts and outperform unsupervised baselines on real data; (c) Signal vectors faithfully estimate encoding directions, validated via activation maximization.

Conclusion: The method successfully recovers interpretable concept directions, enabling applications in understanding global model behavior, explaining individual predictions, and producing counterfactuals or correcting errors through interventions.

Abstract: Empirical evidence shows that deep vision networks represent concepts as directions in latent space, vectors we call concept embeddings. Each concept has a latent factor-a scalar-indicating its presence in an input patch. For a given patch, multiple latent factors are encoded into a compact representation by linearly combining concept embeddings, with the factors as coefficients. Since these embeddings enable such encoding, we call them encoding directions. A latent factor can be recovered via the inner product with a filter, a vector we call a decoding direction. These encoding-decoding direction pairs are not directly accessible, but recovering them helps open the black box of deep networks, enabling understanding, debugging, and improving models. Decoder directions attribute meaning to latent codes, while encoding directions assess concept influence on predictions, with both enabling model correction by unlearning irrelevant concepts. Unlike prior matrix decomposition, autoencoder, or dictionary learning methods that rely on feature reconstruction, we propose a new perspective: decoding directions are identified via directional clustering of activations, and encoding directions are estimated with signal vectors under a probabilistic view. We further leverage network weights through a novel technique, Uncertainty Region Alignment, which reveals interpretable directions affecting predictions. Our analysis shows that (a) on synthetic data, our method recovers ground-truth direction pairs; (b) on real data, decoding directions map to monosemantic, interpretable concepts and outperform unsupervised baselines; and (c) signal vectors faithfully estimate encoding directions, validated via activation maximization. Finally, we demonstrate applications in understanding global model behavior, explaining individual predictions, and intervening to produce counterfactuals or correct errors.

Method: The approach includes: (1) constructing SAR-GEOVL-1M dataset with geographic projection properties; (2) generating aligned structured text through hierarchical cognitive chain-of-thought (HCoT); (3) designing Self-Consistent Iterative Optimization mechanism for cross-modal alignment; (4) establishing unified evaluation benchmark across 11 downstream tasks.

Result: SAR-KnowLIP demonstrates leading performance compared to 14 leading foundation models, particularly in object counting and land-cover classification tasks.

Conclusion: SAR-KnowLIP’s large-scale multimodal data, transferable model architecture, and comprehensive experimental benchmark are expected to significantly advance the development of SAR multimodal baseline models.

Abstract: Cross-modal artificial intelligence has garnered widespread attention in recent years, achieving significant progress in the study of natural images. However, existing methods are mostly designed for RGB imagery, leaving a significant gap in modeling synthetic aperture radar (SAR) imagery. SAR, with its all-day, all-weather imaging capabilities, plays an irreplaceable role in remote sensing scene understanding. To address this gap, this paper proposes SAR-KnowLIP, the first universal SAR multimodal foundational model, along with reusable data and evaluation baselines. Specifically: (1) This work introduces the critical yet long-overlooked attribute of geographic information into remote sensing research, constructing SAR-GEOVL-1M (the first large-scale SAR dataset with complete geographic projection properties), covering multiple satellite platforms, 120,000 images, and 135 cities. (2) Aligned structured text is generated through a hierarchical cognitive chain-of-thought (HCoT), providing more than one million multi-dimensional semantic annotations of landforms, regional functions, target attributes, and spatial relationships. (3) We design a Self-Consistent Iterative Optimization mechanism that continuously enhances cross-modal alignment through a self-supervised closed loop of contrastive, matching, and reconstruction learning on a transferable multimodal encoder. (4) A unified evaluation benchmark is established across 11 representative downstream vision and vision-language tasks, with comparisons against 14 leading foundation models, where SAR-KnowLIP demonstrates leading performance, particularly in object counting and land-cover classification. We expect that SAR-KnowLIP’s large-scale multimodal data, transferable model architecture, and comprehensive experimental benchmark will significantly advance the development of SAR multimodal baseline models.

Method: Quantifies mutual information between visual and textual tokens, then projects this to a budget-constrained logistic retention curve that adapts to task complexity.

Result: Prunes 89% of visual tokens, reduces FLOPs by 76.8% while retaining 96.7% accuracy on LLaVA-1.5-7B, outperforming PDrop by 9.1%.

Conclusion: Complexity-adaptive pruning effectively reduces computational demands while maintaining performance by aligning token elimination with the model’s reasoning process.

Abstract: The established redundancy in visual tokens within large vision-language models allows pruning to effectively reduce their substantial computational demands. Previous methods typically employ heuristic layer-specific pruning strategies where, although the number of tokens removed may differ across decoder layers, the overall pruning schedule is fixed and applied uniformly to all input samples and tasks, failing to align token elimination with the model’s holistic reasoning trajectory. Cognitive science indicates that human visual processing often begins with broad exploration to accumulate evidence before narrowing focus as the target becomes distinct. Our experiments reveal an analogous pattern in these models. This observation suggests that neither a fixed pruning schedule nor a heuristic layer-wise strategy can optimally accommodate the diverse complexities inherent in different inputs. To overcome this limitation, we introduce Complexity-Adaptive Pruning (AutoPrune), a training-free, plug-and-play framework that tailors pruning policies to varying sample and task complexities. Specifically, AutoPrune quantifies the mutual information between visual and textual tokens, then projects this signal to a budget-constrained logistic retention curve. Each such logistic curve, defined by its unique shape, corresponds to the specific complexity of different tasks and can guarantee adherence to predefined computational constraints. We evaluate AutoPrune on standard vision-language tasks and on Vision-Language-Action models for autonomous driving. Notably, when applied to LLaVA-1.5-7B, our method prunes 89% of visual tokens and reduces inference FLOPs by 76.8% while retaining 96.7% of the original accuracy averaged over all tasks. This corresponds to a 9.1% improvement over the recent work PDrop, demonstrating the effectiveness. Code is available at https://github.com/AutoLab-SAI-SJTU/AutoPrune.

Method: The pipeline performs 3D damage segmentation by up-lifting 2D masks. Uses a learning-free approach for single-view 3D-GS segmentation: projects Gaussians onto image plane using SfM camera parameters, then filters them using Z-buffering with normal distribution model of depth and opacities.

Result: The method is particularly effective for challenging car damage detection scenarios where target objects (scratches, small dents) may only be clearly visible in a single view, making multi-view consistency approaches impractical.

Conclusion: The proposed single-view 3D Gaussian Splatting segmentation approach provides an effective solution for car damage detection in scenarios with limited views, overcoming limitations of multi-view methods.

Abstract: Automatic car damage detection has been a topic of significant interest for the auto insurance industry as it promises faster, accurate, and cost-effective damage assessments. However, few works have gone beyond 2D image analysis to leverage 3D reconstruction methods, which have the potential to provide a more comprehensive and geometrically accurate representation of the damage. Moreover, recent methods employing 3D representations for novel view synthesis, particularly 3D Gaussian Splatting (3D-GS), have demonstrated the ability to generate accurate and coherent 3D reconstructions from a limited number of views. In this work we introduce an automatic car damage detection pipeline that performs 3D damage segmentation by up-lifting 2D masks. Additionally, we propose a simple yet effective learning-free approach for single-view 3D-GS segmentation. Specifically, Gaussians are projected onto the image plane using camera parameters obtained via Structure from Motion (SfM). They are then filtered through an algorithm that utilizes Z-buffering along with a normal distribution model of depth and opacities. Through experiments we found that this method is particularly effective for challenging scenarios like car damage detection, where target objects (e.g., scratches, small dents) may only be clearly visible in a single view, making multi-view consistency approaches impractical or impossible. The code is publicly available at: https://github.com/DragosChileban/CrashSplat.

Method: Used meticulous data curation, advanced architecture design, native Chain-of-Thoughts schema, progressive pre-training, aggressive post-training, and efficient infrastructure. Built a Mixture-of-Experts model with 80B total parameters (13B active per token).

Result: Extensive experiments show HunyuanImage 3.0 rivals previous state-of-the-art models in both automatic and human evaluation of text-image alignment and visual quality.

Conclusion: The model represents a significant advancement in multimodal AI and has been open-sourced to enable community exploration and foster a vibrant multimodal ecosystem.

Abstract: We present HunyuanImage 3.0, a native multimodal model that unifies multimodal understanding and generation within an autoregressive framework, with its image generation module publicly available. The achievement of HunyuanImage 3.0 relies on several key components, including meticulous data curation, advanced architecture design, a native Chain-of-Thoughts schema, progressive model pre-training, aggressive model post-training, and an efficient infrastructure that enables large-scale training and inference. With these advancements, we successfully trained a Mixture-of-Experts (MoE) model comprising over 80 billion parameters in total, with 13 billion parameters activated per token during inference, making it the largest and most powerful open-source image generative model to date. We conducted extensive experiments and the results of automatic and human evaluation of text-image alignment and visual quality demonstrate that HunyuanImage 3.0 rivals previous state-of-the-art models. By releasing the code and weights of HunyuanImage 3.0, we aim to enable the community to explore new ideas with a state-of-the-art foundation model, fostering a dynamic and vibrant multimodal ecosystem. All open source assets are publicly available at https://github.com/Tencent-Hunyuan/HunyuanImage-3.0

Method: Uses Collaborative Multimodal Model Interaction (CMMI) with MLLMs and LLMs for description generation, and Spatial Progressive Augmentation (SPA) to enhance spatial expressiveness among duplicate instances.

Result: Significantly accelerates annotation process while improving quality and discriminability of generated expressions; partially adopted in ICCV 2025 MARS2 Challenge dataset generation.

Conclusion: ColLab enables fully automated REC and REG data generation without human supervision, producing diverse and challenging samples that better reflect real-world reasoning demands.

Abstract: Referring Expression Comprehension (REC) and Referring Expression Generation (REG) are fundamental tasks in multimodal understanding, supporting precise object localization through natural language. However, existing REC and REG datasets rely heavily on manual annotation, which is labor-intensive and difficult to scale. In this paper, we propose ColLab, a collaborative spatial progressive data engine that enables fully automated REC and REG data generation without human supervision. Specifically, our method introduces a Collaborative Multimodal Model Interaction (CMMI) strategy, which leverages the semantic understanding of multimodal large language models (MLLMs) and large language models (LLMs) to generate descriptions. Furthermore, we design a module termed Spatial Progressive Augmentation (SPA) to enhance spatial expressiveness among duplicate instances. Experiments demonstrate that ColLab significantly accelerates the annotation process of REC and REG while improving the quality and discriminability of the generated expressions. In addition to the core methodological contribution, our framework was partially adopted in the data generation pipeline of the ICCV 2025 MARS2 Challenge on Multimodal Reasoning, enriching the dataset with diverse and challenging samples that better reflect real-world reasoning demands.

Method: Proposes RLIR framework that uses an Inverse Dynamics Model to recover input actions from generated videos, mapping high-dimensional video to low-dimensional action space for verifiable reward signals, optimized via Group Relative Policy Optimization.

Result: Experiments show 5-10% gains in action-following, up to 10% improvements in visual quality, and higher human preference scores across autoregressive and diffusion paradigms.

Conclusion: RLIR is the first post-training method specifically designed to enhance action-following in video world models, providing an effective solution without requiring expensive preference annotations.

Abstract: World models simulate dynamic environments, enabling agents to interact with diverse input modalities. Although recent advances have improved the visual quality and temporal consistency of video world models, their ability of accurately modeling human-specified actions remains under-explored. Reinforcement learning presents a promising approach for directly improving the suboptimal action-following capability of pre-trained models, assuming that an appropriate reward function can be defined. However, transferring reinforcement learning post-training methods to world model is impractical due to the prohibitive cost of large-scale preference annotations and the infeasibility of constructing rule-based video verifiers. To address this gap, we propose Reinforcement Learning with Inverse Rewards (RLIR), a post-training framework that derives verifiable reward signals by recovering input actions from generated videos using an Inverse Dynamics Model. By mapping high-dimensional video modality to a low-dimensional action space, RLIR provides an objective and verifiable reward for optimization via Group Relative Policy Optimization. Experiments across autoregressive and diffusion paradigms demonstrate 5-10% gains in action-following, up to 10% improvements in visual quality, and higher human preference scores, establishing RLIR as the first post-training method specifically designed to enhance action-following in video world models.

Method: Couples Adaptive CNN with CDF9/7 wavelets whose detail coefficients are modulated by n-scroll chaotic system to enrich discriminative features. Evaluated on DDTI thyroid ultrasound dataset using 5-fold cross-validation.

Result: Achieves 98.17% accuracy, 98.76% sensitivity, 97.58% specificity, 97.55% F1-score, and AUC of 0.9912. Outperforms state-of-the-art backbones (EfficientNetV2-S, Swin-T, ViT-B/16, ConvNeXt-T) by +1.23 points in accuracy. Chaotic modulation improves accuracy by +8.79 percentage points.

Conclusion: The wavelet-chaos-CNN pipeline delivers state-of-the-art thyroid ultrasound classification with strong generalization, practical runtime characteristics (28.7 ms per image), and suitability for clinical integration.

Abstract: Timely and accurate diagnosis is crucial in addressing the global rise in thyroid cancer, ensuring effective treatment strategies and improved patient outcomes. We present an intelligent classification method that couples an Adaptive Convolutional Neural Network (CNN) with Cohen-Daubechies-Feauveau (CDF9/7) wavelets whose detail coefficients are modulated by an n-scroll chaotic system to enrich discriminative features. We evaluate on the public DDTI thyroid ultrasound dataset (n = 1,638 images; 819 malignant / 819 benign) using 5-fold cross-validation, where the proposed method attains 98.17% accuracy, 98.76% sensitivity, 97.58% specificity, 97.55% F1-score, and an AUC of 0.9912. A controlled ablation shows that adding chaotic modulation to CDF9/7 improves accuracy by +8.79 percentage points over a CDF9/7-only CNN (from 89.38% to 98.17%). To objectively position our approach, we trained state-of-the-art backbones on the same data and splits: EfficientNetV2-S (96.58% accuracy; AUC 0.987), Swin-T (96.41%; 0.986), ViT-B/16 (95.72%; 0.983), and ConvNeXt-T (96.94%; 0.987). Our method outperforms the best of these by +1.23 points in accuracy and +0.0042 in AUC, while remaining computationally efficient (28.7 ms per image; 1,125 MB peak VRAM). Robustness is further supported by cross-dataset testing on TCIA (accuracy 95.82%) and transfer to an ISIC skin-lesion subset (n = 28 unique images, augmented to 2,048; accuracy 97.31%). Explainability analyses (Grad-CAM, SHAP, LIME) highlight clinically relevant regions. Altogether, the wavelet-chaos-CNN pipeline delivers state-of-the-art thyroid ultrasound classification with strong generalization and practical runtime characteristics suitable for clinical integration.

Method: Proposed Validity-First Spatial Intelligence (VFSI) uses energy-based guidance during diffusion sampling to enforce constraints. Incorporates collision avoidance and kinematic constraints as energy functions to guide denoising toward physically valid trajectories.

Result: Across 200 urban scenarios from Waymo Open Motion Dataset, VFSI reduced collision rates by 67% (24.6% to 8.1%) and improved overall validity by 87% (50.3% to 94.2%). Also improved realism metrics (ADE: 1.34m to 1.21m).

Conclusion: Explicit constraint enforcement during inference is both necessary and sufficient for physically valid traffic simulation. Model-agnostic approach demonstrates that physical validity should be an architectural requirement, not emergent property.

Abstract: Modern diffusion models generate realistic traffic simulations but systematically violate physical constraints. In a large-scale evaluation of SceneDiffuser++, a state-of-the-art traffic simulator, we find that 50% of generated trajectories violate basic physical laws - vehicles collide, drive off roads, and spawn inside buildings. This reveals a fundamental limitation: current models treat physical validity as an emergent property rather than an architectural requirement. We propose Validity-First Spatial Intelligence (VFSI), which enforces constraints through energy-based guidance during diffusion sampling, without model retraining. By incorporating collision avoidance and kinematic constraints as energy functions, we guide the denoising process toward physically valid trajectories. Across 200 urban scenarios from the Waymo Open Motion Dataset, VFSI reduces collision rates by 67% (24.6% to 8.1%) and improves overall validity by 87% (50.3% to 94.2%), while simultaneously improving realism metrics (ADE: 1.34m to 1.21m). Our model-agnostic approach demonstrates that explicit constraint enforcement during inference is both necessary and sufficient for physically valid traffic simulation.

Method: Proposes attention specialization routing that assigns attention heads to different patterns according to their intrinsic behaviors, and a progressive training strategy starting with temporally consistent degradations then shifting to inconsistent settings.

Result: Achieves state-of-the-art performance on both synthetic and real-world datasets with 6.2× speedup over one-step diffusion baselines like SeedVR2.

Conclusion: OASIS effectively mitigates redundancy in diffusion models for VSR while preserving pretrained knowledge, enabling efficient and high-performance video super-resolution.

Abstract: Diffusion models have recently shown promising results for video super-resolution (VSR). However, directly adapting generative diffusion models to VSR can result in redundancy, since low-quality videos already preserve substantial content information. Such redundancy leads to increased computational overhead and learning burden, as the model performs superfluous operations and must learn to filter out irrelevant information. To address this problem, we propose OASIS, an efficient $\textbf{o}$ne-step diffusion model with $\textbf{a}$ttention $\textbf{s}$pecialization for real-world v$\textbf{i}$deo $\textbf{s}$uper-resolution. OASIS incorporates an attention specialization routing that assigns attention heads to different patterns according to their intrinsic behaviors. This routing mitigates redundancy while effectively preserving pretrained knowledge, allowing diffusion models to better adapt to VSR and achieve stronger performance. Moreover, we propose a simple yet effective progressive training strategy, which starts with temporally consistent degradations and then shifts to inconsistent settings. This strategy facilitates learning under complex degradations. Extensive experiments demonstrate that OASIS achieves state-of-the-art performance on both synthetic and real-world datasets. OASIS also provides superior inference speed, offering a $\textbf{6.2$\times$}$ speedup over one-step diffusion baselines such as SeedVR2. The code will be available at \href{https://github.com/jp-guo/OASIS}{https://github.com/jp-guo/OASIS}.

Method: Converts 360 images into six-face cubemap representations, uses perspective foundation models to estimate depth and surface normals, and introduces a novel depth scale alignment technique using graph-based optimization with per-face scale parameters to ensure consistency across cubemap faces.

Result: Achieves superior performance across diverse datasets (Matterport3D, Stanford2D3D, 360Loc) and demonstrates benefits in downstream tasks: feature matching (3.2-5.4% improvement) and Structure from Motion (0.2-9.7% improvement in AUC@5).

Conclusion: The proposed RPG360 method provides a robust, training-free solution for 360 monocular depth estimation that leverages foundation models’ zero-shot robustness and graph optimization for depth scale consistency, showing strong performance across multiple datasets and downstream applications.

Abstract: The increasing use of 360 images across various domains has emphasized the need for robust depth estimation techniques tailored for omnidirectional images. However, obtaining large-scale labeled datasets for 360 depth estimation remains a significant challenge. In this paper, we propose RPG360, a training-free robust 360 monocular depth estimation method that leverages perspective foundation models and graph optimization. Our approach converts 360 images into six-face cubemap representations, where a perspective foundation model is employed to estimate depth and surface normals. To address depth scale inconsistencies across different faces of the cubemap, we introduce a novel depth scale alignment technique using graph-based optimization, which parameterizes the predicted depth and normal maps while incorporating an additional per-face scale parameter. This optimization ensures depth scale consistency across the six-face cubemap while preserving 3D structural integrity. Furthermore, as foundation models exhibit inherent robustness in zero-shot settings, our method achieves superior performance across diverse datasets, including Matterport3D, Stanford2D3D, and 360Loc. We also demonstrate the versatility of our depth estimation approach by validating its benefits in downstream tasks such as feature matching 3.2 ~ 5.4% and Structure from Motion 0.2 ~ 9.7% in AUC@5.

Method: Proposes SMART-R1 with R1-style reinforcement fine-tuning for next-token prediction models, featuring metric-oriented policy optimization and iterative SFT-RFT-SFT training strategy alternating between Supervised Fine-Tuning and Reinforcement Fine-Tuning.

Result: Achieves state-of-the-art performance with overall realism meta score of 0.7858 on Waymo Open Sim Agents Challenge, ranking first on the leaderboard.

Conclusion: SMART-R1 demonstrates that simple yet powerful R1-style training framework effectively enhances foundation models for multi-agent traffic simulation, providing better alignment with human preferences and evaluation metrics.

Abstract: Scalable and realistic simulation of multi-agent traffic behavior is critical for advancing autonomous driving technologies. Although existing data-driven simulators have made significant strides in this domain, they predominantly rely on supervised learning to align simulated distributions with real-world driving scenarios. A persistent challenge, however, lies in the distributional shift that arises between training and testing, which often undermines model generalization in unseen environments. To address this limitation, we propose SMART-R1, a novel R1-style reinforcement fine-tuning paradigm tailored for next-token prediction models to better align agent behavior with human preferences and evaluation metrics. Our approach introduces a metric-oriented policy optimization algorithm to improve distribution alignment and an iterative “SFT-RFT-SFT” training strategy that alternates between Supervised Fine-Tuning (SFT) and Reinforcement Fine-Tuning (RFT) to maximize performance gains. Extensive experiments on the large-scale Waymo Open Motion Dataset (WOMD) validate the effectiveness of this simple yet powerful R1-style training framework in enhancing foundation models. The results on the Waymo Open Sim Agents Challenge (WOSAC) showcase that SMART-R1 achieves state-of-the-art performance with an overall realism meta score of 0.7858, ranking first on the leaderboard at the time of submission.

Method: Multimodal deep learning framework integrating echocardiography videos and clinical records using tabular-guided cross-attention and late fusion mechanism.

Result: Outperformed unimodal and non-fused baselines with 67.6% balanced accuracy and 71.1% AUROC. Cross-modality agreement showed 88.6% accuracy for intervention prediction.

Conclusion: TREAT-Net shows potential as a non-invasive tool for timely and accurate patient triage, especially beneficial for underserved populations with limited access to angiography.

Abstract: Coronary angiography remains the gold standard for diagnosing Acute Coronary Syndrome (ACS). However, its resource-intensive and invasive nature can expose patients to procedural risks and diagnostic delays, leading to postponed treatment initiation. In this work, we introduce TREAT-Net, a multimodal deep learning framework for ACS treatment prediction that leverages non-invasive modalities, including echocardiography videos and structured clinical records. TREAT-Net integrates tabular-guided cross-attention to enhance video interpretation, along with a late fusion mechanism to align predictions across modalities. Trained on a dataset of over 9000 ACS cases, the model outperforms unimodal and non-fused baselines, achieving a balanced accuracy of 67.6% and an AUROC of 71.1%. Cross-modality agreement analysis demonstrates 88.6% accuracy for intervention prediction. These findings highlight the potential of TREAT-Net as a non-invasive tool for timely and accurate patient triage, particularly in underserved populations with limited access to coronary angiography.

Method: Introduced a new annotated dataset collected with the NICO robotic platform and evaluated four state-of-the-art gaze estimation models in a shared workspace HRI context.

Result: Angular errors were close to those reported on general-purpose benchmarks, but when converted to actual distance in the shared workspace, the best median error was 16.48 cm, revealing practical limitations of current methods.

Conclusion: Current gaze estimation methods have significant practical limitations in HRI applications despite good angular accuracy, and recommendations are provided for better integration of gaze estimation as a modality in HRI systems.

Abstract: This paper evaluates the current gaze estimation methods within an HRI context of a shared workspace scenario. We introduce a new, annotated dataset collected with the NICO robotic platform. We evaluate four state-of-the-art gaze estimation models. The evaluation shows that the angular errors are close to those reported on general-purpose benchmarks. However, when expressed in terms of distance in the shared workspace the best median error is 16.48 cm quantifying the practical limitations of current methods. We conclude by discussing these limitations and offering recommendations on how to best integrate gaze estimation as a modality in HRI systems.

Method: Fuses identity features from images with semantic text embeddings through a novel conditioning scheme, and introduces a perceptual expression loss using a pre-trained expression classifier to regularize generation.

Result: Significantly improves controllability and realism, outperforming competitive methods in identity preservation and expression fidelity on a single consumer-grade GPU.

Conclusion: SIE3D provides an effective framework for generating high-fidelity 3D avatars with detailed text-based expression control.

Abstract: Generating high-fidelity 3D head avatars from a single image is challenging, as current methods lack fine-grained, intuitive control over expressions via text. This paper proposes SIE3D, a framework that generates expressive 3D avatars from a single image and descriptive text. SIE3D fuses identity features from the image with semantic embedding from text through a novel conditioning scheme, enabling detailed control. To ensure generated expressions accurately match the text, it introduces an innovative perceptual expression loss function. This loss uses a pre-trained expression classifier to regularize the generation process, guaranteeing expression accuracy. Extensive experiments show SIE3D significantly improves controllability and realism, outperforming competitive methods in identity preservation and expression fidelity on a single consumer-grade GPU. Project page: https://blazingcrystal1747.github.io/SIE3D/

Method: FrameMind uses reinforcement learning with Frame-Interleaved Chain-of-Thought (FiCOT) for multi-turn reasoning, Dynamic Resolution Frame Sampling (DRFS) for training, and DRFS-GRPO for policy optimization without frame-level annotations.

Result: Extensive experiments on MLVU and VideoMME benchmarks show significant performance improvements over existing models, advancing state-of-the-art in video understanding.

Conclusion: FrameMind enables flexible and efficient video understanding through dynamic visual information gathering, outperforming traditional static sampling approaches.

Abstract: Current video understanding models rely on fixed frame sampling strategies, processing predetermined visual inputs regardless of the specific reasoning requirements of each question. This static approach limits their ability to adaptively gather visual evidence, leading to suboptimal performance on tasks that require either broad temporal coverage or fine-grained spatial detail. In this paper, we introduce FrameMind, an end-to-end framework trained with reinforcement learning that enables models to dynamically request visual information during reasoning through Frame-Interleaved Chain-of-Thought (FiCOT). Unlike traditional approaches, FrameMind operates in multiple turns where the model alternates between textual reasoning and active visual perception, using tools to extract targeted frames or video clips based on identified knowledge gaps. To train effective dynamic sampling policies, we propose Dynamic Resolution Frame Sampling (DRFS), which exposes models to diverse temporal-spatial trade-offs during learning, and DRFS-GRPO, a group-relative policy optimization algorithm that learns from outcome-based rewards without requiring frame-level annotations. Extensive experiments on challenging benchmarks like MLVU and VideoMME demonstrate that our method significantly outperforms existing models, advancing the state of the art in flexible and efficient video understanding.

Method: Proposes prototype subspace modeling using basis vectors per category with orthogonality and reconstruction constraints for better class structure representation.

Result: Significantly outperforms state-of-the-art GCD methods on real-world HSI datasets.

Conclusion: Establishes a strong foundation for generalized category discovery in hyperspectral settings with the proposed subspace modeling approach.

Abstract: Generalized category discovery~(GCD) seeks to jointly identify both known and novel categories in unlabeled data. While prior works have mainly focused on RGB images, their assumptions and modeling strategies do not generalize well to hyperspectral images~(HSI), which are inherently high-dimensional and exhibit complex spectral structures. In this paper, we propose the first GCD framework tailored for HSI, introducing a prototype subspace modeling model to better capture class structure. Instead of learning a single prototype vector for each category as in existing methods such as SimGCD, we model each category using a set of basis vectors, forming a subspace representation that enables greater expressiveness and discrimination in a high-dimensional feature space. To guide the learning of such bases, we enforce two key constraints: (1) a basis orthogonality constraint that promotes inter-class separability, and (2) a reconstruction constraint that ensures each prototype basis can effectively reconstruct its corresponding class samples. Experimental results on real-world HSI demonstrate that our method significantly outperforms state-of-the-art GCD methods, establishing a strong foundation for generalized category discovery in hyperspectral settings.

Method: Combines differentiable atmospheric scattering model for haze mitigation, adaptive Mamba variant for feature extraction, and heterogeneous graph attention network with spatio-temporal fusion for pedestrian relationship modeling.

Result: 37.2% and 41.5% reduction in minADE/minFDE metrics compared to SOTA in dense haze scenarios (visibility < 30m), with 78% inference speed increase over native Mamba.

Conclusion: Provides a new modeling paradigm for reliable perception in intelligent transportation systems in adverse environments.

Abstract: To address the issues of physical information degradation and ineffective pedestrian interaction modeling in pedestrian trajectory prediction under hazy weather conditions, we propose a deep learning model that combines physical priors of atmospheric scattering with topological modeling of pedestrian relationships. Specifically, we first construct a differentiable atmospheric scattering model that decouples haze concentration from light degradation through a network with physical parameter estimation, enabling the learning of haze-mitigated feature representations. Second, we design an adaptive scanning state space model for feature extraction. Our adaptive Mamba variant achieves a 78% inference speed increase over native Mamba while preserving long-range dependency modeling. Finally, to efficiently model pedestrian relationships, we develop a heterogeneous graph attention network, using graph matrices to model multi-granularity interactions between pedestrians and groups, combined with a spatio-temporal fusion module to capture the collaborative evolution patterns of pedestrian movements. Furthermore, we constructed a new pedestrian trajectory prediction dataset based on ETH/UCY to evaluate the effectiveness of the proposed method. Experiments show that our method reduces the minADE / minFDE metrics by 37.2% and 41.5%, respectively, compared to the SOTA models in dense haze scenarios (visibility < 30m), providing a new modeling paradigm for reliable perception in intelligent transportation systems in adverse environments.

Method: The paper evaluates post-ISP and Bayer reconstruction pipelines, curates Raw-Rain (first public benchmark of real rainy scenes in both 12-bit Bayer and bit-depth-matched sRGB), and introduces Information Conservation Score (ICS) as a color-invariant metric. A raw-domain model is developed for reconstruction.

Result: On the test split, the raw-domain model improves sRGB results by up to +0.99 dB PSNR and +1.2% ICS, while running faster with half of the GFLOPs compared to traditional approaches.

Conclusion: The results advocate for an ISP-last paradigm for low-level vision and open the door to end-to-end learnable camera pipelines by demonstrating superior performance when learning directly on raw Bayer data.

Abstract: Image reconstruction from corrupted images is crucial across many domains. Most reconstruction networks are trained on post-ISP sRGB images, even though the image-signal-processing pipeline irreversibly mixes colors, clips dynamic range, and blurs fine detail. This paper uses the rain degradation problem as a use case to show that these losses are avoidable, and demonstrates that learning directly on raw Bayer mosaics yields superior reconstructions. To substantiate the claim, we (i) evaluate post-ISP and Bayer reconstruction pipelines, (ii) curate Raw-Rain, the first public benchmark of real rainy scenes captured in both 12-bit Bayer and bit-depth-matched sRGB, and (iii) introduce Information Conservation Score (ICS), a color-invariant metric that aligns more closely with human opinion than PSNR or SSIM. On the test split, our raw-domain model improves sRGB results by up to +0.99 dB PSNR and +1.2% ICS, while running faster with half of the GFLOPs. The results advocate an ISP-last paradigm for low-level vision and open the door to end-to-end learnable camera pipelines.

Method: Joint optimization framework with feedback mechanism: self-representation network based on unfolded ADMM guides a differentiable superpixel module, with each superpixel learning a unique compactness parameter for adaptive segmentation.

Result: Extensive experiments on benchmark HSI datasets show superior clustering accuracy compared to state-of-the-art methods, with clustering-aware partitions that preserve both spectral and spatial structure.

Conclusion: The proposed unified framework effectively addresses the misalignment between segmentation and clustering objectives, achieving better performance through joint optimization and adaptive superpixel learning.

Abstract: Subspace clustering is a powerful unsupervised approach for hyperspectral image (HSI) analysis, but its high computational and memory costs limit scalability. Superpixel segmentation can improve efficiency by reducing the number of data points to process. However, existing superpixel-based methods usually perform segmentation independently of the clustering task, often producing partitions that do not align with the subsequent clustering objective. To address this, we propose a unified end-to-end framework that jointly optimizes superpixel segmentation and subspace clustering. Its core is a feedback mechanism: a self-representation network based on unfolded Alternating Direction Method of Multipliers (ADMM) provides a model-driven signal to guide a differentiable superpixel module. This joint optimization yields clustering-aware partitions that preserve both spectral and spatial structure. Furthermore, our superpixel network learns a unique compactness parameter for each superpixel, enabling more flexible and adaptive segmentation. Extensive experiments on benchmark HSI datasets demonstrate that our method consistently achieves superior accuracy compared with state-of-the-art clustering approaches.

Method: Investigate pruning-at-initialization on pre-trained vision models using one task’s data, then evaluate zero-shot performance on unseen tasks and fine-tuning recovery.

Result: Pruning on one task retains zero-shot performance on unseen tasks, and fine-tuning improves performance on both seen and held-out tasks.

Conclusion: Extensive pre-training creates favorable loss landscapes that enable effective pruning without task-specific data, maintaining performance across tasks.

Abstract: The widespread availability of pre-trained vision models has enabled numerous deep learning applications through their transferable representations. However, their computational and storage costs often limit practical deployment. Pruning-at-Initialization has emerged as a promising approach to compress models before training, enabling efficient task-specific adaptation. While conventional wisdom suggests that effective pruning requires task-specific data, this creates a challenge when downstream tasks are unknown in advance. In this paper, we investigate how data influences the pruning of pre-trained vision models. Surprisingly, pruning on one task retains the model’s zero-shot performance also on unseen tasks. Furthermore, fine-tuning these pruned models not only improves performance on original seen tasks but can recover held-out tasks’ performance. We attribute this phenomenon to the favorable loss landscapes induced by extensive pre-training on large-scale datasets.

Method: Used representation analysis to identify Grounding IDs in embedding space and performed causal interventions to confirm their role in object-symbol binding.

Result: Found that Grounding IDs create robust within-partition alignment, reduce modality gap, mediate binding between objects and cues, and strengthen attention between related components.

Conclusion: Grounding IDs are a key symbolic mechanism that explains how external visual cues enhance multimodal binding, offering both interpretability and practical robustness improvements.

Abstract: Large vision-language models (LVLMs) show strong performance across multimodal benchmarks but remain limited in structured reasoning and precise grounding. Recent work has demonstrated that adding simple visual structures, such as partitions and annotations, improves accuracy, yet the internal mechanisms underlying these gains remain unclear. We investigate this phenomenon and propose the concept of Grounding IDs, latent identifiers induced by external cues that bind objects to their designated partitions across modalities. Through representation analysis, we find that these identifiers emerge as robust within-partition alignment in embedding space and reduce the modality gap between image and text. Causal interventions further confirm that these identifiers mediate binding between objects and symbolic cues. We show that Grounding IDs strengthen attention between related components, which in turn improves cross-modal grounding and reduces hallucinations. Taken together, our results identify Grounding IDs as a key symbolic mechanism explaining how external cues enhance multimodal binding, offering both interpretability and practical improvements in robustness.

Method: VideoAR is a unified framework that supports multiple prediction units including full frames, key-detail frames, multiscale refinements, and spatiotemporal cubes. The core innovation is using spatiotemporal cubes as prediction units for autoregressive modeling across both spatial and temporal dimensions.

Result: Cube-based prediction consistently delivers superior quality, speed, and temporal coherence compared to frame-based approaches. The method surpasses state-of-the-art baselines on VBench while achieving faster inference and scaling to minute-long sequences.

Conclusion: The work demonstrates that removing the frame-by-frame constraint enables better video generation and motivates rethinking sequence decomposition in spatiotemporal domains beyond traditional frame-based approaches.

Abstract: Autoregressive models for video generation typically operate frame-by-frame, extending next-token prediction from language to video’s temporal dimension. We question that unlike word as token is universally agreed in language if frame is a appropriate prediction unit? To address this, we present VideoAR, a unified framework that supports a spectrum of prediction units including full frames, key-detail frames, multiscale refinements, and spatiotemporal cubes. Among these designs, we find model video generation using \textit{spatiotemporal} cubes as prediction units, which allows autoregressive models to operate across both spatial and temporal dimensions simultaneously. This approach eliminates the assumption that frames are the natural atomic units for video autoregression. We evaluate VideoAR across diverse prediction strategies, finding that cube-based prediction consistently delivers superior quality, speed, and temporal coherence. By removing the frame-by-frame constraint, our video generator surpasses state-of-the-art baselines on VBench while achieving faster inference and enabling seamless scaling to minute-long sequences. We hope this work will motivate rethinking sequence decomposition in video and other spatiotemporal domains.

Method: Uses rectified flow for deterministic straight-line sampling paths between noise and data, employs Retrieval-Augmented Generation (RAG) with music features and LLM-based text decompositions, contrastive objective for alignment, and synchronization loss for inter-person coordination.

Result: Extensive evaluations show consistent gains in motion quality, responsiveness, and efficiency. Produces temporally coherent and rhythmically synchronized motions.

Conclusion: DualFlow sets state-of-the-art in multi-modal human motion generation, offering improved performance across text-to-motion, music-to-motion, and multi-modal interactive tasks.

Abstract: Generating realistic, context-aware two-person motion conditioned on diverse modalities remains a central challenge in computer graphics, animation, and human-computer interaction. We introduce DualFlow, a unified and efficient framework for multi-modal two-person motion generation. DualFlow conditions 3D motion synthesis on diverse inputs, including text, music, and prior motion sequences. Leveraging rectified flow, it achieves deterministic straight-line sampling paths between noise and data, reducing inference time and mitigating error accumulation common in diffusion-based models. To enhance semantic grounding, DualFlow employs a Retrieval-Augmented Generation (RAG) module that retrieves motion exemplars using music features and LLM-based text decompositions of spatial relations, body movements, and rhythmic patterns. We use contrastive objective that further strengthens alignment with conditioning signals and introduce synchronization loss that improves inter-person coordination. Extensive evaluations across text-to-motion, music-to-motion, and multi-modal interactive benchmarks show consistent gains in motion quality, responsiveness, and efficiency. DualFlow produces temporally coherent and rhythmically synchronized motions, setting state-of-the-art in multi-modal human motion generation.

Method: SVAC uses Anchor-Based Spatio-Temporal Compression (ASTC) to compress visual tokens while preserving structure, and Clip-Specific Allocation (CSA) strategy to handle dynamic object behaviors across video clips.

Result: SVAC achieves state-of-the-art performance on multiple RVOS benchmarks with competitive efficiency.

Conclusion: The proposed SVAC model effectively addresses computational challenges in RVOS while improving segmentation performance through enhanced video-language interaction and temporal dynamics handling.

Abstract: Referring Video Object Segmentation (RVOS) aims to segment target objects in video sequences based on natural language descriptions. While recent advances in Multi-modal Large Language Models (MLLMs) have improved RVOS performance through enhanced text-video understanding, several challenges remain, including insufficient exploitation of MLLMs’ prior knowledge, prohibitive computational and memory costs for long-duration videos, and inadequate handling of complex temporal dynamics. In this work, we propose SVAC, a unified model that improves RVOS by scaling up input frames and segmentation tokens to enhance video-language interaction and segmentation precision. To address the resulting computational challenges, SVAC incorporates the Anchor-Based Spatio-Temporal Compression (ASTC) module to compress visual tokens while preserving essential spatio-temporal structure. Moreover, the Clip-Specific Allocation (CSA) strategy is introduced to better handle dynamic object behaviors across video clips. Experimental results demonstrate that SVAC achieves state-of-the-art performance on multiple RVOS benchmarks with competitive efficiency. Our code is available at https://github.com/lizhang1998/SVAC.

Method: Systematically compares VAE, GAN, and DDPM performance using a dataset of 10,314 Japanese Kanji characters, evaluating image fidelity with FID scores and sampling time.

Result: DDPM achieved highest image fidelity (FID score of 26.2) but was over 2,000 times slower in sampling time compared to VAE and GAN.

Conclusion: GANji is an effective and accessible framework that reveals fundamental trade-offs between model architecture, computational cost, and visual quality, suitable for both educational and research purposes.

Abstract: The comparative study of generative models often requires significant computational resources, creating a barrier for researchers and practitioners. This paper introduces GANji, a lightweight framework for benchmarking foundational AI image generation techniques using a dataset of 10,314 Japanese Kanji characters. It systematically compares the performance of a Variational Autoencoder (VAE), a Generative Adversarial Network (GAN), and a Denoising Diffusion Probabilistic Model (DDPM). The results demonstrate that while the DDPM achieves the highest image fidelity, with a Fr'echet Inception Distance (FID) score of 26.2, its sampling time is over 2,000 times slower than the other models. The GANji framework is an effective and accessible tool for revealing the fundamental trade-offs between model architecture, computational cost, and visual quality, making it ideal for both educational and research purposes.

Method: A synergistic framework with five stages using hierarchical search and cross-modal communication, where general VLMs scan for regions of interest and fine-tuned grounding models locate precise coordinates.

Result: GMS achieves 35.7% accuracy on ScreenSpot-Pro dataset, representing a 10× improvement over individual components (2.0% for Scanner, 3.7% for Locator) and outperforms other baselines.

Conclusion: The GMS framework demonstrates robust performance and potential for general-purpose GUI grounding by effectively leveraging complementary strengths of different model types.

Abstract: Grounding natural language queries in graphical user interfaces (GUIs) presents a challenging task that requires models to comprehend diverse UI elements across various applications and systems, while also accurately predicting the spatial coordinates for the intended operation. To tackle this problem, we propose GMS: Generalist Scanner Meets Specialist Locator, a synergistic coarse-to-fine framework that effectively improves GUI grounding performance. GMS leverages the complementary strengths of general vision-language models (VLMs) and small, task-specific GUI grounding models by assigning them distinct roles within the framework. Specifically, the general VLM acts as a ‘Scanner’ to identify potential regions of interest, while the fine-tuned grounding model serves as a ‘Locator’ that outputs precise coordinates within these regions. This design is inspired by how humans perform GUI grounding, where the eyes scan the interface and the brain focuses on interpretation and localization. Our whole framework consists of five stages and incorporates hierarchical search with cross-modal communication to achieve promising prediction results. Experimental results on the ScreenSpot-Pro dataset show that while the ‘Scanner’ and ‘Locator’ models achieve only $2.0%$ and $3.7%$ accuracy respectively when used independently, their integration within GMS framework yields an overall accuracy of $35.7%$, representing a $10 \times$ improvement. Additionally, GMS significantly outperforms other strong baselines under various settings, demonstrating its robustness and potential for general-purpose GUI grounding.

Method: Benchmarked three pre-trained CNN models (VGG16, VGG19, ResNet50) on 21,577 retinal fundus images from Retinal Disease Dataset, with fine-tuned VGG16 achieving best performance. Integrated Grad-CAM for visual explanations.

Result: Fine-tuned VGG16 achieved state-of-the-art 92.36% test accuracy in classifying 10 retinal conditions, outperforming other models.

Conclusion: EYE-DEX provides accurate automated retinal disease diagnosis with transparent visual explanations, enhancing clinician trust in AI-assisted diagnostics.

Abstract: Retinal disease diagnosis is critical in preventing vision loss and reducing socioeconomic burdens. Globally, over 2.2 billion people are affected by some form of vision impairment, resulting in annual productivity losses estimated at $411 billion. Traditional manual grading of retinal fundus images by ophthalmologists is time-consuming and subjective. In contrast, deep learning has revolutionized medical diagnostics by automating retinal image analysis and achieving expert-level performance. In this study, we present EYE-DEX, an automated framework for classifying 10 retinal conditions using the large-scale Retinal Disease Dataset comprising 21,577 eye fundus images. We benchmark three pre-trained Convolutional Neural Network (CNN) models–VGG16, VGG19, and ResNet50–with our finetuned VGG16 achieving a state-of-the-art global benchmark test accuracy of 92.36%. To enhance transparency and explainability, we integrate the Gradient-weighted Class Activation Mapping (Grad-CAM) technique to generate visual explanations highlighting disease-specific regions, thereby fostering clinician trust and reliability in AI-assisted diagnostics.

Method: Projects images into visual tokens in joint semantic space, trains language model to generate latent states that reconstruct key visual tokens, and interleaves LVR with text generation using adapted GRPO algorithm for reinforcement learning.

Result: Achieves 71.67% on MMVP benchmark compared to 66.67% with Qwen2.5-VL, showing substantial improvement in fine-grained visual understanding and perception.

Conclusion: LVR paradigm enables direct visual reasoning in embedding space, significantly enhancing perception capabilities of multimodal models beyond language-only reasoning approaches.

Abstract: Multimodal Large Language Models (MLLMs) have achieved notable gains in various tasks by incorporating Chain-of-Thought (CoT) reasoning in language spaces. Recent work extends this direction by leveraging external tools for visual editing, thereby enhancing the visual signal along the reasoning trajectories. Nevertheless, these approaches remain fundamentally constrained: reasoning is still confined to the language space, with visual information treated as static preconditions. We introduce Latent Visual Reasoning (LVR), a new paradigm that enables autoregressive reasoning directly in the visual embedding space. A visual encoder first projects images into visual tokens within a joint semantic space shared with the language model. The language model is then trained to generate latent states that reconstruct key visual tokens critical for answering the query, constituting the process of latent visual reasoning. By interleaving LVR with standard text generation, our model achieves substantial gains on perception-intensive visual question answering tasks. In addition, we adapt the GRPO algorithm to conduct reinforcement learning on latent reasoning, further balancing LVR and textual generation. We show that LVR substantially improves fine-grained visual understanding and perception, achieving 71.67% on MMVP compared to 66.67% with Qwen2.5-VL. Code base and model weights will be released later.

Method: Comparative analysis of models trained on SCUT-FBP5500 and MEBeauty datasets using rigorous statistical validation (Kruskal-Wallis H-tests, post hoc Dunn analyses) and cross-dataset validation on the balanced FairFace dataset.

Result: Both models exhibited significant prediction disparities across ethnic groups (p < 0.001), with only 4.8-9.5% of inter-group comparisons satisfying distributional parity criteria, showing algorithmic amplification of societal beauty biases rather than mitigation.

Conclusion: Current AI beauty prediction approaches are inadequate, and mitigation strategies are needed to address the significant ethnicity-based bias in facial beauty prediction systems.

Abstract: Bias can be introduced to AI systems even from seemingly balanced sources, and AI facial beauty prediction is subject to ethnicity-based bias. This work sounds warnings about AI’s role in shaping aesthetic norms while providing potential pathways toward equitable beauty technologies through comparative analysis of models trained on SCUT-FBP5500 and MEBeauty datasets. Employing rigorous statistical validation (Kruskal-Wallis H-tests, post hoc Dunn analyses). It is demonstrated that both models exhibit significant prediction disparities across ethnic groups $(p < 0.001)$, even when evaluated on the balanced FairFace dataset. Cross-dataset validation shows algorithmic amplification of societal beauty biases rather than mitigation based on prediction and error parity. The findings underscore the inadequacy of current AI beauty prediction approaches, with only 4.8-9.5% of inter-group comparisons satisfying distributional parity criteria. Mitigation strategies are proposed and discussed in detail.

Method: Proposes FastVSR with f16 VAE (16x spatial compression), uses pixel shuffle and channel replication for upsampling, and implements a lower-bound-guided training strategy for stable convergence.

Result: Achieves speedups of 111.9x compared to multi-step models and 3.92x compared to existing one-step models while maintaining performance.

Conclusion: FastVSR demonstrates substantial improvements in inference efficiency for video super-resolution through high compression VAE and stable training framework, making diffusion models more practical for real-world applications.

Abstract: Diffusion models have significant advantages in the field of real-world video super-resolution and have demonstrated strong performance in past research. In recent diffusion-based video super-resolution (VSR) models, the number of sampling steps has been reduced to just one, yet there remains significant room for further optimization in inference efficiency. In this paper, we propose FastVSR, which achieves substantial reductions in computational cost by implementing a high compression VAE (spatial compression ratio of 16, denoted as f16). We design the structure of the f16 VAE and introduce a stable training framework. We employ pixel shuffle and channel replication to achieve additional upsampling. Furthermore, we propose a lower-bound-guided training strategy, which introduces a simpler training objective as a lower bound for the VAE’s performance. It makes the training process more stable and easier to converge. Experimental results show that FastVSR achieves speedups of 111.9 times compared to multi-step models and 3.92 times compared to existing one-step models. We will release code and models at https://github.com/JianzeLi-114/FastVSR.

Method: Fine-tuned CNN models (VGG16, ResNet50, Xception) for tumor classification combined with YOLOv8 for tumor localization using bounding boxes on MRI data from Brain Tumor MRI Dataset.

Result: Fine-tuned VGG16 achieved 99.86% test accuracy, substantially exceeding previous benchmarks, with enhanced clinical interpretability through localization and explainable AI.

Conclusion: This approach demonstrates the transformative potential of deep learning for faster, more reliable brain tumor diagnoses, contributing to improved patient care and survival rates.

Abstract: The early and accurate classification of brain tumors is crucial for guiding effective treatment strategies and improving patient outcomes. This study presents BrainFusion, a significant advancement in brain tumor analysis using magnetic resonance imaging (MRI) by combining fine-tuned convolutional neural networks (CNNs) for tumor classification–including VGG16, ResNet50, and Xception–with YOLOv8 for precise tumor localization with bounding boxes. Leveraging the Brain Tumor MRI Dataset, our experiments reveal that the fine-tuned VGG16 model achieves test accuracy of 99.86%, substantially exceeding previous benchmarks. Beyond setting a new accuracy standard, the integration of bounding-box localization and explainable AI techniques further enhances both the clinical interpretability and trustworthiness of the system’s outputs. Overall, this approach underscores the transformative potential of deep learning in delivering faster, more reliable diagnoses, ultimately contributing to improved patient care and survival rates.

Method: Propose two strategies: QBP synthesizes diverse questions into holistic narrative paragraphs reconstructing event structure; QBC generates fine-grained visual rationales grounding answers in specific evidence. Use synthetic data to train models under unified next-token prediction.

Result: Achieves state-of-the-art results: 72.5% on STAR (+4.9%) with 3B model, 80.8% on NExT-QA with 7B model. Both QBP and QBC enhance cross-dataset generalization, with QBP accelerating convergence by over 2.5x.

Conclusion: Shifting from isolated facts to narrative coherence and grounded rationales creates a more accurate, efficient, and generalizable VideoQA training paradigm.

Abstract: The performance of Video Question Answering (VideoQA) models is fundamentally constrained by the nature of their supervision, which typically consists of isolated, factual question-answer pairs. This “bag-of-facts” approach fails to capture the underlying narrative and causal structure of events, limiting models to a shallow understanding of video content. To move beyond this paradigm, we introduce a framework to synthesize richer supervisory signals. We propose two complementary strategies: Question-Based Paraphrasing (QBP), which synthesizes the diverse inquiries (what, how, why) from a video’s existing set of question-answer pairs into a holistic narrative paragraph that reconstructs the video’s event structure; and Question-Based Captioning (QBC), which generates fine-grained visual rationales, grounding the answer to each question in specific, relevant evidence. Leveraging powerful generative models, we use this synthetic data to train VideoQA models under a unified next-token prediction objective. Extensive experiments on STAR and NExT-QA validate our approach, demonstrating significant accuracy gains and establishing new state-of-the-art results, such as improving a 3B model to 72.5% on STAR (+4.9%) and a 7B model to 80.8% on NExT-QA. Beyond accuracy, our analysis reveals that both QBP and QBC substantially enhance cross-dataset generalization, with QBP additionally accelerating model convergence by over 2.5x. These results demonstrate that shifting data synthesis from isolated facts to narrative coherence and grounded rationales yields a more accurate, efficient, and generalizable training paradigm.

Method: A dual-stage architecture with attention-based FFC module for anatomical feature integration and Spatial Deformation Network for morphological variations. Uses cross-modality translation from RGBD input to radiographic domain.

Result: LatXGen produces anatomically accurate radiographs and outperforms existing GAN-based methods in both visual fidelity and quantitative metrics, using a dataset of 3,264 RGBD and lateral radiograph pairs.

Conclusion: The framework offers a promising radiation-free solution for sagittal spine assessment and advances comprehensive AIS evaluation by enabling reliable sagittal alignment estimation without ionizing radiation.

Abstract: Adolescent Idiopathic Scoliosis (AIS) is a complex three-dimensional spinal deformity, and accurate morphological assessment requires evaluating both coronal and sagittal alignment. While previous research has made significant progress in developing radiation-free methods for coronal plane assessment, reliable and accurate evaluation of sagittal alignment without ionizing radiation remains largely underexplored. To address this gap, we propose LatXGen, a novel generative framework that synthesizes realistic lateral spinal radiographs from posterior Red-Green-Blue and Depth (RGBD) images of unclothed backs. This enables accurate, radiation-free estimation of sagittal spinal alignment. LatXGen tackles two core challenges: (1) inferring sagittal spinal morphology changes from a lateral perspective based on posteroanterior surface geometry, and (2) performing cross-modality translation from RGBD input to the radiographic domain. The framework adopts a dual-stage architecture that progressively estimates lateral spinal structure and synthesizes corresponding radiographs. To enhance anatomical consistency, we introduce an attention-based Fast Fourier Convolution (FFC) module for integrating anatomical features from RGBD images and 3D landmarks, and a Spatial Deformation Network (SDN) to model morphological variations in the lateral view. Additionally, we construct the first large-scale paired dataset for this task, comprising 3,264 RGBD and lateral radiograph pairs. Experimental results demonstrate that LatXGen produces anatomically accurate radiographs and outperforms existing GAN-based methods in both visual fidelity and quantitative metrics. This study offers a promising, radiation-free solution for sagittal spine assessment and advances comprehensive AIS evaluation.

Method: Created Euclid30K dataset with 30K geometry problems, then used Group Relative Policy Optimization (GRPO) to finetune Qwen2.5VL and RoboBrain2.0 models, enabling them to identify shapes, count entities, and perform multi-step deductive reasoning using Euclidean principles.

Result: Models achieved substantial zero-shot gains across four spatial reasoning benchmarks (Super-CLEVR, Omni3DBench, VSI-Bench, and MindCube). Mean VSI-Bench accuracy rose from 34.5% to 40.5%, with RoboBrain2.0-Euclid-7B achieving 49.6% accuracy, surpassing previous state-of-the-art.

Conclusion: This is the first systematic study showing that geometry-centric fine-tuning can confer vision-language models with broadly transferable spatial skills, demonstrating the effectiveness of using Euclidean geometry as a surrogate task for spatial intelligence.

Abstract: Spatial intelligence spans a rich suite of abilities, including visualising and transforming shapes, mentally rotating objects, judging relational positions and containment, and estimating numerosity. However, it still remains a critical unresolved challenge for Multimodal Large Language Models (MLLMs).To fill this gap, we propose to treat Euclidean geometry problem-solving as a surrogate task. Specifically, we meticulously constructed a curated multimodal dataset, called Euclid30K, comprising approximately 30K plane and solid geometry problems. To enable the model to acquire and apply Euclidean principles from these geometry problems, we employed Group Relative Policy Optimization (GRPO) to finetune the Qwen2.5VL family and RoboBrain2.0 family, inspiring the models to identify shapes, count, and relate entities, and perform multi-step deductive reasoning using Euclidean principles. Our experiments demonstrate that the resulting models achieve substantial zero-shot gains across four spatial reasoning benchmarks (Super-CLEVR, Omni3DBench, VSI-Bench, and MindCube) without any task-specific adaptations. Notably, after training on the Euclid30K, the mean VSI-Bench accuracy of all evaluated models rose from 34.5% to 40.5%, improving by 5.5 percentage points. Among them, RoboBrain2.0-Euclid-7B achieves 49.6% accuracy, surpassing the previous state-of-the-art model, Spatial-MLLM.To our knowledge, this is the first systematic study showing that geometry-centric fine-tuning can confer vision-language models with broadly transferable spatial skills. Code and Euclid30K dataset can be found in https://zgca-ai4edu.github.io/Euclids_Gift.

Method: Uses shape-wise potential to capture geometric structure of parameter trajectories and easy-to-complex matching strategy that progressively addresses parameters based on complexity.

Result: Experiments on medical image classification show improved distillation performance while preserving privacy and maintaining model accuracy comparable to training on original datasets.

Conclusion: The proposed method effectively addresses limitations of existing trajectory matching approaches for medical dataset distillation, enabling better data sharing while protecting privacy.

Abstract: Medical image analysis faces significant challenges in data sharing due to privacy regulations and complex institutional protocols. Dataset distillation offers a solution to address these challenges by synthesizing compact datasets that capture essential information from real, large medical datasets. Trajectory matching has emerged as a promising methodology for dataset distillation; however, existing methods primarily focus on terminal states, overlooking crucial information in intermediate optimization states. We address this limitation by proposing a shape-wise potential that captures the geometric structure of parameter trajectories, and an easy-to-complex matching strategy that progressively addresses parameters based on their complexity. Experiments on medical image classification tasks demonstrate that our method improves distillation performance while preserving privacy and maintaining model accuracy comparable to training on the original datasets. Our code is available at https://github.com/Bian-jh/HoP-TM.

Method: DECERN introduces a multifaceted informativeness measure combining discrepancy-confusion uncertainty (quantifying category directionality and structural stability) and calibration diversity (uncertainty-weighted clustering to diversify samples while maintaining local representativeness).

Result: Extensive experiments on 7 fine-grained image datasets across 26 experimental settings demonstrate superior performance compared to state-of-the-art methods.

Conclusion: The proposed DECERN method effectively addresses the challenges of active learning in fine-grained image classification by combining uncertainty and diversity measures to select the most valuable samples for annotation.

Abstract: Active learning (AL) aims to build high-quality labeled datasets by iteratively selecting the most informative samples from an unlabeled pool under limited annotation budgets. However, in fine-grained image classification, assessing this informativeness is especially challenging due to subtle inter-class differences. In this paper, we introduce a novel method, combining discrepancy-confusion uncertainty and calibration diversity for active fine-grained image classification (DECERN), to effectively perceive the distinctiveness between fine-grained images and evaluate the sample value. DECERN introduces a multifaceted informativeness measure that combines discrepancy-confusion uncertainty and calibration diversity. The discrepancy-confusion uncertainty quantifies the category directionality and structural stability of fine-grained unlabeled data during local feature fusion. Subsequently, uncertainty-weighted clustering is performed to diversify the uncertainty samples. Then we calibrate the diversity to maximize the global diversity of the selected sample while maintaining its local representativeness. Extensive experiments conducted on 7 fine-grained image datasets across 26 distinct experimental settings demonstrate that our method achieves superior performance compared to state-of-the-art methods.

Method: Developed a scalable automated data generation pipeline to create high-quality video question answering data for the NeMo task, which assesses long-context recall and temporal grounding.

Result: Generated NeMoBench with 31,378 QA pairs from 13,486 videos of varying durations. Evaluated 20 state-of-the-art models, providing extensive results and insights into their capabilities and limitations.

Conclusion: The proposed pipeline reliably generates high-quality evaluation data, enabling continuous updates to NeMoBench with the latest videos, and provides comprehensive assessment of VideoLLMs’ temporal reasoning abilities.

Abstract: Recent advances in video large language models (VideoLLMs) call for new evaluation protocols and benchmarks for complex temporal reasoning in video-language understanding. Inspired by the needle in a haystack test widely used by LLMs, we introduce a novel task of Needle in a Montage (NeMo), designed to assess VideoLLMs’ critical reasoning capabilities, including long-context recall and temporal grounding. To generate video question answering data for our task, we develop a scalable automated data generation pipeline that facilitates high-quality data synthesis. Built upon the proposed pipeline, we present NeMoBench, a video-language benchmark centered on our task. Specifically, our full set of NeMoBench features 31,378 automatically generated question-answer (QA) pairs from 13,486 videos with various durations ranging from seconds to hours. Experiments demonstrate that our pipeline can reliably and automatically generate high-quality evaluation data, enabling NeMoBench to be continuously updated with the latest videos. We evaluate 20 state-of-the-art models on our benchmark, providing extensive results and key insights into their capabilities and limitations. Our project page is available at: https://lavi-lab.github.io/NeMoBench.

Method: GAN-based model for tumor mask generation and diffusion-based approach for tumor texture generation, both conditioned on radiomics features (size, shape, texture).

Result: Successfully tested on tumors in four organs (kidney, lung, breast, brain) across CT and MRI; synthesized images aid downstream task training and pass expert authenticity evaluations.

Conclusion: The method enables flexible tumor manipulation and generation, showing potential for treatment planning with diverse synthesized tumors.

Abstract: Due to privacy concerns, obtaining large datasets is challenging in medical image analysis, especially with 3D modalities like Computed Tomography (CT) and Magnetic Resonance Imaging (MRI). Existing generative models, developed to address this issue, often face limitations in output diversity and thus cannot accurately represent 3D medical images. We propose a tumor-generation model that utilizes radiomics features as generative conditions. Radiomics features are high-dimensional handcrafted semantic features that are biologically well-grounded and thus are good candidates for conditioning. Our model employs a GAN-based model to generate tumor masks and a diffusion-based approach to generate tumor texture conditioned on radiomics features. Our method allows the user to generate tumor images according to user-specified radiomics features such as size, shape, and texture at an arbitrary location. This enables the physicians to easily visualize tumor images to better understand tumors according to changing radiomics features. Our approach allows for the removal, manipulation, and repositioning of tumors, generating various tumor types in different scenarios. The model has been tested on tumors in four different organs (kidney, lung, breast, and brain) across CT and MRI. The synthesized images are shown to effectively aid in training for downstream tasks and their authenticity was also evaluated through expert evaluations. Our method has potential usage in treatment planning with diverse synthesized tumors.

Method: Uses maximum intensity projection images from DCE-MRI and leverages diffusion models with prompt tuning to incorporate clinical factors affecting NAC response, generating post-treatment images from pre-treatment ones.

Result: The model outperformed other generative models in image quality metrics and better reflected tumor size changes according to pathological complete response (pCR). Ablation studies confirmed the method’s design choices.

Conclusion: The proposed approach has potential to contribute to precision medicine by improving NAC response prediction for breast cancer treatment planning.

Abstract: Neoadjuvant chemotherapy (NAC) is a common therapy option before the main surgery for breast cancer. Response to NAC is monitored using follow-up dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI). Accurate prediction of NAC response helps with treatment planning. Here, we adopt maximum intensity projection images from DCE-MRI to generate post-treatment images (i.e., 3 or 12 weeks after NAC) from pre-treatment images leveraging the emerging diffusion model. We introduce prompt tuning to account for the known clinical factors affecting response to NAC. Our model performed better than other generative models in image quality metrics. Our model was better at generating images that reflected changes in tumor size according to pCR compared to other models. Ablation study confirmed the design choices of our method. Our study has the potential to help with precision medicine.

Method: TaSe framework with three components: hierarchical synthetic captioning dataset, Talk in Pieces module for disentangling text embeddings into objects/attributes/relations, and See in Whole module for hierarchical aggregation guided by novel loss functions.

Result: Experimental results on OmniLabel benchmark show 24% performance improvement, demonstrating effectiveness of linguistic compositionality approach.

Conclusion: Disentangling and hierarchically structuring linguistic representations significantly enhances VLMs’ ability to handle complex queries and improves multimodal perception for language-based object detection.

Abstract: While vision-language models (VLMs) have made significant progress in multimodal perception (e.g., open-vocabulary object detection) with simple language queries, state-of-the-art VLMs still show limited ability to perceive complex queries involving descriptive attributes and relational clauses. Our in-depth analysis shows that these limitations mainly stem from text encoders in VLMs. Such text encoders behave like bags-of-words and fail to separate target objects from their descriptive attributes and relations in complex queries, resulting in frequent false positives. To address this, we propose restructuring linguistic representations according to the hierarchical relations within sentences for language-based object detection. A key insight is the necessity of disentangling textual tokens into core components-objects, attributes, and relations (“talk in pieces”)-and subsequently aggregating them into hierarchically structured sentence-level representations (“see in whole”). Building on this principle, we introduce the TaSe framework with three main contributions: (1) a hierarchical synthetic captioning dataset spanning three tiers from category names to descriptive sentences; (2) Talk in Pieces, the three-component disentanglement module guided by a novel disentanglement loss function, transforms text embeddings into subspace compositions; and (3) See in Whole, which learns to aggregate disentangled components into hierarchically structured embeddings with the guide of proposed hierarchical objectives. The proposed TaSe framework strengthens the inductive bias of hierarchical linguistic structures, resulting in fine-grained multimodal representations for language-based object detection. Experimental results under the OmniLabel benchmark show a 24% performance improvement, demonstrating the importance of linguistic compositionality.

Method: MMRQA integrates three innovations: 1) robust metric extraction using MRQy with simulated artifacts, 2) structured transformation of metrics into question-answer pairs using Qwen, and 3) parameter-efficient fusion via LoRA adaptation of LLaVA-OneVision.

Result: The framework achieves state-of-the-art performance on MR-ART, FastMRI, and MyConnectome benchmarks, demonstrating strong zero-shot generalization capabilities as validated by comprehensive ablation studies.

Conclusion: MMRQA successfully bridges quantitative analysis with semantic reasoning, generating clinically interpretable outputs that enhance quality control in dynamic medical settings while maintaining strong generalization across protocols.

Abstract: Magnetic resonance imaging (MRI) quality assessment is crucial for clinical decision-making, yet remains challenging due to data scarcity and protocol variability. Traditional approaches face fundamental trade-offs: signal-based methods like MRIQC provide quantitative metrics but lack semantic understanding, while deep learning approaches achieve high accuracy but sacrifice interpretability. To address these limitations, we introduce the Multimodal MRI Quality Assessment (MMRQA) framework, pioneering the integration of multimodal large language models (MLLMs) with acquisition-aware signal processing. MMRQA combines three key innovations: robust metric extraction via MRQy augmented with simulated artifacts, structured transformation of metrics into question-answer pairs using Qwen, and parameter-efficient fusion through Low-Rank Adaptation (LoRA) of LLaVA-OneVision. Evaluated on MR-ART, FastMRI, and MyConnectome benchmarks, MMRQA achieves state-of-the-art performance with strong zero-shot generalization, as validated by comprehensive ablation studies. By bridging quantitative analysis with semantic reasoning, our framework generates clinically interpretable outputs that enhance quality control in dynamic medical settings.

Method: The T1C-RFlow model uses a pretrained autoencoder to create latent space representations from T1w and T2-FLAIR images, then trains a rectified flow diffusion model in this latent space. The model was trained on a large curated dataset of glioma, meningioma, and metastases patients.

Result: T1C-RFlow outperformed benchmark 3D models (pix2pix, DDPM, DiT-3D) with superior quantitative metrics and significantly faster denoising times (6.9 s/volume vs 37.7s for DDPM). It achieved high SSIM scores (0.905-0.937) and low NMSE values across all tumor types.

Conclusion: The proposed method generates synthetic T1C images that closely resemble ground truth T1C in much less time than previous diffusion models, potentially enabling practical contrast-agent-free MRI for brain tumors.

Abstract: Objective: Gadolinium-based contrast agents (GBCAs) are commonly employed with T1w MRI to enhance lesion visualization but are restricted in patients at risk of nephrogenic systemic fibrosis and variations in GBCA administration can introduce imaging inconsistencies. This study develops an efficient 3D deep-learning framework to generate T1-contrast enhanced images (T1C) from pre-contrast multiparametric MRI. Approach: We propose the 3D latent rectified flow (T1C-RFlow) model for generating high-quality T1C images. First, T1w and T2-FLAIR images are input into a pretrained autoencoder to acquire an efficient latent space representation. A rectified flow diffusion model is then trained in this latent space representation. The T1C-RFlow model was trained on a curated dataset comprised of the BraTS 2024 glioma (GLI; 1480 patients), meningioma (MEN; 1141 patients), and metastases (MET; 1475 patients) datasets. Selected patients were split into train (N=2860), validation (N=612), and test (N=614) sets. Results: Both qualitative and quantitative results demonstrate that the T1C-RFlow model outperforms benchmark 3D models (pix2pix, DDPM, Diffusion Transformers (DiT-3D)) trained in the same latent space. T1C-RFlow achieved the following metrics - GLI: NMSE 0.044 +/- 0.047, SSIM 0.935 +/- 0.025; MEN: NMSE 0.046 +/- 0.029, SSIM 0.937 +/- 0.021; MET: NMSE 0.098 +/- 0.088, SSIM 0.905 +/- 0.082. T1C-RFlow had the best tumor reconstruction performance and significantly faster denoising times (6.9 s/volume, 200 steps) than conventional DDPM models in both latent space (37.7s, 1000 steps) and patch-based in image space (4.3 hr/volume). Significance: Our proposed method generates synthetic T1C images that closely resemble ground truth T1C in much less time than previous diffusion models. Further development may permit a practical method for contrast-agent-free MRI for brain tumors.

Method: Three key components: (1) Complementary Detail Enhancement Network with depthwise separable convolutions and multi-scale fusion for boundary-sensitive features; (2) Low-rank adapters in Vision Transformer blocks for optimized medical feature representation; (3) Low-rank tensor attention mechanism in mask decoder to reduce memory usage by 75%.

Result: BALR-SAM outperforms several state-of-the-art methods including fully fine-tuned MedSAM on standard medical segmentation datasets, while updating only 1.8% (11.7M) of parameters and operating without requiring prompts.

Conclusion: The proposed BALR-SAM framework successfully addresses the domain adaptation challenges of vision foundation models in medical imaging, achieving superior segmentation performance with significantly reduced computational resources and parameter updates.

Abstract: Vision foundation models like the Segment Anything Model (SAM), pretrained on large-scale natural image datasets, often struggle in medical image segmentation due to a lack of domain-specific adaptation. In clinical practice, fine-tuning such models efficiently for medical downstream tasks with minimal resource demands, while maintaining strong performance, is challenging. To address these issues, we propose BALR-SAM, a boundary-aware low-rank adaptation framework that enhances SAM for medical imaging. It combines three tailored components: (1) a Complementary Detail Enhancement Network (CDEN) using depthwise separable convolutions and multi-scale fusion to capture boundary-sensitive features essential for accurate segmentation; (2) low-rank adapters integrated into SAM’s Vision Transformer blocks to optimize feature representation and attention for medical contexts, while simultaneously significantly reducing the parameter space; and (3) a low-rank tensor attention mechanism in the mask decoder, cutting memory usage by 75% and boosting inference speed. Experiments on standard medical segmentation datasets show that BALR-SAM, without requiring prompts, outperforms several state-of-the-art (SOTA) methods, including fully fine-tuned MedSAM, while updating just 1.8% (11.7M) of its parameters.

Method: Couples MLLM with diffusion decoder through lightweight adapter, introduces Temperature Modality Alignment for better prompt adherence and Pyramid Reflection for efficient temporal reasoning via dynamic keyframe selection.

Result: State-of-the-art performance with 2.2% improvement on VBench-Long total score vs EasyAnimateV5.1, and 1.0% and 3.3% accuracy gains on MSVD-QA and ActivityNet-QA respectively vs best prior 7B baselines.

Conclusion: UniVid successfully unifies video generation and understanding capabilities through efficient architectural design and novel techniques for modality alignment and temporal reasoning.

Abstract: Unified video modeling that combines generation and understanding capabilities is increasingly important but faces two key challenges: maintaining semantic faithfulness during flow-based generation due to text-visual token imbalance and the limitations of uniform cross-modal attention across the flow trajectory, and efficiently extending image-centric MLLMs to video without costly retraining. We present UniVid, a unified architecture that couples an MLLM with a diffusion decoder through a lightweight adapter, enabling both video understanding and generation. We introduce Temperature Modality Alignment to improve prompt adherence and Pyramid Reflection for efficient temporal reasoning via dynamic keyframe selection. Extensive experiments on standard benchmarks demonstrate state-of-the-art performance, achieving a 2.2% improvement on VBench-Long total score compared to EasyAnimateV5.1, and 1.0% and 3.3% accuracy gains on MSVD-QA and ActivityNet-QA, respectively, compared with the best prior 7B baselines.

Method: The model uses streaming 3D Gaussian reconstruction with learnable state tokens for temporal consistency, and a motion prediction module with occlusion-aware Gaussian fusion for interpolation. It employs self-supervised retargeting loss and optical flow loss for motion supervision.

Result: Extensive experiments show effectiveness on both in-domain and out-of-domain datasets, demonstrating successful 4D reconstruction and interpolation capabilities.

Conclusion: Forge4D provides an efficient solution for instant 4D human reconstruction from sparse-view videos, enabling both novel view and novel time synthesis through joint streaming reconstruction and motion prediction.

Abstract: Instant reconstruction of dynamic 3D humans from uncalibrated sparse-view videos is critical for numerous downstream applications. Existing methods, however, are either limited by the slow reconstruction speeds or incapable of generating novel-time representations. To address these challenges, we propose Forge4D, a feed-forward 4D human reconstruction and interpolation model that efficiently reconstructs temporally aligned representations from uncalibrated sparse-view videos, enabling both novel view and novel time synthesis. Our model simplifies the 4D reconstruction and interpolation problem as a joint task of streaming 3D Gaussian reconstruction and dense motion prediction. For the task of streaming 3D Gaussian reconstruction, we first reconstruct static 3D Gaussians from uncalibrated sparse-view images and then introduce learnable state tokens to enforce temporal consistency in a memory-friendly manner by interactively updating shared information across different timestamps. For novel time synthesis, we design a novel motion prediction module to predict dense motions for each 3D Gaussian between two adjacent frames, coupled with an occlusion-aware Gaussian fusion process to interpolate 3D Gaussians at arbitrary timestamps. To overcome the lack of the ground truth for dense motion supervision, we formulate dense motion prediction as a dense point matching task and introduce a self-supervised retargeting loss to optimize this module. An additional occlusion-aware optical flow loss is introduced to ensure motion consistency with plausible human movement, providing stronger regularization. Extensive experiments demonstrate the effectiveness of our model on both in-domain and out-of-domain datasets. Project page and code at: https://zhenliuzju.github.io/huyingdong/Forge4D.

Method: Proposes AVF-MAE++ with dual masking strategy across modalities, enhanced modality encoders, Iterative Audio-Visual Correlation Learning Module, and progressive semantic injection strategy with three training stages.

Result: Achieves state-of-the-art performance across 17 datasets covering three major AVFA tasks, with comprehensive ablation studies validating each component’s importance.

Conclusion: The framework successfully demonstrates the importance of scaling in AVFA and provides effective solutions for cross-modal correlation learning, with publicly released code and models.

Abstract: Affective video facial analysis (AVFA) has emerged as a key research field for building emotion-aware intelligent systems, yet this field continues to suffer from limited data availability. In recent years, the self-supervised learning (SSL) technique of Masked Autoencoders (MAE) has gained momentum, with growing adaptations in its audio-visual contexts. While scaling has proven essential for breakthroughs in general multi-modal learning domains, its specific impact on AVFA remains largely unexplored. Another core challenge in this field is capturing both intra- and inter-modal correlations through scalable audio-visual representations. To tackle these issues, we propose AVF-MAE++, a family of audio-visual MAE models designed to efficiently investigate the scaling properties in AVFA while enhancing cross-modal correlation modeling. Our framework introduces a novel dual masking strategy across audio and visual modalities and strengthens modality encoders with a more holistic design to better support scalable pre-training. Additionally, we present the Iterative Audio-Visual Correlation Learning Module, which improves correlation learning within the SSL paradigm, bridging the limitations of previous methods. To support smooth adaptation and reduce overfitting risks, we further introduce a progressive semantic injection strategy, organizing the model training into three structured stages. Extensive experiments conducted on 17 datasets, covering three major AVFA tasks, demonstrate that AVF-MAE++ achieves consistent state-of-the-art performance across multiple benchmarks. Comprehensive ablation studies further highlight the importance of each proposed component and provide deeper insights into the design choices driving these improvements. Our code and models have been publicly released at Github.

Method: Developed using over 1.3 million samples from 23 datasets across 11 imaging modalities and 6 clinical specialties. Uses hybrid training combining supervised and visual reinforcement fine-tuning to enable pixel-level visual grounding and reasoning capabilities.

Result: Achieved highest average accuracy (0.858) and F1-score (0.797) in disease diagnostics, outperforming leading models. In medical visual grounding, achieved average mIOU of 0.743 and Acc@0.5 of 0.837 across nine modalities. Strong zero-shot and few-shot performance confirmed in external validation.

Conclusion: EVLF-FM is an early multi-disease VLM model with explainability and reasoning capabilities that could advance adoption of and trust in foundation models for real-world clinical deployment.

Abstract: Despite the promise of foundation models in medical AI, current systems remain limited - they are modality-specific and lack transparent reasoning processes, hindering clinical adoption. To address this gap, we present EVLF-FM, a multimodal vision-language foundation model (VLM) designed to unify broad diagnostic capability with fine-grain explainability. The development and testing of EVLF-FM encompassed over 1.3 million total samples from 23 global datasets across eleven imaging modalities related to six clinical specialties: dermatology, hepatology, ophthalmology, pathology, pulmonology, and radiology. External validation employed 8,884 independent test samples from 10 additional datasets across five imaging modalities. Technically, EVLF-FM is developed to assist with multiple disease diagnosis and visual question answering with pixel-level visual grounding and reasoning capabilities. In internal validation for disease diagnostics, EVLF-FM achieved the highest average accuracy (0.858) and F1-score (0.797), outperforming leading generalist and specialist models. In medical visual grounding, EVLF-FM also achieved stellar performance across nine modalities with average mIOU of 0.743 and Acc@0.5 of 0.837. External validations further confirmed strong zero-shot and few-shot performance, with competitive F1-scores despite a smaller model size. Through a hybrid training strategy combining supervised and visual reinforcement fine-tuning, EVLF-FM not only achieves state-of-the-art accuracy but also exhibits step-by-step reasoning, aligning outputs with visual evidence. EVLF-FM is an early multi-disease VLM model with explainability and reasoning capabilities that could advance adoption of and trust in foundation models for real-world clinical deployment.

Method: Two inference-time techniques: 1) Action-scaled classifier-free guidance that modulates guidance strength based on action magnitude, and 2) Action-scaled noise truncation that adjusts initial noise distribution to match motion dynamics.

Result: Significant improvements in action coherence and visual quality across diverse robot environments, as demonstrated on real robot manipulation datasets.

Conclusion: Actively incorporating action parameters in diffusion-based video generation through inference-time techniques effectively enhances controllability and realism of generated robot videos.

Abstract: Generating realistic robot videos from explicit action trajectories is a critical step toward building effective world models and robotics foundation models. We introduce two training-free, inference-time techniques that fully exploit explicit action parameters in diffusion-based robot video generation. Instead of treating action vectors as passive conditioning signals, our methods actively incorporate them to guide both the classifier-free guidance process and the initialization of Gaussian latents. First, action-scaled classifier-free guidance dynamically modulates guidance strength in proportion to action magnitude, enhancing controllability over motion intensity. Second, action-scaled noise truncation adjusts the distribution of initially sampled noise to better align with the desired motion dynamics. Experiments on real robot manipulation datasets demonstrate that these techniques significantly improve action coherence and visual quality across diverse robot environments.

Method: A cycle training framework that fine-tunes diffusion models by incorporating cycle constraints to enforce consistency between generated and original images.

Result: Experiments on 3D brain MRI datasets show improved conditioning accuracy and enhanced image quality as measured by FID and SSIM metrics.

Conclusion: The cycle strategy in CDM is effective for refining diffusion-based medical image generation, with applications in data augmentation, counterfactual modeling, and disease progression modeling.

Abstract: Deep generative models have demonstrated remarkable success in medical image synthesis. However, ensuring conditioning faithfulness and high-quality synthetic images for direct or counterfactual generation remains a challenge. In this work, we introduce a cycle training framework to fine-tune diffusion models for improved conditioning adherence and enhanced synthetic image realism. Our approach, Cycle Diffusion Model (CDM), enforces consistency between generated and original images by incorporating cycle constraints, enabling more reliable direct and counterfactual generation. Experiments on a combined 3D brain MRI dataset (from ABCD, HCP aging & young adults, ADNI, and PPMI) show that our method improves conditioning accuracy and enhances image quality as measured by FID and SSIM. The results suggest that the cycle strategy used in CDM can be an effective method for refining diffusion-based medical image generation, with applications in data augmentation, counterfactual, and disease progression modeling.

Method: Uses CLIP’s shallow-layer attention to generate importance maps for adaptive bit allocation, and integrates a lightweight decoder adapter with multi-level loss function to reconstruct both low-level details and high-level semantic context.

Result: Achieves up to 35.99% bitrate saving while maintaining same performance on MLLM tasks, outperforming previous state-of-the-art neural codecs.

Conclusion: The proposed CoTAM codec effectively addresses the unique requirements of MLLMs by adaptively protecting multi-level features, demonstrating significant bandwidth efficiency gains without compromising task performance.

Abstract: The increasing deployment of powerful Multimodal Large Language Models (MLLMs), typically hosted on cloud platforms, urgently requires effective compression techniques to efficiently transmit signal inputs (e.g., images, videos) from edge devices with minimal bandwidth usage. However, conventional image codecs are optimized for fidelity to serve the Human Visual System (HVS) and ill-suited for MLLMs, in which diverse downstream tasks are jointly considered. In this paper, we first systematically analyze the impact of compression artifacts on several mainstream MLLMs. We find that: Compression distortion unevenly impacts different-level image features, leading to varying effects on MLLMs’ downstream tasks depending on their feature-level reliance. Motivated by this discovery, we propose an image Codec TAilored to MLLMs (CoTAM) designed to adaptively protect multi-level features and suit different demands of downstream tasks. The encoder leverages CLIP’s shallow-layer attention to generate an importance map for bit allocation, preserving critical semantic regions. Concurrently, the decoder integrates a lightweight adapter with a multi-level loss function to ensure the faithful reconstruction both of low-level details and high-level semantic context for robust synthesis of cross-level features. Extensive experiments validate that our method achieves up to 35.99% bitrate saving while maintaining the same performance on the MLLM tasks, outperforming previous SOTA neural codecs.

Method: Created TemMed-Bench with three tasks (VQA, report generation, image-pair selection) and 17K+ knowledge corpus. Evaluated 12 LVLMs and explored multi-modal retrieval augmentation combining visual and textual modalities.

Result: Most LVLMs lack temporal reasoning ability, with many performing at random-guessing level. GPT o3, o4-mini and Claude 3.5 Sonnet performed best but still below desired level. Multi-modal retrieval improved VQA performance by 2.59% on average.

Conclusion: LVLMs have significant limitations in temporal medical image reasoning. Multi-modal retrieval augmentation shows promise for addressing this challenge and should be further explored.

Abstract: Existing medical reasoning benchmarks for vision-language models primarily focus on analyzing a patient’s condition based on an image from a single visit. However, this setting deviates significantly from real-world clinical practice, where doctors typically refer to a patient’s historical conditions to provide a comprehensive assessment by tracking their changes over time. In this paper, we introduce TemMed-Bench, the first benchmark designed for analyzing changes in patients’ conditions between different clinical visits, which challenges large vision-language models (LVLMs) to reason over temporal medical images. TemMed-Bench consists of a test set comprising three tasks - visual question-answering (VQA), report generation, and image-pair selection - and a supplementary knowledge corpus of over 17,000 instances. With TemMed-Bench, we conduct an evaluation of six proprietary and six open-source LVLMs. Our results show that most LVLMs lack the ability to analyze patients’ condition changes over temporal medical images, and a large proportion perform only at a random-guessing level in the closed-book setting. In contrast, GPT o3, o4-mini and Claude 3.5 Sonnet demonstrate comparatively decent performance, though they have yet to reach the desired level. Furthermore, we explore augmenting the input with both retrieved visual and textual modalities in the medical domain. We also show that multi-modal retrieval augmentation yields notably higher performance gains than no retrieval and textual retrieval alone across most models on our benchmark, with the VQA task showing an average improvement of 2.59%. Overall, we compose a benchmark grounded on real-world clinical practice, and it reveals LVLMs’ limitations in temporal medical image reasoning, as well as highlighting the use of multi-modal retrieval augmentation as a potentially promising direction worth exploring to address this challenge.

Method: Proposes S²NNs with outlier-aware sub-bit weight quantization (OS-Quant) to mitigate codeword selection bias and membrane potential-based feature distillation (MPFD) for improved performance guidance.

Result: Extensive experiments on vision and non-vision tasks show S²NNs outperform existing quantized SNNs in both performance and efficiency.

Conclusion: S²NNs represent a promising approach for energy-efficient edge computing applications by achieving sub-bit weight representation with improved compression and acceleration capabilities.

Abstract: Spiking Neural Networks (SNNs) offer an energy-efficient paradigm for machine intelligence, but their continued scaling poses challenges for resource-limited deployment. Despite recent advances in binary SNNs, the storage and computational demands remain substantial for large-scale networks. To further explore the compression and acceleration potential of SNNs, we propose Sub-bit Spiking Neural Networks (S$^2$NNs) that represent weights with less than one bit. Specifically, we first establish an S$^2$NN baseline by leveraging the clustering patterns of kernels in well-trained binary SNNs. This baseline is highly efficient but suffers from \textit{outlier-induced codeword selection bias} during training. To mitigate this issue, we propose an \textit{outlier-aware sub-bit weight quantization} (OS-Quant) method, which optimizes codeword selection by identifying and adaptively scaling outliers. Furthermore, we propose a \textit{membrane potential-based feature distillation} (MPFD) method, improving the performance of highly compressed S$^2$NN via more precise guidance from a teacher model. Extensive results on vision and non-vision tasks reveal that S$^2$NN outperforms existing quantized SNNs in both performance and efficiency, making it promising for edge computing applications.

Method: Proposes a four-agent framework: Retriever (gathers external info), Reflector (assesses adequacy and triggers reformulation), Answerer (drafts responses in parallel), Improver (refines answers through iterative checks and multi-image alignment).

Result: Achieves competitive performance on the AgMMU benchmark for multi-image agricultural QA.

Conclusion: The multi-agent collaborative framework effectively addresses limitations of existing approaches by enabling context enrichment, reflective reasoning, and iterative improvement for agricultural VQA.

Abstract: Agricultural visual question answering is essential for providing farmers and researchers with accurate and timely knowledge. However, many existing approaches are predominantly developed for evidence-constrained settings such as text-only queries or single-image cases. This design prevents them from coping with real-world agricultural scenarios that often require multi-image inputs with complementary views across spatial scales, and growth stages. Moreover, limited access to up-to-date external agricultural context makes these systems struggle to adapt when evidence is incomplete. In addition, rigid pipelines often lack systematic quality control. To address this gap, we propose a self-reflective and self-improving multi-agent framework that integrates four roles, the Retriever, the Reflector, the Answerer, and the Improver. They collaborate to enable context enrichment, reflective reasoning, answer drafting, and iterative improvement. A Retriever formulates queries and gathers external information, while a Reflector assesses adequacy and triggers sequential reformulation and renewed retrieval. Two Answerers draft candidate responses in parallel to reduce bias. The Improver refines them through iterative checks while ensuring that information from multiple images is effectively aligned and utilized. Experiments on the AgMMU benchmark show that our framework achieves competitive performance on multi-image agricultural QA.

Method: Systematically mapped GSM8K text samples into visual form using an automated image-generation pipeline combined with human annotation, creating 1,319 high-quality multi-image mathematical reasoning samples.

Result: Current VLMs show substantial performance gap between text-based GSM8K (95.22% accuracy for best model) and visual GSM8K-V (46.93% accuracy), indicating visual mathematical reasoning remains a challenging task.

Conclusion: GSM8K-V provides a new benchmark for visual mathematical reasoning that reveals current model limitations and guides development of more robust and generalizable vision language models.

Abstract: Vision language models (VLMs) achieve unified modeling of images and text, enabling them to accomplish complex real-world tasks through perception, planning, and reasoning. Among these tasks, reasoning is particularly representative, with mathematical reasoning serving as a prominent example. It highlights the high-level capability of VLMs to comprehend mathematical information in images and to perform sophisticated reasoning. Recently, numerous visual mathematical reasoning benchmarks have been proposed, but they are often restricted to geometry, lack coverage of math word problems, and rarely assess reasoning across multiple images. To address these gaps, we introduce GSM8K-V, a purely visual multi-image mathematical reasoning benchmark. GSM8K-V is built by systematically mapping each sample from the widely used text-based GSM8K into visual form. Through a carefully designed automated image-generation pipeline combined with meticulous human annotation, we curate 1,319 high-quality samples. We evaluate a wide range of open-source and closed-source models on GSM8K-V. Results show that although existing VLMs have nearly saturated performance on text-based GSM8K, there remains substantial room for improvement on GSM8K-V. For example, the best-performing model, Gemini-2.5-Pro, achieves 95.22% accuracy on GSM8K but only 46.93% on GSM8K-V. We conduct a comprehensive analysis of GSM8K-V, examining the limitations of current models as well as potential directions for improvement. GSM8K-V offers a new perspective on visual mathematical reasoning and establishes a benchmark to guide the development of more robust and generalizable VLMs.

Method: The framework introduces a corruption-resilient skeletal representation and integrates skeletal structures into registration. It combines transformations from corrupted point cloud alignment and skeleton alignment, using a distribution distance loss function to enforce consistency between source and target skeletons.

Result: Experimental evaluations on diverse corrupted datasets show SRRF consistently outperforms state-of-the-art registration methods across various corruption scenarios including density distortions, noise contamination, and geometric deformations.

Conclusion: SRRF demonstrates robustness in handling corrupted point clouds by considering both local geometric features and global skeleton structure stability, making it a potential approach for 3D perception tasks in real-world scenarios.

Abstract: Point cloud registration is fundamental in 3D vision applications, including autonomous driving, robotics, and medical imaging, where precise alignment of multiple point clouds is essential for accurate environment reconstruction. However, real-world point clouds are often affected by sensor limitations, environmental noise, and preprocessing errors, making registration challenging due to density distortions, noise contamination, and geometric deformations. Existing registration methods rely on direct point matching or surface feature extraction, which are highly susceptible to these corruptions and lead to reduced alignment accuracy. To address these challenges, a skeleton-based robust registration framework is presented, which introduces a corruption-resilient skeletal representation to improve registration robustness and accuracy. The framework integrates skeletal structures into the registration process and combines the transformations obtained from both the corrupted point cloud alignment and its skeleton alignment to achieve optimal registration. In addition, a distribution distance loss function is designed to enforce the consistency between the source and target skeletons, which significantly improves the registration performance. This framework ensures that the alignment considers both the original local geometric features and the global stability of the skeleton structure, resulting in robust and accurate registration results. Experimental evaluations on diverse corrupted datasets demonstrate that SRRF consistently outperforms state-of-the-art registration methods across various corruption scenarios, including density distortions, noise contamination, and geometric deformations. The results confirm the robustness of SRRF in handling corrupted point clouds, making it a potential approach for 3D perception tasks in real-world scenarios.

Method: Proposes LamFormer with Linear Attention Mamba (LAM) in pyramid encoder, Parallel Hierarchical Feature Aggregation (PHFA) module, and Reduced Transformer (RT) for global modeling of up-sampled features.

Result: Outperforms existing segmentation methods on seven complex and diverse datasets, achieving a balance between model performance and complexity.

Conclusion: LamFormer demonstrates exceptional performance in multi-organ medical image segmentation while addressing computational efficiency and local detail extraction limitations of Transformer-based methods.

Abstract: In the field of multi-organ medical image segmentation, recent methods frequently employ Transformers to capture long-range dependencies from image features. However, these methods overlook the high computational cost of Transformers and their deficiencies in extracting local detailed information. To address high computational costs and inadequate local detail information, we reassess the design of feature extraction modules and propose a new deep-learning network called LamFormer for fine-grained segmentation tasks across multiple organs. LamFormer is a novel U-shaped network that employs Linear Attention Mamba (LAM) in an enhanced pyramid encoder to capture multi-scale long-range dependencies. We construct the Parallel Hierarchical Feature Aggregation (PHFA) module to aggregate features from different layers of the encoder, narrowing the semantic gap among features while filtering information. Finally, we design the Reduced Transformer (RT), which utilizes a distinct computational approach to globally model up-sampled features. RRT enhances the extraction of detailed local information and improves the network’s capability to capture long-range dependencies. LamFormer outperforms existing segmentation methods on seven complex and diverse datasets, demonstrating exceptional performance. Moreover, the proposed network achieves a balance between model performance and model complexity.

Method: Uses hybrid overlap confidence estimation (semantic descriptors + geometric similarity) and context-aware matching with global attention to assign soft confidence scores, guiding a differentiable weighted SVD solver for transformation computation.

Result: Outperforms state-of-the-art methods on ModelNet40, ScanObjectNN, and 7Scenes datasets in accuracy, robustness, and generalization.

Conclusion: CEGC provides an interpretable and scalable solution for partial point cloud registration under challenging conditions.

Abstract: Partial point cloud registration is essential for autonomous perception and 3D scene understanding, yet it remains challenging owing to structural ambiguity, partial visibility, and noise. We address these issues by proposing Confidence Estimation under Global Context (CEGC), a unified, confidence-driven framework for robust partial 3D registration. CEGC enables accurate alignment in complex scenes by jointly modeling overlap confidence and correspondence reliability within a shared global context. Specifically, the hybrid overlap confidence estimation module integrates semantic descriptors and geometric similarity to detect overlapping regions and suppress outliers early. The context-aware matching strategy smitigates ambiguity by employing global attention to assign soft confidence scores to correspondences, improving robustness. These scores guide a differentiable weighted singular value decomposition solver to compute precise transformations. This tightly coupled pipeline adaptively down-weights uncertain regions and emphasizes contextually reliable matches. Experiments on ModelNet40, ScanObjectNN, and 7Scenes 3D vision datasets demonstrate that CEGC outperforms state-of-the-art methods in accuracy, robustness, and generalization. Overall, CEGC offers an interpretable and scalable solution to partial point cloud registration under challenging conditions.

Method: Uses SFT with GRPO for understanding tasks, DPO for generation tasks, and proposes an industrial workflow including LLM-friendly DSL, training strategies, rendering processes, and evaluation metrics.

Result: Achieves SOTA performance on understanding tasks, outperforming larger general-purpose MLLMs and similarly-sized UI-specialized models. Matches larger MLLMs in UI generation at fraction of computational cost.

Conclusion: Integrating understanding and generation tasks improves accuracy and quality for both tasks, demonstrating the effectiveness of the unified approach.

Abstract: Although Multimodal Large Language Models (MLLMs) have been widely applied across domains, they are still facing challenges in domain-specific tasks, such as User Interface (UI) understanding accuracy and UI generation quality. In this paper, we introduce UI-UG (a unified MLLM for UI Understanding and Generation), integrating both capabilities. For understanding tasks, we employ Supervised Fine-tuning (SFT) combined with Group Relative Policy Optimization (GRPO) to enhance fine-grained understanding on the modern complex UI data. For generation tasks, we further use Direct Preference Optimization (DPO) to make our model generate human-preferred UIs. In addition, we propose an industrially effective workflow, including the design of an LLM-friendly domain-specific language (DSL), training strategies, rendering processes, and evaluation metrics. In experiments, our model achieves state-of-the-art (SOTA) performance on understanding tasks, outperforming both larger general-purpose MLLMs and similarly-sized UI-specialized models. Our model is also on par with these larger MLLMs in UI generation performance at a fraction of the computational cost. We also demonstrate that integrating understanding and generation tasks can improve accuracy and quality for both tasks.

Method: Leverages text-to-image diffusion models (Stable Diffusion) to transfer segmentations from 2D to 3D. Optimizes text tokens for each segment, fine-tunes with cross-view part correspondence loss, segments multi-view renderings, fuses labels in UV-space via voting, refines with Noise Optimization, and maps back to mesh.

Result: Outperforms existing methods by a noticeable margin in both quantitative and qualitative evaluations. Provides practical and generalizable solution for both semantic and non-semantic 3D segmentation tasks.

Conclusion: ASIA offers a practical framework for controllable 3D segmentation using minimal image annotations, effectively handling both semantic and non-semantic parts even when annotated and target objects differ significantly in geometry or structure.

Abstract: We introduce ASIA (Adaptive 3D Segmentation using few Image Annotations), a novel framework that enables segmentation of possibly non-semantic and non-text-describable “parts” in 3D. Our segmentation is controllable through a few user-annotated in-the-wild images, which are easier to collect than multi-view images, less demanding to annotate than 3D models, and more precise than potentially ambiguous text descriptions. Our method leverages the rich priors of text-to-image diffusion models, such as Stable Diffusion (SD), to transfer segmentations from image space to 3D, even when the annotated and target objects differ significantly in geometry or structure. During training, we optimize a text token for each segment and fine-tune our model with a novel cross-view part correspondence loss. At inference, we segment multi-view renderings of the 3D mesh, fuse the labels in UV-space via voting, refine them with our novel Noise Optimization technique, and finally map the UV-labels back onto the mesh. ASIA provides a practical and generalizable solution for both semantic and non-semantic 3D segmentation tasks, outperforming existing methods by a noticeable margin in both quantitative and qualitative evaluations.

Method: Proposed Uni-X architecture with modality-specific initial and final layers for low-level processing, while maintaining shared parameters in middle layers for high-level semantic fusion, creating an X-shaped design that eliminates gradient conflicts at both ends.

Result: Uni-X achieves superior training efficiency and, when scaled to 3B parameters, matches or surpasses 7B AR-based UMMs, achieving GenEval score of 82 for image generation alongside strong performance in text and vision understanding tasks.

Conclusion: Uni-X establishes a parameter-efficient and scalable foundation for future unified multimodal modeling by effectively resolving gradient conflicts through its X-shaped architecture.

Abstract: Unified Multimodal Models (UMMs) built on shared autoregressive (AR) transformers are attractive for their architectural simplicity. However, we identify a critical limitation: when trained on multimodal inputs, modality-shared transformers suffer from severe gradient conflicts between vision and text, particularly in shallow and deep layers. We trace this issue to the fundamentally different low-level statistical properties of images and text, while noting that conflicts diminish in middle layers where representations become more abstract and semantically aligned. To overcome this challenge, we propose Uni-X, a two-end-separated, middle-shared architecture. Uni-X dedicates its initial and final layers to modality-specific processing, while maintaining shared parameters in the middle layers for high-level semantic fusion. This X-shaped design not only eliminates gradient conflicts at both ends but also further alleviates residual conflicts in the shared layers. Extensive experiments validate the effectiveness of Uni-X. Under identical training conditions, Uni-X achieves superior training efficiency compared to strong baselines. When scaled to 3B parameters with larger training data, Uni-X matches or surpasses 7B AR-based UMMs, achieving a GenEval score of 82 for image generation alongside strong performance in text and vision understanding tasks. These results establish Uni-X as a parameter-efficient and scalable foundation for future unified multimodal modeling. Our code is available at https://github.com/CURRENTF/Uni-X

Method: Uses a pipeline that renders primitives sequentially, annotates images and code with multimodal models, builds stepwise updates mirroring primitive addition, and trains LLMs with supervised fine-tuning that exposes chain-of-thought reasoning.

Result: Produces more stable, editable, and higher-quality SVGs than state-of-the-art baselines while preserving vector graphics structural advantages and enabling precise hierarchical editing.

Conclusion: SVGThinker opens new directions for design, content creation, and automated graphics generation by enabling precise and hierarchical editing unlike image-based methods.

Abstract: Scalable Vector Graphics (SVG) is a code-based representation for 2D visuals. Leveraging recent advances in large language models (LLMs), we study text-to-SVG generation and address two persistent gaps: weak generalization and poor adherence to input instructions. We present SVGThinker, a reasoning-driven framework that aligns the production of SVG code with the visualization process and supports the full set of SVG primitives. Our pipeline first renders each primitive in sequence and uses a multimodal model to annotate the image and code; we then build stepwise updates that mirror the incremental addition of primitives. On this data, we train an LLM with supervised fine-tuning that exposes its chain-of-thought as intermediate reasoning, improving robustness and reducing errors and hallucinations. Experiments against state-of-the-art baselines show that SVGThinker produces more stable, editable, and higher-quality SVGs while preserving the structural advantages of vector graphics. Unlike image-based methods, our outputs enable precise and hierarchical editing, opening new directions for design, content creation, and automated graphics generation.

Method: Uses Regularized Fused Partial Gromov-Wasserstein Optimal Transport (R-FPGWOT) to jointly model visual correspondences and temporal relations, integrated with inter-sequence contrastive learning for stable training.

Result: Achieves up to 18.9% average F1-score improvements and over 30% temporal IoU gains across benchmarks (EgoProceL, ProceL, CrossTask), with more interpretable transport maps.

Conclusion: REALIGN provides a robust framework for procedure learning that effectively handles real-world video variability while preserving key-step orderings and filtering noise.

Abstract: Learning from procedural videos remains a core challenge in self-supervised representation learning, as real-world instructional data often contains background segments, repeated actions, and steps presented out of order. Such variability violates the strong monotonicity assumptions underlying many alignment methods. Prior state-of-the-art approaches, such as OPEL, leverage Kantorovich Optimal Transport (KOT) to build frame-to-frame correspondences, but rely solely on feature similarity and fail to capture the higher-order temporal structure of a task. In this paper, we introduce REALIGN, a self-supervised framework for procedure learning based on Regularized Fused Partial Gromov-Wasserstein Optimal Transport (R-FPGWOT). In contrast to KOT, our formulation jointly models visual correspondences and temporal relations under a partial alignment scheme, enabling robust handling of irrelevant frames, repeated actions, and non-monotonic step orders common in instructional videos. To stabilize training, we integrate FPGWOT distances with inter-sequence contrastive learning, avoiding the need for multiple regularizers and preventing collapse to degenerate solutions. Across egocentric (EgoProceL) and third-person (ProceL, CrossTask) benchmarks, REALIGN achieves up to 18.9% average F1-score improvements and over 30% temporal IoU gains, while producing more interpretable transport maps that preserve key-step orderings and filter out noise.

Method: Proposes a two-phase training strategy: 1) Supervised Fine-Tuning (SFT) to teach fundamental action capabilities, followed by 2) Reinforcement Learning (RL) with comprehensive reward design to optimize strategic decision-making for frame selection and reasoning.

Result: Achieves significant improvements (+10.4% average) over baselines while drastically reducing processed frames. The 7B model establishes new SOTA on LongVideo-Reason (76.1% accuracy) using only 20.6 frames on average - outperforming LongVILA-R1 (72.0%) with over 20x fewer frames (vs. 512).

Conclusion: FrameThinker demonstrates unparalleled efficiency and effectiveness in long video reasoning, enabling LVLMs to think strategically about video content through iterative interrogation and optimized frame selection.

Abstract: While Large Vision-Language Models (LVLMs) have achieved substantial progress in video understanding, their application to long video reasoning is hindered by uniform frame sampling and static textual reasoning, which are inefficient and struggle to handle visually intensive video tasks. To overcome these challenges, in this paper, we introduce the concept of thinking with long videos and propose a novel framework FrameThinker. Within this framework, LVLMs are able to iteratively interrogate video content. Developing such video reasoning capabilities in LVLMs presents notable challenges, particularly in adapting the model to new video actions (e.g. select frame), and designing reward functions to guide LVLMs to adopt the newly introduced action. To solve these challenges, we propose a two-phase training strategy, first employing Supervised Fine-Tuning (SFT) to instill fundamental action capabilities, followed by Reinforcement Learning (RL) to optimize a strategic decision-making policy.Notably, in this RL phase, we conduct an in-depth and comprehensive exploration of the reward design for each action and format reward. Extensive experiments on reasoning benchmarks like Video-Holmes, LongVideo-Reason, and long-video understanding benchmarks such as LongVideoBench, MLVU, VideoMME, and LVBench, demonstrate that FrameThinker achieves a significant average improvement of +10.4% over baselines while drastically reducing the number of processed frames. Most notably, our 7B model, FrameThinker establishes a new state-of-the-art on LongVideo-Reason, achieving 76.1% accuracy using an average of only 20.6 frames. This not only outperforms the competitive LongVILA-R1 (72.0%) but does so with over 20x fewer frames (vs. 512), demonstrating unparalleled efficiency and effectiveness.

Method: Uses video inputs without external 3D data, integrates geometric priors via Cross-Task Adapter, employs Metric Depth Model for geometric consistency, and applies two-stage distillation optimization for training.

Result: Achieves superior performance on 3D Question Answering, 3D Dense Captioning, and 3D Visual Grounding tasks across diverse benchmarks, demonstrating strong multi-task capabilities.

Conclusion: Vid-LLM provides a practical and scalable solution for 3D scene understanding by leveraging video inputs and geometric priors without requiring external 3D data, enabling real-world deployment.

Abstract: Recent developments in Multimodal Large Language Models (MLLMs) have significantly improved Vision-Language (VL) reasoning in 2D domains. However, extending these capabilities to 3D scene understanding remains a major challenge. Existing 3D Multimodal Large Language Models (3D-MLLMs) often depend on 3D data inputs, which limits scalability and generalization. To address this limitation, we propose Vid-LLM, a video-based 3D-MLLM that directly processes video inputs without requiring external 3D data, making it practical for real-world deployment. In our method, the geometric prior are directly used to improve the performance of the sceen perception. To integrate the geometric cues into the MLLM compactly, we design a Cross-Task Adapter (CTA) module to align the 3D geometric priors with the vision-language representations. To ensure geometric consistency and integrity, we introduce a Metric Depth Model that recovers real-scale geometry from the reconstruction outputs. Finally, the model is fine-tuned with a two-stage distillation optimization strategy, realizing fast convergence and stabilizes training. Extensive experiments across diverse benchmarks verified the effectiveness of our method on 3D Question Answering, 3D Dense Captioning and 3D Visual Grounding tasks, demonstrating the superior multi-task capabilities.

Method: Joint optimization of explicit triangle mesh and 2D Gaussian splats via flexible binding strategy, integrating mesh constraints and monocular normal supervision, with iterative mesh-refinement that splits high-error faces and prunes unreliable ones.

Result: Achieves state-of-the-art performance on indoor reconstruction benchmarks, reducing Chamfer-L1 by 47.3% over 2DGS baseline while maintaining competitive novel-view rendering quality.

Conclusion: OMeGa effectively addresses prior limitations in indoor texture-less reconstruction through joint optimization and mesh-guided splat learning.

Abstract: Neural rendering with Gaussian splatting has advanced novel view synthesis, and most methods reconstruct surfaces via post-hoc mesh extraction. However, existing methods suffer from two limitations: (i) inaccurate geometry in texture-less indoor regions, and (ii) the decoupling of mesh extraction from optimization, thereby missing the opportunity to leverage mesh geometry to guide splat optimization. In this paper, we present OMeGa, an end-to-end framework that jointly optimizes an explicit triangle mesh and 2D Gaussian splats via a flexible binding strategy, where spatial attributes of Gaussian Splats are expressed in the mesh frame and texture attributes are retained on splats. To further improve reconstruction accuracy, we integrate mesh constraints and monocular normal supervision into the optimization, thereby regularizing geometry learning. In addition, we propose a heuristic, iterative mesh-refinement strategy that splits high-error faces and prunes unreliable ones to further improve the detail and accuracy of the reconstructed mesh. OMeGa achieves state-of-the-art performance on challenging indoor reconstruction benchmarks, reducing Chamfer-$L_1$ by 47.3% over the 2DGS baseline while maintaining competitive novel-view rendering quality. The experimental results demonstrate that OMeGa effectively addresses prior limitations in indoor texture-less reconstruction.

Method: Developed CryoEngine for generating 904k synthetic subtomograms from 452 particle classes; designed APT-ViT with adaptive phase tokenization for geometric and semantic robustness; implemented NRCL strategy for noise-resilient representation learning.

Result: Achieved state-of-the-art performance on all three major subtomogram tasks across 24 synthetic and real datasets, with strong generalization to unseen datasets.

Conclusion: The proposed foundation model advances scalable and robust subtomogram analysis in cryo-ET by addressing key challenges through synthetic data generation, equivariance-enhancing architecture, and noise-resilient learning strategies.

Abstract: Cryo-electron tomography (cryo-ET) enables in situ visualization of macromolecular structures, where subtomogram analysis tasks such as classification, alignment, and averaging are critical for structural determination. However, effective analysis is hindered by scarce annotations, severe noise, and poor generalization. To address these challenges, we take the first step towards foundation models for cryo-ET subtomograms. First, we introduce CryoEngine, a large-scale synthetic data generator that produces over 904k subtomograms from 452 particle classes for pretraining. Second, we design an Adaptive Phase Tokenization-enhanced Vision Transformer (APT-ViT), which incorporates adaptive phase tokenization as an equivariance-enhancing module that improves robustness to both geometric and semantic variations. Third, we introduce a Noise-Resilient Contrastive Learning (NRCL) strategy to stabilize representation learning under severe noise conditions. Evaluations across 24 synthetic and real datasets demonstrate state-of-the-art (SOTA) performance on all three major subtomogram tasks and strong generalization to unseen datasets, advancing scalable and robust subtomogram analysis in cryo-ET.

Method: Uses selective state-space models (SSMs) with a similarity-aware selective scan mechanism adapted from Mamba’s linear-complexity algorithm to refine 4D correlation maps without compromising feature map resolution or receptive field.

Result: Achieves state-of-the-art performance on standard semantic correspondence benchmarks.

Conclusion: MambaMatcher effectively overcomes limitations of previous methods by efficiently modeling high-dimensional correlations using SSMs.

Abstract: Establishing semantic correspondences between images is a fundamental yet challenging task in computer vision. Traditional feature-metric methods enhance visual features but may miss complex inter-correlation relationships, while recent correlation-metric approaches are hindered by high computational costs due to processing 4D correlation maps. We introduce MambaMatcher, a novel method that overcomes these limitations by efficiently modeling high-dimensional correlations using selective state-space models (SSMs). By implementing a similarity-aware selective scan mechanism adapted from Mamba’s linear-complexity algorithm, MambaMatcher refines the 4D correlation map effectively without compromising feature map resolution or receptive field. Experiments on standard semantic correspondence benchmarks demonstrate that MambaMatcher achieves state-of-the-art performance.

Method: CLQ uses three key designs: cross-block calibration (CBC) for accurate calibration data, orthogonal-based smoothing (OBS) to quantify and smooth outlier channels using block Hadamard matrix, and cross-layer parameter searching (CLPS).

Result: CLQ successfully compresses DiTs into W4A4 format with negligible degradation in visual quality and metrics, achieving 3.98x memory saving and 3.95x speedup for both image and video generation models.

Conclusion: CLQ provides an effective quantization solution for DiTs that maintains high visual quality while significantly reducing memory usage and accelerating inference, making DiTs more practical for edge device deployment.

Abstract: Visual generation quality has been greatly promoted with the rapid advances in diffusion transformers (DiTs), which is attributed to the scaling of model size and complexity. However, these attributions also hinder the practical deployment of DiTs on edge devices, limiting their development and application. Serve as an efficient model compression technique, model post-training quantization (PTQ) can reduce the memory consumption and speed up the inference, with inevitable performance degradation. To alleviate the degradation, we propose CLQ, a cross-layer guided orthogonal-based quantization method for DiTs. To be specific, CLQ consists of three key designs. First, we observe that the calibration data used by most of the PTQ methods can not honestly represent the distribution of the activations. Therefore, we propose cross-block calibration (CBC) to obtain accurate calibration data, with which the quantization can be better guided. Second, we propose orthogonal-based smoothing (OBS), which quantifies the outlier score of each channel and leverages block Hadamard matrix to smooth the outliers with negligible overhead. Third, we propose cross-layer parameter searching (CLPS) to search. We evaluate CLQ with both image generation and video generation models and successfully compress the model into W4A4 with negligible degradation in visual quality and metrics. CLQ achieves 3.98x memory saving and 3.95x speedup. Our code is available at \hyperlink{https://github.com/Kai-Liu001/CLQ}{https://github.com/Kai-Liu001/CLQ}.

Method: Leverages geometric projection and tri-plane fusion to integrate features from multiple cameras onto a unified ground plane, extracting single-view features, aligning them via spatial transformation, and decoding a scene-level density map.

Result: Significantly outperforms single-view and conventional fusion comparisons, achieving 95.1% accuracy and strong robustness in dense, occluded scenarios.

Conclusion: Demonstrates practical potential for intelligent agriculture with superior performance in challenging farming conditions.

Abstract: Accurate animal counting is essential for smart farming but remains difficult in crowded scenes due to occlusions and limited camera views. To address this, we propose a tri-plane-based multi-view chicken counting model (TP-MVCC), which leverages geometric projection and tri-plane fusion to integrate features from multiple cameras onto a unified ground plane. The framework extracts single-view features, aligns them via spatial transformation, and decodes a scene-level density map for precise chicken counting. In addition, we construct the first multi-view dataset of silkie chickens under real farming conditions. Experiments show that TP-MVCC significantly outperforms single-view and conventional fusion comparisons, achieving 95.1% accuracy and strong robustness in dense, occluded scenarios, demonstrating its practical potential for intelligent agriculture.

Method: Proposes SphereAR which constrains all AR inputs and outputs to lie on a fixed-radius hypersphere using hyperspherical VAEs, removing the scale component that causes variance collapse and stabilizing AR decoding.

Result: SphereAR-H (943M) achieves FID 1.34 on ImageNet, setting new SOTA for AR models. Smaller variants SphereAR-L (479M) and SphereAR-B (208M) achieve FID 1.54 and 1.92 respectively, matching or surpassing much larger baselines like MAR-H (943M, 1.55) and VAR-d30 (2B, 1.92).

Conclusion: SphereAR is the first pure next-token autoregressive image generator with raster order that surpasses diffusion and masked-generation models at comparable parameter scales, demonstrating the effectiveness of hyperspherical constraints for stabilizing AR image generation.

Abstract: Autoregressive (AR) models are promising for image generation, yet continuous-token AR variants often trail latent diffusion and masked-generation models. The core issue is heterogeneous variance in VAE latents, which is amplified during AR decoding, especially under classifier-free guidance (CFG), and can cause variance collapse. We propose SphereAR to address this issue. Its core design is to constrain all AR inputs and outputs – including after CFG – to lie on a fixed-radius hypersphere (constant $\ell_2$ norm), leveraging hyperspherical VAEs. Our theoretical analysis shows that hyperspherical constraint removes the scale component (the primary cause of variance collapse), thereby stabilizing AR decoding. Empirically, on ImageNet generation, SphereAR-H (943M) sets a new state of the art for AR models, achieving FID 1.34. Even at smaller scales, SphereAR-L (479M) reaches FID 1.54 and SphereAR-B (208M) reaches 1.92, matching or surpassing much larger baselines such as MAR-H (943M, 1.55) and VAR-d30 (2B, 1.92). To our knowledge, this is the first time a pure next-token AR image generator with raster order surpasses diffusion and masked-generation models at comparable parameter scales.

Method: Two-stage framework: 1) Hypernetwork-based tokenizer encodes videos to neural parameter space, 2) Implicit diffusion transformer denoises latent INR weights. Uses Gaussian-distributed INR weights with reused bottleneck latent, reformed weight assignment, upsampling connections, and input coordinates.

Result: Superior video generation quality over previous INR-based models and comparable performance to state-of-the-art non-implicit models on UCF-101 and Kinetics-600 benchmarks. Creates smooth INR weight space for seamless interpolations.

Conclusion: NeRV-Diffusion successfully demonstrates that generating videos via neural network weights enables efficient high-quality synthesis with smooth interpolation capabilities, bridging the gap between INR-based and traditional video generation approaches.

Abstract: We present NeRV-Diffusion, an implicit latent video diffusion model that synthesizes videos via generating neural network weights. The generated weights can be rearranged as the parameters of a convolutional neural network, which forms an implicit neural representation (INR), and decodes into videos with frame indices as the input. Our framework consists of two stages: 1) A hypernetworkbased tokenizer that encodes raw videos from pixel space to neural parameter space, where the bottleneck latent serves as INR weights to decode. 2) An implicit diffusion transformer that denoises on the latent INR weights. In contrast to traditional video tokenizers that encode videos into frame-wise feature maps, NeRV-Diffusion compresses and generates a video holistically as a unified neural network. This enables efficient and high-quality video synthesis via obviating temporal cross-frame attentions in the denoiser and decoding video latent with dedicated decoders. To achieve Gaussian-distributed INR weights with high expressiveness, we reuse the bottleneck latent across all NeRV layers, as well as reform its weight assignment, upsampling connection and input coordinates. We also introduce SNR-adaptive loss weighting and scheduled sampling for effective training of the implicit diffusion model. NeRV-Diffusion reaches superior video generation quality over previous INR-based models and comparable performance to most recent state-of-the-art non-implicit models on real-world video benchmarks including UCF-101 and Kinetics-600. It also brings a smooth INR weight space that facilitates seamless interpolations between frames or videos.

Method: Proposes a zero-shot inference-time optimization method that updates text embeddings of pretrained diffusion models during sampling to guide motion generation toward desired Laban Effort and Shape components without additional motion data.

Result: The approach successfully manipulates motion attributes according to target Laban tags, yielding diverse expressive motion qualities while preserving motion identity.

Conclusion: The method enables interpretable and expressive control of human motion generation by integrating Laban movement analysis quantification methods into text-guided motion generation models.

Abstract: Diverse human motion generation is an increasingly important task, having various applications in computer vision, human-computer interaction and animation. While text-to-motion synthesis using diffusion models has shown success in generating high-quality motions, achieving fine-grained expressive motion control remains a significant challenge. This is due to the lack of motion style diversity in datasets and the difficulty of expressing quantitative characteristics in natural language. Laban movement analysis has been widely used by dance experts to express the details of motion including motion quality as consistent as possible. Inspired by that, this work aims for interpretable and expressive control of human motion generation by seamlessly integrating the quantification methods of Laban Effort and Shape components into the text-guided motion generation models. Our proposed zero-shot, inference-time optimization method guides the motion generation model to have desired Laban Effort and Shape components without any additional motion data by updating the text embedding of pretrained diffusion models during the sampling step. We demonstrate that our approach yields diverse expressive motion qualities while preserving motion identity by successfully manipulating motion attributes according to target Laban tags.

Method: DRIFT uses a stochastic ensemble of lightweight, learnable filters trained to actively disrupt gradient consensus through gradient dissonance. It combines prediction consistency, Jacobian separation, logit-space separation, and adversarial robustness in training.

Result: DRIFT achieves substantial robustness gains on ImageNet across CNNs and Vision Transformers, outperforming state-of-the-art preprocessing, adversarial training, and diffusion-based defenses under various attack types.

Conclusion: Gradient divergence is established as a practical and generalizable principle for adversarial defense, with DRIFT delivering strong protection with negligible runtime and memory costs.

Abstract: Deep neural networks remain highly vulnerable to adversarial examples, and most defenses collapse once gradients can be reliably estimated. We identify \emph{gradient consensus} – the tendency of randomized transformations to yield aligned gradients – as a key driver of adversarial transferability. Attackers exploit this consensus to construct perturbations that remain effective across transformations. We introduce \textbf{DRIFT} (Divergent Response in Filtered Transformations), a stochastic ensemble of lightweight, learnable filters trained to actively disrupt gradient consensus. Unlike prior randomized defenses that rely on gradient masking, DRIFT enforces \emph{gradient dissonance} by maximizing divergence in Jacobian- and logit-space responses while preserving natural predictions. Our contributions are threefold: (i) we formalize gradient consensus and provide a theoretical analysis linking consensus to transferability; (ii) we propose a consensus-divergence training strategy combining prediction consistency, Jacobian separation, logit-space separation, and adversarial robustness; and (iii) we show that DRIFT achieves substantial robustness gains on ImageNet across CNNs and Vision Transformers, outperforming state-of-the-art preprocessing, adversarial training, and diffusion-based defenses under adaptive white-box, transfer-based, and gradient-free attacks. DRIFT delivers these improvements with negligible runtime and memory cost, establishing gradient divergence as a practical and generalizable principle for adversarial defense.

Method: Proposes Semantic Curriculum Preference Optimization (SCPO) with progressive easy-to-hard curriculum using Semantic Curriculum Preference Pairs dataset, dynamic reference model, and symmetric bidirectional objective for simultaneous text and visual preference learning.

Result: SCPO reduces hallucination rate by up to 62.9% on LLaVA models across various scales and versions, improves factuality while preserving general capabilities, and maintains stable performance on general vision-language benchmarks.

Conclusion: SCPO is the first framework to unify semantics, symmetry, and curriculum for MLLM alignment, effectively mitigating visual hallucinations while maintaining model capabilities.

Abstract: Multimodal Large Language Models (MLLMs) have significantly improved the performance of various tasks, but continue to suffer from visual hallucinations, a critical issue where generated responses contradict visual evidence. While Direct Preference Optimization(DPO) is widely used for alignment, its application to MLLMs often fails to capture fine-grained semantic differences and encourages shortcut learning. To address these challenges, we propose Semantic Curriculum Preference Optimization (SCPO), a novel framework for MLLM alignment. SCPO employs a progressive, easy-to-hard curriculum built upon our Semantic Curriculum Preference Pairs dataset, which provides fine-grained semantic contrasts sorted by difficulty. This curriculum is trained with a dynamic reference model and a novel symmetric, bidirectional objective to facilitate simultaneous learning from both textual and visual preferences. To our knowledge, SCPO is the first framework to unify semantics, symmetry, and curriculum for MLLMs alignment, effectively mitigating visual hallucinations. Extensive experiments on LLaVA models across various scales and versions validate that SCPO demonstrates superior performance compared to baseline models on multiple hallucination benchmarks, reducing the hallucination rate by up to 62.9%. Moreover, evaluations on generalized benchmarks show that SCPO improves factuality while preserving general capabilities, with its performance remaining stable across general vision-language benchmarks.

Method: Real-aware Residual Model Merging (R²M) estimates a shared Real component via low-rank factorization, decomposes specialists into Real-aligned parts and Fake residuals, denoises residuals with layerwise rank truncation, and aggregates them with per-task norm matching.

Result: R²M outperforms joint training and other merging baselines across in-distribution, cross-dataset, and unseen-dataset scenarios. It maintains composability - new forgery families only require fine-tuning one specialist and re-merging.

Conclusion: R²M provides an effective training-free solution for deepfake detection that adapts to evolving generators through composable model merging, eliminating the need for exhaustive retraining while maintaining strong performance across diverse scenarios.

Abstract: Deepfake generators evolve quickly, making exhaustive data collection and repeated retraining impractical. We argue that model merging is a natural fit for deepfake detection: unlike generic multi-task settings with disjoint labels, deepfake specialists share the same binary decision and differ in generator-specific artifacts. Empirically, we show that simple weight averaging preserves Real representations while attenuating Fake-specific cues. Building upon these findings, we propose Real-aware Residual Model Merging (R$^2$M), a training-free parameter-space merging framework. R$^2$M estimates a shared Real component via a low-rank factorization of task vectors, decomposes each specialist into a Real-aligned part and a Fake residual, denoises residuals with layerwise rank truncation, and aggregates them with per-task norm matching to prevent any single generator from dominating. A concise rationale explains why a simple head suffices: the Real component induces a common separation direction in feature space, while truncated residuals contribute only minor off-axis variations. Across in-distribution, cross-dataset, and unseen-dataset, R$^2$M outperforms joint training and other merging baselines. Importantly, R$^2$M is also composable: when a new forgery family appears, we fine-tune one specialist and re-merge, eliminating the need for retraining.

Method: Uses DINOv2 to extract visual features from images and fuses them with geometric features at patch level. Introduces mixed positional embedding to encode both image space and point cloud space positional information.

Result: Achieves significant improvements on RGBD-3DMatch and RGBD-3DLoMatch datasets: 14.2% increase in patch inlier ratio and 15.7% increase in registration recall compared to state-of-the-art methods.

Conclusion: The proposed DINOReg effectively combines visual and geometric information through patch-level fusion and mixed positional embedding, demonstrating superior performance in point cloud registration tasks.

Abstract: Point cloud registration is a fundamental task in 3D computer vision. Most existing methods rely solely on geometric information for feature extraction and matching. Recently, several studies have incorporated color information from RGB-D data into feature extraction. Although these methods achieve remarkable improvements, they have not fully exploited the abundant texture and semantic information in images, and the feature fusion is performed in an image-lossy manner, which limit their performance. In this paper, we propose DINOReg, a registration network that sufficiently utilizes both visual and geometric information to solve the point cloud registration problem. Inspired by advances in vision foundation models, we employ DINOv2 to extract informative visual features from images, and fuse visual and geometric features at the patch level. This design effectively combines the rich texture and global semantic information extracted by DINOv2 with the detailed geometric structure information captured by the geometric backbone. Additionally, a mixed positional embedding is proposed to encode positional information from both image space and point cloud space, which enhances the model’s ability to perceive spatial relationships between patches. Extensive experiments on the RGBD-3DMatch and RGBD-3DLoMatch datasets demonstrate that our method achieves significant improvements over state-of-the-art geometry-only and multi-modal registration methods, with a 14.2% increase in patch inlier ratio and a 15.7% increase in registration recall. The code is publicly available at https://github.com/ccjccjccj/DINOReg.

Method: Proposed Mask Clustering-based Annotation Engine (MCAE) inspired by spatial autocorrelation principle. It treats semantically consistent mask groups as minimal annotating units to enable simultaneous annotation of multiple instances.

Result: Built HiCity-LC dataset with ~14 billion labeled pixels, supporting city-scale land cover maps across five major Chinese cities with classification accuracies above 85%. First publicly available submeter resolution city-level land cover benchmark.

Conclusion: MCAE demonstrates scalability and practical utility for large-scale submeter resolution mapping, significantly improving annotation efficiency while preserving label quality, semantic diversity, and spatial representativeness.

Abstract: Recent advances in remote sensing technology have made submeter resolution imagery increasingly accessible, offering remarkable detail for fine-grained land cover analysis. However, its full potential remains underutilized - particularly for large-scale land cover mapping - due to the lack of sufficient, high-quality annotated datasets. Existing labels are typically derived from pre-existing products or manual annotation, which are often unreliable or prohibitively expensive, particularly given the rich visual detail and massive data volumes of submeter imagery. Inspired by the spatial autocorrelation principle, which suggests that objects of the same class tend to co-occur with similar visual features in local neighborhoods, we propose the Mask Clustering-based Annotation Engine (MCAE), which treats semantically consistent mask groups as the minimal annotating units to enable efficient, simultaneous annotation of multiple instances. It significantly improves annotation efficiency by one to two orders of magnitude, while preserving label quality, semantic diversity, and spatial representativeness. With MCAE, we build a high-quality annotated dataset of about 14 billion labeled pixels, referred to as HiCity-LC, which supports the generation of city-scale land cover maps across five major Chinese cities with classification accuracies above 85%. It is the first publicly available submeter resolution city-level land cover benchmark, highlighting the scalability and practical utility of MCAE for large-scale, submeter resolution mapping. The dataset is available at https://github.com/chenhaocs/MCAE

Method: CMT trains a model to map points along a solver trajectory from a pre-trained diffusion model directly to the solver-generated clean sample, creating a trajectory-consistent and stable initialization for flow map training.

Result: CMT achieves state-of-the-art two-step FIDs: 1.97 on CIFAR-10, 1.32 on ImageNet 64x64, and 1.84 on ImageNet 512x512, while using up to 98% less training data and GPU time compared to Consistency Models.

Conclusion: CMT provides a principled, efficient, and general framework for training flow map models that significantly reduces training time and computational requirements while improving generation quality.

Abstract: Flow map models such as Consistency Models (CM) and Mean Flow (MF) enable few-step generation by learning the long jump of the ODE solution of diffusion models, yet training remains unstable, sensitive to hyperparameters, and costly. Initializing from a pre-trained diffusion model helps, but still requires converting infinitesimal steps into a long-jump map, leaving instability unresolved. We introduce mid-training, the first concept and practical method that inserts a lightweight intermediate stage between the (diffusion) pre-training and the final flow map training (i.e., post-training) for vision generation. Concretely, Consistency Mid-Training (CMT) is a compact and principled stage that trains a model to map points along a solver trajectory from a pre-trained model, starting from a prior sample, directly to the solver-generated clean sample. It yields a trajectory-consistent and stable initialization. This initializer outperforms random and diffusion-based baselines and enables fast, robust convergence without heuristics. Initializing post-training with CMT weights further simplifies flow map learning. Empirically, CMT achieves state of the art two step FIDs: 1.97 on CIFAR-10, 1.32 on ImageNet 64x64, and 1.84 on ImageNet 512x512, while using up to 98% less training data and GPU time, compared to CMs. On ImageNet 256x256, CMT reaches 1-step FID 3.34 while cutting total training time by about 50% compared to MF from scratch (FID 3.43). This establishes CMT as a principled, efficient, and general framework for training flow map models.

Method: Extended SMIRK dataset to MoreSMIRK with multi-pedestrian crossing situations. Used space-filling curves (SFCs) to transform multi-dimensional scenario features into characteristic patterns for matching with MoreSMIRK dictionary entries.

Result: Successfully evaluated PCICF on PIE dataset (150+ annotated pedestrian crossing videos). Framework can identify and classify complex pedestrian crossings, including when groups merge or split. Computationally efficient enough for potential onboard ODD detection.

Conclusion: PCICF provides effective systematic identification and classification of VRU situations for robotaxi ODD analysis. Framework shows promise for real-world deployment and is available as open-source with complete dataset and algorithms.

Abstract: We have recently observed the commercial roll-out of robotaxis in various countries. They are deployed within an operational design domain (ODD) on specific routes and environmental conditions, and are subject to continuous monitoring to regain control in safety-critical situations. Since ODDs typically cover urban areas, robotaxis must reliably detect vulnerable road users (VRUs) such as pedestrians, bicyclists, or e-scooter riders. To better handle such varied traffic situations, end-to-end AI, which directly compute vehicle control actions from multi-modal sensor data instead of only for perception, is on the rise. High quality data is needed for systematically training and evaluating such systems within their OOD. In this work, we propose PCICF, a framework to systematically identify and classify VRU situations to support ODD’s incident analysis. We base our work on the existing synthetic dataset SMIRK, and enhance it by extending its single-pedestrian-only design into the MoreSMIRK dataset, a structured dictionary of multi-pedestrian crossing situations constructed systematically. We then use space-filling curves (SFCs) to transform multi-dimensional features of scenarios into characteristic patterns, which we match with corresponding entries in MoreSMIRK. We evaluate PCICF with the large real-world dataset PIE, which contains more than 150 manually annotated pedestrian crossing videos. We show that PCICF can successfully identify and classify complex pedestrian crossings, even when groups of pedestrians merge or split. By leveraging computationally efficient components like SFCs, PCICF has even potential to be used onboard of robotaxis for OOD detection for example. We share an open-source replication package for PCICF containing its algorithms, the complete MoreSMIRK dataset and dictionary, as well as our experiment results presented in: https://github.com/Claud1234/PCICF

Method: Leverages SemanticSAM with progressive granularity refinement for more accurate object-level masks, and employs context-aware CLIP encoding with multiple contextual views and empirical weighting for richer semantic context.

Result: Experimental evaluation on multiple 3D scene understanding tasks shows significant improvements over existing methods in 3D semantic segmentation and object retrieval from language queries across several benchmark datasets.

Conclusion: The approach effectively addresses limitations of previous methods by improving mask generation quality and semantic context integration, demonstrating superior performance in 3D scene understanding applications.

Abstract: 3D scene understanding is fundamental for embodied AI and robotics, supporting reliable perception for interaction and navigation. Recent approaches achieve zero-shot, open-vocabulary 3D semantic mapping by assigning embedding vectors to 2D class-agnostic masks generated via vision-language models (VLMs) and projecting these into 3D. However, these methods often produce fragmented masks and inaccurate semantic assignments due to the direct use of raw masks, limiting their effectiveness in complex environments. To address this, we leverage SemanticSAM with progressive granularity refinement to generate more accurate and numerous object-level masks, mitigating the over-segmentation commonly observed in mask generation models such as vanilla SAM, and improving downstream 3D semantic segmentation. To further enhance semantic context, we employ a context-aware CLIP encoding strategy that integrates multiple contextual views of each mask using empirically determined weighting, providing much richer visual context. We evaluate our approach on multiple 3D scene understanding tasks, including 3D semantic segmentation and object retrieval from language queries, across several benchmark datasets. Experimental results demonstrate significant improvements over existing methods, highlighting the effectiveness of our approach.

Method: Proposes a novel spatio-angular latent space that encodes both spatial appearance and angular viewpoint deviations into a single latent, with a strategically decomposed multi-step training process.

Result: Outperforms existing methods in consistency and latency, with competitive quality and text-image alignment, generating 32 multi-view images in approximately 5 seconds.

Conclusion: RapidMV provides an efficient and consistent solution for text-to-multi-view generation, serving as a crucial bridge for synthetic 3D asset creation with superior speed and multi-view consistency.

Abstract: Generating synthetic multi-view images from a text prompt is an essential bridge to generating synthetic 3D assets. In this work, we introduce RapidMV, a novel text-to-multi-view generative model that can produce 32 multi-view synthetic images in just around 5 seconds. In essence, we propose a novel spatio-angular latent space, encoding both the spatial appearance and angular viewpoint deviations into a single latent for improved efficiency and multi-view consistency. We achieve effective training of RapidMV by strategically decomposing our training process into multiple steps. We demonstrate that RapidMV outperforms existing methods in terms of consistency and latency, with competitive quality and text-image alignment.

Method: A fast proxy system generates precise occlusion depth maps (1000x1000 resolution in <1ms) that guides: 1) culling of anchors and Gaussians for rendering acceleration, and 2) densification towards surfaces during training to avoid inconsistencies in occluded regions.

Result: In heavily occluded scenarios like MatrixCity Streets, Proxy-GS achieves over 2.5x speedup compared to Octree-GS while delivering substantially higher rendering quality, especially for MLP-based Gaussian splatting variants.

Conclusion: Proxy-GS effectively addresses occlusion redundancy in 3D Gaussian Splatting through proxy-guided culling and training, achieving both faster rendering speeds and improved visual fidelity in complex scenes.

Abstract: 3D Gaussian Splatting (3DGS) has emerged as an efficient approach for achieving photorealistic rendering. Recent MLP-based variants further improve visual fidelity but introduce substantial decoding overhead during rendering. To alleviate computation cost, several pruning strategies and level-of-detail (LOD) techniques have been introduced, aiming to effectively reduce the number of Gaussian primitives in large-scale scenes. However, our analysis reveals that significant redundancy still remains due to the lack of occlusion awareness. In this work, we propose Proxy-GS, a novel pipeline that exploits a proxy to introduce Gaussian occlusion awareness from any view. At the core of our approach is a fast proxy system capable of producing precise occlusion depth maps at a resolution of 1000x1000 under 1ms. This proxy serves two roles: first, it guides the culling of anchors and Gaussians to accelerate rendering speed. Second, it guides the densification towards surfaces during training, avoiding inconsistencies in occluded regions, and improving the rendering quality. In heavily occluded scenarios, such as the MatrixCity Streets dataset, Proxy-GS not only equips MLP-based Gaussian splatting with stronger rendering capability but also achieves faster rendering speed. Specifically, it achieves more than 2.5x speedup over Octree-GS, and consistently delivers substantially higher rendering quality. Code will be public upon acceptance.

Method: Uses collaborative training of modality transfer and flow estimation networks with a decoupled optimization strategy, geometry-aware data synthesis pipeline, outlier-robust loss, and cross-modal consistency constraint.

Result: Achieves state-of-the-art performance among unsupervised approaches and can be integrated with various flow estimation networks, as demonstrated on a comprehensive cross-modal flow benchmark.

Conclusion: DCFlow effectively addresses cross-modal flow estimation challenges through its novel decoupled optimization and consistency constraints, outperforming previous unsupervised methods.

Abstract: This work presents DCFlow, a novel unsupervised cross-modal flow estimation framework that integrates a decoupled optimization strategy and a cross-modal consistency constraint. Unlike previous approaches that implicitly learn flow estimation solely from appearance similarity, we introduce a decoupled optimization strategy with task-specific supervision to address modality discrepancy and geometric misalignment distinctly. This is achieved by collaboratively training a modality transfer network and a flow estimation network. To enable reliable motion supervision without ground-truth flow, we propose a geometry-aware data synthesis pipeline combined with an outlier-robust loss. Additionally, we introduce a cross-modal consistency constraint to jointly optimize both networks, significantly improving flow prediction accuracy. For evaluation, we construct a comprehensive cross-modal flow benchmark by repurposing public datasets. Experimental results demonstrate that DCFlow can be integrated with various flow estimation networks and achieves state-of-the-art performance among unsupervised approaches.

Method: Proposes UI2V-Bench with four evaluation dimensions (spatial understanding, attribute binding, category understanding, reasoning) using MLLM-based evaluation methods including instance-level pipeline and feedback-based reasoning pipeline, plus human evaluation.

Result: Evaluated various open-source and closed-source I2V models using ~500 text-image pairs, showing strong alignment between MLLM-based metrics and human evaluations.

Conclusion: UI2V-Bench fills a critical gap in I2V evaluation by emphasizing semantic comprehension and reasoning ability, providing a robust framework for future research and model development.

Abstract: Generative diffusion models are developing rapidly and attracting increasing attention due to their wide range of applications. Image-to-Video (I2V) generation has become a major focus in the field of video synthesis. However, existing evaluation benchmarks primarily focus on aspects such as video quality and temporal consistency, while largely overlooking the model’s ability to understand the semantics of specific subjects in the input image or to ensure that the generated video aligns with physical laws and human commonsense. To address this gap, we propose UI2V-Bench, a novel benchmark for evaluating I2V models with a focus on semantic understanding and reasoning. It introduces four primary evaluation dimensions: spatial understanding, attribute binding, category understanding, and reasoning. To assess these dimensions, we design two evaluation methods based on Multimodal Large Language Models (MLLMs): an instance-level pipeline for fine-grained semantic understanding, and a feedback-based reasoning pipeline that enables step-by-step causal assessment for more accurate evaluation. UI2V-Bench includes approximately 500 carefully constructed text-image pairs and evaluates a range of both open source and closed-source I2V models across all defined dimensions. We further incorporate human evaluations, which show strong alignment with the proposed MLLM-based metrics. Overall, UI2V-Bench fills a critical gap in I2V evaluation by emphasizing semantic comprehension and reasoning ability, offering a robust framework and dataset to support future research and model development in the field.

Method: Created SPLICE benchmark with 3,381 human-filtered videos from COIN dataset, segmented into 11,423 event clips. Evaluated humans and state-of-the-art VLMs on rearranging clips into coherent sequences. Used human-annotated textual descriptions to test reliance on language vs visual understanding.

Result: Significant performance gap between humans and VLMs. Human-annotated text improved model accuracy but not human performance, indicating VLMs rely more on language priors. VLMs performed better on temporal/causal reasoning tasks than contextual/spatial ones, and better on everyday tasks than specialized ones.

Conclusion: VLMs still fall short of human-level visual reasoning capabilities, with persistent challenges in understanding complex event sequences. Models show stronger performance on temporal/causal reasoning and everyday tasks, but struggle with contextual/spatial reasoning and specialized domains.

Abstract: In this work, we introduce SPLICE, a human-curated benchmark derived from the COIN instructional video dataset, designed to probe event-based reasoning across multiple dimensions: temporal, causal, spatial, contextual, and general knowledge. SPLICE includes 3,381 human-filtered videos spanning 12 categories and 180 sub-categories, such as sports, engineering, and housework. These videos are segmented into a total of 11,423 event clips. We evaluate both human participants and state-of-the-art vision-language models (VLMs) on the task of rearranging these clips into coherent event sequences to assess visual reasoning capabilities. Results reveal a significant gap: VLMs struggle to match human performance. While human-annotated textual descriptions improve model accuracy, they do not affect human performance, suggesting that models rely more on language priors than on visual understanding. Even with annotations, VLMs fall short of human-level reasoning, underscoring persistent challenges in visual reasoning. A deeper analysis across sub-categories shows that VLMs perform relatively better on videos where temporal and causal reasoning are dominant, compared to those where contextual and spatial reasoning are dominant. They also perform better on everyday tasks than on specialized ones.

Method: Uses object-centric 3D representations with hybrid scene structure - key foreground objects modeled in full 3D while backgrounds and non-interacted regions synthesized in 2D. Implements cutting-edge representation learning and object-to-3D techniques for flexible viewpoint manipulation and scene animation controlled by natural language commands.

Result: Significantly outperforms existing 2D and depth-layered 2.5D methods on the WorldScore benchmark. Enables progressive unfolding of virtual worlds with increasing 3D detail as users interact, delivering dynamic, immersive, and visually coherent exploration experiences.

Conclusion: NeoWorld successfully demonstrates a novel approach to interactive 3D world generation that balances visual realism with computational efficiency through its hybrid 3D/2D rendering strategy, enabling physically plausible scene animation and natural language control.

Abstract: We introduce NeoWorld, a deep learning framework for generating interactive 3D virtual worlds from a single input image. Inspired by the on-demand worldbuilding concept in the science fiction novel Simulacron-3 (1964), our system constructs expansive environments where only the regions actively explored by the user are rendered with high visual realism through object-centric 3D representations. Unlike previous approaches that rely on global world generation or 2D hallucination, NeoWorld models key foreground objects in full 3D, while synthesizing backgrounds and non-interacted regions in 2D to ensure efficiency. This hybrid scene structure, implemented with cutting-edge representation learning and object-to-3D techniques, enables flexible viewpoint manipulation and physically plausible scene animation, allowing users to control object appearance and dynamics using natural language commands. As users interact with the environment, the virtual world progressively unfolds with increasing 3D detail, delivering a dynamic, immersive, and visually coherent exploration experience. NeoWorld significantly outperforms existing 2D and depth-layered 2.5D methods on the WorldScore benchmark.

Method: Hybrid approach alternating between discrete Volterra feature extraction and continuous ODE-driven state evolution, integrating nonlinear Volterra filtering with neural ordinary differential equations.

Result: Consistently outperforms state-of-the-art models on benchmark datasets (CIFAR10, Imagenet1K) with improved computational complexity and substantially fewer parameters than conventional deep architectures.

Conclusion: VNODE successfully demonstrates that combining discrete Volterra filtering with continuous neural ODEs creates an efficient and effective architecture for image classification tasks.

Abstract: This paper introduces Volterra Neural Ordinary Differential Equations (VNODE), a piecewise continuous Volterra Neural Network that integrates nonlinear Volterra filtering with continuous time neural ordinary differential equations for image classification. Drawing inspiration from the visual cortex, where discrete event processing is interleaved with continuous integration, VNODE alternates between discrete Volterra feature extraction and ODE driven state evolution. This hybrid formulation captures complex patterns while requiring substantially fewer parameters than conventional deep architectures. VNODE consistently outperforms state of the art models with improved computational complexity as exemplified on benchmark datasets like CIFAR10 and Imagenet1K.

Method: Proposes a dual-model ensemble with teacher-student knowledge distillation: an Encoder-Decoder model for patch-level industrial defects and an Encoder-Encoder model for semantic AD, both using shared DINOv2 encoder. Uses Noisy-OR objective and joint probability scoring.

Result: Achieved state-of-the-art performance on 8 benchmarks: 99.7% AUROC on MVTec-AD (industrial) and 97.8% on CIFAR-10 (semantic), outperforming prior general AD models and even specialist models in multi-class settings.

Conclusion: The proposed method successfully bridges the gap between industrial and semantic anomaly detection, demonstrating strong generalization across multiple domains in both single-class and multi-class settings.

Abstract: Anomaly detection (AD) plays an important role in various real-world applications. Recent advancements in AD, however, are often biased towards industrial inspection, struggle to generalize to broader tasks like semantic anomaly detection and vice versa. Although recent methods have attempted to address general anomaly detection, their performance remains sensitive to dataset-specific settings and single-class tasks. In this paper, we propose a novel dual-model ensemble approach based on knowledge distillation (KD) to bridge this gap. Our framework consists of a teacher and two student models: an Encoder-Decoder model, specialized in detecting patch-level minor defects for industrial AD and an Encoder-Encoder model, optimized for semantic AD. Both models leverage a shared pre-trained encoder (DINOv2) to extract high-quality feature representations. The dual models are jointly learned using the Noisy-OR objective, and the final anomaly score is obtained using the joint probability via local and semantic anomaly scores derived from the respective models. We evaluate our method on eight public benchmarks under both single-class and multi-class settings: MVTec-AD, MVTec-LOCO, VisA and Real-IAD for industrial inspection and CIFAR-10/100, FMNIST and View for semantic anomaly detection. The proposed method achieved state-of-the-art accuracies in both domains, in multi-class as well as single-class settings, demonstrating generalization across multiple domains of anomaly detection. Our model achieved an image-level AUROC of 99.7% on MVTec-AD and 97.8% on CIFAR-10, which is significantly better than the prior general AD models in multi-class settings and even higher than the best specialist models on individual benchmarks.

Method: Combines clustering and coreset selection to identify representative samples, plus neighbor-homogeneity consistency score for outlier removal to filter uncharacteristic samples before selection.

Result: Maintains near-optimal retrieval accuracy while reducing computational costs by 90% (database size reduced to 10%), with outlier removal further improving performance by eliminating non-discriminative samples.

Conclusion: Strategic pruning and selective representation enables efficient second-hand fashion retrieval with minimal accuracy loss, supporting scalable marketplace operations.

Abstract: The fashion industry has been identified as a major contributor to waste and emissions, leading to an increased interest in promoting the second-hand market. Machine learning methods play an important role in facilitating the creation and expansion of second-hand marketplaces by enabling the large-scale valuation of used garments. We contribute to this line of work by addressing the scalability of second-hand image retrieval from databases. By introducing a selective representation framework, we can shrink databases to 10% of their original size without sacrificing retrieval accuracy. We first explore clustering and coreset selection methods to identify representative samples that capture the key features of each garment and its internal variability. Then, we introduce an efficient outlier removal method, based on a neighbour-homogeneity consistency score measure, that filters out uncharacteristic samples prior to selection. We evaluate our approach on three public datasets: DeepFashion Attribute, DeepFashion Con2Shop, and DeepFashion2. The results demonstrate a clear performance-efficiency trade-off by strategically pruning and selecting representative vectors of images. The retrieval system maintains near-optimal accuracy, while greatly reducing computational costs by reducing the images added to the vector database. Furthermore, applying our outlier removal method to clustering techniques yields even higher retrieval performance by removing non-discriminative samples before the selection.

Method: Proposes EQUISeg with four-stage Cross-modal Transformer Block (CMTB) for efficient multimodal fusion and hierarchical selection, plus Self-guided Module (SGM) with mutual guidance mechanism to adaptively adjust modality contributions.

Result: Extensive experiments on multiple datasets demonstrate significant performance gains and effective alleviation of modality imbalance adverse effects.

Conclusion: EQUISeg successfully balances modality contributions and enhances robustness under degraded conditions in multimodal segmentation tasks.

Abstract: Multimodal semantic segmentation enhances model robustness by exploiting cross-modal complementarities. However, existing methods often suffer from imbalanced modal dependencies, where overall performance degrades significantly once a dominant modality deteriorates in real-world scenarios. Thus, modality balance has become acritical challenge for practical multimodal segmentation. To address this issue, we propose EQUISeg, a multimodal segmentation framework that balances modality contributions through equal encoding of modalities. Built upon a four-stage Cross-modal Transformer Block(CMTB), EQUISeg enables efficient multimodal fusion and hierarchical selection. Furthermore, we design a Self-guided Module(SGM) that mitigates modality imbalance by introducing a mutual guidance mechanism, enabling each modality to adaptively adjust its contribution and enhance robustness under degraded conditions. Extensive experiments on multiple datasets demonstrate that EQUISeg achieves significant performance gains and effectively alleviates the adverse effects of modality imbalance in segmentation tasks.

Method: QL-Adapter uses two modules: ILFM fuses layout priors with ViT patch tokens from CLIP image encoder to enhance spatial understanding, and CMAM injects image features into the text branch to enrich textual embeddings and improve instruction following.

Result: QL-Adapter achieves state-of-the-art performance on the QL-Edit task and significantly outperforms existing models, as demonstrated through extensive experiments on the QL-Dataset benchmark.

Conclusion: The proposed QL-Adapter framework effectively addresses the challenges of multiple object editing by combining layout fusion and cross-modal augmentation, enabling better instruction following and spatial structure understanding in complex scenes.

Abstract: Instruction driven image editing with standard CLIP text encoders often fails in complex scenes with many objects. We present QL-Adapter, a framework for multiple object editing that tackles two challenges: enforcing object counts and spatial layouts, and accommodating diverse categories. QL-Adapter consists of two core modules: the Image-Layout Fusion Module (ILFM) and the Cross-Modal Augmentation Module (CMAM). ILFM fuses layout priors with ViT patch tokens from the CLIP image encoder to strengthen spatial structure understanding. CMAM injects image features into the text branch to enrich textual embeddings and improve instruction following. We further build QL-Dataset, a benchmark that spans broad category, layout, and count variations, and define the task of quantity and layout consistent image editing (QL-Edit). Extensive experiments show that QL-Adapter achieves state of the art performance on QL-Edit and significantly outperforms existing models.

Method: Uses Linear DiT with linear attention for efficiency and constant-memory KV cache for block linear attention to enable long video generation with fixed memory cost.

Result: Achieves competitive performance with state-of-the-art models while being 16x faster in latency, with 2.4x speedup in inference (29s vs 71s for 5-second 720p video), trained in only 12 days on 64 H100 GPUs (1% of MovieGen’s cost).

Conclusion: SANA-Video enables low-cost, high-quality video generation with efficient architecture designs that make high-resolution, long-duration video synthesis practical on consumer hardware.

Abstract: We introduce SANA-Video, a small diffusion model that can efficiently generate videos up to 720x1280 resolution and minute-length duration. SANA-Video synthesizes high-resolution, high-quality and long videos with strong text-video alignment at a remarkably fast speed, deployable on RTX 5090 GPU. Two core designs ensure our efficient, effective and long video generation: (1) Linear DiT: We leverage linear attention as the core operation, which is more efficient than vanilla attention given the large number of tokens processed in video generation. (2) Constant-Memory KV cache for Block Linear Attention: we design block-wise autoregressive approach for long video generation by employing a constant-memory state, derived from the cumulative properties of linear attention. This KV cache provides the Linear DiT with global context at a fixed memory cost, eliminating the need for a traditional KV cache and enabling efficient, minute-long video generation. In addition, we explore effective data filters and model training strategies, narrowing the training cost to 12 days on 64 H100 GPUs, which is only 1% of the cost of MovieGen. Given its low cost, SANA-Video achieves competitive performance compared to modern state-of-the-art small diffusion models (e.g., Wan 2.1-1.3B and SkyReel-V2-1.3B) while being 16x faster in measured latency. Moreover, SANA-Video can be deployed on RTX 5090 GPUs with NVFP4 precision, accelerating the inference speed of generating a 5-second 720p video from 71s to 29s (2.4x speedup). In summary, SANA-Video enables low-cost, high-quality video generation.

Method: Recast both frameworks through Stochastic Optimal Control and Optimal Transport perspectives, propose a novel Diffusion Bridge architecture using latent Transformer, and implement Flow Matching with same structure for fair comparison across multiple tasks.

Result: Theoretical analysis shows Diffusion Bridge has lower cost function and more stable trajectories. Flow Matching’s interpolation coefficients become ineffective with reduced training data. Experiments across 6 tasks confirm theoretical predictions.

Conclusion: The paper systematically delineates the respective advantages and disadvantages of Diffusion Bridge and Flow Matching models, providing guidance for practitioners on when to use each approach based on theoretical and empirical evidence.

Abstract: Diffusion Bridge and Flow Matching have both demonstrated compelling empirical performance in transformation between arbitrary distributions. However, there remains confusion about which approach is generally preferable, and the substantial discrepancies in their modeling assumptions and practical implementations have hindered a unified theoretical account of their relative merits. We have, for the first time, provided a unified theoretical and experimental validation of these two models. We recast their frameworks through the lens of Stochastic Optimal Control and prove that the cost function of the Diffusion Bridge is lower, guiding the system toward more stable and natural trajectories. Simultaneously, from the perspective of Optimal Transport, interpolation coefficients $t$ and $1-t$ of Flow Matching become increasingly ineffective when the training data size is reduced. To corroborate these theoretical claims, we propose a novel, powerful architecture for Diffusion Bridge built on a latent Transformer, and implement a Flow Matching model with the same structure to enable a fair performance comparison in various experiments. Comprehensive experiments are conducted across Image Inpainting, Super-Resolution, Deblurring, Denoising, Translation, and Style Transfer tasks, systematically varying both the distributional discrepancy (different difficulty) and the training data size. Extensive empirical results align perfectly with our theoretical predictions and allow us to delineate the respective advantages and disadvantages of these two models. Our code is available at https://anonymous.4open.science/r/DBFM-3E8E/.

Method: 1) Differentiable atmospheric scattering model with transmittance dynamic estimation and scattering parameter calibration; 2) MSA-KAN network based on Kolmogorov-Arnold Theorem for learnable edge activation; 3) Weather-aware GCN that dynamically constructs spatial adjacency matrices.

Result: Achieves 12.2%-27.5% reduction in MAE metrics compared to mainstream algorithms on four public datasets in dense fog scenarios.

Conclusion: The synergistic optimization of physical mechanisms and data-driven approaches significantly improves crowd counting accuracy under complex meteorological conditions.

Abstract: Aiming at the key challenges of crowd counting in foggy environments, such as long-range target blurring, local feature degradation, and image contrast attenuation, this paper proposes a crowd-counting method with a physical a priori of atmospheric scattering, which improves crowd counting accuracy under complex meteorological conditions through the synergistic optimization of the physical mechanism and data-driven.Specifically, first, the method introduces a differentiable atmospheric scattering model and employs transmittance dynamic estimation and scattering parameter adaptive calibration techniques to accurately quantify the nonlinear attenuation laws of haze on targets with different depths of field.Secondly, the MSA-KAN was designed based on the Kolmogorov-Arnold Representation Theorem to construct a learnable edge activation function. By integrating a multi-layer progressive architecture with adaptive skip connections, it significantly enhances the model’s nonlinear representation capability in feature-degraded regions, effectively suppressing feature confusion under fog interference.Finally, we further propose a weather-aware GCN that dynamically constructs spatial adjacency matrices using deep features extracted by MSA-KAN. Experiments on four public datasets demonstrate that our method achieves a 12.2%-27.5% reduction in MAE metrics compared to mainstream algorithms in dense fog scenarios.

Method: Two-stage framework: Stage 1 uses perception-augmented fine-tuning, Stage 2 applies perception-aware reinforcement learning with visual, textual, and consistency rewards.

Result: Significantly improves reasoning accuracy and robustness across diverse multimodal tasks, with best improvements seen in smaller models.

Conclusion: VTPerception-R1 provides a scalable and auditable solution for perception-grounded multimodal reasoning, demonstrating the effectiveness of explicit perception strategies.

Abstract: Multimodal large language models (MLLMs) often struggle to ground reasoning in perceptual evidence. We present a systematic study of perception strategies-explicit, implicit, visual, and textual-across four multimodal benchmarks and two MLLMs. Our findings show that explicit perception, especially when paired with textual cues, consistently yields the best improvements, particularly for smaller models. Based on this insight, we propose VTPerception-R1, a unified two-stage framework that decouples perception from reasoning. Stage 1 introduces perception-augmented fine-tuning, and Stage 2 applies perception-aware reinforcement learning with novel visual, textual, and consistency rewards. Experiments demonstrate that VTPerception-R1 significantly improves reasoning accuracy and robustness across diverse tasks, offering a scalable and auditable solution for perception-grounded multimodal reasoning. Our code is available at: https://github.com/yizhuoDi/VTPerceprion-R1.

Method: TokenSwap injects visual triggers into training samples while swapping grammatical roles of key tokens in textual answers. It uses an adaptive token-weighted loss to emphasize learning of swapped tokens, associating visual triggers with bags-of-words behavior.

Result: TokenSwap achieves high attack success rates while maintaining superior evasiveness and stealthiness across multiple benchmarks and various LVLM architectures.

Conclusion: TokenSwap demonstrates that backdoor attacks can be made more stealthy by targeting compositional understanding rather than using fixed patterns, highlighting new security vulnerabilities in LVLMs.

Abstract: Large vision-language models (LVLMs) have achieved impressive performance across a wide range of vision-language tasks, while they remain vulnerable to backdoor attacks. Existing backdoor attacks on LVLMs aim to force the victim model to generate a predefined target pattern, which is either inserted into or replaces the original content. We find that these fixed-pattern attacks are relatively easy to detect, because the attacked LVLM tends to memorize such frequent patterns in the training dataset, thereby exhibiting overconfidence on these targets given poisoned inputs. To address these limitations, we introduce TokenSwap, a more evasive and stealthy backdoor attack that focuses on the compositional understanding capabilities of LVLMs. Instead of enforcing a fixed targeted content, TokenSwap subtly disrupts the understanding of object relationships in text. Specifically, it causes the backdoored model to generate outputs that mention the correct objects in the image but misrepresent their relationships (i.e., bags-of-words behavior). During training, TokenSwap injects a visual trigger into selected samples and simultaneously swaps the grammatical roles of key tokens in the corresponding textual answers. However, the poisoned samples exhibit only subtle differences from the original ones, making it challenging for the model to learn the backdoor behavior. To address this, TokenSwap employs an adaptive token-weighted loss that explicitly emphasizes the learning of swapped tokens, such that the visual triggers and bags-of-words behavior are associated. Extensive experiments demonstrate that TokenSwap achieves high attack success rates while maintaining superior evasiveness and stealthiness across multiple benchmarks and various LVLM architectures.

Method: Combines DINOv2 features with photorealistic training data and a noise model for point normals. Uses cross-attention to inject continuous semantic priors for learning canonicalized object coordinate systems across instances.

Result: Outperforms state-of-the-art in synthetically trained category-level object pose estimation with 31.9% relative improvement on 5°5cm metric. Achieves up to 100% success rate in grasping unseen objects from unknown categories.

Conclusion: SCOPE enables effective object pose estimation and manipulation for both known and unknown object categories through continuous semantic priors, bridging the Sim2Real gap and supporting generalization beyond training distribution.

Abstract: Object manipulation requires accurate object pose estimation. In open environments, robots encounter unknown objects, which requires semantic understanding in order to generalize both to known categories and beyond. To resolve this challenge, we present SCOPE, a diffusion-based category-level object pose estimation model that eliminates the need for discrete category labels by leveraging DINOv2 features as continuous semantic priors. By combining these DINOv2 features with photorealistic training data and a noise model for point normals, we reduce the Sim2Real gap in category-level object pose estimation. Furthermore, injecting the continuous semantic priors via cross-attention enables SCOPE to learn canonicalized object coordinate systems across object instances beyond the distribution of known categories. SCOPE outperforms the current state of the art in synthetically trained category-level object pose estimation, achieving a relative improvement of 31.9% on the 5$^\circ$5cm metric. Additional experiments on two instance-level datasets demonstrate generalization beyond known object categories, enabling grasping of unseen objects from unknown categories with a success rate of up to 100%. Code available: https://github.com/hoenigpeter/scope.

Method: Leverages structural causal modeling with two attribute regularization strategies: prompt-aligned injection for semantic control and conditioned token contrastive loss for disentangling attribute factors and reducing spurious correlations.

Result: Achieves state-of-the-art performance with 91% MAE reduction on Pendulum dataset for accurate attribute control and 87% FID reduction on ADNI for high-fidelity MRI image generation.

Conclusion: The approach enables robust, generalizable counterfactual editing with faithful attribute modification and strong identity preservation in image generation.

Abstract: We present Causal-Adapter, a modular framework that adapts frozen text-to-image diffusion backbones for counterfactual image generation. Our method enables causal interventions on target attributes, consistently propagating their effects to causal dependents without altering the core identity of the image. In contrast to prior approaches that rely on prompt engineering without explicit causal structure, Causal-Adapter leverages structural causal modeling augmented with two attribute regularization strategies: prompt-aligned injection, which aligns causal attributes with textual embeddings for precise semantic control, and a conditioned token contrastive loss to disentangle attribute factors and reduce spurious correlations. Causal-Adapter achieves state-of-the-art performance on both synthetic and real-world datasets, with up to 91% MAE reduction on Pendulum for accurate attribute control and 87% FID reduction on ADNI for high-fidelity MRI image generation. These results show that our approach enables robust, generalizable counterfactual editing with faithful attribute modification and strong identity preservation.

Method: Constructed dataset with 200+ samples including normal and craniofacial conditions, proposed dense ray matching registration for topological consistency, and developed 3D bidirectional morphable model with tissue thickness modeling.

Result: Extensive experiments confirm robustness and accuracy; model enables 3D face-skull reconstruction from single images and surgical planning prediction.

Conclusion: BFSM successfully addresses key challenges in face-skull modeling and demonstrates strong potential for medical applications including diagnostics and surgical planning.

Abstract: Building a joint face-skull morphable model holds great potential for applications such as remote diagnostics, surgical planning, medical education, and physically based facial simulation. However, realizing this vision is constrained by the scarcity of paired face-skull data, insufficient registration accuracy, and limited exploration of reconstruction and clinical applications. Moreover, individuals with craniofacial deformities are often overlooked, resulting in underrepresentation and limited inclusivity. To address these challenges, we first construct a dataset comprising over 200 samples, including both normal cases and rare craniofacial conditions. Each case contains a CT-based skull, a CT-based face, and a high-fidelity textured face scan. Secondly, we propose a novel dense ray matching registration method that ensures topological consistency across face, skull, and their tissue correspondences. Based on this, we introduce the 3D Bidirectional Face-Skull Morphable Model (BFSM), which enables shape inference between the face and skull through a shared coefficient space, while also modeling tissue thickness variation to support one-to-many facial reconstructions from the same skull, reflecting individual changes such as fat over time. Finally, we demonstrate the potential of BFSM in medical applications, including 3D face-skull reconstruction from a single image and surgical planning prediction. Extensive experiments confirm the robustness and accuracy of our method. BFSM is available at https://github.com/wang-zidu/BFSM

Method: Curated large-scale annotated dataset (16,891 images, 298,850 cells across 12 PBC classes plus RBC class) and conducted comprehensive evaluation of five YOLOv11 variants (Nano to XLarge) under two data splitting strategies (70:20:10 and 80:10:10).

Result: YOLOv11 Medium variant achieves best trade-off with mAP@0.5 of 0.934 under 8:1:1 split. Larger models provide only marginal accuracy gains at substantially higher computational cost. 8:1:1 split consistently outperforms 7:2:1 split across all models.

Conclusion: YOLOv11, particularly the Medium variant, is highly effective for automated fine-grained PBS detection. The publicly released dataset provides valuable resource for advancing blood cell detection research in hematology.

Abstract: Manual peripheral blood smear (PBS) analysis is labor intensive and subjective. While deep learning offers a promising alternative, a systematic evaluation of state of the art models such as YOLOv11 for fine grained PBS detection is still lacking. In this work, we make two key contributions. First, we curate a large scale annotated dataset for blood cell detection and classification, comprising 16,891 images across 12 peripheral blood cell (PBC) classes, along with the red blood cell class, all carefully re annotated for object detection tasks. In total, the dataset contains 298,850 annotated cells. Second, we leverage this dataset to conduct a comprehensive evaluation of five YOLOv11 variants (ranging from Nano to XLarge). These models are rigorously benchmarked under two data splitting strategies (70:20:10 and 80:10:10) and systematically assessed using multiple performance criteria, including mean Average Precision (mAP), precision, recall, F1 score, and computational efficiency. Our experiments show that the YOLOv11 Medium variant achieves the best trade off, reaching a mAP@0.5 of 0.934 under the 8:1:1 split. Larger models (Large and XLarge) provide only marginal accuracy gains at substantially higher computational cost. Moreover, the 8:1:1 split consistently outperforms the 7:2:1 split across all models. These findings highlight YOLOv11, particularly the Medium variant, as a highly effective framework for automated, fine grained PBS detection. Beyond benchmarking, our publicly released dataset (github.com/Mohamad-AbouAli/OI-PBC-Dataset) offers a valuable resource to advance research on blood cell detection and classification in hematology.

Method: Unsupervised framework combining structured optimal transport (SOT) with Attention-based Spatio-Temporal Graph Convolutional Network (ASTGCN) for contextually aware motion phase segmentation.

Result: Achieved 71.02% mean average precision (mAP) and 74.61% F1-score on test data, substantially outperforming competing unsupervised baselines. Released dataset of 211 annotated professional javelin-throw videos.

Conclusion: The proposed unsupervised framework successfully enables automatic identification of motion phase transitions in javelin-throw without manual labeling, providing a scalable solution for sports analytics.

Abstract: Precise analysis of athletic motion is central to sports analytics, particularly in disciplines where nuanced biomechanical phases directly impact performance outcomes. Traditional analytics techniques rely on manual annotation or laboratory-based instrumentation, which are time-consuming, costly, and lack scalability. Automatic extraction of relevant kinetic variables requires a robust and contextually appropriate temporal segmentation. Considering the specific case of elite javelin-throw, we present a novel unsupervised framework for such a contextually aware segmentation, which applies the structured optimal transport (SOT) concept to augment the well-known Attention-based Spatio-Temporal Graph Convolutional Network (ASTGCN). This enables the identification of motion phase transitions without requiring expensive manual labeling. Extensive experiments demonstrate that our approach outperforms state-of-the-art unsupervised methods, achieving 71.02% mean average precision (mAP) and 74.61% F1-score on test data, substantially higher than competing baselines. We also release a new dataset of 211 manually annotated professional javelin-throw videos with frame-level annotations, covering key biomechanical phases: approach steps, drive, throw, and recovery.

Method: FreeRet uses a two-stage approach: 1) derives semantically grounded embeddings directly from MLLMs for fast candidate search, bypassing lexical alignment layers; 2) exploits MLLM’s reasoning ability for precise reranking, using explicit priors and neutral choice framing to mitigate framing effects.

Result: On MMEB and MMEB-V2 benchmarks spanning 46 datasets, FreeRet substantially outperforms models trained on millions of pairs. It is model-agnostic, scales across MLLM families and sizes, preserves generative abilities, supports arbitrary modality combinations, and unifies retrieval, reranking, and generation in end-to-end RAG.

Conclusion: Pretrained MLLMs, when carefully harnessed, can serve as strong retrieval engines without training, closing a critical gap in their role as generalists.

Abstract: Multimodal large language models (MLLMs) are emerging as versatile foundations for mixed-modality retrieval. Yet, they often require heavy post-hoc training to convert them into contrastive encoders for retrieval. This work asks: Can off-the-shelf MLLMs serve as powerful retrievers without additional training? We present FreeRet, a plug-and-play framework that turns any MLLM into a two-stage retriever. FreeRet first derives semantically grounded embeddings directly from the model for fast candidate search, and then exploits its reasoning ability for precise reranking. The framework contributes three advances: bypassing lexical alignment layers to obtain semantically faithful embeddings, conditioning representation generation with explicit priors, and mitigating framing effect in reranking via neutral choice framing. On the MMEB and MMEB-V2 benchmarks spanning 46 datasets, FreeRet substantially outperforms models trained on millions of pairs. Beyond benchmarks, FreeRet is model-agnostic and scales seamlessly across MLLM families and sizes, preserves their generative abilities, supports arbitrary modality combinations, and unifies retrieval, reranking, and generation into end-to-end RAG within a single model. Our findings demonstrate that pretrained MLLMs, when carefully harnessed, can serve as strong retrieval engines without training, closing a critical gap in their role as generalists.

Method: Proposes RIFLE framework with flicker-banding prior estimator to predict banding attributes, masked loss for focused supervision, and a simulation pipeline to generate realistic training data with stochastic jitter and noise.

Result: RIFLE outperforms recent image reconstruction baselines across quantitative metrics and visual comparisons on real-world datasets, from mild to severe flicker-banding.

Conclusion: This is the first work to research simulation and removal of flicker-banding, establishing foundations for future research in dataset construction and removal model design.

Abstract: Capturing screens is now routine in our everyday lives. But the photographs of emissive displays are often influenced by the flicker-banding (FB), which is alternating bright%u2013dark stripes that arise from temporal aliasing between a camera’s rolling-shutter readout and the display’s brightness modulation. Unlike moire degradation, which has been extensively studied, the FB remains underexplored despite its frequent and severe impact on readability and perceived quality. We formulate FB removal as a dedicated restoration task and introduce Removal of Image Flicker-Banding via Latent Diffusion Enhancement, RIFLE, a diffusion-based framework designed to remove FB while preserving fine details. We propose the flicker-banding prior estimator (FPE) that predicts key banding attributes and injects it into the restoration network. Additionally, Masked Loss (ML) is proposed to concentrate supervision on banded regions without sacrificing global fidelity. To overcome data scarcity, we provide a simulation pipeline that synthesizes FB in the luminance domain with stochastic jitter in banding angle, banding spacing, and banding width. Feathered boundaries and sensor noise are also applied for a more realistic simulation. For evaluation, we collect a paired real-world FB dataset with pixel-aligned banding-free references captured via long exposure. Across quantitative metrics and visual comparisons on our real-world dataset, RIFLE consistently outperforms recent image reconstruction baselines from mild to severe flicker-banding. To the best of our knowledge, it is the first work to research the simulation and removal of FB. Our work establishes a great foundation for subsequent research in both the dataset construction and the removal model design. Our dataset and code will be released soon.

Method: Integrates lightweight slot-based conditioning into pretrained diffusion models using register tokens for background/style and slot-conditioned modules for objects. For video, uses Invariant Slot Attention (ISA) to separate object identity from pose and Transformer-based temporal aggregation.

Result: Achieves state-of-the-art results in object discovery, segmentation, compositional editing, and controllable image generation. For video, establishes new benchmarks in unsupervised video object segmentation and reconstruction, supporting advanced editing tasks without explicit supervision.

Conclusion: Establishes a general and scalable approach to object-centric generative modeling that bridges human perception and machine learning, expanding possibilities for interactive and structured generative tools across various domains.

Abstract: A central goal in AI is to represent scenes as compositions of discrete objects, enabling fine-grained, controllable image and video generation. Yet leading diffusion models treat images holistically and rely on text conditioning, creating a mismatch for object-level editing. This thesis introduces a framework that adapts powerful pretrained diffusion models for object-centric synthesis while retaining their generative capacity. We identify a core challenge: balancing global scene coherence with disentangled object control. Our method integrates lightweight, slot-based conditioning into pretrained models, preserving their visual priors while providing object-specific manipulation. For images, SlotAdapt augments diffusion models with a register token for background/style and slot-conditioned modules for objects, reducing text-conditioning bias and achieving state-of-the-art results in object discovery, segmentation, compositional editing, and controllable image generation. We further extend the framework to video. Using Invariant Slot Attention (ISA) to separate object identity from pose and a Transformer-based temporal aggregator, our approach maintains consistent object representations and dynamics across frames. This yields new benchmarks in unsupervised video object segmentation and reconstruction, and supports advanced editing tasks such as object removal, replacement, and insertion without explicit supervision. Overall, this work establishes a general and scalable approach to object-centric generative modeling for images and videos. By bridging human object-based perception and machine learning, it expands the design space for interactive, structured, and user-driven generative tools in creative, scientific, and practical domains.

Method: Created 16-class corpus (~47k images) from multiple sources, used SMOTE oversampling and undersampling, benchmarked lightweight ensembles (Random Forest, AdaBoost, soft-voting) against configurable ResNet-style CNN with strong augmentation and label smoothing.

Result: Best ensemble (SMOTE-combined) achieved 74.8% test accuracy, CNN achieved 79.19% on full test set and 81.25% on unseen inference batch. Barge class remained a failure mode despite rebalancing.

Conclusion: Deep models have advantage but rebalancing alone has limits; need additional minority-class collection, cost-sensitive objectives like focal loss, and hybrid ensemble-CNN pipelines to combine interpretability with representational power.

Abstract: Accurate vehicle type recognition underpins intelligent transportation and logistics, but severe class imbalance in public datasets suppresses performance on rare categories. We curate a 16-class corpus (~47k images) by merging Kaggle, ImageNet, and web-crawled data, and create six balanced variants via SMOTE oversampling and targeted undersampling. Lightweight ensembles, such as Random Forest, AdaBoost, and a soft-voting combiner built on MobileNet-V2 features are benchmarked against a configurable ResNet-style CNN trained with strong augmentation and label smoothing. The best ensemble (SMOTE-combined) attains 74.8% test accuracy, while the CNN achieves 79.19% on the full test set and 81.25% on an unseen inference batch, confirming the advantage of deep models. Nonetheless, the most under-represented class (Barge) remains a failure mode, highlighting the limits of rebalancing alone. Results suggest prioritizing additional minority-class collection and cost-sensitive objectives (e.g., focal loss) and exploring hybrid ensemble or CNN pipelines to combine interpretability with representational power.

Method: Proposes a lightweight text adapter with novel layered asymmetric initialization to improve the classification component in a classifier-centric adaptive framework.

Result: Achieves substantial improvements: cIoU from 0.443 to 0.493, cSm from 0.579 to 0.658, and cMAE from 0.336 to 0.239 on OVCamo benchmark compared to OVCoser baseline.

Conclusion: Targeted classification enhancement effectively advances camouflaged object segmentation performance.

Abstract: Open-vocabulary camouflaged object segmentation requires models to segment camouflaged objects of arbitrary categories unseen during training, placing extremely high demands on generalization capabilities. Through analysis of existing methods, it is observed that the classification component significantly affects overall segmentation performance. Accordingly, a classifier-centric adaptive framework is proposed to enhance segmentation performance by improving the classification component via a lightweight text adapter with a novel layered asymmetric initialization. Through the classification enhancement, the proposed method achieves substantial improvements in segmentation metrics compared to the OVCoser baseline on the OVCamo benchmark: cIoU increases from 0.443 to 0.493, cSm from 0.579 to 0.658, and cMAE reduces from 0.336 to 0.239. These results demonstrate that targeted classification enhancement provides an effective approach for advancing camouflaged object segmentation performance.

Method: Leveraged various architectures within the nnUNet framework for initial segmentation, complemented by post-processing strategies to enhance evaluation metrics.

Result: Achieved accuracy of 0.8451, Dice scores of 0.4711 (images with visible lesions) and 0.8514 (images without visible lesions), with overall Dice score of 0.5973, ranking among top-6 methods in AIMS-TBI 2025 challenge.

Conclusion: The developed automated segmentation pipeline effectively addresses the challenge of TBI lesion segmentation and is publicly available for use.

Abstract: The identification and segmentation of moderate-severe traumatic brain injury (TBI) lesions pose a significant challenge in neuroimaging. This difficulty arises from the extreme heterogeneity of these lesions, which vary in size, number, and laterality, thereby complicating downstream image processing tasks such as image registration and brain parcellation, reducing the analytical accuracy. Thus, developing methods for highly accurate segmentation of TBI lesions is essential for reliable neuroimaging analysis. This study aims to develop an effective automated segmentation pipeline to automatically detect and segment TBI lesions in T1-weighted MRI scans. We evaluate multiple approaches to achieve accurate segmentation of the TBI lesions. The core of our pipeline leverages various architectures within the nnUNet framework for initial segmentation, complemented by post-processing strategies to enhance evaluation metrics. Our final submission to the challenge achieved an accuracy of 0.8451, Dice score values of 0.4711 and 0.8514 for images with and without visible lesions, respectively, with an overall Dice score of 0.5973, ranking among the top-6 methods in the AIMS-TBI 2025 challenge. The Python implementation of our pipeline is publicly available.

Method: Combines hierarchical task taxonomy with automated data generation using structured resource pools and GPT-4o, covering fundamental capabilities and challenging categories like scientific imagery and complex instruction editing.

Result: Fine-tuning leading models on this dataset achieved significant performance gains: up to 18% improvement on editing tasks (UniWorld-V1 on ImgEdit-Bench) and 13% on generation tasks (Harmon on GenEval).

Conclusion: Systematic data construction is key to advancing multimodal AI capabilities, as demonstrated by the substantial performance improvements achieved through the OpenGPT-4o-Image dataset.

Abstract: The performance of unified multimodal models for image generation and editing is fundamentally constrained by the quality and comprehensiveness of their training data. While existing datasets have covered basic tasks like style transfer and simple object manipulation, they often lack the systematic structure and challenging scenarios required for real-world applications. To address this bottleneck, we introduce OpenGPT-4o-Image, a large-scale dataset constructed using a novel methodology that combines hierarchical task taxonomy with automated data generation. Our taxonomy not only includes fundamental capabilities such as text rendering and style control but also introduces highly practical yet challenging categories like scientific imagery for chemistry illustrations and complex instruction editing requiring simultaneous execution of multiple operations. Through an automated pipeline leveraging structured resource pools and GPT-4o, we generate 80k high-quality instruction-image pairs with controlled diversity, covering 11 major domains and 51 subtasks. Extensive experiments show that fine-tuning leading models on our dataset achieves significant performance gains across multiple benchmarks, with improvements of up to 18% on editing tasks (UniWorld-V1 on ImgEdit-Bench) and 13% on generation tasks (Harmon on GenEval). Our work demonstrates that systematic data construction is key to advancing multimodal AI capabilities.

Method: Uses lightweight physics-aware reasoning to create counterfactual prompts encoding physics-violating behaviors, and proposes Synchronized Decoupled Guidance (SDG) with synchronized directional normalization and trajectory-decoupled denoising to suppress implausible content throughout denoising.

Result: Experiments show substantial enhancement of physical fidelity while maintaining photorealism across different physical domains, with no additional training required. Ablation studies confirm the effectiveness of both components.

Conclusion: Establishes a new plug-and-play physics-aware paradigm for video generation that explicitly reasons about physical plausibility at inference time.

Abstract: Diffusion models can generate realistic videos, but existing methods rely on implicitly learning physical reasoning from large-scale text-video datasets, which is costly, difficult to scale, and still prone to producing implausible motions that violate fundamental physical laws. We introduce a training-free framework that improves physical plausibility at inference time by explicitly reasoning about implausibility and guiding the generation away from it. Specifically, we employ a lightweight physics-aware reasoning pipeline to construct counterfactual prompts that deliberately encode physics-violating behaviors. Then, we propose a novel Synchronized Decoupled Guidance (SDG) strategy, which leverages these prompts through synchronized directional normalization to counteract lagged suppression and trajectory-decoupled denoising to mitigate cumulative trajectory bias, ensuring that implausible content is suppressed immediately and consistently throughout denoising. Experiments across different physical domains show that our approach substantially enhances physical fidelity while maintaining photorealism, despite requiring no additional training. Ablation studies confirm the complementary effectiveness of both the physics-aware reasoning component and SDG. In particular, the aforementioned two designs of SDG are also individually validated to contribute critically to the suppression of implausible content and the overall gains in physical plausibility. This establishes a new and plug-and-play physics-aware paradigm for video generation.

Method: Segmentor-guided Counterfactual Fine-Tuning (Seg-CFT) preserves the simplicity of intervening on scalar-valued, structure-specific variables while producing locally coherent counterfactuals without requiring hypothetical segmentations.

Result: The method demonstrates capability of generating realistic chest radiographs and shows promising results for modeling coronary artery disease, producing effective counterfactuals with local coherence.

Conclusion: Seg-CFT provides a practical approach for structure-specific counterfactual image generation that avoids the need for tedious pixel-level annotations while maintaining intervention simplicity and producing realistic medical images.

Abstract: Counterfactual image generation is a powerful tool for augmenting training data, de-biasing datasets, and modeling disease. Current approaches rely on external classifiers or regressors to increase the effectiveness of subject-level interventions (e.g., changing the patient’s age). For structure-specific interventions (e.g., changing the area of the left lung in a chest radiograph), we show that this is insufficient, and can result in undesirable global effects across the image domain. Previous work used pixel-level label maps as guidance, requiring a user to provide hypothetical segmentations which are tedious and difficult to obtain. We propose Segmentor-guided Counterfactual Fine-Tuning (Seg-CFT), which preserves the simplicity of intervening on scalar-valued, structure-specific variables while producing locally coherent and effective counterfactuals. We demonstrate the capability of generating realistic chest radiographs, and we show promising results for modeling coronary artery disease. Code: https://github.com/biomedia-mira/seg-cft.

Method: Created IWR-Bench with 113 tasks from 100 real-world websites, featuring 1,001 actions and diverse interaction complexities. Includes user interaction videos and crawled static assets. Uses agent-as-a-judge framework with comprehensive metrics to assess functional correctness and visual fidelity.

Result: Extensive experiments on 28 LVLMs show the best model achieves only 36.35% overall score, with functional correctness (24.39%) lagging significantly behind visual fidelity (64.25%).

Conclusion: Current models have critical limitations in reasoning about temporal dynamics and synthesizing event-driven logic. IWR-Bench establishes a challenging frontier for vision-language research.

Abstract: The webpage-to-code task requires models to understand visual representations of webpages and generate corresponding code. However, existing benchmarks primarily focus on static screenshot-to-code tasks, thereby overlooking the dynamic interactions fundamental to real-world web applications. To address this limitation, this paper introduces IWR-Bench, a novel benchmark for evaluating the capabilities of Large Vision-Language Models (LVLMs) in interactive webpage reconstruction from video. IWR-Bench comprises 113 meticulously curated tasks from 100 real-world websites, with 1,001 actions and featuring diverse interaction complexities (e.g., web games), visual styles, and domains. Aligning with standard web development practices, each task includes not only user interaction videos but also all crawled static assets (e.g., images, videos). This benchmark evaluates models on two fundamental challenges: comprehensive multi-modal reasoning to infer interaction logic from video and assets, and advanced code generation to translate this logic into functional code. An agent-as-a-judge framework with a comprehensive metric system automatically assesses the functional correctness and visual fidelity of generated webpages. Extensive experiments on 28 LVLMs reveal a significant challenge: the best model achieves an overall score of only 36.35%, as functional correctness (24.39% IFS) lags significantly behind visual fidelity (64.25% VFS). These results highlight critical limitations in current models’ ability to reason about temporal dynamics and synthesize event-driven logic, establishing IWR-Bench as a challenging frontier for vision-language research. The benchmark and evaluation code will be made publicly available. Code is available at https://github.com/L-O-I/IWR-Bench.

Method: Uses two key design choices: training in compact VAE latent space for efficiency, and adopting purely transformer-based generators/discriminators. Addresses scaling issues with lightweight intermediate supervision and width-aware learning-rate adjustment.

Result: GAT can be reliably trained across various capacities (S through XL). GAT-XL/2 achieves state-of-the-art FID of 2.96 on ImageNet-256 in only 40 epochs, 6x faster than strong baselines.

Conclusion: The proposed GAT architecture demonstrates that GANs can be effectively scaled using transformer-based designs and latent-space training, achieving competitive performance with significantly improved training efficiency.

Abstract: Scalability has driven recent advances in generative modeling, yet its principles remain underexplored for adversarial learning. We investigate the scalability of Generative Adversarial Networks (GANs) through two design choices that have proven to be effective in other types of generative models: training in a compact Variational Autoencoder latent space and adopting purely transformer-based generators and discriminators. Training in latent space enables efficient computation while preserving perceptual fidelity, and this efficiency pairs naturally with plain transformers, whose performance scales with computational budget. Building on these choices, we analyze failure modes that emerge when naively scaling GANs. Specifically, we find issues as underutilization of early layers in the generator and optimization instability as the network scales. Accordingly, we provide simple and scale-friendly solutions as lightweight intermediate supervision and width-aware learning-rate adjustment. Our experiments show that GAT, a purely transformer-based and latent-space GANs, can be easily trained reliably across a wide range of capacities (S through XL). Moreover, GAT-XL/2 achieves state-of-the-art single-step, class-conditional generation performance (FID of 2.96) on ImageNet-256 in just 40 epochs, 6x fewer epochs than strong baselines.

Method: Benchmark state-of-the-art fusion networks on PoTATO dataset of polarimetric images, comparing with single-image baselines using traditional models.

Result: Polarimetric cues help recover low-contrast objects and suppress reflection-induced false positives, improving mean IoU and lowering contour error compared to RGB inputs.

Conclusion: Polarized cameras provide sharper masks but increase computational load and risk of new false positives; benchmark helps researchers decide if suitable for their applications.