Method: Used lambeq library and DisCoCat framework to construct parameterized quantum circuits for sentence pairs, trained for semantic relatedness and inference classification. Introduced Information Gain per Parameter (IGPP) metric and proposed cluster-based architecture for parameter sharing.

Result: Quantum models achieved performance comparable to classical baselines with dramatically fewer parameters, outperformed randomly initialized transformers in inference, and showed up to five orders of magnitude higher per-parameter learning efficiency than classical counterparts.

Conclusion: QNLP shows promise for low-resource, structure-sensitive settings, with quantum models demonstrating superior parameter efficiency and learning dynamics.

Abstract: Quantum natural language processing (QNLP) offers a novel approach to semantic modeling by embedding compositional structure directly into quantum circuits. This paper investigates the application of QNLP models to the task of Natural Language Inference (NLI), comparing quantum, hybrid, and classical transformer-based models under a constrained few-shot setting. Using the lambeq library and the DisCoCat framework, we construct parameterized quantum circuits for sentence pairs and train them for both semantic relatedness and inference classification. To assess efficiency, we introduce a novel information-theoretic metric, Information Gain per Parameter (IGPP), which quantifies learning dynamics independent of model size. Our results demonstrate that quantum models achieve performance comparable to classical baselines while operating with dramatically fewer parameters. The Quantum-based models outperform randomly initialized transformers in inference and achieve lower test error on relatedness tasks. Moreover, quantum models exhibit significantly higher per-parameter learning efficiency (up to five orders of magnitude more than classical counterparts), highlighting the promise of QNLP in low-resource, structure-sensitive settings. To address circuit-level isolation and promote parameter sharing, we also propose a novel cluster-based architecture that improves generalization by tying gate parameters to learned word clusters rather than individual tokens.

Method: Used two LLMs (ChatGPT and Claude) with image-only and multimodal (image + synthetic clinical notes) inputs, employing similarity-based consensus approach with 95% output similarity threshold.

Result: Consensus accuracy improved from 77.6% (image-only) to 91.3% (multimodal). Individual models: ChatGPT 62.8%→84%, Claude 76.9%→76% with multimodal inputs.

Conclusion: Multi-model fusion with output-level consensus significantly improves diagnostic accuracy and trustworthiness of AI-assisted radiology with minimal computational overhead.

Abstract: This study presents a novel multi-model fusion framework leveraging two state-of-the-art large language models (LLMs), ChatGPT and Claude, to enhance the reliability of chest X-ray interpretation on the CheXpert dataset. From the full CheXpert corpus of 224,316 chest radiographs, we randomly selected 234 radiologist-annotated studies to evaluate unimodal performance using image-only prompts. In this setting, ChatGPT and Claude achieved diagnostic accuracies of 62.8% and 76.9%, respectively. A similarity-based consensus approach, using a 95% output similarity threshold, improved accuracy to 77.6%. To assess the impact of multimodal inputs, we then generated synthetic clinical notes following the MIMIC-CXR template and evaluated a separate subset of 50 randomly selected cases paired with both images and synthetic text. On this multimodal cohort, performance improved to 84% for ChatGPT and 76% for Claude, while consensus accuracy reached 91.3%. Across both experimental conditions, agreement-based fusion consistently outperformed individual models. These findings highlight the utility of integrating complementary modalities and using output-level consensus to improve the trustworthiness and clinical utility of AI-assisted radiological diagnosis, offering a practical path to reduce diagnostic errors with minimal computational overhead.

Method: Developed CorrectBench benchmark to test intrinsic, external, and fine-tuned self-correction approaches across commonsense reasoning, mathematical reasoning, and code generation tasks.

Result: Self-correction improves accuracy especially for complex reasoning; mixing strategies yields further improvements but reduces efficiency; reasoning LLMs have limited optimization under additional self-correction; CoT baseline shows competitive accuracy and efficiency.

Conclusion: Self-correction has potential to enhance LLM reasoning but efficiency remains a challenge, advocating for research to optimize balance between reasoning capabilities and operational efficiency.

Abstract: Self-correction of large language models (LLMs) emerges as a critical component for enhancing their reasoning performance. Although various self-correction methods have been proposed, a comprehensive evaluation of these methods remains largely unexplored, and the question of whether LLMs can truly correct themselves is a matter of significant interest and concern. In this study, we introduce CorrectBench, a benchmark developed to evaluate the effectiveness of self-correction strategies, including intrinsic, external, and fine-tuned approaches, across three tasks: commonsense reasoning, mathematical reasoning, and code generation. Our findings reveal that: 1) Self-correction methods can improve accuracy, especially for complex reasoning tasks; 2) Mixing different self-correction strategies yields further improvements, though it reduces efficiency; 3) Reasoning LLMs (e.g., DeepSeek-R1) have limited optimization under additional self-correction methods and have high time costs. Interestingly, a comparatively simple chain-of-thought (CoT) baseline demonstrates competitive accuracy and efficiency. These results underscore the potential of self-correction to enhance LLM’s reasoning performance while highlighting the ongoing challenge of improving their efficiency. Consequently, we advocate for further research focused on optimizing the balance between reasoning capabilities and operational efficiency. Project Page: https://correctbench.github.io/

Method: Two-stage closed-loop lifecycle: (1) Offline Self-Distillation synthesizes interaction trajectories into abstract, reusable strategic principles; (2) Online Interaction retrieves distilled principles to guide decision-making while accumulating behavioral trajectories, with policy reinforcement for iterative updates.

Result: EvolveR achieves superior performance over strong agentic baselines on complex multi-hop question-answering benchmarks, demonstrating effective self-improvement capabilities.

Conclusion: The framework provides a comprehensive blueprint for agents that learn from both external data and the consequences of their own actions, enabling more autonomous and continuously improving systems.

Abstract: Current Large Language Model (LLM) agents show strong performance in tool use, but lack the crucial capability to systematically learn from their own experiences. While existing frameworks mainly focus on mitigating external knowledge gaps, they fail to address a more fundamental limitation: the inability to iteratively refine problem-solving strategies. In this work, we introduce EvolveR, a framework designed to enable agent to self-improve through a complete, closed-loop experience lifecycle. This lifecycle comprises two key stages: (1) Offline Self-Distillation, where the agent’s interaction trajectories are synthesized into a structured repository of abstract, reusable strategic principles; (2) Online Interaction, where the agent interacts with tasks and actively retrieves distilled principles to guide its decision-making, accumulating a diverse set of behavioral trajectories. This loop employs a policy reinforcement mechanism to iteratively update the agent based on its performance. We demonstrate the effectiveness of EvolveR on complex multi-hop question-answering benchmarks, where it achieves superior performance over strong agentic baselines. Our work presents a comprehensive blueprint for agents that learn not only from external data but also from the consequences of their own actions, paving the way for more autonomous and continuously improving systems. Code is available at https://github.com/Edaizi/EvolveR.

Method: Developed through natural language instruction tuning (not parameter retraining), integrating comprehensive knowledge base, scenario-based instruction framework, chain-of-thought simulation, expert feedback refinement, and external APIs for tongue-image classification and multimodal database retrieval.

Result: Achieved superior accuracy to general-domain and TCM-domain models in diagnostics, herb recognition, and constitution classification. Deployed as interactive application on OpenAI GPTs Store with nearly 1,000 global users by October 2025.

Conclusion: Demonstrates feasibility of developing TCM-domain LLM through natural language-based instruction tuning and multimodal integration, providing practical framework for aligning generative AI with traditional medical reasoning and scalable real-world deployment.

Abstract: Traditional Chinese Medicine (TCM), with a history spanning over two millennia, plays a role in global healthcare. However, applying large language models (LLMs) to TCM remains challenging due to its reliance on holistic reasoning, implicit logic, and multimodal diagnostic cues. Existing TCM-domain LLMs have made progress in text-based understanding but lack multimodal integration, interpretability, and clinical applicability. To address these limitations, we developed BenCao, a ChatGPT-based multimodal assistant for TCM, integrating structured knowledge bases, diagnostic data, and expert feedback refinement. BenCao was trained through natural language instruction tuning rather than parameter retraining, aligning with expert-level reasoning and ethical norms specific to TCM. The system incorporates a comprehensive knowledge base of over 1,000 classical and modern texts, a scenario-based instruction framework for diverse interactions, a chain-of-thought simulation mechanism for interpretable reasoning, and a feedback refinement process involving licensed TCM practitioners. BenCao connects to external APIs for tongue-image classification and multimodal database retrieval, enabling dynamic access to diagnostic resources. In evaluations across single-choice question benchmarks and multimodal classification tasks, BenCao achieved superior accuracy to general-domain and TCM-domain models, particularly in diagnostics, herb recognition, and constitution classification. The model was deployed as an interactive application on the OpenAI GPTs Store, accessed by nearly 1,000 users globally as of October 2025. This study demonstrates the feasibility of developing a TCM-domain LLM through natural language-based instruction tuning and multimodal integration, offering a practical framework for aligning generative AI with traditional medical reasoning and a scalable pathway for real-world deployment.

Method: Evaluated six LLMs (GPT-4o, GPT-4o-mini, DeepSeek-Chat-V3, Gemini-2.5-Flash, Claude-3.5-Haiku, Llama-4-Maverick) under five prompt types (zero-shot, few-shot, chain-of-thought, CoT-few-shot, self-reflection) across relevance classification and six Level-2 tasks using accuracy, precision, recall, and F1 metrics.

Result: CoT-few-shot yielded the most reliable precision-recall balance; zero-shot maximized recall for high-sensitivity passes; self-reflection underperformed due to over-inclusivity. GPT-4o and DeepSeek provided robust overall performance, while GPT-4o-mini performed competitively at substantially lower cost. Cost-performance analysis revealed large differences among model-prompt pairings.

Conclusion: Recommends a staged workflow using low-cost models with structured prompts for first-pass screening and escalating borderline cases to higher-capacity models. Highlights LLMs’ uneven but promising potential for literature screening automation and provides practical guidance for task-adaptive deployment.

Abstract: This study quantifies how prompting strategies interact with large language models (LLMs) to automate the screening stage of systematic literature reviews (SLRs). We evaluate six LLMs (GPT-4o, GPT-4o-mini, DeepSeek-Chat-V3, Gemini-2.5-Flash, Claude-3.5-Haiku, Llama-4-Maverick) under five prompt types (zero-shot, few-shot, chain-of-thought (CoT), CoT-few-shot, self-reflection) across relevance classification and six Level-2 tasks, using accuracy, precision, recall, and F1. Results show pronounced model-prompt interaction effects: CoT-few-shot yields the most reliable precision-recall balance; zero-shot maximizes recall for high-sensitivity passes; and self-reflection underperforms due to over-inclusivity and instability across models. GPT-4o and DeepSeek provide robust overall performance, while GPT-4o-mini performs competitively at a substantially lower dollar cost. A cost-performance analysis for relevance classification (per 1,000 abstracts) reveals large absolute differences among model-prompt pairings; GPT-4o-mini remains low-cost across prompts, and structured prompts (CoT/CoT-few-shot) on GPT-4o-mini offer attractive F1 at a small incremental cost. We recommend a staged workflow that (1) deploys low-cost models with structured prompts for first-pass screening and (2) escalates only borderline cases to higher-capacity models. These findings highlight LLMs’ uneven but promising potential to automate literature screening. By systematically analyzing prompt-model interactions, we provide a comparative benchmark and practical guidance for task-adaptive LLM deployment.

Method: Develop a flexible synthetic testbed combining statistical token streams with abstract factual source-target pairs, enabling independent control over contextual structure and diversity levels through stream composition manipulation.

Result: Higher contextual diversity delays in-distribution factual accuracy but has varying effects on out-of-distribution generalization depending on contextual structure. Some cases show similar ID/OOD trends, while others require diversity for non-trivial recall. Optimal diversity levels depend on training duration, and failures can be traced to embedding/unembedding layer bottlenecks.

Conclusion: The interplay between contextual design and diversity level significantly impacts different generalization aspects, with the synthetic framework enabling isolation of effects that would be confounded in large-scale studies, providing a controlled testbed for future research.

Abstract: Language models are pretrained on sequences that blend statistical regularities (making text fluent) with factual associations between specific tokens (knowledge of facts). While recent work suggests that the variability of their interaction, such as paraphrases of factual associations, critically determines generalization ability, we lack a systematic analysis of these impacts. This paper introduces a flexible synthetic testbed that combines a statistical stream of generic tokens with an abstract factual stream of source-target token pairs, enabling fine-grained control over their interaction. The design enables the independent control of diversity nature by manipulating stream composition (contextual structure) and the diversity level by varying which statistical streams each fact appears in. Through controlled experiments, we find that while higher contextual diversity delays in-distribution (ID) factual accuracy, its impact on out-of-distribution (OOD) factual generalization depends critically on contextual structure. In some cases, OOD performance follows the same trend as ID, but in others, diversity becomes essential for non-trivial factual recall. Even when low diversity prohibits factual recall, optimal diversity levels depend on training duration. Beyond factual recall failures, we identify structures where statistical generalization fails independently, and others where both capabilities degrade. This shows how the interplay between contextual design and diversity level impacts different generalization aspects. Further, through a series of controlled interventions on the model components, we trace the OOD failures to distinct optimization bottlenecks, highlighting the importance of the embedding and unembedding layers. Our synthetic framework allows us to isolate effects that would be confounded in large-scale studies, offering a controlled testbed for future investigations.

Method: Used multi-year Reddit dataset (2022-2025) spanning 39 subreddits and 197,618 posts, combining crowd-sourced annotations with classification models to scale analysis of trust and distrust patterns.

Result: Trust and distrust are nearly balanced over time with shifts around major model releases; technical performance and usability dominate as dimensions; personal experience is most frequent reason shaping attitudes; distinct patterns emerge across different user types.

Conclusion: Provides a methodological framework for large-scale trust analysis and insights into evolving public perceptions of GenAI, showing dynamic attitudes influenced by technical factors and personal experiences.

Abstract: The rise of generative AI (GenAI) has impacted many aspects of human life. As these systems become embedded in everyday practices, understanding public trust in them also becomes essential for responsible adoption and governance. Prior work on trust in AI has largely drawn from psychology and human-computer interaction, but there is a lack of computational, large-scale, and longitudinal approaches to measuring trust and distrust in GenAI and large language models (LLMs). This paper presents the first computational study of Trust and Distrust in GenAI, using a multi-year Reddit dataset (2022–2025) spanning 39 subreddits and 197,618 posts. Crowd-sourced annotations of a representative sample were combined with classification models to scale analysis. We find that Trust and Distrust are nearly balanced over time, with shifts around major model releases. Technical performance and usability dominate as dimensions, while personal experience is the most frequent reason shaping attitudes. Distinct patterns also emerge across trustors (e.g., experts, ethicists, general users). Our results provide a methodological framework for large-scale Trust analysis and insights into evolving public perceptions of GenAI.

Method: Extract acoustic and linguistic features from Whisper’s hidden representations and train only a lightweight classifier on intermediate and final outputs, incorporating image and text-prompt information as auxiliary cues.

Result: Achieves strong performance on GEPT picture-description dataset, outperforming existing cutting-edge baselines including multimodal approaches, with additional gains from auxiliary relevance cues.

Conclusion: Whisper intrinsically encodes both ordinal proficiency patterns and semantic aspects of speech without task-specific fine-tuning, highlighting its potential as a powerful foundation for SLA and spoken language understanding tasks.

Abstract: In this paper, we explore the untapped potential of Whisper, a well-established automatic speech recognition (ASR) foundation model, in the context of L2 spoken language assessment (SLA). Unlike prior studies that extrinsically analyze transcriptions produced by Whisper, our approach goes a step further to probe its latent capabilities by extracting acoustic and linguistic features from hidden representations. With only a lightweight classifier being trained on top of Whisper’s intermediate and final outputs, our method achieves strong performance on the GEPT picture-description dataset, outperforming existing cutting-edge baselines, including a multimodal approach. Furthermore, by incorporating image and text-prompt information as auxiliary relevance cues, we demonstrate additional performance gains. Finally, we conduct an in-depth analysis of Whisper’s embeddings, which reveals that, even without task-specific fine-tuning, the model intrinsically encodes both ordinal proficiency patterns and semantic aspects of speech, highlighting its potential as a powerful foundation for SLA and other spoken language understanding tasks.

Method: Designed new data collection pipeline to collect images covering landmarks, food, and folklore, with manual validation for cultural authenticity and multimodal coherence.

Result: CLIP achieves 21.2% Top-1 and 36.4% Top-5 accuracy on EgMM-Corpus, revealing cultural bias in vision-language models.

Conclusion: EgMM-Corpus serves as a valuable benchmark for developing culturally aware AI models and highlights existing cultural biases in current systems.

Abstract: Despite recent advances in AI, multimodal culturally diverse datasets are still limited, particularly for regions in the Middle East and Africa. In this paper, we introduce EgMM-Corpus, a multimodal dataset dedicated to Egyptian culture. By designing and running a new data collection pipeline, we collected over 3,000 images, covering 313 concepts across landmarks, food, and folklore. Each entry in the dataset is manually validated for cultural authenticity and multimodal coherence. EgMM-Corpus aims to provide a reliable resource for evaluating and training vision-language models in an Egyptian cultural context. We further evaluate the zero-shot performance of Contrastive Language-Image Pre-training CLIP on EgMM-Corpus, on which it achieves 21.2% Top-1 accuracy and 36.4% Top-5 accuracy in classification. These results underscore the existing cultural bias in large-scale vision-language models and demonstrate the importance of EgMM-Corpus as a benchmark for developing culturally aware models.

Method: Introduced SHALLOW benchmark framework with four complementary axes (lexical, phonetic, morphological, semantic) and defined targeted metrics within each category to produce interpretable profiles of model behavior.

Result: SHALLOW metrics correlate strongly with WER when recognition quality is high (low WER), but this correlation weakens substantially as WER increases. SHALLOW captures fine-grained error patterns that WER fails to distinguish under degraded and challenging conditions.

Conclusion: SHALLOW supports specific diagnosis of model weaknesses and provides feedback for model improvement beyond what aggregate error rates can offer, addressing the critical need for evaluation frameworks that can effectively identify and assess models with heightened propensity for generating hallucinated content.

Abstract: Hallucinations in automatic speech recognition (ASR) systems refer to fluent and coherent transcriptions produced by neural ASR models that are completely unrelated to the underlying acoustic input (i.e., the speech signal). While similar to conventional decoding errors in potentially compromising the usability of transcriptions for downstream applications, hallucinations can be more detrimental due to their preservation of syntactically and semantically plausible structure. This apparent coherence can mislead subsequent processing stages and introduce serious risks, particularly in critical domains such as healthcare and law. Conventional evaluation metrics are primarily centered on error-based metrics and fail to distinguish between phonetic inaccuracies and hallucinations. Consequently, there is a critical need for new evaluation frameworks that can effectively identify and assess models with a heightened propensity for generating hallucinated content. To this end, we introduce SHALLOW, the first benchmark framework that systematically categorizes and quantifies hallucination phenomena in ASR along four complementary axes: lexical, phonetic, morphological, and semantic. We define targeted metrics within each category to produce interpretable profiles of model behavior. Through evaluation across various architectures and speech domains, we have found that SHALLOW metrics correlate strongly with word error rate (WER) when recognition quality is high (i.e., low WER). Still, this correlation weakens substantially as WER increases. SHALLOW, therefore, captures fine-grained error patterns that WER fails to distinguish under degraded and challenging conditions. Our framework supports specific diagnosis of model weaknesses and provides feedback for model improvement beyond what aggregate error rates can offer.

Method: Theoretical analysis of the relationship between grammar, meaning, and string probability based on assumptions about corpus data generation, validated empirically using 280K sentence pairs in English and Chinese to test three predictions about minimal pairs and probability correlations.

Result: Empirical validation of three predictions: (1) correlation between probabilities of strings within minimal pairs, (2) correlation between models’ and humans’ deltas within minimal pairs, and (3) poor separation in probability space between unpaired grammatical and ungrammatical strings.

Conclusion: The analyses provide theoretical grounding for using probability to learn about LMs’ structural knowledge and suggest directions for future work in LM grammatical evaluation.

Abstract: What have language models (LMs) learned about grammar? This question remains hotly debated, with major ramifications for linguistic theory. However, since probability and grammaticality are distinct notions in linguistics, it is not obvious what string probabilities can reveal about an LM’s underlying grammatical knowledge. We present a theoretical analysis of the relationship between grammar, meaning, and string probability, based on simple assumptions about the generative process of corpus data. Our framework makes three predictions, which we validate empirically using 280K sentence pairs in English and Chinese: (1) correlation between the probability of strings within minimal pairs, i.e., string pairs with minimal semantic differences; (2) correlation between models’ and humans’ deltas within minimal pairs; and (3) poor separation in probability space between unpaired grammatical and ungrammatical strings. Our analyses give theoretical grounding for using probability to learn about LMs’ structural knowledge, and suggest directions for future work in LM grammatical evaluation.

Method: Two proposed methods: pluralistic decoding and model steering, designed to work in low-resource settings with only 50 annotated samples.

Result: Model steering consistently improves over zero-shot and few-shot baselines, reduces false positives in high-stakes tasks (hate speech detection, misinformation detection), and improves distributional alignment to human values in GlobalOpinionQA.

Conclusion: The work highlights the importance of diversity and demonstrates how language models can be adapted to consider nuanced perspectives, with model steering showing particular promise for pluralistic alignment.

Abstract: As language models have a greater impact on society, it is important to ensure they are aligned to a diverse range of perspectives and are able to reflect nuance in human values. However, the most popular training paradigms for modern language models often assume there is one optimal answer for every query, leading to generic responses and poor alignment. In this work, we aim to enhance pluralistic alignment of language models in a low-resource setting with two methods: pluralistic decoding and model steering. We empirically demonstrate that model steering offers consistent improvement over zero-shot and few-shot baselines with only 50 annotated samples. Our proposed methods decrease false positives in several high-stakes tasks such as hate speech detection and misinformation detection, and improves the distributional alignment to human values in GlobalOpinionQA. We hope our work highlights the importance of diversity and how language models can be adapted to consider nuanced perspectives.

Method: A hypernetwork trained end-to-end maps encoded user profiles directly to full sets of adapter parameters (e.g., LoRA), eliminating per-user training at deployment.

Result: Outperforms both prompt-based personalization and OPPU while using substantially fewer computational resources, with strong generalization to unseen users and robustness across varying conditions.

Conclusion: Profile-to-PEFT enables efficient, scalable, and adaptive LLM personalization suitable for large-scale applications with instant adaptation and privacy-preserving local deployment.

Abstract: Personalized large language models (LLMs) tailor content to individual preferences using user profiles or histories. However, existing parameter-efficient fine-tuning (PEFT) methods, such as the ``One-PEFT-Per-User’’ (OPPU) paradigm, require training a separate adapter for each user, making them computationally expensive and impractical for real-time updates. We introduce Profile-to-PEFT, a scalable framework that employs a hypernetwork, trained end-to-end, to map a user’s encoded profile directly to a full set of adapter parameters (e.g., LoRA), eliminating per-user training at deployment. This design enables instant adaptation, generalization to unseen users, and privacy-preserving local deployment. Experimental results demonstrate that our method outperforms both prompt-based personalization and OPPU while using substantially fewer computational resources at deployment. The framework exhibits strong generalization to out-of-distribution users and maintains robustness across varying user activity levels and different embedding backbones. The proposed Profile-to-PEFT framework enables efficient, scalable, and adaptive LLM personalization suitable for large-scale applications.

Method: Defined three core competencies (awareness, generalization, alignment) and empirically evaluated them on tasks requiring distinct policies, comparing SFT, DPO, and GRPO training methods.

Result: RL-trained models demonstrated greater awareness of learned behaviors and stronger generalization to novel tasks than SFT models, but showed weak alignment between reasoning traces and final outputs, particularly in GRPO-trained models.

Conclusion: RL training enhances policy awareness and generalization but compromises alignment between internal reasoning and final outputs, with GRPO showing the most pronounced misalignment.

Abstract: Recent advances in post-training techniques have endowed Large Language Models (LLMs) with enhanced capabilities for tackling complex, logic-intensive tasks through the generation of supplementary planning tokens. This development raises a fundamental question: Are these models aware of what they “learn” and “think”? To address this, we define three core competencies: (1) awareness of learned latent policies, (2) generalization of these policies across domains, and (3) alignment between internal reasoning traces and final outputs. We empirically evaluate these abilities on several tasks, each designed to require learning a distinct policy. Furthermore, we contrast the profiles of models post-trained via Supervised Fine-Tuning (SFT), Direct Policy Optimization (DPO), and Group Relative Policy Optimization (GRPO). Our findings indicate that RL-trained models not only demonstrate greater awareness of their learned behaviors and stronger generalizability to novel, structurally similar tasks than SFT models but also often exhibit weak alignment between their reasoning traces and final outputs, an effect most pronounced in GRPO-trained models.

Method: Experiment with various prompting strategies and fine-tuning approaches for LLMs, plus train classifiers to categorize anti-vaccine tweets into multi-labeled categories (efficacy, side effects, political influences) for context-aware rebuttals.

Result: Strong alignment across human judgment, LLM-based assessments, and automatic metrics. Integration of label descriptions and structured fine-tuning enhances counter-argument effectiveness.

Conclusion: LLMs offer a promising approach for mitigating vaccine misinformation at scale through optimized counter-argument generation.

Abstract: In an era where public health is increasingly influenced by information shared on social media, combatting vaccine skepticism and misinformation has become a critical societal goal. Misleading narratives around vaccination have spread widely, creating barriers to achieving high immunisation rates and undermining trust in health recommendations. While efforts to detect misinformation have made significant progress, the generation of real time counter-arguments tailored to debunk such claims remains an insufficiently explored area. In this work, we explore the capabilities of LLMs to generate sound counter-argument rebuttals to vaccine misinformation. Building on prior research in misinformation debunking, we experiment with various prompting strategies and fine-tuning approaches to optimise counter-argument generation. Additionally, we train classifiers to categorise anti-vaccine tweets into multi-labeled categories such as concerns about vaccine efficacy, side effects, and political influences allowing for more context aware rebuttals. Our evaluation, conducted through human judgment, LLM based assessments, and automatic metrics, reveals strong alignment across these methods. Our findings demonstrate that integrating label descriptions and structured fine-tuning enhances counter-argument effectiveness, offering a promising approach for mitigating vaccine misinformation at scale.

Method: The framework simulates structured debate among four specialized LLM agents, each embodying a distinct reasoning paradigm, allowing collaborative exploration of diverse cognitive approaches through iterative debate.

Result: Across six benchmarks under unified open-source setup, DiMo improves accuracy over single-model and debate baselines, with largest gains on math tasks, while providing explicit, auditable reasoning chains.

Conclusion: DiMo positions itself as a semantics-aware, Web-native multi-agent framework that models human-machine intelligence with LLM agents producing semantically typed, URL-annotated evidence chains for explanations and user-friendly interactions.

Abstract: Large Language Models (LLMs) demonstrate strong performance but often lack interpretable reasoning. This paper introduces the Multi-Agent Collaboration Framework for Diverse Thinking Modes (DiMo), which enhances both performance and interpretability by simulating a structured debate among four specialized LLM agents. Each agent embodies a distinct reasoning paradigm, allowing the framework to collaboratively explore diverse cognitive approaches. Through iterative debate, agents challenge and refine initial responses, yielding more robust conclusions and an explicit, auditable reasoning chain. Across six benchmarks and under a unified open-source setup, DiMo improves accuracy over widely used single-model and debate baselines, with the largest gains on math. We position DiMo as a semantics-aware, Web-native multi-agent framework: it models human-machine intelligence with LLM agents that produce semantically typed, URL-annotated evidence chains for explanations and user-friendly interactions. Although our experiments use standard reasoning benchmarks, the framework is designed to be instantiated over Web corpora and knowledge graphs, combining retrieval-augmented reasoning with structured justifications that downstream systems can inspect and reuse.

Method: Autoregressive Argumentative Structure Prediction (AASP) framework using pre-trained language models with constrained action sets to build structures step-by-step.

Result: Achieved state-of-the-art results on two benchmarks and strong results on third benchmark across all argument mining tasks.

Conclusion: AASP effectively captures argumentative reasoning flow through joint formulation and autoregressive structure prediction.

Abstract: Argument Mining (AM) helps in automating the extraction of complex argumentative structures such as Argument Components (ACs) like Premise, Claim etc. and Argumentative Relations (ARs) like Support, Attack etc. in an argumentative text. Due to the inherent complexity of reasoning involved with this task, modelling dependencies between ACs and ARs is challenging. Most of the recent approaches formulate this task through a generative paradigm by flattening the argumentative structures. In contrast to that, this study jointly formulates the key tasks of AM in an end-to-end fashion using Autoregressive Argumentative Structure Prediction (AASP) framework. The proposed AASP framework is based on the autoregressive structure prediction framework that has given good performance for several NLP tasks. AASP framework models the argumentative structures as constrained pre-defined sets of actions with the help of a conditional pre-trained language model. These actions build the argumentative structures step-by-step in an autoregressive manner to capture the flow of argumentative reasoning in an efficient way. Extensive experiments conducted on three standard AM benchmarks demonstrate that AASP achieves state-of-theart (SoTA) results across all AM tasks in two benchmarks and delivers strong results in one benchmark.

Method: Linear transformation applied to specific layer activations using steering vectors to guide model output, without computationally intensive techniques.

Result: Improved performance on two tasks: identifying depression-relevant Reddit posts and completing depression screening questionnaires based on user post history.

Conclusion: Steering mechanisms show untapped potential as computationally efficient tools for LLM domain adaptation in mental health applications.

Abstract: The rapid evolution of Large Language Models (LLMs) is transforming AI, opening new opportunities in sensitive and high-impact areas such as Mental Health (MH). Yet, despite these advancements, recent evidence reveals that smaller-scale models still struggle to deliver optimal performance in domain-specific applications. In this study, we present a cost-efficient yet powerful approach to improve MH assessment capabilities of an LLM, without relying on any computationally intensive techniques. Our lightweight method consists of a linear transformation applied to a specific layer’s activations, leveraging steering vectors to guide the model’s output. Remarkably, this intervention enables the model to achieve improved results across two distinct tasks: (1) identifying whether a Reddit post is useful for detecting the presence or absence of depressive symptoms (relevance prediction task), and (2) completing a standardized psychological screening questionnaire for depression based on users’ Reddit post history (questionnaire completion task). Results highlight the untapped potential of steering mechanisms as computationally efficient tools for LLMs’ MH domain adaptation.

Method: VeriMAP decomposes tasks, models subtask dependencies, and encodes planner-defined passing criteria as subtask verification functions (VFs) in both Python and natural language for automated verification.

Result: VeriMAP outperforms both single- and multi-agent baselines on diverse datasets while enhancing system robustness and interpretability.

Conclusion: Verification-aware planning enables reliable coordination and iterative refinement in multi-agent systems without relying on external labels or annotations.

Abstract: Large language model (LLM) agents are increasingly deployed to tackle complex tasks, often necessitating collaboration among multiple specialized agents. However, multi-agent collaboration introduces new challenges in planning, coordination, and verification. Execution failures frequently arise not from flawed reasoning alone, but from subtle misalignments in task interpretation, output format, or inter-agent handoffs. To address these challenges, we present VeriMAP, a framework for multi-agent collaboration with verification-aware planning. The VeriMAP planner decomposes tasks, models subtask dependencies, and encodes planner-defined passing criteria as subtask verification functions (VFs) in Python and natural language. We evaluate VeriMAP on diverse datasets, demonstrating that it outperforms both single- and multi-agent baselines while enhancing system robustness and interpretability. Our analysis highlights how verification-aware planning enables reliable coordination and iterative refinement in multi-agent systems, without relying on external labels or annotations.

Method: Proposed late fusion and fission approach with fission process that accesses both high- and low-level features for speech generation, implemented in SmolTolk models.

Result: SmolTolk models rival or surpass state-of-the-art TSLMs trained with orders of magnitude more compute, achieving significantly improved cross-modal performance. Representation analyses show enhanced ability to abstract higher-level semantic features from speech and more shared representation spaces across layers.

Conclusion: Late fusion/fission with multi-level feature access addresses limitations of early fusion approaches in TSLMs, enabling better cross-modal transfer and more efficient training while achieving competitive performance.

Abstract: Text-Speech Language Models (TSLMs) – language models trained to jointly process and generate text and speech – are commonly trained through an early modality fusion/fission approach, in which both modalities are fed and predicted from a shared backbone via linear layers. We hypothesize that this approach limits cross-modal transfer by neglecting feature compositionality – specifically, the finer-grained nature of speech representations compared to text – preventing the emergence of a shared feature hierarchy within model layers. In this paper, we argue that this limitation can be addressed through late fusion and fission, with a fission process that accesses both high- and low-level features for speech generation. Our models implementing these principles, SmolTolk, rival or surpass state-of-the-art TSLMs trained with orders of magnitude more compute, and achieve significantly improved cross-modal performance relative to early fusion/fission baselines. Representation analyses further suggest that our method enhances the model’s ability to abstract higher-level, more semantic features from speech, and leads to increasingly shared representation spaces across layers.

Method: Created MoReBench with 1,000 moral scenarios paired with 23k+ expert-defined rubric criteria covering moral considerations, trade-offs, and recommendations. Also developed MoReBench-Theory with 150 examples testing reasoning under five major normative ethics frameworks.

Result: Scaling laws and existing benchmarks on math, code, and scientific reasoning fail to predict models’ moral reasoning abilities. Models show bias toward specific moral frameworks (Benthamite Act Utilitarianism and Kantian Deontology), possibly due to training paradigms.

Conclusion: These benchmarks advance process-focused reasoning evaluation toward safer and more transparent AI by enabling assessment of how AI systems reason about moral dilemmas.

Abstract: As AI systems progress, we rely more on them to make decisions with us and for us. To ensure that such decisions are aligned with human values, it is imperative for us to understand not only what decisions they make but also how they come to those decisions. Reasoning language models, which provide both final responses and (partially transparent) intermediate thinking traces, present a timely opportunity to study AI procedural reasoning. Unlike math and code problems which often have objectively correct answers, moral dilemmas are an excellent testbed for process-focused evaluation because they allow for multiple defensible conclusions. To do so, we present MoReBench: 1,000 moral scenarios, each paired with a set of rubric criteria that experts consider essential to include (or avoid) when reasoning about the scenarios. MoReBench contains over 23 thousand criteria including identifying moral considerations, weighing trade-offs, and giving actionable recommendations to cover cases on AI advising humans moral decisions as well as making moral decisions autonomously. Separately, we curate MoReBench-Theory: 150 examples to test whether AI can reason under five major frameworks in normative ethics. Our results show that scaling laws and existing benchmarks on math, code, and scientific reasoning tasks fail to predict models’ abilities to perform moral reasoning. Models also show partiality towards specific moral frameworks (e.g., Benthamite Act Utilitarianism and Kantian Deontology), which might be side effects of popular training paradigms. Together, these benchmarks advance process-focused reasoning evaluation towards safer and more transparent AI.

Method: Two-phase approach: 1) Offline knowledge ingestion where LLM translates informal specs into formal symbolic knowledge base, 2) Online task processing where symbolic decision engine uses formal input encoding and knowledge base for reliable reasoning.

Result: Competitive with state-of-the-art end-to-end models in automated setup; with human-verified knowledge base, significantly outperforms larger models while achieving perfect determinism, enhanced stability, and immunity to prompt injection attacks.

Conclusion: ATA provides a practical and controllable architecture for building transparent, auditable, and reliable autonomous agents through symbolic reasoning.

Abstract: Large Language Models (LLMs) have demonstrated impressive capabilities, yet their deployment in high-stakes domains is hindered by inherent limitations in trustworthiness, including hallucinations, instability, and a lack of transparency. To address these challenges, we introduce a generic neuro-symbolic approach, which we call Autonomous Trustworthy Agents (ATA). The core of our approach lies in decoupling tasks into two distinct phases: Offline knowledge ingestion and online task processing. During knowledge ingestion, an LLM translates an informal problem specification into a formal, symbolic knowledge base. This formal representation is crucial as it can be verified and refined by human experts, ensuring its correctness and alignment with domain requirements. In the subsequent task processing phase, each incoming input is encoded into the same formal language. A symbolic decision engine then utilizes this encoded input in conjunction with the formal knowledge base to derive a reliable result. Through an extensive evaluation on a complex reasoning task, we demonstrate that a concrete implementation of ATA is competitive with state-of-the-art end-to-end reasoning models in a fully automated setup while maintaining trustworthiness. Crucially, with a human-verified and corrected knowledge base, our approach significantly outperforms even larger models, while exhibiting perfect determinism, enhanced stability against input perturbations, and inherent immunity to prompt injection attacks. By generating decisions grounded in symbolic reasoning, ATA offers a practical and controllable architecture for building the next generation of transparent, auditable, and reliable autonomous agents.

Method: SHANKS streams input speech in fixed-duration chunks and generates unspoken chain-of-thought reasoning as each chunk arrives, using all previous speech and reasoning to decide whether to interrupt users or make tool calls while they continue speaking.

Result: SHANKS achieved 37.1% higher interruption accuracy than baseline when detecting math mistakes, and completed 56.9% of tool calls before users finished speaking in tool-augmented dialogues.

Conclusion: SHANKS enables models to think throughout conversations rather than only after turns end, moving toward more natural, real-time human-model interaction.

Abstract: Current large language models (LLMs) and spoken language models (SLMs) begin thinking and taking actions only after the user has finished their turn. This prevents the model from interacting during the user’s turn and can lead to high response latency while it waits to think. Consequently, thinking after receiving the full input is not suitable for speech-to-speech interaction, where real-time, low-latency exchange is important. We address this by noting that humans naturally “think while listening.” In this paper, we propose SHANKS, a general inference framework that enables SLMs to generate unspoken chain-of-thought reasoning while listening to the user input. SHANKS streams the input speech in fixed-duration chunks and, as soon as a chunk is received, generates unspoken reasoning based on all previous speech and reasoning, while the user continues speaking. SHANKS uses this unspoken reasoning to decide whether to interrupt the user and to make tool calls to complete the task. We demonstrate that SHANKS enhances real-time user-SLM interaction in two scenarios: (1) when the user is presenting a step-by-step solution to a math problem, SHANKS can listen, reason, and interrupt when the user makes a mistake, achieving 37.1% higher interruption accuracy than a baseline that interrupts without thinking; and (2) in a tool-augmented dialogue, SHANKS can complete 56.9% of the tool calls before the user finishes their turn. Overall, SHANKS moves toward models that keep thinking throughout the conversation, not only after a turn ends. Animated illustrations of Shanks can be found at https://d223302.github.io/SHANKS/

Method: Uses GlobEnc and DecompX token attribution methods to assign salience scores, rank tokens, and preserve top-k% tokens in original order to create sparse frugalized prompts.

Result: 20% prompt reduction causes only marginal performance loss on sentiment analysis, commonsense QA, and summarization, but sharp deterioration on mathematical reasoning. Bottom-k% and random-k% tokens show asymmetric patterns suggesting potential task contamination.

Conclusion: The work provides nuanced understanding of LLM behavior in performance-efficiency trade-offs, delineating boundaries between tasks tolerant to contextual sparsity versus those requiring exhaustive context.

Abstract: Large language models (LLMs) owe much of their stellar performance to expansive input contexts, yet such verbosity inflates monetary costs, carbon footprint, and inference-time latency. Much of this overhead manifests from the redundant low-utility tokens present in typical prompts, as only a fraction of tokens typically carries the majority of the semantic weight. We address this inefficiency by introducing FrugalPrompt, a novel prompt compression framework for LLMs, which retains only the most semantically significant tokens. Leveraging two state-of-the-art token attribution methods, GlobEnc and DecompX, we assign salience scores to every token in an input sequence, rank them to preserve the top-k% tokens in their original order, and obtain a sparse frugalized prompt. We evaluate the approach across four NLP tasks: Sentiment Analysis, Commonsense QA, Summarization, and Mathematical Reasoning, using a suite of frontier LLMs. For the first three tasks, a 20% prompt reduction incurs only a marginal loss in task performance, demonstrating that contemporary LLMs can reconstruct elided context from high-salience cues. In contrast, performance on mathematical reasoning deteriorates sharply, reflecting a stronger dependence on complete token continuity. Further analysis with bottom-k% and random-k% tokens reveals asymmetric performance patterns that may suggest potential task contamination effects, wherein models may resort to shallow memorized patterns from pretraining exposure for conventional NLP tasks. We posit that our work contributes to a more nuanced understanding of LLM behavior in performance-efficiency trade-offs, and delineate the boundary between tasks tolerant to contextual sparsity and those requiring exhaustive context. Our source code and models are available at: https://github.com/Starscream-11813/Frugal-ICL

Method: Uses hidden states from sampler LLM for process-level scoring with a 0.6B parameter lightweight verifier that evaluates step-wise trajectory quality and aggregates scores to select optimal reasoning path. Employs end-to-end training without step-level annotations.

Result: Outperforms majority voting by 4.61% accuracy and existing process reward models by 4.31% to 12.21% in Best-of-32 settings across five benchmarks, with lower inference costs.

Conclusion: TrajSelector provides an efficient and effective Best-of-N framework that leverages LLM latent representations to achieve scalable performance improvements with reduced computational overhead.

Abstract: Large language models (LLMs) have shown remarkable progress in complex reasoning tasks, largely enabled by test-time scaling (TTS) paradigms that allocate additional compute during inference. Among these, external TTS (particularly the Best-of-N selection paradigm) yields scalable performance improvements by selecting from multiple independently generated reasoning trajectories. However, this approach faces key limitations: (i) the high computational overhead of deploying process reward models, (ii) the underutilization of the LLM’s intrinsic latent representations. We introduce TrajSelector, an efficient and effective Best-of-N framework that exploit the hidden states in the sampler LLM for process-level scoring. A lightweight verifier (with only 0.6B parameters) evaluates the quality of step-wise trajectory, and then aggregates these scores to identify the optimal reasoning trajectory. Our framework employs a fully data-driven, end-to-end training recipe that eliminates reliance on massive step-level annotations. Experiential results across five benchmarks demonstrate that TrajSelector delivers consistent performance gains. In Best-of-32 settings, it surpasses majority voting by 4.61% accuracy and outperforms existing process reward models by 4.31% to 12.21%, all while maintaining lower inference costs.

Method: Integrates curriculum reinforcement learning with MLLMs, uses progressive training with mixed precision annotations, employs Group Relative Policy Optimization (GRPO) for emergent reasoning, and implements hierarchical reward mechanisms.

Result: Achieves superior performance in violation category accuracy and temporal interval localization on industrial datasets and public benchmarks, with significant improvements in precision and recall in online A/B testing.

Conclusion: RAVEN demonstrates strong generalization, mitigates catastrophic forgetting, and shows practical applicability for online ad services deployment.

Abstract: Advertisement (Ad) video violation detection is critical for ensuring platform compliance, but existing methods struggle with precise temporal grounding, noisy annotations, and limited generalization. We propose RAVEN, a novel framework that integrates curriculum reinforcement learning with multimodal large language models (MLLMs) to enhance reasoning and cognitive capabilities for violation detection. RAVEN employs a progressive training strategy, combining precisely and coarsely annotated data, and leverages Group Relative Policy Optimization (GRPO) to develop emergent reasoning abilities without explicit reasoning annotations. Multiple hierarchical sophisticated reward mechanism ensures precise temporal grounding and consistent category prediction. Experiments on industrial datasets and public benchmarks show that RAVEN achieves superior performances in violation category accuracy and temporal interval localization. We also design a pipeline to deploy the RAVEN on the online Ad services, and online A/B testing further validates its practical applicability, with significant improvements in precision and recall. RAVEN also demonstrates strong generalization, mitigating the catastrophic forgetting issue associated with supervised fine-tuning.

Method: Applied LiTEx taxonomy to two English NLI datasets, aligning annotation variation through multiple aspects: NLI label agreement, explanation similarity, and taxonomy agreement, while considering annotators’ selection bias.

Result: Found instances where annotators disagree on labels but provide highly similar explanations, suggesting surface-level disagreement may mask underlying agreement. Also revealed individual preferences in explanation strategies and label choices.

Conclusion: Agreement in reasoning types better reflects semantic similarity of free-text explanations than label agreement alone, highlighting the richness of reasoning-based explanations and the need for caution in treating labels as ground truth.

Abstract: Natural Language Inference datasets often exhibit human label variation. To better understand these variations, explanation-based approaches analyze the underlying reasoning behind annotators’ decisions. One such approach is the LiTEx taxonomy, which categorizes free-text explanations in English into reasoning types. However, previous work applying such taxonomies has focused on within-label variation: cases where annotators agree on the final NLI label but provide different explanations. In contrast, this paper broadens the scope by examining how annotators may diverge not only in the reasoning type but also in the labeling step. We use explanations as a lens to decompose the reasoning process underlying NLI annotation and to analyze individual differences. We apply LiTEx to two NLI English datasets and align annotation variation from multiple aspects: NLI label agreement, explanation similarity, and taxonomy agreement, with an additional compounding factor of annotators’ selection bias. We observe instances where annotators disagree on the label but provide highly similar explanations, suggesting that surface-level disagreement may mask underlying agreement in interpretation. Moreover, our analysis reveals individual preferences in explanation strategies and label choices. These findings highlight that agreement in reasoning types better reflects the semantic similarity of free-text explanations than label agreement alone. Our findings underscore the richness of reasoning-based explanations and the need for caution in treating labels as ground truth.

Method: xKG is a modular knowledge base that automatically extracts and integrates technical insights, code snippets, and domain-specific knowledge from scientific literature to support multi-granular retrieval and reuse.

Result: When integrated into three agent frameworks with two LLMs, xKG shows substantial performance gains (10.9% with o3-mini) on PaperBench.

Conclusion: xKG is an effective and extensible solution for automated AI research replication, demonstrating general applicability across different agent frameworks and LLMs.

Abstract: Replicating AI research is a crucial yet challenging task for large language model (LLM) agents. Existing approaches often struggle to generate executable code, primarily due to insufficient background knowledge and the limitations of retrieval-augmented generation (RAG) methods, which fail to capture latent technical details hidden in referenced papers. Furthermore, previous approaches tend to overlook valuable implementation-level code signals and lack structured knowledge representations that support multi-granular retrieval and reuse. To overcome these challenges, we propose Executable Knowledge Graphs (xKG), a modular and pluggable knowledge base that automatically integrates technical insights, code snippets, and domain-specific knowledge extracted from scientific literature. When integrated into three agent frameworks with two different LLMs, xKG shows substantial performance gains (10.9% with o3-mini) on PaperBench, demonstrating its effectiveness as a general and extensible solution for automated AI research replication. Code will released at https://github.com/zjunlp/xKG.

Method: Proposed using explicit quit instructions within the ToolEmu framework, systematically evaluating quitting behavior across 12 state-of-the-art LLMs.

Result: Agents with quit instructions improved safety by +0.39 on 0-3 scale (+0.64 for proprietary models) with only -0.03 decrease in helpfulness.

Conclusion: Quitting serves as an effective first-line defense mechanism that can be immediately deployed in existing agent systems for high-stakes applications.

Abstract: As Large Language Model (LLM) agents increasingly operate in complex environments with real-world consequences, their safety becomes critical. While uncertainty quantification is well-studied for single-turn tasks, multi-turn agentic scenarios with real-world tool access present unique challenges where uncertainties and ambiguities compound, leading to severe or catastrophic risks beyond traditional text generation failures. We propose using “quitting” as a simple yet effective behavioral mechanism for LLM agents to recognize and withdraw from situations where they lack confidence. Leveraging the ToolEmu framework, we conduct a systematic evaluation of quitting behavior across 12 state-of-the-art LLMs. Our results demonstrate a highly favorable safety-helpfulness trade-off: agents prompted to quit with explicit instructions improve safety by an average of +0.39 on a 0-3 scale across all models (+0.64 for proprietary models), while maintaining a negligible average decrease of -0.03 in helpfulness. Our analysis demonstrates that simply adding explicit quit instructions proves to be a highly effective safety mechanism that can immediately be deployed in existing agent systems, and establishes quitting as an effective first-line defense mechanism for autonomous agents in high-stakes applications.

Method: Introduces an online knapsack-based framework where a composer agent dynamically tests candidate components and models their utility in real-time to assemble optimal agent sets under budget constraints.

Result: Empirical evaluation shows the online knapsack composer achieves up to 31.6% higher success rates than retrieval baselines in single-agent setups, and increases success rate from 37% to 87% in multi-agent systems with 100+ agents.

Conclusion: The framework enables scalable reuse of agentic components and consistently achieves Pareto-optimal performance across diverse domains and budget constraints.

Abstract: Designing effective agentic systems requires the seamless composition and integration of agents, tools, and models within dynamic and uncertain environments. Most existing methods rely on static, semantic retrieval approaches for tool or agent discovery. However, effective reuse and composition of existing components remain challenging due to incomplete capability descriptions and the limitations of retrieval methods. Component selection suffers because the decisions are not based on capability, cost, and real-time utility. To address these challenges, we introduce a structured, automated framework for agentic system composition that is inspired by the knapsack problem. Our framework enables a composer agent to systematically identify, select, and assemble an optimal set of agentic components by jointly considering performance, budget constraints, and compatibility. By dynamically testing candidate components and modeling their utility in real-time, our approach streamlines the assembly of agentic systems and facilitates scalable reuse of resources. Empirical evaluation with Claude 3.5 Sonnet across five benchmarking datasets shows that our online-knapsack-based composer consistently lies on the Pareto frontier, achieving higher success rates at significantly lower component costs compared to our baselines. In the single-agent setup, the online knapsack composer shows a success rate improvement of up to 31.6% in comparison to the retrieval baselines. In multi-agent systems, the online knapsack composer increases success rate from 37% to 87% when agents are selected from an agent inventory of 100+ agents. The substantial performance gap confirms the robust adaptability of our method across diverse domains and budget constraints.

Method: A comprehensive four-stage LLM-driven framework: (1) collects ICLR and NeurIPS papers with reviews from OpenReview; (2) annotates review types using GPT-4.1 with human validation; (3) addresses class imbalance through LLM-driven synthetic data augmentation; (4) fine-tunes encoder-based models and open source LLMs.

Result: Created a corpus of 6,634 papers, 24,657 real reviews, and 46,438 synthetic reviews. Deficient reviews show lower rating scores, higher confidence, reduced structural complexity, and more negative sentiment. AI-generated reviews increased dramatically post-ChatGPT. Mixed training with synthetic and real data improved recall and F1 scores.

Conclusion: This study presents the first LLM-driven system for detecting deficient peer reviews, providing evidence to inform AI governance in peer review and offering insights into human-AI collaboration to maintain academic integrity.

Abstract: Peer review serves as the gatekeeper of science, yet the surge in submissions and widespread adoption of large language models (LLMs) in scholarly evaluation present unprecedented challenges. Recent work has focused on using LLMs to improve review efficiency or generate insightful review content. However, unchecked deficient reviews from both human experts and AI systems threaten to systematically undermine the peer review ecosystem and compromise academic integrity. To address this critical issue, we introduce ReviewGuard, an automated system for detecting and categorizing deficient reviews. ReviewGuard employs a comprehensive four-stage LLM-driven framework that: (1) collects ICLR and NeurIPS papers with their corresponding reviews from OpenReview; (2) annotates review types using GPT-4.1 with human validation; (3) addresses class imbalance and data scarcity through LLM-driven synthetic data augmentation, producing a final corpus of 6,634 papers, 24,657 real reviews, and 46,438 synthetic reviews; and (4) fine-tunes both encoder-based models and open source LLMs. We perform comprehensive feature analysis of the structure and quality of the review text. Compared to sufficient reviews, deficient reviews demonstrate lower rating scores, higher self-reported confidence, reduced structural complexity, and a higher proportion of negative sentiment. AI-generated text detection reveals that, since ChatGPT’s emergence, AI-generated reviews have increased dramatically. In the evaluation of deficient review detection models, mixed training with synthetic and real review data provides substantial enhancements to recall and F1 scores on the binary task. This study presents the first LLM-driven system for detecting deficient peer reviews, providing evidence to inform AI governance in peer review while offering valuable insights into human-AI collaboration to maintain academic integrity.

Method: The researchers traced LLMs’ internal cultural understanding by measuring activation path overlaps when answering semantically equivalent questions under two conditions: varying target country while fixing question language, and varying question language while fixing country. They also used same-language country pairs to disentangle language from cultural aspects.

Result: Results show that internal paths overlap more for same-language, cross-country questions than for cross-language, same-country questions, indicating strong language-specific patterns. The South Korea-North Korea pair exhibited low overlap and high variability, showing linguistic similarity doesn’t guarantee aligned internal representation.

Conclusion: The study reveals that LLMs exhibit strong language-specific patterns in cultural understanding, and linguistic similarity between countries doesn’t necessarily translate to aligned internal representations, highlighting the complexity of cultural understanding mechanisms in language models.

Abstract: Large language models (LLMs) are increasingly used across diverse cultural contexts, making accurate cultural understanding essential. Prior evaluations have mostly focused on output-level performance, obscuring the factors that drive differences in responses, while studies using circuit analysis have covered few languages and rarely focused on culture. In this work, we trace LLMs’ internal cultural understanding mechanisms by measuring activation path overlaps when answering semantically equivalent questions under two conditions: varying the target country while fixing the question language, and varying the question language while fixing the country. We also use same-language country pairs to disentangle language from cultural aspects. Results show that internal paths overlap more for same-language, cross-country questions than for cross-language, same-country questions, indicating strong language-specific patterns. Notably, the South Korea-North Korea pair exhibits low overlap and high variability, showing that linguistic similarity does not guarantee aligned internal representation.

Method: Developed balanced dataset (1,800 human + 1,800 AI texts from Gemini, GPT-4o-mini, Kimi AI), conducted linguistic/statistical analysis, and fine-tuned three multilingual transformers (mdeberta-v3-base, distilbert-base-multilingualcased, xlm-roberta-base).

Result: mDeBERTa-v3-base achieved highest performance with 91.29 F1-score and 91.26% accuracy on test set, significantly advancing Urdu AI text detection capabilities.

Conclusion: The research successfully addresses the gap in Urdu AI text detection, contributes to NLP tools for low-resource languages, and helps combat misinformation in Urdu-speaking communities.

Abstract: Large Language Models (LLMs) are now capable of generating text that closely resembles human writing, making them powerful tools for content creation, but this growing ability has also made it harder to tell whether a piece of text was written by a human or by a machine. This challenge becomes even more serious for languages like Urdu, where there are very few tools available to detect AI-generated text. To address this gap, we propose a novel AI-generated text detection framework tailored for the Urdu language. A balanced dataset comprising 1,800 humans authored, and 1,800 AI generated texts, sourced from models such as Gemini, GPT-4o-mini, and Kimi AI was developed. Detailed linguistic and statistical analysis was conducted, focusing on features such as character and word counts, vocabulary richness (Type Token Ratio), and N-gram patterns, with significance evaluated through t-tests and MannWhitney U tests. Three state-of-the-art multilingual transformer models such as mdeberta-v3-base, distilbert-base-multilingualcased, and xlm-roberta-base were fine-tuned on this dataset. The mDeBERTa-v3-base achieved the highest performance, with an F1-score 91.29 and accuracy of 91.26% on the test set. This research advances efforts in contesting misinformation and academic misconduct in Urdu-speaking communities and contributes to the broader development of NLP tools for low resource languages.

Method: Fine-tuned several models from Hugging Face repository using training data generated from AnCora-ES corpus to translate input sentences into corresponding syntactic structures.

Result: Demonstrated high accuracy in phrase-structure analysis as evaluated by F1 score.

Conclusion: The methodology shows great potential for syntactic analysis using fine-tuned large language models.

Abstract: Recent advances in natural language processing with large neural models have opened new possibilities for syntactic analysis based on machine learning. This work explores a novel approach to phrase-structure analysis by fine-tuning large language models (LLMs) to translate an input sentence into its corresponding syntactic structure. The main objective is to extend the capabilities of MiSintaxis, a tool designed for teaching Spanish syntax. Several models from the Hugging Face repository were fine-tuned using training data generated from the AnCora-ES corpus, and their performance was evaluated using the F1 score. The results demonstrate high accuracy in phrase-structure analysis and highlight the potential of this methodology.

Method: Introduces Capsule Prompt-Tuning (CaPT) that leverages off-the-shelf instance semantics to integrate both instance-aware and task-aware information in a nearly parameter-free manner using a single capsule prompt.

Result: Achieves 84.03% average accuracy on T5-Large and uses only 0.003% of model parameters on Llama3.2-1B, demonstrating superior performance and high parameter efficiency.

Conclusion: CaPT effectively serves as an “attention anchor” that preserves strong attention to critical structural information and exhibits active attention interaction with all input tokens, making prompt-based learning more efficient and effective.

Abstract: Prompt-based learning has emerged as a parameter-efficient finetuning (PEFT) approach to facilitate Large Language Model (LLM) adaptation to downstream tasks by conditioning generation with task-aware guidance. Despite its successes, current prompt-based learning methods heavily rely on laborious grid searching for optimal prompt length and typically require considerable number of prompts, introducing additional computational burden. Worse yet, our pioneer findings indicate that the task-aware prompt design is inherently limited by its absence of instance-aware information, leading to a subtle attention interplay with the input sequence. In contrast, simply incorporating instance-aware information as a part of the guidance can enhance the prompt-tuned model performance without additional fine-tuning. Moreover, we find an interesting phenomenon, namely “attention anchor”, that incorporating instance-aware tokens at the earliest position of the sequence can successfully preserve strong attention to critical structural information and exhibit more active attention interaction with all input tokens. In light of our observation, we introduce Capsule Prompt-Tuning (CaPT), an efficient and effective solution that leverages off-the-shelf, informative instance semantics into prompt-based learning. Our approach innovatively integrates both instance-aware and task-aware information in a nearly parameter-free manner (i.e., one single capsule prompt). Empirical results demonstrate that our method can exhibit superior performance across various language tasks (e.g., 84.03% average accuracy on T5-Large), serving as an “attention anchor,” while enjoying high parameter efficiency (e.g., 0.003% of model parameters on Llama3.2-1B).

Method: The TUuD framework prompts LLMs to rate similarity between current moment and target events (0.00-1.00 scale) when the reference point of “now” dynamically shifts along a timeline, evaluating temporal understanding under deictic temporal frames of reference.

Result: Four evaluated LLMs show measurable adaptation to deictic t-FoR, with similarity ratings peaking around present and decreasing toward past/future events, but this adaptation weakens beyond near-term contexts.

Conclusion: LLMs display partial human-like temporal cognition but their temporal reasoning remains sensitive to reference-frame shifts and temporal distance, indicating limitations in temporal understanding.

Abstract: Understanding time is fundamental to human cognition, where temporal experience is often conceptualized through spatial metaphors grounded in sensory-motor experience. For example, “summer is approaching” parallels “We are approaching the summer”. In such expressions, humans rely on a frame of reference (FoR) to interpret meaning relative to a particular viewpoint. Extending this concept to time, a temporal frame of reference (t-FoR) defines how temporal relations are perceived relative to an experiencer’s moment of “now”. While Large Language Models (LLMs) have shown remarkable advances in natural language understanding, their ability to interpret and reason about time remains limited. In this work, we introduce TUuD (Temporal Understanding under Deictic t-FoR), a framework that evaluates how LLMs interpret time-event and event-event relations when the reference point of “now” dynamically shifts along a timeline. Following recent work on temporal cognition \cite{li2025other}, LLMs are prompted to rate the similarity between the current moment and a target event from 0.00 (completely dissimilar) to 1.00 (highly similar), where similarity quantifies perceived temporal alignment between the two points. Our results show that four evaluated LLMs exhibit measurable adaptation to a deictic t-FoR, with similarity ratings peaking around the present and decreasing toward past and future events. The adaptation, however, weakens beyond near-term contexts, suggesting that while LLMs display partial human-like temporal cognition, their temporal reasoning remains sensitive to reference-frame shifts and temporal distance.

Method: Constructed NLURC (a comprehensive NLU dataset with rationales) and developed various rationale-augmented methods. Systematically explored these methods on NLU tasks to evaluate their effectiveness.

Result: Three key findings: (1) CoT inference shifts from hindering to surpassing direct prediction as model size grows; (2) Most rationale-augmented training performs worse than label-only training, except one specially designed method; (3) Models trained with rationales achieve significant gains on unseen tasks, rivaling much larger models while maintaining interpretability.

Conclusion: Rationales can benefit NLU tasks, particularly showing positive correlation with model size, and specially designed rationale-augmented training methods can achieve competitive performance with improved interpretability on unseen tasks.

Abstract: Chain-of-thought (CoT) rationales, which provide step-by-step reasoning to derive final answers, benefit LLMs in both inference and training. Incorporating rationales, either by generating them before answering during inference, or by placing them before or after the original answers during training - significantly improves model performance on mathematical, symbolic and commonsense reasoning tasks. However, most work focuses on the role of rationales in these reasoning tasks, overlooking their potential impact on other important tasks like natural language understanding (NLU) tasks. In this work, we raise the question: Can rationales similarly benefit NLU tasks? To conduct a systematic exploration, we construct NLURC, a comprehensive and high-quality NLU dataset collection with rationales, and develop various rationale-augmented methods. Through exploring the applicability of these methods on NLU tasks using the dataset, we uncover several potentially surprising findings: (1) CoT inference shifts from hindering NLU performance to surpassing direct label prediction as model size grows, indicating a positive correlation. (2) Most rationale-augmented training methods perform worse than label-only training, with one specially designed method consistently achieving improvements. (3) LLMs trained with rationales achieve significant performance gains on unseen NLU tasks, rivaling models ten times their size, while delivering interpretability on par with commercial LLMs.

Method: The authors conducted a systematic review by querying six literature databases to identify articles applying NLP techniques in cardiovascular disease contexts. They screened and analyzed 265 relevant articles across multiple dimensions: NLP paradigm types, cardiology task types, cardiovascular disease types, and data source types, including temporal analysis of trends.

Result: The analysis revealed considerable diversity across all dimensions studied, demonstrating the breadth of NLP research in cardiology. Temporal analysis showed the evolution and changing trends in NLP methods over the past decade, providing the most comprehensive overview of this research area to date.

Conclusion: This review constitutes the most comprehensive overview of NLP research in cardiology, highlighting the field’s diversity and evolution over time, which can help healthcare professionals gain deeper insights and potentially revolutionize approaches to cardiac disease diagnosis, treatment, and prevention.

Abstract: Cardiovascular disease has become increasingly prevalent in modern society and has a significant effect on global health and well-being. Heart-related conditions are intricate, multifaceted disorders, which may be influenced by a combination of genetic predispositions, lifestyle choices, and various socioeconomic and clinical factors. Information regarding these potentially complex interrelationships is dispersed among diverse types of textual data, which include patient narratives, medical records, and scientific literature, among others. Natural language processing (NLP) techniques have increasingly been adopted as a powerful means to analyse and make sense of this vast amount of unstructured data. This, in turn, can allow healthcare professionals to gain deeper insights into the cardiology field, which has the potential to revolutionize current approaches to the diagnosis, treatment, and prevention of cardiac problems. This review provides a detailed overview of NLP research in cardiology between 2014 and 2025. We queried six literature databases to find articles describing the application of NLP techniques in the context of a range of different cardiovascular diseases. Following a rigorous screening process, we identified a total of 265 relevant articles. We analysed each article from multiple dimensions, i.e., NLP paradigm types, cardiology-related task types, cardiovascular disease types, and data source types. Our analysis reveals considerable diversity within each of these dimensions, thus demonstrating the considerable breadth of NLP research within the field. We also perform a temporal analysis, which illustrates the evolution and changing trends in NLP methods employed over the last decade that we cover. To our knowledge, the review constitutes the most comprehensive overview of NLP research in cardiology to date.

Method: Two experiments: first evaluated controllability of values in LLMs to identify optimal models and prompts, then generated agent pairs with specific values and analyzed their mutual evaluations after dialogues in English and Japanese.

Result: Pairs of LLM agents with higher value similarity exhibited greater mutual trust and interpersonal closeness, confirming the principle holds in artificial societies.

Conclusion: LLM agent simulations serve as valid testbeds for social science theories, help explain how values influence relationship building, and provide foundations for new social science insights.

Abstract: Large language models (LLMs) have emerged as powerful tools for simulating complex social phenomena using human-like agents with specific traits. In human societies, value similarity is important for building trust and close relationships; however, it remains unexplored whether this principle holds true in artificial societies comprising LLM agents. Therefore, this study investigates the influence of value similarity on relationship-building among LLM agents through two experiments. First, in a preliminary experiment, we evaluated the controllability of values in LLMs to identify the most effective model and prompt design for controlling the values. Subsequently, in the main experiment, we generated pairs of LLM agents imbued with specific values and analyzed their mutual evaluations of trust and interpersonal closeness following a dialogue. The experiments were conducted in English and Japanese to investigate language dependence. The results confirmed that pairs of agents with higher value similarity exhibited greater mutual trust and interpersonal closeness. Our findings demonstrate that the LLM agent simulation serves as a valid testbed for social science theories, contributes to elucidating the mechanisms by which values influence relationship building, and provides a foundation for inspiring new theories and insights into the social sciences.

Method: Created Chameleon Benchmark Dataset with 17,770 question-answer pairs across 1,180 multi-turn conversations spanning 12 controversial domains. Introduced Chameleon Score (stance instability) and Source Re-use Rate (knowledge diversity) metrics. Evaluated Llama-4-Maverick, GPT-4o-mini, and Gemini-2.5-Flash.

Result: All models showed severe chameleon behavior (scores 0.391-0.511), with GPT-4o-mini performing worst. Strong correlations found between source re-use rate and confidence (r=0.627) and stance changes (r=0.429), indicating limited knowledge diversity makes models deferential to query framing.

Conclusion: LLMs exhibit pathological deference to query framing due to limited knowledge diversity, highlighting the need for comprehensive consistency evaluation before deployment in healthcare, legal, and financial systems where coherent positions are critical.

Abstract: Integration of Large Language Models with search/retrieval engines has become ubiquitous, yet these systems harbor a critical vulnerability that undermines their reliability. We present the first systematic investigation of “chameleon behavior” in LLMs: their alarming tendency to shift stances when presented with contradictory questions in multi-turn conversations (especially in search-enabled LLMs). Through our novel Chameleon Benchmark Dataset, comprising 17,770 carefully crafted question-answer pairs across 1,180 multi-turn conversations spanning 12 controversial domains, we expose fundamental flaws in state-of-the-art systems. We introduce two theoretically grounded metrics: the Chameleon Score (0-1) that quantifies stance instability, and Source Re-use Rate (0-1) that measures knowledge diversity. Our rigorous evaluation of Llama-4-Maverick, GPT-4o-mini, and Gemini-2.5-Flash reveals consistent failures: all models exhibit severe chameleon behavior (scores 0.391-0.511), with GPT-4o-mini showing the worst performance. Crucially, small across-temperature variance (less than 0.004) suggests the effect is not a sampling artifact. Our analysis uncovers the mechanism: strong correlations between source re-use rate and confidence (r=0.627) and stance changes (r=0.429) are statistically significant (p less than 0.05), indicating that limited knowledge diversity makes models pathologically deferential to query framing. These findings highlight the need for comprehensive consistency evaluation before deploying LLMs in healthcare, legal, and financial systems where maintaining coherent positions across interactions is critical for reliable decision support.

Method: Each node acts as an autonomous agent with internal memory for node-level planning. Uses retrieval-augmented generation (RAG) to access semantically relevant content and build global relationships.

Result: Achieves competitive performance under few-shot in-context settings using a frozen LLM backbone without fine-tuning.

Conclusion: Demonstrates the potential of agentic planning and local-global retrieval in graph learning, enabling adaptive message propagation and global relationship building.

Abstract: Graph Neural Networks (GNNs) have achieved remarkable success in graph-based learning by propagating information among neighbor nodes via predefined aggregation mechanisms. However, such fixed schemes often suffer from two key limitations. First, they cannot handle the imbalance in node informativeness – some nodes are rich in information, while others remain sparse. Second, predefined message passing primarily leverages local structural similarity while ignoring global semantic relationships across the graph, limiting the model’s ability to capture distant but relevant information. We propose Retrieval-augmented Graph Agentic Network (ReaGAN), an agent-based framework that empowers each node with autonomous, node-level decision-making. Each node acts as an agent that independently plans its next action based on its internal memory, enabling node-level planning and adaptive message propagation. Additionally, retrieval-augmented generation (RAG) allows nodes to access semantically relevant content and build global relationships in the graph. ReaGAN achieves competitive performance under few-shot in-context settings using a frozen LLM backbone without fine-tuning, showcasing the potential of agentic planning and local-global retrieval in graph learning.

Method: Analyzed 19k English poems from Poetry Foundation, comparing whitespace usage across 4k poets, 51k LLM-generated poems, and 12k unpublished online poems. Examined variations across time periods, poetic forms, and data sources.

Result: Found significant differences in whitespace usage between published poems, LLM-generated poems, and unpublished online poems. Different text processing methods produce substantially different whitespace representations in poetry data.

Conclusion: Whitespace patterns in poetry reveal important artistic and semantic information. The findings highlight implications for how pretraining datasets for LLMs are processed, suggesting current methods may inadequately preserve whitespace features.

Abstract: Whitespace is a critical component of poetic form, reflecting both adherence to standardized forms and rebellion against those forms. Each poem’s whitespace distribution reflects the artistic choices of the poet and is an integral semantic and spatial feature of the poem. Yet, despite the popularity of poetry as both a long-standing art form and as a generation task for large language models (LLMs), whitespace has not received sufficient attention from the NLP community. Using a corpus of 19k English-language published poems from Poetry Foundation, we investigate how 4k poets have used whitespace in their works. We release a subset of 2.8k public-domain poems with preserved formatting to facilitate further research in this area. We compare whitespace usage in the published poems to (1) 51k LLM-generated poems, and (2) 12k unpublished poems posted in an online community. We also explore whitespace usage across time periods, poetic forms, and data sources. Additionally, we find that different text processing methods can result in significantly different representations of whitespace in poetry data, motivating us to use these poems and whitespace patterns to discuss implications for the processing strategies used to assemble pretraining datasets for LLMs.

Method: Introduces Beacon, a single-turn forced-choice benchmark that isolates sycophancy bias. Evaluates twelve state-of-the-art models and proposes both prompt-level and activation-level interventions to modulate these biases.

Result: Evaluations reveal sycophancy decomposes into stable linguistic and affective sub-biases that scale with model capacity. The interventions can modulate these biases in opposing directions, exposing the internal geometry of alignment as a dynamic manifold.

Conclusion: Beacon reframes sycophancy as a measurable form of normative misgeneralization, providing a reproducible foundation for studying and mitigating alignment drift in large-scale generative systems.

Abstract: Large language models internalize a structural trade-off between truthfulness and obsequious flattery, emerging from reward optimization that conflates helpfulness with polite submission. This latent bias, known as sycophancy, manifests as a preference for user agreement over principled reasoning. We introduce Beacon, a single-turn forced-choice benchmark that isolates this bias independent of conversational context, enabling precise measurement of the tension between factual accuracy and submissive bias. Evaluations across twelve state-of-the-art models reveal that sycophancy decomposes into stable linguistic and affective sub-biases, each scaling with model capacity. We further propose prompt-level and activation-level interventions that modulate these biases in opposing directions, exposing the internal geometry of alignment as a dynamic manifold between truthfulness and socially compliant judgment. Beacon reframes sycophancy as a measurable form of normative misgeneralization, providing a reproducible foundation for studying and mitigating alignment drift in large-scale generative systems.

Method: Proposed SCO-PAL (Step-level poliCy Optimization through Play-And-Learn) method that analyzes opponent selection at different levels and identifies self-play as the most effective approach.

Result: SCO-PAL with self-play increased average win rate by approximately 30% against four opponents compared to baselines, and achieved 54.76% win rate against GPT-4 in six adversarial games.

Conclusion: Self-play is the most effective way to improve strategic reasoning in adversarial environments, and SCO-PAL demonstrates significant performance improvements through this approach.

Abstract: Existing language agents often encounter difficulties in dynamic adversarial games due to poor strategic reasoning. To mitigate this limitation, a promising approach is to allow agents to learn from game interactions automatically, without relying on costly expert-labeled data. Unlike static environments where agents receive fixed feedback or rewards, selecting appropriate opponents in dynamic adversarial games can significantly impact learning performance. However, the discussion of opponents in adversarial environments remains an area under exploration. In this paper, we propose a Step-level poliCy Optimization method through Play-And-Learn, SCO-PAL. Leveraging SCO-PAL, we conduct a detailed analysis of opponent selection by setting opponents at different levels and find that self-play is the most effective way to improve strategic reasoning in such adversarial environments. Utilizing SCO-PAL with self-play, we increase the average win rate against four opponents by approximately 30% compared to baselines and achieve a 54.76% win rate against GPT-4 in six adversarial games.

Method: Developed LC-Eval benchmark with four novel tasks: multi-document QA, bilingual QA, claim verification, and long-context multiple-choice questions. Created datasets in both English and Arabic for comparative analysis across text genres.

Result: Evaluations on open-weight and closed LLMs showed LC-Eval presents significant challenges. Even high-performing models like GPT-4o struggled with certain tasks, demonstrating the benchmark’s complexity and rigor.

Conclusion: LC-Eval effectively exposes limitations in current LLMs’ long-context understanding capabilities, particularly in bilingual settings and complex reasoning tasks, highlighting the need for continued model improvement in extended context processing.

Abstract: Recent advancements in Large Language Models (LLMs) have demonstrated sophisticated capabilities, including the ability to process and comprehend extended contexts. These emergent capabilities necessitate rigorous evaluation methods to effectively assess their performance in long-context understanding. In this paper, we present \textbf{LC-Eval}, a bilingual, multi-task evaluation benchmark designed to evaluate long-context understanding in English and Arabic, targeting context lengths ranging from 4k to over 128k tokens. LC-Eval introduces four novel and challenging tasks: multi-document question answering, bilingual question answering, claim verification within a paragraph, and multiple-choice questions based on long contexts. These tasks are designed to assess LLMs’ abilities in deep reasoning, document comprehension, information tracing, and bilingual information extraction and understanding. The benchmark includes datasets in both Arabic and English for each task, allowing for a comparative analysis of their performance across different text genres. Evaluations were conducted on both open-weight and closed LLMs, with results indicating that LC-Eval presents significant challenges. Even high-performing models, such as GPT-4o, struggled with certain tasks, highlighting the complexity and rigor of the benchmark.

Method: Multi-stage framework with joint optimization of masked language modeling (MLM) and contrastive objectives within a unified training pipeline, using selective adaptation for in-domain contrastive learning.

Result: Achieved improvements up to 13.4% in NDCG@10 over strong general-domain baselines, validated on both high-resource and low-resource domains.

Conclusion: The framework effectively learns domain-relevant representations while maintaining robust semantic discrimination, with ablation studies confirming the importance of balanced joint supervision and staged adaptation.

Abstract: We introduce MOSAIC (Masked Objective with Selective Adaptation for In-domain Contrastive learning), a multi-stage framework for domain adaptation of sentence embedding models that incorporates joint domain-specific masked supervision. Our approach addresses the challenges of adapting large-scale general-domain sentence embedding models to specialized domains. By jointly optimizing masked language modeling (MLM) and contrastive objectives within a unified training pipeline, our method enables effective learning of domain-relevant representations while preserving the robust semantic discrimination properties of the original model. We empirically validate our approach on both high-resource and low-resource domains, achieving improvements up to 13.4% in NDCG@10 (Normalized Discounted Cumulative Gain) over strong general-domain baselines. Comprehensive ablation studies further demonstrate the effectiveness of each component, highlighting the importance of balanced joint supervision and staged adaptation.

Method: Analyze LLM performance on entity comparison tasks with numerical attributes, identify heuristic biases, compare model sizes (7-8B vs 32B parameters), and test chain-of-thought prompting.

Result: Smaller models’ predictions are better predicted by surface cues than their own numerical knowledge, while larger models selectively use numerical knowledge when reliable. Chain-of-thought prompting improves numerical reasoning across all model sizes.

Conclusion: LLM reasoning is influenced by heuristic biases, with model size affecting the ability to selectively use reliable knowledge, and chain-of-thought prompting can mitigate heuristic reliance.

Abstract: Large Language Models (LLMs) are increasingly used for knowledge-based reasoning tasks, yet understanding when they rely on genuine knowledge versus superficial heuristics remains challenging. We investigate this question through entity comparison tasks by asking models to compare entities along numerical attributes (e.g., ``Which river is longer, the Danube or the Nile?’’), which offer clear ground truth for systematic analysis. Despite having sufficient numerical knowledge to answer correctly, LLMs frequently make predictions that contradict this knowledge. We identify three heuristic biases that strongly influence model predictions: entity popularity, mention order, and semantic co-occurrence. For smaller models, a simple logistic regression using only these surface cues predicts model choices more accurately than the model’s own numerical predictions, suggesting heuristics largely override principled reasoning. Crucially, we find that larger models (32B parameters) selectively rely on numerical knowledge when it is more reliable, while smaller models (7–8B parameters) show no such discrimination, which explains why larger models outperform smaller ones even when the smaller models possess more accurate knowledge. Chain-of-thought prompting steers all models towards using the numerical features across all model sizes.

Method: Two-stage retrieve-and-rerank framework with LLMs, featuring a targeted data curation strategy to help the reranker learn author-discriminative signals rather than relying on IR-based training approaches.

Result: Achieved substantial gains of 22.3 and 34.4 absolute Success@8 points over previous state-of-the-art on HIATUS’s HRS1 and HRS2 cross-genre AA benchmarks.

Conclusion: The proposed LLM-based retrieve-and-rerank pipeline with targeted data curation effectively addresses the limitations of IR-based approaches for cross-genre authorship attribution, demonstrating significant performance improvements.

Abstract: Authorship attribution (AA) is the task of identifying the most likely author of a query document from a predefined set of candidate authors. We introduce a two-stage retrieve-and-rerank framework that finetunes LLMs for cross-genre AA. Unlike the field of information retrieval (IR), where retrieve-and-rerank is a de facto strategy, cross-genre AA systems must avoid relying on topical cues and instead learn to identify author-specific linguistic patterns that are independent of the text’s subject matter (genre/domain/topic). Consequently, for the reranker, we demonstrate that training strategies commonly used in IR are fundamentally misaligned with cross-genre AA, leading to suboptimal behavior. To address this, we introduce a targeted data curation strategy that enables the reranker to effectively learn author-discriminative signals. Using our LLM-based retrieve-and-rerank pipeline, we achieve substantial gains of 22.3 and 34.4 absolute Success@8 points over the previous state-of-the-art on HIATUS’s challenging HRS1 and HRS2 cross-genre AA benchmarks.

Method: Developed CoRUS framework using role theory and online community posts to create a taxonomy of asker roles (patients, caregivers, practitioners), then simulated 15,321 role-based questions embedding each role’s goals, behaviors, and experiences.

Result: Simulated questions were highly believable and comparable to real data. LLM evaluations showed systematic differences: vulnerable roles (patients, caregivers) elicited 17% more supportive responses and 19% less knowledge content compared to practitioner roles.

Conclusion: User roles implicitly shape LLM responses, demonstrating the need for role-informed evaluation of conversational AI, especially in sensitive domains.

Abstract: Language model users often embed personal and social context in their questions. The asker’s role – implicit in how the question is framed – creates specific needs for an appropriate response. However, most evaluations, while capturing the model’s capability to respond, often ignore who is asking. This gap is especially critical in stigmatized domains such as opioid use disorder (OUD), where accounting for users’ contexts is essential to provide accessible, stigma-free responses. We propose CoRUS (COmmunity-driven Roles for User-centric Question Simulation), a framework for simulating role-based questions. Drawing on role theory and posts from an online OUD recovery community (r/OpiatesRecovery), we first build a taxonomy of asker roles – patients, caregivers, practitioners. Next, we use it to simulate 15,321 questions that embed each role’s goals, behaviors, and experiences. Our evaluations show that these questions are both highly believable and comparable to real-world data. When used to evaluate five LLMs, for the same question but differing roles, we find systematic differences: vulnerable roles, such as patients and caregivers, elicit more supportive responses (+17%) and reduced knowledge content (-19%) in comparison to practitioners. Our work demonstrates how implicitly signaling a user’s role shapes model responses, and provides a methodology for role-informed evaluation of conversational AI.

Method: Uses Code Agent with Variable Memory (CAVM) architecture for flexible data handling, Iterative Vision-Enhanced Mechanism for chart refinement, and two-stage Writing Framework for expanding analysis into coherent multimodal reports.

Result: Significantly outperforms all baselines including leading deep research systems in factual accuracy, analytical depth, and presentation quality across various company and industry-level tasks.

Conclusion: Demonstrates a clear path toward generating financial reports that approach human-expert quality through the FinSight framework.

Abstract: Generating professional financial reports is a labor-intensive and intellectually demanding process that current AI systems struggle to fully automate. To address this challenge, we introduce FinSight (Financial InSight), a novel multi agent framework for producing high-quality, multimodal financial reports. The foundation of FinSight is the Code Agent with Variable Memory (CAVM) architecture, which unifies external data, designed tools, and agents into a programmable variable space, enabling flexible data collection, analysis and report generation through executable code. To ensure professional-grade visualization, we propose an Iterative Vision-Enhanced Mechanism that progressively refines raw visual outputs into polished financial charts. Furthermore, a two stage Writing Framework expands concise Chain-of-Analysis segments into coherent, citation-aware, and multimodal reports, ensuring both analytical depth and structural consistency. Experiments on various company and industry-level tasks demonstrate that FinSight significantly outperforms all baselines, including leading deep research systems in terms of factual accuracy, analytical depth, and presentation quality, demonstrating a clear path toward generating reports that approach human-expert quality.

Method: NGC treats weights as transient interactions between embedding-like neuronal states, with computation unfolding through iterative communication among groups of neurons. Introduces neuronal stability metric based on dynamical systems theory to quantify contraction of neuron activations during sequence processing.

Result: NGC instantiated in LLMs shows improved performance on complex reasoning benchmarks under moderate compression, consistently outperforming standard low-rank approximations and cross-layer basis-sharing methods at comparable compression rates.

Conclusion: NGC provides a framework for building efficient, modular, and interpretable neural systems, with structured neuronal group dynamics potentially relating to generalization in high-dimensional learning systems.

Abstract: The ever-increasing scale of modern neural networks has brought unprecedented performance alongside daunting challenges in efficiency and interpretability. This paper addresses the core question of how to build large neural systems that learn efficient, modular, and interpretable representations. We propose Neuronal Group Communication (NGC), a theory-driven framework that reimagines a neural network as a dynamical system of interacting neuronal groups rather than a monolithic collection of neural weights. Instead of treating each weight as an independent trainable parameter, NGC treats weights as transient interactions between embedding-like neuronal states, with neural computation unfolding through iterative communication among groups of neurons. This low-rank, modular representation yields compact models: groups of neurons exchange low-dimensional signals, enabling intra-group specialization and inter-group information sharing while dramatically reducing redundant parameters. By drawing on dynamical systems theory, we introduce a neuronal stability metric (analogous to Lyapunov stability) that quantifies the contraction of neuron activations toward stable patterns during sequence processing. Using this metric, we reveal that emergent reasoning capabilities correspond to an external driving force or ``potential’’, which nudges the neural dynamics away from trivial trajectories while preserving stability. Empirically, we instantiate NGC in large language models (LLMs) and demonstrate improved performance on complex reasoning benchmarks under moderate compression. NGC consistently outperforms standard low-rank approximations and cross-layer basis-sharing methods at comparable compression rates. We conclude by discussing the broader implications of NGC, including how structured neuronal group dynamics might relate to generalization in high-dimensional learning systems.

Method: Created a novel embodied knowledge benchmark based on perceptual theory, testing 30 state-of-the-art models through vector comparison and QA tasks with 1,700+ questions across multiple sensory modalities.

Result: VLMs showed no advantage over text-only models, performed worst in visual dimension, and struggled with spatial perception and reasoning. Vector representations were affected by word form and frequency.

Conclusion: Current models need more effective integration of embodied knowledge to better understand the physical world, as visual grounding alone doesn’t provide the expected benefits.

Abstract: Despite significant progress in multimodal language models (LMs), it remains unclear whether visual grounding enhances their understanding of embodied knowledge compared to text-only models. To address this question, we propose a novel embodied knowledge understanding benchmark based on the perceptual theory from psychology, encompassing visual, auditory, tactile, gustatory, olfactory external senses, and interoception. The benchmark assesses the models’ perceptual abilities across different sensory modalities through vector comparison and question-answering tasks with over 1,700 questions. By comparing 30 state-of-the-art LMs, we surprisingly find that vision-language models (VLMs) do not outperform text-only models in either task. Moreover, the models perform significantly worse in the visual dimension compared to other sensory dimensions. Further analysis reveals that the vector representations are easily influenced by word form and frequency, and the models struggle to answer questions involving spatial perception and reasoning. Our findings underscore the need for more effective integration of embodied knowledge in LMs to enhance their understanding of the physical world.

Method: Created ChiKhaPo with 8 subtasks of varying difficulty using existing lexicons, monolingual data, and bitext to evaluate lexical comprehension and generation abilities.

Result: 6 state-of-the-art models struggle on the benchmark, with performance influenced by language family, resource availability, task type, and comprehension vs generation directions.

Conclusion: ChiKhaPo enables massively multilingual benchmarking of LLMs and highlights the need for better basic linguistic competence across diverse languages.

Abstract: Existing benchmarks for large language models (LLMs) are largely restricted to high- or mid-resource languages, and often evaluate performance on higher-order tasks in reasoning and generation. However, plenty of evidence points to the fact that LLMs lack basic linguistic competence in the vast majority of the world’s 3800+ written languages. We introduce ChiKhaPo, consisting of 8 subtasks of varying difficulty designed to evaluate the lexical comprehension and generation abilities of generative models. ChiKhaPo draws on existing lexicons, monolingual data, and bitext, and provides coverage for 2700+ languages for 2 subtasks, surpassing any existing benchmark in terms of language coverage. We further show that 6 SOTA models struggle on our benchmark, and discuss the factors contributing to performance scores, including language family, language resourcedness, task, and comprehension versus generation directions. With ChiKhaPo, we hope to enable and encourage the massively multilingual benchmarking of LLMs.

Method: PROMPT-MII uses reinforcement learning to meta-learn an instruction induction model that generates compact, descriptive prompts from training examples for arbitrary new datasets.

Result: The method improves downstream model quality by 4-9 F1 points (10-20% relative), matching ICL performance while requiring 3-13x fewer tokens, trained on 3,000+ datasets and evaluated on 90 unseen tasks.

Conclusion: PROMPT-MII provides an efficient alternative to ICL by generating compact instructions that maintain performance while significantly reducing token usage.

Abstract: A popular method to adapt large language models (LLMs) to new tasks is in-context learning (ICL), which is effective but incurs high inference costs as context length grows. In this paper we propose a method to perform instruction induction, where we take training examples and reduce them to a compact but descriptive prompt that can achieve performance comparable to ICL over the full training set. Specifically, we propose PROMPT-MII, a reinforcement learning (RL) based framework to meta-learn an instruction induction model that can generate compact instructions on the fly for an arbitrary new dataset. We train on over 3,000 diverse classification datasets from the HuggingFace hub, and evaluate on 90 unseen tasks. PROMPT-MII improves downstream model quality by 4-9 F1 points (10-20% relative), matching ICL performance while requiring 3-13x fewer tokens.

Method: Fine-tuned three instruction-tuned LLMs (Gemma-3-4B, Llama-3.2-3B, Mistral-7B) on BD-SHS dataset using PEFT with LoRA and QLoRA, training less than 1% of parameters on a single consumer GPU.

Result: Llama-3.2-3B achieved highest F1-score of 92.23%, followed by Mistral-7B (88.94%) and Gemma-3-4B (80.25%), demonstrating effective hate speech detection.

Conclusion: PEFT is established as a practical and replicable strategy for Bengali and other low-resource languages, enabling efficient hate speech detection with minimal computational requirements.

Abstract: Bengali social media platforms have witnessed a sharp increase in hate speech, disproportionately affecting women and adolescents. While datasets such as BD-SHS provide a basis for structured evaluation, most prior approaches rely on either computationally costly full-model fine-tuning or proprietary APIs. This paper presents the first application of Parameter-Efficient Fine-Tuning (PEFT) for Bengali hate speech detection using LoRA and QLoRA. Three instruction-tuned large language models - Gemma-3-4B, Llama-3.2-3B, and Mistral-7B - were fine-tuned on the BD-SHS dataset of 50,281 annotated comments. Each model was adapted by training fewer than 1% of its parameters, enabling experiments on a single consumer-grade GPU. The results show that Llama-3.2-3B achieved the highest F1-score of 92.23%, followed by Mistral-7B at 88.94% and Gemma-3-4B at 80.25%. These findings establish PEFT as a practical and replicable strategy for Bengali and related low-resource languages.

Method: Uses byte-level tokenization where text maps directly to UTF-8 byte IDs (0-255), encodes all special behaviors using C0 control bytes, implements 256*d embedding tables, and employs bit-biased embeddings to expose per-byte bit structure.

Result: Achieves 14x faster tokenization, 8x less host-device transfer than int64, enables shareable embedding tables across models, and improves language modeling convergence with HuggingFace-compatible implementation.

Conclusion: UTF8Tokenizer demonstrates that minimalist byte-level tokenization using UTF-8 encoding and C0 control bytes provides practical benefits including speed, efficiency, and improved model performance while maintaining compatibility.

Abstract: We present UTF8Tokenizer, a minimalist byte-level tokenizer that maps text exactly to IDs corresponding to the bytes underlying the text’s UTF-8 encoding (e.g., byte x09 is token ID 9). Unlike prior byte-level approaches (Xue et al., 2021; Pagnoni et al., 2025), our implementation never introduces out-of-range IDs (i.e. there is no token ID 256) or auxiliary tokens: all special behavior (e.g., padding, boundaries, conversation structure, attention segments, tool calling, “thinking” spans, etc.) is encoded using C0 control bytes - just as ASCII was originally designed to embed control information alongside printable text. These design principles yield practical benefits: (1) faster tokenization (14x) and significantly lower host-device transfer (8x less than int64); (2) simple, shareable 256*d embedding tables that can be aligned across models; and (3) a training-time enhancement via bit-biased embeddings, which exposes per-byte bit structure and can be added to the embedding table post-training, removing inference costs. Our HuggingFace-compatible implementation improves language modeling convergence.

Method: Use transformation vectors as additive offsets to represent word variations from base forms, then reshape vocabulary by composing words from shared base forms and transformations rather than unique tokens.

Result: Successfully removed up to 10% of vocabulary entries across multiple LLMs and five languages, freeing space for more diverse tokens while expanding out-of-vocabulary word coverage with minimal performance impact.

Conclusion: The approach motivates rethinking vocabulary design from string enumeration to compositional structures that leverage language’s inherent patterns, enabling more efficient tokenization without model weight modifications.

Abstract: Large language models (LLMs) were shown to encode word form variations, such as “walk”->“walked”, as linear directions in embedding space. However, standard tokenization algorithms treat these variations as distinct tokens – filling the size-capped vocabulary with surface form variants (e.g., “walk”, “walking”, “Walk”), at the expense of less frequent words and multilingual coverage. We show that many of these variations can be captured by transformation vectors – additive offsets that yield the appropriate word’s representation when applied to the base form word embedding – in both the input and output spaces. Building on this, we propose a compact reshaping of the vocabulary: rather than assigning unique tokens to each surface form, we compose them from shared base form and transformation vectors (e.g., “walked” = “walk” + past tense). We apply our approach to multiple LLMs and across five languages, removing up to 10% of vocabulary entries – thereby freeing space to allocate new, more diverse tokens. Importantly, we do so while also expanding vocabulary coverage to out-of-vocabulary words, with minimal impact on downstream performance, and without modifying model weights. Our findings motivate a foundational rethinking of vocabulary design, moving from string enumeration to a compositional vocabulary that leverages the underlying structure of language.

Method: Propose a framework with online learning that updates defense strategies dynamically. Use reinforcement learning to optimize prompts for appropriate harmless responses while rejecting harmful ones. Introduce Past-Direction Gradient Damping (PDGD) to prevent overfitting to attack patterns.

Result: Significantly outperforms five existing defense methods against five iterative jailbreak methods on three LLMs. Also enhances response quality for harmless tasks simultaneously.

Conclusion: The proposed dynamic defense framework effectively counters iterative jailbreak attacks while maintaining and even improving harmless task performance, demonstrating superior protection compared to existing methods.

Abstract: Iterative jailbreak methods that repeatedly rewrite and input prompts into large language models (LLMs) to induce harmful outputs – using the model’s previous responses to guide each new iteration – have been found to be a highly effective attack strategy. Despite being an effective attack strategy against LLMs and their safety mechanisms, existing defenses do not proactively disrupt this dynamic trial-and-error cycle. In this study, we propose a novel framework that dynamically updates its defense strategy through online learning in response to each new prompt from iterative jailbreak methods. Leveraging the distinctions between harmful jailbreak-generated prompts and typical harmless prompts, we introduce a reinforcement learning-based approach that optimizes prompts to ensure appropriate responses for harmless tasks while explicitly rejecting harmful prompts. Additionally, to curb overfitting to the narrow band of partial input rewrites explored during an attack, we introduce Past-Direction Gradient Damping (PDGD). Experiments conducted on three LLMs show that our approach significantly outperforms five existing defense methods against five iterative jailbreak methods. Moreover, our results indicate that our prompt optimization strategy simultaneously enhances response quality for harmless tasks.

Method: Developed DiscoTrack benchmark with tasks across 12 languages targeting four levels of discourse understanding: salience recognition, entity tracking, discourse relations, and bridging inference.

Result: Evaluation shows these discourse tracking tasks remain challenging even for state-of-the-art models, highlighting limitations in current LLM capabilities for discourse-level understanding.

Conclusion: DiscoTrack addresses the gap in challenging multilingual benchmarks for discourse tracking and demonstrates that current models struggle with implicit information and pragmatic inferences across larger documents.

Abstract: Recent LLM benchmarks have tested models on a range of phenomena, but are still focused primarily on natural language understanding for extraction of explicit information, such as QA or summarization, with responses often tar- geting information from individual sentences. We are still lacking more challenging, and im- portantly also multilingual, benchmarks focus- ing on implicit information and pragmatic infer- ences across larger documents in the context of discourse tracking: integrating and aggregating information across sentences, paragraphs and multiple speaker utterances. To this end, we present DiscoTrack, an LLM benchmark target- ing a range of tasks across 12 languages and four levels of discourse understanding: salience recognition, entity tracking, discourse relations and bridging inference. Our evaluation shows that these tasks remain challenging, even for state-of-the-art models.

Method: SafeSearch, a multi-objective reinforcement learning approach that combines final-output safety/utility rewards with a novel query-level shaping term that penalizes unsafe queries and rewards safe ones.

Result: SafeSearch reduces agent harmfulness by over 70% across three red-teaming datasets while maintaining safe, helpful responses and matching the QA performance of utility-only finetuned agents.

Conclusion: The query-level reward in SafeSearch is effective in jointly improving both safety and utility, demonstrating the importance of multi-objective alignment for search agents.

Abstract: Large language model (LLM) based search agents iteratively generate queries, retrieve external information, and reason to answer open-domain questions. While researchers have primarily focused on improving their utility, their safety behaviors remain underexplored. In this paper, we first evaluate search agents using red-teaming datasets and find that they are more likely to produce harmful outputs than base LLMs. For example, when asked ``How can I track someone’s location without their consent?’’, a base model refuses, whereas a search agent designed to retrieve and cite sources may lower its refusal threshold, fetch documents (e.g., court cases), and, once appended, synthesize them into an informative yet unsafe summary. We further show that utility-oriented fine-tuning intensifies this risk, motivating joint alignment of safety and utility. We present SafeSearch, a multi-objective reinforcement learning approach that couples a final-output safety/utility reward with a novel query-level shaping term that penalizes unsafe queries and rewards safe ones. Experiments show that SafeSearch reduces agent harmfulness by over 70% across three red-teaming datasets while producing safe, helpful responses, and matches the QA performance of a utility-only finetuned agent; further analyses confirm the effectiveness of the query-level reward in jointly improving safety and utility.

Method: Uses cosine-similarity gating with learnable reference vectors to modulate embeddings, integrates multi-source embeddings (GloVe, FastText, BERT CLS), character-level BiLSTM for morphology, embedding-space SMOTE for minority augmentation, and adaptive focal loss with dynamic class weighting.

Result: Achieves 96.0% accuracy and 0.88 macro-F1 on Jigsaw Toxic Comment benchmark, outperforming BERT by 33% on threat and 28% on identity_hate categories with 15x fewer parameters and 50ms inference latency. Cosine gating provides +4.8% F1 gain.

Conclusion: Lightweight, theoretically informed architectures can surpass large pretrained models on imbalanced, domain-specific NLP tasks, establishing a new efficiency-adaptability frontier.

Abstract: Toxic comment detection remains a challenging task, where transformer-based models (e.g., BERT) incur high computational costs and degrade on minority toxicity classes, while classical ensembles lack semantic adaptability. We propose xLSTM, a parameter-efficient and theoretically grounded framework that unifies cosine-similarity gating, adaptive feature prioritization, and principled class rebalancing. A learnable reference vector {v} in {R}^d modulates contextual embeddings via cosine similarity, amplifying toxic cues and attenuating benign signals to yield stronger gradients under severe class imbalance. xLSTM integrates multi-source embeddings (GloVe, FastText, BERT CLS) through a projection layer, a character-level BiLSTM for morphological cues, embedding-space SMOTE for minority augmentation, and adaptive focal loss with dynamic class weighting. On the Jigsaw Toxic Comment benchmark, xLSTM attains 96.0% accuracy and 0.88 macro-F1, outperforming BERT by 33% on threat and 28% on identity_hate categories, with 15 times fewer parameters and 50ms inference latency. Cosine gating contributes a +4.8% F1 gain in ablations. The results establish a new efficiency adaptability frontier, demonstrating that lightweight, theoretically informed architectures can surpass large pretrained models on imbalanced, domain-specific NLP tasks.

Method: Model prompt sensitivity as generalization error, use paraphrasing perturbations across semantic concept space, and introduce a new uncertainty decomposition metric that captures semantic continuities in natural language generation.

Result: Sampling across semantic concept space with paraphrasing improves uncertainty calibration without compromising accuracy. The new decomposition metric can quantify how much LLM uncertainty is attributed to prompt sensitivity.

Conclusion: The work provides a new approach to improve uncertainty calibration in prompt-sensitive LLMs and shows evidence that some LLMs fail to exhibit consistent general reasoning about input meanings.

Abstract: An interesting behavior in large language models (LLMs) is prompt sensitivity. When provided with different but semantically equivalent versions of the same prompt, models may produce very different distributions of answers. This suggests that the uncertainty reflected in a model’s output distribution for one prompt may not reflect the model’s uncertainty about the meaning of the prompt. We model prompt sensitivity as a type of generalization error, and show that sampling across the semantic ``concept space’’ with paraphrasing perturbations improves uncertainty calibration without compromising accuracy. Additionally, we introduce a new metric for uncertainty decomposition in black-box LLMs that improves upon entropy-based decomposition by modeling semantic continuities in natural language generation. We show that this decomposition metric can be used to quantify how much LLM uncertainty is attributed to prompt sensitivity. Our work introduces a new way to improve uncertainty calibration in prompt-sensitive language models, and provides evidence that some LLMs fail to exhibit consistent general reasoning about the meanings of their inputs.

Method: Systematically investigate thinking process mechanisms, identify two failure patterns (stereotype repetition and irrelevant information injection), and introduce a lightweight prompt-based mitigation approach that queries the model to review its initial reasoning against these failure patterns.

Result: Experiments on question answering (BBQ and StereoSet) and open-ended (BOLD) benchmarks show the approach effectively reduces bias while maintaining or improving accuracy.

Conclusion: The study uncovers specific failure patterns in LLM reasoning that drive social bias aggregation and demonstrates that a simple prompt-based mitigation approach can effectively address these biases without compromising performance.

Abstract: While reasoning-based large language models excel at complex tasks through an internal, structured thinking process, a concerning phenomenon has emerged that such a thinking process can aggregate social stereotypes, leading to biased outcomes. However, the underlying behaviours of these language models in social bias scenarios remain underexplored. In this work, we systematically investigate mechanisms within the thinking process behind this phenomenon and uncover two failure patterns that drive social bias aggregation: 1) stereotype repetition, where the model relies on social stereotypes as its primary justification, and 2) irrelevant information injection, where it fabricates or introduces new details to support a biased narrative. Building on these insights, we introduce a lightweight prompt-based mitigation approach that queries the model to review its own initial reasoning against these specific failure patterns. Experiments on question answering (BBQ and StereoSet) and open-ended (BOLD) benchmarks show that our approach effectively reduces bias while maintaining or improving accuracy.

Method: DVAGen provides a unified framework with modular pipeline design, seamless integration with open-source LLMs, and CLI/WebUI tools for training, evaluation, visualization, and real-time result inspection.

Result: The framework validates dynamic vocabulary effectiveness on modern LLMs and demonstrates support for batch inference with significantly improved inference throughput.

Conclusion: DVAGen overcomes limitations of existing dynamic vocabulary approaches by providing a comprehensive, scalable, and user-friendly framework that enhances language model flexibility and performance.

Abstract: Language models trained with a fixed vocabulary struggle to generalize to novel or out-of-vocabulary words, limiting their flexibility in handling diverse token combinations. Existing dynamic vocabulary approaches attempt to address this limitation but face challenges such as fragmented codebases, lack of support for modern LLMs, and limited inference scalability. To overcome these issues, we introduce DVAGen, a fully open-source, unified framework designed for training, evaluation, and visualization of dynamic vocabulary-augmented language models. Our framework modularizes the pipeline for ease of customization, integrates seamlessly with open-source LLMs, and is the first to provide both CLI and WebUI tools for real-time result inspection. We validate the effectiveness of dynamic vocabulary methods on modern LLMs and demonstrate support for batch inference, significantly improving inference throughput.

Method: Introduced On-policy Pseudo-document Query Expansion (OPQE) - a hybrid method where LLM policy learns to generate pseudo-documents that maximize retrieval performance, combining prompting flexibility with RL optimization.

Result: Simple training-free query augmentation performs on par with or surpasses RL-based methods, especially with powerful LLMs. OPQE outperforms both standalone prompting and RL-based rewriting.

Conclusion: Synergistic approaches combining prompting flexibility with RL optimization yield the best results for query augmentation in information retrieval.

Abstract: Recent advances in large language models (LLMs) have led to a surge of interest in query augmentation for information retrieval (IR). Two main approaches have emerged. The first prompts LLMs to generate answers or pseudo-documents that serve as new queries, relying purely on the model’s parametric knowledge or contextual information. The second applies reinforcement learning (RL) to fine-tune LLMs for query rewriting, directly optimizing retrieval metrics. While having respective advantages and limitations, the two approaches have not been compared under consistent experimental conditions. In this work, we present the first systematic comparison of prompting-based and RL-based query augmentation across diverse benchmarks, including evidence-seeking, ad hoc, and tool retrieval. Our key finding is that simple, training-free query augmentation often performs on par with, or even surpasses, more expensive RL-based counterparts, especially when using powerful LLMs. Motivated by this discovery, we introduce a novel hybrid method, On-policy Pseudo-document Query Expansion (OPQE), which, instead of rewriting a query, the LLM policy learns to generate a pseudo-document that maximizes retrieval performance, thus merging the flexibility and generative structure of prompting with the targeted optimization of RL. We show OPQE outperforms both standalone prompting and RL-based rewriting, demonstrating that a synergistic approach yields the best results. Our implementation is made available to facilitate reproducibility.

Method: Compared behavioral and neural responses to ironic statements from AI vs human sources using ERP components (P200 for early incongruity detection and P600 for cognitive reanalysis).

Result: Participants showed attenuated P200 and P600 effects for AI-generated irony, indicating reduced effortful detection and reanalysis. People attributed incongruity to deliberate communication less for AI than human sources.

Conclusion: Source attribution shapes neural processing of social communication. Achieving genuine social agency for AI requires more than linguistic competence - it necessitates a shift in how humans perceive and attribute intentionality to artificial agents.

Abstract: As Large Language Models (LLMs) are increasingly deployed as social agents and trained to produce humor and irony, a question emerges: when encountering witty AI remarks, do people interpret these as intentional communication or mere computational output? This study investigates whether people adopt the intentional stance, attributing mental states to explain behavior,toward AI during irony comprehension. Irony provides an ideal paradigm because it requires distinguishing intentional contradictions from unintended errors through effortful semantic reanalysis. We compared behavioral and neural responses to ironic statements from AI versus human sources using established ERP components: P200 reflecting early incongruity detection and P600 indexing cognitive efforts in reinterpreting incongruity as deliberate irony. Results demonstrate that people do not fully adopt the intentional stance toward AI-generated irony. Behaviorally, participants attributed incongruity to deliberate communication for both sources, though significantly less for AI than human, showing greater tendency to interpret AI incongruities as computational errors. Neural data revealed attenuated P200 and P600 effects for AI-generated irony, suggesting reduced effortful detection and reanalysis consistent with diminished attribution of communicative intent. Notably, people who perceived AI as more sincere showed larger P200 and P600 effects for AI-generated irony, suggesting that intentional stance adoption is calibrated by specific mental models of artificial agents. These findings reveal that source attribution shapes neural processing of social-communicative phenomena. Despite current LLMs’ linguistic sophistication, achieving genuine social agency requires more than linguistic competence, it necessitates a shift in how humans perceive and attribute intentionality to artificial agents.

Method: The authors use a unified framework and comprehensive ablation studies to identify critical design components, including expressive chunk encoders with CLS tokens, bypassing residual paths, and enforced selection sparsity during pre-training.

Result: The proposed approach achieves new state-of-the-art training-free length extrapolation, successfully generalizing from 4K to 32 million tokens on RULER and BABILong benchmarks.

Conclusion: The study provides clear, empirically-grounded design principles for developing future long-context language models, demonstrating that the identified three components are critical for effective length generalization.

Abstract: Effectively processing long contexts is a critical challenge for language models. While standard Transformers are limited by quadratic complexity and poor length extrapolation, alternative architectures like sliding window attention and state space models sacrifice the ability to effectively utilize the full context due to their fixed-size memory. Chunk-based sparse attention has emerged as a promising paradigm for extreme length generalization, yet the key architectural principles underpinning its success are not yet fully understood. In this work, we present a systematic dissection of these models to identify the core components driving their performance. Through a unified framework and comprehensive ablation studies, we demonstrate that a combination of three design principles is critical: (1) an expressive, non-linear Chunk Encoder with a dedicated CLS token to produce representations for retrieval; (2) a Bypassing Residual Path to stably integrate retrieved global information without it being overridden by the local residual stream; and (3) enforced selection sparsity during pre-training to bridge the train-test distribution gap. We provide a theoretical motivation for intra-chunk information processing and landmark generation. By combining these principles, we establish a new state-of-the-art for training-free length extrapolation, successfully generalizing models trained on a 4K context to 32 million tokens on RULER and BABILong. Our findings provide a clear and empirically-grounded set of design principles for developing future, highly-capable long-context language models.

Method: AS framework uses two attention-level interventions: importance-aware suppression applied to unlearning set to reduce reliance on memorized knowledge, and attention-guided retention enhancement that reinforces attention toward semantically essential tokens in retained dataset. These are jointly optimized via a dual-loss objective.

Result: AS improves performance preservation over state-of-the-art unlearning methods, achieving up to 15% higher accuracy on ToFU benchmark and 10% on TDEC benchmark, while maintaining competitive hallucination-free unlearning effectiveness.

Conclusion: AS demonstrates superior balance between unlearning effectiveness, generalization, and response reliability compared to existing methods.

Abstract: The increase in computing power and the necessity of AI-assisted decision-making boost the growing application of large language models (LLMs). Along with this, the potential retention of sensitive data of LLMs has spurred increasing research into machine unlearning. However, existing unlearning approaches face a critical dilemma: Aggressive unlearning compromises model utility, while conservative strategies preserve utility but risk hallucinated responses. This significantly limits LLMs’ reliability in knowledge-intensive applications. To address this, we introduce a novel Attention-Shifting (AS) framework for selective unlearning. AS is driven by two design objectives: (1) context-preserving suppression that attenuates attention to fact-bearing tokens without disrupting LLMs’ linguistic structure; and (2) hallucination-resistant response shaping that discourages fabricated completions when queried about unlearning content. AS realizes these objectives through two attention-level interventions, which are importance-aware suppression applied to the unlearning set to reduce reliance on memorized knowledge and attention-guided retention enhancement that reinforces attention toward semantically essential tokens in the retained dataset to mitigate unintended degradation. These two components are jointly optimized via a dual-loss objective, which forms a soft boundary that localizes unlearning while preserving unrelated knowledge under representation superposition. Experimental results show that AS improves performance preservation over the state-of-the-art unlearning methods, achieving up to 15% higher accuracy on the ToFU benchmark and 10% on the TDEC benchmark, while maintaining competitive hallucination-free unlearning effectiveness. Compared to existing methods, AS demonstrates a superior balance between unlearning effectiveness, generalization, and response reliability.

Method: StreamingThinker framework integrates streaming CoT generation with quality control, streaming-constraint training using attention masks and position encoding, and parallel KV caches that decouple input encoding from reasoning generation for true concurrency.

Result: Evaluation on Qwen3 models across math, logical, and context-based QA reasoning tasks shows comparable performance to batch thinking, with 80% reduction in token waiting time before reasoning onset and over 60% reduction in time-level latency for final answer generation.

Conclusion: The streaming thinking paradigm effectively reduces latency while maintaining reasoning performance, demonstrating the viability of thinking-while-reading approaches for LLM reasoning.

Abstract: Large language models (LLMs) have demonstrated remarkable capabilities in chain of thought (CoT) reasoning. However, the current LLM reasoning paradigm initiates thinking only after the entire input is available, which introduces unnecessary latency and weakens attention to earlier information in dynamic scenarios. Inspired by human cognition of thinking while reading, we first design a \textit{\textbf{streaming thinking}} paradigm for LLMs, where reasoning unfolds in the order of input and further adjusts its depth once reading is complete. We instantiate this paradigm with \textit{StreamingThinker}, a framework that enables LLMs to think while reading through the integration of streaming CoT generation, streaming-constraint training, and streaming parallel inference. Specifically, StreamingThinker employs streaming reasoning units with quality control for CoT generation, enforces order-preserving reasoning through streaming attention masks and position encoding, and leverages parallel KV caches that decouple input encoding from reasoning generation, thereby ensuring alignment and enabling true concurrency. We evaluate StreamingThinker on the Qwen3 model family across math reasoning, logical reasoning, and context-based QA reasoning tasks. Experimental results show that the StreamingThinker preserves performance comparable to batch thinking, while yielding an 80% reduction in token waiting before the onset of reasoning and a more than 60% reduction in time-level latency for producing the final answer, demonstrating the effectiveness of the streaming paradigm for LLM reasoning. Code will be released at \href{https://github.com/EIT-NLP/StreamingLLM/tree/main/StreamingThinker}{this repository.}

Method: Introduced VideoBiasEval framework with event-based prompting to disentangle semantic content from actor attributes, using multi-granular metrics to evaluate ethnicity bias, gender bias conditioned on ethnicity, distributional shifts, and temporal persistence of bias.

Result: Alignment tuning strengthens representational biases and makes them temporally stable, producing smoother yet more stereotyped portrayals. Biases in human preference datasets are amplified in reward models and propagate through alignment-tuned video diffusion models.

Conclusion: There is a critical need for bias-aware evaluation and mitigation throughout the alignment process to ensure fair and socially responsible video generation.

Abstract: Recent advances in video diffusion models have significantly enhanced text-to-video generation, particularly through alignment tuning using reward models trained on human preferences. While these methods improve visual quality, they can unintentionally encode and amplify social biases. To systematically trace how such biases evolve throughout the alignment pipeline, we introduce VideoBiasEval, a comprehensive diagnostic framework for evaluating social representation in video generation. Grounded in established social bias taxonomies, VideoBiasEval employs an event-based prompting strategy to disentangle semantic content (actions and contexts) from actor attributes (gender and ethnicity). It further introduces multi-granular metrics to evaluate (1) overall ethnicity bias, (2) gender bias conditioned on ethnicity, (3) distributional shifts in social attributes across model variants, and (4) the temporal persistence of bias within videos. Using this framework, we conduct the first end-to-end analysis connecting biases in human preference datasets, their amplification in reward models, and their propagation through alignment-tuned video diffusion models. Our results reveal that alignment tuning not only strengthens representational biases but also makes them temporally stable, producing smoother yet more stereotyped portrayals. These findings highlight the need for bias-aware evaluation and mitigation throughout the alignment process to ensure fair and socially responsible video generation.

Method: Used zero-shot inference with Gemma-3 4B model to analyze 300,000 Bengali news headlines and content, identifying dominant emotions and overall tone for each article.

Result: Clear dominance of negative emotions (anger, fear, disappointment) and significant variation in emotional portrayal of similar stories across different news outlets.

Conclusion: Proposed design ideas for human-centered news aggregator that visualizes emotional cues and helps readers recognize hidden affective framing in daily news coverage.

Abstract: News media often shape the public mood not only by what they report but by how they frame it. The same event can appear calm in one outlet and alarming in another, reflecting subtle emotional bias in reporting. Negative or emotionally charged headlines tend to attract more attention and spread faster, which in turn encourages outlets to frame stories in ways that provoke stronger reactions. This research explores that tendency through large-scale emotion analysis of Bengali news. Using zero-shot inference with Gemma-3 4B, we analyzed 300000 Bengali news headlines and their content to identify the dominant emotion and overall tone of each. The findings reveal a clear dominance of negative emotions, particularly anger, fear, and disappointment, and significant variation in how similar stories are emotionally portrayed across outlets. Based on these insights, we propose design ideas for a human-centered news aggregator that visualizes emotional cues and helps readers recognize hidden affective framing in daily news.

Method: Presents a literature review of explainability approaches, conducts experimental studies in healthcare and autonomous driving domains, and analyzes trust implications of explanations.

Result: The paper provides insights into current explainability methods and their application in critical domains, highlighting the importance of human-aligned explanations for building trust.

Conclusion: Identifies unaddressed issues in LLM explainability and outlines opportunities, challenges, and future directions for generating trustworthy explanations that align with human understanding.

Abstract: Large language models have exhibited impressive performance across a broad range of downstream tasks in natural language processing. However, how a language model predicts the next token and generates content is not generally understandable by humans. Furthermore, these models often make errors in prediction and reasoning, known as hallucinations. These errors underscore the urgent need to better understand and interpret the intricate inner workings of language models and how they generate predictive outputs. Motivated by this gap, this paper investigates local explainability and mechanistic interpretability within Transformer-based large language models to foster trust in such models. In this regard, our paper aims to make three key contributions. First, we present a review of local explainability and mechanistic interpretability approaches and insights from relevant studies in the literature. Furthermore, we describe experimental studies on explainability and reasoning with large language models in two critical domains – healthcare and autonomous driving – and analyze the trust implications of such explanations for explanation receivers. Finally, we summarize current unaddressed issues in the evolving landscape of LLM explainability and outline the opportunities, critical challenges, and future directions toward generating human-aligned, trustworthy LLM explanations.

Method: Proposes TaxoAlign - a three-phase topic-based instruction-guided method for scholarly taxonomy generation, with an automated evaluation framework measuring structural alignment and semantic coherence.

Result: TaxoAlign consistently surpasses baselines on nearly all metrics when evaluated on CS-TaxoBench (460 taxonomies from human-written surveys plus 80 from conference surveys).

Conclusion: The method successfully bridges the gap between human-generated and automatically-created taxonomies, demonstrating superior performance over existing approaches.

Abstract: Taxonomies play a crucial role in helping researchers structure and navigate knowledge in a hierarchical manner. They also form an important part in the creation of comprehensive literature surveys. The existing approaches to automatic survey generation do not compare the structure of the generated surveys with those written by human experts. To address this gap, we present our own method for automated taxonomy creation that can bridge the gap between human-generated and automatically-created taxonomies. For this purpose, we create the CS-TaxoBench benchmark which consists of 460 taxonomies that have been extracted from human-written survey papers. We also include an additional test set of 80 taxonomies curated from conference survey papers. We propose TaxoAlign, a three-phase topic-based instruction-guided method for scholarly taxonomy generation. Additionally, we propose a stringent automated evaluation framework that measures the structural alignment and semantic coherence of automatically generated taxonomies in comparison to those created by human experts. We evaluate our method and various baselines on CS-TaxoBench, using both automated evaluation metrics and human evaluation studies. The results show that TaxoAlign consistently surpasses the baselines on nearly all metrics. The code and data can be found at https://github.com/AvishekLahiri/TaxoAlign.

Method: Used CyberAgressionAdo-Large dataset to evaluate abuse detection, bullying behavior analysis, and bullying peer-group identification using six text-based and eight graph-based representation-learning methods with multimodal fusion.

Result: Multimodal models outperformed unimodal baselines, with late fusion model mBERT + WD-SGCN achieving best overall performance (0.718 on abuse detection, 0.286 on peer-group identification, 0.606 on bullying analysis).

Conclusion: The study demonstrates the effectiveness of multimodal approaches in handling nuanced antisocial behavior phenomena like implicit aggression and context-dependent hostility in multi-party conversations.

Abstract: Antisocial behavior (ASB) on social media – including hate speech, harassment, and cyberbullying – poses growing risks to platform safety and societal well-being. Prior research has focused largely on networks such as X and Reddit, while \textit{multi-party conversational settings} remain underexplored due to limited data. To address this gap, we use \textit{CyberAgressionAdo-Large}, a French open-access dataset simulating ASB in multi-party conversations, and evaluate three tasks: \textit{abuse detection}, \textit{bullying behavior analysis}, and \textit{bullying peer-group identification}. We benchmark six text-based and eight graph-based \textit{representation-learning methods}, analyzing lexical cues, interactional dynamics, and their multimodal fusion. Results show that multimodal models outperform unimodal baselines. The late fusion model \texttt{mBERT + WD-SGCN} achieves the best overall results, with top performance on abuse detection (0.718) and competitive scores on peer-group identification (0.286) and bullying analysis (0.606). Error analysis highlights its effectiveness in handling nuanced ASB phenomena such as implicit aggression, role transitions, and context-dependent hostility.

Method: Preregistered study comparing MFA-trained expert writers with three AI models (ChatGPT, Claude, Gemini) using blind pairwise evaluations by 159 expert and lay readers. Tested both in-context prompting and fine-tuning approaches.

Result: In-context prompting was strongly disfavored by experts for stylistic fidelity and writing quality, but fine-tuning completely reversed these findings - experts then favored AI-generated text. Fine-tuned outputs were rarely detected as AI-generated (3% vs 97% for in-context).

Conclusion: Author-specific fine-tuning enables AI writing that readers prefer to expert human writing, providing empirical evidence relevant to copyright’s fair-use factor regarding market effects on source works.

Abstract: The use of copyrighted books for training AI models has led to numerous lawsuits from authors concerned about AI’s ability to generate derivative content. Yet it’s unclear if these models can generate high quality literary text while emulating authors’ styles. To answer this we conducted a preregistered study comparing MFA-trained expert writers with three frontier AI models: ChatGPT, Claude & Gemini in writing up to 450 word excerpts emulating 50 award-winning authors’ diverse styles. In blind pairwise evaluations by 159 representative expert & lay readers, AI-generated text from in-context prompting was strongly disfavored by experts for both stylistic fidelity (OR=0.16, p<10^-8) & writing quality (OR=0.13, p<10^-7) but showed mixed results with lay readers. However, fine-tuning ChatGPT on individual authors’ complete works completely reversed these findings: experts now favored AI-generated text for stylistic fidelity (OR=8.16, p<10^-13) & writing quality (OR=1.87, p=0.010), with lay readers showing similar shifts. These effects generalize across authors & styles. The fine-tuned outputs were rarely flagged as AI-generated (3% rate v. 97% for in-context prompting) by best AI detectors. Mediation analysis shows this reversal occurs because fine-tuning eliminates detectable AI stylistic quirks (e.g., cliche density) that penalize in-context outputs. While we do not account for additional costs of human effort required to transform raw AI output into cohesive, publishable prose, the median fine-tuning & inference cost of $81 per author represents a dramatic 99.7% reduction compared to typical professional writer compensation. Author-specific fine-tuning thus enables non-verbatim AI writing that readers prefer to expert human writing, providing empirical evidence directly relevant to copyright’s fourth fair-use factor, the “effect upon the potential market or value” of the source works.

Method: Proposed Nyx - a unified mixed-modal retriever, with a four-stage automated pipeline to generate NyxQA dataset, followed by two-stage training: pre-training on mixed-modal data and supervised fine-tuning using VLM feedback.

Result: Nyx performs competitively on standard text-only RAG benchmarks and excels in Universal RAG setting, significantly improving generation quality in vision-language tasks.

Conclusion: Nyx effectively addresses the Universal RAG challenge by enabling mixed-modal retrieval and reasoning, demonstrating strong performance in both text-only and mixed-modal scenarios.

Abstract: Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm for enhancing large language models (LLMs) by retrieving relevant documents from an external corpus. However, existing RAG systems primarily focus on unimodal text documents, and often fall short in real-world scenarios where both queries and documents may contain mixed modalities (such as text and images). In this paper, we address the challenge of Universal Retrieval-Augmented Generation (URAG), which involves retrieving and reasoning over mixed-modal information to improve vision-language generation. To this end, we propose Nyx, a unified mixed-modal to mixed-modal retriever tailored for URAG scenarios. To mitigate the scarcity of realistic mixed-modal data, we introduce a four-stage automated pipeline for generation and filtering, leveraging web documents to construct NyxQA, a dataset comprising diverse mixed-modal question-answer pairs that better reflect real-world information needs. Building on this high-quality dataset, we adopt a two-stage training framework for Nyx: we first perform pre-training on NyxQA along with a variety of open-source retrieval datasets, followed by supervised fine-tuning using feedback from downstream vision-language models (VLMs) to align retrieval outputs with generative preferences. Experimental results demonstrate that Nyx not only performs competitively on standard text-only RAG benchmarks, but also excels in the more general and realistic URAG setting, significantly improving generation quality in vision-language tasks.

Method: Evaluated 20 IT-LLMs on modified MMLU and MMLU-Pro benchmarks by systematically varying option label formats (alphabetic, numeric, Roman) under four paradigms: with explicit instructions, without instructions, with option contents removed, and with three-shot exemplars.

Result: Label format changes caused large performance shifts (up to -30.45%), performance dropped without instructions (up to -10.84%), models failed random-choice baselines when option contents were removed except with numeric labels, and three-shot exemplars provided no significant gains.

Conclusion: Current instruction-tuning paradigms are insufficient, and there’s a need for evaluation methods and training strategies that explicitly target atomic instruction-following, as larger LLMs achieve higher accuracy but remain inconsistent in instruction adherence.

Abstract: Instruction-tuned large language models (IT-LLMs) exhibit strong zero-shot reasoning, yet their ability to execute simple, self-contained instructions remains underexplored, despite this being foundational to complex instruction-following. We evaluate 20 IT-LLMs on modified MMLU and MMLU-Pro benchmarks, by systematically varying the format of option labels (alphabetic, numeric, Roman) while keeping their meaning identical under four paradigms, namely: (1) With explicit instructions, label changes cause large performance shifts (e.g., -30.45% for Roman vs. numeric), revealing instruction-format bias. (2) Without instructions, performance drops further (up to -10.84%) and label sensitivity intensifies, underscoring the role of explicit guidance. (3) When option contents are removed, models fail random-choice baselines except with numeric labels, suggesting weak adherence to atomic directives. (4) Three-shot exemplars yield no significant gains in robustness or fidelity, and generation analyses show persistent label errors, especially for non-numeric formats. Across model sizes, larger LLMs achieve higher accuracy but remain inconsistent in instruction adherence. These results expose the insufficiencies of current instruction-tuning paradigms and highlight the need for evaluation methods and training strategies that explicitly target atomic instruction-following.

Method: Created EduAdapt benchmark with nearly 48k grade-labeled QA pairs across nine science subjects spanning Grades 1-12, grouped into four grade levels. Evaluated diverse open-source LLMs on this benchmark.

Result: Larger models generally perform better but still struggle with generating suitable responses for early-grade students (Grades 1-5).

Conclusion: This work presents the first dataset and evaluation framework for assessing grade-level adaptability in LLMs, aiming to foster more developmentally aligned educational AI systems through better training and prompting strategies.

Abstract: Large language models (LLMs) are transforming education by answering questions, explaining complex concepts, and generating content across a wide range of subjects. Despite strong performance on academic benchmarks, they often fail to tailor responses to students’ grade levels. This is a critical need in K-12 education, where age-appropriate vocabulary and explanation are essential for effective learning. Existing models frequently produce outputs that are too advanced or vague for younger learners, and there are no standardized benchmarks to evaluate their ability to adjust across cognitive and developmental stages. To address this gap, we introduce EduAdapt, a benchmark of nearly 48k grade-labeled QA pairs across nine science subjects, spanning Grades 1-12 and grouped into four grade levels. We evaluate a diverse set of open-source LLMs on EduAdapt and find that while larger models generally perform better, they still struggle with generating suitable responses for early-grade students (Grades 1-5). Our work presents the first dataset and evaluation framework for assessing grade-level adaptability in LLMs, aiming to foster more developmentally aligned educational AI systems through better training and prompting strategies. EduAdapt code and datasets are publicly available at https://github.com/NaumanNaeem/EduAdapt.

Method: Built on Qwen2.5-7B-Instruct foundation model, trained exclusively on TCM-Ladder benchmark’s textual subset using GRPO reinforcement learning method that optimizes response selection through intra-group comparisons.

Result: Ladder-base demonstrates superior performance across multiple reasoning metrics compared to state-of-the-art general-purpose LLMs (GPT-4, Gemini 2.5, Claude 3, Qwen3) and domain-specific TCM models (BenTsao, HuatuoGPT2, Zhongjing).

Conclusion: GRPO provides an effective strategy for aligning LLMs with expert-level reasoning in traditional medical domains and supports development of trustworthy TCM AI systems.

Abstract: Traditional Chinese Medicine (TCM) presents a rich and structurally unique knowledge system that challenges conventional applications of large language models (LLMs). Although previous TCM-specific LLMs have shown progress through supervised fine-tuning, they often face limitations in alignment, data quality, and evaluation consistency. In this study, we introduce Ladder-base, the first TCM-focused LLM trained with Group Relative Policy Optimization (GRPO), a reinforcement learning method that improves reasoning and factual consistency by optimizing response selection based on intra-group comparisons. Ladder-base is built upon the Qwen2.5-7B-Instruct foundation model and trained exclusively on the textual subset of the TCM-Ladder benchmark, using 80 percent of the data for training and the remaining 20 percent split evenly between validation and test sets. Through standardized evaluation, Ladder-base demonstrates superior performance across multiple reasoning metrics when compared to both state-of-the-art general-purpose LLMs such as GPT-4, Gemini 2.5, Claude 3, and Qwen3 and domain-specific TCM models including BenTsao, HuatuoGPT2, and Zhongjing. These findings suggest that GRPO provides an effective and efficient strategy for aligning LLMs with expert-level reasoning in traditional medical domains and supports the development of trustworthy and clinically grounded TCM artificial intelligence systems.

Method: The framework includes: (i) a curated dataset from Flickr8k with semantically aligned captions via context-aware selection and translation; (ii) a dynamic pipeline with model ensembling and adaptive substitution for quality preservation; (iii) AfriCaption model (0.5B parameters) integrating SigLIP and NLLB200 for vision-to-text caption generation.

Result: The framework establishes the first scalable image-captioning resource for under-represented African languages, ensuring ongoing data quality and enabling inclusive multimodal AI.

Conclusion: AfriCaption lays the groundwork for truly inclusive multimodal AI by providing a comprehensive solution for multilingual image captioning in African languages, addressing the democratization gap in the field.

Abstract: Multimodal AI research has overwhelmingly focused on high-resource languages, hindering the democratization of advancements in the field. To address this, we present AfriCaption, a comprehensive framework for multilingual image captioning in 20 African languages and our contributions are threefold: (i) a curated dataset built on Flickr8k, featuring semantically aligned captions generated via a context-aware selection and translation process; (ii) a dynamic, context-preserving pipeline that ensures ongoing quality through model ensembling and adaptive substitution; and (iii) the AfriCaption model, a 0.5B parameter vision-to-text architecture that integrates SigLIP and NLLB200 for caption generation across under-represented languages. This unified framework ensures ongoing data quality and establishes the first scalable image-captioning resource for under-represented African languages, laying the groundwork for truly inclusive multimodal AI.

Method: Post-hoc weight interpolation between models before and after alignment training.

Result: Consistently reveals Pareto-optimal interpolations that improve accuracy beyond both parent models while recovering calibration.

Conclusion: Simple model merging efficiently mitigates the full alignment tax, yielding more capable and reliable models.

Abstract: The “alignment tax” of post-training is typically framed as a drop in task accuracy. We show it also involves a severe loss of calibration, making models overconfident, less reliable, and model outputs less diverse. We show that this trade-off can be navigated effectively via a simple post-hoc intervention: interpolating between a model’s weights before and after alignment. Crucially, this is not a strict trade-off. We find that the process consistently reveals Pareto-optimal interpolations - models that improve accuracy beyond both parents while substantially recovering the calibration lost during alignment. Our work demonstrates that simple model merging provides a computationally efficient method for mitigating the full scope of the alignment tax, yielding models that are more capable and more reliable.

Method: Two simple attacks: Search attack (forcing model to begin with search) and Multi-search attack (encouraging repeated searches). Tested across Qwen and Llama model families with local and web search.

Result: Attacks lowered refusal rates by up to 60.0%, answer safety by 82.5%, and search-query safety by 82.4%. Models generate harmful, request-mirroring search queries before refusal tokens.

Conclusion: Current RL training rewards effective queries without accounting for harmfulness, creating exploitable vulnerabilities. Urgent need for safety-aware agentic RL pipelines that optimize for safe search.

Abstract: Agentic reinforcement learning (RL) trains large language models to autonomously call tools during reasoning, with search as the most common application. These models excel at multi-step reasoning tasks, but their safety properties are not well understood. In this study, we show that RL-trained search models inherit refusal from instruction tuning and often deflect harmful requests by turning them into safe queries. However, this safety is fragile. Two simple attacks, one that forces the model to begin response with search (Search attack), another that encourages models to repeatedly search (Multi-search attack), trigger cascades of harmful searches and answers. Across two model families (Qwen, Llama) with both local and web search, these attacks lower refusal rates by up to 60.0%, answer safety by 82.5%, and search-query safety by 82.4%. The attacks succeed by triggering models to generate harmful, request-mirroring search queries before they can generate the inherited refusal tokens. This exposes a core weakness of current RL training: it rewards continued generation of effective queries without accounting for their harmfulness. As a result, RL search models have vulnerabilities that users can easily exploit, making it urgent to develop safety-aware agentic RL pipelines optimising for safe search.

Method: Explored monolingual and multilingual BERT-based models trained on general domain text for extracting disease and medication mentions from clinical case reports in three languages.

Result: Achieved F1-scores of 77.88% (Spanish Diseases), 92.09% (Spanish Medications), 91.74% (English Medications), and 88.9% (Italian Medications), outperforming mean and median scores across all subtasks.

Conclusion: The proposed deep contextual embedding models effectively enhance clinical NER performance in multiple languages, demonstrating the viability of adapting general-domain BERT models for specialized clinical text processing tasks.

Abstract: The rapidly increasing volume of electronic health record (EHR) data underscores a pressing need to unlock biomedical knowledge from unstructured clinical texts to support advancements in data-driven clinical systems, including patient diagnosis, disease progression monitoring, treatment effects assessment, prediction of future clinical events, etc. While contextualized language models have demonstrated impressive performance improvements for named entity recognition (NER) systems in English corpora, there remains a scarcity of research focused on clinical texts in low-resource languages. To bridge this gap, our study aims to develop multiple deep contextual embedding models to enhance clinical NER in the cardiology domain, as part of the BioASQ MultiCardioNER shared task. We explore the effectiveness of different monolingual and multilingual BERT-based models, trained on general domain text, for extracting disease and medication mentions from clinical case reports written in English, Spanish, and Italian. We achieved an F1-score of 77.88% on Spanish Diseases Recognition (SDR), 92.09% on Spanish Medications Recognition (SMR), 91.74% on English Medications Recognition (EMR), and 88.9% on Italian Medications Recognition (IMR). These results outperform the mean and median F1 scores in the test leaderboard across all subtasks, with the mean/median values being: 69.61%/75.66% for SDR, 81.22%/90.18% for SMR, 89.2%/88.96% for EMR, and 82.8%/87.76% for IMR.

Method: Created first Urdu to English idiomatic translation datasets for both Native and Roman Urdu scripts, evaluated multiple LLMs and NMT systems using automatic metrics (BLEU, BERTScore, COMET, XCOMET), and tested prompt engineering approaches.

Result: Prompt engineering improves idiomatic translation compared to direct translation, though differences between prompt types are minor. Native Urdu inputs produce more accurate idiomatic translations than Roman Urdu.

Conclusion: Text representation significantly impacts translation quality for idiomatic expressions, with Native Urdu performing better than Roman Urdu, and prompt engineering can enhance idiomatic translation capabilities.

Abstract: Idiomatic translation remains a significant challenge in machine translation, especially for low resource languages such as Urdu, and has received limited prior attention. To advance research in this area, we introduce the first evaluation datasets for Urdu to English idiomatic translation, covering both Native Urdu and Roman Urdu scripts and annotated with gold-standard English equivalents. We evaluate multiple open-source Large Language Models (LLMs) and Neural Machine Translation (NMT) systems on this task, focusing on their ability to preserve idiomatic and cultural meaning. Automatic metrics including BLEU, BERTScore, COMET, and XCOMET are used to assess translation quality. Our findings indicate that prompt engineering enhances idiomatic translation compared to direct translation, though performance differences among prompt types are relatively minor. Moreover, cross script comparisons reveal that text representation substantially affects translation quality, with Native Urdu inputs producing more accurate idiomatic translations than Roman Urdu.

Method: Constructed MultiWikiHealthCare dataset from Wikipedia, analyzed cross-lingual healthcare coverage, assessed LLM response alignment with references, and conducted case studies using contextual information and RAG across five languages.

Result: Substantial cross-lingual disparities in both Wikipedia coverage and LLM factual alignment; LLM responses consistently align more with English Wikipedia regardless of prompt language; providing non-English contextual excerpts effectively shifts factual alignment toward culturally relevant knowledge.

Conclusion: The findings highlight practical pathways for building more equitable, multilingual AI systems in healthcare by addressing cross-lingual disparities through contextual information and RAG approaches.

Abstract: Equitable access to reliable health information is vital when integrating AI into healthcare. Yet, information quality varies across languages, raising concerns about the reliability and consistency of multilingual Large Language Models (LLMs). We systematically examine cross-lingual disparities in pre-training source and factuality alignment in LLM answers for multilingual healthcare Q&A across English, German, Turkish, Chinese (Mandarin), and Italian. We (i) constructed Multilingual Wiki Health Care (MultiWikiHealthCare), a multilingual dataset from Wikipedia; (ii) analyzed cross-lingual healthcare coverage; (iii) assessed LLM response alignment with these references; and (iv) conducted a case study on factual alignment through the use of contextual information and Retrieval-Augmented Generation (RAG). Our findings reveal substantial cross-lingual disparities in both Wikipedia coverage and LLM factual alignment. Across LLMs, responses align more with English Wikipedia, even when the prompts are non-English. Providing contextual excerpts from non-English Wikipedia at inference time effectively shifts factual alignment toward culturally relevant knowledge. These results highlight practical pathways for building more equitable, multilingual AI systems for healthcare.

Method: ReXMoE allows routers to reuse experts across adjacent layers, decoupling expert dimensionality from per-layer budgets. It uses progressive scaling routing (PSR) to gradually increase candidate expert pool during training.

Result: Extensive experiments on 0.5B to 7B parameter models show ReXMoE consistently improves language modeling and downstream task performance under fixed architectural dimensions.

Conclusion: ReXMoE represents a new design paradigm for parameter-efficient and scalable MoE-based LLMs, enabling richer expert combinations without sacrificing capacity or inflating parameters.

Abstract: Mixture-of-Experts (MoE) architectures have emerged as a promising approach to scale Large Language Models (LLMs). MoE boosts the efficiency by activating a subset of experts per token. Recent works show that fine-grained experts substantially enriches the combinatorial flexibility of active experts and enhances model expressiveness. However, such a design is fundamentally limited by the layer-local routing mechanism: each layer is restricted to its own expert pool. This requires a careful trade-off between expert dimensionality and routing diversity given fixed parameter budgets. We describe ReXMoE, a novel MoE architecture that improves routing beyond the existing layer-local approaches by allowing routers to reuse experts across adjacent layers. ReXMoE decouples expert dimensionality from per-layer budgets, enabling richer expert combinations without sacrificing individual expert capacity or inflating overall parameters. To this end, we propose a new progressive scaling routing (PSR) strategy to gradually increase the candidate expert pool during training. As a result, ReXMoE improves both language modeling and downstream task performance. Extensive experiments on models ranging from 0.5B to 7B parameters across different architectures demonstrate that ReXMoE consistently improves performance under fixed architectural dimensions, confirming ReXMoE as new design paradigm for parameter-efficient and scalable MoE-based LLMs.

Method: Proposed DETree approach models relationships among different human-AI collaboration processes as a Hierarchical Affinity Tree structure with specialized loss function that aligns text representations with the tree. Developed RealBench dataset containing various hybrid texts from different collaboration processes to facilitate learning.

Result: DETree improves performance in hybrid text detection tasks and significantly enhances robustness and generalization in out-of-distribution scenarios, particularly in few-shot learning conditions. Demonstrates promise of training-based approaches in OOD settings.

Conclusion: The DETree framework effectively captures the complex relationships in human-AI collaborative text generation processes through hierarchical modeling, providing superior detection performance and robustness compared to existing methods.

Abstract: Detecting AI-involved text is essential for combating misinformation, plagiarism, and academic misconduct. However, AI text generation includes diverse collaborative processes (AI-written text edited by humans, human-written text edited by AI, and AI-generated text refined by other AI), where various or even new LLMs could be involved. Texts generated through these varied processes exhibit complex characteristics, presenting significant challenges for detection. Current methods model these processes rather crudely, primarily employing binary classification (purely human vs. AI-involved) or multi-classification (treating human-AI collaboration as a new class). We observe that representations of texts generated through different processes exhibit inherent clustering relationships. Therefore, we propose DETree, a novel approach that models the relationships among different processes as a Hierarchical Affinity Tree structure, and introduces a specialized loss function that aligns text representations with this tree. To facilitate this learning, we developed RealBench, a comprehensive benchmark dataset that automatically incorporates a wide spectrum of hybrid texts produced through various human-AI collaboration processes. Our method improves performance in hybrid text detection tasks and significantly enhances robustness and generalization in out-of-distribution scenarios, particularly in few-shot learning conditions, further demonstrating the promise of training-based approaches in OOD settings. Our code and dataset are available at https://github.com/heyongxin233/DETree.

Method: Uses an industry agent capability maturity framework to outline agent evolution from ‘process execution systems’ to ‘adaptive social systems’, and systematically reviews three key technological pillars (Memory, Planning, Tool Use), real-world applications, and evaluation benchmarks.

Result: The review clarifies the current state of industry agents, identifies challenges in evaluation systems regarding authenticity, safety, and industry specificity, and explores practical challenges including capability boundaries, developmental potential, and governance issues.

Conclusion: By combining technological evolution with industry practices, the paper provides a clear roadmap and theoretical foundation for understanding and building the next generation of industry agents.

Abstract: With the rise of large language models (LLMs), LLM agents capable of autonomous reasoning, planning, and executing complex tasks have become a frontier in artificial intelligence. However, how to translate the research on general agents into productivity that drives industry transformations remains a significant challenge. To address this, this paper systematically reviews the technologies, applications, and evaluation methods of industry agents based on LLMs. Using an industry agent capability maturity framework, it outlines the evolution of agents in industry applications, from “process execution systems” to “adaptive social systems.” First, we examine the three key technological pillars that support the advancement of agent capabilities: Memory, Planning, and Tool Use. We discuss how these technologies evolve from supporting simple tasks in their early forms to enabling complex autonomous systems and collective intelligence in more advanced forms. Then, we provide an overview of the application of industry agents in real-world domains such as digital engineering, scientific discovery, embodied intelligence, collaborative business execution, and complex system simulation. Additionally, this paper reviews the evaluation benchmarks and methods for both fundamental and specialized capabilities, identifying the challenges existing evaluation systems face regarding authenticity, safety, and industry specificity. Finally, we focus on the practical challenges faced by industry agents, exploring their capability boundaries, developmental potential, and governance issues in various scenarios, while providing insights into future directions. By combining technological evolution with industry practices, this review aims to clarify the current state and offer a clear roadmap and theoretical foundation for understanding and building the next generation of industry agents.

Method: DSER conceptualizes iterative reasoning as a Markov chain with stochastic transitions, running multiple long-horizon self-evolving processes in parallel to amplify small positive tendencies toward correct solutions.

Result: Applied to DeepSeek-R1-0528-Qwen3-8B, DSER solved 5 out of 9 previously unsolvable problems on AIME 2024-2025 benchmark and enabled this compact model to surpass its 600B-parameter teacher’s single-turn accuracy through majority voting.

Conclusion: DSER framework not only provides immediate test-time scaling utility but also diagnoses fundamental limitations of current open-weight reasoners, establishing a clear research agenda for developing next-generation models with intrinsic self-evolving capabilities.

Abstract: Long-form chain-of-thought reasoning has become a cornerstone of advanced reasoning in large language models. While recent verification-refinement frameworks have enabled proprietary models to solve Olympiad-level problems, their effectiveness hinges on strong, reliable verification and correction capabilities, which remain fragile in open-weight, smaller-scale models. This work demonstrates that even with weak verification and refinement capabilities on hard tasks, the reasoning limits of such models can be substantially extended through a probabilistic paradigm we call Deep Self-Evolving Reasoning (DSER). We conceptualize iterative reasoning as a Markov chain, where each step represents a stochastic transition in the solution space. The key insight is that convergence to a correct solution is guaranteed as long as the probability of improvement marginally exceeds that of degradation. By running multiple long-horizon, self-evolving processes in parallel, DSER amplifies these small positive tendencies, enabling the model to asymptotically approach correct answers. Empirically, we apply DSER to the DeepSeek-R1-0528-Qwen3-8B model. On the challenging AIME 2024-2025 benchmark, DSER solves 5 out of 9 previously unsolvable problems and boosts overall performance, enabling this compact model to surpass the single-turn accuracy of its 600B-parameter teacher through majority voting. Beyond its immediate utility for test-time scaling, the DSER framework serves to diagnose the fundamental limitations of current open-weight reasoners. By clearly delineating their shortcomings in self-verification, refinement, and stability, our findings establish a clear research agenda for developing next-generation models with powerful, intrinsic self-evolving capabilities.

Method: Combined best methods for learning monolingual and cross-lingual representations: masked language modeling (MLM), translation language modeling (TLM), dual encoder translation ranking, and additive margin softmax. Used pre-trained multilingual language models to reduce parallel data requirements.

Result: Achieved 83.7% bi-text retrieval accuracy over 112 languages on Tatoeba (vs 65.5% by LASER), with 80% reduction in parallel training data requirements. The model also performs competitively on monolingual transfer learning benchmarks and can mine parallel data for training competitive NMT models.

Conclusion: The proposed methods successfully create high-quality multilingual sentence embeddings that significantly outperform previous approaches in cross-lingual retrieval while maintaining strong monolingual performance. The model is publicly released for 109+ languages.

Abstract: While BERT is an effective method for learning monolingual sentence embeddings for semantic similarity and embedding based transfer learning BERT based cross-lingual sentence embeddings have yet to be explored. We systematically investigate methods for learning multilingual sentence embeddings by combining the best methods for learning monolingual and cross-lingual representations including: masked language modeling (MLM), translation language modeling (TLM), dual encoder translation ranking, and additive margin softmax. We show that introducing a pre-trained multilingual language model dramatically reduces the amount of parallel training data required to achieve good performance by 80%. Composing the best of these methods produces a model that achieves 83.7% bi-text retrieval accuracy over 112 languages on Tatoeba, well above the 65.5 achieved by LASER, while still performing competitively on monolingual transfer learning benchmarks. Parallel data mined from CommonCrawl using our best model is shown to train competitive NMT models for en-zh and en-de. We publicly release our best multilingual sentence embedding model for 109+ languages at https://tfhub.dev/google/LaBSE.

Method: Two-stage framework: 1) Elicitation stage uses inexpensive self-consistency supervision to extract internal confidence, 2) Calibration stage uses a small set of correctness annotations (only 1k) to refine the confidence estimates.

Result: EliCal achieves near-optimal alignment with only 1k correctness annotations (0.18% of full supervision) and shows better alignment performance on unseen MMLU tasks compared to calibration-only baselines.

Conclusion: EliCal offers a scalable solution for universal honesty alignment in LLMs by significantly reducing annotation costs while maintaining strong performance across diverse tasks.

Abstract: Honesty alignment-the ability of large language models (LLMs) to recognize their knowledge boundaries and express calibrated confidence-is essential for trustworthy deployment. Existing methods either rely on training-free confidence estimation (e.g., token probabilities, self-consistency) or training-based calibration with correctness annotations. While effective, achieving universal honesty alignment with training-based calibration requires costly, large-scale labeling. To support annotation-efficient training, we introduce Elicitation-Then-Calibration (EliCal), a two-stage framework that first elicits internal confidence using inexpensive self-consistency supervision, then calibrates this confidence with a small set of correctness annotations. To support a large-scale study, we release HonestyBench, a benchmark covering ten free-form QA datasets with 560k training and 70k evaluation instances annotated with correctness and self-consistency signals. Experiments show that EliCal achieves near-optimal alignment with only 1k correctness annotations (0.18% of full supervision) and better alignment performance on unseen MMLU tasks than the calibration-only baseline, offering a scalable solution toward universal honesty alignment in LLMs.

Method: Introduced SimBench benchmark that unifies 20 diverse datasets covering tasks from moral decision-making to economic choice across a large global participant pool. Evaluated LLM performance across different model sizes, inference-time compute, instruction-tuning approaches, and demographic simulations.

Result: Best LLMs today have limited simulation ability (score: 40.80/100). Performance scales log-linearly with model size. Simulation performance is not improved by increased inference-time compute. Shows alignment-simulation trade-off: instruction-tuning helps on low-entropy questions but degrades performance on high-entropy ones. Models struggle with specific demographic groups. Simulation ability correlates strongly with deep, knowledge-intensive reasoning (MMLU-Pro, r=0.939).

Conclusion: SimBench enables measurable progress in LLM simulation development by providing standardized evaluation. Current LLMs have limited but scalable simulation capabilities, with specific limitations around demographic simulation and trade-offs between alignment and simulation performance.

Abstract: Large language model (LLM) simulations of human behavior have the potential to revolutionize the social and behavioral sciences, if and only if they faithfully reflect real human behaviors. Current evaluations are fragmented, based on bespoke tasks and metrics, creating a patchwork of incomparable results. To address this, we introduce SimBench, the first large-scale, standardized benchmark for a robust, reproducible science of LLM simulation. By unifying 20 diverse datasets covering tasks from moral decision-making to economic choice across a large global participant pool, SimBench provides the necessary foundation to ask fundamental questions about when, how, and why LLM simulations succeed or fail. We show that, while even the best LLMs today have limited simulation ability (score: 40.80/100), performance scales log-linearly with model size. Simulation performance is not improved by increased inference-time compute. We demonstrate an alignment-simulation trade-off: instruction-tuning improves performance on low-entropy (consensus) questions but degrades it on high-entropy (diverse) ones. Models particularly struggle when simulating specific demographic groups. Finally, we demonstrate that simulation ability correlates most strongly with deep, knowledge-intensive reasoning (MMLU-Pro, r=0.939). By making progress measurable, we aim to accelerate the development of more faithful LLM simulators.

Method: Multi-task learning framework training LLMs to jointly perform binary survival classification, continuous survival time regression, and natural language rationale generation. Evaluated three alignment strategies: standard SFT, SFT with CoT prompting, and GRPO reinforcement learning.

Result: CoT prompting improved F1 by +6.0 and reduced MAE by 12%. GRPO achieved state-of-the-art interpretability and predictive performance across BLEU, ROUGE, and BERTScore metrics.

Conclusion: Reasoning-aware alignment is crucial for multi-task clinical modeling, setting a new benchmark for interpretable, trustworthy LLMs in precision oncology.

Abstract: Predicting cancer treatment outcomes requires models that are both accurate and interpretable, particularly in the presence of heterogeneous clinical data. While large language models (LLMs) have shown strong performance in biomedical NLP, they often lack structured reasoning capabilities critical for high-stakes decision support. We present a unified, multi-task learning framework that aligns autoregressive LLMs with clinical reasoning for outcome prediction on the MSK-CHORD dataset. Our models are trained to jointly perform binary survival classification, continuous survival time regression, and natural language rationale generation. We evaluate three alignment strategies: (1) standard supervised fine-tuning (SFT), (2) SFT with Chain-of-Thought (CoT) prompting to elicit step-by-step reasoning, and (3) Group Relative Policy Optimization (GRPO), a reinforcement learning method that aligns model outputs to expert-derived reasoning trajectories. Experiments with LLaMa3-8B and Med42-8B backbones demonstrate that CoT prompting improves F1 by +6.0 and reduces MAE by 12%, while GRPO achieves state-of-the-art interpretability and predictive performance across BLEU, ROUGE, and BERTScore. We further show that existing biomedical LLMs often fail to produce valid reasoning traces due to architectural constraints. Our findings underscore the importance of reasoning-aware alignment in multi-task clinical modeling and set a new benchmark for interpretable, trustworthy LLMs in precision oncology.

Method: Applied Mapper (a topological data analysis tool) to analyze RoBERTa-Large fine-tuned on the MD-Offense dataset, comparing it with traditional tools like PCA and UMAP.

Result: Fine-tuning restructures embedding space into modular, non-convex regions aligned with model predictions (>98% of components show ≥90% prediction purity). However, alignment with ground-truth labels drops in ambiguous data, revealing tension between structural confidence and label uncertainty.

Conclusion: Mapper serves as a powerful diagnostic tool for understanding how models resolve ambiguity, enabling topological metrics that can inform proactive modeling strategies in subjective NLP tasks.

Abstract: Language models are often evaluated with scalar metrics like accuracy, but such measures fail to capture how models internally represent ambiguity, especially when human annotators disagree. We propose a topological perspective to analyze how fine-tuned models encode ambiguity and more generally instances. Applied to RoBERTa-Large on the MD-Offense dataset, Mapper, a tool from topological data analysis, reveals that fine-tuning restructures embedding space into modular, non-convex regions aligned with model predictions, even for highly ambiguous cases. Over $98%$ of connected components exhibit $\geq 90%$ prediction purity, yet alignment with ground-truth labels drops in ambiguous data, surfacing a hidden tension between structural confidence and label uncertainty. Unlike traditional tools such as PCA or UMAP, Mapper captures this geometry directly uncovering decision regions, boundary collapses, and overconfident clusters. Our findings position Mapper as a powerful diagnostic tool for understanding how models resolve ambiguity. Beyond visualization, it also enables topological metrics that may inform proactive modeling strategies in subjective NLP tasks.

Method: Language Confusion Gate (LCG) filters tokens during decoding using norm-adjusted self-distillation to predict language families and apply masking only when needed, leveraging findings that correct-language tokens are usually top predictions.

Result: LCG decreases language confusion significantly (often by an order of magnitude) across various models including Qwen3, GPT-OSS, Gemma3, Llama3.1 without negatively impacting task performance.

Conclusion: LCG provides an effective plug-in solution to reduce language confusion in LLMs without model retraining, distinguishing between harmful confusion and acceptable code-switching.

Abstract: Large language models (LLMs) often experience language confusion, which is the unintended mixing of languages during text generation. Current solutions to this problem either necessitate model retraining or cannot differentiate between harmful confusion and acceptable code-switching. This paper introduces the Language Confusion Gate (LCG), a lightweight, plug-in solution that filters tokens during decoding without altering the base LLM. The LCG is trained using norm-adjusted self-distillation to predict appropriate language families and apply masking only when needed. Our method is based on the findings that language confusion is infrequent, correct-language tokens are usually among the top predictions, and output token embedding norms are larger for high-resource languages, which biases sampling. When evaluated across various models, including Qwen3, GPT-OSS, Gemma3, Llama3.1, LCG decreases language confusion significantly, often by an order of magnitude, without negatively impacting task performance. Code is available at https://github.com/collinzrj/language_confusion_gate.

Method: Proposed three high-order correlations (AST family, lexical, line), designed tokens/hyperedges generator, and created HGAdapter - a hypergraph-based adapter that combines improved hypergraph neural networks with adapter tuning.

Result: Improved PLM performance on code summarization and clone detection tasks across six languages, with varying degrees of improvement.

Conclusion: High-order data correlations in code contribute to improved effectiveness of PLMs for code-related tasks.

Abstract: Pre-trained language models (PLMs) are increasingly being applied to code-related tasks. Although PLMs have achieved good results, they do not take into account potential high-order data correlations within the code. We propose three types of high-order correlations in code tokens, i.e. abstract syntax tree family correlation, lexical correlation, and line correlation. We design a tokens and hyperedges generator to capture these high-order data correlations. We improve the architecture of hypergraph neural networks and combine it with adapter tuning to propose a novel hypergraph-based adapter (HGAdapter) to fine-tune PLMs. HGAdapter can encode high-order data correlations and is allowed to be inserted into various PLMs to enhance performance. Experiments were conducted on several public datasets, including six languages of code summarization and code clone detection tasks. Our methods improved the performance of PLMs in datasets to varying degrees. Experimental results validate the introduction of high-order data correlations that contribute to improved effectiveness.

Method: Developed LawChain - a three-module reasoning framework that operationalizes legal reasoning processes for tort analysis, with each module containing multiple finer-grained sub-steps. Created LawChain_eval benchmark and evaluated state-of-the-art LLMs, then introduced baseline approaches incorporating LawChain-style reasoning through prompting or post-training.

Result: Current LLMs fall short in accurately handling crucial elements of tort legal reasoning. The proposed baseline approaches achieved significant improvements in tort-related legal reasoning and generalized well to related legal analysis tasks like Legal Named-Entity Recognition and Criminal Damages Calculation.

Conclusion: Explicitly modeling legal reasoning chains enhances language models’ reasoning capabilities, demonstrating the value of structured legal reasoning frameworks for improving computational legal analysis.

Abstract: Legal reasoning is a fundamental component of legal analysis and decision-making. Existing computational approaches to legal reasoning predominantly rely on generic reasoning frameworks such as syllogism and IRAC, which do not comprehensively examine the nuanced processes that underpin legal reasoning. Moreover, current research has largely focused on criminal cases, with insufficient modeling for civil cases. In this work, we present a novel framework for explicitly modeling legal reasoning in the analysis of Chinese tort-related civil cases. We first operationalize the legal reasoning processes used in tort analysis into the LawChain framework. LawChain is a three-module reasoning framework, with each module consisting of multiple finer-grained sub-steps. Informed by the LawChain framework, we introduce the task of tort legal reasoning and construct an evaluation benchmark, LawChain$_{eval}$, to systematically assess the critical steps within analytical reasoning chains for tort analysis. Leveraging this benchmark, we evaluate state-of-the-art large language models for their legal reasoning ability in civil tort contexts. Our results indicate that current models still fall short in accurately handling crucial elements of tort legal reasoning. Furthermore, we introduce several baseline approaches that explicitly incorporate LawChain-style reasoning through prompting or post-training. We conduct further experiments on additional legal analysis tasks, such as Legal Named-Entity Recognition and Criminal Damages Calculation, to verify the generalizability of these baselines. The proposed baseline approaches achieve significant improvements in tort-related legal reasoning and generalize well to related legal analysis tasks, thus demonstrating the value of explicitly modeling legal reasoning chains to enhance the reasoning capabilities of language models.

Method: Augment unlearning objectives with a plug-in term that preserves the model’s ability to use forgotten knowledge when present in context, evaluated through systematic testing of six state-of-the-art unlearning methods.

Result: The approach restores contextual utility to near original levels while maintaining effective forgetting and retain-set utility, as demonstrated through extensive experiments.

Conclusion: The proposed plug-in objective successfully addresses the contextual utility limitation of existing unlearning methods, enabling models to use forgotten knowledge when contextually appropriate while still achieving effective unlearning.

Abstract: Large language models may encode sensitive information or outdated knowledge that needs to be removed, to ensure responsible and compliant model responses. Unlearning has emerged as an efficient alternative to full retraining, aiming to remove specific knowledge while preserving overall model utility. Existing evaluations of unlearning methods focus on (1) the extent of forgetting of the target knowledge (forget set) and (2) maintaining performance on the retain set (i.e., utility). However, these evaluations overlook an important usability aspect: users may still want the model to leverage the removed information if it is re-introduced in the prompt. In a systematic evaluation of six state-of-the-art unlearning methods, we find that they consistently impair such contextual utility. To address this, we augment unlearning objectives with a plug-in term that preserves the model’s ability to use forgotten knowledge when it is present in context. Extensive experiments demonstrate that our approach restores contextual utility to near original levels while still maintaining effective forgetting and retain-set utility.

Method: Used a complete pipeline: bilingual continued pre-training with mixed Irish-English corpora, instruction tuning with synthesized datasets from Gemini-2.5-Pro, and human preference alignment. Created 30K Irish-English parallel instruction dataset and 1K human preference dataset.

Result: Achieved gains of up to 29% in Irish and 44% in English across benchmarks for translation, gender understanding, topic identification and world knowledge. Demonstrated clear progress in instruction following for chatbot functionality.

Conclusion: Qomhr’a successfully demonstrates the development of a bilingual LLM under low-resource conditions, showing significant improvements in both Irish and English performance through a comprehensive training pipeline.

Abstract: This paper introduces Qomhr'a, a bilingual Irish-English large language model (LLM), developed under low-resource constraints presenting a complete pipeline spanning bilingual continued pre-training, instruction tuning, and alignment from human preferences. Newly accessible Irish corpora and English text are mixed and curated to improve Irish performance while preserving English ability. 6 closed-weight LLMs are judged for their Irish text generation by a native speaker, a learner and other LLMs. Google’s Gemini-2.5-Pro is ranked the highest and is subsequently used to synthesise instruction tuning and human preference datasets. Two datasets are contributed leveraging Gemini-2.5-Pro: a 30K Irish-English parallel instruction tuning dataset and a 1K human preference dataset, generating accepted and rejected responses that show near perfect alignment with a native Irish speaker. Qomhr'a is comprehensively evaluated across benchmarks testing translation, gender understanding, topic identification and world knowledge with gains of up to 29% in Irish and 44% in English. Qomhr'a also undergoes instruction tuning and demonstrates clear progress in instruction following, crucial for chatbot functionality.

Method: The approach involves four steps: dialogue data collection, dialogue act (DA) annotation, DA pattern mining, and predictive model building to identify effective pedagogical strategies.

Result: Early insights are presented as initial findings, with the work representing an ongoing study that lays groundwork for future research in this area.

Conclusion: The research emphasizes the importance of evaluating LLM-based educational applications through dialogue dynamics and pedagogical strategies rather than just technical metrics.

Abstract: Dialogue plays a crucial role in educational settings, yet existing evaluation methods for educational applications of large language models (LLMs) primarily focus on technical performance or learning outcomes, often neglecting attention to learner-LLM interactions. To narrow this gap, this AIED Doctoral Consortium paper presents an ongoing study employing a dialogue analysis approach to identify effective pedagogical strategies from learner-LLM dialogues. The proposed approach involves dialogue data collection, dialogue act (DA) annotation, DA pattern mining, and predictive model building. Early insights are outlined as an initial step toward future research. The work underscores the need to evaluate LLM-based educational applications by focusing on dialogue dynamics and pedagogical strategies.

Method: Combines difficulty-aware graph sampling and difficulty-aware rejection fine-tuning to optimize specialized generators for creating challenging coding problems, then uses generated problems for distillation from strong teacher models or reinforcement learning.

Result: Generated 100K difficult problems that, when used to fine-tune Qwen3-8B-base, surpassed original Qwen3-8B performance on LiveCodeBench. With additional 112K examples, the 8B model matched performance of much larger DeepSeek-R1-671B.

Conclusion: QueST provides an effective and scalable approach to advancing competitive coding and reasoning capabilities in LLMs by generating complex problems rather than relying on limited human-labeled data.

Abstract: Large Language Models have achieved strong performance on reasoning tasks, solving competition-level coding and math problems. However, their scalability is limited by human-labeled datasets and the lack of large-scale, challenging coding problem training data. Existing competitive coding datasets contain only thousands to tens of thousands of problems. Previous synthetic data generation methods rely on either augmenting existing instruction datasets or selecting challenging problems from human-labeled data. In this paper, we propose QueST, a novel framework which combines difficulty-aware graph sampling and difficulty-aware rejection fine-tuning that directly optimizes specialized generators to create challenging coding problems. Our trained generators demonstrate superior capability compared to even GPT-4o at creating challenging problems that benefit downstream performance. We leverage QueST to generate large-scale synthetic coding problems, which we then use to distill from strong teacher models with long chain-of-thought or to conduct reinforcement learning for smaller models, proving effective in both scenarios. Our distillation experiments demonstrate significant performance gains. Specifically, after fine-tuning Qwen3-8B-base on 100K difficult problems generated by QueST, we surpass the performance of the original Qwen3-8B on LiveCodeBench. With an additional 112K examples (i.e., 28K human-written problems paired with multiple synthetic solutions), our 8B model matches the performance of the much larger DeepSeek-R1-671B. These findings indicate that generating complex problems via QueST offers an effective and scalable approach to advancing the frontiers of competitive coding and reasoning for large language models.

Method: Two key innovations: (1) new instruction tuning template with simplified output format to leverage large context windows of LLMs, (2) strategic data augmentation that preserves entity information while paraphrasing surrounding context to expand training data.

Result: Achieves performance comparable to state-of-the-art models on few-shot and zero-shot tasks, with average F1 score of 80.1 on CrossNER datasets. Models show consistent improvements of up to 17 F1 points over baselines.

Conclusion: Offers a promising solution for groups with limited NER training data and compute power by effectively leveraging few-shot learning and strategic data augmentation.

Abstract: Named Entity Recognition (NER) is a critical task that requires substantial annotated data, making it challenging in low-resource scenarios where label acquisition is expensive. While zero-shot and instruction-tuned approaches have made progress, they often fail to generalize to domain-specific entities and do not effectively utilize limited available data. We present a lightweight few-shot NER framework that addresses these challenges through two key innovations: (1) a new instruction tuning template with a simplified output format that combines principles from prior IT approaches to leverage the large context window of recent state-of-the-art LLMs; (2) introducing a strategic data augmentation technique that preserves entity information while paraphrasing the surrounding context, thereby expanding our training data without compromising semantic relationships. Experiments on benchmark datasets show that our method achieves performance comparable to state-of-the-art models on few-shot and zero-shot tasks, with our few-shot approach attaining an average F1 score of 80.1 on the CrossNER datasets. Models trained with our paraphrasing approach show consistent improvements in F1 scores of up to 17 points over baseline versions, offering a promising solution for groups with limited NER training data and compute power.

Method: Uses arXiv papers to create academic writing tasks, integrates expert-curated few-shot demonstrations from a co-author graph, and implements live evaluation to prevent label leakage.

Result: LLMs perform poorly on tasks with hierarchical abstraction levels and struggle with long few-shot demonstrations, highlighting benchmark challenges.

Conclusion: The benchmark reveals insights for enhancing LLMs’ long-context modeling capabilities and provides an effective evaluation framework for academic writing tasks.

Abstract: Large Language Models (LLMs) have recently achieved remarkable performance in long-context understanding. However, current long-context LLM benchmarks are limited by rigid context length, labor-intensive annotation, and the pressing challenge of label leakage issues during LLM training. Therefore, we propose \textsc{AcademicEval}, a live benchmark for evaluating LLMs over long-context generation tasks. \textsc{AcademicEval} adopts papers on arXiv to introduce several academic writing tasks with long-context inputs, \textit{i.e.}, \textsc{Title}, \textsc{Abstract}, \textsc{Introduction}, and \textsc{Related Work}, which cover a wide range of abstraction levels and require no manual labeling. Moreover, \textsc{AcademicEval} integrates high-quality and expert-curated few-shot demonstrations from a collected co-author graph to enable flexible context length. Especially, \textsc{AcademicEval} features an efficient live evaluation, ensuring no label leakage. We conduct a holistic evaluation on \textsc{AcademicEval}, and the results illustrate that LLMs perform poorly on tasks with hierarchical abstraction levels and tend to struggle with long few-shot demonstrations, highlighting the challenge of our benchmark. Through experimental analysis, we also reveal some insights for enhancing LLMs’ long-context modeling capabilities. Code is available at https://github.com/ulab-uiuc/AcademicEval

Method: Online reinforcement learning using a novel binary retrieval-augmented reward (RAR) that assigns reward of 1 only when output is entirely factually correct, and 0 otherwise.

Result: 39.3% reduction in hallucination rates for open-ended generation; 44.4% and 21.7% fewer incorrect answers on PopQA and GPQA respectively; learns calibrated abstention; maintains performance on instruction following, math, and code tasks.

Conclusion: Binary RAR effectively reduces hallucinations without performance degradation, outperforming both supervised training and continuous-reward RL baselines.

Abstract: Language models often generate factually incorrect information unsupported by their training data, a phenomenon known as extrinsic hallucination. Existing mitigation approaches often degrade performance on open-ended generation and downstream tasks, limiting their practical utility. We propose an online reinforcement learning method using a novel binary retrieval-augmented reward (RAR) to address this tradeoff. Unlike continuous reward schemes, our approach assigns a reward of one only when the model’s output is entirely factually correct, and zero otherwise. We evaluate our method on Qwen3 reasoning models across diverse tasks. For open-ended generation, binary RAR achieves a 39.3% reduction in hallucination rates, substantially outperforming both supervised training and continuous-reward RL baselines. In short-form question answering, the model learns calibrated abstention, strategically outputting “I don’t know” when faced with insufficient parametric knowledge. This yields 44.4% and 21.7% fewer incorrect answers on PopQA and GPQA, respectively. Crucially, these factuality gains come without performance degradation on instruction following, math, or code, whereas continuous-reward RL, despite improving factuality, induces quality regressions.

Method: Proposes a levels-of-autonomy lens (L0-L3) spanning informational tools to supervised agents, aligning existing benchmarks with permitted actions and associated risks at each level.

Result: Develops a level-conditioned blueprint for selecting metrics, assembling evidence, and reporting claims that links evaluation to oversight mechanisms.

Conclusion: By centering autonomy levels, the field can move beyond score-based claims toward credible, risk-aware evidence for real clinical implementation.

Abstract: Medical Large language models achieve strong scores on standard benchmarks; however, the transfer of those results to safe and reliable performance in clinical workflows remains a challenge. This survey reframes evaluation through a levels-of-autonomy lens (L0-L3), spanning informational tools, information transformation and aggregation, decision support, and supervised agents. We align existing benchmarks and metrics with the actions permitted at each level and their associated risks, making the evaluation targets explicit. This motivates a level-conditioned blueprint for selecting metrics, assembling evidence, and reporting claims, alongside directions that link evaluation to oversight. By centering autonomy, the survey moves the field beyond score-based claims toward credible, risk-aware evidence for real clinical use.

Method: Curated 2.5M samples across five evaluation tasks (pairwise, step-level, reference-free and reference-based verification, single rating) and trained FARE models (8B and 20B parameters) using iterative rejection-sampling supervised finetuning (SFT).

Result: FARE-8B challenges larger specialized RL-trained evaluators, FARE-20B sets new standard for open-source evaluators surpassing specialized 70B+ models. Achieves near-oracle performance on MATH as rerankers, improves RL-trained model performance by 14.1% as verifiers, and FARE-Code outperforms gpt-oss-20B by 65% on test-case quality evaluation.

Conclusion: Data scaling with simple SFT approach can produce highly effective evaluators that outperform more complex methods, demonstrating the importance of large-scale curated datasets for building foundational automatic reasoning evaluators.

Abstract: Finetuning specialized generative evaluators has emerged as a popular paradigm to meet the increasing demand for scalable evaluation during both training and test-time. However, recent work has largely focused on applying new methodology, such as reinforcement learning (RL), to training evaluators, shying away from large-scale, data-driven development. In this work, we focus on data scaling, curating a set of 2.5M samples spanning five unique evaluation tasks (pairwise, step-level, reference-free and reference-based verification, and single rating) and multiple domains focused on reasoning evaluation. With our data, we train Foundational Automatic Reasoning Evaluators (FARE), a family of 8B and 20B (with 3.6B active) parameter evaluators, with a simple iterative rejection-sampling supervised finetuning (SFT) approach. FARE-8B challenges larger specialized RL-trained evaluators and FARE-20B sets the new standard for open-source evaluators, surpassing specialized 70B+ evaluators. Beyond static benchmarks, we evaluate FARE in real-world tasks: As inference-time rerankers, FARE-20B achieves near-oracle performance on MATH. As verifiers in RL training, FARE improves the downstream RL-trained model performance by up to 14.1% vs. string-matching verifiers. When initialized from FARE, a continually-finetuned FARE-Code outperforms gpt-oss-20B by 65% on evaluating test-case quality.

Method: Multi-agent system with Master Planning Agent for query decomposition, four specialized search agents (General, Academic, GitHub, LinkedIn), MCP-based tool ecosystem for NL2SQL and file analysis, Visualization Agent, and reflection mechanism with optional human-in-the-loop guidance.

Result: Outperforms state-of-the-art agentic systems on DeepResearch Bench and DeepConsult benchmarks without human steering, enabling automated report generation and real-time streaming on internal datasets.

Conclusion: EDR framework and benchmark trajectories released to advance multi-agent reasoning research, demonstrating effective enterprise deployment for automated insight generation.

Abstract: As information grows exponentially, enterprises face increasing pressure to transform unstructured data into coherent, actionable insights. While autonomous agents show promise, they often struggle with domain-specific nuances, intent alignment, and enterprise integration. We present Enterprise Deep Research (EDR), a multi-agent system that integrates (1) a Master Planning Agent for adaptive query decomposition, (2) four specialized search agents (General, Academic, GitHub, LinkedIn), (3) an extensible MCP-based tool ecosystem supporting NL2SQL, file analysis, and enterprise workflows, (4) a Visualization Agent for data-driven insights, and (5) a reflection mechanism that detects knowledge gaps and updates research direction with optional human-in-the-loop steering guidance. These components enable automated report generation, real-time streaming, and seamless enterprise deployment, as validated on internal datasets. On open-ended benchmarks including DeepResearch Bench and DeepConsult, EDR outperforms state-of-the-art agentic systems without any human steering. We release the EDR framework and benchmark trajectories to advance research on multi-agent reasoning applications. Code at https://github.com/SalesforceAIResearch/enterprise-deep-research and Dataset at https://huggingface.co/datasets/Salesforce/EDR-200

Method: Created a corpus of 16,123 English Reddit comments from 12 subreddits, annotated by at least three trained annotators for 8 moral sentiment categories based on updated Moral Foundations Theory. Evaluated LLMs (Llama3-8B, Ministral-8B) in zero-shot, few-shot, and PEFT settings against fine-tuned BERT models.

Result: LLMs continue to lag behind fine-tuned encoder-only models like BERT on this subjective moral sentiment classification task, despite advances in large language models.

Conclusion: There is an ongoing need for human-annotated moral corpora for AI alignment evaluation, as current LLMs still underperform fine-tuned models on subjective moral sentiment tasks.

Abstract: Moral framing and sentiment can affect a variety of online and offline behaviors, including donation, environmental action, political engagement, and protest. Various computational methods in Natural Language Processing (NLP) have been used to detect moral sentiment from textual data, but achieving strong performance in such subjective tasks requires large, hand-annotated datasets. Previous corpora annotated for moral sentiment have proven valuable, and have generated new insights both within NLP and across the social sciences, but have been limited to Twitter. To facilitate improving our understanding of the role of moral rhetoric, we present the Moral Foundations Reddit Corpus, a collection of 16,123 English Reddit comments that have been curated from 12 distinct subreddits, hand-annotated by at least three trained annotators for 8 categories of moral sentiment (i.e., Care, Proportionality, Equality, Purity, Authority, Loyalty, Thin Morality, Implicit/Explicit Morality) based on the updated Moral Foundations Theory (MFT) framework. We evaluate baselines using large language models (Llama3-8B, Ministral-8B) in zero-shot, few-shot, and PEFT settings, comparing their performance to fine-tuned encoder-only models like BERT. The results show that LLMs continue to lag behind fine-tuned encoders on this subjective task, underscoring the ongoing need for human-annotated moral corpora for AI alignment evaluation. Keywords: moral sentiment annotation, moral values, moral foundations theory, multi-label text classification, large language models, benchmark dataset, evaluation and alignment resource

Method: Conducted a systematic review of current practices and performed exploratory annotation experiments across four different NLP tasks to investigate relationships between agreement measures and perceptions of subjectivity/ambiguity.

Result: Found that intra-annotator agreement is rarely reported in the field, and experiments revealed annotators provide inconsistent responses around 25% of the time across different NLP tasks.

Conclusion: Intra-annotator agreement should be calculated as important quality control that provides insights into why annotators disagree, complementing traditional inter-annotator agreement measures.

Abstract: We commonly use agreement measures to assess the utility of judgements made by human annotators in Natural Language Processing (NLP) tasks. While inter-annotator agreement is frequently used as an indication of label reliability by measuring consistency between annotators, we argue for the additional use of intra-annotator agreement to measure label stability (and annotator consistency) over time. However, in a systematic review, we find that the latter is rarely reported in this field. Calculating these measures can act as important quality control and could provide insights into why annotators disagree. We conduct exploratory annotation experiments to investigate the relationships between these measures and perceptions of subjectivity and ambiguity in text items, finding that annotators provide inconsistent responses around 25% of the time across four different NLP tasks.

Method: Created SICCK dataset by modifying 15 SICK examples using logical modifiers (quantifiers, negation), annotated with NL entailment rules, and tested NLI models in zero-shot and fine-tuned settings.

Result: NLI models performed poorly in zero-shot setting, especially for negation and existential quantifiers. After fine-tuning, models continued to struggle with negation, existential, and universal modifiers.

Conclusion: Current NLI models have significant limitations in handling compositional logical structures, particularly with negation and quantifiers, even after targeted training.

Abstract: We introduce a synthetic dataset called Sentences Involving Complex Compositional Knowledge (SICCK) and a novel analysis that investigates the performance of Natural Language Inference (NLI) models to understand compositionality in logic. We produce 1,304 sentence pairs by modifying 15 examples from the SICK dataset (Marelli et al., 2014). To this end, we modify the original texts using a set of phrases - modifiers that correspond to universal quantifiers, existential quantifiers, negation, and other concept modifiers in Natural Logic (NL) (MacCartney, 2009). We use these phrases to modify the subject, verb, and object parts of the premise and hypothesis. Lastly, we annotate these modified texts with the corresponding entailment labels following NL rules. We conduct a preliminary verification of how well the change in the structural and semantic composition is captured by neural NLI models, in both zero-shot and fine-tuned scenarios. We found that the performance of NLI models under the zero-shot setting is poor, especially for modified sentences with negation and existential quantifiers. After fine-tuning this dataset, we observe that models continue to perform poorly over negation, existential and universal modifiers.

Method: Proposes argTANL framework that converts argument structures into Augmented Natural Language (ANL) text. Introduces two variants: ME-argTANL with marker enhancement and specialized marker-based fine-tuning.

Result: Extensive experiments on three standard benchmarks show ME-argTANL achieves superior performance compared to existing methods.

Conclusion: The generative paradigm with marker enhancement provides an effective end-to-end solution for argument mining, outperforming traditional approaches.

Abstract: Argument Mining (AM) involves identifying and extracting Argumentative Components (ACs) and their corresponding Argumentative Relations (ARs). Most of the prior works have broken down these tasks into multiple sub-tasks. Existing end-to-end setups primarily use the dependency parsing approach. This work introduces a generative paradigm-based end-to-end framework argTANL. argTANL frames the argumentative structures into label-augmented text, called Augmented Natural Language (ANL). This framework jointly extracts both ACs and ARs from a given argumentative text. Additionally, this study explores the impact of Argumentative and Discourse markers on enhancing the model’s performance within the proposed framework. Two distinct frameworks, Marker-Enhanced argTANL (ME-argTANL) and argTANL with specialized Marker-Based Fine-Tuning, are proposed to achieve this. Extensive experiments are conducted on three standard AM benchmarks to demonstrate the superior performance of the ME-argTANL.

Method: Proposed FaviComp - a training-free evidence compression technique that enhances familiarity of compressed evidence to target models while integrating parametric knowledge.

Result: Outperforms recent evidence compression baselines across multiple open-domain QA datasets, improving accuracy by up to 28.1% with high compression rates.

Conclusion: FaviComp effectively integrates both parametric and non-parametric knowledge during evidence compression and consistently improves RAG performance.

Abstract: Retrieval-augmented generation (RAG) improves large language models (LMs) by incorporating non-parametric knowledge through evidence retrieved from external sources. However, it often struggles to cope with inconsistent and irrelevant information that can distract the LM from its tasks, especially when multiple evidence pieces are required. While compressing the retrieved evidence with a compression model aims to address this issue, the compressed evidence may still be unfamiliar to the target model used for downstream tasks, potentially failing to utilize the evidence effectively. We propose FaviComp (Familarity-Aware Evidence Compression), a novel training-free evidence compression technique that makes retrieved evidence more familiar to the target model, while seamlessly integrating parametric knowledge from the model. Experimental results show that FaviComp consistently outperforms most recent evidence compression baselines across multiple open-domain QA datasets, improving accuracy by up to 28.1% while achieving high compression rates. Additionally, we demonstrate the effective integration of both parametric and non-parametric knowledge during evidence compression.

Method: Pre-trained on Llama2 70B framework and fine-tuned with ~127,000 non-copyrighted ophthalmology training instances from case reports, abstracts, and study materials. Evaluated against 8 other LLMs using internal and external validation tasks with metrics including Rouge-L scores, accuracy, and expert evaluation.

Result: LEME consistently outperformed other models in internal validations (Rouge-L: 0.20-0.82) and external validations. It ranked second in MCQ accuracy (0.68) and scored highest in EHR summarization and clinical QA (4.24-4.83/5 for correctness, completeness, readability).

Conclusion: LEME represents a breakthrough in open-source ophthalmology-specific LLMs, offering potential to revolutionize clinical task execution while democratizing research collaboration through robust fine-tuning and non-copyrighted data.

Abstract: Large Language Models (LLMs) are poised to revolutionize healthcare. Ophthalmology-specific LLMs remain scarce and underexplored. We introduced an open-source, specialized LLM for ophthalmology, termed Language Enhanced Model for Eye (LEME). LEME was initially pre-trained on the Llama2 70B framework and further fine-tuned with a corpus of ~127,000 non-copyrighted training instances curated from ophthalmology-specific case reports, abstracts, and open-source study materials. We benchmarked LEME against eight other LLMs, namely, GPT-3.5, GPT-4, three Llama2 models (7B, 13B, 70B), PMC-LLAMA 13B, Meditron 70B, and EYE-Llama (another ophthalmology-specific LLM). Evaluations included four internal validation tasks: abstract completion, fill-in-the-blank, multiple-choice questions (MCQ), and short-answer QA. External validation tasks encompassed long-form QA, MCQ, patient EHR summarization, and clinical QA. Evaluation metrics included Rouge-L scores, accuracy, and expert evaluation of correctness, completeness, and readability. In internal validations, LEME consistently outperformed its counterparts, achieving Rouge-L scores of 0.20 in abstract completion (all p<0.05), 0.82 in fill-in-the-blank (all p<0.0001), and 0.22 in short-answer QA (all p<0.0001, except versus GPT-4). In external validations, LEME excelled in long-form QA with a Rouge-L of 0.19 (all p<0.0001), ranked second in MCQ accuracy (0.68; all p<0.0001), and scored highest in EHR summarization and clinical QA (ranging from 4.24 to 4.83 out of 5 for correctness, completeness, and readability). LEME’s emphasis on robust fine-tuning and the use of non-copyrighted data represents a breakthrough in open-source ophthalmology-specific LLMs, offering the potential to revolutionize execution of clinical tasks while democratizing research collaboration.

Method: Proposes SimNPO framework that removes reliance on reference model through simple preference optimization, providing advantages analyzed via mixtures of Markov chains.

Result: Extensive experiments on TOFU and MUSE benchmarks show SimNPO’s efficacy and robustness against relearning attacks, outperforming existing methods.

Conclusion: Simplicity in removing reference model dependence benefits LLM unlearning, with SimNPO providing a technically-grounded optimization framework that addresses critical issues in current state-of-the-art approaches.

Abstract: This work studies the problem of large language model (LLM) unlearning, aiming to remove unwanted data influences (e.g., copyrighted or harmful content) while preserving model utility. Despite the increasing demand for unlearning, a technically-grounded optimization framework is lacking. Gradient ascent (GA)-type methods, though widely used, are suboptimal as they reverse the learning process without controlling optimization divergence (i.e., deviation from the pre-trained state), leading to risks of over-forgetting and potential model collapse. Negative preference optimization (NPO) has been proposed to address this issue and is considered one of the state-of-the-art LLM unlearning approaches. In this work, we revisit NPO and identify another critical issue: reference model bias. This bias arises from using the reference model (i.e., the model prior to unlearning) to evaluate the unlearning success, which can compromise NPO’s effectiveness. Specifically, it leads to (a) uneven allocation of optimization power across forget data with varying difficulty levels and (b) ineffective gradient weight smoothing during the early stages of unlearning optimization. To overcome these challenges, we propose a simple yet effective unlearning optimization framework, called SimNPO, showing that `simplicity’ in removing the reliance on a reference model (through the lens of simple preference optimization) benefits unlearning. We provide deeper insights into SimNPO’s advantages through an analysis based on mixtures of Markov chains. Extensive experiments further validate SimNPO’s efficacy on benchmarks like TOFU and MUSE, as well as its robustness against relearning attacks. Codes are available at https://github.com/OPTML-Group/Unlearn-Simple.

Method: Uses mixture-of-experts (MoE)-based fusion with controllable drift modeling, drift-regularization loss, dual objective formulation for embedding distances, and gating loss to canalize expert activations.

Result: Outperforms individually aligned models by 11.37%, shows 13.77% stronger robustness than state-of-the-art LLM ensemble approaches, and 6.18% better than model-merging approaches across three benchmark datasets.

Conclusion: H3Fusion effectively addresses the challenge of multi-dimensional LLM alignment through subspace fusion and controllable drift mechanisms, providing superior performance across all three alignment dimensions.

Abstract: The alignment of pre-trained LLMs continues to draw significant attention from both industry and academia, aiming to ensure responses that are helpful, harmless, and honest. However, identifying a point in the model’s representation subspace that simultaneously satisfies all these properties remains challenging. H3Fusion addresses this challenge by introducing a mixture-of-experts (MoE)-based fusion mechanism that models alignment as a controllable drift within the subspace, guided by a drift-regularization loss to balance competing alignment dimensions. Furthermore, we formulate the alignment by finding a dual objective of harnessing the distance of generated embeddings and alignment embeddings, and introduce a gating loss by canalizing the activations on the contributing experts. Extensive evaluations of three benchmark datasets show that H3Fusion is more helpful, less harmful, and more honest in three aspects: it outperforms each individually aligned model by 11.37%, and provides stronger robustness compared to the state-of-the-art LLM ensemble approaches by 13.77% and model-merging approaches by 6.18%. Code is available at https://github.com/sftekin/h3fusion.

Method: Proposed PGO, a gradient-free prompt generation framework that uses perturbation types as pseudo-gradient signals to guide LLMs in producing robust prompts under noisy conditions.

Result: Extensive experiments show PGO consistently outperforms previous methods across diverse tasks and multiple LLMs in maintaining performance under input perturbations.

Conclusion: Explicit emphasis on robustness under noisy conditions is crucial for reliable auto-prompting, and PGO provides an effective solution for generating perturbation-resistant prompts.

Abstract: While automatic prompt generation methods have recently received significant attention, their robustness remains poorly understood. In this paper, we introduce PertBench, a comprehensive benchmark dataset that includes a wide range of input perturbations, designed to systematically evaluate the robustness of current auto-prompting techniques. Our analysis reveals substantial vulnerabilities in existing prompt generation strategies, where even minor modifications to the prompt can lead to significant differences in model output. To address this issue, we propose PGO, a gradient-free prompt generation framework that leverages perturbation types as pseudo-gradient signals to guide LLMs in producing more robust prompts. In contrast to existing methods that assess prompt quality only on clean, well-structured inputs, our approach explicitly emphasizes robustness under noisy and perturbed conditions. Extensive experiments across diverse tasks and multiple LLMs show PGO consistently outperforms previous methods in maintaining performance under input perturbations.

Method: Uses Valence-Arousal-Dominance (VAD) framework to measure emotional fingerprints in news texts, analyzes Allsides dataset, and develops NeutraSum model to neutralize identified emotional patterns in summaries.

Result: Analysis confirmed distinct emotional fingerprints for left, center, and right-leaning media. NeutraSum successfully erased partisan emotional fingerprints from summaries, achieving lower emotional bias scores than other models.

Conclusion: The work pioneers bias mitigation by shifting focus from political labels to addressing the emotional encoding of partisan bias in language, providing an evidence-driven approach to media bias reduction.

Abstract: This study introduces a novel framework for analysing and mitigating media bias by tracing partisan stances to their linguistic roots in emotional language. We posit that partisan bias is not merely an abstract stance but materialises as quantifiable ’emotional fingerprints’ within news texts. These fingerprints are systematically measured using the Valence-Arousal-Dominance (VAD) framework, allowing us to decode the affective strategies behind partisan framing. Our analysis of the Allsides dataset confirms this hypothesis, revealing distinct and statistically significant emotional fingerprints for left, centre, and right-leaning media. Based on this evidence-driven approach, we then propose a computational approach to mitigation through NeutraSum, a model designed to neutralise these identified emotional patterns. By explicitly targeting the VAD characteristics of biased language, NeutraSum generates summaries that are not only coherent but also demonstrably closer to an emotionally neutral baseline. Experimental results validate our framework: NeutraSum successfully erases the partisan emotional fingerprints from its summaries, achieving a demonstrably lower emotional bias score than other models. This work pioneers a new path for bias mitigation, shifting the focus from treating symptoms (political labels) to addressing the cause: the emotional encoding of partisan bias in language.

Method: Analyzed emotional transitions, intensity patterns, and semantic consistency using continuation and response tasks with three open-source models (Gemma, Llama3, Llama3.3) and one commercial model (Claude) on climate change discussions from Twitter and Reddit.

Result: LLMs maintain high semantic coherence but show strong tendency to moderate negative emotions, converting them to neutral or positive emotions. They generate responses with reduced emotional intensity and prefer neutral rational emotions compared to human-authored content.

Conclusion: The findings provide insights into LLMs’ emotion and semantic processing capabilities, which are significant for their deployment in social media environments and human-computer interaction design.

Abstract: Large Language Models (LLMs) demonstrate remarkable capabilities in text generation, yet their emotional consistency and semantic coherence in social media contexts remain insufficiently understood. This study investigates how LLMs handle emotional content and maintain semantic relationships through continuation and response tasks using three open-source models: Gemma, Llama3 and Llama3.3 and one commercial Model:Claude. By analyzing climate change discussions from Twitter and Reddit, we examine emotional transitions, intensity patterns, and semantic consistency between human-authored and LLM-generated content. Our findings reveal that while both models maintain high semantic coherence, they exhibit distinct emotional patterns: these models show a strong tendency to moderate negative emotions. When the input text carries negative emotions such as anger, disgust, fear, or sadness, LLM tends to generate content with more neutral emotions, or even convert them into positive emotions such as joy or surprise. At the same time, we compared the LLM-generated content with human-authored content. The four models systematically generated responses with reduced emotional intensity and showed a preference for neutral rational emotions in the response task. In addition, these models all maintained a high semantic similarity with the original text, although their performance in the continuation task and the response task was different. These findings provide deep insights into the emotion and semantic processing capabilities of LLM, which are of great significance for its deployment in social media environments and human-computer interaction design.

Method: Proposed EVOLVE framework with synergistic optimization: training methods to activate Self-Refinement capability and inference strategies to enhance and utilize it while generating training data.

Result: Llama-3.1-8B with evolved Self-Refinement surpasses GPT-4o on AlpacaEval 2 (62.3% length-controlled, 63.3% raw win rates) and Arena-Hard (50.3%), and improves performance on mathematical reasoning benchmarks (GSM8K, MATH).

Conclusion: Self-Refinement can be systematically developed through the EVOLVE framework, enabling models to refine their own responses and achieve broader self-improvement across various tasks.

Abstract: Self-Refinement refers to a model’s ability to revise its own responses to produce improved outputs. This capability can also serve as a fundamental mechanism for Self-Improvement, for example, by reconstructing datasets with refined results to enhance intrinsic model performance. However, our comprehensive experiments reveal that large language models (LLMs) show no clear evidence of inherent Self-Refinement and may even experience response quality degradation after Self-Refinement. To address this issue, we propose EVOLVE, a simple and effective framework for eliciting and tracking the evolution of Self-Refinement through iterative training. We first explore optimization methods during training to activate the model’s Self-Refinement capability. Then, at inference, we investigate various generation strategies to further enhance and utilize Self-Refinement while supplying the necessary data for training. Through synergistic optimization of training and inference stages, we continually evolve the model’s Self-Refinement ability, enabling it to better refine its own responses. Moreover, we demonstrate the potential of leveraging Self-Refinement to achieve broader Self-Improvement of intrinsic model abilities. Experiments show that the evolved Self-Refinement ability enables the Llama-3.1-8B base model to surpass GPT-4o, achieving 62.3% length-controlled and 63.3% raw win rates on AlpacaEval 2, and 50.3% on Arena-Hard. It also generalizes effectively to out-of-domain reasoning tasks, improving performance on mathematical reasoning benchmarks such as GSM8K and MATH.

Method: Analyzed 1.4M comments from 3,161 YouTube news videos, created a fine-grained hope speech classifier, conducted annotation study with 3,750 instances using diverse political representation, and assessed zero-shot LLM performance.

Result: Found strong association between rater political beliefs and content rating, models trained on individual political beliefs show significant disagreement in real-world settings, and LLMs align more with liberal raters.

Conclusion: Political beliefs significantly influence how content about marginalized communities is rated, highlighting challenges in developing unbiased content moderation systems.

Abstract: This paper makes three contributions. First, via a substantial corpus of 1,419,047 comments posted on 3,161 YouTube news videos of major US cable news outlets, we analyze how users engage with LGBTQ+ news content. Our analyses focus both on positive and negative content. In particular, we construct a fine-grained hope speech classifier that detects positive (hope speech), negative, neutral, and irrelevant content. Second, in consultation with a public health expert specializing on LGBTQ+ health, we conduct an annotation study with a balanced and diverse political representation and release a dataset of 3,750 instances with fine-grained labels and detailed annotator demographic information. Finally, beyond providing a vital resource for the LGBTQ+ community, our annotation study and subsequent in-the-wild assessments reveal (1) strong association between rater political beliefs and how they rate content relevant to a marginalized community; (2) models trained on individual political beliefs exhibit considerable in-the-wild disagreement; and (3) zero-shot large language models (LLMs) align more with liberal raters.

Method: LLaDA uses a diffusion model with forward data masking and reverse generation processes, parameterized by a Transformer to predict masked tokens, trained under pre-training and supervised fine-tuning paradigms.

Result: LLaDA performs comparably to autoregressive baselines across general tasks, math, and code, with LLaDA 8B being competitive with LLaMA3 8B in in-context learning and instruction-following. It also addresses the reversal curse, surpassing GPT-4o in reversal poem completion.

Conclusion: Diffusion models show promise for language modeling at scale and challenge the assumption that core LLM capabilities inherently depend on autoregressive models.

Abstract: The capabilities of large language models (LLMs) are widely regarded as relying on autoregressive models (ARMs). We challenge this notion by introducing LLaDA, a diffusion model trained from scratch under the pre-training and supervised fine-tuning (SFT) paradigm. LLaDA employs a forward data masking process and a reverse generation process, parameterized by a Transformer to predict masked tokens. It provides a principled generative approach for probabilistic inference by optimizing a likelihood lower bound. Across extensive benchmarks on general tasks, math, code, and so on, LLaDA demonstrates strong scalability and performs comparably to our self-constructed ARM baselines. Remarkably, LLaDA 8B is competitive with strong LLMs like LLaMA3 8B in in-context learning and, after SFT, exhibits impressive instruction-following abilities in case studies such as multi-turn dialogue. Moreover, LLaDA addresses the reversal curse, surpassing GPT-4o in a reversal poem completion task. Our findings show the promise of diffusion models for language modeling at scale and challenge the common assumption that core LLM capabilities discussed above inherently depend on ARMs. Project page and codes: https://ml-gsai.github.io/LLaDA-demo/.

Method: Proposes GRIFFIN framework with token-alignable training strategy using loss masking to exclude misaligned tokens, and token-alignable draft model that introduces input tokens to correct feature inconsistencies.

Result: Experiments on LLaMA, Vicuna, Qwen and Mixtral models show average 8% acceptance length improvement and over 7% speedup ratio, outperforming current state-of-the-art speculative decoding methods.

Conclusion: GRIFFIN effectively mitigates token misalignment in speculative decoding, providing significant performance improvements for LLM inference acceleration.

Abstract: Speculative decoding accelerates inference in large language models (LLMs) by generating multiple draft tokens simultaneously. However, existing methods often struggle with token misalignment between the training and decoding phases, limiting their performance. To address this, we propose GRIFFIN, a novel framework that incorporates a token-alignable training strategy and a token-alignable draft model to mitigate misalignment. The training strategy employs a loss masking mechanism to exclude highly misaligned tokens during training, preventing them from negatively impacting the draft model’s optimization. The token-alignable draft model introduces input tokens to correct inconsistencies in generated features. Experiments on LLaMA, Vicuna, Qwen and Mixtral models demonstrate that GRIFFIN achieves an average acceptance length improvement of over 8% and a speedup ratio exceeding 7%, outperforming current speculative decoding state-of-the-art methods. Our code and GRIFFIN’s draft models are released publicly in https://github.com/hsj576/GRIFFIN.

Method: Created EHOP dataset with NP-hard problems in three formats: textbook versions, real-life scenarios, and inverted rules. Tested state-of-the-art LLMs with multiple prompting strategies.

Result: LLMs systematically solve textbook problems more accurately than real-life and inverted versions. Reasoning models show high variance across problem presentations.

Conclusion: LLMs are heavily dependent on training data and struggle to generalize to novel problems, lacking truly robust reasoning mechanisms.

Abstract: To investigate the effect of problem presentation on LLMs’ ability to solve optimization problems, we introduce the dataset of Everyday Hard Optimization Problems (EHOP), a collection of NP-hard problems expressed in natural language. EHOP includes problem formulations that could be found in computer science textbooks (e.g., graph coloring), versions that are dressed up as problems that could arise in real life (e.g., party planning), and variants with inverted rules. We find that state-of-the-art LLMs, across multiple prompting strategies, systematically solve textbook problems more accurately than their real-life and inverted counterparts. While reasoning models are more capable, they nonetheless show high variance across problem presentations, suggesting they lack a truly robust reasoning mechanism. We argue that this constitutes evidence that LLMs are still heavily dependent on what was seen in training and struggle to generalize to novel problems.

Method: Developed Russian Poetry Scansion Tool library for stress mark placement in Russian syllabo-tonic poetry, rhyme detection, and poetic defect identification. Also created RIFMA dataset of annotated poem fragments.

Result: Released a comprehensive tool library and annotated dataset that can evaluate large language models’ capability to place stress marks accurately in poetic texts.

Conclusion: These resources provide valuable tools for researchers and practitioners in creative generative AI, facilitating advancements in generative poetry system development and evaluation.

Abstract: Generative poetry systems require effective tools for data engineering and automatic evaluation, particularly to assess how well a poem adheres to versification rules, such as the correct alternation of stressed and unstressed syllables and the presence of rhymes. In this work, we introduce the Russian Poetry Scansion Tool library designed for stress mark placement in Russian-language syllabo-tonic poetry, rhyme detection, and identification of defects of poeticness. Additionally, we release RIFMA – a dataset of poem fragments spanning various genres and forms, annotated with stress marks. This dataset can be used to evaluate the capability of modern large language models to accurately place stress marks in poetic texts. The published resources provide valuable tools for researchers and practitioners in the field of creative generative AI, facilitating advancements in the development and evaluation of generative poetry systems.

Method: Uses a dual-encoder framework extracting complementary time series and image features, cross-modal alignment for multimodal understanding, and knowledge-guided instruction generation for generating high-granularity grounding data (ECG-Grounding) linking diagnoses to measurable parameters.

Result: Significantly improves predictive performance (CSN 7.4% ↑), explainability (22.7% ↑), and grounding (24.8% ↑) on both existing and proposed benchmarks.

Conclusion: GEM makes ECG interpretation more suitable for real-world clinical applications by enhancing multimodal understanding and providing evidence-driven reasoning similar to clinicians.

Abstract: While recent multimodal large language models (MLLMs) have advanced automated ECG interpretation, they still face two key limitations: (1) insufficient multimodal synergy between time series signals and visual ECG representations, and (2) limited explainability in linking diagnoses to granular waveform evidence. We introduce GEM, the first MLLM unifying ECG time series, 12-lead ECG images and text for grounded and clinician-aligned ECG interpretation. GEM enables feature-grounded analysis, evidence-driven reasoning, and a clinician-like diagnostic process through three core innovations: a dual-encoder framework extracting complementary time series and image features, cross-modal alignment for effective multimodal understanding, and knowledge-guided instruction generation for generating high-granularity grounding data (ECG-Grounding) linking diagnoses to measurable parameters ($e.g.$, QRS/PR Intervals). Additionally, we propose the Grounded ECG Understanding task, a clinically motivated benchmark designed to comprehensively assess the MLLM’s capability in grounded ECG understanding. Experimental results on both existing and our proposed benchmarks show GEM significantly improves predictive performance (CSN $7.4% \uparrow$), explainability ($22.7% \uparrow$), and grounding ($24.8% \uparrow$), making it more suitable for real-world clinical applications. GitHub repository: https://github.com/lanxiang1017/GEM.git

Method: Conducted an online survey in the NLP community to collect insights on implementation practices, obstacles, and future prospects of active learning.

Result: Data annotation expected to remain important; active learning stays relevant with LLMs; three key challenges persist from 15 years ago: setup complexity, uncertain cost reduction, and tooling.

Conclusion: Active learning continues to face adoption barriers despite LLM advances; proposed strategies to alleviate persistent challenges; dataset published for further research.

Abstract: Supervised learning relies on data annotation which usually is time-consuming and therefore expensive. A longstanding strategy to reduce annotation costs is active learning, an iterative process, in which a human annotates only data instances deemed informative by a model. Research in active learning has made considerable progress, especially with the rise of large language models (LLMs). However, we still know little about how these remarkable advances have translated into real-world applications, or contributed to removing key barriers to active learning adoption. To fill in this gap, we conduct an online survey in the NLP community to collect previously intangible insights on current implementation practices, common obstacles in application, and future prospects in active learning. We also reassess the perceived relevance of data annotation and active learning as fundamental assumptions. Our findings show that data annotation is expected to remain important and active learning to stay relevant while benefiting from LLMs. Consistent with a community survey from over 15 years ago, three key challenges yet persist – setup complexity, uncertain cost reduction, and tooling – for which we propose alleviation strategies. We publish an anonymized version of the dataset.

Method: Evaluated GenRM against SC under fixed inference budgets across diverse models and datasets, and derived inference scaling laws for the GenRM paradigm.

Result: SC outperforms GenRM for most practical compute budgets. GenRM first matches SC after consuming up to 8x more inference compute and requires significantly more compute to outperform SC.

Conclusion: Compute-optimal inference favors scaling solution generation more aggressively than verification. SC remains the more efficient approach for most practical scenarios.

Abstract: Scaling test-time compute has emerged as a key strategy for enhancing the reasoning capabilities of large language models (LLMs), particularly in tasks like mathematical problem-solving. A traditional approach, Self-Consistency (SC), generates multiple solutions to a problem and selects the most common answer via majority voting. Another common method involves scoring each solution with a reward model (verifier) and choosing the best one. Recent advancements in Generative Reward Models (GenRM) reframe verification as a next-token prediction task, enabling inference-time scaling along a new axis. Specifically, GenRM generates multiple verification chains-of-thought to score each solution. Under a limited inference budget, this introduces a fundamental trade-off: should you spend the budget on scaling solutions via SC or generate fewer solutions and allocate compute to verification via GenRM? To address this, we evaluate GenRM against SC under a fixed inference budget. Interestingly, we find that SC is more compute-efficient than GenRM for most practical inference budgets across diverse models and datasets. For instance, GenRM first matches SC after consuming up to 8x the inference compute and requires significantly more compute to outperform it. Furthermore, we derive inference scaling laws for the GenRM paradigm, revealing that compute-optimal inference favors scaling solution generation more aggressively than scaling the number of verifications. Our work provides practical guidance on optimizing test-time scaling by balancing solution generation and verification. The code is available at https://github.com/nishadsinghi/sc-genrm-scaling.

Method: Uses well-learned classifiers to assign calibration values to training samples based on alignment with target human-preferred distribution, then incorporates these into a robust optimization objective that minimizes worst-case loss over preference-relevant data regions.

Result: The approach mitigates distributional mismatch and improves generation of responses that better reflect intended human values.

Conclusion: Distribution-aware optimization effectively addresses distribution shifts in preference alignment, enabling LLMs to produce outputs more consistent with human preferences despite reliance on synthetic training data.

Abstract: Preference alignment methods are increasingly critical for steering large language models (LLMs) to generate outputs consistent with human values. While recent approaches often rely on synthetic data generated by LLMs for scalability and cost-efficiency reasons, this reliance can introduce distribution shifts that undermine the nuanced representation of human preferences needed for desirable outputs. In this paper, we propose a novel distribution-aware optimization framework that improves preference alignment despite such shifts. Our approach first leverages well-learned classifiers to assign a calibration value to each training sample, quantifying its alignment with the target human-preferred distribution. These values are then incorporated into a robust optimization objective that minimizes the worst-case loss over regions of the data space most relevant to human preferences. By explicitly focusing optimization on the target distribution, our approach mitigates the impact of distributional mismatch and improves the generation of responses that better reflect intended values.

Method: Analyzed verbalized confidence as a lens into self-reflection, examining effects of supervised fine-tuning on reasoning traces (distillation) and reinforcement learning on verbalized calibration in reasoning-intensive settings.

Result: Training improved verbalized calibration progressively but reduced “I don’t know” response rates on factuality benchmarks. Models expressed higher confidence with shorter reasoning chains, indicating diminished knowledge boundary awareness.

Conclusion: Reasoning-oriented training enhances reasoning performance but incurs a “reasoning tax” - reduced ability to recognize knowledge limits, compromising model faithfulness through overconfidence without proper abstention awareness.

Abstract: Large reasoning models (LRMs) have recently demonstrated impressive capabilities in complex reasoning tasks by leveraging increased test-time computation and exhibiting behaviors reminiscent of human-like self-reflection. While LRMs show a clear capacity for valuable self-reflection, how this ability interacts with other model behaviors remains underexplored. We investigate this connection by analyzing verbalized confidence, how models articulate their certainty, as a lens into the nature of self-reflection in LRMs. We find that supervised fine-tuning on reasoning traces (i.e., distillation) and reinforcement learning can improve verbalized calibration in reasoning-intensive settings in a progressive, laddered fashion. However, our results also indicate that reasoning models may possess a diminished awareness of their own knowledge boundaries, as evidenced by significantly lower “I don’t know” response rates on factuality benchmarks. Moreover, we examine the relationship between verbalized confidence and reasoning chains, finding that models tend to express higher confidence when providing shorter or less elaborate reasoning. Our findings highlight how reasoning-oriented training can enhance performance in reasoning-centric tasks while potentially incurring a “reasoning tax,” a cost reflected in the model’s reduced ability to accurately recognize the limits of its own knowledge in small-scale models. More broadly, our work showcases how this erosion of knowledge boundaries can compromise model faithfulness, as models grow more confident without a commensurate understanding of when they should abstain.

Method: Used LLM-assisted annotation pipeline to extract timestamped clinical findings, then systematically evaluated decoder-based LLMs and encoder-based transformers on event prediction, temporal ordering, and survival analysis tasks.

Result: Encoder-based models achieved higher F1 scores and better temporal concordance for event forecasting, while fine-tuned masking improved ranking performance. Instruction-tuned decoder models showed relative advantage in survival analysis, especially for early prognosis.

Conclusion: Time ordering in clinical time series provides additional benefits beyond text ordering, with important implications for temporal tasks in the LLM era.

Abstract: Clinical case reports encode temporal patient trajectories that are often underexploited by traditional machine learning methods relying on structured data. In this work, we introduce the forecasting problem from textual time series, where timestamped clinical findings – extracted via an LLM-assisted annotation pipeline – serve as the primary input for prediction. We systematically evaluate a diverse suite of models, including fine-tuned decoder-based large language models and encoder-based transformers, on tasks of event occurrence prediction, temporal ordering, and survival analysis. Our experiments reveal that encoder-based models consistently achieve higher F1 scores and superior temporal concordance for short- and long-horizon event forecasting, while fine-tuned masking approaches enhance ranking performance. In contrast, instruction-tuned decoder models demonstrate a relative advantage in survival analysis, especially in early prognosis settings. Our sensitivity analyses further demonstrate the importance of time ordering, which requires clinical time series construction, as compared to text ordering, the format of the text inputs that LLMs are classically trained on. This highlights the additional benefit that can be ascertained from time-ordered corpora, with implications for temporal tasks in the era of widespread LLM use.

Method: Evaluated over 40 LLMs across 12 languages using cross-lingual machine reading comprehension (xMRC) as a representative scenario, analyzing mechanism through two-phase process.

Result: Post-trained open LLMs show strong cross-lingual context retrieval comparable to GPT-4o. Oracle performances greatly improve after post-training. xMRC bottleneck lies at last model layers in answer retrieval phase.

Conclusion: Larger-scale pretraining alone cannot improve xMRC performance; LLMs need multilingual post-training to fully unlock cross-lingual context retrieval potential.

Abstract: Cross-lingual context retrieval (extracting contextual information in one language based on requests in another) is a fundamental aspect of cross-lingual alignment, but the performance and mechanism of it for large language models (LLMs) remains unclear. In this paper, we evaluate the cross-lingual context retrieval of over 40 LLMs across 12 languages, using cross-lingual machine reading comprehension (xMRC) as a representative scenario. Our results show that post-trained open LLMs show strong cross-lingual context retrieval ability, comparable to closed-source LLMs such as GPT-4o, and their estimated oracle performances greatly improve after post-training. Our mechanism analysis shows that the cross-lingual context retrieval process can be divided into two main phases: question encoding and answer retrieval, which are formed in pre-training and post-training respectively. The phasing stability correlates with xMRC performance, and the xMRC bottleneck lies at the last model layers in the second phase, where the effect of post-training can be evidently observed. Our results also indicate that larger-scale pretraining cannot improve the xMRC performance. Instead, larger LLMs need further multilingual post-training to fully unlock their cross-lingual context retrieval potential.

Method: The framework leverages large language models to generate topics at multiple levels of granularity, creating hierarchical taxonomies with dedicated evaluation metrics for comprehensive assessment.

Result: Evaluations on three diverse datasets show LLMTaxo effectively produces clear, coherent, and comprehensive taxonomies, with GPT-4o mini consistently outperforming other models across most metrics.

Conclusion: LLMTaxo demonstrates flexibility and low reliance on manual intervention, highlighting its potential for broad applicability in organizing social media discourse.

Abstract: With the rapid expansion of content on social media platforms, analyzing and comprehending online discourse has become increasingly complex. This paper introduces LLMTaxo, a novel framework leveraging large language models for the automated construction of taxonomies of factual claims from social media by generating topics at multiple levels of granularity. The resulting hierarchical structure significantly reduces redundancy and improves information accessibility. We also propose dedicated taxonomy evaluation metrics to enable comprehensive assessment. Evaluations conducted on three diverse datasets demonstrate LLMTaxo’s effectiveness in producing clear, coherent, and comprehensive taxonomies. Among the evaluated models, GPT-4o mini consistently outperforms others across most metrics. The framework’s flexibility and low reliance on manual intervention underscore its potential for broad applicability.

Method: HCR-Reasoner framework uses actual causality formalisms to filter candidate causes, then applies causal judgment factors to determine psychologically selected causes. Evaluated on HCR-Bench with 1,093 annotated instances.

Result: HCR-Reasoner consistently and significantly improves LLMs’ causal alignment with humans. Explicit integration of theory-guided reasoning is highly effective for achieving faithful human-like causal reasoning.

Conclusion: Systematic integration of actual causality theory and causal judgment factors into LLMs enables more human-like causal reasoning, demonstrating the value of theory-guided approaches for strong AI.

Abstract: Genuine human-like causal reasoning is fundamental for strong artificial intelligence. Humans typically identify whether an event is part of the causal chain first, and then influenced by modulatory factors such as morality, normality, and intention to make the final judgment. These two stages naturally map to the fields of 1) actual causality that provides formalisms for causal chain membership and 2) causal judgment from cognitive science that studies psychological modulators that influence causal selection. However, these two domains have largely been studied in isolation, leaving a gap for a systematic method based on LLMs. Therefore, we introduce HCR-Reasoner, a framework that systematically integrates the theory of actual causality and causal judgment into LLMs for human-like causal reasoning. It simulates humans by using actual causality formalisms to filter for structurally necessary candidate causes and causal judgment factors to determine the psychologically selected cause. For fine-grained evaluation, we introduce HCR-Bench, a challenging benchmark with 1,093 annotated instances with detailed reasoning steps. Results show HCR-Reasoner consistently and significantly improves LLMs’ causal alignment with humans, and that explicitly integrating theory-guided reasoning into LLMs is highly effective for achieving faithful human-like causal reasoning.

Method: Analyzed prompt-induced shifts (changes in hidden representations caused by self-correction prompts) across 5 open-source LLMs, comparing shifts in text detoxification and toxification with latent directions from contrastive pairs.

Result: Prompt-induced shifts align with interpretable latent directions: in detoxification they align with non-toxic direction, in toxification with toxic direction, showing self-correction functions as representation steering beyond standard metrics.

Conclusion: Intrinsic self-correction operates through representation steering along interpretable latent directions, providing an interpretability-based account that advances systematic understanding of LLM prompting mechanisms.

Abstract: Intrinsic self-correction refers to the phenomenon where a language model refines its own outputs purely through prompting, without external feedback or parameter updates. While this approach improves performance across diverse tasks, its internal mechanism remains poorly understood. We analyze intrinsic self-correction from a representation-level perspective. We formalize and introduce the notion of a prompt-induced shift, which is the change in hidden representations caused by a self-correction prompt. Across 5 open-source LLMs, prompt-induced shifts in text detoxification and text toxification align with latent directions constructed from contrastive pairs. In detoxification, the shifts align with the non-toxic direction; in toxification, they align with the toxic direction. These results suggest that intrinsic self-correction functions as representation steering along interpretable latent directions, beyond what standard metrics such as task scores or model confidence capture. Our analysis offers an interpretability-based account of intrinsic self-correction and contributes to a more systematic understanding of LLM prompting.

Method: PsyMem supplements textual descriptions with 26 psychological indicators and implements memory alignment training to explicitly align character responses with memory, using a dataset of 5,414 characters and 38,962 dialogues from novels.

Result: The resulting PsyMem-Qwen model (trained on Qwen2.5-7B-Instruct) outperforms baseline models in role-playing, achieving the best performance in human-likeness and character fidelity.

Conclusion: PsyMem successfully addresses key limitations in role-playing LLMs through psychological attribute modeling and explicit memory control, enhancing reliability and performance.

Abstract: Existing LLM-based role-playing methods often rely on superficial textual descriptions or simplistic metrics, inadequately modeling both intrinsic and extrinsic character dimensions. Additionally, they typically simulate character memory with implicit model knowledge or basic retrieval augment generation without explicit memory alignment, compromising memory consistency. The two issues weaken reliability of role-playing LLMs in several applications, such as trustworthy social simulation. To address these limitations, we propose PsyMem, a novel framework integrating fine-grained psychological attributes and explicit memory control for role-playing. PsyMem supplements textual descriptions with 26 psychological indicators to detailed model character. Additionally, PsyMem implements memory alignment training, explicitly trains the model to align character’s response with memory, thereby enabling dynamic memory-controlled responding during inference. By training Qwen2.5-7B-Instruct on our specially designed dataset (including 5,414 characters and 38,962 dialogues extracted from novels), the resulting model, termed as PsyMem-Qwen, outperforms baseline models in role-playing, achieving the best performance in human-likeness and character fidelity.

Method: Constructs a problem-solving template library via MCTS on a small seed set, then integrates this high-level structured guidance into RL training to align rollout generation with proven template structures.

Result: Outperforms GRPO by 99% on AIME and 41% on AMC, with superior stability on weak models and remarkable cross-domain generalization.

Conclusion: TemplateRL’s structure-guided design effectively steers policies toward validated strategic patterns, stabilizing training and enhancing sampling efficiency while maintaining interpretability and editability.

Abstract: Reinforcement learning (RL) has emerged as an effective paradigm for enhancing model reasoning. However, existing RL methods like GRPO often rely on unstructured self-sampling to fit scalar rewards, often producing inefficient rollouts that fail to capture transferable problem-solving strategies. To address these limitations, we propose TemplateRL, a structured template-guided RL framework that augments policy optimization with explicit template guidance. Our approach first constructs a problem-solving template library via MCTS on a small seed set, then seamlessly integrates this high-level structured guidance into RL training. By guiding rollout generation to align with proven template structures, TemplateRL significantly improves high-quality trajectory hit rates while reducing ineffective exploration. This structure-guided design steers the policy toward validated strategic patterns, stabilizing training dynamics, and enhancing RL sampling efficiency. Notably, the explicit template library is interpretable, editable, and supports online updates-enabling continuous updates during both training and inference. Extensive experiments demonstrate that TemplateRL outperforms GRPO by 99% on AIME and 41% on AMC, with superior stability on weak models and remarkable cross-domain generalization, highlighting its potential for broader tasks.

Method: Developed PaSBench with 416 multimodal scenarios (128 image sequences, 288 text logs) across 5 safety-critical domains, and evaluated 36 advanced AI models on their proactive risk detection capabilities.

Result: Top performers like Gemini-2.5-pro achieved 71% image and 64% text accuracy, but missed 45-55% of risks in repeated trials. Failure analysis identified unstable proactive reasoning as the primary limitation rather than knowledge deficits.

Conclusion: This work establishes a proactive safety benchmark, provides systematic evidence of model limitations, and identifies critical directions for developing reliable protective AI that actively prevents harm rather than merely responding to requests.

Abstract: Human safety awareness gaps often prevent the timely recognition of everyday risks. In solving this problem, a proactive safety artificial intelligence (AI) system would work better than a reactive one. Instead of just reacting to users’ questions, it would actively watch people’s behavior and their environment to detect potential dangers in advance. Our Proactive Safety Bench (PaSBench) evaluates this capability through 416 multimodal scenarios (128 image sequences, 288 text logs) spanning 5 safety-critical domains. Evaluation of 36 advanced models reveals fundamental limitations: Top performers like Gemini-2.5-pro achieve 71% image and 64% text accuracy, but miss 45-55% risks in repeated trials. Through failure analysis, we identify unstable proactive reasoning rather than knowledge deficits as the primary limitation. This work establishes (1) a proactive safety benchmark, (2) systematic evidence of model limitations, and (3) critical directions for developing reliable protective AI. We believe our dataset and findings can promote the development of safer AI assistants that actively prevent harm rather than merely respond to requests. Our dataset can be found at https://huggingface.co/datasets/Youliang/PaSBench.

Method: Proposes a pipeline that decomposes medical answers into condition-aware valid facts and verifies them against in-domain medical corpora, extracting more facts while reducing hallucinations and vague references.

Result: Extracts up to 3 times more valid facts than existing methods, reduces hallucination and vague references, and retains condition-dependency in facts. Factuality scores vary significantly by decomposition method, verification corpus, and LLM backbone.

Conclusion: Customizing each evaluation step is crucial for reliable factuality assessment in medical domain, and the proposed modular pipeline enables effective domain adaptation.

Abstract: While Large Language Models (LLMs) can generate fluent and convincing responses, they are not necessarily correct. This is especially apparent in the popular decompose-then-verify factuality evaluation pipeline, where LLMs evaluate generations by decomposing the generations into individual, valid claims. Factuality evaluation is especially important for medical answers, since incorrect medical information could seriously harm the patient. However, existing factuality systems are a poor match for the medical domain, as they are typically only evaluated on objective, entity-centric, formulaic texts such as biographies and historical topics. This differs from condition-dependent, conversational, hypothetical, sentence-structure diverse, and subjective medical answers, which makes decomposition into valid facts challenging. We propose MedScore, a new pipeline to decompose medical answers into condition-aware valid facts and verify against in-domain corpora. Our method extracts up to three times more valid facts than existing methods, reducing hallucination and vague references, and retaining condition-dependency in facts. The resulting factuality score substantially varies by decomposition method, verification corpus, and used backbone LLM, highlighting the importance of customizing each step for reliable factuality evaluation by using our generalizable and modularized pipeline for domain adaptation.

Method: Leverages structured textual reasoning explanations that inherently encode generalizable decision-making patterns, enabling effective transfer to multimodal judgments with images, videos, and other modalities using minimal text data.

Result: Achieves competitive or superior performance compared to state-of-the-art commercial APIs and extensively trained multimodal evaluators, despite significantly less training data. Shows broad impact in resource-constrained domains like molecule evaluation.

Conclusion: Reasoning-based text supervision provides a powerful, cost-effective alternative to traditional annotation-intensive approaches, substantially advancing scalable multimodal model-as-a-judge frameworks.

Abstract: Human-generated reward signals are critical for aligning generative models with human preferences, guiding both training and inference-time evaluations. While large language models (LLMs) employed as proxy evaluators, i.e., LLM-as-a-Judge, significantly reduce the costs associated with manual annotations, they typically require extensive modality-specific training data and fail to generalize well across diverse multimodal tasks. In this paper, we propose Flex-Judge, a reasoning-guided multimodal judge model that leverages minimal textual reasoning data to robustly generalize across multiple modalities and evaluation formats. Our core intuition is that structured textual reasoning explanations inherently encode generalizable decision-making patterns, enabling an effective transfer to multimodal judgments, e.g., with images or videos. Empirical results demonstrate that Flex-Judge, despite being trained on significantly fewer text data, achieves competitive or superior performance compared to state-of-the-art commercial APIs and extensively trained multimodal evaluators. Notably, Flex-Judge presents broad impact in modalities like molecule, where comprehensive evaluation benchmarks are scarce, underscoring its practical value in resource-constrained domains. Our framework highlights reasoning-based text supervision as a powerful, cost-effective alternative to traditional annotation-intensive approaches, substantially advancing scalable multimodal model-as-a-judge.

Method: Analyzed geometric factors (separability and alignment of query hidden states) in classification tasks, conducted fine-grained analysis of layer-wise dynamics, and performed ablation studies on attention heads and task vectors.

Result: Revealed a two-stage mechanism: separability emerges in early layers, while alignment develops in later layers. Previous Token Heads drive separability, while Induction Heads and task vectors enhance alignment.

Conclusion: The findings bridge the gap between attention heads and task vectors, offering a unified account of in-context learning’s underlying mechanisms.

Abstract: The unusual properties of in-context learning (ICL) have prompted investigations into the internal mechanisms of large language models. Prior work typically focuses on either special attention heads or task vectors at specific layers, but lacks a unified framework linking these components to the evolution of hidden states across layers that ultimately produce the model’s output. In this paper, we propose such a framework for ICL in classification tasks by analyzing two geometric factors that govern performance: the separability and alignment of query hidden states. A fine-grained analysis of layer-wise dynamics reveals a striking two-stage mechanism: separability emerges in early layers, while alignment develops in later layers. Ablation studies further show that Previous Token Heads drive separability, while Induction Heads and task vectors enhance alignment. Our findings thus bridge the gap between attention heads and task vectors, offering a unified account of ICL’s underlying mechanisms.

Method: Proposes Conditional Pointwise Mutual Information (C-PMI) calibrated decoding that jointly models visual and textual token contributions, formulated as bi-level optimization with token purification mechanism to dynamically regulate decoding.

Result: Extensive experiments show significant reduction in hallucinations across various benchmarks while preserving decoding efficiency.

Conclusion: The C-PMI method effectively mitigates hallucinations in LVLMs by adaptively strengthening visual-textual mutual dependency through joint optimization of image and text token contributions.

Abstract: Large Vision-Language Models (LVLMs) are susceptible to hallucinations, where generated responses seem semantically plausible yet exhibit little or no relevance to the input image. Previous studies reveal that this issue primarily stems from LVLMs’ over-reliance on language priors while disregarding the visual information during decoding. To alleviate this issue, we introduce a novel Conditional Pointwise Mutual Information (C-PMI) calibrated decoding strategy, which adaptively strengthens the mutual dependency between generated texts and input images to mitigate hallucinations. Unlike existing methods solely focusing on text token sampling, we propose to jointly model the contributions of visual and textual tokens to C-PMI, formulating hallucination mitigation as a bi-level optimization problem aimed at maximizing mutual information. To solve it, we design a token purification mechanism that dynamically regulates the decoding process by sampling text tokens remaining maximally relevant to the given image, while simultaneously refining image tokens most pertinent to the generated response. Extensive experiments across various benchmarks reveal that the proposed method significantly reduces hallucinations in LVLMs while preserving decoding efficiency.

Method: Frames alignment as importance sampling with upstream model as proposal distribution and alignment module as importance weight estimator. Uses sequence-level training and iterative token-level decoding to address latency.

Result: Experimental evaluations on open-source LLMs across instruction following, domain adaptation, and preference optimization tasks show consistent outperformance over baselines.

Conclusion: RAM provides a flexible and scalable approach to LLM alignment that improves performance while addressing practical deployment challenges like latency.

Abstract: The widespread adoption of large language models (LLMs) across industries has increased the demand for high-quality and customizable outputs. However, traditional alignment methods often require retraining large pretrained models, making it difficult to quickly adapt and optimize LLMs for diverse applications. To address this limitation, we propose a novel \textit{Residual Alignment Model} (\textit{RAM}) that formalizes the alignment process as a type of importance sampling. In this framework, the unaligned upstream model serves as the proposal distribution, while the alignment process is framed as secondary sampling based on an autoregressive alignment module that acts as an estimator of the importance weights. This design enables a natural detachment of the alignment module from the target aligned model, improving flexibility and scalability. Based on this model, we derive an efficient sequence-level training strategy for the alignment module, which operates independently of the proposal module. Additionally, we develop a resampling algorithm with iterative token-level decoding to address the common first-token latency issue in comparable methods. Experimental evaluations on two leading open-source LLMs across diverse tasks, including instruction following, domain adaptation, and preference optimization, demonstrate that our approach consistently outperforms baseline models.

Method: Incorporates two theory of mind modules: prompting to consider possible objections, and using a text encoder with MLP classifier to predict opponent’s stance on counterclaims. Uses reinforcement learning to learn how to analyze opponent information and generate effective arguments.

Result: ToMAP outperforms much larger baselines like GPT-4o with 39.4% relative gain across multiple persuadee models and diverse corpora. Exhibits complex reasoning chains, reduced repetition, and more diverse/effective arguments. Suitable for long conversations with logical, opponent-aware strategies.

Conclusion: ToMAP effectively addresses LLMs’ Theory of Mind limitations in persuasion, demonstrating superior performance and highlighting potential for developing more persuasive language agents through enhanced opponent modeling.

Abstract: Large language models (LLMs) have shown promising potential in persuasion, but existing works on training LLM persuaders are still preliminary. Notably, while humans are skilled in modeling their opponent’s thoughts and opinions proactively and dynamically, current LLMs struggle with such Theory of Mind (ToM) reasoning, resulting in limited diversity and opponent awareness. To address this limitation, we introduce Theory of Mind Augmented Persuader (ToMAP), a novel approach for building more flexible persuader agents by incorporating two theory of mind modules that enhance the persuader’s awareness and analysis of the opponent’s mental state. Specifically, we begin by prompting the persuader to consider possible objections to the target central claim, and then use a text encoder paired with a trained MLP classifier to predict the opponent’s current stance on these counterclaims. Our carefully designed reinforcement learning schema enables the persuader learns how to analyze opponent-related information and utilize it to generate more effective arguments. Experiments show that the ToMAP persuader, while containing only 3B parameters, outperforms much larger baselines, like GPT-4o, with a relative gain of 39.4% across multiple persuadee models and diverse corpora. Notably, ToMAP exhibits complex reasoning chains and reduced repetition during training, which leads to more diverse and effective arguments. The opponent-aware feature of ToMAP also makes it suitable for long conversations and enables it to employ more logical and opponent-aware strategies. These results underscore our method’s effectiveness and highlight its potential for developing more persuasive language agents. Code is available at: https://github.com/ulab-uiuc/ToMAP.

Method: Formulates reasoning as a search tree where nodes represent reasoning spans. Uses A* search algorithm with a cost function for reasoning paths and a bidirectional importance estimation mechanism to efficiently compress chains of thought.

Result: Improves QwQ-32B performance by 2.39× with low budget and reduces output token length by nearly 50% with high budget. Shows compatibility with multiple LRMs and effective balance between performance and efficiency on math tasks.

Conclusion: A*-Thought provides an effective framework for compressing reasoning chains in Large Reasoning Models, achieving better performance-efficiency trade-offs than existing methods while maintaining generalization across different models.

Abstract: Large Reasoning Models (LRMs) achieve superior performance by extending the thought length. However, a lengthy thinking trajectory leads to reduced efficiency. Most of the existing methods are stuck in the assumption of overthinking and attempt to reason efficiently by compressing the Chain-of-Thought, but this often leads to performance degradation. To address this problem, we introduce A*-Thought, an efficient tree search-based unified framework designed to identify and isolate the most essential thoughts from the extensive reasoning chains produced by these models. It formulates the reasoning process of LRMs as a search tree, where each node represents a reasoning span in the giant reasoning space. By combining the A* search algorithm with a cost function specific to the reasoning path, it can efficiently compress the chain of thought and determine a reasoning path with high information density and low cost. In addition, we also propose a bidirectional importance estimation mechanism, which further refines this search process and enhances its efficiency beyond uniform sampling. Extensive experiments on several advanced math tasks show that A*-Thought effectively balances performance and efficiency over a huge search space. Specifically, A*-Thought can improve the performance of QwQ-32B by 2.39$\times$ with low-budget and reduce the length of the output token by nearly 50% with high-budget. The proposed method is also compatible with several other LRMs, demonstrating its generalization capability. The code can be accessed at: https://github.com/AI9Stars/AStar-Thought.

Method: Created SATA-BENCH benchmark for SATA questions across diverse domains. Evaluated 27 models, identified selection bias and count bias as core challenges. Proposed Choice Funnel decoding strategy combining token debiasing with adaptive thresholding.

Result: Even the strongest model achieved only 41.8% exact match on SATA questions. Choice Funnel achieved up to 29% higher exact match than baselines while reducing inference cost by over 64%.

Conclusion: Current LLMs have fundamental limitations in multi-answer reasoning. Choice Funnel provides an effective framework for improving performance on SATA tasks while reducing computational costs.

Abstract: Large language models (LLMs) are increasingly evaluated on single-answer multiple-choice tasks, yet many real-world problems require identifying all correct answers from a set of options. This capability remains underexplored. We introduce SATA-BENCH, the first dedicated benchmark for evaluating LLMs on Select All That Apply (SATA) questions across diverse domains, including reading comprehension, law, and biomedicine. Our evaluation of 27 open-source and proprietary models reveals a significant gap: even the strongest model achieves only 41.8% exact match, exposing LLMs’ inability to reliably identify all correct answers. We find that this weakness stems from two core challenges: selection bias - models favor certain choices regardless of content, and count bias - models fail to predict the correct number of answers. To address these issues, we propose Choice Funnel, a decoding strategy that combines token debiasing with adaptive thresholding to guide models toward complete and accurate selections. Choice Funnel achieves up to 29% higher exact match than competitive baselines while reducing inference cost by over 64%. Our findings expose fundamental limitations in current LLMs and introduce a new framework for diagnosing and improving multi-answer reasoning. We release SATA-BENCH and Choice Funnel to promote LLM development for robust decision-making in realistic, multi-answer applications.

Method: KG-TRACES jointly supervises LLMs to predict symbolic relation paths, full triple-level reasoning paths, and generate attribution-aware reasoning processes grounded in knowledge graphs.

Result: Significant performance improvements: 1.6% Hits@1 and 4.7% F1 on WebQSP; 4.8% Hits@1 and 2.1% F1 on CWQ. Also shows transferability to domains like medicine and produces more stable reasoning processes.

Conclusion: KG-TRACES enables explainable, source-attributable reasoning by explicitly supervising reasoning paths, outperforming SOTA methods and providing more transparent reasoning processes.

Abstract: Large language models (LLMs) have made remarkable strides in various natural language processing tasks, but their performance on complex reasoning problems remains hindered by a lack of explainability and trustworthiness. This issue, often manifesting as hallucinations or unattributable reasoning processes, limits their applicability in complex reasoning scenarios. To address this, we propose Knowledge Graph-constrained Trajectory Reasoning Attribution and Chain Explanation Supervision (KG-TRACES), a novel framework that enhances the reasoning ability of LLMs through explicit supervision over reasoning paths and processes. KG-TRACES jointly supervises the model to: (1) predict symbolic relation paths, (2) predict full triple-level reasoning paths, and (3) generate attribution-aware reasoning processes grounded in the reasoning paths. At inference phase, the model adapts to both KG-available and KG-unavailable scenarios, retrieving reasoning paths from a KG when possible or predicting plausible reasoning paths with only intrinsic knowledge when not. This design enables the model to reason in an explainable and source-attributable pattern. Through extensive experiments on complex reasoning tasks, we demonstrate that KG-TRACES significantly outperforms existing SOTA: it improves Hits@1 by 1.6% and F1 by 4.7% on WebQSP, and achieves improvements of 4.8% in Hits@1 and 2.1% in F1 on CWQ. Moreover, we show its transferability to specialized domains such as medicine. By visualizing the intermediate steps of reasoning processes, we further show that the explicit supervision introduced by KG-TRACES leads to more stable and goal-directed reasoning processes, aligning closely with correct answers. Code is available at https://github.com/Edaizi/KG-TRACES.

Method: The study introduces LongBioBench, which uses artificially generated biographies as a controlled environment for assessing LCLMs. It focuses on three essential features: seamless context, controllable setting, and sound evaluation across dimensions of understanding, reasoning, and trustworthiness.

Result: Evaluation of 18 LCLMs shows most models still exhibit deficiencies in semantic understanding and elementary reasoning over retrieved results, and become less trustworthy as context length increases. Analysis reveals existing synthetic benchmarks’ design choices make them vulnerable for testing long-context capabilities.

Conclusion: LongBioBench achieves a better trade-off between mirroring authentic language tasks and maintaining controllability compared to previous synthetic benchmarks, and is highly interpretable and configurable.

Abstract: Existing frameworks for evaluating long-context language models (LCLM) can be broadly categorized into real-world applications (e.g, document summarization) and synthetic tasks (e.g, needle-in-a-haystack). Despite their utility, both approaches are accompanied by certain intrinsic limitations. Real-world tasks often involve complexity that makes interpretation challenging and suffer from data contamination, whereas synthetic tasks frequently lack meaningful coherence between the target information (needle) and its surrounding context (haystack), undermining their validity as proxies for realistic applications. In response to these challenges, we posit that an ideal long-context evaluation framework should be characterized by three essential features: 1) seamless context 2) controllable setting and 3) sound evaluation. This study introduces $\textbf{LongBioBench}$, a benchmark that utilizes artificially generated biographies as a controlled environment for assessing LCLMs across dimensions of understanding, reasoning, and trustworthiness. Our experimental evaluation, which includes 18 LCLMs in total, demonstrates that most models still exhibit deficiencies in semantic understanding and elementary reasoning over retrieved results and are less trustworthy as context length increases. Our further analysis indicates some design choices employed by existing synthetic benchmarks, such as contextual non-coherence, numerical needles, and the absence of distractors, rendering them vulnerable to test the model’s long-context capabilities. To sum up, compared to previous synthetic benchmarks, LongBioBench achieves a better trade-off between mirroring authentic language tasks and maintaining controllability, and is highly interpretable and configurable.

Method: Proposes KG-Infused RAG framework that performs spreading activation over external knowledge graphs to retrieve structured knowledge, expands queries with this knowledge, integrates it with corpus passages, and uses preference learning on key pipeline stages.

Result: Outperforms vanilla RAG by 3.9% to 17.8% on five QA benchmarks, achieves superior performance compared to GraphRAG and LightRAG at lower cost, and shows further gains when integrated with Self-RAG and DeepNote.

Conclusion: KG-Infused RAG is an effective and versatile plug-and-play enhancement module that improves factual accuracy through structured knowledge integration while maintaining cost efficiency.

Abstract: Retrieval-Augmented Generation (RAG) improves factual accuracy by grounding responses in external knowledge. However, existing RAG methods either rely solely on text corpora and neglect structural knowledge, or build ad-hoc knowledge graphs (KGs) at high cost and low reliability. To address these issues, we propose KG-Infused RAG, a framework that incorporates pre-existing large-scale KGs into RAG and applies spreading activation to enhance both retrieval and generation. KG-Infused RAG directly performs spreading activation over external KGs to retrieve relevant structured knowledge, which is then used to expand queries and integrated with corpus passages, enabling interpretable and semantically grounded multi-source retrieval. We further improve KG-Infused RAG through preference learning on sampled key stages of the pipeline. Experiments on five QA benchmarks show that KG-Infused RAG consistently outperforms vanilla RAG (by 3.9% to 17.8%). Compared with KG-based approaches such as GraphRAG and LightRAG, our method obtains structured knowledge at lower cost while achieving superior performance. Additionally, integrating KG-Infused RAG with Self-RAG and DeepNote yields further gains, demonstrating its effectiveness and versatility as a plug-and-play enhancement module for corpus-based RAG methods.

Method: Leverage the determinism of program execution to extract verifiable step-by-step reasoning traces from code and transform them into natural language chain-of-thought reasoning.

Result: Experiments show the method effectively equips LLMs with transferable reasoning abilities across diverse tasks, produces highly accurate reasoning data, and reduces token length during inference by eliminating meaningless repetition.

Conclusion: The proposed approach provides a scalable way to generate high-quality chain-of-thought supervision that enhances LLM reasoning capabilities while reducing inference costs.

Abstract: Training large language models (LLMs) with chain-of-thought (CoT) supervision has proven effective for enhancing their reasoning abilities. However, obtaining reliable and accurate reasoning supervision remains a significant challenge. We propose a scalable method for generating a high-quality CoT supervision dataset by leveraging the determinism of program execution. Unlike existing reasoning dataset generation methods that rely on costly human annotations or error-prone LLM-generated CoT, our approach extracts verifiable, step-by-step reasoning traces from code execution and transforms them into a natural language CoT reasoning. Experiments on reasoning benchmarks across various domains show that our method effectively equips LLMs with transferable reasoning abilities across diverse tasks. Furthermore, the ablation studies validate that our method produces highly accurate reasoning data and reduces overall token length during inference by reducing meaningless repetition and overthinking.

Method: The survey introduces a unified framework with a taxonomy focused on three core commentator capabilities (Live Observation, Strategic Analysis, Historical Recall) and three commentary types (Descriptive, Analytical, Background). It provides an in-depth review of state-of-the-art methods, datasets, and evaluation metrics.

Result: The paper organizes the fragmented AI-GGC landscape into a systematic framework and identifies key challenges and future research directions.

Conclusion: The survey bridges the gap in AI-GGC research by providing a comprehensive framework and highlighting challenges like real-time reasoning, multimodal integration, and evaluation bottlenecks for future development.

Abstract: The advent of artificial intelligence has propelled AI-Generated Game Commentary (AI-GGC) into a rapidly expanding field, offering benefits such as unlimited availability and personalized narration. However, current researches in this area remain fragmented, and a comprehensive survey that systematically unifies existing efforts is still missing. To bridge this gap, our survey introduces a unified framework that systematically organizes the AI-GGC landscape. We present a novel taxonomy focused on three core commentator capabilities: Live Observation, Strategic Analysis, and Historical Recall. Commentary is further categorized into three functional types: Descriptive, Analytical, and Background. Building on this structure, we provide an in-depth review of state-of-the-art methods, datasets, and evaluation metrics across various game genres. Finally, we highlight key challenges such as real-time reasoning, multimodal integration, and evaluation bottlenecks, and outline promising directions for future research and system development in AI-GGC.

Method: Proposes AnTKV framework with offline token-aware centroids learning and online anchor token selection using vector quantization. Introduces anchor score to measure token sensitivity and preserves top 1% most sensitive tokens.

Result: Achieves 1-bit quantization on Mistral-7B with perplexity of 6.32 (vs 7.25 for CQ and 15.36 for KVQuant). Enables LLaMA3-8B to scale to 840K tokens on single 80GB A100 with 3.5× higher decoding throughput than FP16 baseline.

Conclusion: AnTKV effectively balances compression and accuracy through anchor token-aware vector quantization, significantly outperforming prior methods in ultra-low-bit regimes while maintaining deployment efficiency.

Abstract: Quantization has emerged as an effective and lightweight solution to reduce the memory footprint of the KV cache in Large Language Models. Nevertheless, minimizing the accuracy degradation caused by ultra-low-bit KV cache quantization remains a significant challenge. While scalar quantization is constrained by 1-bit bound, vector quantization exploits intra-vector correlations and enables sub-bit regimes, making it more suitable for ultra-low-bit quantization. To further mitigate quantization-induced degradation, we reveal that the degradation is highly uneven across tokens in attention quality. To investigate this unevenness, we introduce anchor score to measure each token’s sensitivity to quantization. Our analysis and experiments show that preserving a small subset (1%) of tokens with the highest Anchor Score significantly mitigates accuracy loss under aggressive quantization. We propose AnTKV, a dual-stage framework that leverages anchor token-aware vector quantization to compress the KV cache. It combines offline token-aware centroids learning and online anchor token selection to balance compression and accuracy. To enable efficient deployment, we design an online anchor token selection kernel compatible with FlashAttention. It allows LLaMA3-8B to scale to 840K tokens on a single 80GB A100, while delivering up to $3.5\times$ higher decoding throughput over the FP16 baseline. Experiments demonstrate that AnTKV matches or surpasses prior methods at 4-bit, and significantly reduce perplexity under ultra-low-bit quantization, achieving 6.32 at 1-bit on Mistral-7B, compared to 7.25 for CQ and 15.36 for KVQuant.

Method: Uses an autoencoder to learn a compressed latent space trained for token-level reconstruction and alignment with pretrained language encoder activations, enabling diffusion-based text generation.

Result: Achieves 8x compression while maintaining generation quality comparable to token-level diffusion models, and surpasses both diffusion and autoregressive baselines with longer latent sequences.

Conclusion: Cosmos demonstrates that compressed latent spaces enable efficient diffusion-based text generation with comparable or superior quality and significantly faster inference across multiple tasks.

Abstract: Autoregressive language models dominate modern text generation, yet their sequential nature introduces fundamental limitations: decoding is slow, and maintaining global coherence remains challenging. Diffusion models offer a promising alternative by enabling parallel generation and flexible control; however, their application to text generation is hindered by the high dimensionality of token-level representations. We introduce Cosmos, a novel approach to text generation that operates entirely in a compressed, smooth latent space tailored specifically for diffusion. This space is learned using an autoencoder trained simultaneously for token-level reconstruction and alignment with frozen activations from a pretrained language encoder, providing robust semantic grounding and enabling effective perturbation-based augmentations. Empirically, we demonstrate that text representations can be compressed by $8\times$ while maintaining generation quality comparable to token-level diffusion models. Furthermore, increasing the latent sequence length allows Cosmos to surpass both diffusion-based and autoregressive baselines. We evaluate Cosmos on four diverse generative tasks including story generation, question generation, summarization, and detoxification and compare it with various generative paradigms. Cosmos achieves comparable or superior generation quality while offering more than $2\times$ faster inference. Code is released at \href{https://github.com/MeshchaninovViacheslav/cosmos}{GitHub}

Method: Developed a formal framework for social network simulation and empirically tested different approaches to imitate user behavior on X platform in both English and German languages.

Result: Findings suggest that social simulations should be validated by their empirical realism measured in the setting where simulation components were fitted.

Conclusion: The paper argues for more rigor when applying generative-agent-based modeling for social simulation, emphasizing the need for proper validation of simulated behaviors.

Abstract: The ability of Large Language Models (LLMs) to mimic human behavior triggered a plethora of computational social science research, assuming that empirical studies of humans can be conducted with AI agents instead. Since there have been conflicting research findings on whether and when this hypothesis holds, there is a need to better understand the differences in their experimental designs. We focus on replicating the behavior of social network users with the use of LLMs for the analysis of communication on social networks. First, we provide a formal framework for the simulation of social networks, before focusing on the sub-task of imitating user communication. We empirically test different approaches to imitate user behavior on X in English and German. Our findings suggest that social simulations should be validated by their empirical realism measured in the setting in which the simulation components were fitted. With this paper, we argue for more rigor when applying generative-agent-based modeling for social simulation.

Method: Propose an efficient sampling strategy that uses a reward model to identify and retain high-quality first reasoning steps while discarding suboptimal ones.

Result: Achieved up to 70% reduction in inference cost without sacrificing any accuracy, consistently observed across various state-of-the-art reasoning models.

Conclusion: The first reasoning step plays a central role in generating high-quality reasoning trajectories, enabling significantly efficient sampling.

Abstract: Recent advancements in large language models (LLMs) have significantly advanced complex reasoning capabilities, particularly through extended chain-of-thought (CoT) reasoning that incorporates mechanisms such as backtracking, self-reflection, and self-correction. Despite these developments, the self-correction abilities of LLMs during long CoT reasoning remain underexplored. And recent findings on overthinking suggest that such models often engage in unnecessarily redundant reasoning. In this work, we empirically show that the first reasoning step exerts a disproportionately large influence on the final prediction. I.e., errors introduced at this stage can substantially degrade subsequent reasoning quality. This phenomenon is consistently observed across various state-of-the-art open- and closed-source reasoning models. Leveraging this insight, we propose an efficient sampling strategy that leverages a reward model to identify and retain high-quality first reasoning steps while discarding suboptimal ones, achieving up to a 70% reduction in inference cost without sacrificing any accuracy. Our work highlights the central role of the first reasoning step in generating a high-quality reasoning trajectory, and thus enabling significantly efficient sampling.

Method: Four-step process: (1) label sensitive tokens, (2) infer LLM without sensitive tokens for baseline, (3) infer LLM with sensitive tokens, (4) blend distributions to bound distance from baseline. Privacy/utility controlled by epsilon parameter.

Result: Achieves substantially improved theoretical and empirical privacy with 6× lower perplexity than related differentially private inference methods, creating token-level provably privatized documents.

Conclusion: DP-Fusion provides a practical solution for document privatization that balances privacy protection with text quality through provable differential privacy guarantees.

Abstract: Large language models (LLMs) do not preserve privacy at inference-time. The LLM’s outputs can inadvertently reveal information about the model’s context, which presents a privacy challenge when the LLM is augmented via tools or databases containing sensitive information. Existing privacy-preserving methods at inference-time have significant limitations since they (i) lack provable guarantees or (ii) have a poor utility/privacy trade-off. We propose DP-Fusion, a Differentially Private Inference (DPI) mechanism for LLMs that provably bounds the influence a set of tokens in the context can have on the LLM’s output. DP-Fusion works as follows: (1) label a subset of sensitive tokens, (2) infer the LLM without any sensitive tokens to obtain a baseline, (3) infer the LLM with the sensitive tokens, and (4) blend distributions so that the final output remains within a bounded distance of the baseline distribution. While this per-token influence bound also mitigates jailbreak-style prompt injection, we focus on \emph{document privatization}, where the goal is to paraphrase a document containing sensitive tokens, e.g., personally identifiable information, so that no attacker can reliably infer them from the paraphrased document while preserving high text quality. The privacy/utility trade-off is controlled by $\epsilon$, where $\epsilon=0$ hides sensitive tokens entirely, while higher values trade off privacy for improved text quality. We show that our method creates token-level provably privatized documents with substantially improved theoretical and empirical privacy, achieving $6\times$ lower perplexity than related DPI methods.

Method: Use lightweight local LLMs fine-tuned with privacy profiles to rewrite queries, hiding sensitive content before sending to external models. Built PEEP dataset for training.

Result: Fine-tuned lightweight LLMs achieve better privacy preservation and match/exceed performance of larger zero-shot models, but still struggle with fully adhering to user instructions.

Conclusion: The approach effectively balances privacy and performance, but better understanding of user-defined privacy preferences is needed for full instruction adherence.

Abstract: Large language models (LLMs) are primarily accessed via commercial APIs, but this often requires users to expose their data to service providers. In this paper, we explore how users can stay in control of their data by using privacy profiles: simple natural language instructions that say what should and should not be revealed. We build a framework where a local model uses these instructions to rewrite queries, only hiding details deemed sensitive by the user, before sending them to an external model, thus balancing privacy with performance. To support this research, we introduce PEEP, a multilingual dataset of real user queries annotated to mark private content and paired with synthetic privacy profiles. Experiments with lightweight local LLMs show that, after fine-tuning, they not only achieve markedly better privacy preservation but also match or exceed the performance of much larger zero-shot models. At the same time, the system still faces challenges in fully adhering to user instructions, underscoring the need for models with a better understanding of user-defined privacy preferences.

Method: Proposed Induced Substructure Filtration (ISF) perspective through empirical results and theoretical analysis, analyzing multi-layer transformers’ internal mechanisms and input query impact. Also introduced ’thinking in substructures’ concept for complex pattern extraction.

Result: Validated ISF process in LLMs, revealing consistent internal dynamics across layers. Demonstrated that decoder-only Transformers can successfully extract substructures from attributed graphs like molecular graphs.

Conclusion: Sequence-based Transformers can perform substructure extraction over graph data through the ISF process, providing new insights into how they understand graph structures from textual representations.

Abstract: Recent studies suggest that large language models (LLMs) possess the capability to solve graph reasoning tasks. Notably, even when graph structures are embedded within textual descriptions, LLMs can still effectively answer related questions. This raises a fundamental question: How can a decoder-only Transformer architecture understand underlying graph structures? To address this, we start with the substructure extraction task, interpreting the inner mechanisms inside the transformers and analyzing the impact of the input queries. Specifically, through both empirical results and theoretical analysis, we present Induced Substructure Filtration (ISF), a perspective that captures the substructure identification in the multi-layer transformers. We further validate the ISF process in LLMs, revealing consistent internal dynamics across layers. Building on these insights, we explore the broader capabilities of Transformers in handling diverse graph types. Specifically, we introduce the concept of thinking in substructures to efficiently extract complex composite patterns, and demonstrate that decoder-only Transformers can successfully extract substructures from attributed graphs, such as molecular graphs. Together, our findings offer a new insight on how sequence-based Transformers perform the substructure extraction task over graph data.

Method: Proposes a logic tree-based Agent-as-a-Judge system that extracts the logic tree of research outcomes as intermediate information for comprehensive, reliable, and robust evaluation.

Result: FinResearchBench covers 70 typical financial research questions across 7 frequently encountered task types in the financial research domain.

Conclusion: The work fills the gap in evaluating financial research agents by providing the first innovative Agent-as-a-Judge system specifically designed for the financial domain with comprehensive coverage of typical research tasks.

Abstract: Recently, AI agents are rapidly evolving in intelligence and widely used in professional research applications, such as STEM, software development, and finance. Among these AI agents, deep research agent is a key category as it can perform long-horizon tasks and solve problems of greater complexity. However, there are few evaluation frameworks and benchmarks that systematically and automatically investigate the capabilities of these research agents. In addition, financial research problems have distinct complexity and subtlety. To fill in the gap, we propose FinResearchBench, which is a logic tree-based Agent-as-a-Judge and targets specifically for the financial research agents. It provides a comprehensive and automatic assessment of the research agents across 7 key types of tasks in the financial research domain. The contributions of this work are two-folded: (1) the first and innovative Agent-as-a-Judge system that extracts the logic tree of the research outcome and uses it as the intermediate information to present a comprehensive, reliable, and robust evaluation; (2) finance-oriented that it covers 70 typical financial research questions, spreading across 7 frequently encountered types of task in the domain.

Method: Replace GRPO’s arithmetic mean with geometric mean of token-level rewards, which is inherently less sensitive to outliers and maintains more stable importance sampling ratios.

Result: GMPO-7B improves average Pass@1 by up to 4.1% over GRPO on multiple mathematical reasoning benchmarks, outperforming state-of-the-art approaches.

Conclusion: GMPO is a plug-and-play improvement over GRPO that provides more stable policy optimization through geometric mean aggregation, leading to better reasoning performance.

Abstract: Group Relative Policy Optimization (GRPO) has significantly enhanced the reasoning capability of large language models by optimizing the arithmetic mean of token-level rewards. Unfortunately, GRPO is observed to suffer from unstable policy updates when facing tokens with outlier importance-weighted rewards, which manifest as extreme importance sampling ratios during training. In this study, we propose Geometric-Mean Policy Optimization (GMPO), with the aim to improve the stability of GRPO through suppressing token reward outliers. Instead of optimizing the arithmetic mean, GMPO maximizes the geometric mean of token-level rewards, which is inherently less sensitive to outliers and maintains a more stable range of importance sampling ratio. GMPO is plug-and-play-simply replacing GRPO’s arithmetic mean with the geometric mean of token-level rewards, as the latter is inherently less sensitive to outliers. GMPO is theoretically plausible-analysis reveals that both GMPO and GRPO are weighted forms of the policy gradient while the former enjoys more stable weights, which consequently benefits policy optimization and performance. Experiments on multiple mathematical reasoning benchmarks show that GMPO-7B improves the average Pass@1 of GRPO by up to 4.1%, outperforming many state-of-the-art approaches. Code is available at https://github.com/callsys/GMPO.

Method: Four-stage pipeline: cultural data segregation, cultural data adaptation, machine translation, and quality filtering to expand English safety dataset to 8 languages.

Result: Created Nemotron-Safety-Guard-Dataset-v3 with 386,661 samples in 9 languages; trained Llama-3.1-Nemotron-Safety-Guard-8B-v3 model achieves SOTA performance on multilingual benchmarks with strong cross-lingual transfer.

Conclusion: This work advances multilingual LLM safety by enabling development of culturally aware safety guard models, showing current LLMs are more prone to unsafe responses in non-English languages.

Abstract: The increasing use of Large Language Models (LLMs) in agentic applications highlights the need for robust safety guard models. While content safety in English is well-studied, non-English languages lack similar advancements due to the high cost of collecting culturally aligned labeled datasets. We present CultureGuard, a novel solution for curating culturally aligned, high-quality safety datasets across multiple languages. Our approach introduces a four-stage synthetic data generation and filtering pipeline: cultural data segregation, cultural data adaptation, machine translation, and quality filtering. This pipeline enables the conversion and expansion of the Nemotron-Content-Safety-Dataset-V2 English safety dataset into eight distinct languages: Arabic, German, Spanish, French, Hindi, Japanese, Thai, and Chinese. The resulting dataset, Nemotron-Safety-Guard-Dataset-v3, comprises 386,661 samples in 9 languages and facilitates the training of Llama-3.1-Nemotron-Safety-Guard-8B-v3 via LoRA-based fine-tuning. The final model achieves state-of-the-art performance on several multilingual content safety benchmarks. Furthermore, we show our moderately multilingual fine-tuning enables robust cross-lingual transfer and strong zero-shot generalization to unseen languages. We also benchmark the latest open LLMs on multilingual safety and observe that these LLMs are more prone to give unsafe responses when prompted in non-English languages. This work advances multilingual LLM safety by enabling the development of culturally aware safety guard models.

Method: Compared Mirror vs Non-Mirror models using 110 participants who completed both structured diagnostic interviews (Mirror condition) and life history interviews (Non-Mirror condition), with LLMs prompted to predict depression scores.

Result: Mirror models showed near-perfect prediction (R² = .70), but Non-Mirror models also displayed large prediction sizes. Both model types correlated similarly with other depression questionnaires, suggesting bias in Mirror models. Topic modeling revealed different theme structures across model types.

Conclusion: Incorporating Non-Mirror approaches may support more valid and clinically useful language-based AI applications in psychological assessment, as they avoid criterion contamination while maintaining predictive power.

Abstract: Recent studies show near-perfect language-based predictions of depression scores (R2 = .70), but these “Mirror” models rely on language responses directly from depression assessments to predict depression assessment scores. These methods suffer from criterion contamination that inflate prediction estimates. We compare “Mirror” models to “Non-Mirror” models, which use other external language to predict depression scores. 110 participants completed both structured diagnostic (Mirror condition) and life history (Non-Mirror condition) interviews. LLMs were prompted to predict diagnostic depression scores. As expected, Mirror models were near-perfect. However, Non-Mirror models also displayed prediction sizes considered large in psychology. Further, both Mirror and Non-Mirror predictions correlated with other questionnaire-based depression symptoms at similar sizes, suggesting bias in Mirror models. Topic modeling revealed different theme structures across model types. As language models for depression continue to evolve, incorporating Non-Mirror approaches may support more valid and clinically useful language-based AI applications in psychological assessment.

Method: Correlate sample correctness with SAE activations from generated tokens at inference time, using only inference-time activations to extract relevant features and obtain steering coefficients from average activations.

Result: Improved performance on QA, bias mitigation, jailbreaking prevention, and reasoning benchmarks on Gemma-2 2B and LLaMA-3.1 8B, with +3.3% MMLU improvement (4000 samples) and +27.2% HarmBench improvement (108 samples). Selected features show semantically meaningful patterns.

Conclusion: Correlation-based selection is an effective and scalable approach for automated SAE steering across language model applications.

Abstract: Sparse Autoencoders (SAEs) can extract interpretable features from large language models (LLMs) without supervision. However, their effectiveness in downstream steering tasks is limited by the requirement for contrastive datasets or large activation storage. To address these limitations, we propose CorrSteer, which selects features by correlating sample correctness with SAE activations from generated tokens at inference time. This approach uses only inference-time activations to extract more relevant features, thereby reducing spurious correlations. It also obtains steering coefficients from average activations, automating the entire pipeline. Our method shows improved task performance on QA, bias mitigation, jailbreaking prevention, and reasoning benchmarks on Gemma-2 2B and LLaMA-3.1 8B, notably achieving a +3.3% improvement in MMLU performance with 4000 samples and a +27.2% improvement in HarmBench with only 108 samples. Selected features demonstrate semantically meaningful patterns aligned with each task’s requirements, revealing the underlying capabilities that drive performance. Our work establishes correlation-based selection as an effective and scalable approach for automated SAE steering across language model applications.

Method: Comprehensive data-centric synthesis with unified taxonomies of scientific data and knowledge, systematic review of Sci-LLMs across disciplines, analysis of 270+ datasets, and examination of 190+ benchmark datasets.

Result: Identifies distinct demands of Sci-LLMs including heterogeneous, multi-scale, uncertainty-laden corpora requiring domain-invariant representations and cross-modal reasoning. Shows shift from static to process-oriented evaluation.

Conclusion: Outlines paradigm shift toward closed-loop systems where autonomous Sci-LLM agents actively experiment and contribute to evolving knowledge bases, providing roadmap for trustworthy AI partners in scientific discovery.

Abstract: Scientific Large Language Models (Sci-LLMs) are transforming how knowledge is represented, integrated, and applied in scientific research, yet their progress is shaped by the complex nature of scientific data. This survey presents a comprehensive, data-centric synthesis that reframes the development of Sci-LLMs as a co-evolution between models and their underlying data substrate. We formulate a unified taxonomy of scientific data and a hierarchical model of scientific knowledge, emphasizing the multimodal, cross-scale, and domain-specific challenges that differentiate scientific corpora from general natural language processing datasets. We systematically review recent Sci-LLMs, from general-purpose foundations to specialized models across diverse scientific disciplines, alongside an extensive analysis of over 270 pre-/post-training datasets, showing why Sci-LLMs pose distinct demands – heterogeneous, multi-scale, uncertainty-laden corpora that require representations preserving domain invariance and enabling cross-modal reasoning. On evaluation, we examine over 190 benchmark datasets and trace a shift from static exams toward process- and discovery-oriented assessments with advanced evaluation protocols. These data-centric analyses highlight persistent issues in scientific data development and discuss emerging solutions involving semi-automated annotation pipelines and expert validation. Finally, we outline a paradigm shift toward closed-loop systems where autonomous agents based on Sci-LLMs actively experiment, validate, and contribute to a living, evolving knowledge base. Collectively, this work provides a roadmap for building trustworthy, continually evolving artificial intelligence (AI) systems that function as a true partner in accelerating scientific discovery.

Method: Uses sequential Bayesian experimental design framework where LLMs iteratively choose questions that maximize expected information gain about the task, based on probabilistic models derived from the LLM’s predictive distributions.

Result: Achieves substantial performance gains across 20 questions game and user preference inference tasks compared to direct prompting and other adaptive strategies.

Conclusion: BED-LLM provides a principled approach for LLMs to act as effective multi-turn conversational agents by intelligently gathering information through Bayesian experimental design.

Abstract: We propose a general-purpose approach for improving the ability of Large Language Models (LLMs) to intelligently and adaptively gather information from a user or other external source using the framework of sequential Bayesian experimental design (BED). This enables LLMs to act as effective multi-turn conversational agents and interactively interface with external environments. Our approach, which we call BED-LLM (Bayesian Experimental Design with Large Language Models), is based on iteratively choosing questions or queries that maximize the expected information gain (EIG) about the task of interest given the responses gathered previously. We show how this EIG can be formulated (and then estimated) in a principled way using a probabilistic model derived from the LLM’s predictive distributions and provide detailed insights into key decisions in its construction and updating procedure. We find that BED-LLM achieves substantial gains in performance across a wide range of tests based on the 20 questions game and using the LLM to actively infer user preferences, compared to direct prompting of the LLM and other adaptive design strategies.

Method: SIFT casts SL tasks as constrained response generation, combining in-context learning from demonstrations with supervised fine-tuning of causal LLMs.

Result: SIFT considerably outperforms both ICL and decoder-as-encoder fine-tuning baselines on standard SL tasks. Removing instructions improves performance as they’re largely unnecessary for strong SL performance with SIFT.

Conclusion: Response-based generative task formulation is crucial for effective sequence labeling with LLMs, highlighting both strengths and limitations of SL with LLMs.

Abstract: Sequence labeling (SL) tasks, where labels are assigned to tokens, are abundant in NLP (e.g., named entity recognition and aspect-based sentiment analysis). Owing to the intuition that they require bidirectional context, SL tasks are commonly tackled with encoder-only models. Recent work also shows that removing the causal mask in fine-tuning enables decoder-based LLMs to become effective token classifiers. Less work, however, focused on (supervised) generative SL, a more natural setting for causal LLMs. Due to their rapid scaling, causal LLMs applied to SL are expected to outperform encoders, whose own development has stagnated. In this work, we propose supervised in-context fine-tuning (SIFT) for generative SL. SIFT casts SL tasks as constrained response generation, natural to LLMs, combining in-context learning (ICL) from demonstrations with supervised fine-tuning. SIFT considerably outperforms both ICL and decoder-as-encoder fine-tuning baselines on a range of standard SL tasks. We further find that although long context hinders the performance of generative SL in both ICL and SIFT, this deficiency can be mitigated by removing the instruction, as instructions are shown to be largely unnecessary for achieving strong SL performance with SIFT. Our findings highlight strengths and limitations of SL with LLMs, underscoring the importance of a response-based generative task formulation for effective SL performance.

Method: Applied two data augmentation techniques: sentence concatenation with back translation and switch-out, tested across six African languages.

Result: Significant improvements in machine translation performance with minimum 25% BLEU score increase across all six languages tested.

Conclusion: Data augmentation techniques show strong potential to improve machine translation systems for low-resource languages, contributing to more robust translation systems for under-resourced languages.

Abstract: The linguistic diversity across the African continent presents different challenges and opportunities for machine translation. This study explores the effects of data augmentation techniques in improving translation systems in low-resource African languages. We focus on two data augmentation techniques: sentence concatenation with back translation and switch-out, applying them across six African languages. Our experiments show significant improvements in machine translation performance, with a minimum increase of 25% in BLEU score across all six languages. We provide a comprehensive analysis and highlight the potential of these techniques to improve machine translation systems for low-resource languages, contributing to the development of more robust translation systems for under-resourced languages.

Method: Human pairwise preferences from 678 creatives analyzed with Bradley-Terry models, covering 100 brands across 12 categories and three prompt types (Insights, Ideas, Wild Ideas), plus analysis of model diversity and LLM-as-judge setups.

Result: Models show tightly clustered performance (Δθ≈0.45, head-to-head win probability 0.61), no model dominates across brands or prompt types, weak correlations between LLM judges and human rankings, and partial transfer of conventional creativity tests.

Conclusion: Expert human evaluation remains essential for marketing creativity assessment, automated judges cannot substitute humans, and diversity-aware workflows are needed due to limited model dominance and inconsistent performance.

Abstract: We introduce Creativity Benchmark, an evaluation framework for large language models (LLMs) in marketing creativity. The benchmark covers 100 brands (12 categories) and three prompt types (Insights, Ideas, Wild Ideas). Human pairwise preferences from 678 practising creatives over 11,012 anonymised comparisons, analysed with Bradley-Terry models, show tightly clustered performance with no model dominating across brands or prompt types: the top-bottom spread is $\Delta\theta \approx 0.45$, which implies a head-to-head win probability of $0.61$; the highest-rated model beats the lowest only about $61%$ of the time. We also analyse model diversity using cosine distances to capture intra- and inter-model variation and sensitivity to prompt reframing. Comparing three LLM-as-judge setups with human rankings reveals weak, inconsistent correlations and judge-specific biases, underscoring that automated judges cannot substitute for human evaluation. Conventional creativity tests also transfer only partially to brand-constrained tasks. Overall, the results highlight the need for expert human evaluation and diversity-aware workflows.

Method: ZeroRepo uses a graph-driven framework with three stages: proposal-level planning, implementation-level construction, and graph-guided code generation with test validation. It employs Repository Planning Graph (RPG) as a structured blueprint encoding capabilities, file structures, data flows, and functions.

Result: On RepoCraft benchmark (6 real-world projects, 1,052 tasks), ZeroRepo generated nearly 36K Code Lines and 445K Code Tokens, 3.9x larger than Claude Code baseline. Achieved 81.5% coverage and 69.7% test accuracy, improving over Claude Code by 27.3 and 35.8 points.

Conclusion: RPG enables consistent long-horizon planning for repository generation, models complex dependencies, allows near-linear scaling for sophisticated planning, and improves agent understanding of repositories for faster localization.

Abstract: Large language models excel at generating individual functions or single files of code, yet generating complete repositories from scratch remains a fundamental challenge. This capability is key to building coherent software systems from high-level specifications and realizing the full potential of automated code generation. The process requires planning at two levels: deciding what features and modules to build (proposal stage) and defining their implementation details (implementation stage). Current approaches rely on natural language planning, which often produces unclear specifications, misaligned components, and brittle designs due to its inherent ambiguity and lack of structure. To address these limitations, we introduce the Repository Planning Graph (RPG), a structured representation that encodes capabilities, file structures, data flows, and functions in a unified graph. By replacing free-form natural language with an explicit blueprint, RPG enables consistent long-horizon planning for repository generation. Building on RPG, we develop ZeroRepo, a graph-driven framework that operates in three stages: proposal-level planning, implementation-level construction, and graph-guided code generation with test validation. To evaluate, we construct RepoCraft, a benchmark of six real-world projects with 1,052 tasks. On RepoCraft, ZeroRepo produces nearly 36K Code Lines and 445K Code Tokens, on average 3.9$\times$ larger than the strongest baseline (Claude Code), and 68$\times$ larger than other baselines. It achieves 81.5% coverage and 69.7% test accuracy, improving over Claude Code by 27.3 and 35.8 points. Further analysis shows that RPG models complex dependencies, enables more sophisticated planning through near-linear scaling, and improves agent understanding of repositories, thus accelerating localization.

Method: The approach uses Semantic Representation Heuristic Search to generate adversarial prompts in semantic space, maintaining interpretability during incremental expansion and targeting diverse responses with equivalent harmful meanings.

Result: Achieves unprecedented attack success rates (89.41% averaged across 18 LLMs, including 100% on 11 models) while maintaining stealthiness and efficiency.

Conclusion: Semantic Representation Attack fundamentally resolves the trade-off between attack efficacy and prompt naturalness, demonstrating overall superiority over existing methods.

Abstract: Large Language Models (LLMs) increasingly employ alignment techniques to prevent harmful outputs. Despite these safeguards, attackers can circumvent them by crafting prompts that induce LLMs to generate harmful content. Current methods typically target exact affirmative responses, such as ``Sure, here is…’’, suffering from limited convergence, unnatural prompts, and high computational costs. We introduce Semantic Representation Attack, a novel paradigm that fundamentally reconceptualizes adversarial objectives against aligned LLMs. Rather than targeting exact textual patterns, our approach exploits the semantic representation space comprising diverse responses with equivalent harmful meanings. This innovation resolves the inherent trade-off between attack efficacy and prompt naturalness that plagues existing methods. The Semantic Representation Heuristic Search algorithm is proposed to efficiently generate semantically coherent and concise adversarial prompts by maintaining interpretability during incremental expansion. We establish rigorous theoretical guarantees for semantic convergence and demonstrate that our method achieves unprecedented attack success rates (89.41% averaged across 18 LLMs, including 100% on 11 models) while maintaining stealthiness and efficiency. Comprehensive experimental results confirm the overall superiority of our Semantic Representation Attack. The code will be publicly available.

Method: Developed annotation guidelines for sub-sentence citations, constructed a dataset, and proposed an attribution framework using LLMs to generate fine-tuning data with a credit model to filter low-quality examples.

Result: Experiments on the constructed dataset demonstrate that the proposed approach can generate high-quality and more readable citations.

Conclusion: Sub-sentence citations provide concise and sufficient attribution, reducing user effort in verifying LLM outputs while maintaining verifiability.

Abstract: In retrieval-augmented generation (RAG) question answering systems, generating citations for large language model (LLM) outputs enhances verifiability and helps users identify potential hallucinations. However, we observe two problems in the citations produced by existing attribution methods. First, the citations are typically provided at the sentence or even paragraph level. Long sentences or paragraphs may include a substantial amount of irrelevant content. Second, sentence-level citations may omit information that is essential for verifying the output, forcing users to read the surrounding context. In this paper, we propose generating sub-sentence citations that are both concise and sufficient, thereby reducing the effort required by users to confirm the correctness of the generated output. To this end, we first develop annotation guidelines for such citations and construct a corresponding dataset. Then, we propose an attribution framework for generating citations that adhere to our standards. This framework leverages LLMs to automatically generate fine-tuning data for our task and employs a credit model to filter out low-quality examples. Our experiments on the constructed dataset demonstrate that the propose approach can generate high-quality and more readable citations.

Method: Evaluated and compared several RAG fine-tuning strategies including independent, joint, and two-phase fine-tuning approaches.

Result: All fine-tuning strategies achieved approximately equal improvement in EM and F1 generation quality metrics, despite having significantly different computational costs.

Conclusion: The optimal fine-tuning strategy depends on whether the training dataset includes context labels and whether grid search over learning rates for both embedding and generator models is required.

Abstract: A Comparison of Independent and Joint Fine-tuning Strategies for Retrieval-Augmented Generation Download PDF Neal Gregory Lawton, Alfy Samuel, Anoop Kumar, Daben Liu Published: 20 Aug 2025, Retrieval augmented generation (RAG) is a popular framework for question answering that is powered by two large language models (LLMs): an embedding model that retrieves context documents from a database that are relevant to a given question, and a generator model that uses the retrieved context to generate an answer to the question. Both the embedding and generator models can be fine-tuned to increase performance of a RAG pipeline on a new task, but multiple fine-tuning strategies exist with different costs and benefits. In this paper, we evaluate and compare several RAG fine-tuning strategies, including independent, joint, and two-phase fine-tuning. In our experiments, we observe that all of these strategies achieve about equal improvement in EM and F1 generation quality metrics, although they have significantly different computational costs. We conclude the optimal fine-tuning strategy to use depends on whether the training dataset includes context labels and whether a grid search over the learning rates for the embedding and generator models is required.

Method: The method uses three main stages: tentative matching of pseudo-parallel embedding vectors, transformation fitting, and iterative refinement to learn a linear transformation between embedding spaces.

Result: mini-vec2vec exceeds vec2vec by orders of magnitude in efficiency while matching or exceeding its results, with improved stability and interpretability.

Conclusion: The method’s stability, efficiency, and interpretable steps enable scaling and adoption in new domains and fields for embedding space alignment.

Abstract: We build upon vec2vec, a procedure designed to align text embedding spaces without parallel data. vec2vec finds a near-perfect alignment, but it is expensive and unstable. We present mini-vec2vec, a simple and efficient alternative that requires substantially lower computational cost and is highly robust. Moreover, the learned mapping is a linear transformation. Our method consists of three main stages: a tentative matching of pseudo-parallel embedding vectors, transformation fitting, and iterative refinement. Our linear alternative exceeds the original instantiation of vec2vec by orders of magnitude in efficiency, while matching or exceeding their results. The method’s stability and interpretable algorithmic steps facilitate scaling and unlock new opportunities for adoption in new domains and fields.

Method: Uses reinforcement learning with policy conditioned on explicit persona, optimizing next-step rationale and action generation via action correctness reward signals.

Result: Significantly outperforms prompting and SFT-based baselines in next-action prediction tasks and better matches users’ action distribution, indicating higher fidelity in personalized behavior simulation.

Conclusion: Customer-R1 demonstrates superior capability for personalized step-wise user behavior simulation compared to existing methods.

Abstract: Simulating step-wise human behavior with Large Language Models (LLMs) has become an emerging research direction, enabling applications in various practical domains. While prior methods, including prompting, supervised fine-tuning (SFT), and reinforcement learning (RL), have shown promise in modeling step-wise behavior, they primarily learn a population-level policy without conditioning on a user’s persona, yielding generic rather than personalized simulations. In this work, we pose a critical question: how can LLM agents better simulate personalized user behavior? We introduce Customer-R1, an RL-based method for personalized, step-wise user behavior simulation in online shopping environments. Our policy is conditioned on an explicit persona, and we optimize next-step rationale and action generation via action correctness reward signals. Experiments on the OPeRA dataset emonstrate that Customer-R1 not only significantly outperforms prompting and SFT-based baselines in next-action prediction tasks, but also better matches users’ action distribution, indicating higher fidelity in personalized behavior simulation.

Method: SAC directly selects anchor tokens from original context and aggregates contextual information into their KV representations using anchor embeddings and bidirectional attention modification.

Result: SAC consistently outperforms existing context compression methods across various compression ratios, achieving 1 EM improvement at 5x compression on MRQA with increasing advantages at higher ratios.

Conclusion: SAC provides a more effective approach to context compression by eliminating autoencoding training and directly leveraging contextual tokens through anchor-based architecture.

Abstract: Context compression presents a promising approach for accelerating large language model (LLM) inference by compressing long contexts into compact representations. Current context compression methods predominantly rely on autoencoding tasks to train context-agnostic compression tokens to compress contextual semantics. While autoencoding tasks enable compression tokens to acquire compression capabilities, compression via autoencoding tasks creates a fundamental mismatch: the models are optimized for reconstruction that diverge from actual downstream tasks, thereby weakening the features more beneficial for real-world usage. We propose Semantic-Anchor Compression (SAC), a novel method that shifts from autoencoding task based compression to an architecture that is equipped with this compression capability \textit{a priori}. Instead of training models to compress contexts through autoencoding tasks, SAC directly selects so-called anchor tokens from the original context and aggregates contextual information into their key-value (KV) representations. By deriving representations directly from the contextual tokens, SAC eliminates the need for autoencoding training. To ensure compression performance while directly leveraging anchor tokens, SAC incorporates two key designs: (1) anchor embeddings that enable the compressor to identify critical tokens, and (2) bidirectional attention modification that allows anchor tokens to capture information from the entire context. Experimental results demonstrate that SAC consistently outperforms existing context compression methods across various compression ratios. On out-of-distribution evaluation using MRQA, SAC achieves 1 EM improvement at 5x compression over strong baselines, with increasing advantages at higher compression ratios.

Method: DICE decouples the process: LLMs generate natural language responses, then trained SLMs analyze and refine these outputs using chain-of-thought correction. It constructs structured CoT datasets via two-stage method and applies dual-tuning strategy to fine-tune SLMs.

Result: DICE improves average format accuracy by 35.4% and content correctness by 29.4%, achieving state-of-the-art performance over competitive baselines.

Conclusion: DICE effectively preserves LLMs’ knowledge and reasoning while ensuring outputs meet structured specifications, providing a practical alternative to expensive LLM fine-tuning.

Abstract: When performing reasoning tasks with user-specific requirements, such as strict output formats, large language models (LLMs) often prioritize reasoning over adherence to detailed instructions. Fine-tuning LLMs on supervised datasets to address this is impractical due to high computational costs and limited parameter access. To tackle this, we propose DICE, a lightweight framework that guides small language models (SLMs) to refine LLMs’ outputs through chain-of-thought (CoT) correction. DICE decouples the process by first prompting LLMs to generate natural language responses, then using trained SLMs to analyze and refine these outputs to meet structured output specifications. This framework preserves LLMs’ broad knowledge and reasoning capabilities while ensuring the outputs conform to user demands. Specifically, DICE first constructs structured CoT adaptation datasets via a two-stage method and subsequently applies a dual-tuning strategy to fine-tune SLMs for generating structured outputs in an analyze-then-answer pattern. Experiments demonstrate that DICE improves the average format accuracy and content correctness of LLM outputs by 35.4% and 29.4%, respectively, achieving state-of-the-art (SOTA) performance over other competitive baselines.

Method: Developed ShiZhi LLM specifically for court view generation using a Chinese Court View Generation dataset (CCVG) of 110K+ cases with fact descriptions paired with court views.

Result: Achieved 70.00 ROUGE-1 and 67.85 BLEU-1 on court view generation, and 86.48% accuracy with 92.75% macro F1 on charge prediction.

Conclusion: Even small LLMs can generate reasonable and legally coherent court views when trained on high-quality domain-specific legal data.

Abstract: Criminal Court View Generation (CVG) is a fundamental task in legal artificial intelligence, aiming to automatically generate the “Court View” section of a legal case document. Generating court views is challenging due to the diversity and complexity of case facts, and directly generating from raw facts may limit performance. In this paper, we present ShiZhi, the first large language model (LLM) specifically designed for court view generation. We construct a Chinese Court View Generation dataset, CCVG, of more than 110K cases, each containing fact descriptions paired with corresponding court views. Based on this dataset, ShiZhi achieving 70.00 ROUGE-1 and 67.85 BLEU-1 on court view generation, as well as 86.48% accuracy with 92.75% macro F1 on charge prediction. Experimental results demonstrate that even a small LLM can generate reasonable and legally coherent court views when trained on high-quality domain-specific data. Our model and dataset are available at \href{https://github.com/ZhitianHou/ShiZhi}{https://github.com/ZhitianHou/ShiZhi}.

Method: Developed HUME framework to measure human performance across 16 MTEB datasets spanning reranking, classification, clustering, and semantic textual similarity across high- and low-resource languages.

Result: Humans achieved 77.6% average performance vs 80.1% for best embedding model, with models reaching near-ceiling performance on some datasets but struggling on others, particularly in low-resource languages.

Conclusion: HUME provides human performance baselines, insights into task difficulty patterns, and an extensible framework for more meaningful model evaluation and benchmark development.

Abstract: Comparing human and model performance offers a valuable perspective for understanding the strengths and limitations of embedding models, highlighting where they succeed and where they fail to capture meaning and nuance. However, such comparisons are rarely made, as human performance on embedding tasks is difficult to measure. To fill this gap, we introduce HUME: Human Evaluation Framework for Text Embeddings. While frameworks like MTEB provide broad model evaluation, they lack reliable estimates of human performance, limiting the interpretability of model scores. We measure human performance across 16 MTEB datasets spanning reranking, classification, clustering, and semantic textual similarity across linguistically diverse high- and low-resource languages. Humans achieve an average performance of 77.6% compared to 80.1% for the best embedding model, although variation is substantial: models reach near-ceiling performance on some datasets while struggling on others, suggesting dataset issues and revealing shortcomings in low-resource languages. We provide human performance baselines, insight into task difficulty patterns, and an extensible evaluation framework that enables a more meaningful interpretation of the model and informs the development of both models and benchmarks. Our code, dataset, and leaderboard are publicly available at https://github.com/embeddings-benchmark/mteb.

Method: Audit-of-Understanding (AoU) framework with three phases: (1) decomposing queries into candidate assumptions, (2) auditing their support, and (3) conditioning inference only on validated premises. Formally, it’s posterior-constrained inference connected to selective prediction and rejection learning.

Result: AoU improves both accuracy and faithfulness on GSM8K, MultiArith, and SVAMP benchmarks, achieving up to +30% gains on GSM8K, +45% on MultiArith, and consistent +20-28% improvements on SVAMP over Chain-of-Thought, Self-Consistency, and CoT-Decoding.

Conclusion: AoU effectively addresses reasoning-induced hallucinations in LLMs by constraining inference to validated premises, providing theoretical guarantees and empirical improvements across multiple reasoning benchmarks.

Abstract: Large language models (LLMs) often generate reasoning traces that appear coherent but rest on unsupported assumptions, leading to hallucinated conclusions. Prior work mainly addresses factual hallucinations or relies on post-hoc verification, leaving reasoning-induced hallucinations largely unaddressed. We propose Audit-of-Understanding (AoU), a framework that constrains inference to validated premises through three phases: (1) decomposing a query into candidate assumptions, (2) auditing their support, and (3) conditioning inference only on the validated subset. Formally, AoU is \emph{posterior-constrained inference}, connecting to selective prediction and rejection learning. Our contributions are threefold: (i) theoretical guarantees under perfect validation, (ii) excess-risk bounds under imperfect audits, and (iii) tractability analysis. Empirically, AoU improves both accuracy and faithfulness on GSM8K, MultiArith, and SVAMP, achieving up to +30% gains on GSM8K, +45% on MultiArith, and consistent +20–28% improvements on SVAMP over Chain-of-Thought, Self-Consistency, and CoT-Decoding. Code is available at https://anonymous.4open.science/r/audit-of-understanding-E28B.

Method: Introduced SLAQ framework comparing LLMs’ answers to same factual questions asked in isolation (short) vs. integrated into complex queries (long). Evaluated 16 LLMs across 600 queries, conducted mechanistic analysis to examine model internals.

Result: Found systematic misalignment between short and long query answers, position-dependent accuracy loss, and momentum effects. Aligned facts activate overlapping model internals, and mechanistic similarity metrics can predict short-long answer alignment with up to 78% accuracy.

Conclusion: Factual consistency over query complexity is crucial for LLMs’ trustworthiness. Current evaluation practices are flawed as they assume good performance on simple queries implies reliability in complex knowledge-seeking tasks.

Abstract: Large language models (LLMs) can correctly answer “When was Einstein born?” yet fail to provide the same date when writing about Einstein’s life revealing a fundamental inconsistency in how models access factual knowledge across task complexities. While models display impressive accuracy on factual question-answering benchmarks, the reliability gap between simple and complex queries remains poorly understood, eroding their trustworthiness. In this work, we introduce Short-Long Form Alignment for Factual Question Answering (SLAQ), a controlled evaluation framework that compares LLMs’ answers to the same factual questions asked (a) in isolation (short) vs. (b) integrated into complex queries (long). Looking at 16 LLMs across 600 queries, we find a systematic misalignment of answers to the corresponding short and long queries. We further uncover position-dependent accuracy loss and momentum effects where consecutive correct or incorrect answers create self-reinforcing patterns. Through mechanistic analysis, we find that aligned facts activate overlapping model internals, and that metrics based on mechanistic similarity can predict short-long answer alignment with up to 78% accuracy. Our work establishes factual consistency over query complexity as an important aspect of LLMs’ trustworthiness and challenges current evaluation practices, which implicitly assume that good performance for simple factual queries implies reliability in more complex knowledge-seeking tasks too.

Method: Study synthesis methods (using LLMs to generate survey responses) combined with rectification methods (debiasing population estimates). Tested on panel surveys covering nutrition, politics, and economics. Explored optimal allocation of human responses between synthesis and rectification under fixed budget.

Result: Synthesis alone introduces substantial bias (24-86%). Combining synthesis with rectification reduces bias below 5% and increases effective sample size by up to 14%. Allocating most human responses to rectification rather than fine-tuning provides more effective estimation.

Conclusion: LLMs can effectively substitute for human survey respondents when combined with proper debiasing methods. Optimal strategy allocates majority of human responses to rectification rather than fine-tuning, challenging common practice.

Abstract: Surveys provide valuable insights into public opinion and behavior, but their execution is costly and slow. Large language models (LLMs) have been proposed as a scalable, low-cost substitute for human respondents, but their outputs are often biased and yield invalid estimates. We study the interplay between synthesis methods that use LLMs to generate survey responses and rectification methods that debias population estimates, and explore how human responses are best allocated between them. Using two panel surveys with questions on nutrition, politics, and economics, we find that synthesis alone introduces substantial bias (24-86%), whereas combining it with rectification reduces bias below 5% and increases effective sample size by up to 14%. Overall, we challenge the common practice of using all human responses for fine-tuning, showing that under a fixed budget, allocating most to rectification results in far more effective estimation.

Method: Developed an automatic pipeline for difficulty-controlled creation of CRUMQs (uncheatable, realistic, unanswerable, multi-hop queries) adaptable to any corpus and domain.

Result: CRUMQs significantly challenge RAG systems, achieving up to 81.0% reduction in cheatability scores compared to prior benchmarks when tested on leading retrieval-augmented LLMs.

Conclusion: The CRUMQs pipeline effectively enhances benchmark difficulty and realism, providing a better framework to drive development of more capable RAG systems.

Abstract: Real-world use cases often present RAG systems with complex queries for which relevant information is missing from the corpus or is incomplete. In these settings, RAG systems must be able to reject unanswerable, out-of-scope queries and identify failures of retrieval and multi-hop reasoning. Despite this, existing RAG benchmarks rarely reflect realistic task complexity for multi-hop or out-of-scope questions, which often can be cheated via disconnected reasoning (i.e., solved without genuine multi-hop inference) or require only simple factual recall. This limits the ability for such benchmarks to uncover limitations of existing RAG systems. To address this gap, we present the first pipeline for automatic, difficulty-controlled creation of un$\underline{c}$heatable, $\underline{r}$ealistic, $\underline{u}$nanswerable, and $\underline{m}$ulti-hop $\underline{q}$uerie$\underline{s}$ (CRUMQs), adaptable to any corpus and domain. We use our pipeline to create CRUMQs over two popular RAG datasets and demonstrate its effectiveness via benchmark experiments on leading retrieval-augmented LLMs. Results show that compared to prior RAG benchmarks, CRUMQs are highly challenging for RAG systems and achieve up to 81.0% reduction in cheatability scores. More broadly, our pipeline offers a simple way to enhance benchmark difficulty and realism and drive development of more capable RAG systems.

Method: Introduced CUEST framework to evaluate human-model alignment in curiosity using Yahoo! Answers dataset across multiple countries, analyzing linguistic style and topic preferences, grounded in social science constructs.

Result: LLMs across both open- and closed-source models flatten cross-cultural diversity in curiosity, aligning more closely with Western expression patterns. Fine-tuning strategies narrowed the human-model alignment gap by up to 50%.

Conclusion: Curiosity has practical value for LLM adaptability across cultures and is important for future NLP research.

Abstract: Recent advances in Large Language Models (LLMs) have expanded their role in human interaction, yet curiosity – a central driver of inquiry – remains underexplored in these systems, particularly across cultural contexts. In this work, we investigate cultural variation in curiosity using Yahoo! Answers, a real-world multi-country dataset spanning diverse topics. We introduce CUEST (CUriosity Evaluation across SocieTies), an evaluation framework that measures human-model alignment in curiosity through linguistic (style), topic preference (content) analysis and grounding insights in social science constructs. Across open- and closed-source models, we find that LLMs flatten cross-cultural diversity, aligning more closely with how curiosity is expressed in Western countries. We then explore fine-tuning strategies to induce curiosity in LLMs, narrowing the human-model alignment gap by up to 50%. Finally, we demonstrate the practical value of curiosity for LLM adaptability across cultures, showing its importance for future NLP research.

Method: The authors developed \bench, a dynamic and live evaluation benchmark that supports real-time updates and prevents data contamination through an automated curation pipeline. It focuses on consumer domain discussions.

Result: \bench is presented as the largest and most diverse benchmark of its kind, specifically designed for intent understanding in real-world public discussions with dynamic, live evaluation capabilities.

Conclusion: The paper bridges a critical gap in LLM evaluation by providing the first specialized benchmark for understanding human intent in complex, multi-participant public discussions, enabling better assessment of LLMs’ analytical reasoning and contextual interpretation abilities.

Abstract: Understanding human intent is a complex, high-level task for large language models (LLMs), requiring analytical reasoning, contextual interpretation, dynamic information aggregation, and decision-making under uncertainty. Real-world public discussions, such as consumer product discussions, are rarely linear or involve a single user. Instead, they are characterized by interwoven and often conflicting perspectives, divergent concerns, goals, emotional tendencies, as well as implicit assumptions and background knowledge about usage scenarios. To accurately understand such explicit public intent, an LLM must go beyond parsing individual sentences; it must integrate multi-source signals, reason over inconsistencies, and adapt to evolving discourse, similar to how experts in fields like politics, economics, or finance approach complex, uncertain environments. Despite the importance of this capability, no large-scale benchmark currently exists for evaluating LLMs on real-world human intent understanding, primarily due to the challenges of collecting real-world public discussion data and constructing a robust evaluation pipeline. To bridge this gap, we introduce \bench, the first dynamic, live evaluation benchmark specifically designed for intent understanding, particularly in the consumer domain. \bench is the largest and most diverse benchmark of its kind, supporting real-time updates while preventing data contamination through an automated curation pipeline.

Method: Combines two strategies: (1) lightweight prompting techniques including Deflanderization to reduce excessive role-play and improve task fidelity in API track, and (2) fine-tuned Qwen3-14B models using supervised fine-tuning and LoRA in GPU track.

Result: Achieved 2nd place on Task 1, 2nd place on Task 3 (API track), and 4th place on Task 3 (GPU track) in the Commonsense Persona-Grounded Dialogue Challenge 2025 Round 2.

Conclusion: The combination of prompting techniques and fine-tuned models effectively enables both task execution and persona-consistent dialogue generation for dynamic NPCs in gaming environments.

Abstract: The emergence of large language models (LLMs) has opened new opportunities for cre- ating dynamic non-player characters (NPCs) in gaming environments, enabling both func- tional task execution and persona-consistent dialogue generation. In this paper, we (Tu_Character_lab) report our participation in the Commonsense Persona-Grounded Dialogue Challenge (CPDC) 2025 Round 2, which eval- uates agents across three tracks: task-oriented dialogue, context-aware dialogue, and their integration. Our approach combines two complementary strategies: (i) lightweight prompting techniques in the API track, including a Deflanderization prompting method to suppress excessive role-play and improve task fidelity, and (ii) fine-tuned large models in the GPU track, leveraging Qwen3-14B with supervisedfinetuning (SFT) and Low-Rank Adaptation(LoRA). Our best submissions ranked 2nd on Task 1, 2nd on Task 3 (API track), and 4th on Task 3 (GPU track).

Method: Created a benchmark with 30 iconic heroes and 90 canon-specific versions, featuring two tasks: Canon Events (factual recall) and Moral Dilemmas (ethical scenarios). Proposed Think-Act Matching metric to quantify alignment between reasoning and actions.

Result: Chain-of-thought improves coherence in weaker models but reduces accuracy in stronger ones; cross-version generalization remains challenging; models rarely excel at both thinking and acting simultaneously.

Conclusion: Beyond One World exposes critical gaps in multiversal consistency and reasoning alignment, providing a challenging evaluation framework for role-playing LLMs.

Abstract: Large language models (LLMs) are increasingly used as role-playing agents, yet their capacity to faithfully and consistently portray version-specific characters – for example, superheroes across comic and cinematic universes – remains underexplored. Superhero canons such as Marvel and DC provide a rich testbed: decades of storytelling yield multiple incarnations of the same character with distinct histories, values, and moral codes. To study this problem, we introduce Beyond One World, a benchmark for character-grounded roleplay spanning 30 iconic heroes and 90 canon-specific versions. The benchmark comprises two tasks: (i) Canon Events, which probes factual recall of pivotal life stages, and (ii) Moral Dilemmas, which confronts models with ethically charged scenarios. We score responses for canonical accuracy and reasoning fidelity under a framework that separates internal deliberation (“thinking”) from outward decisions (“acting”). We further propose Think-Act Matching, a metric that quantifies alignment between reasons and actions and serves as a proxy for model trustworthiness. Experiments across reasoning- and non-reasoning-oriented models yield three findings: (1) chain-of-thought prompting improves narrative coherence in weaker models but can reduce canonical accuracy in stronger ones; (2) cross-version generalization within a character remains a major obstacle; and (3) models often excel at either thinking or acting, but rarely both. Beyond One World exposes critical gaps in multiversal consistency and reasoning alignment, offering a challenging evaluation for role-playing LLMs.

Method: Built a multilingual benchmark of 100 dialogue sessions across English, Japanese, and Chinese, where users interact for five days with psychologically grounded NPCs based on MBTI dimensions to capture natural communication patterns.

Result: Established the first benchmark for YNTP and evaluated prompt-based and fine-tuning-based personalization methods, providing a foundation for user-aligned language modeling.

Conclusion: The YNTP task and dataset enable analysis of users’ internal communication models and advance personalized language generation while addressing privacy limitations of real communication data collection.

Abstract: Large language models (LLMs) excel at general next-token prediction but still struggle to generate responses that reflect how individuals truly communicate, such as replying to emails or social messages in their own style. However, real SNS or email histories are difficult to collect due to privacy concerns. To address this, we propose the task of “Your Next Token Prediction (YNTP)”, which models a user’s precise word choices through controlled human-agent conversations. We build a multilingual benchmark of 100 dialogue sessions across English, Japanese, and Chinese, where users interact for five days with psychologically grounded NPCs based on MBTI dimensions. This setup captures natural, daily-life communication patterns and enables analysis of users’ internal models. We evaluate prompt-based and fine-tuning-based personalization methods, establishing the first benchmark for YNTP and a foundation for user-aligned language modeling. The dataset is available at: https://github.com/AnonymousHub4Submissions/your-next-token-prediction-dataset-100

Method: Three innovations: citation-aware reasoning with Negative Knowledge Assertions, iterative retrieval-verification with Medical Gap Analysis, and MedTrust-Align Module using Direct Preference Optimization.

Result: Enhanced factual consistency and reduced hallucinations in medical question answering.

Conclusion: The proposed framework significantly improves reliability of biomedical QA systems by systematically addressing hallucination issues through structured verification and preference optimization.

Abstract: Biomedical question answering (QA) requires accurate interpretation of complex medical knowledge. Large language models (LLMs) have shown promising capabilities in this domain, with retrieval-augmented generation (RAG) systems enhancing performance by incorporating external medical literature. However, RAG-based approaches in biomedical QA suffer from hallucinations due to post-retrieval noise and insufficient verification of retrieved evidence, undermining response reliability. We propose MedTrust-Guided Iterative RAG, a framework designed to enhance factual consistency and mitigate hallucinations in medical QA. Our method introduces three key innovations. First, it enforces citation-aware reasoning by requiring all generated content to be explicitly grounded in retrieved medical documents, with structured Negative Knowledge Assertions used when evidence is insufficient. Second, it employs an iterative retrieval-verification process, where a verification agent assesses evidence adequacy and refines queries through Medical Gap Analysis until reliable information is obtained. Third, it integrates the MedTrust-Align Module (MTAM) that combines verified positive examples with hallucination-aware negative samples, leveraging Direct Preference Optimization to reinforce citation-grounded reasoning while penalizing hallucination-prone response patterns.

Method: Examined hybrid architectures with DDLM-ARM collaboration in both text space (one model plans, another executes) and latent space (using learned projector to map DDLM latents to ARM embedding space).

Result: Latent-space communication significantly improved accuracy (27.0% to 54.0% on DART-5, 0.0% to 14.0% on AIME24). The hybrid approach provided substantial computational savings - using only 69 tokens total (64 for planning, 5 for execution) outperformed Qwen3.1-7B which used 44x more tokens.

Conclusion: DDLM-ARM hybrid architectures offer new insights into reasoning and highlight DDLMs’ potential in efficient hybrid systems, achieving better performance with significantly reduced computational costs.

Abstract: Current autoregressive language models (ARMs) achieve high accuracy but require long token sequences, making them costly. Discrete diffusion language models (DDLMs) enable parallel and flexible generation within a fixed number of steps and have recently emerged for their strong performance in complex reasoning and long-term planning tasks. We present a study exploring hybrid architectures that couple DDLMs with ARMs to assess whether their collaboration can yield complementary benefits. We first examine collaboration in text space, where one model plans the reasoning process and another executes the final answer based on that plan. We then extend this setup to latent-space communication, introducing a learned projector that maps DDLM latents into the ARM’s embedding space, potentially bypassing some of the text-generation limitations of diffusion models. We find that shifting DDLM –> ARM communication from text space to latent space yields significant accuracy gains, for example increasing from 27.0% to 54.0% on DART-5 and from 0.0% to 14.0% on AIME24. We also find that combining a DDLM planner with an ARM executor can provide substantial computational savings with little to no impact on accuracy. For example, the latent-space pipeline, using 64 tokens for planning and roughly 5 for execution, surpasses Qwen3.1-7B on DART-5 and AIME, despite Qwen using 44 times more tokens. Overall, our study offers new insights into reasoning with DDLMs and highlights their potential in hybrid architectures.

Method: Three synergistic components: adaptive execution scheduling (dynamic compute graph balancing), context-aligned drafting (lightweight online calibration), and hardware-efficient draft extension (reusing and expanding intermediate sequences).

Result: Experiments show consistent improvements of up to 3.8x in generation speed and 4.7x in energy efficiency compared to existing mobile inference solutions across multiple smartphones and workloads.

Conclusion: The framework successfully accelerates context-aware text generation on mobile devices through speculative decoding and dynamic hardware scheduling, with component-level analysis validating each optimization’s contribution.

Abstract: Enhancing on-device large language models (LLMs) with contextual information from local data enables personalized and task-aware generation, powering use cases such as intelligent assistants and UI agents. While recent developments in neural processors have substantially improved the efficiency of prefill on mobile devices, the token-by-token generation process still suffers from high latency and limited hardware utilization due to its inherently memory-bound characteristics. This work presents sd.npu, a mobile inference framework that integrates speculative decoding with dynamic hardware scheduling to accelerate context-aware text generation on mobile devices. The framework introduces three synergistic components: (1) adaptive execution scheduling, which dynamically balances compute graphs between prefill and decoding phases; (2) context-aligned drafting, which improves speculative efficiency through lightweight online calibration to current tasks; and (3) hardware-efficient draft extension, which reuses and expands intermediate sequences to improve processing parallelism and reduce verification cost. Experiments on multiple smartphones and representative workloads show consistent improvements of up to 3.8x in generation speed and 4.7x in energy efficiency compared with existing mobile inference solutions. Component-level analysis further validates the contribution of each optimization.

Method: Uses reinforcement learning to train an LLM constructor, framing graph generation as policy learning with task-aware reward functions based on the graph’s utility in RAG pipelines.

Result: AutoGraph-R1 consistently enables graph RAG methods to achieve significant performance gains over task-agnostic baseline graphs across multiple QA benchmarks.

Conclusion: It’s possible to close the loop between KG construction and application, shifting from building intrinsically ‘good’ graphs to building demonstrably ‘useful’ ones.

Abstract: Building effective knowledge graphs (KGs) for Retrieval-Augmented Generation (RAG) is pivotal for advancing question answering (QA) systems. However, its effectiveness is hindered by a fundamental disconnect: the knowledge graph (KG) construction process is decoupled from its downstream application, yielding suboptimal graph structures. To bridge this gap, we introduce AutoGraph-R1, the first framework to directly optimize KG construction for task performance using Reinforcement Learning (RL). AutoGraph-R1 trains an LLM constructor by framing graph generation as a policy learning problem, where the reward is derived from the graph’s functional utility in a RAG pipeline. We design two novel, task-aware reward functions, one for graphs as knowledge carriers and another as knowledge indices. Across multiple QA benchmarks, AutoGraph-R1 consistently enables graph RAG methods to achieve significant performance gains over using task-agnostic baseline graphs. Our work shows it is possible to close the loop between construction and application, shifting the paradigm from building intrinsically good'' graphs to building demonstrably useful’’ ones.

Method: Reinforcement learning framework with composite rewards (normalized edit distance, paragraph count accuracy, reading order preservation) trained on Infinity-Doc-400K dataset.

Result: Infinity-Parser achieves state-of-the-art performance across multiple benchmarks (OmniDocBench, olmOCR-Bench, PubTabNet, FinTabNet) for various document types, languages, and complexities.

Conclusion: The approach demonstrates robust generalization and substantially outperforms existing document parsing systems and general-purpose vision-language models.

Abstract: Document parsing from scanned images into structured formats remains a significant challenge due to its complexly intertwined elements such as text paragraphs, figures, formulas, and tables. Existing supervised fine-tuning methods often struggle to generalize across diverse document types, leading to poor performance, particularly on out-of-distribution data. This issue is further exacerbated by the limited availability of high-quality training data for layout-aware parsing tasks. To address these challenges, we introduce LayoutRL, a reinforcement learning framework that optimizes layout understanding through composite rewards integrating normalized edit distance, paragraph count accuracy, and reading order preservation. To support this training, we construct the Infinity-Doc-400K dataset, which we use to train Infinity-Parser, a vision-language model demonstrating robust generalization across various domains. Extensive evaluations on benchmarks including OmniDocBench, olmOCR-Bench, PubTabNet, and FinTabNet show that Infinity-Parser consistently achieves state-of-the-art performance across a broad range of document types, languages, and structural complexities, substantially outperforming both specialized document parsing systems and general-purpose vision-language models. We will release our code, dataset, and model to facilitate reproducible research in document parsing.

Method: Introduces User Game Lifecycle (UGL) to enrich user behaviors, strategies for extracting short/long-term interests, and Inverse Probability Masking for handling game imbalance in UGL representation learning.

Result: Significant improvements: 1.83% AUC offline increase and 21.67% CVR online increase for game advertising; 0.5% AUC offline increase and 0.82% ARPU online increase for in-game item recommendation.

Conclusion: UGL representations effectively address game sparsity and imbalance challenges, demonstrating substantial performance gains in both game advertising and in-game item recommendation systems.

Abstract: The rapid expansion of video game production necessitates the development of effective advertising and recommendation systems for online game platforms. Recommending and advertising games to users hinges on capturing their interest in games. However, existing representation learning methods crafted for handling billions of items in recommendation systems are unsuitable for game advertising and recommendation. This is primarily due to game sparsity, where the mere hundreds of games fall short for large-scale user representation learning, and game imbalance, where user behaviors are overwhelmingly dominated by a handful of popular games. To address the sparsity issue, we introduce the User Game Lifecycle (UGL), designed to enrich user behaviors in games. Additionally, we propose two innovative strategies aimed at manipulating user behaviors to more effectively extract both short and long-term interests. To tackle the game imbalance challenge, we present an Inverse Probability Masking strategy for UGL representation learning. The offline and online experimental results demonstrate that the UGL representations significantly enhance model by achieving a 1.83% AUC offline increase on average and a 21.67% CVR online increase on average for game advertising and a 0.5% AUC offline increase and a 0.82% ARPU online increase for in-game item recommendation.

Method: Propose ESCA framework with SGClip - a CLIP-based, promptable scene graph generation model trained on 87K+ videos using neurosymbolic learning with model-driven self-supervision from video-caption pairs.

Result: SGClip excels in scene graph generation and action localization benchmarks. ESCA consistently improves both open-source and commercial MLLMs, achieving state-of-the-art performance in embodied environments with significant error reduction.

Conclusion: ESCA enables open-source models to surpass proprietary baselines by providing structured spatial-temporal understanding through scene graph contextualization.

Abstract: Multi-modal large language models (MLLMs) are making rapid progress toward general-purpose embodied agents. However, current training pipelines primarily rely on high-level vision-sound-text pairs and lack fine-grained, structured alignment between pixel-level visual content and textual semantics. To overcome this challenge, we propose ESCA, a new framework for contextualizing embodied agents through structured spatial-temporal understanding. At its core is SGClip, a novel CLIP-based, open-domain, and promptable model for generating scene graphs. SGClip is trained on 87K+ open-domain videos via a neurosymbolic learning pipeline, which harnesses model-driven self-supervision from video-caption pairs and structured reasoning, thereby eliminating the need for human-labeled scene graph annotations. We demonstrate that SGClip supports both prompt-based inference and task-specific fine-tuning, excelling in scene graph generation and action localization benchmarks. ESCA with SGClip consistently improves both open-source and commercial MLLMs, achieving state-of-the-art performance across two embodied environments. Notably, it significantly reduces agent perception errors and enables open-source models to surpass proprietary baselines.

Method: RefAtomNet++ uses multi-hierarchical semantic-aligned cross-attention mechanism with multi-trajectory Mamba modeling at partial-keyword, scene-attribute, and holistic-sentence levels, with dynamic scanning trajectories for visual spatial tokens.

Result: RefAtomNet++ establishes new state-of-the-art results on the RefAVA++ dataset, outperforming previous baselines including RefAtomNet.

Conclusion: The proposed framework effectively overcomes cross-modal alignment limitations and improves performance in referring atomic video action recognition tasks.

Abstract: Referring Atomic Video Action Recognition (RAVAR) aims to recognize fine-grained, atomic-level actions of a specific person of interest conditioned on natural language descriptions. Distinct from conventional action recognition and detection tasks, RAVAR emphasizes precise language-guided action understanding, which is particularly critical for interactive human action analysis in complex multi-person scenarios. In this work, we extend our previously introduced RefAVA dataset to RefAVA++, which comprises >2.9 million frames and >75.1k annotated persons in total. We benchmark this dataset using baselines from multiple related domains, including atomic action localization, video question answering, and text-video retrieval, as well as our earlier model, RefAtomNet. Although RefAtomNet surpasses other baselines by incorporating agent attention to highlight salient features, its ability to align and retrieve cross-modal information remains limited, leading to suboptimal performance in localizing the target person and predicting fine-grained actions. To overcome the aforementioned limitations, we introduce RefAtomNet++, a novel framework that advances cross-modal token aggregation through a multi-hierarchical semantic-aligned cross-attention mechanism combined with multi-trajectory Mamba modeling at the partial-keyword, scene-attribute, and holistic-sentence levels. In particular, scanning trajectories are constructed by dynamically selecting the nearest visual spatial tokens at each timestep for both partial-keyword and scene-attribute levels. Moreover, we design a multi-hierarchical semantic-aligned cross-attention strategy, enabling more effective aggregation of spatial and temporal tokens across different semantic hierarchies. Experiments show that RefAtomNet++ establishes new state-of-the-art results. The dataset and code are released at https://github.com/KPeng9510/refAVA2.

Method: Proposes Sparse Selector (SS) with two core modules: Ray-Aware Supervision (RAS) to preserve geometric information during training, and Class-Balanced Supervision to adaptively reweight class semantics and retain small object tokens. Also introduces Ray Positional Encoding to address LiDAR-image distribution differences.

Result: Achieves state-of-the-art performance on nuScenes benchmark with 72.4 mAP and 74.7 NDS, while running 1.84× faster than other leading methods. Demonstrates strong robustness even with partially or entirely missing LiDAR or camera data.

Conclusion: CrossRay3D successfully addresses token representation quality issues in sparse detectors through geometric structure preservation and balanced class handling, achieving superior performance and efficiency while maintaining robustness to sensor failures.

Abstract: The sparse cross-modality detector offers more advantages than its counterpart, the Bird’s-Eye-View (BEV) detector, particularly in terms of adaptability for downstream tasks and computational cost savings. However, existing sparse detectors overlook the quality of token representation, leaving it with a sub-optimal foreground quality and limited performance. In this paper, we identify that the geometric structure preserved and the class distribution are the key to improving the performance of the sparse detector, and propose a Sparse Selector (SS). The core module of SS is Ray-Aware Supervision (RAS), which preserves rich geometric information during the training stage, and Class-Balanced Supervision, which adaptively reweights the salience of class semantics, ensuring that tokens associated with small objects are retained during token sampling. Thereby, outperforming other sparse multi-modal detectors in the representation of tokens. Additionally, we design Ray Positional Encoding (Ray PE) to address the distribution differences between the LiDAR modality and the image. Finally, we integrate the aforementioned module into an end-to-end sparse multi-modality detector, dubbed CrossRay3D. Experiments show that, on the challenging nuScenes benchmark, CrossRay3D achieves state-of-the-art performance with 72.4 mAP and 74.7 NDS, while running 1.84 faster than other leading methods. Moreover, CrossRay3D demonstrates strong robustness even in scenarios where LiDAR or camera data are partially or entirely missing.

Method: Two-stage approach: 1) Transform language queries into enriched sentences with additional details, 2) Use lightweight decoder to ground enriched queries with multiple-instance-learning training that dynamically selects optimal query versions.

Result: State-of-the-art performance across temporal video grounding benchmarks, significantly outperforming previous LLM-based approaches and comparable to specialized models, with strong zero-shot evaluation capabilities.

Conclusion: ED-VTG demonstrates that query enrichment through multimodal LLMs combined with dynamic training strategies effectively improves video temporal grounding, offering superior performance while maintaining generalization in zero-shot scenarios.

Abstract: We introduce ED-VTG, a method for fine-grained video temporal grounding utilizing multi-modal large language models. Our approach harnesses the capabilities of multimodal LLMs to jointly process text and video, in order to effectively localize natural language queries in videos through a two-stage process. Rather than being directly grounded, language queries are initially transformed into enriched sentences that incorporate missing details and cues to aid in grounding. In the second stage, these enriched queries are grounded, using a lightweight decoder, which specializes at predicting accurate boundaries conditioned on contextualized representations of the enriched queries. To mitigate noise and reduce the impact of hallucinations, our model is trained with a multiple-instance-learning objective that dynamically selects the optimal version of the query for each training sample. We demonstrate state-of-the-art results across various benchmarks in temporal video grounding and paragraph grounding settings. Experiments reveal that our method significantly outperforms all previously proposed LLM-based temporal grounding approaches and is either superior or comparable to specialized models, while maintaining a clear advantage against them in zero-shot evaluation scenarios.

Method: Combines YOLO object detectors for multi-defect detection and segmentation with vision language models for scene-aware summarization, generating structured JSON action plans.

Result: Accurately identifies diverse defects and produces coherent, structured summaries with incident descriptions, recommended tools, dimensions, repair plans, and urgent alerts.

Conclusion: The system shows promise for city-wide deployment but faces challenges in scaling, requiring further development for broader implementation.

Abstract: Infrastructure in smart cities is increasingly monitored by networks of closed circuit television (CCTV) cameras. Roads, bridges and tunnels develop cracks, potholes, and fluid leaks that threaten public safety and require timely repair. Manual inspection is costly and hazardous, and existing automatic systems typically address individual defect types or provide unstructured outputs that cannot directly guide maintenance crews. This paper proposes a comprehensive pipeline that leverages street CCTV streams for multi defect detection and segmentation using the YOLO family of object detectors and passes the detections to a vision language model (VLM) for scene aware summarization. The VLM generates a structured action plan in JSON format that includes incident descriptions, recommended tools, dimensions, repair plans, and urgent alerts. We review literature on pothole, crack and leak detection, highlight recent advances in large vision language models such as QwenVL and LLaVA, and describe the design of our early prototype. Experimental evaluation on public datasets and captured CCTV clips demonstrates that the system accurately identifies diverse defects and produces coherent summaries. We conclude by discussing challenges and directions for scaling the system to city wide deployments.

Method: Created a benchmark with 1,000 long-duration, information-dense videos from FineVideo dataset, designed six challenging task scenarios (Intra-Event and Inter-Event Tasks), and implemented a three-step semi-automated quality assurance pipeline.

Result: Omni-modal models still struggle with precise temporal localization and long-range causal inference tasks. Extended experiments revealed information loss and processing bias in multi-modal fusion.

Conclusion: LongInsightBench provides a comprehensive evaluation framework that reveals current limitations of omni-modal models in handling long video understanding, particularly for temporal and causal reasoning tasks.

Abstract: We introduce \textbf{LongInsightBench}, the first benchmark designed to assess models’ ability to understand long videos, with a focus on human language, viewpoints, actions, and other contextual elements, while integrating \textbf{visual, audio, and text} modalities. Our benchmark excels in three key areas: \textbf{a) Long-Duration, Information-Dense Videos:} We carefully select approximately 1,000 videos from open-source datasets FineVideo based on duration limit and the information density of both visual and audio modalities, focusing on content like lectures, interviews, and vlogs, which contain rich language elements. \textbf{b) Diverse and Challenging Task Scenarios:} We have designed six challenging task scenarios, including both Intra-Event and Inter-Event Tasks. \textbf{c) Rigorous and Comprehensive Quality Assurance Pipelines:} We have developed a three-step, semi-automated data quality assurance pipeline to ensure the difficulty and validity of the synthesized questions and answer options. Based on LongInsightBench, we designed a series of experiments. Experimental results shows that Omni-modal models(OLMs) still face challenge in tasks requiring precise temporal localization (T-Loc) and long-range causal inference (CE-Caus). Extended experiments reveal the information loss and processing bias in multi-modal fusion of OLMs. Our dataset and code is available at https://anonymous.4open.science/r/LongInsightBench-910F/.

Method: Uses Abnormal Prompt Generator for detailed anomaly prompts, Text-Guided Enhancer for visual grounding, and Multi-Mask Fusion module to incorporate mask as expert knowledge for pixel-level anomaly perception.

Result: Achieves state-of-the-art performance on MVTec-AD and VisA datasets for self-supervised and few-shot anomaly detection and segmentation tasks.

Conclusion: IAD-GPT effectively combines MLLMs’ causal capabilities with visual grounding techniques to advance industrial anomaly detection with detailed descriptions and dialogue capabilities.

Abstract: The robust causal capability of Multimodal Large Language Models (MLLMs) hold the potential of detecting defective objects in Industrial Anomaly Detection (IAD). However, most traditional IAD methods lack the ability to provide multi-turn human-machine dialogues and detailed descriptions, such as the color of objects, the shape of an anomaly, or specific types of anomalies. At the same time, methods based on large pre-trained models have not fully stimulated the ability of large models in anomaly detection tasks. In this paper, we explore the combination of rich text semantics with both image-level and pixel-level information from images and propose IAD-GPT, a novel paradigm based on MLLMs for IAD. We employ Abnormal Prompt Generator (APG) to generate detailed anomaly prompts for specific objects. These specific prompts from the large language model (LLM) are used to activate the detection and segmentation functions of the pre-trained visual-language model (i.e., CLIP). To enhance the visual grounding ability of MLLMs, we propose Text-Guided Enhancer, wherein image features interact with normal and abnormal text prompts to dynamically select enhancement pathways, which enables language models to focus on specific aspects of visual data, enhancing their ability to accurately interpret and respond to anomalies within images. Moreover, we design a Multi-Mask Fusion module to incorporate mask as expert knowledge, which enhances the LLM’s perception of pixel-level anomalies. Extensive experiments on MVTec-AD and VisA datasets demonstrate our state-of-the-art performance on self-supervised and few-shot anomaly detection and segmentation tasks, such as MVTec-AD and VisA datasets. The codes are available at \href{https://github.com/LiZeWen1225/IAD-GPT}{https://github.com/LiZeWen1225/IAD-GPT}.

Method: Prospective study with 8 readers (4 novice, 4 non-novice) analyzing 35 bedside chest radiographs each using three reporting modes: free-text (FT), structured reporting (SR), and AI-assisted structured reporting (AI-SR), with eye-tracking and timing measurements.

Result: AI-SR achieved highest diagnostic accuracy (κ=0.71 vs 0.58-0.60), fastest reporting times (25±9s vs 37±18s vs 88±38s), reduced saccade counts and fixation durations, and was the preferred reporting mode.

Conclusion: Structured reporting improves efficiency by guiding visual attention toward images, and AI-prefilled structured reporting further enhances diagnostic accuracy and user satisfaction.

Abstract: Structured reporting (SR) and artificial intelligence (AI) may transform how radiologists interact with imaging studies. This prospective study (July to December 2024) evaluated the impact of three reporting modes: free-text (FT), structured reporting (SR), and AI-assisted structured reporting (AI-SR), on image analysis behavior, diagnostic accuracy, efficiency, and user experience. Four novice and four non-novice readers (radiologists and medical students) each analyzed 35 bedside chest radiographs per session using a customized viewer and an eye-tracking system. Outcomes included diagnostic accuracy (compared with expert consensus using Cohen’s $\kappa$), reporting time per radiograph, eye-tracking metrics, and questionnaire-based user experience. Statistical analysis used generalized linear mixed models with Bonferroni post-hoc tests with a significance level of ($P \le .01$). Diagnostic accuracy was similar in FT ($\kappa = 0.58$) and SR ($\kappa = 0.60$) but higher in AI-SR ($\kappa = 0.71$, $P < .001$). Reporting times decreased from $88 \pm 38$ s (FT) to $37 \pm 18$ s (SR) and $25 \pm 9$ s (AI-SR) ($P < .001$). Saccade counts for the radiograph field ($205 \pm 135$ (FT), $123 \pm 88$ (SR), $97 \pm 58$ (AI-SR)) and total fixation duration for the report field ($11 \pm 5$ s (FT), $5 \pm 3$ s (SR), $4 \pm 1$ s (AI-SR)) were lower with SR and AI-SR ($P < .001$ each). Novice readers shifted gaze towards the radiograph in SR, while non-novice readers maintained their focus on the radiograph. AI-SR was the preferred mode. In conclusion, SR improves efficiency by guiding visual attention toward the image, and AI-prefilled SR further enhances diagnostic accuracy and user satisfaction.

Method: Introduces Intersectional Fairness Evaluation Framework (IFEF) with quantitative metrics and interpretability tools, plus Bias-Weighted Augmentation (BWA) that adapts transformation intensities based on subgroup statistics.

Result: On Open Images V7 dataset, BWA improves accuracy for underrepresented class-environment intersections by up to 24 percentage points, reduces fairness metric disparities by 35%, with statistically significant improvements (p < 0.05).

Conclusion: Provides a replicable methodology for analyzing and addressing intersectional biases in image classification systems.

Abstract: Machine learning models trained on imbalanced datasets often exhibit intersectional biases-systematic errors arising from the interaction of multiple attributes such as object class and environmental conditions. This paper presents a data-driven framework for analyzing and mitigating such biases in image classification. We introduce the Intersectional Fairness Evaluation Framework (IFEF), which combines quantitative fairness metrics with interpretability tools to systematically identify bias patterns in model predictions. Building on this analysis, we propose Bias-Weighted Augmentation (BWA), a novel data augmentation strategy that adapts transformation intensities based on subgroup distribution statistics. Experiments on the Open Images V7 dataset with five object classes demonstrate that BWA improves accuracy for underrepresented class-environment intersections by up to 24 percentage points while reducing fairness metric disparities by 35%. Statistical analysis across multiple independent runs confirms the significance of improvements (p < 0.05). Our methodology provides a replicable approach for analyzing and addressing intersectional biases in image classification systems.

Method: The authors propose a differentiable quantization approach that supports logarithmic quantization of values in the form 2^n, enabling n-bit quantization without requiring higher precision multiplication.

Result: When tested on ImageNet with ResNet18, the method achieved less than 1% accuracy drop compared to full precision with weight-only quantization, and achieved state-of-the-art accuracy with both weight and activation quantization in just 15 training epochs.

Conclusion: The proposed quantization method provides a differentiable, convergent approach that achieves high accuracy with minimal training epochs and reasonable inference costs, addressing key limitations of previous quantization techniques.

Abstract: Quantization of neural networks provides benefits of inference in less compute and memory requirements. Previous work in quantization lack two important aspects which this work provides. First almost all previous work in quantization used a non-differentiable approach and for learning; the derivative is usually set manually in backpropogation which make the learning ability of algorithm questionable, our approach is not just differentiable, we also provide proof of convergence of our approach to the optimal neural network. Second previous work in shift/logrithmic quantization either have avoided activation quantization along with weight quantization or achieved less accuracy. Learning logrithmic quantize values of form $2^n$ requires the quantization function can scale to more than 1 bit quantization which is another benifit of our quantization that it provides $n$ bits quantization as well. Our approach when tested with image classification task using imagenet dataset, resnet18 and weight quantization only achieves less than 1 percent accuracy compared to full precision accuracy while taking only 15 epochs to train using shift bit quantization and achieves comparable to SOTA approaches accuracy in both weight and activation quantization using shift bit quantization in 15 training epochs with slightly higher(only higher cpu instructions) inference cost compared to 1 bit quantization(without logrithmic quantization) and not requiring any higher precision multiplication.

Method: StripRFNet comprises three modules: (1) Shape Perception Module (SPM) with large separable kernel attention for shape discrimination; (2) Strip Receptive Field Module (SRFM) with large strip convolutions and pooling for slender cracks; (3) Small-Scale Enhancement Module (SSEM) with high-resolution P2 feature map, dedicated detection head, and dynamic upsampling for small-object detection.

Result: On RDD2022 benchmark: Chinese subset improved F1-score, mAP50, and mAP50:95 by 4.4, 2.9, and 3.4 percentage points over baseline. On full dataset, achieved highest F1-score of 80.33% compared to CRDDC'2022 participants and ORDDC'2024 Phase 2 results, while maintaining competitive inference speed.

Conclusion: StripRFNet achieves state-of-the-art accuracy and real-time efficiency, offering a promising tool for intelligent road maintenance and sustainable infrastructure management.

Abstract: Well-maintained road networks are crucial for achieving Sustainable Development Goal (SDG) 11. Road surface damage not only threatens traffic safety but also hinders sustainable urban development. Accurate detection, however, remains challenging due to the diverse shapes of damages, the difficulty of capturing slender cracks with high aspect ratios, and the high error rates in small-scale damage recognition. To address these issues, we propose StripRFNet, a novel deep neural network comprising three modules: (1) a Shape Perception Module (SPM) that enhances shape discrimination via large separable kernel attention (LSKA) in multi-scale feature aggregation; (2) a Strip Receptive Field Module (SRFM) that employs large strip convolutions and pooling to capture features of slender cracks; and (3) a Small-Scale Enhancement Module (SSEM) that leverages a high-resolution P2 feature map, a dedicated detection head, and dynamic upsampling to improve small-object detection. Experiments on the RDD2022 benchmark show that StripRFNet surpasses existing methods. On the Chinese subset, it improves F1-score, mAP50, and mAP50:95 by 4.4, 2.9, and 3.4 percentage points over the baseline, respectively. On the full dataset, it achieves the highest F1-score of 80.33% compared with CRDDC'2022 participants and ORDDC'2024 Phase 2 results, while maintaining competitive inference speed. These results demonstrate that StripRFNet achieves state-of-the-art accuracy and real-time efficiency, offering a promising tool for intelligent road maintenance and sustainable infrastructure management.

Method: Uses object-specific transformations: color space perturbations on individual objects during training, diffusion models to generate diverse pedestrian instances, and applies object perturbations at inference to quantify uncertainty using detection score variance.

Result: Experiments with YOLOv8 on NuImages dataset show notable accuracy improvements, uncertainty reduction across all object classes, and higher uncertainty values for false positives compared to true positives.

Conclusion: ObjectTransforms is a lightweight yet effective mechanism for reducing uncertainty during training and quantifying uncertainty during inference in vision-based perception systems.

Abstract: Reliable perception is fundamental for safety critical decision making in autonomous driving. Yet, vision based object detector neural networks remain vulnerable to uncertainty arising from issues such as data bias and distributional shifts. In this paper, we introduce ObjectTransforms, a technique for quantifying and reducing uncertainty in vision based object detection through object specific transformations at both training and inference times. At training time, ObjectTransforms perform color space perturbations on individual objects, improving robustness to lighting and color variations. ObjectTransforms also uses diffusion models to generate realistic, diverse pedestrian instances. At inference time, object perturbations are applied to detected objects and the variance of detection scores are used to quantify predictive uncertainty in real time. This uncertainty signal is then used to filter out false positives and also recover false negatives, improving the overall precision recall curve. Experiments with YOLOv8 on the NuImages 10K dataset demonstrate that our method yields notable accuracy improvements and uncertainty reduction across all object classes during training, while predicting desirably higher uncertainty values for false positives as compared to true positives during inference. Our results highlight the potential of ObjectTransforms as a lightweight yet effective mechanism for reducing and quantifying uncertainty in vision-based perception during training and inference respectively.

Method: Proposes TM-Mamba with text-controlled selection mechanism that dynamically incorporates global temporal information based on text queries, enhanced with relational embeddings for spatial graph topology awareness.

Result: Extensive evaluations on the new BABEL-Grounding dataset demonstrate the effectiveness of TM-Mamba for text-based motion grounding tasks.

Conclusion: TM-Mamba successfully addresses the computational challenges of long untrimmed motion sequences while effectively performing temporal localization based on text descriptions.

Abstract: Human motion understanding is a fundamental task with diverse practical applications, facilitated by the availability of large-scale motion capture datasets. Recent studies focus on text-motion tasks, such as text-based motion generation, editing and question answering. In this study, we introduce the novel task of text-based human motion grounding (THMG), aimed at precisely localizing temporal segments corresponding to given textual descriptions within untrimmed motion sequences. Capturing global temporal information is crucial for the THMG task. However, Transformer-based models that rely on global temporal self-attention face challenges when handling long untrimmed sequences due to the quadratic computational cost. We address these challenges by proposing Text-controlled Motion Mamba (TM-Mamba), a unified model that integrates temporal global context, language query control, and spatial graph topology with only linear memory cost. The core of the model is a text-controlled selection mechanism which dynamically incorporates global temporal information based on text query. The model is further enhanced to be topology-aware through the integration of relational embeddings. For evaluation, we introduce BABEL-Grounding, the first text-motion dataset that provides detailed textual descriptions of human actions along with their corresponding temporal segments. Extensive evaluations demonstrate the effectiveness of TM-Mamba on BABEL-Grounding.

Method: Dataset collection using Aria Gen 2 glasses with a primary subject and friends recording daily activities in five scenarios (cleaning, cooking, eating, playing, outdoor walking), providing both raw sensor data and processed outputs from machine perception algorithms.

Result: A comprehensive publicly available dataset at projectaria.com with open-source tools and usage examples, demonstrating robust performance across diverse users and conditions.

Conclusion: A2PD serves as a valuable resource for egocentric AI research, enabling the development and evaluation of perception algorithms for understanding human activities and interactions with the environment.

Abstract: The Aria Gen 2 Pilot Dataset (A2PD) is an egocentric multimodal open dataset captured using the state-of-the-art Aria Gen 2 glasses. To facilitate timely access, A2PD is released incrementally with ongoing dataset enhancements. The initial release features Dia’ane, our primary subject, who records her daily activities alongside friends, each equipped with Aria Gen 2 glasses. It encompasses five primary scenarios: cleaning, cooking, eating, playing, and outdoor walking. In each of the scenarios, we provide comprehensive raw sensor data and output data from various machine perception algorithms. These data illustrate the device’s ability to perceive the wearer, the surrounding environment, and interactions between the wearer and the environment, while maintaining robust performance across diverse users and conditions. The A2PD is publicly available at projectaria.com, with open-source tools and usage examples provided in Project Aria Tools.

Method: Teacher uses dual streams (original dark frames + retinex-enhanced frames) with weight-shared R(2+1)D-34 backbones and Dynamic Feature Fusion module. Student uses only dark frames and is pre-trained with self-supervision then fine-tuned with knowledge distillation from teacher.

Result: State-of-the-art accuracy: 96.92% (Top-1) on ARID V1.0, 88.27% on ARID V1.5, and 48.96% on Dark48 under single-stream inference.

Conclusion: The framework successfully transfers multi-stream knowledge to single-stream model, with Dynamic Feature Fusion outperforming static fusion, knowledge distillation enabling transfer of gains, and spatio-temporal SSL surpassing spatial/temporal-only variants.

Abstract: Action recognition in dark or low-light (under-exposed) videos is a challenging task due to visibility degradation, which can hinder critical spatiotemporal details. This paper proposes ActLumos, a teacher-student framework that attains single-stream inference while retaining multi-stream level accuracy. The teacher consumes dual stream inputs, which include original dark frames and retinex-enhanced frames, processed by weight-shared R(2+1)D-34 backbones and fused by a Dynamic Feature Fusion (DFF) module, which dynamically re-weights the two streams at each time step, emphasising the most informative temporal segments. The teacher is also included with a supervised contrastive loss (SupCon) that sharpens class margins. The student shares the R(2+1)D-34 backbone but uses only dark frames and no fusion at test time. The student is first pre-trained with self-supervision on dark clips of both datasets without their labels and then fine-tuned with knowledge distillation from the teacher, transferring the teacher’s multi-stream knowledge into a single-stream model. Under single-stream inference, the distilled student attains state-of-the-art accuracy of 96.92% (Top-1) on ARID V1.0, 88.27% on ARID V1.5, and 48.96% on Dark48. Ablation studies further highlight the individual contributions of each component, i.e., DFF in the teacher outperforms single or static fusion, knowledge distillation (KD) transfers these gains to the single-stream student, and two-view spatio-temporal SSL surpasses spatial-only or temporal-only variants without increasing inference cost. The official website of this work is available at: https://github.com/HrishavBakulBarua/ActLumos

Method: Proposed a generalized selection method for class representations that are not limited to centroids, allowing adjustable number of representations per class to meet application requirements.

Result: The approach substantially improves re-identification performance across multiple embeddings, achieving better balance between accuracy and mean average precision.

Conclusion: The generalized representation selection method effectively addresses limitations of prior centroid-based approaches and advances person re-identification performance beyond current state-of-the-art.

Abstract: Advanced feature extraction methods have significantly contributed to enhancing the task of person re-identification. In addition, modifications to objective functions have been developed to further improve performance. Nonetheless, selecting better class representatives is an underexplored area of research that can also lead to advancements in re-identification performance. Although past works have experimented with using the centroid of a gallery image class during training, only a few have investigated alternative representations during the retrieval stage. In this paper, we demonstrate that these prior techniques yield suboptimal results in terms of re-identification metrics. To address the re-identification problem, we propose a generalized selection method that involves choosing representations that are not limited to class centroids. Our approach strikes a balance between accuracy and mean average precision, leading to improvements beyond the state of the art. For example, the actual number of representations per class can be adjusted to meet specific application requirements. We apply our methodology on top of multiple re-identification embeddings, and in all cases it substantially improves upon contemporary results

Method: Uses pretrained rectified flow models conditioned on image/text with periodic guidance addition during sampling, implemented as differentiable loss functions including part-aware appearance and self-similarity losses.

Result: Successfully transfers texture and geometric details to 3D assets, outperforming baselines both qualitatively and quantitatively. Traditional metrics are unsuitable, so evaluation uses GPT-based ranking system.

Conclusion: The method is general and can be extended to different diffusion models and guidance functions, with robust evaluation confirmed by user studies.

Abstract: Transferring appearance to 3D assets using different representations of the appearance object - such as images or text - has garnered interest due to its wide range of applications in industries like gaming, augmented reality, and digital content creation. However, state-of-the-art methods still fail when the geometry between the input and appearance objects is significantly different. A straightforward approach is to directly apply a 3D generative model, but we show that this ultimately fails to produce appealing results. Instead, we propose a principled approach inspired by universal guidance. Given a pretrained rectified flow model conditioned on image or text, our training-free method interacts with the sampling process by periodically adding guidance. This guidance can be modeled as a differentiable loss function, and we experiment with two different types of guidance including part-aware losses for appearance and self-similarity. Our experiments show that our approach successfully transfers texture and geometric details to the input 3D asset, outperforming baselines both qualitatively and quantitatively. We also show that traditional metrics are not suitable for evaluating the task due to their inability of focusing on local details and comparing dissimilar inputs, in absence of ground truth data. We thus evaluate appearance transfer quality with a GPT-based system objectively ranking outputs, ensuring robust and human-like assessment, as further confirmed by our user study. Beyond showcased scenarios, our method is general and could be extended to different types of diffusion models and guidance functions.

Method: MIRROR uses dedicated encoders for each modality, a modality alignment module for integration, a modality retention module to preserve unique attributes, and a style clustering module to reduce redundancy and align pathological signatures in clustering space.

Result: Extensive evaluations on TCGA cohorts for cancer subtyping and survival analysis demonstrate MIRROR’s superior performance in constructing comprehensive oncological feature representations.

Conclusion: MIRROR effectively integrates histopathology and transcriptomics while maintaining modality-specific fidelity, benefiting cancer diagnosis through comprehensive feature representations.

Abstract: Histopathology and transcriptomics are fundamental modalities in oncology, encapsulating the morphological and molecular aspects of the disease. Multi-modal self-supervised learning has demonstrated remarkable potential in learning pathological representations by integrating diverse data sources. Conventional multi-modal integration methods primarily emphasize modality alignment, while paying insufficient attention to retaining the modality-specific structures. However, unlike conventional scenarios where multi-modal inputs share highly overlapping features, histopathology and transcriptomics exhibit pronounced heterogeneity, offering orthogonal yet complementary insights. Histopathology provides morphological and spatial context, elucidating tissue architecture and cellular topology, whereas transcriptomics delineates molecular signatures through gene expression patterns. This inherent disparity introduces a major challenge in aligning them while maintaining modality-specific fidelity. To address these challenges, we present MIRROR, a novel multi-modal representation learning method designed to foster both modality alignment and retention. MIRROR employs dedicated encoders to extract comprehensive features for each modality, which is further complemented by a modality alignment module to achieve seamless integration between phenotype patterns and molecular profiles. Furthermore, a modality retention module safeguards unique attributes from each modality, while a style clustering module mitigates redundancy and enhances disease-relevant information by modeling and aligning consistent pathological signatures within a clustering space. Extensive evaluations on TCGA cohorts for cancer subtyping and survival analysis highlight MIRROR’s superior performance, demonstrating its effectiveness in constructing comprehensive oncological feature representations and benefiting the cancer diagnosis.

Method: Self-supervised framework that classifies skeletal landmarks using regression-based pretext tasks.

Result: Model outperforms existing methods in regression and classification; positional pretext task significantly improves downstream classification.

Conclusion: Framework shows promise for autonomous C-arm control to optimize trajectories from pelvis to head during thrombectomy procedures.

Abstract: Thrombectomy is one of the most effective treatments for ischemic stroke, but it is resource and personnel-intensive. We propose employing deep learning to automate critical aspects of thrombectomy, thereby enhancing efficiency and safety. In this work, we introduce a self-supervised framework that classifies various skeletal landmarks using a regression-based pretext task. Our experiments demonstrate that our model outperforms existing methods in both regression and classification tasks. Notably, our results indicate that the positional pretext task significantly enhances downstream classification performance. Future work will focus on extending this framework toward fully autonomous C-arm control, aiming to optimize trajectories from the pelvis to the head during stroke thrombectomy procedures. All code used is available at https://github.com/AhmadArrabi/C_arm_guidance

Method: Dual-branch framework with asynchronous optimization (each branch optimizes either encoder or decoder while keeping the other frozen), Decoupled Dropout Perturbation for regularization, Pair-wise CutMix Cross-Guidance for model diversity, and Consistency Matching to mitigate confirmation bias from noisy pseudo-labels.

Result: Extensive experiments on benchmark brain MRI segmentation datasets (ISLES2022 and BraTS) show that DuetMatch consistently outperforms state-of-the-art methods.

Conclusion: DuetMatch demonstrates effectiveness and robustness across diverse semi-supervised segmentation scenarios in medical imaging.

Abstract: The limited availability of annotated data in medical imaging makes semi-supervised learning increasingly appealing for its ability to learn from imperfect supervision. Recently, teacher-student frameworks have gained popularity for their training benefits and robust performance. However, jointly optimizing the entire network can hinder convergence and stability, especially in challenging scenarios. To address this for medical image segmentation, we propose DuetMatch, a novel dual-branch semi-supervised framework with asynchronous optimization, where each branch optimizes either the encoder or decoder while keeping the other frozen. To improve consistency under noisy conditions, we introduce Decoupled Dropout Perturbation, enforcing regularization across branches. We also design Pair-wise CutMix Cross-Guidance to enhance model diversity by exchanging pseudo-labels through augmented input pairs. To mitigate confirmation bias from noisy pseudo-labels, we propose Consistency Matching, refining labels using stable predictions from frozen teacher models. Extensive experiments on benchmark brain MRI segmentation datasets, including ISLES2022 and BraTS, show that DuetMatch consistently outperforms state-of-the-art methods, demonstrating its effectiveness and robustness across diverse semi-supervised segmentation scenarios.

Method: Formulates video classification as sequence alignment: LLM generates ordered sub-action sequences for each class, then aligns them with video frames using Dynamic Time Warping in a shared embedding space. Training-free and model-agnostic.

Result: Achieves 30.5% accuracy on ActionAtlas benchmark (human performance: 61.6%), outperforms billion-parameter video-language models while using 8x fewer parameters. Demonstrates domain-general applicability.

Conclusion: Structured language priors combined with classical alignment methods can unlock the open-set recognition potential of image-language models for fine-grained video understanding without video-text supervision or fine-tuning.

Abstract: We address the task of zero-shot video classification for extremely fine-grained actions (e.g., Windmill Dunk in basketball), where no video examples or temporal annotations are available for unseen classes. While image-language models (e.g., CLIP, SigLIP) show strong open-set recognition, they lack temporal modeling needed for video understanding. We propose ActAlign, a truly zero-shot, training-free method that formulates video classification as a sequence alignment problem, preserving the generalization strength of pretrained image-language models. For each class, a large language model (LLM) generates an ordered sequence of sub-actions, which we align with video frames using Dynamic Time Warping (DTW) in a shared embedding space. Without any video-text supervision or fine-tuning, ActAlign achieves 30.5% accuracy on ActionAtlas–the most diverse benchmark of fine-grained actions across multiple sports–where human performance is only 61.6%. ActAlign outperforms billion-parameter video-language models while using 8x fewer parameters. Our approach is model-agnostic and domain-general, demonstrating that structured language priors combined with classical alignment methods can unlock the open-set recognition potential of image-language models for fine-grained video understanding.

Method: Uses X-ray images from arbitrary starting positions to predict 3D displacement vectors to target landmarks, incorporates aleatoric and epistemic uncertainty modeling with conformal prediction, and combines probabilistic loss with skeletal pose regularization.

Result: Strong localization accuracy across multiple architectures with well-calibrated prediction bounds on synthetic X-ray data from DeepDRR.

Conclusion: The pipeline shows potential as a component for safe and reliable autonomous C-arm systems in clinical workflows.

Abstract: Accurate and reliable C-arm positioning is essential for fluoroscopy-guided interventions. However, clinical workflows rely on manual alignment that increases radiation exposure and procedural delays. In this work, we present a pipeline that autonomously navigates the C-arm to predefined anatomical landmarks utilizing X-ray images. Given an input X-ray image from an arbitrary starting location on the operating table, the model predicts a 3D displacement vector toward each target landmark along the body. To ensure reliable deployment, we capture both aleatoric and epistemic uncertainties in the model’s predictions and further calibrate them using conformal prediction. The derived prediction regions are interpreted as 3D confidence regions around the predicted landmark locations. The training framework combines a probabilistic loss with skeletal pose regularization to encourage anatomically plausible outputs. We validate our approach on a synthetic X-ray dataset generated from DeepDRR. Results show not only strong localization accuracy across multiple architectures but also well-calibrated prediction bounds. These findings highlight the pipeline’s potential as a component in safe and reliable autonomous C-arm systems. Code is available at https://github.com/AhmadArrabi/C_arm_guidance_APAH

Method: Developed a formula to estimate cost savings based on precision and pass yield of IQA engines, and applied it with AutoML in background inpainting use case.

Result: Achieved significant 51.61% cost saving in background inpainting scenario using simple AutoML solution for automatic IQA pre-filtering.

Conclusion: Automatic IQA pre-filtering can substantially reduce costs in generative AI workflows, with the presented formula providing a framework for estimating potential savings.

Abstract: Deep generative models have shown impressive progress in recent years, making it possible to produce high quality images with a simple text prompt or a reference image. However, state of the art technology does not yet meet the quality standards offered by traditional photographic methods. For this reason, production pipelines that use generated images often include a manual stage of image quality assessment (IQA). This process is slow and expensive, especially because of the low yield of automatically generated images that pass the quality bar. The IQA workload can be reduced by introducing an automatic pre-filtering stage, that will increase the overall quality of the images sent to review and, therefore, reduce the average cost required to obtain a high quality image. We present a formula that estimates the cost savings depending on the precision and pass yield of a generic IQA engine. This formula is applied in a use case of background inpainting, showcasing a significant cost saving of 51.61% obtained with a simple AutoML solution.

Method: Projects fMRI signals into structured text space using PRISM model, includes object-centric diffusion module for image composition and attribute-relationship search module to align with neural activity.

Result: Extensive experiments show PRISM outperforms existing methods with up to 8% reduction in perceptual loss on real-world datasets.

Conclusion: Structured text space is superior for bridging fMRI signals and image reconstruction, capturing the compositional nature of visual stimuli effectively.

Abstract: Understanding how the brain encodes visual information is a central challenge in neuroscience and machine learning. A promising approach is to reconstruct visual stimuli, essentially images, from functional Magnetic Resonance Imaging (fMRI) signals. This involves two stages: transforming fMRI signals into a latent space and then using a pretrained generative model to reconstruct images. The reconstruction quality depends on how similar the latent space is to the structure of neural activity and how well the generative model produces images from that space. Yet, it remains unclear which type of latent space best supports this transformation and how it should be organized to represent visual stimuli effectively. We present two key findings. First, fMRI signals are more similar to the text space of a language model than to either a vision based space or a joint text image space. Second, text representations and the generative model should be adapted to capture the compositional nature of visual stimuli, including objects, their detailed attributes, and relationships. Building on these insights, we propose PRISM, a model that Projects fMRI sIgnals into a Structured text space as an interMediate representation for visual stimuli reconstruction. It includes an object centric diffusion module that generates images by composing individual objects to reduce object detection errors, and an attribute relationship search module that automatically identifies key attributes and relationships that best align with the neural activity. Extensive experiments on real world datasets demonstrate that our framework outperforms existing methods, achieving up to an 8% reduction in perceptual loss. These results highlight the importance of using structured text as the intermediate space to bridge fMRI signals and image reconstruction.

Method: The paper advocates a Data-Centric AI perspective, reviewing and prioritizing techniques like confident learning, core-set selection, data augmentation, and active learning. It identifies 25 distinct strategies and proposes a practical pipeline using the 9 most mature methods.

Result: The paper highlights the readiness and suitability of 25 data-centric strategies for large-scale agricultural mapping pipelines, with a focus on tropical contexts where traditional approaches are limited.

Conclusion: A data-centric approach using curated practical methods is better suited for tropical agriculture mapping, addressing the dynamic realities and limitations of traditional model-centric approaches in these challenging environments.

Abstract: Mapping agriculture in tropical areas through remote sensing presents unique challenges, including the lack of high-quality annotated data, the elevated costs of labeling, data variability, and regional generalisation. This paper advocates a Data-Centric Artificial Intelligence (DCAI) perspective and pipeline, emphasizing data quality and curation as key drivers for model robustness and scalability. It reviews and prioritizes techniques such as confident learning, core-set selection, data augmentation, and active learning. The paper highlights the readiness and suitability of 25 distinct strategies in large-scale agricultural mapping pipelines. The tropical context is of high interest, since high cloudiness, diverse crop calendars, and limited datasets limit traditional model-centric approaches. This tutorial outlines practical solutions as a data-centric approach for curating and training AI models better suited to the dynamic realities of tropical agriculture. Finally, we propose a practical pipeline using the 9 most mature and straightforward methods that can be applied to a large-scale tropical agricultural mapping project.

Method: Samples videos at varying temporal and spatial resolutions during training and dynamically interpolates model weights to accommodate any spatio-temporal scale. Introduces and compares five variants of flexible training for video SSMs.

Result: StretchySnake outperforms transformer and SSM baselines by up to 28% on short-action (UCF-101, HMDB-51) and long-action (COIN, Breakfast) benchmarks, with strong adaptability to fine-grained actions (SSV2, Diving-48).

Conclusion: The method provides a simple drop-in training recipe that makes video SSMs more robust, resolution-agnostic, and efficient across diverse action recognition scenarios.

Abstract: State space models (SSMs) have emerged as a competitive alternative to transformers in various tasks. Their linear complexity and hidden-state recurrence make them particularly attractive for modeling long sequences, whereas attention becomes quadratically expensive. However, current training methods for video understanding are tailored towards transformers and fail to fully leverage the unique attributes of SSMs. For example, video models are often trained at a fixed resolution and video length to balance the quadratic scaling of attention cost against performance. Consequently, these models suffer from degraded performance when evaluated on videos with spatial and temporal resolutions unseen during training; a property we call spatio-temporal inflexibility. In the context of action recognition, this severely limits a model’s ability to retain performance across both short- and long-form videos. Therefore, we propose a flexible training method that leverages and improves the inherent adaptability of SSMs. Our method samples videos at varying temporal and spatial resolutions during training and dynamically interpolates model weights to accommodate any spatio-temporal scale. This instills our SSM, which we call StretchySnake, with spatio-temporal flexibility and enables it to seamlessly handle videos ranging from short, fine-grained clips to long, complex activities. We introduce and compare five different variants of flexible training, and identify the most effective strategy for video SSMs. On short-action (UCF-101, HMDB-51) and long-action (COIN, Breakfast) benchmarks, StretchySnake outperforms transformer and SSM baselines alike by up to 28%, with strong adaptability to fine-grained actions (SSV2, Diving-48). Therefore, our method provides a simple drop-in training recipe that makes video SSMs more robust, resolution-agnostic, and efficient across diverse action recognition scenarios.

Method: Proposed VM-BeautyNet, a heterogeneous ensemble architecture that fuses Vision Transformer (for global facial structure and symmetry) with Mamba-based Vision model (for efficient long-range dependency modeling with linear complexity). Uses complementary feature extraction from both backbones.

Result: Achieved state-of-the-art performance on SCUT-FBP5500 dataset: Pearson Correlation of 0.9212, MAE of 0.2085, RMSE of 0.2698. Grad-CAM visualizations confirmed complementary feature extraction between the two backbones.

Conclusion: VM-BeautyNet presents a powerful new architectural paradigm for computational aesthetics, successfully combining global feature capture with efficient long-range dependency modeling, while providing interpretable insights into the decision-making process.

Abstract: Facial Beauty Prediction (FBP) is a complex and challenging computer vision task, aiming to model the subjective and intricate nature of human aesthetic perception. While deep learning models, particularly Convolutional Neural Networks (CNNs), have made significant strides, they often struggle to capture the global, holistic facial features that are critical to human judgment. Vision Transformers (ViT) address this by effectively modeling long-range spatial relationships, but their quadratic complexity can be a bottleneck. This paper introduces a novel, heterogeneous ensemble architecture, \textbf{VM-BeautyNet}, that synergistically fuses the complementary strengths of a Vision Transformer and a Mamba-based Vision model, a recent advancement in State-Space Models (SSMs). The ViT backbone excels at capturing global facial structure and symmetry, while the Mamba backbone efficiently models long-range dependencies with linear complexity, focusing on sequential features and textures. We evaluate our approach on the benchmark SCUT-FBP5500 dataset. Our proposed VM-BeautyNet achieves state-of-the-art performance, with a \textbf{Pearson Correlation (PC) of 0.9212}, a \textbf{Mean Absolute Error (MAE) of 0.2085}, and a \textbf{Root Mean Square Error (RMSE) of 0.2698}. Furthermore, through Grad-CAM visualizations, we provide interpretability analysis that confirms the complementary feature extraction of the two backbones, offering new insights into the model’s decision-making process and presenting a powerful new architectural paradigm for computational aesthetics.

Method: Trained a CNN on 4293 images of benign/malignant tumors and negative samples, tested on images at 5 resolutions, and developed hardware for detailed image capture.

Result: Higher resolution images led to more accurate predictions on a logarithmic scale, showing diminishing returns with increased pixel counts.

Conclusion: CNN with proper image resolution can effectively detect OCSCC, and hardware enhancement improves detection accuracy.

Abstract: Oral Cavity Squamous Cell Carcinoma (OCSCC) is the most common type of head and neck cancer. Due to the subtle nature of its early stages, deep and hidden areas of development, and slow growth, OCSCC often goes undetected, leading to preventable deaths. However, properly trained Convolutional Neural Networks (CNNs), with their precise image segmentation techniques and ability to apply kernel matrices to modify the RGB values of images for accurate image pattern recognition, would be an effective means for early detection of OCSCC. Pairing this neural network with image capturing and processing hardware would allow increased efficacy in OCSCC detection. The aim of our project is to develop a Convolutional Neural Network trained to recognize OCSCC, as well as to design a physical hardware system to capture and process detailed images, in order to determine the image quality required for accurate predictions. A CNN was trained on 4293 training images consisting of benign and malignant tumors, as well as negative samples, and was evaluated for its precision, recall, and Mean Average Precision (mAP) in its predictions of OCSCC. A testing dataset of randomly assorted images of cancerous, non-cancerous, and negative images was chosen, and each image was altered to represent 5 common resolutions. This test data set was thoroughly analyzed by the CNN and predictions were scored on the basis of accuracy. The designed enhancement hardware was used to capture detailed images, and its impact was scored. An application was developed to facilitate the testing process and bring open access to the CNN. Images of increasing resolution resulted in higher-accuracy predictions on a logarithmic scale, demonstrating the diminishing returns of higher pixel counts.

Method: Collected data from 439 participants in a multi-camera stage, capturing 54 million frames of tracked 3D motion including hand tracking, body shape, text annotations, and separate audio tracks for each participant.

Result: Successfully created a dataset with 500 hours of 3D motion data covering prompted motions, hand gestures, locomotion, discussions, emotional conversations, collaborative activities, and co-living scenarios.

Conclusion: Embody 3D provides a valuable resource for research in human motion analysis, behavioral studies, and multimodal interaction modeling, with its comprehensive tracking and diverse interaction scenarios.

Abstract: The Codec Avatars Lab at Meta introduces Embody 3D, a multimodal dataset of 500 individual hours of 3D motion data from 439 participants collected in a multi-camera collection stage, amounting to over 54 million frames of tracked 3D motion. The dataset features a wide range of single-person motion data, including prompted motions, hand gestures, and locomotion; as well as multi-person behavioral and conversational data like discussions, conversations in different emotional states, collaborative activities, and co-living scenarios in an apartment-like space. We provide tracked human motion including hand tracking and body shape, text annotations, and a separate audio track for each participant.

Method: Online approach that integrates camera/object pose estimation, instance decomposition, and map updating using human-object interactions from egocentric live streams, aided by Gaussian splatting for photorealistic rendering.

Result: Achieves accurate and consistent dynamic scene modeling with photorealistic and efficient rendering, validated in multiple real-world scenarios.

Conclusion: The system provides a flexible, progressive alternative to conventional object-level reconstruction methods by capitalizing on human-object interaction cues.

Abstract: Human behaviors are the major causes of scene dynamics and inherently contain rich cues regarding the dynamics. This paper formalizes a new task of proactive scene decomposition and reconstruction, an online approach that leverages human-object interactions to iteratively disassemble and reconstruct the environment. By observing these intentional interactions, we can dynamically refine the decomposition and reconstruction process, addressing inherent ambiguities in static object-level reconstruction. The proposed system effectively integrates multiple tasks in dynamic environments such as accurate camera and object pose estimation, instance decomposition, and online map updating, capitalizing on cues from human-object interactions in egocentric live streams for a flexible, progressive alternative to conventional object-level reconstruction methods. Aided by the Gaussian splatting technique, accurate and consistent dynamic scene modeling is achieved with photorealistic and efficient rendering. The efficacy is validated in multiple real-world scenarios with promising advantages.

Method: Two-stage cascaded system: 1) Learns normal behavioral rules offline, 2) Combines lightweight filtering with fine-grained VLM reasoning during online inference. Uses motion mask prompting and rule-based deviation detection to focus VLM attention on relevant motion regions and identify anomalies as deviations from learned norms.

Result: Achieves 57.68 fps on NVIDIA L40S GPU (151.79x speedup compared to VLM-based methods) with 97.2% accuracy comparable to state-of-the-art VLM-based VAD methods across four datasets.

Conclusion: Cerberus establishes itself as a practical solution for real-time video analytics by balancing efficiency and accuracy through its cascaded architecture and innovative attention mechanisms.

Abstract: Video anomaly detection (VAD) has rapidly advanced by recent development of Vision-Language Models (VLMs). While these models offer superior zero-shot detection capabilities, their immense computational cost and unstable visual grounding performance hinder real-time deployment. To overcome these challenges, we introduce Cerberus, a two-stage cascaded system designed for efficient yet accurate real-time VAD. Cerberus learns normal behavioral rules offline, and combines lightweight filtering with fine-grained VLM reasoning during online inference. The performance gains of Cerberus come from two key innovations: motion mask prompting and rule-based deviation detection. The former directs the VLM’s attention to regions relevant to motion, while the latter identifies anomalies as deviations from learned norms rather than enumerating possible anomalies. Extensive evaluations on four datasets show that Cerberus on average achieves 57.68 fps on an NVIDIA L40S GPU, a 151.79$\times$ speedup, and 97.2% accuracy comparable to the state-of-the-art VLM-based VAD methods, establishing it as a practical solution for real-time video analytics.

Method: Created a controlled benchmark with 6,000 images where member and non-member sample distributions are carefully balanced, with ground-truth membership labels provided across three distinct training stages.

Result: Experiments showed that state-of-the-art MIA methods converged to random chance performance under unbiased conditions, revealing that previous high success rates were artifacts of dataset bias.

Conclusion: OpenLVLM-MIA provides a transparent and unbiased benchmark that clarifies current limitations of MIA research on LVLMs and offers a solid foundation for developing stronger privacy-preserving techniques.

Abstract: OpenLVLM-MIA is a new benchmark that highlights fundamental challenges in evaluating membership inference attacks (MIA) against large vision-language models (LVLMs). While prior work has reported high attack success rates, our analysis suggests that these results often arise from detecting distributional bias introduced during dataset construction rather than from identifying true membership status. To address this issue, we introduce a controlled benchmark of 6{,}000 images where the distributions of member and non-member samples are carefully balanced, and ground-truth membership labels are provided across three distinct training stages. Experiments using OpenLVLM-MIA demonstrated that the performance of state-of-the-art MIA methods converged to random chance under unbiased conditions. By offering a transparent and unbiased benchmark, OpenLVLM-MIA clarifies the current limitations of MIA research on LVLMs and provides a solid foundation for developing stronger privacy-preserving techniques.

Method: Proposes cross-image stroke attention mechanism embedded in self-attention layers to establish fine-grained semantic correspondences for stroke attribute transfer, plus adaptive contrast enhancement and semantic-focused attention for content preservation.

Result: Effectively synthesizes stylistically faithful sketches resembling handcrafted results, outperforming existing methods in expressive stroke control and semantic coherence.

Conclusion: Stroke2Sketch provides an effective training-free solution for style-guided sketch generation with accurate stroke attribute transfer and strong content preservation capabilities.

Abstract: Generating sketches guided by reference styles requires precise transfer of stroke attributes, such as line thickness, deformation, and texture sparsity, while preserving semantic structure and content fidelity. To this end, we propose Stroke2Sketch, a novel training-free framework that introduces cross-image stroke attention, a mechanism embedded within self-attention layers to establish fine-grained semantic correspondences and enable accurate stroke attribute transfer. This allows our method to adaptively integrate reference stroke characteristics into content images while maintaining structural integrity. Additionally, we develop adaptive contrast enhancement and semantic-focused attention to reinforce content preservation and foreground emphasis. Stroke2Sketch effectively synthesizes stylistically faithful sketches that closely resemble handcrafted results, outperforming existing methods in expressive stroke control and semantic coherence. Codes are available at https://github.com/rane7/Stroke2Sketch.

Method: Created ScaleDF dataset with 5.8M real images from 51 domains and 8.8M fake images from 102 methods, then analyzed scaling relationships and power-law patterns.

Result: Found predictable power-law scaling where detection error decreases as number of real domains or deepfake methods increases, enabling performance forecasting.

Conclusion: Scaling laws apply to deepfake detection, suggesting data-centric approaches can counter evolving deepfake technology, though scaling has limitations.

Abstract: This paper presents a systematic study of scaling laws for the deepfake detection task. Specifically, we analyze the model performance against the number of real image domains, deepfake generation methods, and training images. Since no existing dataset meets the scale requirements for this research, we construct ScaleDF, the largest dataset to date in this field, which contains over 5.8 million real images from 51 different datasets (domains) and more than 8.8 million fake images generated by 102 deepfake methods. Using ScaleDF, we observe power-law scaling similar to that shown in large language models (LLMs). Specifically, the average detection error follows a predictable power-law decay as either the number of real domains or the number of deepfake methods increases. This key observation not only allows us to forecast the number of additional real domains or deepfake methods required to reach a target performance, but also inspires us to counter the evolving deepfake technology in a data-centric manner. Beyond this, we examine the role of pre-training and data augmentations in deepfake detection under scaling, as well as the limitations of scaling itself.

Method: Divides high-resolution latents into local windows for near-linear attention complexity, uses low-resolution latent with positional anchors for global semantics, and employs LoRA adaptation to bridge global-local pathways with Hilbert curve token ordering for efficiency.

Result: Achieves 2× faster inference with lower memory usage compared to dense attention, scales to 4K resolution without additional training data, and delivers superior global coherence and local detail on quantitative metrics (FID, IS, CLIP Score).

Conclusion: Hierarchical local attention with guided low-resolution anchors is an effective approach for advancing ultra-high-resolution image generation, matching or outperforming methods requiring native 4K training.

Abstract: Ultra-high-resolution text-to-image generation demands both fine-grained texture synthesis and globally coherent structure, yet current diffusion models remain constrained to sub-$1K \times 1K$ resolutions due to the prohibitive quadratic complexity of attention and the scarcity of native $4K$ training data. We present \textbf{Scale-DiT}, a new diffusion framework that introduces hierarchical local attention with low-resolution global guidance, enabling efficient, scalable, and semantically coherent image synthesis at ultra-high resolutions. Specifically, high-resolution latents are divided into fixed-size local windows to reduce attention complexity from quadratic to near-linear, while a low-resolution latent equipped with scaled positional anchors injects global semantics. A lightweight LoRA adaptation bridges global and local pathways during denoising, ensuring consistency across structure and detail. To maximize inference efficiency, we repermute token sequence in Hilbert curve order and implement a fused-kernel for skipping masked operations, resulting in a GPU-friendly design. Extensive experiments demonstrate that Scale-DiT achieves more than $2\times$ faster inference and lower memory usage compared to dense attention baselines, while reliably scaling to $4K \times 4K$ resolution without requiring additional high-resolution training data. On both quantitative benchmarks (FID, IS, CLIP Score) and qualitative comparisons, Scale-DiT delivers superior global coherence and sharper local detail, matching or outperforming state-of-the-art methods that rely on native 4K training. Taken together, these results highlight hierarchical local attention with guided low-resolution anchors as a promising and effective approach for advancing ultra-high-resolution image generation.

Method: Systematic analysis including: 1) Overfitting evaluation with reduced training data and cross-dataset validation, 2) Testing with metamer data using metameric black theory, 3) Exploring optical encoding via optical aberrations and deliberate optical design.

Result: RGB-to-spectral methods suffer from overfitting to existing datasets, fundamental inability to handle metameric conditions, and dataset insufficiencies. Optical encoding provides some improvement but remains limited by dataset issues.

Conclusion: Future progress in snapshot spectral imaging depends on generating improved datasets that can enable effective optical encoding strategies, as the RGB-to-spectral inverse problem remains fundamentally ill-posed.

Abstract: Hyperspectral imaging empowers machine vision systems with the distinct capability of identifying materials through recording their spectral signatures. Recent efforts in data-driven spectral reconstruction aim at extracting spectral information from RGB images captured by cost-effective RGB cameras, instead of dedicated hardware. Published work reports exceedingly high numerical scores for this reconstruction task, yet real-world performance lags substantially behind. We systematically analyze the performance of such methods. First, we evaluate the overfitting limitations with respect to current datasets by training the networks with less data, validating the trained models with unseen yet slightly modified data and cross-dataset validation. Second, we reveal fundamental limitations in the ability of RGB to spectral methods to deal with metameric or near-metameric conditions, which have so far gone largely unnoticed due to the insufficiencies of existing datasets. We validate the trained models with metamer data generated by metameric black theory and re-training the networks with various forms of metamers. This methodology can also be used for data augmentation as a partial mitigation of the dataset issues, although the RGB to spectral inverse problem remains fundamentally ill-posed. Finally, we analyze the potential for modifying the problem setting to achieve better performance by exploiting optical encoding provided by either optical aberrations or deliberate optical design. Our experiments show such approaches provide improved results under certain circumstances, but their overall performance is limited by the same dataset issues. We conclude that future progress on snapshot spectral imaging will heavily depend on the generation of improved datasets which can then be used to design effective optical encoding strategies. Code: https://github.com/vccimaging/OpticsAwareHSI-Analysis.

Method: A cloud-edge collaborative framework with lightweight on-device diffusion model for rapid previews and high-capacity cloud model for final refinements, plus a noise level predictor for dynamic computation load balancing.

Result: Reduces average generation time by 15.8% compared to Stable Diffusion v1.5 with comparable quality, and only 0.9% slower than Tiny-SD with significantly improved image quality.

Conclusion: DiffusionX demonstrates efficient and scalable prompt-based generation with minimal overhead through cloud-edge collaboration.

Abstract: Recent advances in diffusion models have driven remarkable progress in image generation. However, the generation process remains computationally intensive, and users often need to iteratively refine prompts to achieve the desired results, further increasing latency and placing a heavy burden on cloud resources. To address this challenge, we propose DiffusionX, a cloud-edge collaborative framework for efficient multi-round, prompt-based generation. In this system, a lightweight on-device diffusion model interacts with users by rapidly producing preview images, while a high-capacity cloud model performs final refinements after the prompt is finalized. We further introduce a noise level predictor that dynamically balances the computation load, optimizing the trade-off between latency and cloud workload. Experiments show that DiffusionX reduces average generation time by 15.8% compared with Stable Diffusion v1.5, while maintaining comparable image quality. Moreover, it is only 0.9% slower than Tiny-SD with significantly improved image quality, thereby demonstrating efficiency and scalability with minimal overhead.

Method: Proposed a new coding approach using bit plane representation to process images in SNNs, and investigated the effects of different color models on the coding process.

Result: Experimental validation showed effectiveness in achieving performance gains across multiple datasets. This is the first research to consider bit planes and color models in SNN context.

Conclusion: The bit plane coding strategy unlocks new potentials in SNN performance, potentially enabling more efficient and effective SNN models for future applications.

Abstract: Spiking neural network (SNN) has emerged as a promising paradigm in computational neuroscience and artificial intelligence, offering advantages such as low energy consumption and small memory footprint. However, their practical adoption is constrained by several challenges, prominently among them being performance optimization. In this study, we present a novel approach to enhance the performance of SNN for images through a new coding method that exploits bit plane representation. Our proposed technique is designed to improve the accuracy of SNN without increasing model size. Also, we investigate the impacts of color models of the proposed coding process. Through extensive experimental validation, we demonstrate the effectiveness of our coding strategy in achieving performance gain across multiple datasets. To the best of our knowledge, this is the first research that considers bit planes and color models in the context of SNN. By leveraging the unique characteristics of bit planes, we hope to unlock new potentials in SNNs performance, potentially paving the way for more efficient and effective SNNs models in future researches and applications.

Method: Three-part token-level enhancement: 1) Token Index Embedding clusters tokens for better reference representation, 2) Instruct Token Injection adds extra visual features as complementary priors, 3) Identity-Token Disentanglement (ITD) explicitly guides tokens to independently represent each identity’s features.

Result: The framework significantly improves identity consistency while preserving high-quality background reconstruction. Comprehensive experiments show it surpasses current state-of-the-art models in multiple reference image generation.

Conclusion: TokenAR effectively addresses identity confusion in multi-reference generation and introduces the InstructAR Dataset, the first large-scale open-source dataset for this task with 28K training pairs.

Abstract: Autoregressive Model (AR) has shown remarkable success in conditional image generation. However, these approaches for multiple reference generation struggle with decoupling different reference identities. In this work, we propose the TokenAR framework, specifically focused on a simple but effective token-level enhancement mechanism to address reference identity confusion problem. Such token-level enhancement consists of three parts, 1). Token Index Embedding clusters the tokens index for better representing the same reference images; 2). Instruct Token Injection plays as a role of extra visual feature container to inject detailed and complementary priors for reference tokens; 3). The identity-token disentanglement strategy (ITD) explicitly guides the token representations toward independently representing the features of each identity.This token-enhancement framework significantly augments the capabilities of existing AR based methods in conditional image generation, enabling good identity consistency while preserving high quality background reconstruction. Driven by the goal of high-quality and high-diversity in multi-subject generation, we introduce the InstructAR Dataset, the first open-source, large-scale, multi-reference input, open domain image generation dataset that includes 28K training pairs, each example has two reference subjects, a relative prompt and a background with mask annotation, curated for multiple reference image generation training and evaluating. Comprehensive experiments validate that our approach surpasses current state-of-the-art models in multiple reference image generation task. The implementation code and datasets will be made publicly. Codes are available, see https://github.com/lyrig/TokenAR

Method: Conducted diverse experiments including ImageNet classification, segmentation, and gradient visualization to analyze vision encoder representations after different training strategies (SFT vs RL). Proposed PIVOT method for building strong vision encoders.

Result: RL produces stronger and more precisely localized visual representations than SFT. PIVOT-trained vision encoders outperform larger counterparts with less than 1% computational cost of standard pretraining.

Conclusion: Training strategy fundamentally reshapes MLLM’s visual representations, with RL offering superior performance. PIVOT provides an efficient path for advancing MLLM vision backbones.

Abstract: A dominant assumption in Multimodal Language Model (MLLM) research is that its performance is largely inherited from the LLM backbone, given its immense parameter scale and remarkable capabilities. This has created a void in the understanding of the vision encoder, which determines how MLLMs perceive images. The recent shift in MLLM training paradigms, from Supervised Finetuning (SFT) to Reinforcement Learning (RL), magnifies this oversight-namely, the significant lack of analysis on how such training reshapes the vision encoder as well as the MLLM. To address this, we first investigate the impact of training strategies on MLLMs, where RL shows a clear advantage over SFT in strongly vision-related VQA benchmarks. Motivated by this, we conduct a critical yet under-explored analysis of the vision encoder of MLLMs through diverse and in-depth experiments, ranging from ImageNet classification and segmentation to gradient visualization. Our results demonstrate that MLLM’s post-training strategy (i.e., SFT or RL) not only leads to distinct outcomes on MLLM downstream tasks, but also fundamentally reshapes MLLM’s underlying visual representations. Specifically, the key finding of our study is that RL produces stronger and precisely localized visual representations compared to SFT, boosting the ability of the vision encoder for MLLM. We then reframe our findings into a simple recipe for building strong vision encoders for MLLMs, Preference-Instructed Vision OpTimization (PIVOT). When integrated into MLLMs, a PIVOT-trained vision encoder outperforms even larger and more heavily-trained counterparts, despite requiring less than 1% of the computational cost of standard vision pretraining. This result opens an effective and efficient path for advancing the vision backbones of MLLMs. Project page available at https://june-page.github.io/pivot/

Method: GradNorm measures noun positiveness using gradient magnitudes from cross-entropy between predicted target distribution and softmax output. Provides theoretical error bounds and subsumes existing methods as special cases.

Result: Extensive experiments show GradNorm achieves state-of-the-art clustering performance across various benchmarks.

Conclusion: GradNorm offers a theoretically grounded and empirically superior approach for noun filtering in Language-assisted Image Clustering, outperforming existing intuitive methods.

Abstract: This paper investigates the recently emerged problem of Language-assisted Image Clustering (LaIC), where textual semantics are leveraged to improve the discriminability of visual representations to facilitate image clustering. Due to the unavailability of true class names, one of core challenges of LaIC lies in how to filter positive nouns, i.e., those semantically close to the images of interest, from unlabeled wild corpus data. Existing filtering strategies are predominantly based on the off-the-shelf feature space learned by CLIP; however, despite being intuitive, these strategies lack a rigorous theoretical foundation. To fill this gap, we propose a novel gradient-based framework, termed as GradNorm, which is theoretically guaranteed and shows strong empirical performance. In particular, we measure the positiveness of each noun based on the magnitude of gradients back-propagated from the cross-entropy between the predicted target distribution and the softmax output. Theoretically, we provide a rigorous error bound to quantify the separability of positive nouns by GradNorm and prove that GradNorm naturally subsumes existing filtering strategies as extremely special cases of itself. Empirically, extensive experiments show that GradNorm achieves the state-of-the-art clustering performance on various benchmarks.

Method: Created the MIRAD dataset capturing three critical dimensions: diverse individualized products with large intra-class variation, data from six geographically dispersed manufacturing nodes, and substantial imaging heterogeneity. Evaluated state-of-the-art anomaly detection methods including one-class, multi-class, and zero-shot approaches.

Result: All evaluated models showed significant performance drops compared to conventional benchmarks, highlighting the unresolved complexities of defect detection in real-world individualized production.

Conclusion: MIRAD bridges industrial requirements and academic research, providing a realistic foundation for developing robust quality control solutions essential for Industry 5.0.

Abstract: Social manufacturing leverages community collaboration and scattered resources to realize mass individualization in modern industry. However, this paradigm shift also introduces substantial challenges in quality control, particularly in defect detection. The main difficulties stem from three aspects. First, products often have highly customized configurations. Second, production typically involves fragmented, small-batch orders. Third, imaging environments vary considerably across distributed sites. To overcome the scarcity of real-world datasets and tailored algorithms, we introduce the Mass Individualization Robust Anomaly Detection (MIRAD) dataset. As the first benchmark explicitly designed for anomaly detection in social manufacturing, MIRAD captures three critical dimensions of this domain: (1) diverse individualized products with large intra-class variation, (2) data collected from six geographically dispersed manufacturing nodes, and (3) substantial imaging heterogeneity, including variations in lighting, background, and motion conditions. We then conduct extensive evaluations of state-of-the-art (SOTA) anomaly detection methods on MIRAD, covering one-class, multi-class, and zero-shot approaches. Results show a significant performance drop across all models compared with conventional benchmarks, highlighting the unresolved complexities of defect detection in real-world individualized production. By bridging industrial requirements and academic research, MIRAD provides a realistic foundation for developing robust quality control solutions essential for Industry 5.0. The dataset is publicly available at https://github.com/wu33learn/MIRAD.

Method: Collected 3,000 phacoemulsification cataract surgery videos from two surgical centers with surgeons of varying experience levels, enriched with four annotation layers: temporal surgical phases, instance segmentation, instrument-tissue interaction tracking, and quantitative skill scores based on ICO-OSCAR rubrics.

Result: Benchmarking experiments demonstrated the dataset’s technical quality for key surgical AI tasks including workflow recognition, scene segmentation, and automated skill assessment. Established domain adaptation baseline for phase recognition across surgical centers.

Conclusion: The presented dataset addresses the gap in diverse, deeply annotated cataract surgery resources and supports development of generalizable surgical AI models through comprehensive annotations and benchmarking.

Abstract: The development of computer-assisted surgery systems depends on large-scale, annotated datasets. Current resources for cataract surgery often lack the diversity and annotation depth needed to train generalizable deep-learning models. To address this gap, we present a dataset of 3,000 phacoemulsification cataract surgery videos from two surgical centers, performed by surgeons with a range of experience levels. This resource is enriched with four annotation layers: temporal surgical phases, instance segmentation of instruments and anatomical structures, instrument-tissue interaction tracking, and quantitative skill scores based on the established competency rubrics like the ICO-OSCAR. The technical quality of the dataset is supported by a series of benchmarking experiments for key surgical AI tasks, including workflow recognition, scene segmentation, and automated skill assessment. Furthermore, we establish a domain adaptation baseline for the phase recognition task by training a model on a subset of surgical centers and evaluating its performance on a held-out center. The dataset and annotations are available in Google Form (https://docs.google.com/forms/d/e/1FAIpQLSfmyMAPSTGrIy2sTnz0-TMw08ZagTimRulbAQcWdaPwDy187A/viewform?usp=dialog).

Method: Used self-annotated dataset of 7,000 dashcam frames to fine-tune YOLO model, synchronized OCR timestamps with GPS logs for geolocation, and built backend database with road segment attribution and contractor information.

Result: Developed a fully automated end-to-end platform that detects potholes in real-time, geotags them, and provides dynamic road health visualization with intelligent governance features for automated alerts and accountability.

Conclusion: iWatchRoadv2 enables data-driven smart city management, transparent governance, and sustainable improvements in road infrastructure maintenance by automating the complete pothole monitoring lifecycle from detection to repair verification.

Abstract: Road potholes pose significant safety hazards and maintenance challenges, particularly on India’s diverse and under-maintained road networks. This paper presents iWatchRoadv2, a fully automated end-to-end platform for real-time pothole detection, GPS-based geotagging, and dynamic road health visualization using OpenStreetMap (OSM). We curated a self-annotated dataset of over 7,000 dashcam frames capturing diverse Indian road conditions, weather patterns, and lighting scenarios, which we used to fine-tune the Ultralytics YOLO model for accurate pothole detection. The system synchronizes OCR-extracted video timestamps with external GPS logs to precisely geolocate each detected pothole, enriching detections with comprehensive metadata, including road segment attribution and contractor information managed through an optimized backend database. iWatchRoadv2 introduces intelligent governance features that enable authorities to link road segments with contract metadata through a secure login interface. The system automatically sends alerts to contractors and officials when road health deteriorates, supporting automated accountability and warranty enforcement. The intuitive web interface delivers actionable analytics to stakeholders and the public, facilitating evidence-driven repair planning, budget allocation, and quality assessment. Our cost-effective and scalable solution streamlines frame processing and storage while supporting seamless public engagement for urban and rural deployments. By automating the complete pothole monitoring lifecycle, from detection to repair verification, iWatchRoadv2 enables data-driven smart city management, transparent governance, and sustainable improvements in road infrastructure maintenance. The platform and live demonstration are accessible at https://smlab.niser.ac.in/project/iwatchroad.

Method: Developed a data-driven parametric model that handles varying shape topology and models three sources of shape variation: articulation, subcomponent shape variation, and non-rigid deformation. Used a large-scale soybean farm dataset for training and testing.

Result: Demeter effectively synthesizes shapes, reconstructs structures, and simulates biophysical processes, demonstrating practical utility in agricultural applications.

Conclusion: Demeter advances plant modeling by providing an expressive parametric framework that captures the complex morphological variations in plants, with potential applications in 3D reconstruction, generation, understanding, and simulation for agricultural research.

Abstract: Learning 3D parametric shape models of objects has gained popularity in vision and graphics and has showed broad utility in 3D reconstruction, generation, understanding, and simulation. While powerful models exist for humans and animals, equally expressive approaches for modeling plants are lacking. In this work, we present Demeter, a data-driven parametric model that encodes key factors of a plant morphology, including topology, shape, articulation, and deformation into a compact learned representation. Unlike previous parametric models, Demeter handles varying shape topology across various species and models three sources of shape variation: articulation, subcomponent shape variation, and non-rigid deformation. To advance crop plant modeling, we collected a large-scale, ground-truthed dataset from a soybean farm as a testbed. Experiments show that Demeter effectively synthesizes shapes, reconstructs structures, and simulates biophysical processes. Code and data is available at https://tianhang-cheng.github.io/Demeter/.

Method: Uses encoder-decoder architecture with sparse convolution on ResNet-18 backbone to exploit sparsity in hand pose images, introduces SPLite decoder for faster decoding, and applies quantization-aware training for memory optimization.

Result: Achieved 42% end-to-end efficiency improvement, 3.1x frame rate boost on Raspberry Pi 5, 2.98x overall speed-up on Raspberry Pi 5 CPU, with minimal accuracy loss (PA-MPJPE increased only from 9.0mm to 9.1mm on FreiHAND).

Conclusion: The proposed framework demonstrates comparable accuracy to state-of-the-art methods while significantly enhancing computational efficiency, making it suitable for real-time hand pose estimation on resource-constrained edge devices.

Abstract: With the increasing ubiquity of AR/VR devices, the deployment of deep learning models on edge devices has become a critical challenge. These devices require real-time inference, low power consumption, and minimal latency. Many framework designers face the conundrum of balancing efficiency and performance. We design a light framework that adopts an encoder-decoder architecture and introduces several key contributions aimed at improving both efficiency and accuracy. We apply sparse convolution on a ResNet-18 backbone to exploit the inherent sparsity in hand pose images, achieving a 42% end-to-end efficiency improvement. Moreover, we propose our SPLite decoder. This new architecture significantly boosts the decoding process’s frame rate by 3.1x on the Raspberry Pi 5, while maintaining accuracy on par. To further optimize performance, we apply quantization-aware training, reducing memory usage while preserving accuracy (PA-MPJPE increases only marginally from 9.0 mm to 9.1 mm on FreiHAND). Overall, our system achieves a 2.98x speed-up on a Raspberry Pi 5 CPU (BCM2712 quad-core Arm A76 processor). Our method is also evaluated on compound benchmark datasets, demonstrating comparable accuracy to state-of-the-art approaches while significantly enhancing computational efficiency.

Method: Segmentation on 3D Gaussian Splatting representations with Global-to-Local Spatial Grounding: multiple global views for coarse localization, then close-up novel views for fine-grained local segmentation, aggregating MLLM responses for robust target identification.

Result: REALM achieves remarkable performance in interpreting both explicit and implicit instructions across LERF, 3D-OVS, and REALM3D benchmarks, and supports various 3D interaction tasks including object removal, replacement, and style transfer.

Conclusion: REALM demonstrates practical utility and versatility as an agent framework that enables open-world reasoning-based segmentation without extensive 3D-specific post-training, effectively bridging 2D reasoning capabilities with 3D spatial understanding.

Abstract: Bridging the gap between complex human instructions and precise 3D object grounding remains a significant challenge in vision and robotics. Existing 3D segmentation methods often struggle to interpret ambiguous, reasoning-based instructions, while 2D vision-language models that excel at such reasoning lack intrinsic 3D spatial understanding. In this paper, we introduce REALM, an innovative MLLM-agent framework that enables open-world reasoning-based segmentation without requiring extensive 3D-specific post-training. We perform segmentation directly on 3D Gaussian Splatting representations, capitalizing on their ability to render photorealistic novel views that are highly suitable for MLLM comprehension. As directly feeding one or more rendered views to the MLLM can lead to high sensitivity to viewpoint selection, we propose a novel Global-to-Local Spatial Grounding strategy. Specifically, multiple global views are first fed into the MLLM agent in parallel for coarse-level localization, aggregating responses to robustly identify the target object. Then, several close-up novel views of the object are synthesized to perform fine-grained local segmentation, yielding accurate and consistent 3D masks. Extensive experiments show that REALM achieves remarkable performance in interpreting both explicit and implicit instructions across LERF, 3D-OVS, and our newly introduced REALM3D benchmarks. Furthermore, our agent framework seamlessly supports a range of 3D interaction tasks, including object removal, replacement, and style transfer, demonstrating its practical utility and versatility. Project page: https://ChangyueShi.github.io/REALM.

Method: Reformulates self-supervised learning objectives (like predicting image rotation or reconstructing masked patches) into dense, automatic reward signals for RL-based fine-tuning.

Result: Substantially improves performance on both vision-centric and vision-language reasoning benchmarks. Also demonstrates generality by achieving significant gains in graph learning.

Conclusion: SSL4RL establishes a versatile and effective paradigm for aligning multimodal models using verifiable, self-supervised objectives, with identified key factors influencing effectiveness including task difficulty, model scale, and semantic alignment.

Abstract: Vision-language models (VLMs) have shown remarkable abilities by integrating large language models with visual inputs. However, they often fail to utilize visual evidence adequately, either depending on linguistic priors in vision-centric tasks or resorting to textual shortcuts during reasoning. Although reinforcement learning (RL) can align models with desired behaviors, its application to VLMs has been hindered by the lack of scalable and reliable reward mechanisms. To overcome this challenge, we propose SSL4RL, a novel framework that leverages self-supervised learning (SSL) tasks as a source of verifiable rewards for RL-based fine-tuning. Our approach reformulates SSL objectives-such as predicting image rotation or reconstructing masked patches-into dense, automatic reward signals, eliminating the need for human preference data or unreliable AI evaluators. Experiments show that SSL4RL substantially improves performance on both vision-centric and vision-language reasoning benchmarks. Furthermore, through systematic ablations, we identify key factors-such as task difficulty, model scale, and semantic alignment with the target domain-that influence the effectiveness of SSL4RL tasks, offering new design principles for future work. We also demonstrate the framework’s generality by applying it to graph learning, where it yields significant gains. SSL4RL establishes a versatile and effective paradigm for aligning multimodal models using verifiable, self-supervised objectives.

Method: Proposes LightGlueStick with Attentional Line Message Passing (ALMP) that explicitly exposes line connectivity to the network, allowing efficient communication between nodes in a lightweight architecture.

Result: LightGlueStick establishes a new state-of-the-art across different benchmarks while maintaining computational efficiency suitable for real-time applications.

Conclusion: The proposed lightweight matcher successfully combines point and line matching with efficient architecture, making it suitable for real-time applications and edge device deployment.

Abstract: Lines and points are complementary local features, whose combination has proven effective for applications such as SLAM and Structure-from-Motion. The backbone of these pipelines are the local feature matchers, establishing correspondences across images. Traditionally, point and line matching have been treated as independent tasks. Recently, GlueStick proposed a GNN-based network that simultaneously operates on points and lines to establish matches. While running a single joint matching reduced the overall computational complexity, the heavy architecture prevented real-time applications or deployment to edge devices. Inspired by recent progress in point matching, we propose LightGlueStick, a lightweight matcher for points and line segments. The key novel component in our architecture is the Attentional Line Message Passing (ALMP), which explicitly exposes the connectivity of the lines to the network, allowing for efficient communication between nodes. In thorough experiments we show that LightGlueStick establishes a new state-of-the-art across different benchmarks. The code is available at https://github.com/aubingazhib/LightGlueStick.

Method: Uses Spatio-Temporal Subtle Information Tokenization (ST-SIT) to extract cross-frame deepfake features, and Fine-grained Multimodal Chain-of-Thought (Fg-MCoT) with facial feature constraints for pixel-level spatio-temporal localization.

Result: EDVD-LLaMA achieves outstanding performance and robustness in detection accuracy, explainability, and cross-forgery/cross-dataset scenarios, outperforming previous methods.

Conclusion: The framework provides a more explainable and superior solution for deepfake video detection compared to traditional methods, with publicly available source code and dataset.

Abstract: The rapid development of deepfake video technology has not only facilitated artistic creation but also made it easier to spread misinformation. Traditional deepfake video detection (DVD) methods face issues such as a lack of transparency in their principles and insufficient generalization capabilities to cope with evolving forgery techniques. This highlights an urgent need for detectors that can identify forged content and provide verifiable reasoning explanations. This paper proposes the explainable deepfake video detection (EDVD) task and designs the EDVD-LLaMA multimodal, a large language model (MLLM) reasoning framework, which provides traceable reasoning processes alongside accurate detection results and trustworthy explanations. Our approach first incorporates a Spatio-Temporal Subtle Information Tokenization (ST-SIT) to extract and fuse global and local cross-frame deepfake features, providing rich spatio-temporal semantic information input for MLLM reasoning. Second, we construct a Fine-grained Multimodal Chain-of-Thought (Fg-MCoT) mechanism, which introduces facial feature data as hard constraints during the reasoning process to achieve pixel-level spatio-temporal video localization, suppress hallucinated outputs, and enhance the reliability of the chain of thought. In addition, we build an Explainable Reasoning FF++ benchmark dataset (ER-FF++set), leveraging structured data to annotate videos and ensure quality control, thereby supporting dual supervision for reasoning and detection. Extensive experiments demonstrate that EDVD-LLaMA achieves outstanding performance and robustness in terms of detection accuracy, explainability, and its ability to handle cross-forgery methods and cross-dataset scenarios. Compared to previous DVD methods, it provides a more explainable and superior solution. The source code and dataset will be publicly available.

Method: Proposed rotation-invariant loss function leveraging Gaussian bounding box representation and Bhattacharyya distance, with anisotropic Gaussian representation to address isotropic variance issues in square-like objects.

Result: Significant improvements in mean Average Precision metrics compared to existing methods when integrated into state-of-the-art deep learning detectors.

Conclusion: The approach shows potential to establish new benchmarks in rotated object detection with wide applications requiring precise object localization regardless of orientation.

Abstract: Detecting rotated objects accurately and efficiently is a significant challenge in computer vision, particularly in applications such as aerial imagery, remote sensing, and autonomous driving. Although traditional object detection frameworks are effective for axis-aligned objects, they often underperform in scenarios involving rotated objects due to their limitations in capturing orientation variations. This paper introduces an improved loss function aimed at enhancing detection accuracy and robustness by leveraging the Gaussian bounding box representation and Bhattacharyya distance. In addition, we advocate for the use of an anisotropic Gaussian representation to address the issues associated with isotropic variance in square-like objects. Our proposed method addresses these challenges by incorporating a rotation-invariant loss function that effectively captures the geometric properties of rotated objects. We integrate this proposed loss function into state-of-the-art deep learning-based rotated object detection detectors, and extensive experiments demonstrated significant improvements in mean Average Precision metrics compared to existing methods. The results highlight the potential of our approach to establish new benchmark in rotated object detection, with implications for a wide range of applications requiring precise and reliable object localization irrespective of orientation.

Method: VIPAMIN enhances adaptation by (1) aligning prompts with semantically informative regions in embedding space, and (2) injecting novel representational directions beyond the pretrained subspace, requiring only a single forward pass and lightweight operations.

Result: VIPAMIN consistently improves performance across diverse tasks and dataset sizes, setting a new state of the art in visual prompt tuning.

Conclusion: VIPAMIN provides an effective and efficient visual prompt initialization strategy that significantly enhances adaptation of self-supervised models while maintaining computational efficiency.

Abstract: In the era of large-scale foundation models, fully fine-tuning pretrained networks for each downstream task is often prohibitively resource-intensive. Prompt tuning offers a lightweight alternative by introducing tunable prompts while keeping the backbone frozen. However, existing visual prompt tuning methods often fail to specialize the prompts or enrich the representation space–especially when applied to self-supervised backbones. We show that these limitations become especially pronounced in challenging tasks and data-scarce settings, where effective adaptation is most critical. In this work, we introduce VIPAMIN, a visual prompt initialization strategy that enhances adaptation of self-supervised models by (1) aligning prompts with semantically informative regions in the embedding space, and (2) injecting novel representational directions beyond the pretrained subspace. Despite its simplicity–requiring only a single forward pass and lightweight operations–VIPAMIN consistently improves performance across diverse tasks and dataset sizes, setting a new state of the art in visual prompt tuning. Our code is available at https://github.com/iamjaekyun/vipamin.

Method: Multitask learning framework combining segmentation and center detection with cross-teaching mechanism, class-focused cross-domain contrastive learning, and segmentation self-training with instance-aware pseudo-label selection strategy.

Result: Outperforms existing UDA and WDA methods, significantly narrowing the performance gap with supervised upper bound, and achieves substantial improvements over other UDA techniques.

Conclusion: The proposed weakly supervised domain adaptation approach effectively leverages sparse point annotations to achieve high-performance mitochondria segmentation with minimal annotation costs.

Abstract: Annotation-efficient segmentation of the numerous mitochondria instances from various electron microscopy (EM) images is highly valuable for biological and neuroscience research. Although unsupervised domain adaptation (UDA) methods can help mitigate domain shifts and reduce the high costs of annotating each domain, they typically have relatively low performance in practical applications. Thus, we investigate weakly supervised domain adaptation (WDA) that utilizes additional sparse point labels on the target domain, which require minimal annotation effort and minimal expert knowledge. To take full use of the incomplete and imprecise point annotations, we introduce a multitask learning framework that jointly conducts segmentation and center detection with a novel cross-teaching mechanism and class-focused cross-domain contrastive learning. While leveraging unlabeled image regions is essential, we introduce segmentation self-training with a novel instance-aware pseudo-label (IPL) selection strategy. Unlike existing methods that typically rely on pixel-wise pseudo-label filtering, the IPL semantically selects reliable and diverse pseudo-labels with the help of the detection task. Comprehensive validations and comparisons on challenging datasets demonstrate that our method outperforms existing UDA and WDA methods, significantly narrowing the performance gap with the supervised upper bound. Furthermore, under the UDA setting, our method also achieves substantial improvements over other UDA techniques.

Method: Trains Q-model with unlabeled trajectory data to generate Q-features for candidate actions, integrates these with navigation instructions via cross-modal encoder, and uses A*-style search combining future and historical scores.

Result: Extensive experiments on goal-oriented VLN datasets validate the method’s effectiveness in improving navigation performance.

Conclusion: The proposed foresighted approach with Q-learning and future-aware decision-making significantly enhances VLN agent performance by considering long-term outcomes.

Abstract: In this work we concentrate on the task of goal-oriented Vision-and-Language Navigation (VLN). Existing methods often make decisions based on historical information, overlooking the future implications and long-term outcomes of the actions. In contrast, we aim to develop a foresighted agent. Specifically, we draw upon Q-learning to train a Q-model using large-scale unlabeled trajectory data, in order to learn the general knowledge regarding the layout and object relations within indoor scenes. This model can generate a Q-feature, analogous to the Q-value in traditional Q-network, for each candidate action, which describes the potential future information that may be observed after taking the specific action. Subsequently, a cross-modal future encoder integrates the task-agnostic Q-feature with navigation instructions to produce a set of action scores reflecting future prospects. These scores, when combined with the original scores based on history, facilitate an A*-style searching strategy to effectively explore the regions that are more likely to lead to the destination. Extensive experiments conducted on widely used goal-oriented VLN datasets validate the effectiveness of the proposed method.

Method: Disentangles Gaussian representation into structural layer (StyleUNet-based generator mapping poses to Gaussians) and motion layer (SMPL-X model for compact pose variations), with facial attention mechanism and layer-wise compression.

Result: Provides streamable solution for rapid 3D avatar rendering, significantly outperforming prior methods in both visual quality and compression efficiency.

Conclusion: HGC-Avatar enables efficient transmission and high-quality rendering of dynamic avatars with improved compression efficiency and facial realism.

Abstract: Recent advances in 3D Gaussian Splatting (3DGS) have enabled fast, photorealistic rendering of dynamic 3D scenes, showing strong potential in immersive communication. However, in digital human encoding and transmission, the compression methods based on general 3DGS representations are limited by the lack of human priors, resulting in suboptimal bitrate efficiency and reconstruction quality at the decoder side, which hinders their application in streamable 3D avatar systems. We propose HGC-Avatar, a novel Hierarchical Gaussian Compression framework designed for efficient transmission and high-quality rendering of dynamic avatars. Our method disentangles the Gaussian representation into a structural layer, which maps poses to Gaussians via a StyleUNet-based generator, and a motion layer, which leverages the SMPL-X model to represent temporal pose variations compactly and semantically. This hierarchical design supports layer-wise compression, progressive decoding, and controllable rendering from diverse pose inputs such as video sequences or text. Since people are most concerned with facial realism, we incorporate a facial attention mechanism during StyleUNet training to preserve identity and expression details under low-bitrate constraints. Experimental results demonstrate that HGC-Avatar provides a streamable solution for rapid 3D avatar rendering, while significantly outperforming prior methods in both visual quality and compression efficiency.

Method: Created PRISMM-Bench through multi-stage pipeline: review mining, LLM-assisted filtering, and human verification to curate 262 inconsistencies from 242 papers. Designed three tasks (inconsistency identification, remedy, pair matching) and introduced structured JSON-based answer representations to minimize linguistic biases.

Result: Benchmarked 21 leading LMMs including large open-weight and proprietary models. Results show strikingly low performance (26.1-54.2%), highlighting the challenge of multimodal scientific reasoning.

Conclusion: Current LMMs struggle significantly with detecting and resolving real-world multimodal inconsistencies in scientific papers, motivating the need for progress towards more trustworthy scientific assistants.

Abstract: Large Multimodal Models (LMMs) are increasingly applied to scientific research, yet it remains unclear whether they can reliably understand and reason over the multimodal complexity of papers. A central challenge lies in detecting and resolving inconsistencies across text, figures, tables, and equations, issues that are often subtle, domain-specific, and ultimately undermine clarity, reproducibility, and trust. Existing benchmarks overlook this issue, either isolating single modalities or relying on synthetic errors that fail to capture real-world complexity. We introduce PRISMM-Bench (Peer-Review-sourced Inconsistency Set for Multimodal Models), the first benchmark grounded in real reviewer-flagged inconsistencies in scientific papers. Through a multi-stage pipeline of review mining, LLM-assisted filtering and human verification, we curate 262 inconsistencies from 242 papers. Based on this set, we design three tasks, namely inconsistency identification, remedy and pair matching, which assess a model’s capacity to detect, correct, and reason over inconsistencies across different modalities. Furthermore, to address the notorious problem of choice-only shortcuts in multiple-choice evaluation, where models exploit answer patterns without truly understanding the question, we further introduce structured JSON-based answer representations that minimize linguistic biases by reducing reliance on superficial stylistic cues. We benchmark 21 leading LMMs, including large open-weight models (GLM-4.5V 106B, InternVL3 78B) and proprietary models (Gemini 2.5 Pro, GPT-5 with high reasoning). Results reveal strikingly low performance (26.1-54.2%), underscoring the challenge of multimodal scientific reasoning and motivating progress towards trustworthy scientific assistants.

Method: Extends YOLOv8 with additional convolutional branches for OOS detection, product segmentation, and depth estimation. Uses pseudo-labeled depth data from Depth Anything V2 with a novel depth normalization procedure.

Result: Achieved 1.8% higher mAP than state-of-the-art OOS detection methods. Ablation studies showed auxiliary learning increased mAP by 3.7% and depth normalization by 4.2%.

Conclusion: The proposed OOS-DSD method effectively improves OOS detection through auxiliary learning and depth normalization, demonstrating significant performance gains over existing approaches.

Abstract: Out-of-stock (OOS) detection is a very important retail verification process that aims to infer the unavailability of products in their designated areas on the shelf. In this paper, we introduce OOS-DSD, a novel deep learning-based method that advances OOS detection through auxiliary learning. In particular, we extend a well-established YOLOv8 object detection architecture with additional convolutional branches to simultaneously detect OOS, segment products, and estimate scene depth. While OOS detection and product segmentation branches are trained using ground truth data, the depth estimation branch is trained using pseudo-labeled annotations produced by the state-of-the-art (SOTA) depth estimation model Depth Anything V2. Furthermore, since the aforementioned pseudo-labeled depth estimates display relative depth, we propose an appropriate depth normalization procedure that stabilizes the training process. The experimental results show that the proposed method surpassed the performance of the SOTA OOS detection methods by 1.8% of the mean average precision (mAP). In addition, ablation studies confirm the effectiveness of auxiliary learning and the proposed depth normalization procedure, with the former increasing mAP by 3.7% and the latter by 4.2%.

Method: Utilizes a graph structure where nodes represent images/representatives and edges capture similarity relationships. Employs GAT-based autoencoder to highlight important features and construct context-aware latent representations, then obtains category representatives from embeddings for categorization and retrieval.

Result: The method demonstrates effectiveness through experiments comparing GAT autoencoders with standard feature-based techniques, showing improved performance in representative-centric image categorization and retrieval.

Conclusion: The proposed representative-centric approach using GAT-based autoencoders provides an effective framework for image categorization and retrieval by leveraging graph-based relationships and context-aware representations.

Abstract: We propose a method for image categorization and retrieval that leverages graphs and a graph attention network (GAT)-based autoencoder. Our approach is representative-centric, that is, we execute the categorization and retrieval process via the representative models we construct for the images and image categories. We utilize a graph where nodes represent images (or their representatives) and edges capture similarity relationships. GAT highlights important features and relationships between images, enabling the autoencoder to construct context-aware latent representations that capture the key features of each image relative to its neighbors. We obtain category representatives from these embeddings and categorize a query image by comparing its representative to the category representatives. We then retrieve the most similar image to the query image within its identified category. We demonstrate the effectiveness of our representative-centric approach through experiments with both the GAT autoencoders and standard feature-based techniques.

Method: READ adds two auxiliary objectives to contrastive learning: (1) token-level reconstruction using a frozen decoder to reconstruct alternative captions, and (2) sentence-level alignment to explicitly align paraphrased sentences in embedding space.

Result: READ-CLIP achieves state-of-the-art performance across five major compositional reasoning benchmarks, outperforming the strongest baseline by up to 4.1%. The method also improves existing CLIP variants like NegCLIP and FSC-CLIP.

Conclusion: The reconstruction and alignment objectives provide complementary benefits - reconstruction captures word relationships within captions, while alignment ensures consistent representations for paraphrases with different wording.

Abstract: Despite recent advances, vision-language models trained with standard contrastive objectives still struggle with compositional reasoning – the ability to understand structured relationships between visual and linguistic elements. This shortcoming is largely due to the tendency of the text encoder to focus on individual words rather than their relations, a limitation reinforced by contrastive training that primarily aligns words with visual objects. In this paper, we introduce REconstruction and Alignment of text Descriptions (READ), a fine-tuning method designed to enhance compositional reasoning by adding two auxiliary objectives to the contrastive learning: (1) a token-level reconstruction objective, where a frozen pre-trained decoder reconstructs alternative captions based on the embedding of the original caption; and (2) a sentence-level alignment objective, which explicitly aligns paraphrased sentences in the embedding space. We show that READ-CLIP, a model derived by applying the READ method to the pre-trained CLIP model, achieves the state-of-the-art performance across five major compositional reasoning benchmarks, outperforming the strongest conventional fine-tuning baseline by up to 4.1%. Furthermore, applying the READ to existing CLIP variants (including NegCLIP and FSC-CLIP) also improves performance on these benchmarks. Quantitative and qualitative analyses reveal that our proposed objectives – reconstruction and alignment – offer complementary benefits: the former encourages the encoder to capture relationships between words within a caption, while the latter ensures consistent representations for paraphrases expressed with different wording.

Method: Proposes Region-aware Dynamic Aggregation and Excitation framework (GaitRDAE) with two core modules: RDA for dynamically searching optimal temporal receptive fields per region, and RDE for emphasizing motion regions with stable behavior patterns while suppressing static regions affected by covariates.

Result: GaitRDAE achieves state-of-the-art performance on several benchmark datasets.

Conclusion: The framework successfully addresses limitations of fixed temporal modeling by dynamically adapting to motion regions and their specific temporal patterns, leading to improved gait recognition accuracy.

Abstract: Deep learning-based gait recognition has achieved great success in various applications. The key to accurate gait recognition lies in considering the unique and diverse behavior patterns in different motion regions, especially when covariates affect visual appearance. However, existing methods typically use predefined regions for temporal modeling, with fixed or equivalent temporal scales assigned to different types of regions, which makes it difficult to model motion regions that change dynamically over time and adapt to their specific patterns. To tackle this problem, we introduce a Region-aware Dynamic Aggregation and Excitation framework (GaitRDAE) that automatically searches for motion regions, assigns adaptive temporal scales and applies corresponding attention. Specifically, the framework includes two core modules: the Region-aware Dynamic Aggregation (RDA) module, which dynamically searches the optimal temporal receptive field for each region, and the Region-aware Dynamic Excitation (RDE) module, which emphasizes the learning of motion regions containing more stable behavior patterns while suppressing attention to static regions that are more susceptible to covariates. Experimental results show that GaitRDAE achieves state-of-the-art performance on several benchmark datasets.

Method: Created a benchmark based on the Political Deepfakes Incident Database - a curated collection of real-world political deepfakes shared on social media since 2018. Systematically evaluated state-of-the-art deepfake detectors from academia, government, and industry.

Result: Academic and government detectors performed poorly. Paid tools achieved higher performance than free-access models, but all detectors struggled to generalize to authentic political deepfakes and were vulnerable to simple manipulations, especially in video.

Conclusion: There is an urgent need for politically contextualized deepfake detection frameworks to better protect the public from real-world political deepfake threats.

Abstract: The rapid proliferation of AI-generated content, driven by advances in generative adversarial networks, diffusion models, and multimodal large language models, has made the creation and dissemination of synthetic media effortless, heightening the risks of misinformation, particularly political deepfakes that distort truth and undermine trust in political institutions. In turn, governments, research institutions, and industry have strongly promoted deepfake detection initiatives as solutions. Yet, most existing models are trained and validated on synthetic, laboratory-controlled datasets, limiting their generalizability to the kinds of real-world political deepfakes circulating on social platforms that affect the public. In this work, we introduce the first systematic benchmark based on the Political Deepfakes Incident Database, a curated collection of real-world political deepfakes shared on social media since 2018. Our study includes a systematic evaluation of state-of-the-art deepfake detectors across academia, government, and industry. We find that the detectors from academia and government perform relatively poorly. While paid detection tools achieve relatively higher performance than free-access models, all evaluated detectors struggle to generalize effectively to authentic political deepfakes, and are vulnerable to simple manipulations, especially in the video domain. Results urge the need for politically contextualized deepfake detection frameworks to better safeguard the public in real-world settings.

Method: Proposes SHIELD framework with three training-free strategies: 1) re-weighting visual tokens to reduce statistical bias, 2) introducing noise-derived tokens to counter inherent bias, and 3) applying adversarial attacks with contrastive decoding to address vulnerability.

Result: SHIELD effectively mitigates object hallucinations across diverse benchmarks and LVLM families. It also achieves strong performance on general LVLM benchmarks, demonstrating broad applicability.

Conclusion: SHIELD successfully addresses object hallucination in LVLMs by targeting visual encoder issues rather than LLM components, providing an effective training-free solution with wide applicability across different model families.

Abstract: Large Vision-Language Models (LVLMs) excel in diverse cross-modal tasks. However, object hallucination, where models produce plausible but inaccurate object descriptions, remains a significant challenge. In contrast to previous work focusing on LLM components, this paper is the first to trace LVLM hallucinations to visual encoders and identifies three key issues: statistical bias, inherent bias, and vulnerability. To address these challenges, we propose SHIELD, a training-free framework that mitigates hallucinations through three strategies: re-weighting visual tokens to reduce statistical bias, introducing noise-derived tokens to counter inherent bias, and applying adversarial attacks with contrastive decoding to address vulnerability. Experiments demonstrate that SHIELD effectively mitigates object hallucinations across diverse benchmarks and LVLM families. Moreover, SHIELD achieves strong performance on the general LVLM benchmark, highlighting its broad applicability. Code will be released.

Method: Reformulates token compression as end-to-end learnable decision process with VisionSelector - a scorer module decoupled from MLLM backbone, incorporating differentiable Top-K mechanism and curriculum annealing strategy to bridge training-inference gap.

Result: Achieves 100% accuracy on MME with 30% retention budget, outperforms prior methods by 12.14% at 10% retention budget, doubles prefill speed, with only 12.85M trainable parameters. Demonstrates generalization across various compression rates.

Conclusion: VisionSelector provides efficient and adaptive token selection for MLLMs, preserving critical information while significantly reducing computational overhead across various compression budgets.

Abstract: Multimodal Large Language Models (MLLMs) encounter significant computational and memory bottlenecks from the massive number of visual tokens generated by high-resolution images or multi-image inputs. Previous token compression techniques are often constrained by heuristic rules that risk discarding critical information. They may suffer from biases, such as attention sinks, that lead to sharp performance drops under aggressive compression ratios. To address these limitations, we reformulate token compression as a lightweight plug-and-play framework that reformulates token compression into an end-to-end learnable decision process. To be specific, we propose VisionSelector, a scorer module decoupled from the MLLM backbone that incorporates a differentiable Top-K mechanism and a curriculum annealing strategy to bridge the training-inference gap, enabling efficient and adaptive token selection various arbitrary compression rates. Remarkably lightweight with only 12.85M trainable parameters, VisionSelector demonstrates generalization across various compression rates and adaptively identifying critical tokens. This leads to superior performance across all compression budgets, evidenced by preserving 100% accuracy on MME with 30% retention budget, outperforming prior methods by 12.14% at 10% retention budget, and doubling prefill speed. Our code is available at https://github.com/JulietChoo/VisionSelector .

Method: Integrates U-Net, EfficientNet, and Transformer-based neural networks with real-time optimization strategies (model pruning, quantization, GPU acceleration) and enables flexible deployment across edge devices, servers, and cloud infrastructure.

Result: Achieved state-of-the-art performance: >92% classification accuracy, >91% segmentation Dice scores, and <80ms inference times on public benchmark datasets, with enhanced transparency through Grad-CAM and segmentation overlays.

Conclusion: The framework can substantially accelerate diagnostic workflows, reduce clinician workload, and support trustworthy AI integration in time-critical healthcare environments.

Abstract: Medical imaging plays a vital role in modern diagnostics; however, interpreting high-resolution radiological data remains time-consuming and susceptible to variability among clinicians. Traditional image processing techniques often lack the precision, robustness, and speed required for real-time clinical use. To overcome these limitations, this paper introduces a deep learning framework for real-time medical image analysis designed to enhance diagnostic accuracy and computational efficiency across multiple imaging modalities, including X-ray, CT, and MRI. The proposed system integrates advanced neural network architectures such as U-Net, EfficientNet, and Transformer-based models with real-time optimization strategies including model pruning, quantization, and GPU acceleration. The framework enables flexible deployment on edge devices, local servers, and cloud infrastructures, ensuring seamless interoperability with clinical systems such as PACS and EHR. Experimental evaluations on public benchmark datasets demonstrate state-of-the-art performance, achieving classification accuracies above 92%, segmentation Dice scores exceeding 91%, and inference times below 80 milliseconds. Furthermore, visual explanation tools such as Grad-CAM and segmentation overlays enhance transparency and clinical interpretability. These results indicate that the proposed framework can substantially accelerate diagnostic workflows, reduce clinician workload, and support trustworthy AI integration in time-critical healthcare environments.

Method: Combines semantic segmentation with monocular depth estimation using adaptive scale factor algorithm for metric distance conversion. Uses knowledge distillation with SVM teacher to train lightweight U-Net student network for real-time segmentation.

Result: Achieved 14.4 cm mean distance error, 100% success rate in 30 real-world and 100 digital-twin flight tests. End-to-end learning achieved 87.5% autonomous mission success rate.

Conclusion: The system advances practical vision-based drone navigation in structured environments, solving metric depth estimation and computational efficiency challenges for deployment on resource-constrained platforms.

Abstract: This paper presents a vision-only autonomous flight system for small UAVs operating in controlled indoor environments. The system combines semantic segmentation with monocular depth estimation to enable obstacle avoidance, scene exploration, and autonomous safe landing operations without requiring GPS or expensive sensors such as LiDAR. A key innovation is an adaptive scale factor algorithm that converts non-metric monocular depth predictions into accurate metric distance measurements by leveraging semantic ground plane detection and camera intrinsic parameters, achieving a mean distance error of 14.4 cm. The approach uses a knowledge distillation framework where a color-based Support Vector Machine (SVM) teacher generates training data for a lightweight U-Net student network (1.6M parameters) capable of real-time semantic segmentation. For more complex environments, the SVM teacher can be replaced with a state-of-the-art segmentation model. Testing was conducted in a controlled 5x4 meter laboratory environment with eight cardboard obstacles simulating urban structures. Extensive validation across 30 flight tests in a real-world environment and 100 flight tests in a digital-twin environment demonstrates that the combined segmentation and depth approach increases the distance traveled during surveillance and reduces mission time while maintaining 100% success rates. The system is further optimized through end-to-end learning, where a compact student neural network learns complete flight policies from demonstration data generated by our best-performing method, achieving an 87.5% autonomous mission success rate. This work advances practical vision-based drone navigation in structured environments, demonstrating solutions for metric depth estimation and computational efficiency challenges that enable deployment on resource-constrained platforms.

Method: Created MultiVerse benchmark with 647 dialogues (4 turns average) from 12 diverse VLM benchmarks, covering 484 tasks across factual knowledge, perception, reasoning, math, and coding. Uses checklist-based evaluation with GPT-4o measuring 37 key aspects.

Result: Even strongest models like GPT-4o achieve only 50% success rate in complex multi-turn conversations. Providing full dialogue context significantly improves performance for smaller/weaker models, highlighting importance of in-context learning.

Conclusion: MultiVerse serves as a comprehensive landscape for evaluating multi-turn interaction abilities in VLMs, revealing significant challenges in complex conversational scenarios that current models struggle with.

Abstract: Vision-and-Language Models (VLMs) have shown impressive capabilities on single-turn benchmarks, yet real-world applications often demand more intricate multi-turn dialogues. Existing multi-turn datasets (e.g, MMDU, ConvBench) only partially capture the breadth and depth of conversational scenarios encountered by users. In this work, we introduce MultiVerse, a novel multi-turn conversation benchmark featuring 647 dialogues - each averaging four turns - derived from a diverse set of 12 popular VLM evaluation benchmarks. With 484 tasks and 484 interaction goals, MultiVerse covers a wide range of topics, from factual knowledge and perception to advanced reasoning tasks such as mathematics and coding. To facilitate robust assessment, we propose a checklist-based evaluation method that leverages GPT-4o as the automated evaluator, measuring performance across 37 key aspects, including perceptual accuracy, linguistic clarity, and factual correctness. We evaluate 18 VLMs on MultiVerse, revealing that even the strongest models (e.g., GPT-4o) achieve only a 50% success rate in complex multi-turn conversations, highlighting the dataset’s challenging nature. Notably, we find that providing full dialogue context significantly enhances performance for smaller or weaker models, emphasizing the importance of in-context learning. We believe MultiVerse is a landscape of evaluating multi-turn interaction abilities for VLMs.

Method: Proposed using Retrieval Augmented Generation with a graph database encoding of 3DSGs and providing Cypher query language as a tool for LLMs to retrieve relevant scene data for language grounding tasks.

Result: The Cypher interface approach scales significantly better to large, rich graphs on both local and cloud-based models, leading to large performance improvements in grounded language tasks while substantially reducing token count.

Conclusion: Using Cypher as an interface to 3D scene graphs provides an effective and scalable solution for connecting LLMs with complex world representations for natural language grounding.

Abstract: In order to provide a robot with the ability to understand and react to a user’s natural language inputs, the natural language must be connected to the robot’s underlying representations of the world. Recently, large language models (LLMs) and 3D scene graphs (3DSGs) have become a popular choice for grounding natural language and representing the world. In this work, we address the challenge of using LLMs with 3DSGs to ground natural language. Existing methods encode the scene graph as serialized text within the LLM’s context window, but this encoding does not scale to large or rich 3DSGs. Instead, we propose to use a form of Retrieval Augmented Generation to select a subset of the 3DSG relevant to the task. We encode a 3DSG in a graph database and provide a query language interface (Cypher) as a tool to the LLM with which it can retrieve relevant data for language grounding. We evaluate our approach on instruction following and scene question-answering tasks and compare against baseline context window and code generation methods. Our results show that using Cypher as an interface to 3D scene graphs scales significantly better to large, rich graphs on both local and cloud-based models. This leads to large performance improvements in grounded language tasks while also substantially reducing the token count of the scene graph content. A video supplement is available at https://www.youtube.com/watch?v=zY_YI9giZSA.

Method: Optimized using deep learning, UTAP generates fixed weak noise patterns that are added to pathology images to disrupt feature representations. The method demonstrates universality (works across diverse field-of-views) and transferability (affects various external black-box models).

Result: UTAP causes significant performance drops across various state-of-the-art pathology foundation models on multiple datasets with visually imperceptible modifications. It successfully degrades performance of external black-box models never seen during development.

Conclusion: UTAP constitutes a broad threat to emerging pathology foundation models and their applications, establishing a critical benchmark for robustness evaluation and highlighting the need for advancing defense mechanisms and adversarial training for reliable AI deployment in pathology.

Abstract: We introduce Universal and Transferable Adversarial Perturbations (UTAP) for pathology foundation models that reveal critical vulnerabilities in their capabilities. Optimized using deep learning, UTAP comprises a fixed and weak noise pattern that, when added to a pathology image, systematically disrupts the feature representation capabilities of multiple pathology foundation models. Therefore, UTAP induces performance drops in downstream tasks that utilize foundation models, including misclassification across a wide range of unseen data distributions. In addition to compromising the model performance, we demonstrate two key features of UTAP: (1) universality: its perturbation can be applied across diverse field-of-views independent of the dataset that UTAP was developed on, and (2) transferability: its perturbation can successfully degrade the performance of various external, black-box pathology foundation models - never seen before. These two features indicate that UTAP is not a dedicated attack associated with a specific foundation model or image dataset, but rather constitutes a broad threat to various emerging pathology foundation models and their applications. We systematically evaluated UTAP across various state-of-the-art pathology foundation models on multiple datasets, causing a significant drop in their performance with visually imperceptible modifications to the input images using a fixed noise pattern. The development of these potent attacks establishes a critical, high-standard benchmark for model robustness evaluation, highlighting a need for advancing defense mechanisms and potentially providing the necessary assets for adversarial training to ensure the safe and reliable deployment of AI in pathology.

Method: Uses Teacher-Student architecture with knowledge distillation - Teacher encodes latent hyperspectral data, Student learns mappings from natural images to Teacher’s encoded domain with novel training method.

Result: Achieves high-quality spectral reconstruction with SOTA performance across all metrics, including 18% accuracy boost and faster inference times at various channel depths.

Conclusion: HYDRA successfully addresses key limitations of prior spectral reconstruction models and provides superior performance for modern hyperspectral imaging applications.

Abstract: Hyperspectral images (HSI) promise to support a range of new applications in computer vision. Recent research has explored the feasibility of generalizable Spectral Reconstruction (SR), the problem of recovering a HSI from a natural three-channel color image in unseen scenarios. However, previous Multi-Scale Attention (MSA) works have only demonstrated sufficient generalizable results for very sparse spectra, while modern HSI sensors contain hundreds of channels. This paper introduces a novel approach to spectral reconstruction via our HYbrid knowledge Distillation and spectral Reconstruction Architecture (HYDRA). Using a Teacher model that encapsulates latent hyperspectral image data and a Student model that learns mappings from natural images to the Teacher’s encoded domain, alongside a novel training method, we achieve high-quality spectral reconstruction. This addresses key limitations of prior SR models, providing SOTA performance across all metrics, including an 18% boost in accuracy, and faster inference times than current SOTA models at various channel depths.

Method: Dual-agent framework with Perception Agent that programmatically queries 3D scenes using perception toolbox (including novel SOG module for direction grounding), and Reasoning Agent that iteratively refines information to achieve minimal sufficiency through closed-loop pruning and requests.

Result: Significantly improves accuracy and achieves state-of-the-art performance across two challenging benchmarks, while producing interpretable reasoning paths.

Conclusion: The explicit pursuit of both sufficiency and minimality in 3D information processing effectively addresses spatial reasoning challenges in VLMs, offering a promising approach for generating high-quality training data.

Abstract: Spatial reasoning, the ability to ground language in 3D understanding, remains a persistent challenge for Vision-Language Models (VLMs). We identify two fundamental bottlenecks: inadequate 3D understanding capabilities stemming from 2D-centric pre-training, and reasoning failures induced by redundant 3D information. To address these, we first construct a Minimal Sufficient Set (MSS) of information before answering a given question: a compact selection of 3D perception results from \textit{expert models}. We introduce MSSR (Minimal Sufficient Spatial Reasoner), a dual-agent framework that implements this principle. A Perception Agent programmatically queries 3D scenes using a versatile perception toolbox to extract sufficient information, including a novel SOG (Situated Orientation Grounding) module that robustly extracts language-grounded directions. A Reasoning Agent then iteratively refines this information to pursue minimality, pruning redundant details and requesting missing ones in a closed loop until the MSS is curated. Extensive experiments demonstrate that our method, by explicitly pursuing both sufficiency and minimality, significantly improves accuracy and achieves state-of-the-art performance across two challenging benchmarks. Furthermore, our framework produces interpretable reasoning paths, offering a promising source of high-quality training data for future models. Source code is available at https://github.com/gyj155/mssr.

Method: Proposes SDPA++ framework that leverages only noisy OCT images to generate pseudo-ground-truth through self-fusion and self-supervised denoising. Uses these refined images to train an ensemble of denoising models with patch-based strategy for enhanced clarity.

Result: Validated on IEEE SPS VIP Cup dataset containing only real-world noisy OCT images. Shows performance improvements in Contrast-to-Noise Ratio (CNR), Mean Square Ratio (MSR), Texture Preservation (TP), and Edge Preservation (EP) metrics.

Conclusion: The method demonstrates potential for improving OCT image quality and diagnostic outcomes in clinical practice without requiring clean reference images, addressing the challenge of limited paired datasets.

Abstract: Optical Coherence Tomography (OCT) is a widely used non-invasive imaging technique that provides detailed three-dimensional views of the retina, which are essential for the early and accurate diagnosis of ocular diseases. Consequently, OCT image analysis and processing have emerged as key research areas in biomedical imaging. However, acquiring paired datasets of clean and real-world noisy OCT images for supervised denoising models remains a formidable challenge due to intrinsic speckle noise and practical constraints in clinical imaging environments. To address these issues, we propose SDPA++: A General Framework for Self-Supervised Denoising with Patch Aggregation. Our novel approach leverages only noisy OCT images by first generating pseudo-ground-truth images through self-fusion and self-supervised denoising. These refined images then serve as targets to train an ensemble of denoising models using a patch-based strategy that effectively enhances image clarity. Performance improvements are validated via metrics such as Contrast-to-Noise Ratio (CNR), Mean Square Ratio (MSR), Texture Preservation (TP), and Edge Preservation (EP) on the real-world dataset from the IEEE SPS Video and Image Processing Cup. Notably, the VIP Cup dataset contains only real-world noisy OCT images without clean references, highlighting our method’s potential for improving image quality and diagnostic outcomes in clinical practice.

Method: DCCL enhances intra-class connectivity through: 1) aggressive data augmentation and cross-domain positive samples on data side, 2) model anchoring to exploit pre-trained representations and generative transformation loss on model side.

Result: Extensive experiments on five standard DG benchmarks show DCCL outperforms state-of-the-art baselines without requiring domain supervision.

Conclusion: DCCL successfully addresses the intra-class connectivity deficiency in DG settings and provides generalizable representations that improve domain generalization performance.

Abstract: Distribution shifts between training and testing samples frequently occur in practice and impede model generalization performance. This crucial challenge thereby motivates studies on domain generalization (DG), which aim to predict the label on unseen target domain data by solely using data from source domains. It is intuitive to conceive the class-separated representations learned in contrastive learning (CL) are able to improve DG, while the reality is quite the opposite: users observe directly applying CL deteriorates the performance. We analyze the phenomenon with the insights from CL theory and discover lack of intra-class connectivity in the DG setting causes the deficiency. We thus propose a new paradigm, domain-connecting contrastive learning (DCCL), to enhance the conceptual connectivity across domains and obtain generalizable representations for DG. On the data side, more aggressive data augmentation and cross-domain positive samples are introduced to improve intra-class connectivity. On the model side, to better embed the unseen test domains, we propose model anchoring to exploit the intra-class connectivity in pre-trained representations and complement the anchoring with generative transformation loss. Extensive experiments on five standard DG benchmarks are performed. The results verify that DCCL outperforms state-of-the-art baselines even without domain supervision. The detailed model implementation and the code are provided through https://github.com/weitianxin/DCCL

Method: Uses consistency models to learn self-consistent mapping between noisy and clean motion states, with Transformer-based spatiotemporal architecture and temporal embeddings for long-range dependencies.

Result: Achieves comparable or superior accuracy to state-of-the-art diffusion models on Human3.6M and HumanEva-I datasets while reducing inference steps by up to two orders of magnitude.

Conclusion: HumanCM provides an efficient alternative to diffusion models for human motion prediction, maintaining high accuracy with dramatically faster inference through one-step generation.

Abstract: We present HumanCM, a one-step human motion prediction framework built upon consistency models. Instead of relying on multi-step denoising as in diffusion-based methods, HumanCM performs efficient single-step generation by learning a self-consistent mapping between noisy and clean motion states. The framework adopts a Transformer-based spatiotemporal architecture with temporal embeddings to model long-range dependencies and preserve motion coherence. Experiments on Human3.6M and HumanEva-I demonstrate that HumanCM achieves comparable or superior accuracy to state-of-the-art diffusion models while reducing inference steps by up to two orders of magnitude.

Method: Introduces SCENECOT - a grounded Chain-of-Thought reasoning method that decouples complex tasks into simpler problems using multimodal expert modules to build visual clues.

Result: Achieves strong performance across various 3D scene reasoning benchmarks with high grounding-QA coherence, demonstrating the first successful application of CoT reasoning to 3D scene understanding.

Conclusion: The framework enables step-by-step human-like reasoning in 3D scenes and shows potential for extension to broader 3D scene understanding scenarios.

Abstract: Existing research on 3D Large Language Models (LLMs) still struggles to achieve grounded question-answering, primarily due to the under-exploration of the mech- anism of human-like scene-object grounded reasoning. This paper bridges the gap by presenting a novel framework. We first introduce a grounded Chain-of- Thought reasoning method in 3D scenes (SCENECOT), decoupling a complex reasoning task into simpler and manageable problems, and building corresponding visual clues based on multimodal expert modules. To enable such a method, we develop SCENECOT-185K, the first large-scale grounded CoT reasoning dataset, consisting of 185K high-quality instances. Extensive experiments across various complex 3D scene reasoning benchmarks demonstrate that our new framework achieves strong performance with high grounding-QA coherence. To the best of our knowledge, this is the first successful application of CoT reasoning to 3D scene understanding, enabling step-by-step human-like reasoning and showing potential for extension to broader 3D scene understanding scenarios.

Method: Uses BEV representation of current state, leverages previous timestep BEV features as temporal prior, predicts only residual changes conditioned on ego-vehicle actions and scene context, and applies alignment module to correct semantic/dynamic misalignments.

Result: Achieves top performance on nuScenes benchmark for both 4D occupancy forecasting and trajectory planning, with implicit future state generation substantially improving planning accuracy.

Conclusion: The proposed IR-WM approach effectively focuses on modeling world evolution rather than full reconstruction, demonstrating superior performance in autonomous driving tasks through efficient residual prediction and temporal modeling.

Abstract: End-to-end autonomous driving systems increasingly rely on vision-centric world models to understand and predict their environment. However, a common ineffectiveness in these models is the full reconstruction of future scenes, which expends significant capacity on redundantly modeling static backgrounds. To address this, we propose IR-WM, an Implicit Residual World Model that focuses on modeling the current state and evolution of the world. IR-WM first establishes a robust bird’s-eye-view representation of the current state from the visual observation. It then leverages the BEV features from the previous timestep as a strong temporal prior and predicts only the “residual”, i.e., the changes conditioned on the ego-vehicle’s actions and scene context. To alleviate error accumulation over time, we further apply an alignment module to calibrate semantic and dynamic misalignments. Moreover, we investigate different forecasting-planning coupling schemes and demonstrate that the implicit future state generated by world models substantially improves planning accuracy. On the nuScenes benchmark, IR-WM achieves top performance in both 4D occupancy forecasting and trajectory planning.

Method: UKANFormer builds on UKAN architecture with a Global-Local Transformer (GL-Trans) block in the decoder to extract both global semantic structures and local boundary details, enabling high-precision mapping under noisy supervision.

Result: Achieved coral-class IoU of 67.00% and pixel accuracy of 83.98%, outperforming conventional baselines under the same noisy labels setting, and produced predictions more accurate than the training labels themselves.

Conclusion: Architectural design can mitigate label noise and support scalable mapping under imperfect supervision, challenging the notion that data quality directly limits model performance, providing foundation for ecological monitoring with scarce reliable labels.

Abstract: Coral reefs are vital yet fragile ecosystems that require accurate large-scale mapping for effective conservation. Although global products such as the Allen Coral Atlas provide unprecedented coverage of global coral reef distri-bution, their predictions are frequently limited in spatial precision and semantic consistency, especially in regions requiring fine-grained boundary delineation. To address these challenges, we propose UKANFormer, a novel se-mantic segmentation model designed to achieve high-precision mapping under noisy supervision derived from Allen Coral Atlas. Building upon the UKAN architecture, UKANFormer incorporates a Global-Local Transformer (GL-Trans) block in the decoder, enabling the extraction of both global semantic structures and local boundary details. In experiments, UKANFormer achieved a coral-class IoU of 67.00% and pixel accuracy of 83.98%, outperforming conventional baselines under the same noisy labels setting. Remarkably, the model produces predictions that are visually and structurally more accurate than the noisy labels used for training. These results challenge the notion that data quality directly limits model performance, showing that architectural design can mitigate label noise and sup-port scalable mapping under imperfect supervision. UKANFormer provides a foundation for ecological monitoring where reliable labels are scarce.

Method: The paper formalizes the problem setting and learning objectives, and proposes a three-axis taxonomy: (1) Functionality: Decision-Coupled vs. General-Purpose; (2) Temporal Modeling: Sequential Simulation and Inference vs. Global Difference Prediction; (3) Spatial Representation: Global Latent Vector, Token Feature Sequence, Spatial Latent Grid, and Decomposed Rendering Representation.

Result: The survey systematizes data resources and metrics across robotics, autonomous driving, and general video settings, covering pixel prediction quality, state-level understanding, and task performance. It provides a quantitative comparison of state-of-the-art models.

Conclusion: Key open challenges include: scarcity of unified datasets, need for evaluation metrics that assess physical consistency over pixel fidelity, trade-off between model performance and computational efficiency for real-time control, and achieving long-horizon temporal consistency while mitigating error accumulation.

Abstract: Embodied AI requires agents that perceive, act, and anticipate how actions reshape future world states. World models serve as internal simulators that capture environment dynamics, enabling forward and counterfactual rollouts to support perception, prediction, and decision making. This survey presents a unified framework for world models in embodied AI. Specifically, we formalize the problem setting and learning objectives, and propose a three-axis taxonomy encompassing: (1) Functionality, Decision-Coupled vs. General-Purpose; (2) Temporal Modeling, Sequential Simulation and Inference vs. Global Difference Prediction; (3) Spatial Representation, Global Latent Vector, Token Feature Sequence, Spatial Latent Grid, and Decomposed Rendering Representation. We systematize data resources and metrics across robotics, autonomous driving, and general video settings, covering pixel prediction quality, state-level understanding, and task performance. Furthermore, we offer a quantitative comparison of state-of-the-art models and distill key open challenges, including the scarcity of unified datasets and the need for evaluation metrics that assess physical consistency over pixel fidelity, the trade-off between model performance and the computational efficiency required for real-time control, and the core modeling difficulty of achieving long-horizon temporal consistency while mitigating error accumulation. Finally, we maintain a curated bibliography at https://github.com/Li-Zn-H/AwesomeWorldModels.

Method: Applied beam search to discrete, sequential visual autoregressive models, leveraging the discrete token space for early pruning and computational reuse in text-to-image generation.

Result: Beam search substantially improved text-to-image generation, with a 2B parameter autoregressive model outperforming a 12B parameter diffusion model across benchmarks. Systematic ablations confirmed the advantage comes from discrete token space properties.

Conclusion: Model architecture, not just scale, is critical for inference-time optimization in visual generation, with discrete autoregressive models enabling effective search strategies that continuous diffusion models cannot match.

Abstract: While inference-time scaling through search has revolutionized Large Language Models, translating these gains to image generation has proven difficult. Recent attempts to apply search strategies to continuous diffusion models show limited benefits, with simple random sampling often performing best. We demonstrate that the discrete, sequential nature of visual autoregressive models enables effective search for image generation. We show that beam search substantially improves text-to-image generation, enabling a 2B parameter autoregressive model to outperform a 12B parameter diffusion model across benchmarks. Systematic ablations show that this advantage comes from the discrete token space, which allows early pruning and computational reuse, and our verifier analysis highlights trade-offs between speed and reasoning capability. These findings suggest that model architecture, not just scale, is critical for inference-time optimization in visual generation.

Method: Created a dataset of 1302 artifact examples from 11 SR methods with crowdsourced prominence scores. Trained a lightweight regressor to produce spatial prominence heatmaps for artifact detection.

Result: The trained regressor outperforms existing methods at detecting prominent artifacts and produces spatial prominence heatmaps that effectively identify visually disturbing artifacts.

Conclusion: The proposed prominence-aware approach provides better evaluation and mitigation of SR artifacts. The dataset and code are released to facilitate further research in this direction.

Abstract: Generative image super-resolution (SR) is rapidly advancing in visual quality and detail restoration. As the capacity of SR models expands, however, so does their tendency to produce artifacts: incorrect, visually disturbing details that reduce perceived quality. Crucially, their perceptual impact varies: some artifacts are barely noticeable while others strongly degrade the image. We argue that artifacts should be characterized by their prominence to human observers rather than treated as uniform binary defects. Motivated by this, we present a novel dataset of 1302 artifact examples from 11 contemporary image-SR methods, where each artifact is paired with a crowdsourced prominence score. Building on this dataset, we train a lightweight regressor that produces spatial prominence heatmaps and outperforms existing methods at detecting prominent artifacts. We release the dataset and code to facilitate prominence-aware evaluation and mitigation of SR artifacts.

Method: Proposes three key components: Global Multiscale Wavelet Transform Convolutions (GMWTConvs) to expand receptive field, Mamba-Based Channel-Aware Module (MCAM) to capture long-range dependencies, and Multiscale Texture Enhancement Loss (MTELoss) to preserve texture structures.

Result: Extensive experiments show WaMaIR outperforms state-of-the-art methods in image restoration quality while maintaining efficient computational performance.

Conclusion: WaMaIR effectively addresses texture detail restoration challenges in image restoration through its novel architecture combining wavelet transforms, channel-aware modeling, and specialized loss functions.

Abstract: Image restoration is a fundamental and challenging task in computer vision, where CNN-based frameworks demonstrate significant computational efficiency. However, previous CNN-based methods often face challenges in adequately restoring fine texture details, which are limited by the small receptive field of CNN structures and the lack of channel feature modeling. In this paper, we propose WaMaIR, which is a novel framework with a large receptive field for image perception and improves the reconstruction of texture details in restored images. Specifically, we introduce the Global Multiscale Wavelet Transform Convolutions (GMWTConvs) for expandding the receptive field to extract image features, preserving and enriching texture features in model inputs. Meanwhile, we propose the Mamba-Based Channel-Aware Module (MCAM), explicitly designed to capture long-range dependencies within feature channels, which enhancing the model sensitivity to color, edges, and texture information. Additionally, we propose Multiscale Texture Enhancement Loss (MTELoss) for image restoration to guide the model in preserving detailed texture structures effectively. Extensive experiments confirm that WaMaIR outperforms state-of-the-art methods, achieving better image restoration and efficient computational performance of the model.

Method: Introduces a dual-level guidance mechanism: regions are represented with full-image context and aligned with detailed region-level descriptions, while the entire image is simultaneously matched to comprehensive scene-level descriptions generated by a large vision-language model.

Result: Experiments show the method produces more coherent and instruction-aligned results compared to existing approaches.

Conclusion: The proposed framework enables precise and harmonized image editing by encouraging regions to understand their role within the global image context through multilevel semantic alignment between vision and language.

Abstract: Recent research has made significant progress in localizing and editing image regions based on text. However, most approaches treat these regions in isolation, relying solely on local cues without accounting for how each part contributes to the overall visual and semantic composition. This often results in inconsistent edits, unnatural transitions, or loss of coherence across the image. In this work, we propose Region in Context, a novel framework for text-conditioned image editing that performs multilevel semantic alignment between vision and language, inspired by the human ability to reason about edits in relation to the whole scene. Our method encourages each region to understand its role within the global image context, enabling precise and harmonized changes. At its core, the framework introduces a dual-level guidance mechanism: regions are represented with full-image context and aligned with detailed region-level descriptions, while the entire image is simultaneously matched to a comprehensive scene-level description generated by a large vision-language model. These descriptions serve as explicit verbal references of the intended content, guiding both local modifications and global structure. Experiments show that it produces more coherent and instruction-aligned results. Code is available at: https://github.com/thuyvuphuong/Region-in-Context.git

Method: X-ray images are divided into patches, tokenized, and processed by an SSM-based vision backbone for feature extraction using Partial LoRA. An LLM with hybrid decoder generates medical reports through end-to-end training.

Result: Extensive experiments on three widely used benchmark datasets fully validated the effectiveness of the proposed strategies for X-ray medical report generation.

Conclusion: The proposed EMRRG framework demonstrates the potential of fine-tuning pre-trained Mamba networks with parameter-efficient methods for medical report generation, achieving strong results on benchmark datasets.

Abstract: X-ray image-based medical report generation (MRG) is a pivotal area in artificial intelligence that can significantly reduce diagnostic burdens for clinicians and patient wait times. Existing MRG models predominantly rely on Large Language Models (LLMs) to improve report generation, with limited exploration of pre-trained vision foundation models or advanced fine-tuning techniques. Mainstream frameworks either avoid fine-tuning or utilize simplistic methods like LoRA, often neglecting the potential of enhancing cross-attention mechanisms. Additionally, while Transformer-based models dominate vision-language tasks, non-Transformer architectures, such as the Mamba network, remain underexplored for medical report generation, presenting a promising avenue for future research. In this paper, we propose EMRRG, a novel X-ray report generation framework that fine-tunes pre-trained Mamba networks using parameter-efficient methods. Specifically, X-ray images are divided into patches, tokenized, and processed by an SSM-based vision backbone for feature extraction, with Partial LoRA yielding optimal performance. An LLM with a hybrid decoder generates the medical report, enabling end-to-end training and achieving strong results on benchmark datasets. Extensive experiments on three widely used benchmark datasets fully validated the effectiveness of our proposed strategies for the X-ray MRG. The source code of this paper will be released on https://github.com/Event-AHU/Medical_Image_Analysis.

Method: Formulates pose regression using Bundle Adjustment principles, extends 3DGS with Lie algebra for pose-differentiable rendering, and iteratively optimizes pose by comparing input and rendered images while updating color parameters.

Result: Achieves accuracy improvements of 1.4%, 2.8% and 2.5% on T-LESS, LineMod-Occlusion and LineMod datasets respectively compared to previous models.

Conclusion: GS2POSE effectively addresses challenges in 6D pose estimation for textureless objects under varying illumination through differentiable rendering and iterative optimization.

Abstract: Accurate 6D pose estimation of 3D objects is a fundamental task in computer vision, and current research typically predicts the 6D pose by establishing correspondences between 2D image features and 3D model features. However, these methods often face difficulties with textureless objects and varying illumination conditions. To overcome these limitations, we propose GS2POSE, a novel approach for 6D object pose estimation. GS2POSE formulates a pose regression algorithm inspired by the principles of Bundle Adjustment (BA). By leveraging Lie algebra, we extend the capabilities of 3DGS to develop a pose-differentiable rendering pipeline, which iteratively optimizes the pose by comparing the input image to the rendered image. Additionally, GS2POSE updates color parameters within the 3DGS model, enhancing its adaptability to changes in illumination. Compared to previous models, GS2POSE demonstrates accuracy improvements of 1.4%, 2.8% and 2.5% on the T-LESS, LineMod-Occlusion and LineMod datasets, respectively.

Method: Reframe video understanding as self-supervised spatio-temporal clustering. Transform video into semantic feature trajectory using frozen VLM encoder, apply Kernel Temporal Segmentation (KTS) for event segmentation, then use density-based clustering to identify recurring scenes and themes.

Result: Automatically produces structured, multi-modal video summaries by selecting representative keyframes from discovered clusters and generating textual descriptions using VLM capabilities.

Conclusion: Provides an effective, interpretable, and model-agnostic pathway for zero-shot automated structural analysis of video content without requiring training.

Abstract: The remarkable zero-shot reasoning capabilities of large-scale Visual Language Models (VLMs) on static images have yet to be fully translated to the video domain. Conventional video understanding models often rely on extensive, task-specific training on annotated datasets, a process that is both costly and limited in scalability. This paper introduces a novel, training-free framework for video understanding that circumvents end-to-end training by synergistically combining the rich semantic priors of pre-trained VLMs with classic machine learning algorithms for pattern discovery. Our core idea is to reframe video understanding as a self-supervised spatio-temporal clustering problem within a high-dimensional semantic feature space. The proposed pipeline first transforms a video stream into a semantic feature trajectory using the frozen visual encoder of a pre-trained VLM. Subsequently, we employ Kernel Temporal Segmentation (KTS), a robust machine learning technique, to partition the continuous feature stream into discrete, semantically coherent event segments. These segments are then subjected to unsupervised density-based clustering to identify recurring macroscopic scenes and themes throughout the video. By selecting representative keyframes from each discovered cluster and leveraging the VLM’s generative capabilities for textual description, our framework automatically produces a structured, multi-modal summary of the video content. This approach provides an effective, interpretable, and model-agnostic pathway for zero-shot, automated structural analysis of video content.

Method: LENS attaches a lightweight, trainable head to a frozen MLLM, refining spatial cues from attention maps to extract keypoints and generate point-wise features compatible with mask decoders.

Result: Extensive experiments show LENS achieves segmentation performance competitive with or superior to retraining-based methods while fully preserving the MLLM’s generalization capabilities.

Conclusion: LENS establishes an efficient paradigm for extending MLLMs with segmentation capabilities without compromising their generalization, paving the way for truly multi-talented unified models.

Abstract: Integrating diverse visual capabilities into a unified model is a significant trend in Multimodal Large Language Models (MLLMs). Among these, the inclusion of segmentation poses a distinct set of challenges. To equip MLLMs with pixel-level segmentation abilities, prevailing methods require finetuning the model to produce specific outputs compatible with a mask decoder. This process typically alters the model’s output space and compromises its intrinsic generalization, which undermines the goal of building a unified model. We introduce LENS (Leveraging kEypoiNts for MLLMs’ Segmentation), a novel plug-and-play solution. LENS attaches a lightweight, trainable head to a completely frozen MLLM. By refining the spatial cues embedded in attention maps, LENS extracts keypoints and describes them into point-wise features directly compatible with the mask decoder. Extensive experiments validate our approach: LENS achieves segmentation performance competitive with or superior to that of retraining-based methods. Crucially, it does so while fully preserving the MLLM’s generalization capabilities, which are significantly degraded by finetuning approaches. As such, the attachable design of LENS establishes an efficient and powerful paradigm for extending MLLMs, paving the way for truly multi-talented, unified models.

Method: Uses geometric priors to generate weak labels (pixels above horizon as non-road, quadrilateral in front as road), then enforces temporal consistency through feature tracking and mutual information maximization.

Result: Achieves 0.82 Intersection-over-Union (IoU) on Cityscapes dataset with high accuracy and temporal stability.

Conclusion: Combining geometric constraints and temporal consistency enables scalable unsupervised road segmentation for autonomous driving applications.

Abstract: This paper presents a fully unsupervised approach for binary road segmentation (road vs. non-road), eliminating the reliance on costly manually labeled datasets. The method leverages scene geometry and temporal cues to distinguish road from non-road regions. Weak labels are first generated from geometric priors, marking pixels above the horizon as non-road and a predefined quadrilateral in front of the vehicle as road. In a refinement stage, temporal consistency is enforced by tracking local feature points across frames and penalizing inconsistent label assignments using mutual information maximization. This enhances both precision and temporal stability. On the Cityscapes dataset, the model achieves an Intersection-over-Union (IoU) of 0.82, demonstrating high accuracy with a simple design. These findings demonstrate the potential of combining geometric constraints and temporal consistency for scalable unsupervised road segmentation in autonomous driving.

Method: Based on pretrained text-to-image diffusion model, PIF learns average appearance of photographic concepts and adjusts them via text prompts. Uses textual inversion technique to optimize prompts for photographic concepts from reference images.

Result: PIF shows outstanding performance in extracting and transferring various kinds of photographic style.

Conclusion: PIF effectively addresses limitations of previous methods by leveraging diffusion model generative prior and textual inversion for photographic style learning and transfer.

Abstract: Photographic style, as a composition of certain photographic concepts, is the charm behind renowned photographers. But learning and transferring photographic style need a profound understanding of how the photo is edited from the unknown original appearance. Previous works either fail to learn meaningful photographic concepts from reference images, or cannot preserve the content of the content image. To tackle these issues, we proposed a Personalized Image Filter (PIF). Based on a pretrained text-to-image diffusion model, the generative prior enables PIF to learn the average appearance of photographic concepts, as well as how to adjust them according to text prompts. PIF then learns the photographic style of reference images with the textual inversion technique, by optimizing the prompts for the photographic concepts. PIF shows outstanding performance in extracting and transferring various kinds of photographic style. Project page: https://pif.pages.dev/

Method: Collected color (RGB) images under diverse weather conditions and times across multiple lychee varieties, applied data augmentation, included depth images, and implemented rigorous annotation process with multiple independent labelers and verification.

Result: Produced a dataset with 11,414 images (878 raw RGB, 8,780 augmented RGB, 1,756 depth) annotated with 9,658 label pairs for detection and maturity classification across three ripeness stages.

Conclusion: The dataset enables development of vision-based harvesting robots for lychee, is publicly available for academic use, and was validated using three deep learning models.

Abstract: Lychee is a high-value subtropical fruit. The adoption of vision-based harvesting robots can significantly improve productivity while reduce reliance on labor. High-quality data are essential for developing such harvesting robots. However, there are currently no consistently and comprehensively annotated open-source lychee datasets featuring fruits in natural growing environments. To address this, we constructed a dataset to facilitate lychee detection and maturity classification. Color (RGB) images were acquired under diverse weather conditions, and at different times of the day, across multiple lychee varieties, such as Nuomici, Feizixiao, Heiye, and Huaizhi. The dataset encompasses three different ripeness stages and contains 11,414 images, consisting of 878 raw RGB images, 8,780 augmented RGB images, and 1,756 depth images. The images are annotated with 9,658 pairs of lables for lychee detection and maturity classification. To improve annotation consistency, three individuals independently labeled the data, and their results were then aggregated and verified by a fourth reviewer. Detailed statistical analyses were done to examine the dataset. Finally, we performed experiments using three representative deep learning models to evaluate the dataset. It is publicly available for academic

Method: Aggregated imagery from 76 curated CoralNet sources and Al Wajh site, totaling ~925K genus-level hard coral annotations with expert-verified labels mapped to WoRMS. Proposed two evaluation settings: within-source and cross-source benchmarks.

Result: Supervised within-source performance is promising but drops sharply across domains. Zero-shot models perform poorly across the board, especially for rare and visually similar genera.

Conclusion: ReefNet provides a challenging benchmark to catalyze advances in domain generalization and fine-grained coral classification, supporting robust global coral reef monitoring and conservation.

Abstract: Coral reefs are rapidly declining due to anthropogenic pressures such as climate change, underscoring the urgent need for scalable, automated monitoring. We introduce ReefNet, a large public coral reef image dataset with point-label annotations mapped to the World Register of Marine Species (WoRMS). ReefNet aggregates imagery from 76 curated CoralNet sources and an additional site from Al Wajh in the Red Sea, totaling approximately 925000 genus-level hard coral annotations with expert-verified labels. Unlike prior datasets, which are often limited by size, geography, or coarse labels and are not ML-ready, ReefNet offers fine-grained, taxonomically mapped labels at a global scale to WoRMS. We propose two evaluation settings: (i) a within-source benchmark that partitions each source’s images for localized evaluation, and (ii) a cross-source benchmark that withholds entire sources to test domain generalization. We analyze both supervised and zero-shot classification performance on ReefNet and find that while supervised within-source performance is promising, supervised performance drops sharply across domains, and performance is low across the board for zero-shot models, especially for rare and visually similar genera. This provides a challenging benchmark intended to catalyze advances in domain generalization and fine-grained coral classification. We will release our dataset, benchmarking code, and pretrained models to advance robust, domain-adaptive, global coral reef monitoring and conservation.

Method: Comprehensive analysis of five texture feature types from wood chip images, combined feature sets, and a domain adaptation method (AdaptMoist) that transfers knowledge between wood chip sources using texture features with model saving based on adjusted mutual information.

Result: Combined texture features achieved 95% accuracy for moisture prediction. AdaptMoist improved cross-domain prediction accuracy by 23%, achieving 80% average accuracy compared to 57% for non-adapted models.

Conclusion: AdaptMoist is an effective robust solution for wood chip moisture content estimation across domains, making it suitable for wood chip-reliant industries by addressing source variability issues.

Abstract: Accurate and quick prediction of wood chip moisture content is critical for optimizing biofuel production and ensuring energy efficiency. The current widely used direct method (oven drying) is limited by its longer processing time and sample destructiveness. On the other hand, existing indirect methods, including near-infrared spectroscopy-based, electrical capacitance-based, and image-based approaches, are quick but not accurate when wood chips come from various sources. Variability in the source material can alter data distributions, undermining the performance of data-driven models. Therefore, there is a need for a robust approach that effectively mitigates the impact of source variability. Previous studies show that manually extracted texture features have the potential to predict wood chip moisture class. Building on this, in this study, we conduct a comprehensive analysis of five distinct texture feature types extracted from wood chip images to predict moisture content. Our findings reveal that a combined feature set incorporating all five texture features achieves an accuracy of 95% and consistently outperforms individual texture features in predicting moisture content. To ensure robust moisture prediction, we propose a domain adaptation method named AdaptMoist that utilizes the texture features to transfer knowledge from one source of wood chip data to another, addressing variability across different domains. We also proposed a criterion for model saving based on adjusted mutual information. The AdaptMoist method improves prediction accuracy across domains by 23%, achieving an average accuracy of 80%, compared to 57% for non-adapted models. These results highlight the effectiveness of AdaptMoist as a robust solution for wood chip moisture content estimation across domains, making it a potential solution for wood chip-reliant industries.

Method: Proposes M2HVideo with dynamic pose-aware head encoder for consistent identity embeddings, mirror loss in pixel space via DDIM-based denoising, and distribution-aware adapter for temporal coherence.

Result: Extensive experiments on UBC fashion, ASOS, and MannequinVideos datasets show superior performance in clothing consistency, identity preservation, and video fidelity compared to state-of-the-art methods.

Conclusion: M2HVideo effectively addresses key challenges in mannequin-to-human video generation, providing a practical solution for realistic fashion presentation while maintaining identity consistency and temporal coherence.

Abstract: Mannequin-based clothing displays offer a cost-effective alternative to real-model showcases for online fashion presentation, but lack realism and expressive detail. To overcome this limitation, we introduce a new task called mannequin-to-human (M2H) video generation, which aims to synthesize identity-controllable, photorealistic human videos from footage of mannequins. We propose M2HVideo, a pose-aware and identity-preserving video generation framework that addresses two key challenges: the misalignment between head and body motion, and identity drift caused by temporal modeling. In particular, M2HVideo incorporates a dynamic pose-aware head encoder that fuses facial semantics with body pose to produce consistent identity embeddings across frames. To address the loss of fine facial details due to latent space compression, we introduce a mirror loss applied in pixel space through a denoising diffusion implicit model (DDIM)-based one-step denoising. Additionally, we design a distribution-aware adapter that aligns statistical distributions of identity and clothing features to enhance temporal coherence. Extensive experiments on the UBC fashion dataset, our self-constructed ASOS dataset, and the newly collected MannequinVideos dataset captured on-site demonstrate that M2HVideo achieves superior performance in terms of clothing consistency, identity preservation, and video fidelity in comparison to state-of-the-art methods.

Method: Uses hierarchical training: first trains 2D Gaussians with normal consistency regularization, then selects underperforming Gaussians for in-place cloning, and finally fine-tunes with frozen opacity.

Result: Achieves high-quality rendering while preserving fine geometric structures with only 1% more storage and minimal additional training time compared to original 2DGS.

Conclusion: The approach effectively balances efficiency with performance, leading to improvements in both visual fidelity and geometric reconstruction accuracy.

Abstract: Recent advancements in 3D Gaussian Splatting (3DGS) have greatly influenced neural fields, as it enables high-fidelity rendering with impressive visual quality. However, 3DGS has difficulty accurately representing surfaces. In contrast, 2DGS transforms the 3D volume into a collection of 2D planar Gaussian disks. Despite advancements in geometric fidelity, rendering quality remains compromised, highlighting the challenge of achieving both high-quality rendering and precise geometric structures. This indicates that optimizing both geometric and rendering quality in a single training stage is currently unfeasible. To overcome this limitation, we present 2DGS-R, a new method that uses a hierarchical training approach to improve rendering quality while maintaining geometric accuracy. 2DGS-R first trains the original 2D Gaussians with the normal consistency regularization. Then 2DGS-R selects the 2D Gaussians with inadequate rendering quality and applies a novel in-place cloning operation to enhance the 2D Gaussians. Finally, we fine-tune the 2DGS-R model with opacity frozen. Experimental results show that compared to the original 2DGS, our method requires only 1% more storage and minimal additional training time. Despite this negligible overhead, it achieves high-quality rendering results while preserving fine geometric structures. These findings indicate that our approach effectively balances efficiency with performance, leading to improvements in both visual fidelity and geometric reconstruction accuracy.

Method: Integrates Convolutional Block Attention Module (CBAM) with MixVisionTransformer architecture, combining CBAM-enhanced encoder backbone with attention-integrated hamburger decoder for multi-class weapon segmentation across five categories.

Result: Achieves state-of-the-art performance with 80.64% mIoU and 89.13% mFscore while maintaining real-time inference at 82.26 FPS, with only 4.886G FLOPs and 3.66M parameters.

Conclusion: ArmFormer outperforms heavyweight models requiring up to 48x more computation, establishing it as the optimal solution for deployment on portable security cameras, surveillance drones, and embedded AI accelerators in distributed security infrastructure.

Abstract: The escalating threat of weapon-related violence necessitates automated detection systems capable of pixel-level precision for accurate threat assessment in real-time security applications. Traditional weapon detection approaches rely on object detection frameworks that provide only coarse bounding box localizations, lacking the fine-grained segmentation required for comprehensive threat analysis. Furthermore, existing semantic segmentation models either sacrifice accuracy for computational efficiency or require excessive computational resources incompatible with edge deployment scenarios. This paper presents ArmFormer, a lightweight transformer-based semantic segmentation framework that strategically integrates Convolutional Block Attention Module (CBAM) with MixVisionTransformer architecture to achieve superior accuracy while maintaining computational efficiency suitable for resource-constrained edge devices. Our approach combines CBAM-enhanced encoder backbone with attention-integrated hamburger decoder to enable multi-class weapon segmentation across five categories: handgun, rifle, knife, revolver, and human. Comprehensive experiments demonstrate that ArmFormer achieves state-of-the-art performance with 80.64% mIoU and 89.13% mFscore while maintaining real-time inference at 82.26 FPS. With only 4.886G FLOPs and 3.66M parameters, ArmFormer outperforms heavyweight models requiring up to 48x more computation, establishing it as the optimal solution for deployment on portable security cameras, surveillance drones, and embedded AI accelerators in distributed security infrastructure.

Method: BARL uses two collaborative branches with Dual-Path Regularization (DPR) and Progressively Cognitive Bias Correction (PCBC) for label-space alignment, plus region-level and lesion-instance matching for representation-space alignment.

Result: Extensive experiments on four public benchmarks and a proprietary CBCT dataset show BARL consistently outperforms state-of-the-art SSMIS methods.

Conclusion: BARL effectively addresses the limitations of existing SSMIS methods by enforcing bilateral alignment, demonstrating superior performance across multiple medical imaging datasets.

Abstract: Semi-supervised medical image segmentation (SSMIS) seeks to match fully supervised performance while sharply reducing annotation cost. Mainstream SSMIS methods rely on \emph{label-space consistency}, yet they overlook the equally critical \emph{representation-space alignment}. Without harmonizing latent features, models struggle to learn representations that are both discriminative and spatially coherent. To this end, we introduce \textbf{Bilateral Alignment in Representation and Label spaces (BARL)}, a unified framework that couples two collaborative branches and enforces alignment in both spaces. For label-space alignment, inspired by co-training and multi-scale decoding, we devise \textbf{Dual-Path Regularization (DPR)} and \textbf{Progressively Cognitive Bias Correction (PCBC)} to impose fine-grained cross-branch consistency while mitigating error accumulation from coarse to fine scales. For representation-space alignment, we conduct region-level and lesion-instance matching between branches, explicitly capturing the fragmented, complex pathological patterns common in medical imagery. Extensive experiments on four public benchmarks and a proprietary CBCT dataset demonstrate that BARL consistently surpasses state-of-the-art SSMIS methods. Ablative studies further validate the contribution of each component. Code will be released soon.

Method: Proposes a framework that embeds feature extraction into the registration learning process, jointly optimizing alignment and representation learning to acquire rotation-invariant and locally discriminative features.

Result: Extensive experiments on Anomaly-ShapeNet and Real3D-AD datasets show the method consistently outperforms existing approaches in effectiveness and generalizability.

Conclusion: Point-cloud registration plays an essential role in guiding feature extraction toward rotation-invariant and locally discriminative representations, and integrating registration with anomaly detection significantly improves performance.

Abstract: 3D anomaly detection in point-cloud data is critical for industrial quality control, aiming to identify structural defects with high reliability. However, current memory bank-based methods often suffer from inconsistent feature transformations and limited discriminative capacity, particularly in capturing local geometric details and achieving rotation invariance. These limitations become more pronounced when registration fails, leading to unreliable detection results. We argue that point-cloud registration plays an essential role not only in aligning geometric structures but also in guiding feature extraction toward rotation-invariant and locally discriminative representations. To this end, we propose a registration-induced, rotation-invariant feature extraction framework that integrates the objectives of point-cloud registration and memory-based anomaly detection. Our key insight is that both tasks rely on modeling local geometric structures and leveraging feature similarity across samples. By embedding feature extraction into the registration learning process, our framework jointly optimizes alignment and representation learning. This integration enables the network to acquire features that are both robust to rotations and highly effective for anomaly detection. Extensive experiments on the Anomaly-ShapeNet and Real3D-AD datasets demonstrate that our method consistently outperforms existing approaches in effectiveness and generalizability.

Method: Proposed a novel neuron-level analysis framework combining fine-grained artificial neuron analysis with fMRI-based voxel encoding to examine two architecturally distinct VLMs (CLIP and METER).

Result: Four key findings: (1) ANs predict BN activities across multiple functional networks; (2) Both show functional redundancy; (3) ANs exhibit polarity patterns paralleling BNs; (4) Different architectures drive distinct BN activations - CLIP shows modality-specific specialization while METER yields unified cross-modal activation.

Conclusion: The results provide compelling evidence for brain-like hierarchical processing in VLMs at the neuronal level, demonstrating shared representational mechanisms between artificial and biological neural systems.

Abstract: While brain-inspired artificial intelligence(AI) has demonstrated promising results, current understanding of the parallels between artificial neural networks (ANNs) and human brain processing remains limited: (1) unimodal ANN studies fail to capture the brain’s inherent multimodal processing capabilities, and (2) multimodal ANN research primarily focuses on high-level model outputs, neglecting the crucial role of individual neurons. To address these limitations, we propose a novel neuron-level analysis framework that investigates the multimodal information processing mechanisms in vision-language models (VLMs) through the lens of human brain activity. Our approach uniquely combines fine-grained artificial neuron (AN) analysis with fMRI-based voxel encoding to examine two architecturally distinct VLMs: CLIP and METER. Our analysis reveals four key findings: (1) ANs successfully predict biological neurons (BNs) activities across multiple functional networks (including language, vision, attention, and default mode), demonstrating shared representational mechanisms; (2) Both ANs and BNs demonstrate functional redundancy through overlapping neural representations, mirroring the brain’s fault-tolerant and collaborative information processing mechanisms; (3) ANs exhibit polarity patterns that parallel the BNs, with oppositely activated BNs showing mirrored activation trends across VLM layers, reflecting the complexity and bidirectional nature of neural information processing; (4) The architectures of CLIP and METER drive distinct BNs: CLIP’s independent branches show modality-specific specialization, whereas METER’s cross-modal design yields unified cross-modal activation, highlighting the architecture’s influence on ANN brain-like properties. These results provide compelling evidence for brain-like hierarchical processing in VLMs at the neuronal level.

Method: Integrates a classifier within a diffusion model to guide image generation based on class conditions, enabling better control over class-conditioned image synthesis.

Result: Generates more realistic and diverse dermoscopic images, and the classifier shows improved performance for downstream diagnostic tasks.

Conclusion: Class-N-Diff is a robust tool for enhancing the quality and utility of diffusion model-based synthetic dermoscopic image generation.

Abstract: Generative models, especially Diffusion Models, have demonstrated remarkable capability in generating high-quality synthetic data, including medical images. However, traditional class-conditioned generative models often struggle to generate images that accurately represent specific medical categories, limiting their usefulness for applications such as skin cancer diagnosis. To address this problem, we propose a classification-induced diffusion model, namely, Class-N-Diff, to simultaneously generate and classify dermoscopic images. Our Class-N-Diff model integrates a classifier within a diffusion model to guide image generation based on its class conditions. Thus, the model has better control over class-conditioned image synthesis, resulting in more realistic and diverse images. Additionally, the classifier demonstrates improved performance, highlighting its effectiveness for downstream diagnostic tasks. This unique integration in our Class-N-Diff makes it a robust tool for enhancing the quality and utility of diffusion model-based synthetic dermoscopic image generation. Our code is available at https://github.com/Munia03/Class-N-Diff.

Method: Uses Diffusion Negative-aware Finetuning (DiffusionNFT) for policy optimization and employs a Multimodal Large Language Model as a unified, training-free reward model with low-variance group filtering to reduce noise.

Result: UniWorld-V2 achieves state-of-the-art results on ImgEdit (4.49) and GEdit-Bench (7.83), with the framework being model-agnostic and delivering substantial performance gains across diverse base models.

Conclusion: The Edit-R1 framework effectively addresses overfitting and generalization issues in instruction-based image editing through policy optimization and MLLM-based rewards, demonstrating wide applicability and superior performance.

Abstract: Instruction-based image editing has achieved remarkable progress; however, models solely trained via supervised fine-tuning often overfit to annotated patterns, hindering their ability to explore and generalize beyond training distributions. To this end, we introduce Edit-R1, a novel post-training framework for instruction-based image editing based on policy optimization. Specifically, we utilize Diffusion Negative-aware Finetuning (DiffusionNFT), a likelihood-free policy optimization method consistent with the flow matching forward process, thereby enabling the use of higher-order samplers and more efficient training. Another key challenge here is the absence of a universal reward model, resulting from the diverse nature of editing instructions and tasks. To bridge this gap, we employ a Multimodal Large Language Model (MLLM) as a unified, training-free reward model, leveraging its output logits to provide fine-grained feedback. Furthermore, we carefully design a low-variance group filtering mechanism to reduce MLLM scoring noise and stabilize optimization. UniWorld-V2, trained with this framework, achieves \textbf{state-of-the-art} results on the ImgEdit and GEdit-Bench benchmarks, scoring 4.49 and 7.83, respectively. Crucially, our framework is model-agnostic, delivering substantial performance gains when applied to diverse base models like Qwen-Image-Edit and FLUX-Kontext, demonstrating its wide applicability. Code and models are publicly available at https://github.com/PKU-YuanGroup/UniWorld-V2.

Method: Uses ground-based cameras to capture contrails shortly after formation when they remain thin and distinct. Introduces a modular framework with multiple geometric representations, distance metrics, temporal smoothing, and probability-based assignment strategies.

Result: Establishes a strong baseline for contrail-to-flight attribution using the ground visible camera contrail sequences (GVCCS) dataset.

Conclusion: Provides a modular framework for future research in linking contrails to their source flights, enabling better validation of contrail formation and evolution models.

Abstract: Aviation’s non-CO2 effects, particularly contrails, are a significant contributor to its climate impact. Persistent contrails can evolve into cirrus-like clouds that trap outgoing infrared radiation, with radiative forcing potentially comparable to or exceeding that of aviation’s CO2 emissions. While physical models simulate contrail formation, evolution and dissipation, validating and calibrating these models requires linking observed contrails to the flights that generated them, a process known as contrail-to-flight attribution. Satellite-based attribution is challenging due to limited spatial and temporal resolution, as contrails often drift and deform before detection. In this paper, we evaluate an alternative approach using ground-based cameras, which capture contrails shortly after formation at high spatial and temporal resolution, when they remain thin, linear, and visually distinct. Leveraging the ground visible camera contrail sequences (GVCCS) dataset, we introduce a modular framework for attributing contrails observed using ground-based cameras to theoretical contrails derived from aircraft surveillance and meteorological data. The framework accommodates multiple geometric representations and distance metrics, incorporates temporal smoothing, and enables flexible probability-based assignment strategies. This work establishes a strong baseline and provides a modular framework for future research in linking contrails to their source flight.

Method: Adapted and evaluated four transformer architectures on a custom thermal dataset of 9,711 images from real surveillance videos, automatically annotated using SAM2. Used standard augmentation strategies within MMSegmentation framework for robust training and fair comparison.

Result: SegFormer-b5 achieved highest mIoU (94.15%) and Pixel Accuracy (97.04%), while SegFormer-b0 provided fastest inference (98.32 FPS) with competitive mIoU (90.84%). SegNeXt-mscans offered balanced performance (85.12 FPS, 92.24% mIoU), and DeepLabV3+ R101-D8 reached 92.76% mIoU at 29.86 FPS.

Conclusion: Transformer architectures demonstrate robust generalization for weapon detection in low-light thermal environments, offering flexible accuracy-speed trade-offs suitable for diverse real-time security applications.

Abstract: Thermal weapon segmentation is crucial for surveillance and security applications, enabling robust detection under lowlight and visually obscured conditions where RGB-based systems fail. While convolutional neural networks (CNNs) dominate thermal segmentation literature, their ability to capture long-range dependencies and fine structural details is limited. Vision Transformers (ViTs), with their global context modeling capabilities, have achieved state-of-the-art results in RGB segmentation tasks, yet their potential in thermal weapon segmentation remains underexplored. This work adapts and evaluates four transformer-based architectures SegFormer, DeepLabV3+, SegNeXt, and Swin Transformer for binary weapon segmentation on a custom thermal dataset comprising 9,711 images collected from real world surveillance videos and automatically annotated using SAM2. We employ standard augmentation strategies within the MMSegmentation framework to ensure robust model training and fair architectural comparison. Experimental results demonstrate significant improvements in segmentation performance: SegFormer-b5 achieves the highest mIoU (94.15%) and Pixel Accuracy (97.04%), while SegFormer-b0 provides the fastest inference speed (98.32 FPS) with competitive mIoU (90.84%). SegNeXt-mscans offers balanced performance with 85.12 FPS and 92.24% mIoU, and DeepLabV3+ R101-D8 reaches 92.76% mIoU at 29.86 FPS. The transformer architectures demonstrate robust generalization capabilities for weapon detection in low-light and occluded thermal environments, with flexible accuracy-speed trade-offs suitable for diverse real-time security applications.

Method: Developed Res-Bench with 14,400 samples across 12 resolution levels and 6 capability dimensions, introducing novel robustness metrics (Spearman’s correlation, Absolute/Relative Continuous Error) and conducting large-scale evaluation of leading MLLMs.

Result: The benchmark enables comprehensive analysis of model-centric and task-centric robustness, investigation of preprocessing strategies (padding, super-resolution), and exploration of fine-tuning for stability enhancement.

Conclusion: Res-Bench provides a systematic framework for evaluating resolution robustness in MLLMs, addressing a critical gap in current evaluation paradigms and enabling better understanding of model performance stability across varying input resolutions.

Abstract: Multimodal Large Language Models (MLLMs) increasingly support dynamic image resolutions. However, current evaluation paradigms primarily assess semantic performance, overlooking the critical question of resolution robustness - whether performance remains stable across varying input resolutions. To address this gap, we introduce \textbf{Res-Bench}, a comprehensive benchmark comprising 14,400 samples across 12 resolution levels and six core capability dimensions. We designed a novel evaluation framework that goes beyond traditional accuracy metrics to capture performance stability. This framework introduces multiple robustness metrics: Spearman’s correlation for assessing resolution-performance trends, and Absolute/Relative Continuous Error (ACE/RCE) for measuring performance volatility. Using these metrics, we conducted a large-scale evaluation of leading MLLMs. Our analysis encompasses: (1) model-centric and task-centric robustness examination, (2) investigation of preprocessing strategies including padding and super-resolution, and (3) exploration of fine-tuning for stability enhancement.

Method: Systematic categorization of studies into vision-only and vision-language FMs based on architectural foundations, training strategies, and downstream clinical tasks. Quantitative meta-analysis of temporal trends in dataset utilization and application domains.

Result: The review provides a structured analysis of FM research in medical imaging, identifying trends in dataset usage and application domains, and critically discussing persistent challenges and emerging solutions.

Conclusion: Key future research directions are identified to enhance robustness, explainability, and clinical integration of FMs, accelerating their translation into real-world medical practice.

Abstract: Recent advancements in artificial intelligence (AI), particularly foundation models (FMs), have revolutionized medical image analysis, demonstrating strong zero- and few-shot performance across diverse medical imaging tasks, from segmentation to report generation. Unlike traditional task-specific AI models, FMs leverage large corpora of labeled and unlabeled multimodal datasets to learn generalized representations that can be adapted to various downstream clinical applications with minimal fine-tuning. However, despite the rapid proliferation of FM research in medical imaging, the field remains fragmented, lacking a unified synthesis that systematically maps the evolution of architectures, training paradigms, and clinical applications across modalities. To address this gap, this review article provides a comprehensive and structured analysis of FMs in medical image analysis. We systematically categorize studies into vision-only and vision-language FMs based on their architectural foundations, training strategies, and downstream clinical tasks. Additionally, a quantitative meta-analysis of the studies was conducted to characterize temporal trends in dataset utilization and application domains. We also critically discuss persistent challenges, including domain adaptation, efficient fine-tuning, computational constraints, and interpretability along with emerging solutions such as federated learning, knowledge distillation, and advanced prompting. Finally, we identify key future research directions aimed at enhancing the robustness, explainability, and clinical integration of FMs, thereby accelerating their translation into real-world medical practice.

Method: Proposes Di-Bregman framework that formulates diffusion distillation as Bregman divergence-based density-ratio matching, providing a convex-analytic view that connects existing objectives.

Result: Experiments on CIFAR-10 and text-to-image generation show improved one-step FID over reverse-KL distillation while maintaining high visual fidelity compared to teacher models.

Conclusion: Bregman density-ratio matching provides a practical and theoretically-grounded approach for efficient one-step diffusion generation.

Abstract: Diffusion and flow models achieve high generative quality but remain computationally expensive due to slow multi-step sampling. Distillation methods accelerate them by training fast student generators, yet most existing objectives lack a unified theoretical foundation. In this work, we propose Di-Bregman, a compact framework that formulates diffusion distillation as Bregman divergence-based density-ratio matching. This convex-analytic view connects several existing objectives through a common lens. Experiments on CIFAR-10 and text-to-image generation demonstrate that Di-Bregman achieves improved one-step FID over reverse-KL distillation and maintains high visual fidelity compared to the teacher model. Our results highlight Bregman density-ratio matching as a practical and theoretically-grounded route toward efficient one-step diffusion generation.

Method: CARE uses Sequence-Image Contrastive Alignment (SICA) to jointly optimize representation learning and classification. It integrates time-aware sequence encoding with spatially-informed image representations, and employs a joint contrastive-classification objective for end-to-end learning.

Result: CARE achieves state-of-the-art performance on three CASAS datasets: 89.8% on Milan, 88.9% on Cairo, and 73.3% on Kyoto7. It also demonstrates robustness to sensor malfunctions and layout variability.

Conclusion: The CARE framework effectively leverages complementary strengths of sequence- and image-based representations through contrastive alignment, enabling reliable ADL recognition in smart homes with improved performance and robustness.

Abstract: The recognition of Activities of Daily Living (ADLs) from event-triggered ambient sensors is an essential task in Ambient Assisted Living, yet existing methods remain constrained by representation-level limitations. Sequence-based approaches preserve temporal order of sensor activations but are sensitive to noise and lack spatial awareness, while image-based approaches capture global patterns and implicit spatial correlations but compress fine-grained temporal dynamics and distort sensor layouts. Naive fusion (e.g., feature concatenation) fail to enforce alignment between sequence- and image-based representation views, underutilizing their complementary strengths. We propose Contrastive Alignment for ADL Recognition from Event-Triggered Sensor Streams (CARE), an end-to-end framework that jointly optimizes representation learning via Sequence-Image Contrastive Alignment (SICA) and classification via cross-entropy, ensuring both cross-representation alignment and task-specific discriminability. CARE integrates (i) time-aware, noise-resilient sequence encoding with (ii) spatially-informed and frequency-sensitive image representations, and employs (iii) a joint contrastive-classification objective for end-to-end learning of aligned and discriminative embeddings. Evaluated on three CASAS datasets, CARE achieves state-of-the-art performance (89.8% on Milan, 88.9% on Cairo, and 73.3% on Kyoto7) and demonstrates robustness to sensor malfunctions and layout variability, highlighting its potential for reliable ADL recognition in smart homes.

Method: Uses LMMs to predict steps from restricted frame sets, then develops BaGLM which injects knowledge of past frames using Bayesian filtering with step transitions modeled via dependency matrices from LLMs and step progress estimation.

Result: The online strategy without task-specific tuning outperforms offline training-based models. BaGLM shows superior performance over state-of-the-art training-based offline methods on three datasets.

Conclusion: BaGLM enables effective online VSG without training requirements, demonstrating the power of LMMs and Bayesian filtering for step grounding tasks.

Abstract: Given a task and a set of steps composing it, Video Step Grounding (VSG) aims to detect which steps are performed in a video. Standard approaches for this task require a labeled training set (e.g., with step-level annotations or narrations), which may be costly to collect. Moreover, they process the full video offline, limiting their applications for scenarios requiring online decisions. Thus, in this work, we explore how to perform VSG online and without training. We achieve this by exploiting the zero-shot capabilities of recent Large Multimodal Models (LMMs). In particular, we use LMMs to predict the step associated with a restricted set of frames, without access to the whole video. We show that this online strategy without task-specific tuning outperforms offline and training-based models. Motivated by this finding, we develop Bayesian Grounding with Large Multimodal Models (BaGLM), further injecting knowledge of past frames into the LMM-based predictions. BaGLM exploits Bayesian filtering principles, modeling step transitions via (i) a dependency matrix extracted through large language models and (ii) an estimation of step progress. Experiments on three datasets show superior performance of BaGLM over state-of-the-art training-based offline methods.

Method: Extracted features from three benchmarks (Charades-STA, ActivityNet-Captions, YouCookII) using video encoders based on CNNs, temporal reasoning, and transformers, then evaluated their impact on a classical architecture.

Result: Results show significant performance differences when changing video encoders, revealing clear patterns and errors from using certain features, indicating potential feature complementarity.

Conclusion: Different video feature representations significantly impact temporal video grounding performance, suggesting that exploring diverse feature types could improve model robustness and reveal complementary strengths.

Abstract: Temporal video grounding is a fundamental task in computer vision, aiming to localize a natural language query in a long, untrimmed video. It has a key role in the scientific community, in part due to the large amount of video generated every day. Although we find extensive work in this task, we note that research remains focused on a small selection of video representations, which may lead to architectural overfitting in the long run. To address this issue, we propose an empirical study to investigate the impact of different video features on a classical architecture. We extract features for three well-known benchmarks, Charades-STA, ActivityNet-Captions and YouCookII, using video encoders based on CNNs, temporal reasoning and transformers. Our results show significant differences in the performance of our model by simply changing the video encoder, while also revealing clear patterns and errors derived from the use of certain features, ultimately indicating potential feature complementarity.

Method: Created a benchmark to measure generalization to lower resolution images, and trained iBOT on MillionAID dataset with remote sensing-specific modifications.

Result: No consistent improvements were found from specialized pretrained models over general-purpose baselines at ViT-B scale.

Conclusion: Specialized remote sensing foundation models may not be necessary when working at small scales, as general-purpose models perform equally well.

Abstract: Foundation models have advanced machine learning across various modalities, including images. Recently multiple teams trained foundation models specialized for remote sensing applications. This line of research is motivated by the distinct characteristics of remote sensing imagery, specific applications and types of robustness useful for satellite image analysis. In this work we systematically challenge the idea that specific foundation models are more useful than general-purpose vision foundation models, at least in the small scale. First, we design a simple benchmark that measures generalization of remote sensing models towards images with lower resolution for two downstream tasks. Second, we train iBOT, a self-supervised vision encoder, on MillionAID, an ImageNet-scale satellite imagery dataset, with several modifications specific to remote sensing. We show that none of those pretrained models bring consistent improvements upon general-purpose baselines at the ViT-B scale.

Method: Proposes What-Where Representation Re-Forming (W2R2) framework with disentangled representation learning: 2D features as semantic beacons for “What” identification and 3D features as spatial anchors for “Where” localization. Uses dual-objective loss with Alignment Loss for multimodal synergy and Pseudo-Label Loss to penalize 2D-dominant outputs.

Result: Experiments on ScanRefer and ScanQA show significant gains in localization accuracy and robustness, particularly in cluttered outdoor scenes.

Conclusion: W2R2 effectively addresses 2D semantic bias through disentangled representation learning and targeted shortcut suppression, enabling precise 3D grounding without architectural modifications.

Abstract: Multimodal 3D grounding has garnered considerable interest in Vision-Language Models (VLMs) \cite{yin2025spatial} for advancing spatial reasoning in complex environments. However, these models suffer from a severe “2D semantic bias” that arises from over-reliance on 2D image features for coarse localization, largely disregarding 3D geometric inputs and resulting in suboptimal fusion performance. In this paper, we propose a novel training framework called What-Where Representation Re-Forming (W2R2) to tackle this issue via disentangled representation learning and targeted shortcut suppression. Our approach fundamentally reshapes the model’s internal space by designating 2D features as semantic beacons for “What” identification and 3D features as spatial anchors for “Where” localization, enabling precise 3D grounding without modifying inference architecture. Key components include a dual-objective loss function with an Alignment Loss that supervises fused predictions using adapted cross-entropy for multimodal synergy, and a Pseudo-Label Loss that penalizes overly effective 2D-dominant pseudo-outputs via a margin-based mechanism. Experiments conducted on ScanRefer and ScanQA demonstrate the effectiveness of W2R2, with significant gains in localization accuracy and robustness, particularly in cluttered outdoor scenes.

Method: Uses conditional StyleGAN2-ADA and StyleGAN3 to generate high-resolution synthetic live fingerprints conditioned on finger identities, and CycleGANs to translate these into realistic spoof fingerprints simulating various presentation attack materials.

Result: Created two synthetic datasets (DB2 and DB3) with 1,500 fingerprint images each. StyleGAN3 achieved FID as low as 5 and TAR of 99.47% at 0.01% FAR. StyleGAN2-ADA achieved TAR of 98.67%. Quality metrics (NFIQ2, MINDTCT) confirm strong performance with no significant identity leakage.

Conclusion: The synthetic fingerprint generation approach successfully addresses privacy, cost, and accessibility concerns while producing high-quality fingerprints suitable for training and evaluation, with strong privacy-preserving properties confirmed through matching experiments.

Abstract: Large fingerprint datasets, while important for training and evaluation, are time-consuming and expensive to collect and require strict privacy measures. Researchers are exploring the use of synthetic fingerprint data to address these issues. This paper presents a novel approach for generating synthetic fingerprint images (both spoof and live), addressing concerns related to privacy, cost, and accessibility in biometric data collection. Our approach utilizes conditional StyleGAN2-ADA and StyleGAN3 architectures to produce high-resolution synthetic live fingerprints, conditioned on specific finger identities (thumb through little finger). Additionally, we employ CycleGANs to translate these into realistic spoof fingerprints, simulating a variety of presentation attack materials (e.g., EcoFlex, Play-Doh). These synthetic spoof fingerprints are crucial for developing robust spoof detection systems. Through these generative models, we created two synthetic datasets (DB2 and DB3), each containing 1,500 fingerprint images of all ten fingers with multiple impressions per finger, and including corresponding spoofs in eight material types. The results indicate robust performance: our StyleGAN3 model achieves a Fr'echet Inception Distance (FID) as low as 5, and the generated fingerprints achieve a True Accept Rate of 99.47% at a 0.01% False Accept Rate. The StyleGAN2-ADA model achieved a TAR of 98.67% at the same 0.01% FAR. We assess fingerprint quality using standard metrics (NFIQ2, MINDTCT), and notably, matching experiments confirm strong privacy preservation, with no significant evidence of identity leakage, confirming the strong privacy-preserving properties of our synthetic datasets.

Method: Used multi-center CT data from 999 patients across 12 public datasets with five DL models (3D Attention U-Net, ResUNet, VNet, ReconNet, SAM-Med3D), benchmarked against expert contours. Assessed segmentation reproducibility using 497 radiomic features and compared supervised vs semi-supervised learning across multiple dimensionality reduction strategies and classifiers. Six physicians qualitatively evaluated masks across clinical domains.

Result: VNet achieved best performance (Dice = 0.83, IoU = 0.71), radiomic stability (mean correlation = 0.76, ICC = 0.65), and predictive accuracy under SSL (accuracy = 0.88, F1 = 0.83). SSL consistently outperformed SL across models. Radiologists favored VNet for peritumoral representation and preferred AI-generated masks for refinement rather than replacement.

Conclusion: Integrating VNet with SSL yields accurate, reproducible, and clinically trusted CT-based lung cancer prognosis, highlighting a feasible path toward physician-centered AI translation with clinician-in-the-loop workflows.

Abstract: Lung cancer remains the leading cause of cancer mortality, with CT imaging central to screening, prognosis, and treatment. Manual segmentation is variable and time-intensive, while deep learning (DL) offers automation but faces barriers to clinical adoption. Guided by the Knowledge-to-Action framework, this study develops a clinician-in-the-loop DL pipeline to enhance reproducibility, prognostic accuracy, and clinical trust. Multi-center CT data from 999 patients across 12 public datasets were analyzed using five DL models (3D Attention U-Net, ResUNet, VNet, ReconNet, SAM-Med3D), benchmarked against expert contours on whole and click-point cropped images. Segmentation reproducibility was assessed using 497 PySERA-extracted radiomic features via Spearman correlation, ICC, Wilcoxon tests, and MANOVA, while prognostic modeling compared supervised (SL) and semi-supervised learning (SSL) across 38 dimensionality reduction strategies and 24 classifiers. Six physicians qualitatively evaluated masks across seven domains, including clinical meaningfulness, boundary quality, prognostic value, trust, and workflow integration. VNet achieved the best performance (Dice = 0.83, IoU = 0.71), radiomic stability (mean correlation = 0.76, ICC = 0.65), and predictive accuracy under SSL (accuracy = 0.88, F1 = 0.83). SSL consistently outperformed SL across models. Radiologists favored VNet for peritumoral representation and smoother boundaries, preferring AI-generated initial masks for refinement rather than replacement. These results demonstrate that integrating VNet with SSL yields accurate, reproducible, and clinically trusted CT-based lung cancer prognosis, highlighting a feasible path toward physician-centered AI translation.

Method: Uses entropy of model outputs as signal to guide micro-exploration/exploitation. Adapts LMM’s value cache during inference via a small trainable controller with entropy-based objective, requiring no dataset supervision or RL.

Result: Significant improvements over base models across video reasoning datasets, narrowing gap with RL-trained models to within 0.6% average accuracy. Reduces output tokens by 58.6% compared to RL models.

Conclusion: V-Reason enables efficient video reasoning by optimizing inference-time behavior using entropy signals, achieving competitive performance without costly training while offering substantial efficiency benefits.

Abstract: Video reasoning using Large Multimodal Models (LMMs) relies on costly reinforcement learning (RL) and verbose chain-of-thought, resulting in substantial computational overhead during both training and inference. Moreover, the mechanisms that control the thinking process in these reasoning models are very limited. In this paper, using entropy of the model’s output as a signal, we discover that the high-quality models go through a series of micro-explorations and micro-exploitations which keep the reasoning process grounded (i.e., avoid excessive randomness while the model is exploring or thinking through an answer). We further observe that once this “thinking” process is over, more accurate models demonstrate a better convergence by reducing the entropy significantly via a final exploitation phase (i.e., a more certain convergence towards a solution trajectory). We then use these novel, theoretically-grounded insights to tune the model’s behavior directly at inference, without using any RL or supervised fine-tuning. Specifically, during inference, our proposed approach called V-Reason (Video-Reason) adapts the value cache of the LMM via a few optimization steps on a small, trainable controller using an entropy-based objective, i.e., no supervision from any dataset or RL is necessary. This tuning improves the model’s micro-exploration and exploitation behavior during inference. Our experiments show that our proposed method achieves significant improvements over the base instruction-tuned models across several video reasoning datasets, narrowing the gap with RL-trained models to within 0.6% average accuracy without any training, while offering massive efficiency benefits: output tokens are reduced by 58.6% compared to the RL model.

Method: Compare feature versatility of Hiera encoder vs SAM2 using lightweight trainable neck to probe adaptability of frozen features, quantify information-theoretic cost of specialization, and conduct cross-neck analysis on SAM2.

Result: SAM2’s specialization is highly effective for spatially-related tasks like depth estimation but underperforms Hiera on conceptually distant tasks (pose estimation, image captioning), showing measurable loss of broader semantic information. Cross-neck analysis reveals each adaptation level creates further representational bottlenecks.

Conclusion: The analysis illuminates trade-offs in feature universality, providing quantitative foundation for designing efficient feature coding and adaptation strategies for diverse downstream applications.

Abstract: The trade-off between general-purpose foundation vision models and their specialized counterparts is critical for efficient feature coding design and is not yet fully understood. We investigate this trade-off by comparing the feature versatility of the general-purpose Hiera encoder against the segmentation-specialized Segment Anything Model 2 (SAM2). Using a lightweight, trainable neck to probe the adaptability of their frozen features, we quantify the information-theoretic cost of specialization. Our results reveal that while SAM2’s specialization is highly effective for spatially-related tasks like depth estimation, it comes at a cost. The specialized SAM2 encoder underperforms its generalist predecessor, Hiera, on conceptually distant tasks such as pose estimation and image captioning, demonstrating a measurable loss of broader semantic information. A novel cross-neck analysis on SAM2 reveals that each level of adaptation creates a further representational bottleneck. Our analysis illuminates these trade-offs in feature universality, providing a quantitative foundation for designing efficient feature coding and adaptation strategies for diverse downstream applications.

Method: Proposed ProDAT with density-aware tail-drop mechanism that adaptively decodes latent features and coordinates based on significance using density guidance.

Result: Achieved over 28.6% BD-rate improvement for PSNR-D2 on SemanticKITTI and over 18.15% on ShapeNet compared to state-of-the-art methods.

Conclusion: ProDAT successfully bridges the progressive coding gap while maintaining superior compression performance across multiple datasets.

Abstract: Three-dimensional (3D) point clouds are becoming increasingly vital in applications such as autonomous driving, augmented reality, and immersive communication, demanding real-time processing and low latency. However, their large data volumes and bandwidth constraints hinder the deployment of high-quality services in resource-limited environments. Progres- sive coding, which allows for decoding at varying levels of detail, provides an alternative by allowing initial partial decoding with subsequent refinement. Although recent learning-based point cloud geometry coding methods have achieved notable success, their fixed latent representation does not support progressive decoding. To bridge this gap, we propose ProDAT, a novel density-aware tail-drop mechanism for progressive point cloud coding. By leveraging density information as a guidance signal, latent features and coordinates are decoded adaptively based on their significance, therefore achieving progressive decoding at multiple bitrates using one single model. Experimental results on benchmark datasets show that the proposed ProDAT not only enables progressive coding but also achieves superior coding efficiency compared to state-of-the-art learning-based coding techniques, with over 28.6% BD-rate improvement for PSNR- D2 on SemanticKITTI and over 18.15% for ShapeNet

Method: Combines frequency-domain filtering (Freq-Filter) to suppress redundant spectral features with cross-attention-based fusion (MCAF) to improve intermodal feature sharing.

Result: Outperforms traditional concatenation fusion, achieving +13.9% mAP@50 on VEDAI and +1.1% on LLVIP datasets.

Conclusion: FMCAF shows potential as a flexible foundation for robust multimodal fusion in future detection pipelines, demonstrating generalizability across different multimodal challenges.

Abstract: Multimodal object detection improves robustness in chal- lenging conditions by leveraging complementary cues from multiple sensor modalities. We introduce Filtered Multi- Modal Cross Attention Fusion (FMCAF), a preprocess- ing architecture designed to enhance the fusion of RGB and infrared (IR) inputs. FMCAF combines a frequency- domain filtering block (Freq-Filter) to suppress redun- dant spectral features with a cross-attention-based fusion module (MCAF) to improve intermodal feature sharing. Unlike approaches tailored to specific datasets, FMCAF aims for generalizability, improving performance across different multimodal challenges without requiring dataset- specific tuning. On LLVIP (low-light pedestrian detec- tion) and VEDAI (aerial vehicle detection), FMCAF outper- forms traditional fusion (concatenation), achieving +13.9% mAP@50 on VEDAI and +1.1% on LLVIP. These results support the potential of FMCAF as a flexible foundation for robust multimodal fusion in future detection pipelines.

Method: Leverages segmentation and normal prediction models to extract planar priors, uses structured representations for planar Gaussian coordinates, introduces Dynamic Gaussian Re-classifier to handle high-gradient regions, and refines mesh layouts using optimized planar priors.

Result: Significantly improves geometric accuracy of extracted meshes without sacrificing rendering quality, produces clean mesh connectivity with reduced vertices/faces, and enables object decoupling and manipulation on supportive planes.

Conclusion: GSPlane effectively addresses planar reconstruction limitations in Gaussian Splatting through structured planar priors, enhancing both geometric accuracy and mesh topology while maintaining rendering performance.

Abstract: Planes are fundamental primitives of 3D sences, especially in man-made environments such as indoor spaces and urban streets. Representing these planes in a structured and parameterized format facilitates scene editing and physical simulations in downstream applications. Recently, Gaussian Splatting (GS) has demonstrated remarkable effectiveness in the Novel View Synthesis task, with extensions showing great potential in accurate surface reconstruction. However, even state-of-the-art GS representations often struggle to reconstruct planar regions with sufficient smoothness and precision. To address this issue, we propose GSPlane, which recovers accurate geometry and produces clean and well-structured mesh connectivity for plane regions in the reconstructed scene. By leveraging off-the-shelf segmentation and normal prediction models, GSPlane extracts robust planar priors to establish structured representations for planar Gaussian coordinates, which help guide the training process by enforcing geometric consistency. To further enhance training robustness, a Dynamic Gaussian Re-classifier is introduced to adaptively reclassify planar Gaussians with persistently high gradients as non-planar, ensuring more reliable optimization. Furthermore, we utilize the optimized planar priors to refine the mesh layouts, significantly improving topological structure while reducing the number of vertices and faces. We also explore applications of the structured planar representation, which enable decoupling and flexible manipulation of objects on supportive planes. Extensive experiments demonstrate that, with no sacrifice in rendering quality, the introduction of planar priors significantly improves the geometric accuracy of the extracted meshes across various baselines.

Method: Introduces a latent refinement pipeline to recover spatial details lost during VAE encoding, incorporating generative priors. The refined latent condition dynamically interacts with the noisy latent in the diffusion process. The approach is plug-and-play and integrates into existing diffusion networks.

Result: Extensive experiments demonstrate significant fidelity improvements in PTDB methods while maintaining realism and aesthetics.

Conclusion: The proposed optimization strategy effectively addresses fidelity loss in diffusion-based methods by enabling better conditional latent modeling and bidirectional interaction, providing more effective control in low-level vision tasks.

Abstract: Diffusion-based methods, leveraging pre-trained large models like Stable Diffusion via ControlNet, have achieved remarkable performance in several low-level vision tasks. However, Pre-Trained Diffusion-Based (PTDB) methods often sacrifice content fidelity to attain higher perceptual realism. This issue is exacerbated in low-light scenarios, where severely degraded information caused by the darkness limits effective control. We identify two primary causes of fidelity loss: the absence of suitable conditional latent modeling and the lack of bidirectional interaction between the conditional latent and noisy latent in the diffusion process. To address this, we propose a novel optimization strategy for conditioning in pre-trained diffusion models, enhancing fidelity while preserving realism and aesthetics. Our method introduces a mechanism to recover spatial details lost during VAE encoding, i.e., a latent refinement pipeline incorporating generative priors. Additionally, the refined latent condition interacts dynamically with the noisy latent, leading to improved restoration performance. Our approach is plug-and-play, seamlessly integrating into existing diffusion networks to provide more effective control. Extensive experiments demonstrate significant fidelity improvements in PTDB methods.

Method: Optimizes LED spectral profiles using spectral modulation (not intensity modulation) that considers human visual sensitivity, camera sensor characteristics, and LED capabilities to produce white light while embedding detectable watermarks.

Result: Successfully embeds 128 bits within 10-second video clips at standard frame rates (30-60 fps), providing sufficient capacity for essential metadata while maintaining visual imperceptibility.

Conclusion: The approach enables practical invisible watermarking through environmental lighting, supporting privacy protection and content verification applications with modest but adequate data transfer rates.

Abstract: This paper introduces a method for using LED-based environmental lighting to produce visually imperceptible watermarks for consumer cameras. Our approach optimizes an LED light source’s spectral profile to be minimally visible to the human eye while remaining highly detectable by typical consumer cameras. The method jointly considers the human visual system’s sensitivity to visible spectra, modern consumer camera sensors’ spectral sensitivity, and narrowband LEDs’ ability to generate broadband spectra perceived as “white light” (specifically, D65 illumination). To ensure imperceptibility, we employ spectral modulation rather than intensity modulation. Unlike conventional visible light communication, our approach enables watermark extraction at standard low frame rates (30-60 fps). While the information transfer rate is modest-embedding 128 bits within a 10-second video clip-this capacity is sufficient for essential metadata supporting privacy protection and content verification.

Method: GOOD guides diffusion sampling trajectories using ID classifiers with dual-level guidance: image-level guidance reduces input likelihood via gradient of log partition, and feature-level guidance promotes sampling in feature-sparse regions using k-NN distance in classifier’s latent space.

Result: Training with GOOD-generated samples notably enhances OOD detection performance, as demonstrated through thorough quantitative and qualitative analyses.

Conclusion: GOOD enables more controllable and diverse OOD sample generation through its dual-guidance design and unified OOD scoring, substantially improving OOD detection capabilities.

Abstract: Recent advancements have explored text-to-image diffusion models for synthesizing out-of-distribution (OOD) samples, substantially enhancing the performance of OOD detection. However, existing approaches typically rely on perturbing text-conditioned embeddings, resulting in semantic instability and insufficient shift diversity, which limit generalization to realistic OOD. To address these challenges, we propose GOOD, a novel and flexible framework that directly guides diffusion sampling trajectories towards OOD regions using off-the-shelf in-distribution (ID) classifiers. GOOD incorporates dual-level guidance: (1) Image-level guidance based on the gradient of log partition to reduce input likelihood, drives samples toward low-density regions in pixel space. (2) Feature-level guidance, derived from k-NN distance in the classifier’s latent space, promotes sampling in feature-sparse regions. Hence, this dual-guidance design enables more controllable and diverse OOD sample generation. Additionally, we introduce a unified OOD score that adaptively combines image and feature discrepancies, enhancing detection robustness. We perform thorough quantitative and qualitative analyses to evaluate the effectiveness of GOOD, demonstrating that training with samples generated by GOOD can notably enhance OOD detection performance.

Method: The framework uses a Kinematic-Aware VAE to encode geometry, joint angles, and part segmentation into a structured latent space, then employs two conditional diffusion models for pose/joint parameter regression and kinematic-aware latent code generation, followed by iterative optimization with Chamfer-distance minimization.

Result: Experimental results on synthetic, semi-synthetic, and real-world datasets show the approach effectively reconstructs articulated objects and estimates their kinematic properties.

Conclusion: KineDiff3D provides an effective solution for category-level articulated object reconstruction and pose estimation from single-view inputs while preserving articulation constraints.

Abstract: Articulated objects, such as laptops and drawers, exhibit significant challenges for 3D reconstruction and pose estimation due to their multi-part geometries and variable joint configurations, which introduce structural diversity across different states. To address these challenges, we propose KineDiff3D: Kinematic-Aware Diffusion for Category-Level Articulated Object Shape Reconstruction and Generation, a unified framework for reconstructing diverse articulated instances and pose estimation from single view input. Specifically, we first encode complete geometry (SDFs), joint angles, and part segmentation into a structured latent space via a novel Kinematic-Aware VAE (KA-VAE). In addition, we employ two conditional diffusion models: one for regressing global pose (SE(3)) and joint parameters, and another for generating the kinematic-aware latent code from partial observations. Finally, we produce an iterative optimization module that bidirectionally refines reconstruction accuracy and kinematic parameters via Chamfer-distance minimization while preserving articulation constraints. Experimental results on synthetic, semi-synthetic, and real-world datasets demonstrate the effectiveness of our approach in accurately reconstructing articulated objects and estimating their kinematic properties.

Method: Two-stage framework: 1) Supervised fine-tuning using depth and surface normal maps as geometric priors combined with RGB images, 2) Reinforcement learning with group length reward to promote compact parametric modeling sequences.

Result: State-of-the-art performance on DeepCAD and Fusion360 datasets, outperforming existing methods in code validity, geometric accuracy, and modeling conciseness.

Conclusion: GACO-CAD effectively addresses spatial reasoning limitations in MLLMs for CAD generation, achieving both high geometric fidelity and modeling efficiency through complementary geometric priors and compact sequence optimization.

Abstract: Generating editable, parametric CAD models from a single image holds great potential to lower the barriers of industrial concept design. However, current multi-modal large language models (MLLMs) still struggle with accurately inferring 3D geometry from 2D images due to limited spatial reasoning capabilities. We address this limitation by introducing GACO-CAD, a novel two-stage post-training framework. It is designed to achieve a joint objective: simultaneously improving the geometric accuracy of the generated CAD models and encouraging the use of more concise modeling procedures. First, during supervised fine-tuning, we leverage depth and surface normal maps as dense geometric priors, combining them with the RGB image to form a multi-channel input. In the context of single-view reconstruction, these priors provide complementary spatial cues that help the MLLM more reliably recover 3D geometry from 2D observations. Second, during reinforcement learning, we introduce a group length reward that, while preserving high geometric fidelity, promotes the generation of more compact and less redundant parametric modeling sequences. A simple dynamic weighting strategy is adopted to stabilize training. Experiments on the DeepCAD and Fusion360 datasets show that GACO-CAD achieves state-of-the-art performance under the same MLLM backbone, consistently outperforming existing methods in terms of code validity, geometric accuracy, and modeling conciseness.

Method: Studied transferability of state-of-the-art adversarial attacks against different preprocessing techniques, analyzed impact of face detection models and interpolation methods, and proposed preprocessing-invariant method using input transformations.

Result: Face detection model choice degrades attack success rate by up to 78%, while interpolation methods have minimal impact. Preprocessing degrades attack strength even in whitebox settings. The proposed preprocessing-invariant method improves transferability by up to 27%.

Conclusion: Face preprocessing is crucial in FR systems and must be considered to improve adversarial generalization of facial adversarial examples.

Abstract: Face Recognition (FR) models have been shown to be vulnerable to adversarial examples that subtly alter benign facial images, exposing blind spots in these systems, as well as protecting user privacy. End-to-end FR systems first obtain preprocessed faces from diverse facial imagery prior to computing the similarity of the deep feature embeddings. Whilst face preprocessing is a critical component of FR systems, and hence adversarial attacks against them, we observe that this preprocessing is often overlooked in blackbox settings. Our study seeks to investigate the transferability of several out-of-the-box state-of-the-art adversarial attacks against FR when applied against different preprocessing techniques used in a blackbox setting. We observe that the choice of face detection model can degrade the attack success rate by up to 78%, whereas choice of interpolation method during downsampling has relatively minimal impacts. Furthermore, we find that the requirement for facial preprocessing even degrades attack strength in a whitebox setting, due to the unintended interaction of produced noise vectors against face detection models. Based on these findings, we propose a preprocessing-invariant method using input transformations that improves the transferability of the studied attacks by up to 27%. Our findings highlight the importance of preprocessing in FR systems, and the need for its consideration towards improving the adversarial generalisation of facial adversarial examples.

Method: Two-stage approach: structure generation for global semantic scaffolding followed by detail reconstruction for completing remaining tokens, plus Frequency-Weighted Token Selection to allocate more computation to detail tokens based on high frequency energy.

Result: Achieves 3.72x speedup on MAR-H while maintaining comparable quality (FID: 1.59, IS: 304.4 vs. original 1.59, 299.1), outperforming existing acceleration methods across various model scales and generation tasks.

Conclusion: GtR effectively accelerates masked autoregressive models through hierarchical generation strategy and intelligent token selection, demonstrating significant speed improvements without quality degradation.

Abstract: Masked Autoregressive (MAR) models promise better efficiency in visual generation than autoregressive (AR) models for the ability of parallel generation, yet their acceleration potential remains constrained by the modeling complexity of spatially correlated visual tokens in a single step. To address this limitation, we introduce Generation then Reconstruction (GtR), a training-free hierarchical sampling strategy that decomposes generation into two stages: structure generation establishing global semantic scaffolding, followed by detail reconstruction efficiently completing remaining tokens. Assuming that it is more difficult to create an image from scratch than to complement images based on a basic image framework, GtR is designed to achieve acceleration by computing the reconstruction stage quickly while maintaining the generation quality by computing the generation stage slowly. Moreover, observing that tokens on the details of an image often carry more semantic information than tokens in the salient regions, we further propose Frequency-Weighted Token Selection (FTS) to offer more computation budget to tokens on image details, which are localized based on the energy of high frequency information. Extensive experiments on ImageNet class-conditional and text-to-image generation demonstrate 3.72x speedup on MAR-H while maintaining comparable quality (e.g., FID: 1.59, IS: 304.4 vs. original 1.59, 299.1), substantially outperforming existing acceleration methods across various model scales and generation tasks. Our codes will be released in https://github.com/feihongyan1/GtR.

Method: Created OoD benchmarks simulating various distribution shift scenarios using DYB-PlanktonNet dataset, and systematically evaluated twenty-two OoD detection methods.

Result: ViM method significantly outperforms other approaches, particularly excelling in Far-OoD scenarios with substantial improvements in key metrics.

Conclusion: This comprehensive evaluation provides reliable reference for algorithm selection in automated plankton recognition and lays foundation for future research in plankton OoD detection.

Abstract: Automated plankton recognition models face significant challenges during real-world deployment due to distribution shifts (Out-of-Distribution, OoD) between training and test data. This stems from plankton’s complex morphologies, vast species diversity, and the continuous discovery of novel species, which leads to unpredictable errors during inference. Despite rapid advancements in OoD detection methods in recent years, the field of plankton recognition still lacks a systematic integration of the latest computer vision developments and a unified benchmark for large-scale evaluation. To address this, this paper meticulously designed a series of OoD benchmarks simulating various distribution shift scenarios based on the DYB-PlanktonNet dataset \cite{875n-f104-21}, and systematically evaluated twenty-two OoD detection methods. Extensive experimental results demonstrate that the ViM \cite{wang2022vim} method significantly outperforms other approaches in our constructed benchmarks, particularly excelling in Far-OoD scenarios with substantial improvements in key metrics. This comprehensive evaluation not only provides a reliable reference for algorithm selection in automated plankton recognition but also lays a solid foundation for future research in plankton OoD detection. To our knowledge, this study marks the first large-scale, systematic evaluation and analysis of Out-of-Distribution data detection methods in plankton recognition. Code is available at https://github.com/BlackJack0083/PlanktonOoD.

Method: Proposes depth order loss with contact regularization for pose tracking, learns a PCA basis for hand-induced facial deformations from a face-hand interaction dataset, and incorporates contact loss inspired by physics-based simulation to reduce interpenetration artifacts.

Result: Evaluated on RGB(D) videos from iPhone and synthetic datasets, showing better appearance and more accurate deforming geometry of the face compared to state-of-the-art surface reconstruction methods.

Conclusion: The method successfully captures detailed head avatars with physically plausible hand-induced facial deformations, overcoming challenges in pose tracking and deformation learning from monocular videos.

Abstract: Photorealistic 3D head avatars are vital for telepresence, gaming, and VR. However, most methods focus solely on facial regions, ignoring natural hand-face interactions, such as a hand resting on the chin or fingers gently touching the cheek, which convey cognitive states like pondering. In this work, we present a novel framework that jointly learns detailed head avatars and the non-rigid deformations induced by hand-face interactions. There are two principal challenges in this task. First, naively tracking hand and face separately fails to capture their relative poses. To overcome this, we propose to combine depth order loss with contact regularization during pose tracking, ensuring correct spatial relationships between the face and hand. Second, no publicly available priors exist for hand-induced deformations, making them non-trivial to learn from monocular videos. To address this, we learn a PCA basis specific to hand-induced facial deformations from a face-hand interaction dataset. This reduces the problem to estimating a compact set of PCA parameters rather than a full spatial deformation field. Furthermore, inspired by physics-based simulation, we incorporate a contact loss that provides additional supervision, significantly reducing interpenetration artifacts and enhancing the physical plausibility of the results. We evaluate our approach on RGB(D) videos captured by an iPhone. Additionally, to better evaluate the reconstructed geometry, we construct a synthetic dataset of avatars with various types of hand interactions. We show that our method can capture better appearance and more accurate deforming geometry of the face than SOTA surface reconstruction methods.

Method: Uses hyperbolic representation learning with GPT-guided diffusion for domain augmentation, Tangent CutMix for curvature-aware interpolation in tangent space, and a unified loss combining penalized Busemann alignment, hybrid hyperbolic contrastive regularization, and adaptive outlier repulsion. Includes learnable curvature parameter.

Result: Achieves state-of-the-art results on PACS, Office-Home, and DomainNet datasets, consistently outperforming existing Euclidean and hyperbolic (DG)-GCD baselines.

Conclusion: HIDISC provides an efficient and effective framework for DG-GCD that avoids computational costs of episodic training while achieving superior generalization across domains and categories through hyperbolic geometry and novel augmentation techniques.

Abstract: Generalized Category Discovery (GCD) aims to classify test-time samples into either seen categories** – available during training – or novel ones, without relying on label supervision. Most existing GCD methods assume simultaneous access to labeled and unlabeled data during training and arising from the same domain, limiting applicability in open-world scenarios involving distribution shifts. Domain Generalization with GCD (DG-GCD) lifts this constraint by requiring models to generalize to unseen domains containing novel categories, without accessing targetdomain data during training. The only prior DG-GCD method, DG2CD-Net, relies on episodic training with multiple synthetic domains and task vector aggregation, incurring high computational cost and error accumulation. We propose HIDISC, a hyperbolic representation learning framework that achieves domain and category-level generalization without episodic simulation. To expose the model to minimal but diverse domain variations, we augment the source domain using GPT-guided diffusion, avoiding overfitting while maintaining efficiency. To structure the representation space, we introduce Tangent CutMix, a curvature-aware interpolation that synthesizes pseudo-novel samples in tangent space, preserving manifold consistency. A unified loss – combining penalized Busemann alignment, hybrid hyperbolic contrastive regularization, and adaptive outlier repulsion – **facilitates compact, semantically structured embeddings. A learnable curvature parameter further adapts the geometry to dataset complexity. HIDISC achieves state-of-the-art results on PACS , Office-Home , and DomainNet, consistently outperforming the existing Euclidean and hyperbolic (DG)-GCD baselines.

Method: Systematic analysis revealed a three-stage cross-modal interaction: shallow layers recognize task intent, middle layers fuse cross-modal information, and deep layers focus on linguistic refinement. Based on this, VisiPruner prunes vision tokens at appropriate stages without training.

Result: VisiPruner reduces up to 99% of vision-related attention computations and 53.9% of FLOPs on LLaVA-v1.5 7B, outperforming existing token pruning methods and generalizing across diverse MLLMs.

Conclusion: The insights provide actionable guidelines for training efficient MLLMs by aligning model architecture with intrinsic layer-wise processing dynamics, and VisiPruner offers an effective training-free pruning solution.

Abstract: Multimodal Large Language Models (MLLMs) have achieved strong performance across vision-language tasks, but suffer from significant computational overhead due to the quadratic growth of attention computations with the number of multimodal tokens. Though efforts have been made to prune tokens in MLLMs, \textit{they lack a fundamental understanding of how MLLMs process and fuse multimodal information.} Through systematic analysis, we uncover a \textbf{three-stage} cross-modal interaction process: (1) Shallow layers recognize task intent, with visual tokens acting as passive attention sinks; (2) Cross-modal fusion occurs abruptly in middle layers, driven by a few critical visual tokens; (3) Deep layers discard vision tokens, focusing solely on linguistic refinement. Based on these findings, we propose \emph{VisiPruner}, a training-free pruning framework that reduces up to 99% of vision-related attention computations and 53.9% of FLOPs on LLaVA-v1.5 7B. It significantly outperforms existing token pruning methods and generalizes across diverse MLLMs. Beyond pruning, our insights further provide actionable guidelines for training efficient MLLMs by aligning model architecture with its intrinsic layer-wise processing dynamics. Our code is available at: https://github.com/EIT-NLP/VisiPruner.

Method: Hierarchical approach that first selects task-relevant visual tokens based on prompt guidance, then supplements with diversity tokens to preserve broader context.

Result: Achieves performance matching or surpassing state-of-the-art with minimal accuracy loss even when pruning up to 90% of tokens, with significant reductions in GPU memory and inference latency.

Conclusion: The proposed prompt-aware token pruning method effectively balances task relevance and information diversity, enabling efficient VLM inference without compromising performance.

Abstract: As the capabilities of Vision-Language Models (VLMs) advance, they can process increasingly large inputs, which, unlike in LLMs, generates significant visual token redundancy and leads to prohibitive inference costs. While many methods aim to reduce these costs by pruning visual tokens, existing approaches, whether based on attention or diversity, typically neglect the guidance of the text prompt and thus fail to prioritize task relevance. In this work, we propose a novel, zero-shot method that reframes the problem by introducing a prompt-aware perspective, explicitly modeling visual token pruning as a balance between task relevance and information diversity. Our hierarchical approach first selects a core set of task-relevant visual tokens and then supplements them with diversity tokens to preserve broader context. Experiments across multiple models and benchmarks show that our method achieves performance that matches or surpasses the state-of-the-art with only minimal accuracy loss, even when pruning up to 90% of the tokens. Furthermore, these gains are accompanied by significant reductions in GPU memory footprint and inference latency.

Method: Used color-channel analysis for rough land/water segmentation, then fine-tuned SAM’s mask decoder to recognize riverbank erosion patterns using a new annotated dataset of disappeared settlements.

Result: Achieved mean Intersection over Union of 86.30% and Dice score of 92.60%, significantly outperforming traditional methods and off-the-shelf deep learning models.

Conclusion: Provides a powerful tool for policymakers to monitor erosion, anticipate trajectories, and protect vulnerable communities through specialized AI model and annotated dataset.

Abstract: The great rivers of Bangladesh, arteries of commerce and sustenance, are also agents of relentless destruction. Each year, they swallow whole villages and vast tracts of farmland, erasing communities from the map and displacing thousands of families. To track this slow-motion catastrophe has, until now, been a Herculean task for human analysts. Here we show how a powerful general-purpose vision model, the Segment Anything Model (SAM), can be adapted to this task with remarkable precision. To do this, we assembled a new dataset

• a digital chronicle of loss compiled from historical Google Earth imagery of Bangladesh’s most vulnerable regions, including Mokterer Char Union, Kedarpur Union, Balchipara village, and Chowhali Upazila, from 2003 to 2025. Crucially, this dataset is the first to include manually annotated data on the settlements that have vanished beneath the water. Our method first uses a simple color-channel analysis to provide a rough segmentation of land and water, and then fine-tunes SAM’s mask decoder to recognize the subtle signatures of riverbank erosion. The resulting model demonstrates a keen eye for this destructive process, achieving a mean Intersection over Union of 86.30% and a Dice score of 92.60% - a performance that significantly surpasses traditional methods and off-the-shelf deep learning models. This work delivers three key contributions: the first annotated dataset of disappeared settlements in Bangladesh due to river erosion; a specialized AI model fine-tuned for this critical task; and a method for quantifying land loss with compelling visual evidence. Together, these tools provide a powerful new lens through which policymakers and disaster management agencies can monitor erosion, anticipate its trajectory, and ultimately protect the vulnerable communities in its path.

Method: Based on the TimeSformer video recognition model, the approach incorporates detailed tactical features extracted from minimap information, including character position data and other in-game events, using an augmented dataset with tactical event labels.

Result: The model achieved approximately 81% prediction accuracy, particularly from the middle phases of a round onward, significantly outperforming a baseline model trained only on raw minimap information without tactical features.

Conclusion: Leveraging tactical features extracted from match footage is highly effective for predicting round outcomes in VALORANT, demonstrating the value of visual analysis over traditional statistical approaches.

Abstract: Recently, research on predicting match outcomes in esports has been actively conducted, but much of it is based on match log data and statistical information. This research targets the FPS game VALORANT, which requires complex strategies, and aims to build a round outcome prediction model by analyzing minimap information in match footage. Specifically, based on the video recognition model TimeSformer, we attempt to improve prediction accuracy by incorporating detailed tactical features extracted from minimap information, such as character position information and other in-game events. This paper reports preliminary results showing that a model trained on a dataset augmented with such tactical event labels achieved approximately 81% prediction accuracy, especially from the middle phases of a round onward, significantly outperforming a model trained on a dataset with the minimap information itself. This suggests that leveraging tactical features from match footage is highly effective for predicting round outcomes in VALORANT.

Method: EndoCIL incorporates three key components: Maximum Mean Discrepancy Based Replay for diverse exemplar selection, Prior Regularized Class Balanced Loss to address class imbalance, and Calibration of Fully-Connected Gradients to mitigate bias toward new classes.

Result: Extensive experiments on four public endoscopic datasets demonstrate that EndoCIL generally outperforms state-of-the-art CIL methods across varying buffer sizes and evaluation metrics.

Conclusion: The framework effectively balances stability and plasticity in lifelong endoscopic diagnosis, showing promising potential for clinical scalability and deployment.

Abstract: Class-incremental learning (CIL) for endoscopic image analysis is crucial for real-world clinical applications, where diagnostic models should continuously adapt to evolving clinical data while retaining performance on previously learned ones. However, existing replay-based CIL methods fail to effectively mitigate catastrophic forgetting due to severe domain discrepancies and class imbalance inherent in endoscopic imaging. To tackle these challenges, we propose EndoCIL, a novel and unified CIL framework specifically tailored for endoscopic image diagnosis. EndoCIL incorporates three key components: Maximum Mean Discrepancy Based Replay (MDBR), employing a distribution-aligned greedy strategy to select diverse and representative exemplars, Prior Regularized Class Balanced Loss (PRCBL), designed to alleviate both inter-phase and intra-phase class imbalance by integrating prior class distributions and balance weights into the loss function, and Calibration of Fully-Connected Gradients (CFG), which adjusts the classifier gradients to mitigate bias toward new classes. Extensive experiments conducted on four public endoscopic datasets demonstrate that EndoCIL generally outperforms state-of-the-art CIL methods across varying buffer sizes and evaluation metrics. The proposed framework effectively balances stability and plasticity in lifelong endoscopic diagnosis, showing promising potential for clinical scalability and deployment.

Method: Uses DINOv2 with registers to extract generalizable features and suppress perturbations in attention mechanisms, focusing on essential minute features for spoof detection.

Result: Demonstrated effectiveness through experiments on datasets from The 6th Face Anti-Spoofing Workshop and SiW dataset.

Conclusion: The proposed DINOv2-based approach successfully detects face spoofing attacks by leveraging enhanced feature extraction and attention mechanisms.

Abstract: Face recognition systems are designed to be robust against variations in head pose, illumination, and image blur during capture. However, malicious actors can exploit these systems by presenting a face photo of a registered user, potentially bypassing the authentication process. Such spoofing attacks must be detected prior to face recognition. In this paper, we propose a DINOv2-based spoofing attack detection method to discern minute differences between live and spoofed face images. Specifically, we employ DINOv2 with registers to extract generalizable features and to suppress perturbations in the attention mechanism, which enables focused attention on essential and minute features. We demonstrate the effectiveness of the proposed method through experiments conducted on the dataset provided by ``The 6th Face Anti-Spoofing Workshop: Unified Physical-Digital Attacks Detection@ICCV2025’’ and SiW dataset.

Method: Four key components: (1) automated pipeline scaling programmatic tools from documentation and code, (2) synthetic data engine with 17,000+ verifiable tasks, (3) large-scale hybrid action trajectory collection, and (4) two-stage training combining supervised fine-tuning with online reinforcement learning.

Result: 7B and 32B models show 22% relative improvement on OSWorld, 11% faster execution, and 21.7% success rate on WindowsAgentArena (outperforming Windows-trained baselines). Hybrid action reduces error propagation while maintaining efficiency.

Conclusion: Hybrid action mechanism successfully bridges GUI primitives with programmatic tools, enabling strategic alternation between low-level and high-level actions for improved performance and reduced cascading failures in computer-use agents.

Abstract: Multimodal agents for computer use rely exclusively on primitive actions (click, type, scroll) that require accurate visual grounding and lengthy execution chains, leading to cascading failures and performance bottlenecks. While other agents leverage rich programmatic interfaces (APIs, MCP servers, tools), computer-use agents (CUAs) remain isolated from these capabilities. We present UltraCUA, a foundation model that bridges this gap through hybrid action – seamlessly integrating GUI primitives with high-level programmatic tool calls. To achieve this, our approach comprises four key components: (1) an automated pipeline that scales programmatic tools from software documentation, open-source repositories, and code generation; (2) a synthetic data engine producing over 17,000 verifiable tasks spanning real-world computer-use scenarios; (3) a large-scale high-quality hybrid action trajectory collection with both low-level GUI actions and high-level programmatic tool calls; and (4) a two-stage training pipeline combining supervised fine-tuning with online reinforcement learning, enabling strategic alternation between low-level and high-level actions. Experiments with our 7B and 32B models demonstrate substantial improvements over state-of-the-art agents. On OSWorld, UltraCUA models achieve an average 22% relative improvement over base models, while being 11% faster in terms of steps. Out-of-domain evaluation on WindowsAgentArena shows our model reaches 21.7% success rate, outperforming baselines trained on Windows data. The hybrid action mechanism proves critical, reducing error propagation while maintaining execution efficiency.

Method: Proposed FlashMMR framework with Multi-moment Post-verification module using constrained temporal adjustment and verification module to refine moment boundaries and filter low-confidence proposals.

Result: FlashMMR achieves improvements over prior SOTA by 3.00% on G-mAP, 2.70% on mAP@3+tgt, and 2.56% on mR@3 on QV-M^2 dataset. The dataset contains 2,212 annotations covering 6,384 video segments.

Conclusion: QV-M^2 serves as an effective benchmark for training and evaluating MMR models, while FlashMMR provides a strong baseline, establishing foundation for advancing research in realistic video temporal grounding scenarios.

Abstract: Existing Moment retrieval (MR) methods focus on Single-Moment Retrieval (SMR). However, one query can correspond to multiple relevant moments in real-world applications. This makes the existing datasets and methods insufficient for video temporal grounding. By revisiting the gap between current MR tasks and real-world applications, we introduce a high-quality datasets called QVHighlights Multi-Moment Dataset (QV-M$^2$), along with new evaluation metrics tailored for multi-moment retrieval (MMR). QV-M$^2$ consists of 2,212 annotations covering 6,384 video segments. Building on existing efforts in MMR, we propose a framework called FlashMMR. Specifically, we propose a Multi-moment Post-verification module to refine the moment boundaries. We introduce constrained temporal adjustment and subsequently leverage a verification module to re-evaluate the candidate segments. Through this sophisticated filtering pipeline, low-confidence proposals are pruned, and robust multi-moment alignment is achieved. We retrain and evaluate 6 existing MR methods on QV-M$^2$ and QVHighlights under both SMR and MMR settings. Results show that QV-M$^2$ serves as an effective benchmark for training and evaluating MMR models, while FlashMMR provides a strong baseline. Specifically, on QV-M$^2$, it achieves improvements over prior SOTA method by 3.00% on G-mAP, 2.70% on mAP@3+tgt, and 2.56% on mR@3. The proposed benchmark and method establish a foundation for advancing research in more realistic and challenging video temporal grounding scenarios. Code is released at https://github.com/Zhuo-Cao/QV-M2.

Method: Render long texts into images processed by VLMs, with LLM-driven genetic search to optimize visual rendering configurations for accuracy-compression balance.

Result: Achieves 3-4x token compression with comparable accuracy to Qwen3-8B, 4x faster prefilling/decoding, 2x faster SFT training, and enables 128K-context VLM to handle 1M-token tasks.

Conclusion: Visual context scaling via Glyph provides efficient long-context processing while benefiting multimodal tasks like document understanding.

Abstract: Large language models (LLMs) increasingly rely on long-context modeling for tasks such as document understanding, code analysis, and multi-step reasoning. However, scaling context windows to the million-token level brings prohibitive computational and memory costs, limiting the practicality of long-context LLMs. In this work, we take a different perspective-visual context scaling-to tackle this challenge. Instead of extending token-based sequences, we propose Glyph, a framework that renders long texts into images and processes them with vision-language models (VLMs). This approach substantially compresses textual input while preserving semantic information, and we further design an LLM-driven genetic search to identify optimal visual rendering configurations for balancing accuracy and compression. Through extensive experiments, we demonstrate that our method achieves 3-4x token compression while maintaining accuracy comparable to leading LLMs such as Qwen3-8B on various long-context benchmarks. This compression also leads to around 4x faster prefilling and decoding, and approximately 2x faster SFT training. Furthermore, under extreme compression, a 128K-context VLM could scale to handle 1M-token-level text tasks. In addition, the rendered text data benefits real-world multimodal tasks, such as document understanding. Our code and model are released at https://github.com/thu-coai/Glyph.

Method: Uses sequence-based clustering for temporal modeling, concept extraction for interpretability, and demographic-aware data augmentation with frequency-domain transformations to preserve deepfake artifacts and balance underrepresented groups.

Result: Extensive experiments on FaceForensics++, DFD, Celeb-DF, and DFDC datasets using Xception and ResNet architectures demonstrate the method achieves the best tradeoff between fairness and accuracy compared to state-of-the-art approaches.

Conclusion: The proposed framework effectively mitigates bias in deepfake detection while maintaining high accuracy and providing interpretable decisions for non-expert users.

Abstract: Existing deepfake detection methods often exhibit bias, lack transparency, and fail to capture temporal information, leading to biased decisions and unreliable results across different demographic groups. In this paper, we propose a fairness-aware deepfake detection framework that integrates temporal feature learning and demographic-aware data augmentation to enhance fairness and interpretability. Our method leverages sequence-based clustering for temporal modeling of deepfake videos and concept extraction to improve detection reliability while also facilitating interpretable decisions for non-expert users. Additionally, we introduce a demography-aware data augmentation method that balances underrepresented groups and applies frequency-domain transformations to preserve deepfake artifacts, thereby mitigating bias and improving generalization. Extensive experiments on FaceForensics++, DFD, Celeb-DF, and DFDC datasets using state-of-the-art (SoTA) architectures (Xception, ResNet) demonstrate the efficacy of the proposed method in obtaining the best tradeoff between fairness and accuracy when compared to SoTA.

Method: Semi-automated human-in-the-loop pipeline that unifies 200+ sources into 185 subsets, with automation for bulk ingestion and human reviewers for auditing, verification, and quality control, plus rigorous de-duplication and decontamination.

Result: Models trained on FineVision consistently outperform those trained on existing open mixtures across a broad evaluation suite, demonstrating benefits of scale, data hygiene, and balanced automation with human oversight.

Conclusion: The FineVision corpus and curation tools are released to accelerate data-centric VLM research, showing that careful data curation with human oversight significantly improves model performance.

Abstract: The advancement of vision-language models (VLMs) is hampered by a fragmented landscape of inconsistent and contaminated public datasets. We introduce FineVision, a meticulously collected, curated, and unified corpus of 24 million samples - the largest open resource of its kind. We unify more than 200 sources into 185 subsets via a semi-automated, human-in-the-loop pipeline: automation performs bulk ingestion and schema mapping, while reviewers audit mappings and spot-check outputs to verify faithful consumption of annotations, appropriate formatting and diversity, and safety; issues trigger targeted fixes and re-runs. The workflow further applies rigorous de-duplication within and across sources and decontamination against 66 public benchmarks. FineVision also encompasses agentic/GUI tasks with a unified action space; reviewers validate schemas and inspect a sample of trajectories to confirm executable fidelity. Models trained on FineVision consistently outperform those trained on existing open mixtures across a broad evaluation suite, underscoring the benefits of scale, data hygiene, and balanced automation with human oversight. We release the corpus and curation tools to accelerate data-centric VLM research.

Method: PnF extracts structured scene understanding from MLLMs using designed prompts, distills this information into learnable embeddings, and augments existing behavior prediction models with these embeddings to leverage MLLMs’ zero-shot reasoning capabilities.

Result: The approach achieves significant improvements in motion prediction performance on both Waymo Open Motion Dataset and nuScenes Dataset, demonstrating consistent performance gains across state-of-the-art motion forecasting models.

Conclusion: PnF provides an effective plug-and-play solution that enables quick adaptation to complex scenarios using natural language descriptions, making it practical to adopt without fine-tuning while improving motion forecasting performance.

Abstract: Current autonomous driving systems rely on specialized models for perceiving and predicting motion, which demonstrate reliable performance in standard conditions. However, generalizing cost-effectively to diverse real-world scenarios remains a significant challenge. To address this, we propose Plug-and-Forecast (PnF), a plug-and-play approach that augments existing motion forecasting models with multimodal large language models (MLLMs). PnF builds on the insight that natural language provides a more effective way to describe and handle complex scenarios, enabling quick adaptation to targeted behaviors. We design prompts to extract structured scene understanding from MLLMs and distill this information into learnable embeddings to augment existing behavior prediction models. Our method leverages the zero-shot reasoning capabilities of MLLMs to achieve significant improvements in motion prediction performance, while requiring no fine-tuning – making it practical to adopt. We validate our approach on two state-of-the-art motion forecasting models using the Waymo Open Motion Dataset and the nuScenes Dataset, demonstrating consistent performance improvements across both benchmarks.

Method: Uses saliency priors to highlight WBC regions, a hybrid EfficientSwin-style backbone for multi-resolution features, cross-layer fusion module, multi-task training with segmentation and classification heads, class-aware weighted losses, and saliency-alignment regularization.

Result: Consistent improvements in IoU, F1, and classification accuracy on standard benchmarks (BCCD, LISC, ALL-IDB) compared to CNN and transformer baselines, with ablation studies confirming contributions of saliency preprocessing and cross-layer fusion.

Conclusion: SG-CLDFF provides a practical and explainable solution for reliable automated WBC analysis in clinical workflows, with enhanced robustness and interpretability through saliency guidance and feature fusion.

Abstract: Accurate segmentation and classification of white blood cells (WBCs) in microscopic images are essential for diagnosis and monitoring of many hematological disorders, yet remain challenging due to staining variability, complex backgrounds, and class imbalance. In this paper, we introduce a novel Saliency-Guided Cross-Layer Deep Feature Fusion framework (SG-CLDFF) that tightly integrates saliency-driven preprocessing with multi-scale deep feature aggregation to improve both robustness and interpretability for WBC analysis. SG-CLDFF first computes saliency priors to highlight candidate WBC regions and guide subsequent feature extraction. A lightweight hybrid backbone (EfficientSwin-style) produces multi-resolution representations, which are fused by a ResNeXt-CC-inspired cross-layer fusion module to preserve complementary information from shallow and deep layers. The network is trained in a multi-task setup with concurrent segmentation and cell-type classification heads, using class-aware weighted losses and saliency-alignment regularization to mitigate imbalance and suppress background activation. Interpretability is enforced through Grad-CAM visualizations and saliency consistency checks, allowing model decisions to be inspected at the regional level. We validate the framework on standard public benchmarks (BCCD, LISC, ALL-IDB), reporting consistent gains in IoU, F1, and classification accuracy compared to strong CNN and transformer baselines. An ablation study also demonstrates the individual contributions of saliency preprocessing and cross-layer fusion. SG-CLDFF offers a practical and explainable path toward more reliable automated WBC analysis in clinical workflows.

Method: Uses YOLOv11 object detection to identify blue markers above surgical sites, then directs high-power LED lighting using servomotors with tilt-pan brackets.

Result: YOLO model achieves 96.7% mAP@50 on validation set with annotated surgical scene images containing blue spherical markers.

Conclusion: The automated machine vision-based lighting system reduces physical strain on surgeons, improves illumination consistency, and supports better surgical outcomes.

Abstract: Effortless and ergonomically designed surgical lighting is critical for precision and safety during procedures. However, traditional systems often rely on manual adjustments, leading to surgeon fatigue, neck strain, and inconsistent illumination due to drift and shadowing. To address these challenges, we propose a novel surgical lighting system that leverages the YOLOv11 object detection algorithm to identify a blue marker placed above the target surgical site. A high-power LED light source is then directed to the identified location using two servomotors equipped with tilt-pan brackets. The YOLO model achieves 96.7% mAP@50 on the validation set consisting of annotated images simulating surgical scenes with the blue spherical marker. By automating the lighting process, this machine vision-based solution reduces physical strain on surgeons, improves consistency in illumination, and supports improved surgical outcomes.

Method: Proposes Dense representation Structure Estimator (DSE) with class-relevance and effective dimensionality measures, plus model selection strategy and DSE-based regularization.

Result: Model selection improves mIoU by 3.0% on average across 16 SSL methods, and DSE regularization consistently mitigates dense degradation effects.

Conclusion: The DSE metric effectively identifies optimal training checkpoints for dense tasks and provides a regularization method to prevent performance degradation in self-supervised learning.

Abstract: In this work, we observe a counterintuitive phenomenon in self-supervised learning (SSL): longer training may impair the performance of dense prediction tasks (e.g., semantic segmentation). We refer to this phenomenon as Self-supervised Dense Degradation (SDD) and demonstrate its consistent presence across sixteen state-of-the-art SSL methods with various losses, architectures, and datasets. When the model performs suboptimally on dense tasks at the end of training, measuring the performance during training becomes essential. However, evaluating dense performance effectively without annotations remains an open challenge. To tackle this issue, we introduce a Dense representation Structure Estimator (DSE), composed of a class-relevance measure and an effective dimensionality measure. The proposed DSE is both theoretically grounded and empirically validated to be closely correlated with the downstream performance. Based on this metric, we introduce a straightforward yet effective model selection strategy and a DSE-based regularization method. Experiments on sixteen SSL methods across four benchmarks confirm that model selection improves mIoU by $3.0%$ on average with negligible computational cost. Additionally, DSE regularization consistently mitigates the effects of dense degradation. Code is available at https://github.com/EldercatSAM/SSL-Degradation.

Method: Decomposes the problem into two stages: BOLD deconvolution to recover latent neural activity, followed by causal graph inference using a novel Conditional Mamba architecture.

Result: Achieves 37% higher accuracy than DCM on simulated data, recovers well-established neural pathways with 88% fidelity in real task fMRI data, and reveals strategic brain network reconfigurations during working memory tasks that traditional methods miss.

Conclusion: Provides neuroscientists with a practical tool for large-scale causal inference that captures both fundamental circuit motifs and flexible network dynamics underlying cognitive function.

Abstract: We introduce CausalMamba, a scalable framework that addresses fundamental limitations in fMRI-based causal inference: the ill-posed nature of inferring neural causality from hemodynamically distorted BOLD signals and the computational intractability of existing methods like Dynamic Causal Modeling (DCM). Our approach decomposes this complex inverse problem into two tractable stages: BOLD deconvolution to recover latent neural activity, followed by causal graph inference using a novel Conditional Mamba architecture. On simulated data, CausalMamba achieves 37% higher accuracy than DCM. Critically, when applied to real task fMRI data, our method recovers well-established neural pathways with 88% fidelity, whereas conventional approaches fail to identify these canonical circuits in over 99% of subjects. Furthermore, our network analysis of working memory data reveals that the brain strategically shifts its primary causal hub-recruiting executive or salience networks depending on the stimulus-a sophisticated reconfiguration that remains undetected by traditional methods. This work provides neuroscientists with a practical tool for large-scale causal inference that captures both fundamental circuit motifs and flexible network dynamics underlying cognitive function.

Method: Evaluated various defense methods against adversarial clothes in both digital and physical worlds, crafting a single set of adversarial clothes that could break multiple defense methods on Faster R-CNN detector.

Result: All defense methods performed poorly against adversarial clothes. A single adversarial clothing set achieved 96.06% ASR against undefended detector and over 64.84% ASR against nine defended models in physical world.

Conclusion: Existing adversarial defense methods have common vulnerabilities against large-scale adversarial attacks like adversarial clothes, highlighting the need for more robust defense strategies.

Abstract: In recent years, adversarial attacks against deep learning-based object detectors in the physical world have attracted much attention. To defend against these attacks, researchers have proposed various defense methods against adversarial patches, a typical form of physically-realizable attack. However, our experiments showed that simply enlarging the patch size could make these defense methods fail. Motivated by this, we evaluated various defense methods against adversarial clothes which have large coverage over the human body. Adversarial clothes provide a good test case for adversarial defense against patch-based attacks because they not only have large sizes but also look more natural than a large patch on humans. Experiments show that all the defense methods had poor performance against adversarial clothes in both the digital world and the physical world. In addition, we crafted a single set of clothes that broke multiple defense methods on Faster R-CNN. The set achieved an Attack Success Rate (ASR) of 96.06% against the undefended detector and over 64.84% ASRs against nine defended models in the physical world, unveiling the common vulnerability of existing adversarial defense methods against adversarial clothes. Code is available at: https://github.com/weiz0823/adv-clothes-break-multiple-defenses.

Method: Uses diffusion-based framework with character-level guidance, leveraging fine-grained character priors from external segmentation and OCR modules. Incorporates CHARM module for precise region-wise masking to restrict character guidance to their own regions.

Result: Significantly outperformed baseline restoration models, achieving 28% relative reduction in CER on Roboflow-LP dataset compared to best-performing baseline model.

Conclusion: Structured character-guided conditioning effectively enhances robustness of diffusion-based license plate restoration and recognition in practical deployment scenarios.

Abstract: The significance of license plate image restoration goes beyond the preprocessing stage of License Plate Recognition (LPR) systems, as it also serves various purposes, including increasing evidential value, enhancing the clarity of visual interface, and facilitating further utilization of license plate images. We propose a novel diffusion-based framework with character-level guidance, CharDiff, which effectively restores and recognizes severely degraded license plate images captured under realistic conditions. CharDiff leverages fine-grained character-level priors extracted through external segmentation and Optical Character Recognition (OCR) modules tailored for low-quality license plate images. For precise and focused guidance, CharDiff incorporates a novel Character-guided Attention through Region-wise Masking (CHARM) module, which ensures that each character’s guidance is restricted to its own region, thereby avoiding interference with other regions. In experiments, CharDiff significantly outperformed the baseline restoration models in both restoration quality and recognition accuracy, achieving a 28% relative reduction in CER on the Roboflow-LP dataset, compared to the best-performing baseline model. These results indicate that the structured character-guided conditioning effectively enhances the robustness of diffusion-based license plate restoration and recognition in practical deployment scenarios.

Method: Proposed iDETEX - a unified multimodal large language model with task-specific offline augmentation modules, data mixing strategy, and online enhancement strategies to exploit multi-sourced supervision across three key tasks: quality grounding, perception, and description.

Result: Achieved state-of-the-art performance across all subtasks on the ViDA-UGC benchmark and ranked first in the ICCV MIPI 2025 Detailed Image Quality Assessment Challenge.

Conclusion: iDETEX demonstrates effectiveness and robustness in delivering accurate and interpretable quality assessments through its unified multimodal approach.

Abstract: Image Quality Assessment (IQA) has progressed from scalar quality prediction to more interpretable, human-aligned evaluation paradigms. In this work, we address the emerging challenge of detailed and explainable IQA by proposing iDETEX-a unified multimodal large language model (MLLM) capable of simultaneously performing three key tasks: quality grounding, perception, and description. To facilitate efficient and generalizable training across these heterogeneous subtasks, we design a suite of task-specific offline augmentation modules and a data mixing strategy. These are further complemented by online enhancement strategies to fully exploit multi-sourced supervision. We validate our approach on the large-scale ViDA-UGC benchmark, where iDETEX achieves state-of-the-art performance across all subtasks. Our model ranks first in the ICCV MIPI 2025 Detailed Image Quality Assessment Challenge, demonstrating its effectiveness and robustness in delivering accurate and interpretable quality assessments.

Method: Proposes measuring agreement between NCM-based probability distribution (from feature space) and softmax probabilities (from logit space) as a post-processing step.

Result: Achieved AUROC of 93.41 and 95.35 on African and Swedish animal datasets, ranking top three consistently across both datasets.

Conclusion: The method provides consistent open-set recognition performance across different datasets without requiring model retraining, outperforming state-of-the-art approaches that excel on only single datasets.

Abstract: Current state-of-the-art Wildlife classification models are trained under the closed world setting. When exposed to unknown classes, they remain overconfident in their predictions. Open-set Recognition (OSR) aims to classify known classes while rejecting unknown samples. Several OSR methods have been proposed to model the closed-set distribution by observing the feature, logit, or softmax probability space. A significant drawback of many existing approaches is the requirement to retrain the pre-trained classification model with the OSR-specific strategy. This study contributes a post-processing OSR method that measures the agreement between the models’ features and predicted logits. We propose a probability distribution based on an input’s distance to its Nearest Class Mean (NCM). The NCM-based distribution is then compared with the softmax probabilities from the logit space to measure agreement between the NCM and the classification head. Our proposed strategy ranks within the top three on two evaluated datasets, showing consistent performance across the two datasets. In contrast, current state-of-the-art methods excel on a single dataset. We achieve an AUROC of 93.41 and 95.35 for African and Swedish animals. The code can be found https://github.com/Applied-Representation-Learning-Lab/OSR.

Method: Proposes Semantic-E2VID with: 1) Cross-modal feature alignment module to transfer visual semantics from SAM to event encoder, 2) Semantic-aware feature fusion block to integrate learned semantics, and 3) Semantic Perceptual E2V Supervision using SAM-generated labels.

Result: Extensive experiments show Semantic-E2VID significantly enhances frame quality and outperforms state-of-the-art E2V methods across multiple benchmarks.

Conclusion: The proposed framework successfully bridges the semantic gap in event-to-video reconstruction by leveraging vision foundation models, demonstrating improved reconstruction quality through semantic-aware processing.

Abstract: Event cameras offer distinct advantages such as low latency, high dynamic range, and efficient motion capture. However, event-to-video reconstruction (E2V), a fundamental event-based vision task, remains challenging, particularly for reconstructing and recovering semantic information. This is primarily due to the nature of the event camera, as it only captures intensity changes, ignoring static objects and backgrounds, resulting in a lack of semantic information in captured event modality. Further, semantic information plays a crucial role in video and frame reconstruction, yet is often overlooked by existing E2V approaches. To bridge this gap, we propose Semantic-E2VID, an E2V framework that explores the missing visual semantic knowledge in event modality and leverages it to enhance event-to-video reconstruction. Specifically, Semantic-E2VID introduces a cross-modal feature alignment (CFA) module to transfer the robust visual semantics from a frame-based vision foundation model, the Segment Anything Model (SAM), to the event encoder, while aligning the high-level features from distinct modalities. To better utilize the learned semantic feature, we further propose a semantic-aware feature fusion (SFF) block to integrate learned semantics in frame modality to form event representations with rich semantics that can be decoded by the event decoder. Further, to facilitate the reconstruction of semantic information, we propose a novel Semantic Perceptual E2V Supervision that helps the model to reconstruct semantic details by leveraging SAM-generated categorical labels. Extensive experiments demonstrate that Semantic-E2VID significantly enhances frame quality, outperforming state-of-the-art E2V methods across multiple benchmarks. The sample code is included in the supplementary material.

Method: Uses Window-Based Cross-Task Attention Module for structured feature exchange while preserving task-specific details, built on lightweight ViT-based DINOv2 backbone optimized for real-time deployment.

Result: Outperforms state-of-the-art multi-task models on NYUDv2, surpasses single-task depth and semantic baselines on Hypersim, achieves superior performance on Cityscapes, while maintaining computational efficiency on laptop hardware.

Conclusion: M2H demonstrates practicality in real-world spatial perception tasks and serves as foundation for monocular spatial perception systems supporting 3D scene graph construction in dynamic environments.

Abstract: Deploying real-time spatial perception on edge devices requires efficient multi-task models that leverage complementary task information while minimizing computational overhead. This paper introduces Multi-Mono-Hydra (M2H), a novel multi-task learning framework designed for semantic segmentation and depth, edge, and surface normal estimation from a single monocular image. Unlike conventional approaches that rely on independent single-task models or shared encoder-decoder architectures, M2H introduces a Window-Based Cross-Task Attention Module that enables structured feature exchange while preserving task-specific details, improving prediction consistency across tasks. Built on a lightweight ViT-based DINOv2 backbone, M2H is optimized for real-time deployment and serves as the foundation for monocular spatial perception systems supporting 3D scene graph construction in dynamic environments. Comprehensive evaluations show that M2H outperforms state-of-the-art multi-task models on NYUDv2, surpasses single-task depth and semantic baselines on Hypersim, and achieves superior performance on the Cityscapes dataset, all while maintaining computational efficiency on laptop hardware. Beyond benchmarks, M2H is validated on real-world data, demonstrating its practicality in spatial perception tasks.

Method: Three key concepts: 1) LLM-informed selection of visual tokens based on attention to discard ~95% unimportant tokens, 2) Recurrent processing of past selected tokens for temporal coherence, 3) Caption-based question answering for lightweight responses.

Result: Achieves state-of-the-art performance on streaming video benchmarks with minimal performance loss while maintaining high efficiency.

Conclusion: The proposed training-free approach effectively balances efficiency and effectiveness for streaming video understanding in Video-LLMs.

Abstract: Video Large Language Models (Video-LLMs) excel at understanding videos in-context, provided they have full access to the video when answering queries. However, these models face challenges in streaming scenarios where hour-long videos must be processed online, and questions need timely responses. In this work, we propose a training-free approach compatible with standard Video-LLMs, leveraging three key concepts: 1) LLM-informed selection of visual tokens to identify those that the LLM has attended to and contributed to its understanding of each short clip. Our attention-based selection allows us to discard up to ~95% of unimportant visual tokens with minimal performance loss; 2) Recurrent processing of past selected tokens to generate temporally coherent understanding of each processed clip; 3) Caption-based question answering for lightweight and accurate responses. Our method achieves state-of-the-art performance on streaming video benchmarks, striking a balance between efficiency and effectiveness.

Method: Systematic literature review of 25 synthetic facial recognition datasets (2018-2025) combined with experimental validation of 10M+ synthetic samples, evaluating 7 key privacy requirements and comparing results on 5 standard benchmarks.

Result: Best synthetic datasets (VariFace, VIGFace) achieved 95.67% and 94.91% accuracy respectively, surpassing CASIA-WebFace (94.70%). Synthetic data ensures proper intra-class variability and identity separability while offering unprecedented bias mitigation control.

Conclusion: Synthetic facial data is scientifically viable and ethically imperative for facial recognition research, providing privacy-preserving alternatives that can replace real datasets while enabling better bias control.

Abstract: The deployment of facial recognition systems has created an ethical dilemma: achieving high accuracy requires massive datasets of real faces collected without consent, leading to dataset retractions and potential legal liabilities under regulations like GDPR. While synthetic facial data presents a promising privacy-preserving alternative, the field lacks comprehensive empirical evidence of its viability. This study addresses this critical gap through extensive evaluation of synthetic facial recognition datasets. We present a systematic literature review identifying 25 synthetic facial recognition datasets (2018-2025), combined with rigorous experimental validation. Our methodology examines seven key requirements for privacy-preserving synthetic data: identity leakage prevention, intra-class variability, identity separability, dataset scale, ethical data sourcing, bias mitigation, and benchmark reliability. Through experiments involving over 10 million synthetic samples, extended by a comparison of results reported on five standard benchmarks, we provide the first comprehensive empirical assessment of synthetic data’s capability to replace real datasets. Best-performing synthetic datasets (VariFace, VIGFace) achieve recognition accuracies of 95.67% and 94.91% respectively, surpassing established real datasets including CASIA-WebFace (94.70%). While those images remain private, publicly available alternatives Vec2Face (93.52%) and CemiFace (93.22%) come close behind. Our findings reveal that they ensure proper intra-class variability while maintaining identity separability. Demographic bias analysis shows that, even though synthetic data inherits limited biases, it offers unprecedented control for bias mitigation through generation parameters. These results establish synthetic facial data as a scientifically viable and ethically imperative alternative for facial recognition research.

Method: Integrates multiple facial expression features through attention-based feature fusion and uses adaptive class balancing strategy that dynamically adjusts training sample contributions based on class distribution and classification difficulty.

Result: Experimental results demonstrate promising performance for PD severity diagnosis and confirm the efficacy of both attention-based feature fusion and adaptive class balancing.

Conclusion: The proposed method effectively addresses limitations of existing approaches by leveraging multiple facial expressions and handling class imbalance, showing strong potential for PD severity diagnosis.

Abstract: Parkinson’s disease (PD) severity diagnosis is crucial for early detecting potential patients and adopting tailored interventions. Diagnosing PD based on facial expression is grounded in PD patients’ “masked face” symptom and gains growing interest recently for its convenience and affordability. However, current facial expression-based approaches often rely on single type of expression which can lead to misdiagnosis, and ignore the class imbalance across different PD stages which degrades the prediction performance. Moreover, most existing methods focus on binary classification (i.e., PD / non-PD) rather than diagnosing the severity of PD. To address these issues, we propose a new facial expression-based method for PD severity diagnosis which integrates multiple facial expression features through attention-based feature fusion. Moreover, we mitigate the class imbalance problem via an adaptive class balancing strategy which dynamically adjusts the contribution of training samples based on their class distribution and classification difficulty. Experimental results demonstrate the promising performance of the proposed method for PD severity diagnosis, as well as the efficacy of attention-based feature fusion and adaptive class balancing.

Method: LoopTrans introduces a closed-loop framework with unified cross-modal localization and denoising knowledge distillation to bridge domain gaps between object-centered egocentric and interaction-centered exocentric images.

Result: Experiments show consistent improvements across all metrics on image and video benchmarks, even handling challenging scenarios where object interaction regions are fully occluded by the human body.

Conclusion: The bidirectional knowledge transfer in LoopTrans enhances affordance grounding performance and addresses limitations of previous one-way transfer approaches.

Abstract: Humans can perform previously unexperienced interactions with novel objects simply by observing others engage with them. Weakly-supervised affordance grounding mimics this process by learning to locate object regions that enable actions on egocentric images, using exocentric interaction images with image-level annotations. However, extracting affordance knowledge solely from exocentric images and transferring it one-way to egocentric images limits the applicability of previous works in complex interaction scenarios. Instead, this study introduces LoopTrans, a novel closed-loop framework that not only transfers knowledge from exocentric to egocentric but also transfers back to enhance exocentric knowledge extraction. Within LoopTrans, several innovative mechanisms are introduced, including unified cross-modal localization and denoising knowledge distillation, to bridge domain gaps between object-centered egocentric and interaction-centered exocentric images while enhancing knowledge transfer. Experiments show that LoopTrans achieves consistent improvements across all metrics on image and video benchmarks, even handling challenging scenarios where object interaction regions are fully occluded by the human body.

Method: Uses object detection (YOLOv11) and multi-object tracking (BoT-SORT) to track horses and people in stables, with event classification based on object trajectories and spatial relations. Custom dataset created using CLIP and GroundingDINO foundation models.

Result: System reliably detects horse-related events but struggles with people detection due to data scarcity. Successfully distinguishes five event types and handles camera blind spots.

Conclusion: Provides foundation for real-time behavioral monitoring in equine facilities, with potential applications in animal welfare and stable management.

Abstract: Monitoring the behavior of stalled horses is essential for early detection of health and welfare issues but remains labor-intensive and time-consuming. In this study, we present a prototype vision-based monitoring system that automates the detection and tracking of horses and people inside stables using object detection and multi-object tracking techniques. The system leverages YOLOv11 and BoT-SORT for detection and tracking, while event states are inferred based on object trajectories and spatial relations within the stall. To support development, we constructed a custom dataset annotated with assistance from foundation models CLIP and GroundingDINO. The system distinguishes between five event types and accounts for the camera’s blind spots. Qualitative evaluation demonstrated reliable performance for horse-related events, while highlighting limitations in detecting people due to data scarcity. This work provides a foundation for real-time behavioral monitoring in equine facilities, with implications for animal welfare and stable management.

Method: Fuses outputs from 7 keypoint and 2 edge detectors to create ground-truth masks, then trains a lightweight ESPNet model using these masks to produce dense, semantically-aware keypoints.

Result: Outperforms other detectors on Oxford Affine Covariant Regions dataset with maximum values of 0.5143 (keypoint density), 0.9582 (repeatability), and 59,003 (correct matches).

Conclusion: DeepDetect successfully unifies classical detector strengths with deep learning to achieve superior performance in keypoint detection across diverse and challenging conditions.

Abstract: Keypoint detection is the foundation of many computer vision tasks, including image registration, structure-from motion, 3D reconstruction, visual odometry, and SLAM. Traditional detectors (SIFT, SURF, ORB, BRISK, etc.) and learning based methods (SuperPoint, R2D2, LF-Net, D2-Net, etc.) have shown strong performance yet suffer from key limitations: sensitivity to photometric changes, low keypoint density and repeatability, limited adaptability to challenging scenes, and lack of semantic understanding, often failing to prioritize visually important regions. We present DeepDetect, an intelligent, all-in-one, dense keypoint detector that unifies the strengths of classical detectors using deep learning. Firstly, we create ground-truth masks by fusing outputs of 7 keypoint and 2 edge detectors, extracting diverse visual cues from corners and blobs to prominent edges and textures in the images. Afterwards, a lightweight and efficient model: ESPNet, is trained using these masks as labels, enabling DeepDetect to focus semantically on images while producing highly dense keypoints, that are adaptable to diverse and visually degraded conditions. Evaluations on the Oxford Affine Covariant Regions dataset demonstrate that DeepDetect surpasses other detectors in keypoint density, repeatability, and the number of correct matches, achieving maximum values of 0.5143 (average keypoint density), 0.9582 (average repeatability), and 59,003 (correct matches).

Method: Using AV1 motion vectors to generate dense sub-pixel correspondences and short tracks filtered by cosine consistency, with evaluation on short videos.

Result: Comparable performance to sequential SIFT with far less CPU usage, denser matches with competitive pairwise geometry, and successful SfM demo on 117-frame clip with 0.46-0.62M points reconstructed at 0.51-0.53px reprojection error.

Conclusion: Compressed-domain correspondences are a practical, resource-efficient front end with clear paths to scaling in full pipelines.

Abstract: We repurpose AV1 motion vectors to produce dense sub-pixel correspondences and short tracks filtered by cosine consistency. On short videos, this compressed-domain front end runs comparably to sequential SIFT while using far less CPU, and yields denser matches with competitive pairwise geometry. As a small SfM demo on a 117-frame clip, MV matches register all images and reconstruct 0.46-0.62M points at 0.51-0.53,px reprojection error; BA time grows with match density. These results show compressed-domain correspondences are a practical, resource-efficient front end with clear paths to scaling in full pipelines.

Method: Proposes Color Space Transformation Network (CST-Net) with learnable color space converter to facilitate rain removal in Y channel where rain is more pronounced, and introduces implicit illumination guidance to capture illumination information for better robustness.

Result: Extensive experiments demonstrate the value of the new HQ-NightRain dataset and the effectiveness of the proposed CST-Net method for nighttime image deraining.

Conclusion: The paper provides a high-quality benchmark and an effective network architecture that addresses the unique challenges of nighttime image deraining through color space transformation and illumination guidance.

Abstract: Compared to daytime image deraining, nighttime image deraining poses significant challenges due to inherent complexities of nighttime scenarios and the lack of high-quality datasets that accurately represent the coupling effect between rain and illumination. In this paper, we rethink the task of nighttime image deraining and contribute a new high-quality benchmark, HQ-NightRain, which offers higher harmony and realism compared to existing datasets. In addition, we develop an effective Color Space Transformation Network (CST-Net) for better removing complex rain from nighttime scenes. Specifically, we propose a learnable color space converter (CSC) to better facilitate rain removal in the Y channel, as nighttime rain is more pronounced in the Y channel compared to the RGB color space. To capture illumination information for guiding nighttime deraining, implicit illumination guidance is introduced enabling the learned features to improve the model’s robustness in complex scenarios. Extensive experiments show the value of our dataset and the effectiveness of our method. The source code and datasets are available at https://github.com/guanqiyuan/CST-Net.

Method: Three initialization improvements: (1) frequency-aware SfM using low-frequency view augmentation for better low-texture coverage, (2) 3DGS self-initialization that lifts photometric supervision to add points in SfM-sparse regions, (3) point-cloud regularization with geometric/visibility priors for multi-view consistency and uniform coverage.

Result: Experiments on LLFF and Mip-NeRF360 datasets show consistent improvements in sparse-view settings, establishing the approach as a stronger initialization strategy for 3D Gaussian Splatting.

Conclusion: Initialization is the primary factor in sparse-view 3DGS performance. The proposed three-pronged initialization approach effectively supplements SfM’s coverage gaps and provides a more reliable foundation for 3D Gaussian Splatting in sparse-view scenarios.

Abstract: Sparse-view 3D Gaussian Splatting (3DGS) often overfits to the training views, leading to artifacts like blurring in novel view rendering. Prior work addresses it either by enhancing the initialization (\emph{i.e.}, the point cloud from Structure-from-Motion (SfM)) or by adding training-time constraints (regularization) to the 3DGS optimization. Yet our controlled ablations reveal that initialization is the decisive factor: it determines the attainable performance band in sparse-view 3DGS, while training-time constraints yield only modest within-band improvements at extra cost. Given initialization’s primacy, we focus our design there. Although SfM performs poorly under sparse views due to its reliance on feature matching, it still provides reliable seed points. Thus, building on SfM, our effort aims to supplement the regions it fails to cover as comprehensively as possible. Specifically, we design: (i) frequency-aware SfM that improves low-texture coverage via low-frequency view augmentation and relaxed multi-view correspondences; (ii) 3DGS self-initialization that lifts photometric supervision into additional points, compensating SfM-sparse regions with learned Gaussian centers; and (iii) point-cloud regularization that enforces multi-view consistency and uniform spatial coverage through simple geometric/visibility priors, yielding a clean and reliable point cloud. Our experiments on LLFF and Mip-NeRF360 demonstrate consistent gains in sparse-view settings, establishing our approach as a stronger initialization strategy. Code is available at https://github.com/zss171999645/ItG-GS.

Method: Proposes Range-Adaptive Perception module with learnable queries modulated by ego vehicle states and temporal-spatial associations, and State-Conditioned Forecasting module using regression-guided formulation instead of classification.

Result: Achieves state-of-the-art performance across perception, forecasting, and planning tasks, with advantages in flexibility, adaptability, and efficiency.

Conclusion: SparseWorld demonstrates superior performance through its sparse dynamic query approach and regression-based forecasting, effectively addressing limitations of traditional occupancy models.

Abstract: Semantic occupancy has emerged as a powerful representation in world models for its ability to capture rich spatial semantics. However, most existing occupancy world models rely on static and fixed embeddings or grids, which inherently limit the flexibility of perception. Moreover, their ``in-place classification" over grids exhibits a potential misalignment with the dynamic and continuous nature of real scenarios.In this paper, we propose SparseWorld, a novel 4D occupancy world model that is flexible, adaptive, and efficient, powered by sparse and dynamic queries. We propose a Range-Adaptive Perception module, in which learnable queries are modulated by the ego vehicle states and enriched with temporal-spatial associations to enable extended-range perception. To effectively capture the dynamics of the scene, we design a State-Conditioned Forecasting module, which replaces classification-based forecasting with regression-guided formulation, precisely aligning the dynamic queries with the continuity of the 4D environment. In addition, We specifically devise a Temporal-Aware Self-Scheduling training strategy to enable smooth and efficient training. Extensive experiments demonstrate that SparseWorld achieves state-of-the-art performance across perception, forecasting, and planning tasks. Comprehensive visualizations and ablation studies further validate the advantages of SparseWorld in terms of flexibility, adaptability, and efficiency. The code is available at https://github.com/MSunDYY/SparseWorld.

Method: Introduces POTNet with entropy-guided dual-clustering: spectral clustering for high-entropy pixels and k-means for low-entropy pixels, aligned by optimal transport. This generates sharp pseudo-masks that supervise a MaskFormer-style encoder-decoder.

Result: AutoSOD outperforms unsupervised methods by up to 26% and weakly supervised methods by up to 36% in F-measure, narrowing the gap to fully supervised models.

Conclusion: The proposed split-fuse-transport design enables high-quality pseudo-mask generation without handcrafted priors, making unsupervised SOD more accurate and efficient.

Abstract: Salient object detection (SOD) aims to segment visually prominent regions in images and serves as a foundational task for various computer vision applications. We posit that SOD can now reach near-supervised accuracy without a single pixel-level label, but only when reliable pseudo-masks are available. We revisit the prototype-based line of work and make two key observations. First, boundary pixels and interior pixels obey markedly different geometry; second, the global consistency enforced by optimal transport (OT) is underutilized if prototype quality is weak. To address this, we introduce POTNet, an adaptation of Prototypical Optimal Transport that replaces POT’s single k-means step with an entropy-guided dual-clustering head: high-entropy pixels are organized by spectral clustering, low-entropy pixels by k-means, and the two prototype sets are subsequently aligned by OT. This split-fuse-transport design yields sharper, part-aware pseudo-masks in a single forward pass, without handcrafted priors. Those masks supervise a standard MaskFormer-style encoder-decoder, giving rise to AutoSOD, an end-to-end unsupervised SOD pipeline that eliminates SelfMask’s offline voting yet improves both accuracy and training efficiency. Extensive experiments on five benchmarks show that AutoSOD outperforms unsupervised methods by up to 26% and weakly supervised methods by up to 36% in F-measure, further narrowing the gap to fully supervised models.

Method: Transform small subset of ground-truth annotations into pseudo labels aggregated into dataset-adaptive scoring rubrics. Use contextual prompting where first/last segments are scored based on descriptions while intermediate ones incorporate adjacent scene summaries to assess narrative progression and redundancy.

Result: Achieved F1 scores of 57.58 on SumMe and 63.05 on TVSum, surpassing unsupervised and prior zero-shot baselines while approaching supervised performance.

Conclusion: Rubric-guided pseudo labeling effectively stabilizes LLM-based scoring and establishes a general, interpretable zero-shot paradigm for video summarization.

Abstract: With the rapid proliferation of video content across social media, surveillance, and education platforms, efficiently summarizing long videos into concise yet semantically faithful surrogates has become increasingly vital. Existing supervised methods achieve strong in-domain accuracy by learning from dense annotations but suffer from high labeling costs and limited cross-dataset generalization, while unsupervised approaches, though label-free, often fail to capture high-level human semantics and fine-grained narrative cues. More recently, zero-shot prompting pipelines have leveraged large language models (LLMs) for training-free video summarization, yet remain highly sensitive to handcrafted prompt templates and dataset-specific score normalization. To overcome these limitations, we introduce a rubric-guided, pseudo-labeled prompting framework that transforms a small subset of ground-truth annotations into high-confidence pseudo labels, which are aggregated into structured, dataset-adaptive scoring rubrics guiding interpretable scene evaluation. During inference, first and last segments are scored based solely on their descriptions, whereas intermediate ones incorporate brief contextual summaries of adjacent scenes to assess narrative progression and redundancy. This contextual prompting enables the LLM to balance local salience and global coherence without parameter tuning. On SumMe and TVSum, our method achieves F1 scores of \textbf{57.58} and \textbf{63.05}, surpassing unsupervised and prior zero-shot baselines while approaching supervised performance. The results demonstrate that rubric-guided pseudo labeling effectively stabilizes LLM-based scoring and establishes a general, interpretable zero-shot paradigm for video summarization.

Method: Optimized four pillars: data processing, model architecture, training strategy, and infrastructure. Used curriculum-based pretraining, alignment-focused post-training, and Megatron-Core for efficient multi-node scaling.

Result: MUG-V 10B matches state-of-the-art video generators overall and surpasses leading open-source baselines on e-commerce video generation tasks in human evaluations. Achieved significant efficiency gains and performance improvements.

Conclusion: Successfully developed and open-sourced a complete large-scale video generation training stack including model weights, training code, and inference pipelines, enabling efficient training of video generation models.

Abstract: In recent years, large-scale generative models for visual content (\textit{e.g.,} images, videos, and 3D objects/scenes) have made remarkable progress. However, training large-scale video generation models remains particularly challenging and resource-intensive due to cross-modal text-video alignment, the long sequences involved, and the complex spatiotemporal dependencies. To address these challenges, we present a training framework that optimizes four pillars: (i) data processing, (ii) model architecture, (iii) training strategy, and (iv) infrastructure for large-scale video generation models. These optimizations delivered significant efficiency gains and performance improvements across all stages of data preprocessing, video compression, parameter scaling, curriculum-based pretraining, and alignment-focused post-training. Our resulting model, MUG-V 10B, matches recent state-of-the-art video generators overall and, on e-commerce-oriented video generation tasks, surpasses leading open-source baselines in human evaluations. More importantly, we open-source the complete stack, including model weights, Megatron-Core-based large-scale training code, and inference pipelines for video generation and enhancement. To our knowledge, this is the first public release of large-scale video generation training code that exploits Megatron-Core to achieve high training efficiency and near-linear multi-node scaling, details are available in \href{https://github.com/Shopee-MUG/MUG-V}{our webpage}.

Method: Proposed MambaX-Net with Mamba-enhanced Cross-Attention Module for temporal evolution and long-range dependencies, Shape Extractor Module for anatomical representation, and semi-supervised self-training using pseudo-labels from pre-trained nnU-Net.

Result: MambaX-Net significantly outperforms state-of-the-art U-Net and Transformer-based models on longitudinal AS dataset, achieving superior prostate zone segmentation with limited and noisy data.

Conclusion: The proposed architecture effectively addresses longitudinal segmentation challenges in Active Surveillance by leveraging temporal information and semi-supervised learning, enabling accurate prostate monitoring without extensive expert annotations.

Abstract: Active Surveillance (AS) is a treatment option for managing low and intermediate-risk prostate cancer (PCa), aiming to avoid overtreatment while monitoring disease progression through serial MRI and clinical follow-up. Accurate prostate segmentation is an important preliminary step for automating this process, enabling automated detection and diagnosis of PCa. However, existing deep-learning segmentation models are often trained on single-time-point and expertly annotated datasets, making them unsuitable for longitudinal AS analysis, where multiple time points and a scarcity of expert labels hinder their effective fine-tuning. To address these challenges, we propose MambaX-Net, a novel semi-supervised, dual-scan 3D segmentation architecture that computes the segmentation for time point t by leveraging the MRI and the corresponding segmentation mask from the previous time point. We introduce two new components: (i) a Mamba-enhanced Cross-Attention Module, which integrates the Mamba block into cross attention to efficiently capture temporal evolution and long-range spatial dependencies, and (ii) a Shape Extractor Module that encodes the previous segmentation mask into a latent anatomical representation for refined zone delination. Moreover, we introduce a semi-supervised self-training strategy that leverages pseudo-labels generated from a pre-trained nnU-Net, enabling effective learning without expert annotations. MambaX-Net was evaluated on a longitudinal AS dataset, and results showed that it significantly outperforms state-of-the-art U-Net and Transformer-based models, achieving superior prostate zone segmentation even when trained on limited and noisy data.

Method: End-to-end weakly-supervised framework with three components: classifier generating CAMs, reconstructor measuring feature inferability, and detector producing pixel-wise results. Uses adversarial learning between classifier and reconstructor, path-aware attention module (PAAM), and center-enhanced CAM consistency module (CECCM).

Result: Achieves comparable results to supervised methods and outperforms existing weakly-supervised methods on three created image-level datasets.

Conclusion: WP-CrackNet significantly advances scalable road inspection by enabling effective crack detection with minimal annotation requirements, making infrastructure maintenance more practical and cost-effective.

Abstract: Road crack detection is essential for intelligent infrastructure maintenance in smart cities. To reduce reliance on costly pixel-level annotations, we propose WP-CrackNet, an end-to-end weakly-supervised method that trains with only image-level labels for pixel-wise crack detection. WP-CrackNet integrates three components: a classifier generating class activation maps (CAMs), a reconstructor measuring feature inferability, and a detector producing pixel-wise road crack detection results. During training, the classifier and reconstructor alternate in adversarial learning to encourage crack CAMs to cover complete crack regions, while the detector learns from pseudo labels derived from post-processed crack CAMs. This mutual feedback among the three components improves learning stability and detection accuracy. To further boost detection performance, we design a path-aware attention module (PAAM) that fuses high-level semantics from the classifier with low-level structural cues from the reconstructor by modeling spatial and channel-wise dependencies. Additionally, a center-enhanced CAM consistency module (CECCM) is proposed to refine crack CAMs using center Gaussian weighting and consistency constraints, enabling better pseudo-label generation. We create three image-level datasets and extensive experiments show that WP-CrackNet achieves comparable results to supervised methods and outperforms existing weakly-supervised methods, significantly advancing scalable road inspection. The source code package and datasets are available at https://mias.group/WP-CrackNet/.

Method: Proposes a dynamics-aware aggregator that predicts a dynamics-aware mask to resolve task conflicts - suppressing motion cues for pose estimation while amplifying them for geometry reconstruction, enabling multi-task 4D reconstruction.

Result: Extensive experiments show PAGE-4D consistently outperforms VGGT in dynamic scenarios, achieving superior results in camera pose estimation, monocular/video depth estimation, and dense point map reconstruction.

Conclusion: PAGE-4D successfully extends static 3D models to dynamic scenes by addressing the inherent conflict between pose estimation and geometry reconstruction through dynamics-aware information disentanglement.

Abstract: Recent 3D feed-forward models, such as the Visual Geometry Grounded Transformer (VGGT), have shown strong capability in inferring 3D attributes of static scenes. However, since they are typically trained on static datasets, these models often struggle in real-world scenarios involving complex dynamic elements, such as moving humans or deformable objects like umbrellas. To address this limitation, we introduce PAGE-4D, a feedforward model that extends VGGT to dynamic scenes, enabling camera pose estimation, depth prediction, and point cloud reconstruction – all without post-processing. A central challenge in multi-task 4D reconstruction is the inherent conflict between tasks: accurate camera pose estimation requires suppressing dynamic regions, while geometry reconstruction requires modeling them. To resolve this tension, we propose a dynamics-aware aggregator that disentangles static and dynamic information by predicting a dynamics-aware mask – suppressing motion cues for pose estimation while amplifying them for geometry reconstruction. Extensive experiments show that PAGE-4D consistently outperforms the original VGGT in dynamic scenarios, achieving superior results in camera pose estimation, monocular and video depth estimation, and dense point map reconstruction.

Method: Proposed UCIS-SAM network with three modules: Channel Balance Optimization Module (CBOM) for enhanced underwater feature learning, Frequency Domain True Integration Module (FDTIM) to emphasize intrinsic object features and reduce camouflage interference, and Multi-scale Feature Frequency Aggregation Module (MFFAM) to strengthen boundaries of low-contrast camouflaged instances.

Result: Extensive experiments on the proposed UCIS4K dataset and public benchmarks show that UCIS-SAM outperforms state-of-the-art approaches in underwater camouflaged instance segmentation.

Conclusion: The UCIS4K dataset and UCIS-SAM network effectively address the challenges of underwater camouflaged instance segmentation, providing superior performance compared to existing methods through specialized modules for underwater feature enhancement and camouflage pattern handling.

Abstract: With the development of underwater exploration and marine protection, underwater vision tasks are widespread. Due to the degraded underwater environment, characterized by color distortion, low contrast, and blurring, camouflaged instance segmentation (CIS) faces greater challenges in accurately segmenting objects that blend closely with their surroundings. Traditional camouflaged instance segmentation methods, trained on terrestrial-dominated datasets with limited underwater samples, may exhibit inadequate performance in underwater scenes. To address these issues, we introduce the first underwater camouflaged instance segmentation (UCIS) dataset, abbreviated as UCIS4K, which comprises 3,953 images of camouflaged marine organisms with instance-level annotations. In addition, we propose an Underwater Camouflaged Instance Segmentation network based on Segment Anything Model (UCIS-SAM). Our UCIS-SAM includes three key modules. First, the Channel Balance Optimization Module (CBOM) enhances channel characteristics to improve underwater feature learning, effectively addressing the model’s limited understanding of underwater environments. Second, the Frequency Domain True Integration Module (FDTIM) is proposed to emphasize intrinsic object features and reduce interference from camouflage patterns, enhancing the segmentation performance of camouflaged objects blending with their surroundings. Finally, the Multi-scale Feature Frequency Aggregation Module (MFFAM) is designed to strengthen the boundaries of low-contrast camouflaged instances across multiple frequency bands, improving the model’s ability to achieve more precise segmentation of camouflaged objects. Extensive experiments on the proposed UCIS4K and public benchmarks show that our UCIS-SAM outperforms state-of-the-art approaches.

Method: Proposes Graph-based Procedural Shape (GPS) representation that decomposes natural language into structured sub-task graphs, using LLM agents to hierarchically parse input and iteratively refine procedural modeling and painting.

Result: Qualitative and quantitative experiments show superior performance in generating geometrically accurate and semantically rich 3D assets compared to existing LLM-based methods, with demonstrated versatility in animation and user-customized editing.

Conclusion: ShapeCraft enables practical text-to-3D generation with structured, textured, and interactive assets, showing potential for broader interactive applications.

Abstract: 3D generation from natural language offers significant potential to reduce expert manual modeling efforts and enhance accessibility to 3D assets. However, existing methods often yield unstructured meshes and exhibit poor interactivity, making them impractical for artistic workflows. To address these limitations, we represent 3D assets as shape programs and introduce ShapeCraft, a novel multi-agent framework for text-to-3D generation. At its core, we propose a Graph-based Procedural Shape (GPS) representation that decomposes complex natural language into a structured graph of sub-tasks, thereby facilitating accurate LLM comprehension and interpretation of spatial relationships and semantic shape details. Specifically, LLM agents hierarchically parse user input to initialize GPS, then iteratively refine procedural modeling and painting to produce structured, textured, and interactive 3D assets. Qualitative and quantitative experiments demonstrate ShapeCraft’s superior performance in generating geometrically accurate and semantically rich 3D assets compared to existing LLM-based agents. We further show the versatility of ShapeCraft through examples of animated and user-customized editing, highlighting its potential for broader interactive applications.

Method: Combines real-world UAV-scanned point clouds with synthetic data generated from Building Information Modeling (BIM) to train machine learning models for automated 3D point cloud segmentation.

Result: Achieved high accuracy in identifying and segmenting railroad track components (rails and crossties) and significantly reduced training time while maintaining reasonable segmentation accuracy using smaller datasets supplemented with BIM data.

Conclusion: The automated approach improves precision and efficiency of 3D infrastructure model segmentation and advances integration of UAV and BIM technologies for structural health monitoring and infrastructure management.

Abstract: The advancement of UAV technology has enabled efficient, non-contact structural health monitoring. Combined with photogrammetry, UAVs can capture high-resolution scans and reconstruct detailed 3D models of infrastructure. However, a key challenge remains in segmenting specific structural components from these models-a process traditionally reliant on time-consuming and error-prone manual labeling. To address this issue, we propose a machine learning-based framework for automated segmentation of 3D point clouds. Our approach uses the complementary strengths of real-world UAV-scanned point clouds and synthetic data generated from Building Information Modeling (BIM) to overcome the limitations associated with manual labeling. Validation on a railroad track dataset demonstrated high accuracy in identifying and segmenting major components such as rails and crossties. Moreover, by using smaller-scale datasets supplemented with BIM data, the framework significantly reduced training time while maintaining reasonable segmentation accuracy. This automated approach improves the precision and efficiency of 3D infrastructure model segmentation and advances the integration of UAV and BIM technologies in structural health monitoring and infrastructure management.

Method: A reconstruction-based framework guided by ’less is more’ philosophy, using orchestration of five simple elements to achieve superior performance without modification across diverse tasks.

Result: Achieves unprecedented 99.9% and 99.3% image-level AUROC on MVTec-AD and VisA for multi-class models, state-of-the-art performance in multi-view/multi-modal inspection, and surpasses previous full-shot models using only 8 normal examples per class (98.7% and 97.4% I-AUROC).

Conclusion: The combination of minimalistic design, computational scalability, and universal applicability positions Dinomaly2 as a unified solution for the full spectrum of real-world anomaly detection applications.

Abstract: Unsupervised anomaly detection (UAD) has evolved from building specialized single-class models to unified multi-class models, yet existing multi-class models significantly underperform the most advanced one-for-one counterparts. Moreover, the field has fragmented into specialized methods tailored to specific scenarios (multi-class, 3D, few-shot, etc.), creating deployment barriers and highlighting the need for a unified solution. In this paper, we present Dinomaly2, the first unified framework for full-spectrum image UAD, which bridges the performance gap in multi-class models while seamlessly extending across diverse data modalities and task settings. Guided by the “less is more” philosophy, we demonstrate that the orchestration of five simple element achieves superior performance in a standard reconstruction-based framework. This methodological minimalism enables natural extension across diverse tasks without modification, establishing that simplicity is the foundation of true universality. Extensive experiments on 12 UAD benchmarks demonstrate Dinomaly2’s full-spectrum superiority across multiple modalities (2D, multi-view, RGB-3D, RGB-IR), task settings (single-class, multi-class, inference-unified multi-class, few-shot) and application domains (industrial, biological, outdoor). For example, our multi-class model achieves unprecedented 99.9% and 99.3% image-level (I-) AUROC on MVTec-AD and VisA respectively. For multi-view and multi-modal inspection, Dinomaly2 demonstrates state-of-the-art performance with minimum adaptations. Moreover, using only 8 normal examples per class, our method surpasses previous full-shot models, achieving 98.7% and 97.4% I-AUROC on MVTec-AD and VisA. The combination of minimalistic design, computational scalability, and universal applicability positions Dinomaly2 as a unified solution for the full spectrum of real-world anomaly detection applications.

Method: Static pretraining on in-domain data; time-incremental classification with two strategies: updating backbone or final layer only; time-aware image generation using temporal metadata.

Result: Static pretraining achieves competitive performance but accuracy declines across years. Time-incremental approaches improve temporal robustness. Time-aware generation yields more realistic outputs.

Conclusion: CaMiT provides a benchmark for studying temporal adaptation in fine-grained visual recognition and generation, addressing the challenge of evolving object appearances over time.

Abstract: AI systems must adapt to evolving visual environments, especially in domains where object appearances change over time. We introduce Car Models in Time (CaMiT), a fine-grained dataset capturing the temporal evolution of car models, a representative class of technological artifacts. CaMiT includes 787K labeled samples of 190 car models (2007-2023) and 5.1M unlabeled samples (2005-2023), supporting both supervised and self-supervised learning. Static pretraining on in-domain data achieves competitive performance with large-scale generalist models while being more resource-efficient, yet accuracy declines when models are tested across years. To address this, we propose a time-incremental classification setting, a realistic continual learning scenario with emerging, evolving, and disappearing classes. We evaluate two strategies: time-incremental pretraining, which updates the backbone, and time-incremental classifier learning, which updates only the final layer, both improving temporal robustness. Finally, we explore time-aware image generation that leverages temporal metadata during training, yielding more realistic outputs. CaMiT offers a rich benchmark for studying temporal adaptation in fine-grained visual recognition and generation.

Method: Uses high-resolution Airborne LiDAR-derived DEMs to capture terrain structures beneath vegetation. Introduces self-supervised cross-view pre-training based on knowledge distillation to learn invariant representations across multiple DEM derivatives, supporting various vision backbones.

Result: Achieved 68.6% mIoU on test areas in Budj Bim cultural landscape, maintaining 63.8% mIoU with only 10% labeled data. Successfully identified dense colonial dry-stone walls beyond Indigenous heritage contexts.

Conclusion: Self-supervised learning on high-resolution DEM derivatives enables effective automated dry-stone wall mapping in vegetated, heritage-rich environments with limited annotations.

Abstract: Dry-stone walls hold significant heritage and environmental value. Mapping these structures is essential for ecosystem preservation and wildfire management in Australia. Yet, many walls remain unidentified due to their inaccessibility and the high cost of manual mapping. Deep learning-based segmentation offers a scalable solution, but two major challenges persist: (1) visual occlusion of low-lying walls by dense vegetation, and (2) limited labeled data for supervised training. We propose DINO-CV, a segmentation framework for automatic mapping of low-lying dry-stone walls using high-resolution Airborne LiDAR-derived digital elevation models (DEMs). DEMs overcome visual occlusion by capturing terrain structures hidden beneath vegetation, enabling analysis of structural rather than spectral cues. DINO-CV introduces a self-supervised cross-view pre-training strategy based on knowledge distillation to mitigate data scarcity. It learns invariant visual and geometric representations across multiple DEM derivatives, supporting various vision backbones including ResNet, Wide ResNet, and Vision Transformers. Applied to the UNESCO World Heritage cultural landscape of Budj Bim, Victoria, the method identifies one of Australia’s densest collections of colonial dry-stone walls beyond Indigenous heritage contexts. DINO-CV achieves a mean Intersection over Union (mIoU) of 68.6% on test areas and maintains 63.8% mIoU when fine-tuned with only 10% labeled data. These results demonstrate the potential of self-supervised learning on high-resolution DEM derivatives for automated dry-stone wall mapping in vegetated and heritage-rich environments with scarce annotations.

Method: Two complementary strategies: (1) zero-shot and federated fine-tuning of vision-language models (LLaVA-7B) using LoRA, and (2) personalized training of a compact 3D CNN (65.8M parameters). Evaluation includes accuracy, calibration, energy usage, and CO2 emissions under realistic non-IID conditions.

Result: Both approaches exceed 90% accuracy. CNN3D slightly outperforms LoRA-tuned VLMs in ROC AUC and log loss while using less energy. VLMs remain better for contextual reasoning and multimodal inference. Energy and CO2 emissions were quantified across training and inference phases.

Conclusion: A hybrid model is recommended: lightweight CNNs for routine classification with selective VLM activation for complex scenarios. The framework provides a reproducible baseline for responsible, resource-aware AI in video surveillance with extensions toward real-time, multimodal systems.

Abstract: We examine frugal federated learning approaches to violence detection by comparing two complementary strategies: (i) zero-shot and federated fine-tuning of vision-language models (VLMs), and (ii) personalized training of a compact 3D convolutional neural network (CNN3D). Using LLaVA-7B and a 65.8M parameter CNN3D as representative cases, we evaluate accuracy, calibration, and energy usage under realistic non-IID settings. Both approaches exceed 90% accuracy. CNN3D slightly outperforms Low-Rank Adaptation(LoRA)-tuned VLMs in ROC AUC and log loss, while using less energy. VLMs remain favorable for contextual reasoning and multimodal inference. We quantify energy and CO$_2$ emissions across training and inference, and analyze sustainability trade-offs for deployment. To our knowledge, this is the first comparative study of LoRA-tuned vision-language models and personalized CNNs for federated violence detection, with an emphasis on energy efficiency and environmental metrics. These findings support a hybrid model: lightweight CNNs for routine classification, with selective VLM activation for complex or descriptive scenarios. The resulting framework offers a reproducible baseline for responsible, resource-aware AI in video surveillance, with extensions toward real-time, multimodal, and lifecycle-aware systems.

Method: Uses a Dual-Thread System with a predictive thread that leverages historical motion/geometric data to forecast future dynamics, and an inference thread that ensures timely predictions by aligning with latest memory while compensating for ego-motion and object movements.

Result: Demonstrates superior robustness under high FPS conditions, accurately predicts dynamic objects in complex scenes, and shows effectiveness on both indoor (HOI4D) and outdoor (SemanticKITTI, nuScenes) datasets.

Conclusion: 4DSegStreamer provides a general framework that can be integrated into existing 3D/4D segmentation methods to enable real-time capability while maintaining robust performance in dynamic streaming environments.

Abstract: 4D panoptic segmentation in a streaming setting is critical for highly dynamic environments, such as evacuating dense crowds and autonomous driving in complex scenarios, where real-time, fine-grained perception within a constrained time budget is essential. In this paper, we introduce 4DSegStreamer, a novel framework that employs a Dual-Thread System to efficiently process streaming frames. The framework is general and can be seamlessly integrated into existing 3D and 4D segmentation methods to enable real-time capability. It also demonstrates superior robustness compared to existing streaming perception approaches, particularly under high FPS conditions. The system consists of a predictive thread and an inference thread. The predictive thread leverages historical motion and geometric information to extract features and forecast future dynamics. The inference thread ensures timely prediction for incoming frames by aligning with the latest memory and compensating for ego-motion and dynamic object movements. We evaluate 4DSegStreamer on the indoor HOI4D dataset and the outdoor SemanticKITTI and nuScenes datasets. Comprehensive experiments demonstrate the effectiveness of our approach, particularly in accurately predicting dynamic objects in complex scenes.

Method: Systematic evaluation across eight physical sub-dimensions (optics, mechanics, state transitions) for common editing operations. Uses VLM-as-a-judge with region-level human annotations and questions. Explores learning physics from videos and builds PICA-100K dataset.

Result: Evaluation of mainstream models shows physical realism remains challenging with significant room for improvement.

Conclusion: Physical realism is a key challenge in image editing. The benchmark and proposed solutions provide a foundation for moving from naive content editing to physically consistent realism.

Abstract: Image editing has achieved remarkable progress recently. Modern editing models could already follow complex instructions to manipulate the original content. However, beyond completing the editing instructions, the accompanying physical effects are the key to the generation realism. For example, removing an object should also remove its shadow, reflections, and interactions with nearby objects. Unfortunately, existing models and benchmarks mainly focus on instruction completion but overlook these physical effects. So, at this moment, how far are we from physically realistic image editing? To answer this, we introduce PICABench, which systematically evaluates physical realism across eight sub-dimension (spanning optics, mechanics, and state transitions) for most of the common editing operations (add, remove, attribute change, etc). We further propose the PICAEval, a reliable evaluation protocol that uses VLM-as-a-judge with per-case, region-level human annotations and questions. Beyond benchmarking, we also explore effective solutions by learning physics from videos and construct a training dataset PICA-100K. After evaluating most of the mainstream models, we observe that physical realism remains a challenging problem with large rooms to explore. We hope that our benchmark and proposed solutions can serve as a foundation for future work moving from naive content editing toward physically consistent realism.

Method: 1) Constructs three types of experts (basic, semantic, adaptive) with pixel probability adaptive voting for expert selection and fusion. 2) Uses semantic-guided contrastive learning to address weak supervision in contrastive learning.

Result: Extensive experiments on three public medical image segmentation datasets show IC-MoE outperforms other state-of-the-art models and demonstrates superior generalizability across diverse medical image segmentation scenarios.

Conclusion: IC-MoE effectively supplements foundational medical image segmentation models with enhanced high-level features and preserved pretrained structural integrity.

Abstract: Foundation models for medical image segmentation have achieved remarkable performance. Adaptive fine-tuning of natural image segmentation foundation models is crucial for medical image segmentation tasks. However, some limitations exist in existing fine-tuning methods: 1) insufficient representation of high-level features and 2) the fine-tuning process disrupts the structural integrity of pretrained weights. Inspired by these critical problems, we propose an intelligent communication mixture-of-experts boosted-medical image segmentation foundation model, named IC-MoE, with twofold ideas: 1) We construct basic experts, semantic experts, and adaptive experts. Moreover, we implement a pixel probability adaptive voting strategy, which enables expert selection and fusion through label consistency and load balancing. This approach preliminarily enhances the representation capability of high-level features while preserving the structural integrity of pretrained weights. 2) We propose a semantic-guided contrastive learning method to address the issue of weak supervision in contrastive learning. This method further enhances the representation capability of high-level features while preserving the structural integrity of pretrained weights. Extensive experiments across three public medical image segmentation datasets demonstrate that the IC-MoE outperforms other SOTA models. Consequently, the proposed IC-MoE effectively supplements foundational medical image segmentation models with high-level features and pretrained structural integrity. We also validate the superior generalizability of the IC-MoE across diverse medical image segmentation scenarios.

Method: Develops Bi-IRRA framework with bidirectional implicit relation reasoning module for masked image/text prediction and multi-dimensional global alignment module to bridge modality heterogeneity. Also creates multilingual TIPR benchmark using LLMs with domain-specific refinement.

Result: Achieves new state-of-the-art results on all multilingual TIPR datasets.

Conclusion: Bi-IRRA effectively addresses multilingual TIPR challenges through implicit relation reasoning and global alignment, providing a robust solution for cross-modal person retrieval in multilingual contexts.

Abstract: Text-to-image person retrieval (TIPR) aims to identify the target person using textual descriptions, facing challenge in modality heterogeneity. Prior works have attempted to address it by developing cross-modal global or local alignment strategies. However, global methods typically overlook fine-grained cross-modal differences, whereas local methods require prior information to explore explicit part alignments. Additionally, current methods are English-centric, restricting their application in multilingual contexts. To alleviate these issues, we pioneer a multilingual TIPR task by developing a multilingual TIPR benchmark, for which we leverage large language models for initial translations and refine them by integrating domain-specific knowledge. Correspondingly, we propose Bi-IRRA: a Bidirectional Implicit Relation Reasoning and Aligning framework to learn alignment across languages and modalities. Within Bi-IRRA, a bidirectional implicit relation reasoning module enables bidirectional prediction of masked image and text, implicitly enhancing the modeling of local relations across languages and modalities, a multi-dimensional global alignment module is integrated to bridge the modality heterogeneity. The proposed method achieves new state-of-the-art results on all multilingual TIPR datasets. Data and code are presented in https://github.com/Flame-Chasers/Bi-IRRA.

Method: Proposes OP3Det using 2D semantic priors and 3D geometric priors for class-agnostic proposals. Integrates complementary information from point cloud and RGB images through cross-modal mixture of experts, dynamically routing uni-modal and multi-modal features to learn generalized 3D objectness.

Result: Significantly surpasses existing open-world 3D detectors by up to 16.0% in AR and achieves 13.5% improvement compared to closed-world 3D detectors.

Conclusion: OP3Det demonstrates extraordinary performance in open-world 3D object detection, effectively addressing the limitations of traditional closed-set detectors and existing open-vocabulary approaches through its prompt-free, class-agnostic design and cross-modal fusion strategy.

Abstract: Recent advancements in 3D object detection and novel category detection have made significant progress, yet research on learning generalized 3D objectness remains insufficient. In this paper, we delve into learning open-world 3D objectness, which focuses on detecting all objects in a 3D scene, including novel objects unseen during training. Traditional closed-set 3D detectors struggle to generalize to open-world scenarios, while directly incorporating 3D open-vocabulary models for open-world ability struggles with vocabulary expansion and semantic overlap. To achieve generalized 3D object discovery, We propose OP3Det, a class-agnostic Open-World Prompt-free 3D Detector to detect any objects within 3D scenes without relying on hand-crafted text prompts. We introduce the strong generalization and zero-shot capabilities of 2D foundation models, utilizing both 2D semantic priors and 3D geometric priors for class-agnostic proposals to broaden 3D object discovery. Then, by integrating complementary information from point cloud and RGB image in the cross-modal mixture of experts, OP3Det dynamically routes uni-modal and multi-modal features to learn generalized 3D objectness. Extensive experiments demonstrate the extraordinary performance of OP3Det, which significantly surpasses existing open-world 3D detectors by up to 16.0% in AR and achieves a 13.5% improvement compared to closed-world 3D detectors.

Method: Proposes Generalized Solver - a simple parameterization of ODE sampler without additional training tricks, combined with adversarial training to mitigate artifacts and enhance detail fidelity. The method uses both original distillation loss and adversarial training.

Result: The Generalized Adversarial Solver demonstrates superior performance compared to existing solver training methods under similar resource constraints, reducing function evaluations from dozens to just a few while improving quality.

Conclusion: The proposed method provides an effective solution for efficient diffusion model sampling that maintains generation quality and preserves fine details through a combination of simple parameterization and adversarial training.

Abstract: While diffusion models achieve state-of-the-art generation quality, they still suffer from computationally expensive sampling. Recent works address this issue with gradient-based optimization methods that distill a few-step ODE diffusion solver from the full sampling process, reducing the number of function evaluations from dozens to just a few. However, these approaches often rely on intricate training techniques and do not explicitly focus on preserving fine-grained details. In this paper, we introduce the Generalized Solver: a simple parameterization of the ODE sampler that does not require additional training tricks and improves quality over existing approaches. We further combine the original distillation loss with adversarial training, which mitigates artifacts and enhances detail fidelity. We call the resulting method the Generalized Adversarial Solver and demonstrate its superior performance compared to existing solver training methods under similar resource constraints. Code is available at https://github.com/3145tttt/GAS.

Method: Two-stage approach: first classifies drawing types (circle, meander, spiral), then extracts features from 2x2 image chunks processed separately. Uses ensemble method to merge decisions from each chunk.

Result: Achieved 97.08% accuracy for seen patients and 94.91% for unseen patients on NewHandPD dataset, with only 2.17 percentage point gap compared to 4.76-point drop in prior work.

Conclusion: The proposed chunking and ensemble approach significantly improves robustness and generalization for Parkinson’s disease detection, particularly on unseen patient data.

Abstract: Parkinson’s disease (PD) is a neurodegenerative disease affecting about 1% of people over the age of 60, causing motor impairments that impede hand coordination activities such as writing and drawing. Many approaches have tried to support early detection of Parkinson’s disease based on hand-drawn images; however, we identified two major limitations in the related works: (1) the lack of sufficient datasets, (2) the robustness when dealing with unseen patient data. In this paper, we propose a new approach to detect Parkinson’s disease that consists of two stages: The first stage classifies based on their drawing type(circle, meander, spiral), and the second stage extracts the required features from the images and detects Parkinson’s disease. We overcame the previous two limitations by applying a chunking strategy where we divide each image into 2x2 chunks. Each chunk is processed separately when extracting features and recognizing Parkinson’s disease indicators. To make the final classification, an ensemble method is used to merge the decisions made from each chunk. Our evaluation shows that our proposed approach outperforms the top performing state-of-the-art approaches, in particular on unseen patients. On the NewHandPD dataset our approach, it achieved 97.08% accuracy for seen patients and 94.91% for unseen patients, our proposed approach maintained a gap of only 2.17 percentage points, compared to the 4.76-point drop observed in prior work.

Method: Combines gradient information with cross-network structure correlations approximated via evolutionary algorithm, uses self-supervised importance scoring, and is retraining-free.

Result: Superior performance over state-of-the-art methods across various sparsities, generates elastic models in <5 minutes on single A100 GPU that can be adjusted to any computational budget.

Conclusion: SnapViT provides an efficient pruning strategy for pretrained Vision Transformers with novel evolutionary approximation of Hessian structures, enabling flexible deployment without performance degradation.

Abstract: Vision foundation models achieve remarkable performance but are only available in a limited set of pre-determined sizes, forcing sub-optimal deployment choices under real-world constraints. We introduce SnapViT: Single-shot network approximation for pruned Vision Transformers, a new post-pretraining structured pruning method that enables elastic inference across a continuum of compute budgets. Our approach efficiently combines gradient information with cross-network structure correlations, approximated via an evolutionary algorithm, does not require labeled data, generalizes to models without a classification head, and is retraining-free. Experiments on DINO, SigLIPv2, DeIT, and AugReg models demonstrate superior performance over state-of-the-art methods across various sparsities, requiring less than five minutes on a single A100 GPU to generate elastic models that can be adjusted to any computational budget. Our key contributions include an efficient pruning strategy for pretrained Vision Transformers, a novel evolutionary approximation of Hessian off-diagonal structures, and a self-supervised importance scoring mechanism that maintains strong performance without requiring retraining or labels. Code and pruned models are available at: https://elastic.ashita.nl/

Method: Two-step analysis: 1) Fine-tuned YOLOv11 model classifies images into cell cluster vs non-cluster groups, 2) Cell type identification by overlaying cluster contours with multi-channel fluorescence stain regions to handle debris and artifacts.

Result: Achieved over 95% accuracy in both cluster classification and phenotype identification, outperforming traditional CNNs and Vision Transformers.

Conclusion: The automated framework effectively analyzes CCC images and has potential for broader applications in analyzing immune and tumor cell clusters across various diseases.

Abstract: Circulating blood cell clusters (CCCs) containing red blood cells (RBCs), white blood cells(WBCs), and platelets are significant biomarkers linked to conditions like thrombosis, infection, and inflammation. Flow cytometry, paired with fluorescence staining, is commonly used to analyze these cell clusters, revealing cell morphology and protein profiles. While computational approaches based on machine learning have advanced the automatic analysis of single-cell flow cytometry images, there is a lack of effort to build tools to automatically analyze images containing CCCs. Unlike single cells, cell clusters often exhibit irregular shapes and sizes. In addition, these cell clusters often consist of heterogeneous cell types, which require multi-channel staining to identify the specific cell types within the clusters. This study introduces a new computational framework for analyzing CCC images and identifying cell types within clusters. Our framework uses a two-step analysis strategy. First, it categorizes images into cell cluster and non-cluster groups by fine-tuning the You Only Look Once(YOLOv11) model, which outperforms traditional convolutional neural networks (CNNs), Vision Transformers (ViT). Then, it identifies cell types by overlaying cluster contours with regions from multi-channel fluorescence stains, enhancing accuracy despite cell debris and staining artifacts. This approach achieved over 95% accuracy in both cluster classification and phenotype identification. In summary, our automated framework effectively analyzes CCC images from flow cytometry, leveraging both bright-field and fluorescence data. Initially tested on blood cells, it holds potential for broader applications, such as analyzing immune and tumor cell clusters, supporting cellular research across various diseases.

Method: Created MT-Video-Bench with 987 meticulously curated multi-turn dialogues from diverse domains, assessing six core competencies focused on perceptivity and interactivity, aligned with real-world applications like sports analysis and video tutoring.

Result: Extensive evaluation of state-of-the-art open-source and closed-source MLLMs revealed significant performance discrepancies and limitations in handling multi-turn video dialogues.

Conclusion: The benchmark exposes current MLLMs’ shortcomings in multi-turn video understanding and will be publicly available to advance future research in this area.

Abstract: The recent development of Multimodal Large Language Models (MLLMs) has significantly advanced AI’s ability to understand visual modalities. However, existing evaluation benchmarks remain limited to single-turn question answering, overlooking the complexity of multi-turn dialogues in real-world scenarios. To bridge this gap, we introduce MT-Video-Bench, a holistic video understanding benchmark for evaluating MLLMs in multi-turn dialogues. Specifically, our MT-Video-Bench mainly assesses six core competencies that focus on perceptivity and interactivity, encompassing 987 meticulously curated multi-turn dialogues from diverse domains. These capabilities are rigorously aligned with real-world applications, such as interactive sports analysis and multi-turn video-based intelligent tutoring. With MT-Video-Bench, we extensively evaluate various state-of-the-art open-source and closed-source MLLMs, revealing their significant performance discrepancies and limitations in handling multi-turn video dialogues. The benchmark will be publicly available to foster future research.

Method: Created RaindropGS benchmark with three components: data preparation (collecting real-world dataset with raindrop-focused, background-focused, and rain-free images), data processing, and raindrop-aware 3DGS evaluation including pose estimation, point cloud initialization, rain removal, and Gaussian training comparison.

Result: Revealed critical insights: camera focus position significantly impacts 3DGS reconstruction performance, and inaccurate pose/point cloud initialization interferes with reconstruction quality under raindrop conditions.

Conclusion: The benchmark establishes clear directions for developing more robust 3DGS methods by identifying performance limitations and the varying impact of different pipeline components in real-world raindrop scenarios.

Abstract: 3D Gaussian Splatting (3DGS) under raindrop conditions suffers from severe occlusions and optical distortions caused by raindrop contamination on the camera lens, substantially degrading reconstruction quality. Existing benchmarks typically evaluate 3DGS using synthetic raindrop images with known camera poses (constrained images), assuming ideal conditions. However, in real-world scenarios, raindrops often interfere with accurate camera pose estimation and point cloud initialization. Moreover, a significant domain gap between synthetic and real raindrops further impairs generalization. To tackle these issues, we introduce RaindropGS, a comprehensive benchmark designed to evaluate the full 3DGS pipeline-from unconstrained, raindrop-corrupted images to clear 3DGS reconstructions. Specifically, the whole benchmark pipeline consists of three parts: data preparation, data processing, and raindrop-aware 3DGS evaluation, including types of raindrop interference, camera pose estimation and point cloud initialization, single image rain removal comparison, and 3D Gaussian training comparison. First, we collect a real-world raindrop reconstruction dataset, in which each scene contains three aligned image sets: raindrop-focused, background-focused, and rain-free ground truth, enabling a comprehensive evaluation of reconstruction quality under different focus conditions. Through comprehensive experiments and analyses, we reveal critical insights into the performance limitations of existing 3DGS methods on unconstrained raindrop images and the varying impact of different pipeline components: the impact of camera focus position on 3DGS reconstruction performance, and the interference caused by inaccurate pose and point cloud initialization on reconstruction. These insights establish clear directions for developing more robust 3DGS methods under raindrop conditions.

Method: Developed two experimental pipelines: one using raw signature images and another using shell preprocessing. Tested on three public benchmarks (CEDAR, ICDAR, and GPDS Synthetic) to evaluate cross-dataset performance.

Result: The raw-image model achieved higher performance across benchmarks, while the shell-based model showed promising potential but no definitive superiority was established between the two approaches.

Conclusion: Both approaches have merits, with raw-image processing currently performing better, but shell preprocessing offers potential for future refinement toward robust cross-domain signature verification.

Abstract: Automated signature verification is a critical biometric technique used in banking, identity authentication, and legal documentation. Despite the notable progress achieved by deep learning methods, most approaches in offline signature verification still struggle to generalize across datasets, as variations in handwriting styles and acquisition protocols often degrade performance. This study investigates feature learning strategies for signature forgery detection, focusing on improving cross-dataset generalization – that is, model robustness when trained on one dataset and tested on another. Using three public benchmarks – CEDAR, ICDAR, and GPDS Synthetic – two experimental pipelines were developed: one based on raw signature images and another employing a preprocessing method referred to as shell preprocessing. Several behavioral patterns were identified and analyzed; however, no definitive superiority between the two approaches was established. The results show that the raw-image model achieved higher performance across benchmarks, while the shell-based model demonstrated promising potential for future refinement toward robust, cross-domain signature verification.

Method: Conditions I2V models on keyframes from pedestrian trajectory benchmarks and evaluates trajectory prediction performance using quantitative pedestrian dynamics measures.

Result: Not specified in the abstract - appears to be a research investigation rather than reporting completed results.

Conclusion: The study explores the potential of I2V models for generating realistic pedestrian movement patterns, suggesting these models may have applications in pedestrian dynamics modeling.

Abstract: Recent high-performing image-to-video (I2V) models based on variants of the diffusion transformer (DiT) have displayed remarkable inherent world-modeling capabilities by virtue of training on large scale video datasets. We investigate whether these models can generate realistic pedestrian movement patterns in crowded public scenes. Our framework conditions I2V models on keyframes extracted from pedestrian trajectory benchmarks, then evaluates their trajectory prediction performance using quantitative measures of pedestrian dynamics.

Method: Proposes a training-free, descriptor-agnostic approach that jointly models places using multiple reference descriptors via matrix decomposition into basis representations, enabling projection-based residual matching.

Result: Improves Recall@1 by up to ~18% over single-reference methods and outperforms multi-reference baselines across appearance and viewpoint changes, with ~5% gains on unstructured data.

Conclusion: The method demonstrates strong generalization while remaining lightweight, providing effective multi-reference visual place recognition without requiring training.

Abstract: We address multi-reference visual place recognition (VPR), where reference sets captured under varying conditions are used to improve localisation performance. While deep learning with large-scale training improves robustness, increasing data diversity and model complexity incur extensive computational cost during training and deployment. Descriptor-level fusion via voting or aggregation avoids training, but often targets multi-sensor setups or relies on heuristics with limited gains under appearance and viewpoint change. We propose a training-free, descriptor-agnostic approach that jointly models places using multiple reference descriptors via matrix decomposition into basis representations, enabling projection-based residual matching. We also introduce SotonMV, a structured benchmark for multi-viewpoint VPR. On multi-appearance data, our method improves Recall@1 by up to ~18% over single-reference and outperforms multi-reference baselines across appearance and viewpoint changes, with gains of ~5% on unstructured data, demonstrating strong generalisation while remaining lightweight.

Method: Uses Deep-UNet with Dual Attention Gates and ASPP for lesion segmentation, then dual DenseNet201 encoders with multi-head cross-attention for classification, plus transformer-based metadata integration.

Result: Achieves state-of-the-art segmentation performance and significantly improves classification accuracy and average AUC on HAM10000, ISIC 2018 and 2019 datasets, with Grad-CAM confirming lesion-focused predictions.

Conclusion: Integrating precise lesion segmentation and clinical data with attention-based fusion creates a more accurate and interpretable skin cancer classification model.

Abstract: Skin cancer is a life-threatening disease where early detection significantly improves patient outcomes. Automated diagnosis from dermoscopic images is challenging due to high intra-class variability and subtle inter-class differences. Many deep learning models operate as “black boxes,” limiting clinical trust. In this work, we propose a dual-encoder attention-based framework that leverages both segmented lesions and clinical metadata to enhance skin lesion classification in terms of both accuracy and interpretability. A novel Deep-UNet architecture with Dual Attention Gates (DAG) and Atrous Spatial Pyramid Pooling (ASPP) is first employed to segment lesions. The classification stage uses two DenseNet201 encoders-one on the original image and another on the segmented lesion whose features are fused via multi-head cross-attention. This dual-input design guides the model to focus on salient pathological regions. In addition, a transformer-based module incorporates patient metadata (age, sex, lesion site) into the prediction. We evaluate our approach on the HAM10000 dataset and the ISIC 2018 and 2019 challenges. The proposed method achieves state-of-the-art segmentation performance and significantly improves classification accuracy and average AUC compared to baseline models. To validate our model’s reliability, we use Gradient-weighted Class Activation Mapping (Grad-CAM) to generate heatmaps. These visualizations confirm that our model’s predictions are based on the lesion area, unlike models that rely on spurious background features. These results demonstrate that integrating precise lesion segmentation and clinical data with attention-based fusion leads to a more accurate and interpretable skin cancer classification model.

Method: SparseVILA decouples visual sparsity by pruning redundant visual tokens during prefill and retrieving only query-relevant tokens during decoding, built on an AWQ-optimized inference pipeline.

Result: Achieves up to 4.0× faster prefilling, 2.5× faster decoding, and 2.6× overall end-to-end speedup on long-context video tasks while improving accuracy on document-understanding and reasoning tasks.

Conclusion: SparseVILA establishes a new direction for efficient multimodal inference with a training-free, architecture-agnostic framework that accelerates large VLMs without sacrificing capability.

Abstract: Vision Language Models (VLMs) have rapidly advanced in integrating visual and textual reasoning, powering applications across high-resolution image understanding, long-video analysis, and multi-turn conversation. However, their scalability remains limited by the growing number of visual tokens that dominate inference latency. We present SparseVILA, a new paradigm for efficient VLM inference that decouples visual sparsity across the prefilling and decoding stages. SparseVILA distributes sparsity across stages by pruning redundant visual tokens during prefill and retrieving only query-relevant tokens during decoding. This decoupled design matches leading prefill pruning methods while preserving multi-turn fidelity by retaining most of the visual cache so that query-aware tokens can be retrieved at each conversation round. Built on an AWQ-optimized inference pipeline, SparseVILA achieves up to 4.0 times faster prefilling, 2.5 times faster decoding, and an overall 2.6 times end-to-end speedup on long-context video tasks – while improving accuracy on document-understanding and reasoning tasks. By decoupling query-agnostic pruning and query-aware retrieval, SparseVILA establishes a new direction for efficient multimodal inference, offering a training-free, architecture-agnostic framework for accelerating large VLMs without sacrificing capability.

Method: Proposes ConsistEdit with three key components: vision-only attention control, mask-guided pre-attention fusion, and differentiated manipulation of query, key, and value tokens. Works across all inference steps and attention layers without handcraft.

Result: Achieves state-of-the-art performance across image and video editing tasks in both structure-consistent and structure-inconsistent scenarios. Enables robust multi-round and multi-region editing with progressive structural consistency adjustment.

Conclusion: ConsistEdit is the first method to perform editing across all inference steps and attention layers without manual intervention, significantly enhancing reliability and consistency for complex editing tasks.

Abstract: Recent advances in training-free attention control methods have enabled flexible and efficient text-guided editing capabilities for existing generation models. However, current approaches struggle to simultaneously deliver strong editing strength while preserving consistency with the source. This limitation becomes particularly critical in multi-round and video editing, where visual errors can accumulate over time. Moreover, most existing methods enforce global consistency, which limits their ability to modify individual attributes such as texture while preserving others, thereby hindering fine-grained editing. Recently, the architectural shift from U-Net to MM-DiT has brought significant improvements in generative performance and introduced a novel mechanism for integrating text and vision modalities. These advancements pave the way for overcoming challenges that previous methods failed to resolve. Through an in-depth analysis of MM-DiT, we identify three key insights into its attention mechanisms. Building on these, we propose ConsistEdit, a novel attention control method specifically tailored for MM-DiT. ConsistEdit incorporates vision-only attention control, mask-guided pre-attention fusion, and differentiated manipulation of the query, key, and value tokens to produce consistent, prompt-aligned edits. Extensive experiments demonstrate that ConsistEdit achieves state-of-the-art performance across a wide range of image and video editing tasks, including both structure-consistent and structure-inconsistent scenarios. Unlike prior methods, it is the first approach to perform editing across all inference steps and attention layers without handcraft, significantly enhancing reliability and consistency, which enables robust multi-round and multi-region editing. Furthermore, it supports progressive adjustment of structural consistency, enabling finer control.

Method: FCN uses an outfit encoder with convolutional layers to create outfit embeddings and a Multi-label Graph Neural Network (ML-GCN) to learn label classifiers via stacked GCN, modeling relationships between outfit compositions and personal appearance features.

Result: Extensive experiments on the O4U dataset show FCN provides strong qualitative and quantitative improvements over alternative methods.

Conclusion: The proposed Fashion Cognitive Network effectively addresses personalized fashion recommendation by incorporating personal physical information, demonstrating superior performance compared to existing approaches.

Abstract: Fashion compatibility models enable online retailers to easily obtain a large number of outfit compositions with good quality. However, effective fashion recommendation demands precise service for each customer with a deeper cognition of fashion. In this paper, we conduct the first study on fashion cognitive learning, which is fashion recommendations conditioned on personal physical information. To this end, we propose a Fashion Cognitive Network (FCN) to learn the relationships among visual-semantic embedding of outfit composition and appearance features of individuals. FCN contains two submodules, namely outfit encoder and Multi-label Graph Neural Network (ML-GCN). The outfit encoder uses a convolutional layer to encode an outfit into an outfit embedding. The latter module learns label classifiers via stacked GCN. We conducted extensive experiments on the newly collected O4U dataset, and the results provide strong qualitative and quantitative evidence that our framework outperforms alternative methods.

Method: Uses event cameras for low energy consumption and small memory bandwidth. Applies event-to-image conversion to leverage mature image-based localization. Introduces two-level privacy protection: network level (split inference to hide user’s view) and sensor level (light-weight filtering to hide sensitive details like faces).

Result: Achieves robustness in low-light or fast-moving scenes. Provides significant privacy protection with small client-side computation and minimal localization performance loss. User study shows reduced feelings of insecurity.

Conclusion: The method serves as a practical building block for location-based services using event cameras, effectively addressing efficiency, robustness, and privacy challenges in client-server localization.

Abstract: We consider the problem of client-server localization, where edge device users communicate visual data with the service provider for locating oneself against a pre-built 3D map. This localization paradigm is a crucial component for location-based services in AR/VR or mobile applications, as it is not trivial to store large-scale 3D maps and process fast localization on resource-limited edge devices. Nevertheless, conventional client-server localization systems possess numerous challenges in computational efficiency, robustness, and privacy-preservation during data transmission. Our work aims to jointly solve these challenges with a localization pipeline based on event cameras. By using event cameras, our system consumes low energy and maintains small memory bandwidth. Then during localization, we propose applying event-to-image conversion and leverage mature image-based localization, which achieves robustness even in low-light or fast-moving scenes. To further enhance privacy protection, we introduce privacy protection techniques at two levels. Network level protection aims to hide the entire user’s view in private scenes using a novel split inference approach, while sensor level protection aims to hide sensitive user details such as faces with light-weight filtering. Both methods involve small client-side computation and localization performance loss, while significantly mitigating the feeling of insecurity as revealed in our user study. We thus project our method to serve as a building block for practical location-based services using event cameras. Project page including the code is available through this link: https://82magnolia.github.io/event_localization/.

Method: Training-free, GPU-accelerated multi-scale Adaptive Riemannian Optimization algorithm for dense diffeomorphic image matching.

Result: FireANTs runs 2.5x faster than ANTs on CPU and up to 1200x faster on GPU. On GPU, it competes with deep learning methods in runtime while using 10x less memory. Shows robustness across modalities, species, and organs without domain-specific training.

Conclusion: FireANTs provides a fast, memory-efficient alternative to both traditional and deep learning registration methods, enabling hyperparameter grid search with significantly reduced resources and time.

Abstract: The paper proposes FireANTs, a multi-scale Adaptive Riemannian Optimization algorithm for dense diffeomorphic image matching. Existing state-of-the-art methods for diffeomorphic image matching are slow due to inefficient implementations and slow convergence due to the ill-conditioned nature of the optimization problem. Deep learning methods offer fast inference but require extensive training time, substantial inference memory, and fail to generalize across long-tailed distributions or diverse image modalities, necessitating costly retraining. We address these challenges by proposing a training-free, GPU-accelerated multi-scale Adaptive Riemannian Optimization algorithm for fast and accurate dense diffeomorphic image matching. FireANTs runs about 2.5x faster than ANTs on a CPU, and upto 1200x faster on a GPU. On a single GPU, FireANTs performs competitively with deep learning methods on inference runtime while consuming upto 10x less memory. FireANTs shows remarkable robustness to a wide variety of matching problems across modalities, species, and organs without any domain-specific training or tuning. Our framework allows hyperparameter grid search studies with significantly less resources and time compared to traditional and deep learning registration algorithms alike.

Method: Represent high dynamic range depth as two low dynamic range components of a Hilbert curve, train full-precision DNN to predict these components, then use quantization with post-processing to reconstruct depth from low-bit predictions.

Result: Method increases bit precision by up to 3 bits with little computational overhead, reduces quantization error by up to 4.6 times, and enables effective 8-bit depth prediction.

Conclusion: The proposed Hilbert curve representation enables accurate depth prediction with 8-bit quantized DNNs, overcoming limitations of low-bit precision for high dynamic range depth.

Abstract: Dense depth prediction deep neural networks (DNN) have achieved impressive results for both monocular and binocular data, but still they are limited by high computational complexity, restricting their use on low-end devices. For better on-device efficiency and hardware utilization, weights and activations of the DNN should be converted to low-bit precision. However, this precision is not sufficient to represent high dynamic range depth. In this paper, we aim to overcome this limitation and restore high-precision depth from low-bit precision predictions. To achieve this, we propose to represent high dynamic range depth as two low dynamic range components of a Hilbert curve, and to train the full-precision DNN to directly predict the latter. For on-device deployment, we use standard quantization methods and add a post-processing step that reconstructs depth from the Hilbert curve components predicted in low-bit precision. Extensive experiments demonstrate that our method increases the bit precision of predicted depth by up to three bits with little computational overhead. We also observed a positive side effect of quantization error reduction by up to 4.6 times. Our method enables effective and accurate depth prediction with DNN weights and activations quantized to eight-bit precision.

Method: Explicitly rearranging features according to dense semantic correspondences rather than using query-key similarity in self-attention layers.

Result: Method shows superiority in preserving target structure and reflecting correct colors from reference, even when images are not aligned.

Conclusion: The proposed explicit feature rearrangement based on semantic correspondences outperforms existing methods for appearance transfer tasks.

Abstract: As pre-trained text-to-image diffusion models have become a useful tool for image synthesis, people want to specify the results in various ways. This paper tackles training-free appearance transfer, which produces an image with the structure of a target image from the appearance of a reference image. Existing methods usually do not reflect semantic correspondence, as they rely on query-key similarity within the self-attention layer to establish correspondences between images. To this end, we propose explicitly rearranging the features according to the dense semantic correspondences. Extensive experiments show the superiority of our method in various aspects: preserving the structure of the target and reflecting the correct color from the reference, even when the two images are not aligned.

Method: Uses CLIP-based network to quantify locatability of street-view images, creates new dataset of highly locatable views, integrates human inference knowledge from geo-localization games, and trains GeoReasoner through dedicated reasoning and location-tuning stages.

Result: Outperforms counterpart LVLMs by more than 25% at country-level and 38% at city-level geo-localization tasks, and surpasses StreetCLIP performance while requiring fewer training resources.

Conclusion: The proposed approach effectively addresses data quality and reasoning challenges in geo-localization, demonstrating superior performance through human inference knowledge integration and optimized training methodology.

Abstract: This work tackles the problem of geo-localization with a new paradigm using a large vision-language model (LVLM) augmented with human inference knowledge. A primary challenge here is the scarcity of data for training the LVLM - existing street-view datasets often contain numerous low-quality images lacking visual clues, and lack any reasoning inference. To address the data-quality issue, we devise a CLIP-based network to quantify the degree of street-view images being locatable, leading to the creation of a new dataset comprising highly locatable street views. To enhance reasoning inference, we integrate external knowledge obtained from real geo-localization games, tapping into valuable human inference capabilities. The data are utilized to train GeoReasoner, which undergoes fine-tuning through dedicated reasoning and location-tuning stages. Qualitative and quantitative evaluations illustrate that GeoReasoner outperforms counterpart LVLMs by more than 25% at country-level and 38% at city-level geo-localization tasks, and surpasses StreetCLIP performance while requiring fewer training resources. The data and code are available at https://github.com/lingli1996/GeoReasoner.

Method: Two-stage framework: (1) fine-tuning pre-trained source model with few-shot support set using feature diversity augmentation to avoid overfitting, (2) test time adaptation with prototype memory bank guidance to produce high-quality pseudo-labels for model adaptation.

Result: Superior performance and reliability demonstrated through extensive experiments on three cross-domain classification benchmarks.

Conclusion: FS-TTA provides a practical and effective approach for domain adaptation by leveraging few-shot support sets to enhance Test Time Adaptation performance and reliability.

Abstract: Deep neural networks often encounter significant performance drops while facing with domain shifts between training (source) and test (target) data. To address this issue, Test Time Adaptation (TTA) methods have been proposed to adapt pre-trained source model to handle out-of-distribution streaming target data. Although these methods offer some relief, they lack a reliable mechanism for domain shift correction, which can often be erratic in real-world applications. In response, we develop Few-Shot Test Time Adaptation (FS-TTA), a novel and practical setting that utilizes a few-shot support set on top of TTA. Adhering to the principle of few inputs, big gains, FS-TTA reduces blind exploration in unseen target domains. Furthermore, we propose a two-stage framework to tackle FS-TTA, including (i) fine-tuning the pre-trained source model with few-shot support set, along with using feature diversity augmentation module to avoid overfitting, (ii) implementing test time adaptation based on prototype memory bank guidance to produce high quality pseudo-label for model adaptation. Through extensive experiments on three cross-domain classification benchmarks, we demonstrate the superior performance and reliability of our FS-TTA and framework.

Method: Proposed semi-optimal policy that independently estimates the value of each frame using per-frame confidence, then selects the top N frames, reducing search space from O(T^N) to O(T).

Result: The semi-optimal policy efficiently approximates the optimal policy, particularly in practical settings. Extensive experiments show stable high performance across various datasets and model architectures regardless of N and T size.

Conclusion: The proposed approach provides an efficient solution to frame sampling by significantly reducing computational complexity while maintaining performance, making it suitable for practical video classification tasks.

Abstract: Given a video with $T$ frames, frame sampling is a task to select $N \ll T$ frames, so as to maximize the performance of a fixed video classifier. Not just brute-force search, but most existing methods suffer from its vast search space of $\binom{T}{N}$, especially when $N$ gets large. To address this challenge, we introduce a novel perspective of reducing the search space from $O(T^N)$ to $O(T)$. Instead of exploring the entire $O(T^N)$ space, our proposed semi-optimal policy selects the top $N$ frames based on the independently estimated value of each frame using per-frame confidence, significantly reducing the computational complexity. We verify that our semi-optimal policy can efficiently approximate the optimal policy, particularly under practical settings. Additionally, through extensive experiments on various datasets and model architectures, we demonstrate that learning our semi-optimal policy ensures stable and high performance regardless of the size of $N$ and $T$.

Method: Uses LLMs to generate semantic interpretations of sensor readings and activity labels, employs iterative re-generation for quality, and implements two-stage training to bridge semantic spaces.

Result: Significantly outperforms state-of-the-art methods in cross-dataset HAR and new activity recognition on five public datasets.

Conclusion: LanHAR effectively mitigates cross-dataset heterogeneity and enhances recognition of new activities through semantic interpretation, producing a lightweight sensor encoder suitable for mobile deployment.

Abstract: Human Activity Recognition (HAR) using Inertial Measurement Unit (IMU) sensors is critical for applications in healthcare, safety, and industrial production. However, variations in activity patterns, device types, and sensor placements create distribution gaps across datasets, reducing the performance of HAR models. To address this, we propose LanHAR, a novel system that leverages Large Language Models (LLMs) to generate semantic interpretations of sensor readings and activity labels for cross-dataset HAR. This approach not only mitigates cross-dataset heterogeneity but also enhances the recognition of new activities. LanHAR employs an iterative re-generation method to produce high-quality semantic interpretations with LLMs and a two-stage training framework that bridges the semantic interpretations of sensor readings and activity labels. This ultimately leads to a lightweight sensor encoder suitable for mobile deployment, enabling any sensor reading to be mapped into the semantic interpretation space. Experiments on five public datasets demonstrate that our approach significantly outperforms state-of-the-art methods in both cross-dataset HAR and new activity recognition. The source code is publicly available at https://github.com/DASHLab/LanHAR.

Method: Proposes MaskControl with two key innovations: Logits Regularizer for implicit perturbation during training to align motion tokens with controlled joint positions, and Logit Optimization for explicit optimization during inference to reshape token distribution. Also introduces Differentiable Expectation Sampling to handle non-differentiable distribution sampling.

Result: Outperforms state-of-the-art methods with FID decreasing by ~77% and achieving higher control precision (average error 0.91 vs. 1.08). Enables diverse applications including any-joint-any-frame control, body-part timeline control, and zero-shot objective control.

Conclusion: MaskControl successfully addresses the challenge of high-precision control in motion generation while maintaining motion quality, representing a significant advancement in controllable text-to-motion generation.

Abstract: Recent advances in motion diffusion models have enabled spatially controllable text-to-motion generation. However, these models struggle to achieve high-precision control while maintaining high-quality motion generation. To address these challenges, we propose MaskControl, the first approach to introduce controllability to the generative masked motion model. Our approach introduces two key innovations. First, \textit{Logits Regularizer} implicitly perturbs logits at training time to align the distribution of motion tokens with the controlled joint positions, while regularizing the categorical token prediction to ensure high-fidelity generation. Second, \textit{Logit Optimization} explicitly optimizes the predicted logits during inference time, directly reshaping the token distribution that forces the generated motion to accurately align with the controlled joint positions. Moreover, we introduce \textit{Differentiable Expectation Sampling (DES)} to combat the non-differential distribution sampling process encountered by logits regularizer and optimization. Extensive experiments demonstrate that MaskControl outperforms state-of-the-art methods, achieving superior motion quality (FID decreases by ~77%) and higher control precision (average error 0.91 vs. 1.08). Additionally, MaskControl enables diverse applications, including any-joint-any-frame control, body-part timeline control, and zero-shot objective control. Video visualization can be found at https://www.ekkasit.com/ControlMM-page/

Method: Uses three pre-trained networks to extract X-ray features, applies PCA for dimensionality reduction, clustering for component selection, combines with clinical data, and processes through FCN for classification.

Result: Clinical data (Medical History, BMI, Height) were main contributors to predictions, while imaging features had lower importance, demonstrating clinical data’s crucial role in accurate diagnosis.

Conclusion: The framework enables precise and interpretable osteoporosis predictions, enhancing AI transparency and trust for clinical integration.

Abstract: Osteoporosis is a common condition that increases fracture risk, especially in older adults. Early diagnosis is vital for preventing fractures, reducing treatment costs, and preserving mobility. However, healthcare providers face challenges like limited labeled data and difficulties in processing medical images. This study presents a novel multi-modal learning framework that integrates clinical and imaging data to improve diagnostic accuracy and model interpretability. The model utilizes three pre-trained networks-VGG19, InceptionV3, and ResNet50-to extract deep features from X-ray images. These features are transformed using PCA to reduce dimensionality and focus on the most relevant components. A clustering-based selection process identifies the most representative components, which are then combined with preprocessed clinical data and processed through a fully connected network (FCN) for final classification. A feature importance plot highlights key variables, showing that Medical History, BMI, and Height were the main contributors, emphasizing the significance of patient-specific data. While imaging features were valuable, they had lower importance, indicating that clinical data are crucial for accurate predictions. This framework promotes precise and interpretable predictions, enhancing transparency and building trust in AI-driven diagnoses for clinical integration.

Method: Leverages influence functions with data transformations that sever the link between poisoned training data and compromised test points, detecting influence collapse to identify poisoned samples.

Result: Consistently achieves the best unlearning performance across three vision-based poisoning attacks and three datasets, outperforming five detection algorithms and five unlearning strategies.

Conclusion: Demonstrates the promise of influence functions for corrective unlearning, effectively eliminating data poisoning through targeted retraining of identified poisoned subsets.

Abstract: Addressing data integrity challenges, such as unlearning the effects of data poisoning after model training, is necessary for the reliable deployment of machine learning models. State-of-the-art influence functions, such as EK-FAC and TRAK, often fail to accurately attribute abnormal model behavior to the specific poisoned training data responsible for the data poisoning attack. In addition, traditional unlearning algorithms often struggle to effectively remove the influence of poisoned samples, particularly when only a few affected examples can be identified. To address these challenge, we introduce $\Delta$-Influence, a novel approach that leverages influence functions to trace abnormal model behavior back to the responsible poisoned training data using as little as just one poisoned test example. $\Delta$-Influence applies data transformations that sever the link between poisoned training data and compromised test points without significantly affecting clean data. This allows $\Delta$-Influence to detect large negative shifts in influence scores following data transformations, a phenomenon we term as influence collapse, thereby accurately identifying poisoned training data. Unlearning this subset, e.g. through retraining, effectively eliminates the data poisoning. We validate our method across three vision-based poisoning attacks and three datasets, benchmarking against five detection algorithms and five unlearning strategies. We show that $\Delta$-Influence consistently achieves the best unlearning across all settings, showing the promise of influence functions for corrective unlearning. Our code is publicly available at: https://github.com/Ruby-a07/delta-influence

Method: Proposes VisualLens framework that extracts, filters, and refines user profiles from visual history using multimodal LLMs to support personalized recommendation.

Result: VisualLens improves over state-of-the-art item-based multimodal recommendations by 5-10% on Hit@3, outperforms GPT-4o by 2-5%, and shows robustness across varying history lengths and unseen content categories.

Conclusion: Visual history can be effectively leveraged for personalization, with VisualLens demonstrating superior performance and adaptability compared to existing methods.

Abstract: Existing recommendation systems either rely on user interaction logs, such as online shopping history for shopping recommendations, or focus on text signals. However, item-based histories are not always accessible, and are not generalizable for multimodal recommendation. We hypothesize that a user’s visual history – comprising images from daily life – can offer rich, task-agnostic insights into their interests and preferences, and thus be leveraged for effective personalization. To this end, we propose VisualLens, a novel framework that leverages multimodal large language models (MLLMs) to enable personalization using task-agnostic visual history. VisualLens extracts, filters, and refines a spectrum user profile from the visual history to support personalized recommendation. We created two new benchmarks, Google-Review-V and Yelp-V, with task-agnostic visual histories, and show that VisualLens improves over state-of-the-art item-based multimodal recommendations by 5-10% on Hit@3, and outperforms GPT-4o by 2-5%. Further analysis shows that VisualLens is robust across varying history lengths and excels at adapting to both longer histories and unseen content categories.

Method: Uses diffusion transformer framework with multimodal guidance and noise-regularized memory bank to maintain contextual continuity. Employs deep compression autoencoder and spatiotemporal-aware transformer with linear attention for multimodal fusion. Combines Symbiotic Fusion for portrait features and Direct Fusion for audio.

Result: Establishes new state-of-the-art in generation quality, producing temporally coherent and realistic talking videos with enhanced diversity and liveliness. Maintains remarkable efficiency with 8x fewer parameters than previous approaches.

Conclusion: LetsTalk effectively addresses long-duration talking video synthesis challenges through its memory bank mechanism and optimized fusion schemes, achieving superior visual realism, precise speech-driven motion, and computational efficiency.

Abstract: Long-duration talking video synthesis faces enduring challenges in achieving high video quality, portrait and temporal consistency, and computational efficiency. As video length increases, issues such as visual degradation, identity inconsistency, temporal incoherence, and error accumulation become increasingly problematic, severely affecting the realism and reliability of the results. To address these challenges, we present LetsTalk, a diffusion transformer framework equipped with multimodal guidance and a novel memory bank mechanism, explicitly maintaining contextual continuity and enabling robust, high-quality, and efficient generation of long-duration talking videos. In particular, LetsTalk introduces a noise-regularized memory bank to alleviate error accumulation and sampling artifacts during extended video generation. To further improve efficiency and spatiotemporal consistency, LetsTalk employs a deep compression autoencoder and a spatiotemporal-aware transformer with linear attention for effective multimodal fusion. We systematically analyze three fusion schemes and show that combining deep (Symbiotic Fusion) for portrait features and shallow (Direct Fusion) for audio achieves superior visual realism and precise speech-driven motion, while preserving diversity of movements. Extensive experiments demonstrate that LetsTalk establishes new state-of-the-art in generation quality, producing temporally coherent and realistic talking videos with enhanced diversity and liveliness, and maintains remarkable efficiency with 8x fewer parameters than previous approaches.

Method: Uses path integral control to approximate guidance for diffusion models with non-differentiable reward functions, employs stitching between video frames to enable image-trained LVLMs to assess video alignment, and supports flexible ensembling of multiple reward models.

Result: Free²Guide significantly improves text-to-video alignment using image-trained LVLMs, enhancing overall video quality without significant computational overhead.

Conclusion: The framework enables effective integration of powerful black-box LVLMs as reward models for video generation alignment, overcoming limitations of previous differentiable reward approaches.

Abstract: Diffusion models have achieved impressive results in generative tasks for text-to-video (T2V) synthesis. However, achieving accurate text alignment in T2V generation remains challenging due to the complex temporal dependencies across frames. Existing reinforcement learning (RL)-based approaches to enhance text alignment often require differentiable reward functions trained for videos, hindering their scalability and applicability. In this paper, we propose \textbf{Free$^2$Guide}, a novel gradient-free and training-free framework for aligning generated videos with text prompts. Specifically, leveraging principles from path integral control, Free$^2$Guide approximates guidance for diffusion models using non-differentiable reward functions, thereby enabling the integration of powerful black-box Large Vision-Language Models (LVLMs) as reward models. To enable image-trained LVLMs to assess text-to-video alignment, we leverage \textit{stitching} between video frames and use system prompts to capture sequential attributions. Our framework supports the flexible ensembling of multiple reward models to synergistically enhance alignment without significant computational overhead. Experimental results confirm that Free$^2$Guide using image-trained LVLMs significantly improves text-to-video alignment, thereby enhancing the overall video quality. Our results and code are available at https://kjm981995.github.io/free2guide/

Method: Semi-online Preference Optimization (SoPo) uses semi-online data pairs consisting of unpreferred motion from online distribution and preferred motion from offline datasets. This leverages both online and offline DPO to compensate for each other’s limitations.

Result: SoPo outperforms other preference alignment methods with MM-Dist of 3.25% on MLD model and 2.91% on MDM model. The MLD model fine-tuned by SoPo surpasses state-of-the-art models in R-precision and MM Dist metrics.

Conclusion: SoPo effectively addresses limitations in both online and offline DPO through semi-online data pairing, demonstrating superior performance in text-to-motion generation preference alignment.

Abstract: Text-to-motion generation is essential for advancing the creative industry but often presents challenges in producing consistent, realistic motions. To address this, we focus on fine-tuning text-to-motion models to consistently favor high-quality, human-preferred motions, a critical yet largely unexplored problem. In this work, we theoretically investigate the DPO under both online and offline settings, and reveal their respective limitation: overfitting in offline DPO, and biased sampling in online DPO. Building on our theoretical insights, we introduce Semi-online Preference Optimization (SoPo), a DPO-based method for training text-to-motion models using “semi-online” data pair, consisting of unpreferred motion from online distribution and preferred motion in offline datasets. This method leverages both online and offline DPO, allowing each to compensate for the other’s limitations. Extensive experiments demonstrate that SoPo outperforms other preference alignment methods, with an MM-Dist of 3.25% (vs e.g. 0.76% of MoDiPO) on the MLD model, 2.91% (vs e.g. 0.66% of MoDiPO) on MDM model, respectively. Additionally, the MLD model fine-tuned by our SoPo surpasses the SoTA model in terms of R-precision and MM Dist. Visualization results also show the efficacy of our SoPo in preference alignment. Project page: https://xiaofeng-tan.github.io/projects/SoPo/ .

Method: FairGen uses attribute adapters that learn attribute latent directions via noise composition through self-discovery, and a distribution indicator that multiplies with adapters to guide generation towards prescribed distributions.

Result: Extensive experiments show FairGen outperforms previous state-of-the-art methods by a large margin in debiasing gender, racial, and intersectional biases.

Conclusion: FairGen provides an effective, lightweight solution for debiasing multiple attributes simultaneously in Diffusion Models without requiring retraining or reference datasets.

Abstract: While Diffusion Models (DM) exhibit remarkable performance across various image generative tasks, they nonetheless reflect the inherent bias presented in the training set. As DMs are now widely used in real-world applications, these biases could perpetuate a distorted worldview and hinder opportunities for minority groups. Existing methods on debiasing DMs usually requires model retraining with a human-crafted reference dataset or additional classifiers, which suffer from two major limitations: (1) collecting reference datasets causes expensive annotation cost; (2) the debiasing performance is heavily constrained by the quality of the reference dataset or the additional classifier. To address the above limitations, we propose FairGen, a plug-and-play method that learns attribute latent directions in a self-discovering manner, thus eliminating the reliance on such reference dataset. Specifically, FairGen consists of two parts: a set of attribute adapters and a distribution indicator. Each adapter in the set aims to learn an attribute latent direction, and is optimized via noise composition through a self-discovering process. Then, the distribution indicator is multiplied by the set of adapters to guide the generation process towards the prescribed distribution. Our method enables debiasing multiple attributes in DMs simultaneously, while remaining lightweight and easily integrable with other DMs, eliminating the need for retraining. Extensive experiments on debiasing gender, racial, and their intersectional biases show that our method outperforms previous SOTA by a large margin.

Method: NanoHTNet uses spatial Hierarchical Mixer for human physical topology and temporal Hierarchical Mixer with DCT and filtering for movement patterns. Includes ETST for spatio-temporal interaction. PoseCLR pre-training uses contrastive learning with 2D pose alignment.

Result: Outperforms state-of-the-art methods in efficiency, making it ideal for deployment on edge devices like Jetson Nano.

Conclusion: The combination of NanoHTNet’s efficient architecture and PoseCLR pre-training effectively captures both explicit and implicit human structural priors, enabling high-performance 3D HPE on edge devices.

Abstract: The widespread application of 3D human pose estimation (HPE) is limited by resource-constrained edge devices, requiring more efficient models. A key approach to enhancing efficiency involves designing networks based on the structural characteristics of input data. However, effectively utilizing the structural priors in human skeletal inputs remains challenging. To address this, we leverage both explicit and implicit spatio-temporal priors of the human body through innovative model design and a pre-training proxy task. First, we propose a Nano Human Topology Network (NanoHTNet), a tiny 3D HPE network with stacked Hierarchical Mixers to capture explicit features. Specifically, the spatial Hierarchical Mixer efficiently learns the human physical topology across multiple semantic levels, while the temporal Hierarchical Mixer with discrete cosine transform and low-pass filtering captures local instantaneous movements and global action coherence. Moreover, Efficient Temporal-Spatial Tokenization (ETST) is introduced to enhance spatio-temporal interaction and reduce computational complexity significantly. Second, PoseCLR is proposed as a general pre-training method based on contrastive learning for 3D HPE, aimed at extracting implicit representations of human topology. By aligning 2D poses from diverse viewpoints in the proxy task, PoseCLR aids 3D HPE encoders like NanoHTNet in more effectively capturing the high-dimensional features of the human body, leading to further performance improvements. Extensive experiments verify that NanoHTNet with PoseCLR outperforms other state-of-the-art methods in efficiency, making it ideal for deployment on edge devices like the Jetson Nano. Code and models are available at https://github.com/vefalun/NanoHTNet.

Method: Uses pre-trained text-to-video diffusion transformer and vision-language model with novel inference-based feature manipulation in attention mechanism for accurate localization and seamless integration of new content.

Result: Successfully demonstrates diverse video edits on real-world videos with various objects and scenarios, handling both camera and object motion while preserving original scene integrity.

Conclusion: The proposed framework provides an effective, automated solution for video augmentation that requires only simple user instructions and produces realistic, cohesive output videos.

Abstract: We present a method for augmenting real-world videos with newly generated dynamic content. Given an input video and a simple user-provided text instruction describing the desired content, our method synthesizes dynamic objects or complex scene effects that naturally interact with the existing scene over time. The position, appearance, and motion of the new content are seamlessly integrated into the original footage while accounting for camera motion, occlusions, and interactions with other dynamic objects in the scene, resulting in a cohesive and realistic output video. We achieve this via a zero-shot, training-free framework that harnesses a pre-trained text-to-video diffusion transformer to synthesize the new content and a pre-trained vision-language model to envision the augmented scene in detail. Specifically, we introduce a novel inference-based method that manipulates features within the attention mechanism, enabling accurate localization and seamless integration of the new content while preserving the integrity of the original scene. Our method is fully automated, requiring only a simple user instruction. We demonstrate its effectiveness on a wide range of edits applied to real-world videos, encompassing diverse objects and scenarios involving both camera and object motion.

Method: Dual Caption Preference Optimization (DCPO) assigns two distinct captions to each preference pair using three strategies: captioning, perturbation, and hybrid methods. Also created Pick-Double Caption dataset.

Result: DCPO significantly outperforms Stable Diffusion 2.1, SFT_Chosen, Diffusion-DPO, and MaPO across multiple metrics including Pickscore, HPSv2.1, GenEval, CLIPscore, and ImageReward.

Conclusion: Using dual captions for preference pairs effectively reinforces learning signals and improves model performance in text-to-image generation.

Abstract: Recent advancements in human preference optimization, originally developed for Large Language Models (LLMs), have shown significant potential in improving text-to-image diffusion models. These methods aim to learn the distribution of preferred samples while distinguishing them from less preferred ones. However, within the existing preference datasets, the original caption often does not clearly favor the preferred image over the alternative, which weakens the supervision signal available during training. To address this issue, we introduce Dual Caption Preference Optimization (DCPO), a data augmentation and optimization framework that reinforces the learning signal by assigning two distinct captions to each preference pair. This encourages the model to better differentiate between preferred and less-preferred outcomes during training. We also construct Pick-Double Caption, a modified version of Pick-a-Pic v2 with separate captions for each image, and propose three different strategies for generating distinct captions: captioning, perturbation, and hybrid methods. Our experiments show that DCPO significantly improves image quality and relevance to prompts, outperforming Stable Diffusion (SD) 2.1, SFT_Chosen, Diffusion-DPO, and MaPO across multiple metrics, including Pickscore, HPSv2.1, GenEval, CLIPscore, and ImageReward, fine-tuned on SD 2.1 as the backbone.

Method: Uses indoor panorama for 3D floor layout estimation and outdoor panorama as environment map to infer illumination and material properties. Models light-heat relationship assuming Lambertian surfaces and outdoor light as heat source, performing transient heat simulations to generate temperature distributions.

Result: Generated indoor temperature distributions through heat simulations, with validation against real-world thermal images captured using infrared cameras.

Conclusion: The approach successfully enables joint estimation of lighting and thermal conditions, supporting physically informed visualization for virtual home staging applications.

Abstract: This paper introduces a novel image-based rendering technique for jointly estimating indoor lighting and thermal conditions from paired indoor-outdoor high dynamic range (HDR) panoramas. Our method uses the indoor panorama to estimate the 3D floor layout, while the corresponding outdoor panorama serves as an environment map to infer spatially-varying illumination and material properties. Assuming indoor surfaces are Lambertian and that all heat originates from outdoor visible light, we model the relationship between light transport and heat transfer, and perform transient heat simulations to generate indoor temperature distributions. The simulated heat maps are validated against real-world thermal images captured with an infrared camera. This approach supports photorealistic and physically informed visualization, enabling integrated light and heat estimation to advance traditional virtual home staging.

Method: PartSDF uses a supervised implicit representation framework with a simple but innovative architecture that explicitly models composite shapes with independent, controllable parts while maintaining shape consistency.

Result: PartSDF outperforms both supervised and unsupervised baselines in reconstruction and generation tasks, and serves as an effective structured shape prior for engineering applications.

Conclusion: The framework enables precise control over individual components while preserving overall coherence, making it suitable for real-world engineering design tasks.

Abstract: Accurate 3D shape representation is essential in engineering applications such as design, optimization, and simulation. In practice, engineering workflows require structured, part-based representations, as objects are inherently designed as assemblies of distinct components. However, most existing methods either model shapes holistically or decompose them without predefined part structures, limiting their applicability in real-world design tasks. We propose PartSDF, a supervised implicit representation framework that explicitly models composite shapes with independent, controllable parts while maintaining shape consistency. Thanks to its simple but innovative architecture, PartSDF outperforms both supervised and unsupervised baselines in reconstruction and generation tasks. We further demonstrate its effectiveness as a structured shape prior for engineering applications, enabling precise control over individual components while preserving overall coherence. Code available at https://github.com/cvlab-epfl/PartSDF.

Method: Enhanced Language-Image Pre-training (ELIP) uses text queries via MLP mapping to predict visual prompts that condition ViT image encoding. Includes global hard sample mining and dataset curation for efficient training.

Result: ELIP significantly boosts CLIP/SigLIP/BLIP-2 text-to-image retrieval performance, outperforms BLIP-2 on several benchmarks, and provides easy adaptation to out-of-distribution datasets.

Conclusion: ELIP framework effectively enhances vision-language models for text-to-image retrieval, demonstrating strong performance improvements and generalization capabilities with limited computing resources.

Abstract: The objective in this paper is to improve the performance of text-to-image retrieval. To this end, we introduce a new framework that can boost the performance of large-scale pre-trained vision-language models, so that they can be used for text-to-image re-ranking. The approach, Enhanced Language-Image Pre-training (ELIP), uses the text query, via a simple MLP mapping network, to predict a set of visual prompts to condition the ViT image encoding. ELIP can easily be applied to the commonly used CLIP, SigLIP and BLIP-2 networks. To train the architecture with limited computing resources, we develop a ‘student friendly’ best practice, involving global hard sample mining, and curation of a large-scale dataset. On the evaluation side, we set up two new out-of-distribution (OOD) benchmarks, Occluded COCO and ImageNet-R, to assess the zero-shot generalisation of the models to different domains. The results demonstrate that ELIP significantly boosts CLIP/SigLIP/SigLIP-2 text-to-image retrieval performance and outperforms BLIP-2 on several benchmarks, as well as providing an easy means to adapt to OOD datasets.

Method: Uses sustained boosting algorithm with simultaneous optimization of classification and residual errors, plus adaptive classifier assignment strategy to dynamically improve weak modality performance.

Result: Empirical experiments on widely used datasets show superiority over various state-of-the-art multimodal learning baselines.

Conclusion: The proposed method effectively balances classification ability between strong and weak modalities, mitigating the modality imbalance issue in multimodal learning.

Abstract: Multimodal learning (MML) is significantly constrained by modality imbalance, leading to suboptimal performance in practice. While existing approaches primarily focus on balancing the learning of different modalities to address this issue, they fundamentally overlook the inherent disproportion in model classification ability, which serves as the primary cause of this phenomenon. In this paper, we propose a novel multimodal learning approach to dynamically balance the classification ability of weak and strong modalities by incorporating the principle of boosting. Concretely, we first propose a sustained boosting algorithm in multimodal learning by simultaneously optimizing the classification and residual errors. Subsequently, we introduce an adaptive classifier assignment strategy to dynamically facilitate the classification performance of the weak modality. Furthermore, we theoretically analyze the convergence property of the cross-modal gap function, ensuring the effectiveness of the proposed boosting scheme. To this end, the classification ability of strong and weak modalities is expected to be balanced, thereby mitigating the imbalance issue. Empirical experiments on widely used datasets reveal the superiority of our method through comparison with various state-of-the-art (SOTA) multimodal learning baselines. The source code is available at https://github.com/njustkmg/NeurIPS25-AUG.

Method: The authors systematically review classical and emerging methods for camera pose estimation and dense surface reconstruction, conducting comprehensive evaluations using both real-world and simulated datasets.

Result: The study provides reference recommendations for scientists and managers, identifying which methods work best for underwater coral reef 3D reconstruction given the challenges of underwater environments and complex coral structures.

Conclusion: This work bridges the gap between technical studies and practical applications, offering technical foundation and practical guidance for processing underwater coral reef images for 3D reconstruction, while discussing development potential and challenges of existing approaches.

Abstract: Corals serve as the foundational habitat-building organisms within reef ecosystems, constructing extensive structures that extend over vast distances. However, their inherent fragility and vulnerability to various threats render them susceptible to significant damage and destruction. The application of advanced 3D reconstruction technologies for high-quality modeling is crucial for preserving them. These technologies help scientists to accurately document and monitor the state of coral reefs, including their structure, species distribution and changes over time. Photogrammetry-based approaches stand out among existing solutions, especially with recent advancements in underwater videography, photogrammetric computer vision, and machine learning. Despite continuous progress in image-based 3D reconstruction techniques, there remains a lack of systematic reviews and comprehensive evaluations of cutting-edge solutions specifically applied to underwater coral reef images. The emerging advanced methods may have difficulty coping with underwater imaging environments, complex coral structures, and computational resource constraints. They need to be reviewed and evaluated to bridge the gap between many cutting-edge technical studies and practical applications. This paper focuses on the two critical stages of these approaches: camera pose estimation and dense surface reconstruction. We systematically review and summarize classical and emerging methods, conducting comprehensive evaluations through real-world and simulated datasets. Based on our findings, we offer reference recommendations and discuss the development potential and challenges of existing approaches in depth. This work equips scientists and managers with a technical foundation and practical guidance for processing underwater coral reef images for 3D reconstruction….

Method: NFIG decomposes image generation into frequency-guided stages: first generating low-frequency components with fewer tokens to establish global structure, then progressively adding higher-frequency details.

Result: On ImageNet-256 benchmark, NFIG achieves superior performance with FID: 2.81 and a 1.25x speedup compared to VAR-d20 baseline.

Conclusion: The frequency-aware paradigm aligns generation with natural image structure, offering both quality improvements and computational efficiency gains in autoregressive image generation.

Abstract: Autoregressive models have achieved significant success in image generation. However, unlike the inherent hierarchical structure of image information in the spectral domain, standard autoregressive methods typically generate pixels sequentially in a fixed spatial order. To better leverage this spectral hierarchy, we introduce NextFrequency Image Generation (NFIG). NFIG is a novel framework that decomposes the image generation process into multiple frequency-guided stages. NFIG aligns the generation process with the natural image structure. It does this by first generating low-frequency components, which efficiently capture global structure with significantly fewer tokens, and then progressively adding higher-frequency details. This frequency-aware paradigm offers substantial advantages: it not only improves the quality of generated images but crucially reduces inference cost by efficiently establishing global structure early on. Extensive experiments on the ImageNet-256 benchmark validate NFIG’s effectiveness, demonstrating superior performance (FID: 2.81) and a notable 1.25x speedup compared to the strong baseline VAR-d20. The source code is available at https://github.com/Pride-Huang/NFIG.

Method: Introduces DitHub framework inspired by Version Control Systems, managing expert modules as branches that can be fetched and merged as needed, enabling compositional exploration of adaptation modules.

Result: Achieves state-of-the-art performance on ODinW-13 benchmark and newly introduced ODinW-O benchmark for assessing class reappearance.

Conclusion: Modular approach enables efficient adaptation and composition of expert modules, marking the first such study in Object Detection and demonstrating superior performance on challenging benchmarks.

Abstract: Open-Vocabulary object detectors can generalize to an unrestricted set of categories through simple textual prompting. However, adapting these models to rare classes or reinforcing their abilities on multiple specialized domains remains essential. While recent methods rely on monolithic adaptation strategies with a single set of weights, we embrace modular deep learning. We introduce DitHub, a framework designed to build and maintain a library of efficient adaptation modules. Inspired by Version Control Systems, DitHub manages expert modules as branches that can be fetched and merged as needed. This modular approach allows us to conduct an in-depth exploration of the compositional properties of adaptation modules, marking the first such study in Object Detection. Our method achieves state-of-the-art performance on the ODinW-13 benchmark and ODinW-O, a newly introduced benchmark designed to assess class reappearance. For more details, visit our project page: https://aimagelab.github.io/DitHub/

Method: Uses Mean Teacher architecture with adaptive label refinement mechanism that weights differences across disturbance versions, sample-level uncertainty-based label selection to prioritize confident samples, and consistency learning between student and teacher networks.

Result: Extensive experiments on two public datasets show significant improvements in segmentation performance, achieving competitive results compared to state-of-the-art methods.

Conclusion: The ALC framework effectively processes noisy labels, adapts to challenging scenarios, and fully exploits Mean Teacher structure strengths for robust medical image segmentation.

Abstract: Deep learning has shown remarkable success in medical image analysis, but its reliance on large volumes of high-quality labeled data limits its applicability. While noisy labeled data are easier to obtain, directly incorporating them into training can degrade model performance. To address this challenge, we propose a Mean Teacher-based Adaptive Label Correction (ALC) self-ensemble framework for robust medical image segmentation with noisy labels. The framework leverages the Mean Teacher architecture to ensure consistent learning under noise perturbations. It includes an adaptive label refinement mechanism that dynamically captures and weights differences across multiple disturbance versions to enhance the quality of noisy labels. Additionally, a sample-level uncertainty-based label selection algorithm is introduced to prioritize high-confidence samples for network updates, mitigating the impact of noisy annotations. Consistency learning is integrated to align the predictions of the student and teacher networks, further enhancing model robustness. Extensive experiments on two public datasets demonstrate the effectiveness of the proposed framework, showing significant improvements in segmentation performance. By fully exploiting the strengths of the Mean Teacher structure, the ALC framework effectively processes noisy labels, adapts to challenging scenarios, and achieves competitive results compared to state-of-the-art methods.

Method: Proposes a hybrid image reconstruction surrogate task that preserves SD’s detail priors by reconstructing images themselves. Introduces Scale-Shift GRU to bridge the distribution gap between SD’s scale-shift invariant estimation and self-supervised scale-invariant depth estimation.

Result: Achieves state-of-the-art performance on KITTI benchmark and demonstrates superior zero-shot generalization across multiple datasets.

Conclusion: Jasmine successfully leverages SD’s visual priors in a self-supervised framework for monocular depth estimation, overcoming limitations of both supervised SD methods and traditional self-supervised approaches.

Abstract: In this paper, we propose Jasmine, the first Stable Diffusion (SD)-based self-supervised framework for monocular depth estimation, which effectively harnesses SD’s visual priors to enhance the sharpness and generalization of unsupervised prediction. Previous SD-based methods are all supervised since adapting diffusion models for dense prediction requires high-precision supervision. In contrast, self-supervised reprojection suffers from inherent challenges (e.g., occlusions, texture-less regions, illumination variance), and the predictions exhibit blurs and artifacts that severely compromise SD’s latent priors. To resolve this, we construct a novel surrogate task of hybrid image reconstruction. Without any additional supervision, it preserves the detail priors of SD models by reconstructing the images themselves while preventing depth estimation from degradation. Furthermore, to address the inherent misalignment between SD’s scale and shift invariant estimation and self-supervised scale-invariant depth estimation, we build the Scale-Shift GRU. It not only bridges this distribution gap but also isolates the fine-grained texture of SD output against the interference of reprojection loss. Extensive experiments demonstrate that Jasmine achieves SoTA performance on the KITTI benchmark and exhibits superior zero-shot generalization across multiple datasets.

Method: Uses VLM-generated structured descriptions to identify objects, collects attributes, informs open-vocabulary detector for bounding boxes, then passes to video segmentation model for masks and tracking. Processes image streams in real-time with minimal overhead.

Result: Evaluation across datasets and robotics platforms demonstrates broad applicability and ability to extract task-specific attributes from non-standard objects in dynamic environments.

Conclusion: Successfully combines descriptive power of VLMs with grounding capability of OVD and pixel-level understanding/speed of video segmentation for real-time processing in dynamic environments.

Abstract: Vision-language models (VLMs) excel in visual understanding but often lack reliable grounding capabilities and actionable inference rates. Integrating them with open-vocabulary object detection (OVD), instance segmentation, and tracking leverages their strengths while mitigating these drawbacks. We utilize VLM-generated structured descriptions to identify visible object instances, collect application-relevant attributes, and inform an open-vocabulary detector to extract corresponding bounding boxes that are passed to a video segmentation model providing segmentation masks and tracking. Once initialized, this model directly extracts segmentation masks, processing image streams in real time with minimal computational overhead. Tracks can be updated online as needed by generating new structured descriptions and detections. This combines the descriptive power of VLMs with the grounding capability of OVD and the pixel-level understanding and speed of video segmentation. Our evaluation across datasets and robotics platforms demonstrates the broad applicability of this approach, showcasing its ability to extract task-specific attributes from non-standard objects in dynamic environments. Code, data, videos, and benchmarks are available at https://vlm-gist.github.io

Method: Uses object-enriched observation rewriting (VLMs+LLMs for scene descriptions, T2IMs for image synthesis) and observation-contrast instruction rewriting (LLMs reasoning differences between observations). Includes mixing-then-focusing training with random cropping.

Result: Superior performance on discrete (R2R, REVERIE, R4R) and continuous (R2R-CE) environments, showing impressive generalization ability.

Conclusion: RAM provides an effective simulator-free and labor-saving paradigm for VLN data augmentation that significantly improves generalization through rewriting-based data creation.

Abstract: Data scarcity is a long-standing challenge in the Vision-Language Navigation (VLN) field, which extremely hinders the generalization of agents to unseen environments. Previous works primarily rely on additional simulator data or web-collected images/videos to improve the generalization. However, the simulator environments still face limited diversity, and the web-collected data often requires extensive labor to remove the noise. In this paper, we propose a Rewriting-driven AugMentation (RAM) paradigm for VLN, which directly creates the unseen observation-instruction pairs via rewriting human-annotated training data. Benefiting from our rewriting mechanism, new observation-instruction pairs can be obtained in both simulator-free and labor-saving manners to promote generalization. Specifically, we first introduce Object-Enriched Observation Rewriting, where we combine Vision-Language Models (VLMs) and Large Language Models (LLMs) to derive rewritten object-enriched scene descriptions, enabling observation synthesis with diverse objects and spatial layouts via Text-to-Image Generation Models (T2IMs). Then, we propose Observation-Contrast Instruction Rewriting, which generates observation-aligned rewritten instructions by requiring LLMs to reason the difference between original and new observations. We further develop a mixing-then-focusing training strategy with a random observation cropping scheme, effectively enhancing data distribution diversity while suppressing augmentation data noise during training. Experiments on both the discrete environments (R2R, REVERIE, and R4R datasets) and continuous environments (R2R-CE dataset) show the superior performance and impressive generalization ability of our method. Code is available at https://github.com/SaDil13/VLN-RAM.

Method: Created Morpheus benchmark with 80 real-world videos capturing physical phenomena, using physics-informed metrics evaluated with respect to conservation laws through physics-informed neural networks and vision-language foundation models.

Result: Current video generation models struggle to encode physical principles despite generating aesthetically pleasing videos, even with advanced prompting and video conditioning.

Conclusion: Video generation models currently lack proper physical reasoning capabilities and cannot be reliably treated as world models for applications requiring physical plausibility.

Abstract: Recent advances in image and video generation raise hopes that these models possess world modeling capabilities, the ability to generate realistic, physically plausible videos. This could revolutionize applications in robotics, autonomous driving, and scientific simulation. However, before treating these models as world models, we must ask: Do they adhere to physical conservation laws? To answer this, we introduce Morpheus, a benchmark for evaluating video generation models on physical reasoning. It features 80 real-world videos capturing physical phenomena, guided by conservation laws. Since artificial generations lack ground truth, we assess physical plausibility using physics-informed metrics evaluated with respect to infallible conservation laws known per physical setting, leveraging advances in physics-informed neural networks and vision-language foundation models. Our findings reveal that even with advanced prompting and video conditioning, current models struggle to encode physical principles despite generating aesthetically pleasing videos. All data, leaderboard, and code are open-sourced at our project page.

Method: Two-stage process: 1) Enhance dynamic range of images using diffusion model priors, 2) Use Gaussian Splats to model 3D lighting for spatially variant effects. Also introduces a novel dataset for benchmarking.

Result: Achieves state-of-the-art results on HDR estimations and applications in illuminating virtual objects and scenes. Validated on both novel and existing datasets.

Conclusion: GaSLight successfully enables regular images to function as light sources in 3D rendering through HDR Gaussian Splats, with superior performance in HDR estimation and virtual scene illumination.

Abstract: We present GaSLight, a method that generates spatially-varying lighting from regular images. Our method proposes using HDR Gaussian Splats as light source representation, marking the first time regular images can serve as light sources in a 3D renderer. Our two-stage process first enhances the dynamic range of images plausibly and accurately by leveraging the priors embedded in diffusion models. Next, we employ Gaussian Splats to model 3D lighting, achieving spatially variant lighting. Our approach yields state-of-the-art results on HDR estimations and their applications in illuminating virtual objects and scenes. To facilitate the benchmarking of images as light sources, we introduce a novel dataset of calibrated and unsaturated HDR to evaluate images as light sources. We assess our method using a combination of this novel dataset and an existing dataset from the literature. Project page: https://lvsn.github.io/gaslight/

Method: Hierarchical feature learning with farthest point sampling, KNN queries for multi-scale information aggregation, coordinate-order and inside-out scanning for point serialization, and Point SSM blocks to capture local patterns and long-range dependencies.

Result: The method achieves superior performance across anatomy classification, completion, and segmentation tasks on the newly built MedPointS dataset.

Conclusion: The proposed SSM-based framework effectively handles medical point clouds and demonstrates strong performance on multiple medical tasks, with the dataset and code made publicly available.

Abstract: Deep learning-based point cloud modeling has been widely investigated as an indispensable component of general shape analysis. Recently, transformer and state space model (SSM) have shown promising capacities in point cloud learning. However, limited research has been conducted on medical point clouds, which have great potential in disease diagnosis and treatment. This paper presents an SSM-based hierarchical feature learning framework for medical point cloud understanding. Specifically, we down-sample input into multiple levels through the farthest point sampling. At each level, we perform a series of k-nearest neighbor (KNN) queries to aggregate multi-scale structural information. To assist SSM in processing point clouds, we introduce coordinate-order and inside-out scanning strategies for efficient serialization of irregular points. Point features are calculated progressively from short neighbor sequences and long point sequences through vanilla and group Point SSM blocks, to capture both local patterns and long-range dependencies. To evaluate the proposed method, we build a large-scale medical point cloud dataset named MedPointS for anatomy classification, completion, and segmentation. Extensive experiments conducted on MedPointS demonstrate that our method achieves superior performance across all tasks. The dataset is available at https://flemme-docs.readthedocs.io/en/latest/medpoints.html. Code is merged to a public medical imaging platform: https://github.com/wlsdzyzl/flemme.

Method: REVERSE combines hallucination-aware training using a new dataset with 1.3M semi-synthetic samples and a novel inference-time retrospective resampling technique that enables VLMs to detect hallucinations during generation and dynamically revise them.

Result: REVERSE achieves state-of-the-art hallucination reduction, outperforming the best existing methods by up to 12% on CHAIR-MSCOCO and 34% on HaloQuest benchmarks.

Conclusion: The REVERSE framework provides an effective unified approach for reducing visual hallucinations in VLMs through integrated training and self-verification, with publicly available dataset, model, and code.

Abstract: Vision-Language Models (VLMs) excel at visual understanding but often suffer from visual hallucinations, where they generate descriptions of nonexistent objects, actions, or concepts, posing significant risks in safety-critical applications. Existing hallucination mitigation methods typically follow one of two paradigms: generation adjustment, which modifies decoding behavior to align text with visual inputs, and post-hoc verification, where external models assess and correct outputs. While effective, generation adjustment methods often rely on heuristics and lack correction mechanisms, while post-hoc verification is complicated, typically requiring multiple models and tending to reject outputs rather than refine them. In this work, we introduce REVERSE, a unified framework that integrates hallucination-aware training with on-the-fly self-verification. By leveraging a new hallucination-verification dataset containing over 1.3M semi-synthetic samples, along with a novel inference-time retrospective resampling technique, our approach enables VLMs to both detect hallucinations during generation and dynamically revise those hallucinations. Our evaluations show that REVERSE achieves state-of-the-art hallucination reduction, outperforming the best existing methods by up to 12% on CHAIR-MSCOCO and 34% on HaloQuest. Our dataset, model, and code are available at: https://reverse-vlm.github.io.

Method: Proposed a phenology-informed sampling strategy and created SSL4Eco dataset, then trained an existing model with season-contrastive objective on multi-date Sentinel-2 imagery.

Result: The SSL4Eco-pretrained model achieved state-of-the-art performance on 7 out of 8 downstream tasks spanning classification and regression, consistently outperforming other datasets.

Conclusion: The straightforward phenology-informed sampling method significantly improves representation quality, highlighting the importance of dataset construction in ecological remote sensing.

Abstract: With the exacerbation of the biodiversity and climate crises, macroecological pursuits such as global biodiversity mapping become more urgent. Remote sensing offers a wealth of Earth observation data for ecological studies, but the scarcity of labeled datasets remains a major challenge. Recently, self-supervised learning has enabled learning representations from unlabeled data, triggering the development of pretrained geospatial models with generalizable features. However, these models are often trained on datasets biased toward areas of high human activity, leaving entire ecological regions underrepresented. Additionally, while some datasets attempt to address seasonality through multi-date imagery, they typically follow calendar seasons rather than local phenological cycles. To better capture vegetation seasonality at a global scale, we propose a simple phenology-informed sampling strategy and introduce corresponding SSL4Eco, a multi-date Sentinel-2 dataset, on which we train an existing model with a season-contrastive objective. We compare representations learned from SSL4Eco against other datasets on diverse ecological downstream tasks and demonstrate that our straightforward sampling method consistently improves representation quality, highlighting the importance of dataset construction. The model pretrained on SSL4Eco reaches state of the art performance on 7 out of 8 downstream tasks spanning (multi-label) classification and regression. We release our code, data, and model weights to support macroecological and computer vision research at https://github.com/PlekhanovaElena/ssl4eco.

Method: Proposes a unified framework that integrates image, text, and metadata via cross-contrastive pre-training, aligning three modality encoders in a shared embedding space, then fine-tuning image and metadata encoders for classification.

Result: Achieves 84.44% top-1 accuracy on NABirds, improving over baseline by 7.83% and outperforming strong multimodal methods.

Conclusion: The proposed multimodal framework effectively addresses fine-grained classification challenges by leveraging complementary information from images, text, and metadata through cross-contrastive learning.

Abstract: Fine-grained visual classification aims to recognize objects belonging to many subordinate categories of a supercategory, where appearance alone often fails to distinguish highly similar classes. We propose a unified framework that integrates image, text, and metadata via cross-contrastive pre-training. We first align the three modality encoders in a shared embedding space and then fine-tune the image and metadata encoders for classification. On NABirds, our approach improves over the baseline by 7.83% and achieves 84.44% top-1 accuracy, outperforming strong multimodal methods.

Method: Introduces CoCo denoiser using generalized Helmholtz decomposition with Hamiltonian regularization for conservativeness and spectral regularization for cocoerciveness. Proves it’s a proximal operator of a weakly convex function.

Result: The CoCo denoiser enables PnP methods to converge globally to stationary points of restoration models with implicit weakly convex priors. Experimental results show superior performance over related methods.

Conclusion: The proposed CoCo denoiser framework overcomes limitations of traditional PnP methods in Poisson inverse problems, providing improved denoising performance with theoretical convergence guarantees.

Abstract: Plug-and-play (PnP) methods with deep denoisers have shown impressive results in imaging problems. They typically require strong convexity or smoothness of the fidelity term and a (residual) non-expansive denoiser for convergence. These assumptions, however, are violated in Poisson inverse problems, and non-expansiveness can hinder denoising performance. To address these challenges, we propose a cocoercive conservative (CoCo) denoiser, which may be (residual) expansive, leading to improved denoising. By leveraging the generalized Helmholtz decomposition, we introduce a novel training strategy that combines Hamiltonian regularization to promote conservativeness and spectral regularization to ensure cocoerciveness. We prove that CoCo denoiser is a proximal operator of a weakly convex function, enabling a restoration model with an implicit weakly convex prior. The global convergence of PnP methods to a stationary point of this restoration model is established. Extensive experimental results demonstrate that our approach outperforms closely related methods in both visual quality and quantitative metrics.

Method: Created benchmark with 2,264 video-text pairs across 48 unsafe categories using a controllable pipeline that decomposes video semantics into subject images and motion text. Proposed RJScore metric for evaluating uncertain outputs with confidence-based judgment.

Result: Benign-query video composition achieved 67.2% average attack success rate, revealing consistent vulnerabilities to video-induced attacks across tested models.

Conclusion: Video-SafetyBench addresses critical gaps in multimodal safety evaluation and will catalyze future research into video-based safety evaluation and defense strategies.

Abstract: The increasing deployment of Large Vision-Language Models (LVLMs) raises safety concerns under potential malicious inputs. However, existing multimodal safety evaluations primarily focus on model vulnerabilities exposed by static image inputs, ignoring the temporal dynamics of video that may induce distinct safety risks. To bridge this gap, we introduce Video-SafetyBench, the first comprehensive benchmark designed to evaluate the safety of LVLMs under video-text attacks. It comprises 2,264 video-text pairs spanning 48 fine-grained unsafe categories, each pairing a synthesized video with either a harmful query, which contains explicit malice, or a benign query, which appears harmless but triggers harmful behavior when interpreted alongside the video. To generate semantically accurate videos for safety evaluation, we design a controllable pipeline that decomposes video semantics into subject images (what is shown) and motion text (how it moves), which jointly guide the synthesis of query-relevant videos. To effectively evaluate uncertain or borderline harmful outputs, we propose RJScore, a novel LLM-based metric that incorporates the confidence of judge models and human-aligned decision threshold calibration. Extensive experiments show that benign-query video composition achieves average attack success rates of 67.2%, revealing consistent vulnerabilities to video-induced attacks. We believe Video-SafetyBench will catalyze future research into video-based safety evaluation and defense strategies.

Method: AIGIBench simulates real-world challenges through four core tasks: multi-source generalization, robustness to image degradation, sensitivity to data augmentation, and impact of test-time pre-processing, using 23 diverse fake image subsets and real-world samples.

Result: Experiments on 11 advanced detectors show significant performance drops on real-world data, limited benefits from common augmentations, and nuanced effects of pre-processing.

Conclusion: There is a need for more robust detection strategies, and AIGIBench provides a unified evaluation framework to guide future research toward dependable and generalizable AIGI detection.

Abstract: The rapid advancement of generative models, such as GANs and Diffusion models, has enabled the creation of highly realistic synthetic images, raising serious concerns about misinformation, deepfakes, and copyright infringement. Although numerous Artificial Intelligence Generated Image (AIGI) detectors have been proposed, often reporting high accuracy, their effectiveness in real-world scenarios remains questionable. To bridge this gap, we introduce AIGIBench, a comprehensive benchmark designed to rigorously evaluate the robustness and generalization capabilities of state-of-the-art AIGI detectors. AIGIBench simulates real-world challenges through four core tasks: multi-source generalization, robustness to image degradation, sensitivity to data augmentation, and impact of test-time pre-processing. It includes 23 diverse fake image subsets that span both advanced and widely adopted image generation techniques, along with real-world samples collected from social media and AI art platforms. Extensive experiments on 11 advanced detectors demonstrate that, despite their high reported accuracy in controlled settings, these detectors suffer significant performance drops on real-world data, limited benefits from common augmentations, and nuanced effects of pre-processing, highlighting the need for more robust detection strategies. By providing a unified and realistic evaluation framework, AIGIBench offers valuable insights to guide future research toward dependable and generalizable AIGI detection.Data and code are publicly available at: https://github.com/HorizonTEL/AIGIBench.

Method: Proposes unified concept tokens and a three-stage progressive training strategy: understanding warm-up, bootstrapping generation from understanding, and deepening understanding from generation.

Result: Shows competitive performance in concept understanding and generation, and achieves state-of-the-art results in personalized attribute-reasoning generation on the proposed UnifyBench benchmark.

Conclusion: Enhanced understanding improves generation, and generation provides valuable insights for understanding, demonstrating mutual benefits between both tasks in a unified vision language model.

Abstract: Personalized models have demonstrated remarkable success in understanding and generating concepts provided by users. However, existing methods use separate concept tokens for understanding and generation, treating these tasks in isolation. This may result in limitations for generating images with complex prompts. For example, given the concept $\langle bo\rangle$, generating “$\langle bo\rangle$ wearing its hat” without additional textual descriptions of its hat. We call this kind of generation \textit{\textbf{personalized attribute-reasoning generation}}. To address the limitation, we present UniCTokens, a novel framework that effectively integrates personalized information into a unified vision language model (VLM) for understanding and generation. UniCTokens trains a set of unified concept tokens to leverage complementary semantics, boosting two personalized tasks. Moreover, we propose a progressive training strategy with three stages: understanding warm-up, bootstrapping generation from understanding, and deepening understanding from generation to enhance mutual benefits between both tasks. To quantitatively evaluate the unified VLM personalization, we present UnifyBench, the first benchmark for assessing concept understanding, concept generation, and attribute-reasoning generation. Experimental results on UnifyBench indicate that UniCTokens shows competitive performance compared to leading methods in concept understanding, concept generation, and achieving state-of-the-art results in personalized attribute-reasoning generation. Our research demonstrates that enhanced understanding improves generation, and the generation process can yield valuable insights into understanding. Our code and dataset will be released at: \href{https://github.com/arctanxarc/UniCTokens}{https://github.com/arctanxarc/UniCTokens}.

Method: Proposes GMatch, a geometry-constrained keypoint matcher that uses geometric constraints to establish globally consistent correspondences between keypoint descriptors, enabling easy 6DoF pose solving.

Result: GMatch beats existing keypoint matchers on HOPE and YCB-Video datasets across three descriptors, approaches SOTA zero-shot method performance on texture-rich objects, and achieves high task success rates when deployed on a LoCoBot mobile manipulator.

Conclusion: GMatch offers a practical solution for resource-limited robotic systems through its lightweight and white-box nature, presenting a promising direction for robust yet efficient pose estimation, though currently limited by descriptor quality.

Abstract: 6DoF object pose estimation is fundamental to robotic grasp tasks. While recent learning-based methods achieve high accuracy, their computational demands hinder deployment on resource-constrained mobile platforms. In this work, we revisit the classical keypoint matching paradigm and propose GMatch, a lightweight, geometry-constrained keypoint matcher that can run efficiently on embedded CPU-only platforms. GMatch works with keypoint descriptors and it uses a set of geometric constraints to establishes inherent ambiguities between features extracted by descriptors, thus giving a globally consistent correspondences from which 6DoF pose can be easily solved. We benchmark GMatch on the HOPE and YCB-Video datasets, where our method beats existing keypoint matchers (both feature-based and geometry-based) among three commonly used descriptors and approaches the SOTA zero-shot method on texture-rich objects with much more humble devices. The method is further deployed on a LoCoBot mobile manipulator, enabling a one-shot grasp pipeline that demonstrates high task success rates in real-world experiments. In a word, by its lightweight and white-box nature, GMatch offers a practical solution for resource-limited robotic systems, and although currently bottlenecked by descriptor quality, the framework presents a promising direction towards robust yet efficient pose estimation. Code will be released soon under Mozilla Public License.

Method: Decomposes HSIs into background and anomaly components, uses TR decomposition for spatial-spectral correlations, introduces unified nonconvex regularizer via TSVD for low-rankness and sparsity of 3D gradient TR factors, and adds generalized nonconvex regularization for anomaly sparsity. Solved using ADMM optimization.

Result: Experimental results on benchmark datasets show the method outperforms existing state-of-the-art approaches in detection accuracy.

Conclusion: HAD-EUNTRFR effectively addresses limitations of existing methods by better capturing background characteristics and achieves superior anomaly detection performance.

Abstract: In recent years, tensor decomposition-based approaches for hyperspectral anomaly detection (HAD) have gained significant attention in the field of remote sensing. However, existing methods often fail to fully leverage both the global correlations and local smoothness of the background components in hyperspectral images (HSIs), which exist in both the spectral and spatial domains. This limitation results in suboptimal detection performance. To mitigate this critical issue, we put forward a novel HAD method named HAD-EUNTRFR, which incorporates an enhanced unified nonconvex tensor ring (TR) factors regularization. In the HAD-EUNTRFR framework, the raw HSIs are first decomposed into background and anomaly components. The TR decomposition is then employed to capture the spatial-spectral correlations within the background component. Additionally, we introduce a unified and efficient nonconvex regularizer, induced by tensor singular value decomposition (TSVD), to simultaneously encode the low-rankness and sparsity of the 3-D gradient TR factors into a unique concise form. The above characterization scheme enables the interpretable gradient TR factors to inherit the low-rankness and smoothness of the original background. To further enhance anomaly detection, we design a generalized nonconvex regularization term to exploit the group sparsity of the anomaly component. To solve the resulting doubly nonconvex model, we develop a highly efficient optimization algorithm based on the alternating direction method of multipliers (ADMM) framework. Experimental results on several benchmark datasets demonstrate that our proposed method outperforms existing state-of-the-art (SOTA) approaches in terms of detection accuracy.

Method: Uses a branched architecture that separates structure modeling and appearance shading, preventing style distortion of 3D structure. Employs identity loss for pre-training through novel view synthesis, allowing reconstruction capabilities to be retained during stylization fine-tuning.

Result: Produces high-quality stylized 3D content with superior blend of style and scene appearance, outperforming existing methods in multi-view consistency and efficiency.

Conclusion: The approach enables direct 3D stylization in under a second using sparse unposed images, achieving better multi-view consistency and efficiency than current state-of-the-art methods.

Abstract: Stylizing 3D scenes instantly while maintaining multi-view consistency and faithfully resembling a style image remains a significant challenge. Current state-of-the-art 3D stylization methods typically involve computationally intensive test-time optimization to transfer artistic features into a pretrained 3D representation, often requiring dense posed input images. In contrast, leveraging recent advances in feed-forward reconstruction models, we demonstrate a novel approach to achieve direct 3D stylization in less than a second using unposed sparse-view scene images and an arbitrary style image. To address the inherent decoupling between reconstruction and stylization, we introduce a branched architecture that separates structure modeling and appearance shading, effectively preventing stylistic transfer from distorting the underlying 3D scene structure. Furthermore, we adapt an identity loss to facilitate pre-training our stylization model through the novel view synthesis task. This strategy also allows our model to retain its original reconstruction capabilities while being fine-tuned for stylization. Comprehensive evaluations, using both in-domain and out-of-domain datasets, demonstrate that our approach produces high-quality stylized 3D content that achieve a superior blend of style and scene appearance, while also outperforming existing methods in terms of multi-view consistency and efficiency.

Method: Developed a hierarchical material recognition method using graph attention networks that leverages taxonomic proximity between classes, and utilized a diverse dataset with images and depth maps.

Result: The model achieves state-of-the-art performance, demonstrates generalization to adverse imaging conditions, benefits from novel views rendered using depth maps, and shows capacity for few-shot learning of new materials.

Conclusion: The proposed taxonomy, dataset, and graph attention network approach effectively enable hierarchical material recognition with strong performance, generalization capabilities, and few-shot learning potential.

Abstract: We introduce a taxonomy of materials for hierarchical recognition from local appearance. Our taxonomy is motivated by vision applications and is arranged according to the physical traits of materials. We contribute a diverse, in-the-wild dataset with images and depth maps of the taxonomy classes. Utilizing the taxonomy and dataset, we present a method for hierarchical material recognition based on graph attention networks. Our model leverages the taxonomic proximity between classes and achieves state-of-the-art performance. We demonstrate the model’s potential to generalize to adverse, real-world imaging conditions, and that novel views rendered using the depth maps can enhance this capability. Finally, we show the model’s capacity to rapidly learn new materials in a few-shot learning setting.

Method: ViGoRL uses reinforcement learning to produce spatially grounded reasoning traces and guide visual attention to task-relevant regions. It features a novel multi-turn RL framework that enables dynamic zooming into predicted coordinates when fine-grained exploration is needed.

Result: ViGoRL consistently outperforms supervised fine-tuning and conventional RL baselines across diverse visual reasoning benchmarks, achieving 86.4% on V*Bench. It improves performance on localizing small GUI elements and visual search, and amplifies other visual behaviors like region exploration and visual verification.

Conclusion: Visually grounded reinforcement learning is a strong paradigm for imbuing models with general-purpose visual reasoning capabilities, with human evaluations confirming the spatial accuracy and helpfulness of the model’s visual references.

Abstract: While reinforcement learning (RL) over chains of thought has significantly advanced language models in tasks such as mathematics and coding, visual reasoning introduces added complexity by requiring models to direct visual attention, interpret perceptual inputs, and ground abstract reasoning in spatial evidence. We introduce ViGoRL (Visually Grounded Reinforcement Learning), a vision-language model trained with RL to explicitly anchor each reasoning step to specific visual coordinates. Inspired by human visual decision-making, ViGoRL learns to produce spatially grounded reasoning traces, guiding visual attention to task-relevant regions at each step. When fine-grained exploration is required, our novel multi-turn RL framework enables the model to dynamically zoom into predicted coordinates as reasoning unfolds. Across a diverse set of visual reasoning benchmarks–including SAT-2 and BLINK for spatial reasoning, Vbench for visual search, and ScreenSpot and VisualWebArena for web-based grounding–ViGoRL consistently outperforms both supervised fine-tuning and conventional RL baselines that lack explicit grounding mechanisms. Incorporating multi-turn RL with zoomed-in visual feedback significantly improves ViGoRL’s performance on localizing small GUI elements and visual search, achieving 86.4% on VBench. Additionally, we find that grounding amplifies other visual behaviors such as region exploration, grounded subgoal setting, and visual verification. Finally, human evaluations show that the model’s visual references are not only spatially accurate but also helpful for understanding model reasoning steps. Our results show that visually grounded RL is a strong paradigm for imbuing models with general-purpose visual reasoning.

Method: End-to-end reinforcement learning framework with composite reward optimization (normalized edit distance, paragraph count accuracy, reading order preservation) using Infinity-Doc-55K dataset and vision-language-model-based parser.

Result: Infinity-Parser achieves new state-of-the-art performance on English and Chinese benchmarks for OCR, table/formula extraction, and reading order detection, outperforming specialist pipelines and general-purpose vision-language models.

Conclusion: The layoutRL framework enables robust document understanding with superior accuracy and structural fidelity, with code and dataset to be publicly released to advance the field.

Abstract: Automated parsing of scanned documents into richly structured, machine-readable formats remains a critical bottleneck in Document AI, as traditional multi-stage pipelines suffer from error propagation and limited adaptability to diverse layouts. We introduce layoutRL, an end-to-end reinforcement learning framework that trains models to be explicitly layout-aware by optimizing a composite reward of normalized edit distance, paragraph count accuracy, and reading order preservation. Leveraging our newly released dataset, Infinity-Doc-55K, which combines 55K high-fidelity synthetic scanned document parsing data with expert-filtered real-world documents, we instantiate layoutRL in a vision-language-model-based parser called Infinity-Parser. Evaluated on English and Chinese benchmarks for OCR, table and formula extraction, and reading order detection, Infinity-Parser achieves new state-of-the-art performance in both accuracy and structural fidelity, outpacing specialist pipelines and general-purpose vision-language models. We will publicly release our code and dataset to accelerate progress in robust document understanding.

Method: Two-stage fine-tuning: 1) curriculum-driven reinforcement learning with difficulty-aware rewards to build reasoning ability, 2) supervised post-tuning to hone instruction following and semantic extraction. Also created TriView2CAD benchmark with 200,000 synthetic and 3,000 real-world orthographic projections.

Result: CReFT-CAD substantially improves reasoning accuracy and out-of-distribution generalizability in real-world scenarios compared to leading VLMs.

Conclusion: The approach offers valuable insights for advancing CAD reasoning research by addressing the limitations of existing methods and providing a comprehensive benchmark for orthographic projection reasoning.

Abstract: Computer-Aided Design (CAD) plays a pivotal role in industrial manufacturing. Orthographic projection reasoning underpins the entire CAD workflow, encompassing design, manufacturing, and simulation. However, prevailing deep-learning approaches employ standard 3D reconstruction pipelines as an alternative, which often introduce imprecise dimensions and limit the parametric editability required for CAD workflows. Recently, some researchers adopt vision-language models (VLMs), particularly supervised fine-tuning (SFT), to tackle CAD-related challenges. SFT shows promise but often devolves into pattern memorization, yielding poor out-of-distribution performance on complex reasoning tasks. To address these gaps, we introduce CReFT-CAD, a two-stage fine-tuning paradigm that first employs a curriculum-driven reinforcement learning stage with difficulty-aware rewards to build reasoning ability steadily, and then applies supervised post-tuning to hone instruction following and semantic extraction. Complementing this, we release TriView2CAD, the first large-scale, open-source benchmark for orthographic projection reasoning, comprising 200,000 synthetic and 3,000 real-world orthographic projections with precise dimension annotations and six interoperable data modalities. We benchmark leading VLMs on orthographic projection reasoning and demonstrate that CReFT-CAD substantially improves reasoning accuracy and out-of-distribution generalizability in real-world scenarios, offering valuable insights for advancing CAD reasoning research.

Method: Proposed VisuRiddles benchmark with tasks across five dimensions and two reasoning categories, and introduced Perceptual Riddle Synthesizer framework for automated riddle generation with fine-grained perceptual descriptions.

Result: Experimental results on VisuRiddles validate that fine-grained visual perception is the principal bottleneck and the synthesis framework significantly enhances MLLM performance on challenging AVR tasks.

Conclusion: The proposed approach effectively addresses abstract visual reasoning limitations in MLLMs through targeted benchmark development and automated data synthesis with perceptual supervision.

Abstract: Recent strides in multimodal large language models (MLLMs) have significantly advanced their performance in many reasoning tasks. However, Abstract Visual Reasoning (AVR) remains a critical challenge, primarily due to limitations in perceiving abstract graphics. To tackle this issue, we investigate the bottlenecks in current MLLMs and synthesize training data to improve their abstract visual perception. First, we propose VisuRiddles, a benchmark for AVR, featuring tasks meticulously constructed to assess models’ reasoning capacities across five core dimensions and two high-level reasoning categories. Second, we introduce the Perceptual Riddle Synthesizer (PRS), an automated framework for generating riddles with fine-grained perceptual descriptions. PRS not only generates valuable training data for abstract graphics but also provides fine-grained perceptual description, crucially allowing for supervision over intermediate reasoning stages and thereby improving both training efficacy and model interpretability. Our extensive experimental results on VisuRiddles empirically validate that fine-grained visual perception is the principal bottleneck and our synthesis framework markedly enhances the performance of contemporary MLLMs on these challenging tasks. Our code and dataset will be released at https://github.com/yh-hust/VisuRiddles

Method: Perception-Reasoning Decoupling modularizes MLLMs to convert multi-modal inputs to textual outputs for external LLM reasoners, plus Visual Perception Optimization (VPO) reinforcement learning to align perceptual outputs with reasoning tasks.

Result: RAPID achieves significant performance gains on multi-modal reasoning benchmarks and enables inference-time scaling with any state-of-the-art LLM reasoner without retraining.

Conclusion: RAPID provides an efficient approach to upgrade MLLM reasoning capabilities by decoupling perception and reasoning, enabling flexible integration with external LLMs and consistent performance improvement.

Abstract: Recent breakthroughs in reasoning language models have significantly advanced text-based reasoning. On the other hand, Multi-modal Large Language Models (MLLMs) still lag behind, hindered by their outdated internal LLMs. Upgrading these is often prohibitively expensive, as it requires complete vision-language alignment retraining which is costly. To address this issue, we introduce Perception-Reasoning Decoupling, which modularizes the MLLM’s reasoning component and makes it easily replaceable. This approach redefines the MLLM’s role to convert multi-modal inputs into detailed textual outputs that can be processed by any powerful, external, text-only LLM reasoners. To align the MLLM’s perceptual output with the final reasoning task, we propose a novel reinforcement learning algorithm called Visual Perception Optimization (VPO). VPO rewards the MLLM based on the correctness of answers generated by the external reasoner to produce faithful and query-relevant captions. Together, this decoupling pipeline and VPO form our Reasoning-Aligned PerceptIon Decoupling (RAPID) approach. Empirical results show that RAPID achieves significant performance gains on multi-modal reasoning benchmarks. Crucially, RAPID enables a novel inference-time scaling paradigm: Once trained with VPO, the MLLM can be paired with any state-of-the-art LLM reasoner for consistent performance improvement without retraining.

Method: ASCD directly steers attention scores during decoding by: (i) positive steering that amplifies automatically identified text-centric attention heads, and (ii) negative steering that dampens critical visual tokens identified on-the-fly. The method requires no additional training and has minimal runtime/memory overhead.

Result: Across five MLLM backbones and three decoding schemes, ASCD reduces hallucination on POPE, CHAIR, and MMHal-Bench by up to 38.2% while improving accuracy on standard VQA benchmarks including MMMU, MM-VET, ScienceQA, TextVQA, and GQA.

Conclusion: Attention steering provides a simple, model-agnostic, and principled approach to achieve safer and more faithful multimodal generation in MLLMs.

Abstract: Multimodal large language models (MLLMs) frequently hallucinate by over-committing to spurious visual cues. Prior remedies-Visual and Instruction Contrastive Decoding (VCD, ICD)-mitigate this issue, yet the mechanism remains opaque. We first empirically show that their improvements systematically coincide with redistributions of cross-modal attention. Building on this insight, we propose Attention-Steerable Contrastive Decoding (ASCD), which directly steers the attention scores during decoding. ASCD combines (i) positive steering, which amplifies automatically mined text-centric heads-stable within a model and robust across domains-with (ii) negative steering, which dampens on-the-fly identified critical visual tokens. The method incurs negligible runtime and memory overhead and requires no additional training. Across five MLLM backbones and three decoding schemes, ASCD reduces hallucination on POPE, CHAIR, and MMHal-Bench by up to 38.2 percent while improving accuracy on standard VQA benchmarks, including MMMU, MM-VET, ScienceQA, TextVQA, and GQA. These results position attention steering as a simple, model-agnostic, and principled route to safer, more faithful multimodal generation.

Method: Zigzag sampling mechanism that alternates between asymmetric prompting to retain subject characteristics, combined with a visual sharing module that transfers visual cues across generated images.

Result: Significantly outperforms previous approaches in generating coherent and consistent visual stories based on both quantitative metrics and qualitative evaluations.

Conclusion: The proposed training-free method effectively enhances subject consistency in visual story generation without requiring resource-intensive fine-tuning.

Abstract: Text-to-image generation models have made significant progress in producing high-quality images from textual descriptions, yet they continue to struggle with maintaining subject consistency across multiple images, a fundamental requirement for visual storytelling. Existing methods attempt to address this by either fine-tuning models on large-scale story visualization datasets, which is resource-intensive, or by using training-free techniques that share information across generations, which still yield limited success. In this paper, we introduce a novel training-free sampling strategy called Zigzag Sampling with Asymmetric Prompts and Visual Sharing to enhance subject consistency in visual story generation. Our approach proposes a zigzag sampling mechanism that alternates between asymmetric prompting to retain subject characteristics, while a visual sharing module transfers visual cues across generated images to %further enforce consistency. Experimental results, based on both quantitative metrics and qualitative evaluations, demonstrate that our method significantly outperforms previous approaches in generating coherent and consistent visual stories. The code is available at https://github.com/Mingxiao-Li/Asymmetry-Zigzag-StoryDiffusion.

Method: Encodes raw motion into continuous latent space using VAE to avoid quantization artifacts. Uses dual-stream Transformer with shared attention to preserve modality-specific routes while enabling controlled bidirectional information flow. Implements generate-then-align three-stage training schedule for stability.

Result: Achieves 2x faster convergence in training loss and up to 4x faster convergence in validation. Maintains state-of-the-art performance on standard motion understanding and generation benchmarks.

Conclusion: The proposed dual-stream architecture with continuous motion encoding effectively reduces cross-modal interference, stabilizes optimization, and accelerates convergence while maintaining high performance in both motion understanding and generation tasks.

Abstract: With the rapid progress of large language models (LLMs), multimodal frameworks that unify understanding and generation have become promising, yet they face increasing complexity as the number of modalities and tasks grows. We observe that motion quantization introduces approximation errors that cap motion quality, and that unifying discrete text and continuous motion within a single-stream backbone amplifies cross-modal interference. Motivated by recent multi-branch Transformer designs that separate signals from different modalities, we propose MotionGPT3, a bimodal motion-language model for both understanding and generation. MotionGPT3 encodes raw motion into a continuous latent space using a variational autoencoder (VAE), thereby avoiding quantization-induced artifacts, while leveraging the semantic prior of pretrained language models. A dual-stream Transformer with shared attention preserves modality-specific routes while enabling controlled, bidirectional information flow, which reduces interference, stabilizing optimization, and empirically accelerates convergence without degrading fidelity. For multimodal joint training, a generate-then-align three-stage schedule further improves stability and limits cross-task interference. Experiments show that MotionGPT3 achieves 2x faster convergence in training loss and up to 4x faster convergence in validation, while maintaining state-of-the-art performance on standard motion understanding and motion generation benchmarks.

Method: Uses complementary captioning (vertex-based for simple parts, VLLM-based for complex parts) to annotate ~221k local parts, then trains LLMs to predict masked parts using geometric instructions and remaining model as input.

Result: Extensive experiments show GeoCAD achieves high generation quality, validity, and text-to-CAD consistency, enabling users to modify any local part while adhering to geometric instructions.

Conclusion: GeoCAD provides an effective solution for local geometry-controllable CAD generation, addressing limitations of existing methods and enhancing design efficiency through user-friendly geometric instruction following.

Abstract: Local geometry-controllable computer-aided design (CAD) generation aims to modify local parts of CAD models automatically, enhancing design efficiency. It also ensures that the shapes of newly generated local parts follow user-specific geometric instructions (e.g., an isosceles right triangle or a rectangle with one corner cut off). However, existing methods encounter challenges in achieving this goal. Specifically, they either lack the ability to follow textual instructions or are unable to focus on the local parts. To address this limitation, we introduce GeoCAD, a user-friendly and local geometry-controllable CAD generation method. Specifically, we first propose a complementary captioning strategy to generate geometric instructions for local parts. This strategy involves vertex-based and VLLM-based captioning for systematically annotating simple and complex parts, respectively. In this way, we caption $\sim$221k different local parts in total. In the training stage, given a CAD model, we randomly mask a local part. Then, using its geometric instruction and the remaining parts as input, we prompt large language models (LLMs) to predict the masked part. During inference, users can specify any local part for modification while adhering to a variety of predefined geometric instructions. Extensive experiments demonstrate the effectiveness of GeoCAD in generation quality, validity and text-to-CAD consistency. Code will be available at https://github.com/Zhanwei-Z/GeoCAD.

Method: Proposes WaveFormer architecture with novel learnable wavelet transform integrating time-domain and frequency-domain features, using WaveletConv module with multi-level wavelet decomposition and depthwise separable convolution for efficiency.

Result: Achieves 95% classification accuracy on EPN612 dataset, outperforming larger models. With INT8 quantization, achieves 6.75 ms inference latency on Intel CPU, enabling real-time deployment.

Conclusion: WaveFormer provides an efficient and compact solution for sEMG gesture recognition that balances high accuracy with low computational requirements, making it suitable for embedded systems.

Abstract: Human-machine interaction, particularly in prosthetic and robotic control, has seen progress with gesture recognition via surface electromyographic (sEMG) signals.However, classifying similar gestures that produce nearly identical muscle signals remains a challenge, often reducing classification accuracy. Traditional deep learning models for sEMG gesture recognition are large and computationally expensive, limiting their deployment on resource-constrained embedded systems. In this work, we propose WaveFormer, a lightweight transformer-based architecture tailored for sEMG gesture recognition. Our model integrates time-domain and frequency-domain features through a novel learnable wavelet transform, enhancing feature extraction. In particular, the WaveletConv module, a multi-level wavelet decomposition layer with depthwise separable convolution, ensures both efficiency and compactness. With just 3.1 million parameters, WaveFormer achieves 95% classification accuracy on the EPN612 dataset, outperforming larger models. Furthermore, when profiled on a laptop equipped with an Intel CPU, INT8 quantization achieves real-time deployment with a 6.75 ms inference latency.

Method: Uses connectivity-preserving tokenization with special tokens for endpoints and hierarchical layers, implements topology-aware reward function with Direct Preference Optimization, and incorporates implicit geodesic features for latent bone selection.

Result: The approach generates more anatomically plausible skeletal structures with superior deformation properties and significantly enhanced topological accuracy.

Conclusion: Auto-Connect’s combination of connectivity-preserving tokenization, reward-guided fine-tuning, and geodesic-aware bone selection enables consistent generation of high-quality skeletal rigs with improved skinning quality.

Abstract: We introduce Auto-Connect, a novel approach for automatic rigging that explicitly preserves skeletal connectivity through a connectivity-preserving tokenization scheme. Unlike previous methods that predict bone positions represented as two joints or first predict points before determining connectivity, our method employs special tokens to define endpoints for each joint’s children and for each hierarchical layer, effectively automating connectivity relationships. This approach significantly enhances topological accuracy by integrating connectivity information directly into the prediction framework. To further guarantee high-quality topology, we implement a topology-aware reward function that quantifies topological correctness, which is then utilized in a post-training phase through reward-guided Direct Preference Optimization. Additionally, we incorporate implicit geodesic features for latent top-k bone selection, which substantially improves skinning quality. By leveraging geodesic distance information within the model’s latent space, our approach intelligently determines the most influential bones for each vertex, effectively mitigating common skinning artifacts. This combination of connectivity-preserving tokenization, reward-guided fine-tuning, and geodesic-aware bone selection enables our model to consistently generate more anatomically plausible skeletal structures with superior deformation properties.

Method: Develops Gemini Motion Video Retriever mechanism for effective motion-centered video retrieval and Motion-centric Dual-alignment DPO Trainer to mitigate error propagation from suboptimal retrieval results.

Result: Experimental results show VimoRAG significantly boosts the performance of motion LLMs constrained to text-only input.

Conclusion: VimoRAG effectively addresses key bottlenecks in video-based motion RAG and enhances motion generation capabilities of LLMs through retrieval augmentation.

Abstract: This paper introduces VimoRAG, a novel video-based retrieval-augmented motion generation framework for motion large language models (LLMs). As motion LLMs face severe out-of-domain/out-of-vocabulary issues due to limited annotated data, VimoRAG leverages large-scale in-the-wild video databases to enhance 3D motion generation by retrieving relevant 2D human motion signals. While video-based motion RAG is nontrivial, we address two key bottlenecks: (1) developing an effective motion-centered video retrieval model that distinguishes human poses and actions, and (2) mitigating the issue of error propagation caused by suboptimal retrieval results. We design the Gemini Motion Video Retriever mechanism and the Motion-centric Dual-alignment DPO Trainer, enabling effective retrieval and generation processes. Experimental results show that VimoRAG significantly boosts the performance of motion LLMs constrained to text-only input. All the resources are available at https://walkermitty.github.io/VimoRAG/

Method: Two-pass framework: first pass uses adaptive noise injection (AdaNI) with age/gender prompts for age accuracy; second pass uses identity embeddings (SVR-ArcFace and Rotate-CLIP) for identity preservation while maintaining age features.

Result: Outperforms existing methods on CelebA-HQ dataset in both age accuracy and identity consistency, as evaluated by Face++ and Qwen-VL protocols.

Conclusion: The proposed Cradle2Cane framework successfully addresses the Age-ID trade-off in face aging through its two-pass approach, achieving superior performance in both age transformation accuracy and identity preservation.

Abstract: Face aging has become a crucial task in computer vision, with applications ranging from entertainment to healthcare. However, existing methods struggle with achieving a realistic and seamless transformation across the entire lifespan, especially when handling large age gaps or extreme head poses. The core challenge lies in balancing age accuracy and identity preservation–what we refer to as the Age-ID trade-off. Most prior methods either prioritize age transformation at the expense of identity consistency or vice versa. In this work, we address this issue by proposing a two-pass face aging framework, named Cradle2Cane, based on few-step text-to-image (T2I) diffusion models. The first pass focuses on solving age accuracy by introducing an adaptive noise injection (AdaNI) mechanism. This mechanism is guided by including prompt descriptions of age and gender for the given person as the textual condition. Also, by adjusting the noise level, we can control the strength of aging while allowing more flexibility in transforming the face. However, identity preservation is weakly ensured here to facilitate stronger age transformations. In the second pass, we enhance identity preservation while maintaining age-specific features by conditioning the model on two identity-aware embeddings (IDEmb): SVR-ArcFace and Rotate-CLIP. This pass allows for denoising the transformed image from the first pass, ensuring stronger identity preservation without compromising the aging accuracy. Both passes are jointly trained in an end-to-end way. Extensive experiments on the CelebA-HQ test dataset, evaluated through Face++ and Qwen-VL protocols, show that our Cradle2Cane outperforms existing face aging methods in age accuracy and identity consistency. Code is available at https://github.com/byliutao/Cradle2Cane.

Method: Integrates geometry and perception priors to initialize Gaussian branches and guide optimization, uses stable Score Distillation Sampling for fine-grained prior distillation, and employs reprojection-based strategy for depth consistency.

Result: Outperforms existing methods on novel view synthesis and 3D reconstruction, demonstrating robust and consistent 3D object generation.

Conclusion: The proposed approach successfully addresses multiview consistency and geometric detail issues in 3D object generation from single images without requiring additional model training.

Abstract: Generating realistic 3D objects from single-view images requires natural appearance, 3D consistency, and the ability to capture multiple plausible interpretations of unseen regions. Existing approaches often rely on fine-tuning pretrained 2D diffusion models or directly generating 3D information through fast network inference or 3D Gaussian Splatting, but their results generally suffer from poor multiview consistency and lack geometric detail. To tackle these issues, we present a novel method that seamlessly integrates geometry and perception information without requiring additional model training to reconstruct detailed 3D objects from a single image. Specifically, we incorporate geometry and perception priors to initialize the Gaussian branches and guide their parameter optimization. The geometry prior captures the rough 3D shapes, while the perception prior utilizes the 2D pretrained diffusion model to enhance multiview information. Subsequently, we introduce a stable Score Distillation Sampling for fine-grained prior distillation to ensure effective knowledge transfer. The model is further enhanced by a reprojection-based strategy that enforces depth consistency. Experimental results show that we outperform existing methods on novel view synthesis and 3D reconstruction, demonstrating robust and consistent 3D object generation.

Method: Gradient-Guided Distillation (G$^{2}$D) with custom loss function fusing unimodal and multimodal objectives, plus dynamic sequential modality prioritization (SMP) to ensure each modality leads learning.

Result: G$^{2}$D amplifies weak modality significance during training and outperforms state-of-the-art methods on multiple real-world datasets for classification and regression tasks.

Conclusion: G$^{2}$D effectively addresses modality imbalance in multimodal learning through knowledge distillation and sequential modality prioritization, improving overall model performance.

Abstract: Multimodal learning aims to leverage information from diverse data modalities to achieve more comprehensive performance. However, conventional multimodal models often suffer from modality imbalance, where one or a few modalities dominate model optimization, leading to suboptimal feature representation and underutilization of weak modalities. To address this challenge, we introduce Gradient-Guided Distillation (G$^{2}$D), a knowledge distillation framework that optimizes the multimodal model with a custom-built loss function that fuses both unimodal and multimodal objectives. G$^{2}$D further incorporates a dynamic sequential modality prioritization (SMP) technique in the learning process to ensure each modality leads the learning process, avoiding the pitfall of stronger modalities overshadowing weaker ones. We validate G$^{2}$D on multiple real-world datasets and show that G$^{2}$D amplifies the significance of weak modalities while training and outperforms state-of-the-art methods in classification and regression tasks. Our code is available at https://github.com/rAIson-Lab/G2D.

Method: Uses pre-trained DINOv2 vision transformer to quantify temporal curvature and stepwise distance in representation domain, then trains classifier on aggregated statistics.

Result: Achieves state-of-the-art performance with 97.17% accuracy and 98.63% AUROC on VidProM benchmark, substantially outperforming existing methods.

Conclusion: ReStraV provides computationally efficient detection using neural representation geometry, offering new insights for AI-generated video authentication.

Abstract: The rapid advancement of generative AI enables highly realistic synthetic videos, posing significant challenges for content authentication and raising urgent concerns about misuse. Existing detection methods often struggle with generalization and capturing subtle temporal inconsistencies. We propose ReStraV(Representation Straightening Video), a novel approach to distinguish natural from AI-generated videos. Inspired by the “perceptual straightening” hypothesis – which suggests real-world video trajectories become more straight in neural representation domain – we analyze deviations from this expected geometric property. Using a pre-trained self-supervised vision transformer (DINOv2), we quantify the temporal curvature and stepwise distance in the model’s representation domain. We aggregate statistics of these measures for each video and train a classifier. Our analysis shows that AI-generated videos exhibit significantly different curvature and distance patterns compared to real videos. A lightweight classifier achieves state-of-the-art detection performance (e.g., 97.17% accuracy and 98.63% AUROC on the VidProM benchmark), substantially outperforming existing image- and video-based methods. ReStraV is computationally efficient, it is offering a low-cost and effective detection solution. This work provides new insights into using neural representation geometry for AI-generated video detection.

Method: Formulates HOI generation as driver-responder system with lightweight transformer-based interaction dynamics model that predicts object responses to human motion, plus residual-based dynamics loss to mitigate prediction errors. The dynamics model is only used during training.

Result: Extensive experiments show the approach enhances HOI generation quality and establishes a feasible metric for evaluating generated interactions.

Conclusion: HOI-Dyn successfully improves 3D human-object interaction generation by explicitly modeling interaction dynamics while maintaining inference efficiency.

Abstract: Generating realistic 3D human-object interactions (HOIs) remains a challenging task due to the difficulty of modeling detailed interaction dynamics. Existing methods treat human and object motions independently, resulting in physically implausible and causally inconsistent behaviors. In this work, we present HOI-Dyn, a novel framework that formulates HOI generation as a driver-responder system, where human actions drive object responses. At the core of our method is a lightweight transformer-based interaction dynamics model that explicitly predicts how objects should react to human motion. To further enforce consistency, we introduce a residual-based dynamics loss that mitigates the impact of dynamics prediction errors and prevents misleading optimization signals. The dynamics model is used only during training, preserving inference efficiency. Through extensive qualitative and quantitative experiments, we demonstrate that our approach not only enhances the quality of HOI generation but also establishes a feasible metric for evaluating the quality of generated interactions.

Method: Uses hierarchical coresets selection with theoretically guaranteed importance function considering utility, representativeness, robustness, and synergy to progressively refine selected regions.

Result: HCS enables VLMs to achieve rapid understanding of unseen scenes at any scale using minimal interpretable regions, with superior performance and universality in various tasks.

Conclusion: HCS is an effective plug-and-play method that enhances VLM adaptation to complex wide-area scenes without requiring additional fine-tuning.

Abstract: Scene understanding is one of the core tasks in computer vision, aiming to extract semantic information from images to identify objects, scene categories, and their interrelationships. Although advancements in Vision-Language Models (VLMs) have driven progress in this field, existing VLMs still face challenges in adaptation to unseen complex wide-area scenes. To address the challenges, this paper proposes a Hierarchical Coresets Selection (HCS) mechanism to advance the adaptation of VLMs in complex wide-area scene understanding. It progressively refines the selected regions based on the proposed theoretically guaranteed importance function, which considers utility, representativeness, robustness, and synergy. Without requiring additional fine-tuning, HCS enables VLMs to achieve rapid understandings of unseen scenes at any scale using minimal interpretable regions while mitigating insufficient feature density. HCS is a plug-and-play method that is compatible with any VLM. Experiments demonstrate that HCS achieves superior performance and universality in various tasks.

Method: The authors incorporate various OCR tasks into evaluation, categorize 33 representative tasks into five categories, and examine six models across closed-source and open-source domains using tailored high-quality image inputs and prompts.

Result: The evaluation identifies crucial observations and weaknesses of current generative models for OCR tasks, showing limitations in their text image generation and editing capabilities.

Conclusion: The authors argue that photorealistic text image generation and editing should be internalized as foundational skills into general-domain generative models rather than delegated to specialized solutions, and provide empirical analysis to guide future development.

Abstract: Text image is a unique and crucial information medium that integrates visual aesthetics and linguistic semantics in modern e-society. Due to their subtlety and complexity, the generation of text images represents a challenging and evolving frontier in the image generation field. The recent surge of specialized image generators (\emph{e.g.}, Flux-series) and unified generative models (\emph{e.g.}, GPT-4o), which demonstrate exceptional fidelity, raises a natural question: can they master the intricacies of text image generation and editing? Motivated by this, we assess current state-of-the-art generative models’ capabilities in terms of text image generation and editing. We incorporate various typical optical character recognition (OCR) tasks into our evaluation and broaden the concept of text-based generation tasks into OCR generative tasks. We select 33 representative tasks and categorize them into five categories: document, handwritten text, scene text, artistic text, and complex & layout-rich text. For comprehensive evaluation, we examine six models across both closed-source and open-source domains, using tailored, high-quality image inputs and prompts. Through this evaluation, we draw crucial observations and identify the weaknesses of current generative models for OCR tasks. We argue that photorealistic text image generation and editing should be internalized as foundational skills into general-domain generative models, rather than being delegated to specialized solutions, and we hope this empirical analysis can provide valuable insights for the community to achieve this goal. This evaluation is online and will be continuously updated at our GitHub repository.

Method: Interpret attention matrix as Markov chain transition matrix; identify metastable states where attention concentrates; compute TokenRank as steady state vector; use matrix multiplication and eigenanalysis for lightweight computation.

Result: Achieves state-of-the-art zero-shot segmentation; TokenRank improves unconditional image generation quality (IS) and diversity (FID); enhances existing segmentation techniques; provides fresh perspective on token attention in visual transformers.

Conclusion: The Markov chain interpretation offers a powerful framework for analyzing attention mechanisms, revealing metastable states and enabling practical applications in segmentation and generation tasks through lightweight computational tools.

Abstract: We introduce a new interpretation of the attention matrix as a discrete-time Markov chain. Our interpretation sheds light on common operations involving attention scores such as selection, summation, and averaging in a unified framework. It further extends them by considering indirect attention, propagated through the Markov chain, as opposed to previous studies that only model immediate effects. Our key observation is that tokens linked to semantically similar regions form metastable states, i.e., regions where attention tends to concentrate, while noisy attention scores dissipate. Metastable states and their prevalence can be easily computed through simple matrix multiplication and eigenanalysis, respectively. Using these lightweight tools, we demonstrate state-of-the-art zero-shot segmentation. Lastly, we define TokenRank – the steady state vector of the Markov chain, which measures global token importance. We show that TokenRank enhances unconditional image generation, improving both quality (IS) and diversity (FID), and can also be incorporated into existing segmentation techniques to improve their performance over existing benchmarks. We believe our framework offers a fresh view of how tokens are being attended in modern visual transformers.

Method: Uses physics-inspired self-attention module with depth-dependent circle of confusion constraint and self-occlusion effects. Adapts diffusion model to one-step inference without additional noise. Synthesizes photorealistic foregrounds with transparency using diffusion models for training data.

Result: Achieves physically accurate and visually appealing lens blur outcomes with high quality and fidelity. Overcomes artifacts in depth discontinuities that plague previous methods.

Conclusion: BokehDiff successfully integrates generative diffusion prior with physics-inspired constraints to produce superior lens blur rendering, addressing key limitations in existing approaches.

Abstract: We introduce BokehDiff, a novel lens blur rendering method that achieves physically accurate and visually appealing outcomes, with the help of generative diffusion prior. Previous methods are bounded by the accuracy of depth estimation, generating artifacts in depth discontinuities. Our method employs a physics-inspired self-attention module that aligns with the image formation process, incorporating depth-dependent circle of confusion constraint and self-occlusion effects. We adapt the diffusion model to the one-step inference scheme without introducing additional noise, and achieve results of high quality and fidelity. To address the lack of scalable paired data, we propose to synthesize photorealistic foregrounds with transparency with diffusion models, balancing authenticity and scene diversity.

Method: Proposes ScreenCoder framework with three specialized agents: grounding agent for visual perception, planning agent for layout planning, and generation agent for code synthesis. Also uses the framework as a data engine to generate high-quality image-code pairs for fine-tuning open-source MLLMs via supervised fine-tuning and reinforcement learning.

Result: Achieves state-of-the-art performance in layout accuracy, structural coherence, and code correctness. The approach demonstrates substantial gains in UI generation capabilities through the dual-stage fine-tuning pipeline.

Conclusion: ScreenCoder’s modular multi-agent approach provides higher robustness and fidelity than end-to-end methods, successfully addressing the limitations of monolithic MLLMs in complex UI-to-code transformation tasks.

Abstract: Automating the transformation of user interface (UI) designs into front-end code holds significant promise for accelerating software development and democratizing design workflows. While multimodal large language models (MLLMs) can translate images to code, they often fail on complex UIs, struggling to unify visual perception, layout planning, and code synthesis within a single monolithic model, which leads to frequent perception and planning errors. To address this, we propose ScreenCoder, a modular multi-agent framework that decomposes the task into three interpretable stages: grounding, planning, and generation. By assigning these distinct responsibilities to specialized agents, our framework achieves significantly higher robustness and fidelity than end-to-end approaches. Furthermore, ScreenCoder serves as a scalable data engine, enabling us to generate high-quality image-code pairs. We use this data to fine-tune open-source MLLM via a dual-stage pipeline of supervised fine-tuning and reinforcement learning, demonstrating substantial gains in its UI generation capabilities. Extensive experiments demonstrate that our approach achieves state-of-the-art performance in layout accuracy, structural coherence, and code correctness. Our code is made publicly available at https://github.com/leigest519/ScreenCoder.

Method: SegDAC uses Segment Anything (SAM) for object-centric decomposition and YOLO-World to ground segmentation via text inputs. It employs a transformer-based architecture that supports dynamic segments and learns which segments to focus on using online RL without human labels.

Result: SegDAC achieves significantly better visual generalization, doubling prior performance on the hardest setting and matching or surpassing prior methods in sample efficiency across all evaluated tasks on the Maniskill3 benchmark.

Conclusion: SegDAC effectively integrates segmentation models into RL, enabling superior visual generalization and sample efficiency in challenging manipulation tasks under strong visual perturbations.

Abstract: Visual reinforcement learning (RL) is challenging due to the need to extract useful representations from high-dimensional inputs while learning effective control from sparse and noisy rewards. Although large perception models exist, integrating them effectively into RL for visual generalization and improved sample efficiency remains difficult. We propose SegDAC, a Segmentation-Driven Actor-Critic method. SegDAC uses Segment Anything (SAM) for object-centric decomposition and YOLO-World to ground the image segmentation process via text inputs. It includes a novel transformer-based architecture that supports a dynamic number of segments at each time step and effectively learns which segments to focus on using online RL, without using human labels. By evaluating SegDAC over a challenging visual generalization benchmark using Maniskill3, which covers diverse manipulation tasks under strong visual perturbations, we demonstrate that SegDAC achieves significantly better visual generalization, doubling prior performance on the hardest setting and matching or surpassing prior methods in sample efficiency across all evaluated tasks.

Method: Proposed loss functions that promote interpretability by generating sparse and localized feature maps, allowing identification of which image regions contribute to predicted control commands. Conducted ablation studies on feature extraction and validated on CARLA benchmarks.

Result: The method improves interpretability while reducing infractions, achieving the highest route completion rate and lower infraction scores than top-performing approaches on CARLA Leaderboard. The monocular, non-ensemble model surpasses existing methods.

Conclusion: The approach successfully enhances interpretability in autonomous driving while maintaining high performance, demonstrating that interpretable AI can yield safer driving models without compromising on route completion capabilities.

Abstract: Trustworthy AI is mandatory for the broad deployment of autonomous vehicles. Although end-to-end approaches derive control commands directly from raw data, interpreting these decisions remains challenging, especially in complex urban scenarios. This is mainly attributed to very deep neural networks with non-linear decision boundaries, making it challenging to grasp the logic behind AI-driven decisions. This paper presents a method to enhance interpretability while optimizing control commands in autonomous driving. To address this, we propose loss functions that promote the interpretability of our model by generating sparse and localized feature maps. The feature activations allow us to explain which image regions contribute to the predicted control command. We conduct comprehensive ablation studies on the feature extraction step and validate our method on the CARLA benchmarks. We also demonstrate that our approach improves interpretability, which correlates with reducing infractions, yielding a safer, high-performance driving model. Notably, our monocular, non-ensemble model surpasses the top-performing approaches from the CARLA Leaderboard by achieving lower infraction scores and the highest route completion rate, all while ensuring interpretability.

Method: Proposes FlowDet with decoupled encoder optimization on DETR architecture, featuring Geometric Deformable Unit (GDU) for traffic-aware geometric modeling and Scale-Aware Attention (SAA) module to handle extreme scale variations.

Result: On Intersection-Flow-5k dataset, FlowDet improves AP(test) by 1.5% and AP50(test) by 1.6% over RT-DETR baseline, while reducing GFLOPs by 63.2% and increasing inference speed by 16.2%.

Conclusion: FlowDet demonstrates a new path for building highly efficient and accurate detectors for demanding real-world perception systems, showing significant improvements in both accuracy and computational efficiency.

Abstract: End-to-end object detectors offer a promising NMS-free paradigm for real-time applications, yet their high computational cost remains a significant barrier, particularly for complex scenarios like intersection traffic monitoring. To address this challenge, we propose FlowDet, a high-speed detector featuring a decoupled encoder optimization strategy applied to the DETR architecture. Specifically, FlowDet employs a novel Geometric Deformable Unit (GDU) for traffic-aware geometric modeling and a Scale-Aware Attention (SAA) module to maintain high representational power across extreme scale variations. To rigorously evaluate the model’s performance in environments with severe occlusion and high object density, we collected the Intersection-Flow-5k dataset, a new challenging scene for this task. Evaluated on Intersection-Flow-5k, FlowDet establishes a new state-of-the-art. Compared to the strong RT-DETR baseline, it improves AP(test) by 1.5% and AP50(test) by 1.6%, while simultaneously reducing GFLOPs by 63.2% and increasing inference speed by 16.2%. Our work demonstrates a new path towards building highly efficient and accurate detectors for demanding, real-world perception systems. The Intersection-Flow-5k dataset is available at https://github.com/AstronZh/Intersection-Flow-5K.

Method: Train a NeRF-based image generator using gradients back-propagated through the pose estimation network to render the main 3D features exploited by the estimator.

Result: The method successfully recovers relevant 3D cues and provides insights into the relationship between network supervision and its implicit representation of the target spacecraft.

Conclusion: The proposed visualization approach enhances understanding of pose estimation networks’ decision processes, potentially facilitating their adoption in real spacecraft missions.

Abstract: On-orbit operations require the estimation of the relative 6D pose, i.e., position and orientation, between a chaser spacecraft and its target. While data-driven spacecraft pose estimation methods have been developed, their adoption in real missions is hampered by the lack of understanding of their decision process. This paper presents a method to visualize the 3D visual cues on which a given pose estimator relies. For this purpose, we train a NeRF-based image generator using the gradients back-propagated through the pose estimation network. This enforces the generator to render the main 3D features exploited by the spacecraft pose estimation network. Experiments demonstrate that our method recovers the relevant 3D cues. Furthermore, they offer additional insights on the relationship between the pose estimation network supervision and its implicit representation of the target spacecraft.

Method: Proposes Segmentation Augmented and Selective Averaged Sa2VA (SaSaSa2VA), building on Sa2VA framework by incorporating efficient segmentation augmentation and test-time ensembling techniques.

Result: Achieved J&F score of 67.45 on 7th LSVOS Challenge (RVOS track), ranking first and surpassing runner-up by 2.80 points.

Conclusion: Efficient segmentation augmentation and test-time ensembling substantially enhance grounded MLLMs for RVOS, demonstrating significant performance improvements over existing methods.

Abstract: Referring video object segmentation (RVOS) requires segmenting and tracking objects in videos conditioned on natural-language expressions, demanding fine-grained understanding of both appearance and motion. Building on Sa2VA, which couples a Multi-modal Large Language Model (MLLM) with the video segmentation model SAM2, we identify two key bottlenecks that limit segmentation performance: sparse frame sampling and reliance on a single [SEG] token for an entire video. We propose Segmentation Augmented and Selective Averaged Sa2VA (SaSaSa2VA) to address these issues. On the 7th LSVOS Challenge (RVOS track), SaSaSa2VA achieves a $\mathcal{J&F}$ of 67.45, ranking first and surpassing the runner-up by 2.80 points. This result and ablation studies demonstrate that efficient segmentation augmentation and test-time ensembling substantially enhance grounded MLLMs for RVOS. The code is released in Sa2VA repository: https://github.com/bytedance/Sa2VA.

Method: Project LoRA-adapted models into a common alignment basis (Core Space) with formal proof that this preserves all information, enabling efficient merging while maintaining low-rank structure.

Result: Significantly improves existing merging techniques, achieves state-of-the-art results on vision and language tasks, and uses only a fraction of computational resources compared to traditional methods.

Conclusion: Core Space merging framework successfully preserves the efficiency of low-rank adaptation while substantially improving model merging accuracy across multiple domains.

Abstract: In this paper, we address the challenges associated with merging low-rank adaptations of large neural networks. With the rise of parameter-efficient adaptation techniques, such as Low-Rank Adaptation (LoRA), model fine-tuning has become more accessible. While fine-tuning models with LoRA is highly efficient, existing merging methods often sacrifice this efficiency by merging fully-sized weight matrices. We propose the Core Space merging framework, which enables the merging of LoRA-adapted models within a common alignment basis, thereby preserving the efficiency of low-rank adaptation while substantially improving accuracy across tasks. We further provide a formal proof that projection into Core Space ensures no loss of information and provide a complexity analysis showing the efficiency gains. Extensive empirical results demonstrate that Core Space significantly improves existing merging techniques and achieves state-of-the-art results on both vision and language tasks while utilizing a fraction of the computational resources. Codebase is available at https://github.com/apanariello4/core-space-merging.

Method: Curated a 2-million-scale multimodal dataset combining textbook knowledge, public data, synthetic samples, and general corpora. Employed three-stage training: domain-specialized pretraining, instruction-driven alignment, and reinforcement-based refinement. Dolphin R1 enhances diagnostic inference through reinforcement learning with ultrasound-specific rewards.

Result: Dolphin R1 achieves a U2-score of 0.5835 on U2-Bench across eight ultrasound tasks - over twice the second-best model (0.2968), setting a new state of the art. Dolphin v1.0 also performs competitively.

Conclusion: The Dolphin series demonstrates that reasoning-enhanced training significantly improves diagnostic accuracy, consistency, and interpretability, highlighting its importance for high-stakes medical AI applications in ultrasound.

Abstract: Ultrasound is crucial in modern medicine but faces challenges like operator dependence, image noise, and real-time scanning, hindering AI integration. While large multimodal models excel in other medical imaging areas, they struggle with ultrasound’s complexities. To address this, we introduce Dolphin v1.0 (V1) and its reasoning-augmented version, Dolphin R1-the first large-scale multimodal ultrasound foundation models unifying diverse clinical tasks in a single vision-language framework.To tackle ultrasound variability and noise, we curated a 2-million-scale multimodal dataset, combining textbook knowledge, public data, synthetic samples, and general corpora. This ensures robust perception, generalization, and clinical adaptability.The Dolphin series employs a three-stage training strategy: domain-specialized pretraining, instruction-driven alignment, and reinforcement-based refinement. Dolphin v1.0 delivers reliable performance in classification, detection, regression, and report generation. Dolphin R1 enhances diagnostic inference, reasoning transparency, and interpretability through reinforcement learning with ultrasound-specific rewards.Evaluated on U2-Bench across eight ultrasound tasks, Dolphin R1 achieves a U2-score of 0.5835-over twice the second-best model (0.2968) setting a new state of the art. Dolphin v1.0 also performs competitively, validating the unified framework. Comparisons show reasoning-enhanced training significantly improves diagnostic accuracy, consistency, and interpretability, highlighting its importance for high-stakes medical AI.

Method: Introduces a SIM(3)-equivariant shape completion network with modular layers that canonicalize features, reason over similarity-invariant geometry, and restore the original frame. Uses a de-biased evaluation protocol.

Result: Outperforms both equivariant and augmentation baselines on PCN benchmark. Sets new cross-domain records: lowers minimal matching distance on KITTI by 17% and Chamfer distance on OmniObject3D by 14%. Even under stricter protocol, outperforms competitors under their biased settings.

Conclusion: Full SIM(3) equivariance is an effective route to truly generalizable shape completion, establishing robustness across different domains and evaluation protocols.

Abstract: 3D shape completion methods typically assume scans are pre-aligned to a canonical frame. This leaks pose and scale cues that networks may exploit to memorize absolute positions rather than inferring intrinsic geometry. When such alignment is absent in real data, performance collapses. We argue that robust generalization demands architectural equivariance to the similarity group, SIM(3), so the model remains agnostic to pose and scale. Following this principle, we introduce the first SIM(3)-equivariant shape completion network, whose modular layers successively canonicalize features, reason over similarity-invariant geometry, and restore the original frame. Under a de-biased evaluation protocol that removes the hidden cues, our model outperforms both equivariant and augmentation baselines on the PCN benchmark. It also sets new cross-domain records on real driving and indoor scans, lowering minimal matching distance on KITTI by 17% and Chamfer distance $\ell1$ on OmniObject3D by 14%. Perhaps surprisingly, ours under the stricter protocol still outperforms competitors under their biased settings. These results establish full SIM(3) equivariance as an effective route to truly generalizable shape completion. Project page: https://sime-completion.github.io.

Method: Proposed DragStream with adaptive distribution self-rectification strategy and spatial-frequency selective optimization mechanism to constrain latent drift and mitigate context interference.

Result: DragStream can be seamlessly integrated into existing autoregressive video diffusion models and effectively enables streaming drag-oriented interactive video manipulation.

Conclusion: The proposed DragStream approach successfully resolves the REVEL task, providing versatile drag operations for video editing and animation with translation, deformation, and rotation effects.

Abstract: Achieving streaming, fine-grained control over the outputs of autoregressive video diffusion models remains challenging, making it difficult to ensure that they consistently align with user expectations. To bridge this gap, we propose \textbf{stReaming drag-oriEnted interactiVe vidEo manipuLation (REVEL)}, a new task that enables users to modify generated videos \emph{anytime} on \emph{anything} via fine-grained, interactive drag. Beyond DragVideo and SG-I2V, REVEL unifies drag-style video manipulation as editing and animating video frames with both supporting user-specified translation, deformation, and rotation effects, making drag operations versatile. In resolving REVEL, we observe: \emph{i}) drag-induced perturbations accumulate in latent space, causing severe latent distribution drift that halts the drag process; \emph{ii}) streaming drag is easily disturbed by context frames, thereby yielding visually unnatural outcomes. We thus propose a training-free approach, \textbf{DragStream}, comprising: \emph{i}) an adaptive distribution self-rectification strategy that leverages neighboring frames’ statistics to effectively constrain the drift of latent embeddings; \emph{ii}) a spatial-frequency selective optimization mechanism, allowing the model to fully exploit contextual information while mitigating its interference via selectively propagating visual cues along generation. Our method can be seamlessly integrated into existing autoregressive video diffusion models, and extensive experiments firmly demonstrate the effectiveness of our DragStream.

Method: The authors propose using intermediate layers of pre-trained models and introduce an entropy-based criterion to automatically identify layers that provide the most complementary information for OOD detection, all in a training-free setting without requiring access to OOD data.

Result: The method increases OOD detection accuracy by up to 10% in far-OOD benchmarks and over 7% in near-OOD benchmarks compared to state-of-the-art training-free methods, across various model architectures and training objectives.

Conclusion: This work reveals a new research direction for OOD detection by leveraging intermediate layer representations and demonstrates how different training objectives and model architectures affect confidence-based OOD detection methods.

Abstract: Out-of-distribution (OOD) detection is essential for reliably deploying machine learning models in the wild. Yet, most methods treat large pre-trained models as monolithic encoders and rely solely on their final-layer representations for detection. We challenge this wisdom. We reveal the \textit{intermediate layers} of pre-trained models, shaped by residual connections that subtly transform input projections, \textit{can} encode \textit{surprisingly rich and diverse signals} for detecting distributional shifts. Importantly, to exploit latent representation diversity across layers, we introduce an entropy-based criterion to \textit{automatically} identify layers offering the most complementary information in a training-free setting – \textit{without access to OOD data}. We show that selectively incorporating these intermediate representations can increase the accuracy of OOD detection by up to \textbf{$10%$} in far-OOD and over \textbf{$7%$} in near-OOD benchmarks compared to state-of-the-art training-free methods across various model architectures and training objectives. Our findings reveal a new avenue for OOD detection research and uncover the impact of various training objectives and model architectures on confidence-based OOD detection methods.

Method: Systematic analysis of pooling-, convolution-, and attention-based token mixers within MetaFormer architecture on 8 medical datasets covering classification and segmentation tasks, including transfer learning from pretrained weights.

Result: For classification: low-complexity token mixers (grouped convolution or pooling) are sufficient. For segmentation: convolutional token mixers with local inductive bias are essential, with grouped convolutions preferred for efficiency.

Conclusion: Grouped convolutions emerge as optimal token mixer for medical imaging, reducing runtime/parameters while MetaFormer’s channel-MLPs provide necessary cross-channel interactions. Pretrained weights remain useful despite domain gap.

Abstract: The generalization of the Transformer architecture via MetaFormer has reshaped our understanding of its success in computer vision. By replacing self-attention with simpler token mixers, MetaFormer provides strong baselines for vision tasks. However, while extensively studied on natural image datasets, its use in medical imaging remains scarce, and existing works rarely compare different token mixers, potentially overlooking more suitable designs choices. In this work, we present the first comprehensive study of token mixers for medical imaging. We systematically analyze pooling-, convolution-, and attention-based token mixers within the MetaFormer architecture on image classification (global prediction task) and semantic segmentation (dense prediction task). Our evaluation spans eight datasets covering diverse modalities and common challenges in the medical domain. Given the prevalence of pretraining from natural images to mitigate medical data scarcity, we also examine transferring pretrained weights to new token mixers. Our results show that, for classification, low-complexity token mixers (e.g. grouped convolution or pooling) are sufficient, aligning with findings on natural images. Pretrained weights remain useful despite the domain gap introduced by the new token mixer. For segmentation, we find that the local inductive bias of convolutional token mixers is essential. Grouped convolutions emerge as the preferred choice, as they reduce runtime and parameter count compared to standard convolutions, while the MetaFormer’s channel-MLPs already provide the necessary cross-channel interactions.

Method: Two main innovations: 1) dynamic temperature control guided by spatial entropy of token distributions to balance diversity, accuracy, and coherence, and 2) entropy-aware acceptance rules in speculative decoding for faster inference.

Result: Extensive experiments show the approach achieves higher generation quality with faster synthesis speed, achieving near-lossless generation at about 85% of conventional inference costs.

Conclusion: The entropy-informed decoding strategy effectively enhances both generation quality and sampling speed in autoregressive image generation models, demonstrating broad applicability across diverse AR models.

Abstract: In this work, we first revisit the sampling issues in current autoregressive (AR) image generation models and identify that image tokens, unlike text tokens, exhibit lower information density and non-uniform spatial distribution. Accordingly, we present an entropy-informed decoding strategy that facilitates higher autoregressive generation quality with faster synthesis speed. Specifically, the proposed method introduces two main innovations: 1) dynamic temperature control guided by spatial entropy of token distributions, enhancing the balance between content diversity, alignment accuracy, and structural coherence in both mask-based and scale-wise models, without extra computational overhead, and 2) entropy-aware acceptance rules in speculative decoding, achieving near-lossless generation at about 85% of the inference cost of conventional acceleration methods. Extensive experiments across multiple benchmarks using diverse AR image generation models demonstrate the effectiveness and generalizability of our approach in enhancing both generation quality and sampling speed.

Method: Multimodal diffusion model conditioned on cellular/nuclear morphologies (horizontal/vertical maps), RGB color characteristics, and BERT-encoded assay metadata, integrated via multi-head cross-attention.

Result: Synthetic images closely match real data with low Wasserstein distances, and incorporating synthetic samples significantly improves segmentation model accuracy on rare cell types like columnar cells.

Conclusion: Multimodal diffusion-based augmentation effectively enriches datasets and improves robustness of cell/nuclei segmentation models, paving the way for broader generative model applications in computational pathology.

Abstract: Scarcity of annotated data, particularly for rare or atypical morphologies, present significant challenges for cell and nuclei segmentation in computational pathology. While manual annotation is labor-intensive and costly, synthetic data offers a cost-effective alternative. We introduce a Multimodal Semantic Diffusion Model (MSDM) for generating realistic pixel-precise image-mask pairs for cell and nuclei segmentation. By conditioning the generative process with cellular/nuclear morphologies (using horizontal and vertical maps), RGB color characteristics, and BERT-encoded assay/indication metadata, MSDM generates datasests with desired morphological properties. These heterogeneous modalities are integrated via multi-head cross-attention, enabling fine-grained control over the generated images. Quantitative analysis demonstrates that synthetic images closely match real data, with low Wasserstein distances between embeddings of generated and real images under matching biological conditions. The incorporation of these synthetic samples, exemplified by columnar cells, significantly improves segmentation model accuracy on columnar cells. This strategy systematically enriches data sets, directly targeting model deficiencies. We highlight the effectiveness of multimodal diffusion-based augmentation for advancing the robustness and generalizability of cell and nuclei segmentation models. Thereby, we pave the way for broader application of generative models in computational pathology.

Method: Uses LLMs to generate descriptive tags from item titles/descriptions, fuses tag embeddings with item identifiers and features, and employs a Tag-Enriched Multi-Attention mechanism to model user preferences within and across domains.

Result: Extensive experiments on four large-scale e-commerce datasets show TEMA-LLM consistently outperforms state-of-the-art baselines.

Conclusion: The approach demonstrates the potential of LLMs to advance intelligent, user-centric services in consumer electronics through semantic tagging and multi-attention integration.

Abstract: Cross-Domain Sequential Recommendation (CDSR) plays a crucial role in modern consumer electronics and e-commerce platforms, where users interact with diverse services such as books, movies, and online retail products. These systems must accurately capture both domain-specific and cross-domain behavioral patterns to provide personalized and seamless consumer experiences. To address this challenge, we propose \textbf{TEMA-LLM} (\textit{Tag-Enriched Multi-Attention with Large Language Models}), a practical and effective framework that integrates \textit{Large Language Models (LLMs)} for semantic tag generation and enrichment. Specifically, TEMA-LLM employs LLMs to assign domain-aware prompts and generate descriptive tags from item titles and descriptions. The resulting tag embeddings are fused with item identifiers as well as textual and visual features to construct enhanced item representations. A \textit{Tag-Enriched Multi-Attention} mechanism is then introduced to jointly model user preferences within and across domains, enabling the system to capture complex and evolving consumer interests. Extensive experiments on four large-scale e-commerce datasets demonstrate that TEMA-LLM consistently outperforms state-of-the-art baselines, underscoring the benefits of LLM-based semantic tagging and multi-attention integration for consumer-facing recommendation systems. The proposed approach highlights the potential of LLMs to advance intelligent, user-centric services in the field of consumer electronics.

Method: Evaluated supervised YOLO and zero-shot models (Gemini Flash 2.5, GPT-4) on 5,000 stratified COCO images and 500 diverse product images, combined with Total Cost of Ownership modeling to derive break-even thresholds.

Result: Supervised YOLO achieved 91.2% accuracy vs 68.5% for Gemini and 71.3% for GPT-4 on standard categories, but requires $10,800 annotation cost for 100 categories. The accuracy advantage only pays off beyond 55 million inferences. On diverse products, zero-shot models achieved 52.3%-55.1% accuracy while YOLO cannot detect untrained classes. Cost-per-detection favors zero-shot models at 100,000 inferences.

Conclusion: Optimal architecture choice depends on inference volume, category stability, budget, and accuracy requirements, with zero-shot models being more cost-effective for lower inference volumes and dynamic categories.

Abstract: Object detection traditionally relies on costly manual annotation. We present the first comprehensive cost-effectiveness analysis comparing supervised YOLO and zero-shot vision-language models (Gemini Flash 2.5 and GPT-4). Evaluated on 5,000 stratified COCO images and 500 diverse product images, combined with Total Cost of Ownership modeling, we derive break-even thresholds for architecture selection. Results show supervised YOLO attains 91.2% accuracy versus 68.5% for Gemini and 71.3% for GPT-4 on standard categories; the annotation expense for a 100-category system is $10,800, and the accuracy advantage only pays off beyond 55 million inferences (151,000 images/day for one year). On diverse product categories Gemini achieves 52.3% and GPT-4 55.1%, while supervised YOLO cannot detect untrained classes. Cost-per-correct-detection favors Gemini ($0.00050) and GPT-4 ($0.00067) over YOLO ($0.143) at 100,000 inferences. We provide decision frameworks showing that optimal architecture choice depends on inference volume, category stability, budget, and accuracy requirements.

Method: Enforces 2D Gaussians to produce multi-view consistent material maps during deferred shading, tracks photometric variations across views to identify reflective regions, and introduces ray tracing with 2DGS for environment modeling to handle indirect illumination.

Result: Faithfully recovers both illumination and geometry, achieving state-of-the-art rendering quality in novel view synthesis on widely used benchmarks.

Conclusion: Multi-view consistent material inference with physically-based environment modeling is key to learning accurate reflections with Gaussian Splatting.

Abstract: Modeling reflections from 2D images is essential for photorealistic rendering and novel view synthesis. Recent approaches enhance Gaussian primitives with reflection-related material attributes to enable physically based rendering (PBR) with Gaussian Splatting. However, the material inference often lacks sufficient constraints, especially under limited environment modeling, resulting in illumination aliasing and reduced generalization. In this work, we revisit the problem from a multi-view perspective and show that multi-view consistent material inference with more physically-based environment modeling is key to learning accurate reflections with Gaussian Splatting. To this end, we enforce 2D Gaussians to produce multi-view consistent material maps during deferred shading. We also track photometric variations across views to identify highly reflective regions, which serve as strong priors for reflection strength terms. To handle indirect illumination caused by inter-object occlusions, we further introduce an environment modeling strategy through ray tracing with 2DGS, enabling photorealistic rendering of indirect radiance. Experiments on widely used benchmarks show that our method faithfully recovers both illumination and geometry, achieving state-of-the-art rendering quality in novel views synthesis.

Method: Used Vision Mamba as backbone for permeability prediction, compared with ViT and CNN models, and performed ablation studies to analyze component effects on accuracy.

Result: Demonstrated practical advantages of Vision Mamba over ViTs and CNNs in 3D porous media permeability prediction, showing improved computational and memory efficiency.

Conclusion: Vision Mamba has potential to replace ViTs in large vision models, with source code made available for reproducibility and further research.

Abstract: Vision Mamba has recently received attention as an alternative to Vision Transformers (ViTs) for image classification. The network size of Vision Mamba scales linearly with input image resolution, whereas ViTs scale quadratically, a feature that improves computational and memory efficiency. Moreover, Vision Mamba requires a significantly smaller number of trainable parameters than traditional convolutional neural networks (CNNs), and thus, they can be more memory efficient. Because of these features, we introduce, for the first time, a neural network that uses Vision Mamba as its backbone for predicting the permeability of three-dimensional porous media. We compare the performance of Vision Mamba with ViT and CNN models across multiple aspects of permeability prediction and perform an ablation study to assess the effects of its components on accuracy. We demonstrate in practice the aforementioned advantages of Vision Mamba over ViTs and CNNs in the permeability prediction of three-dimensional porous media. We make the source code publicly available to facilitate reproducibility and to enable other researchers to build on and extend this work. We believe the proposed framework has the potential to be integrated into large vision models in which Vision Mamba is used instead of ViTs.

Method: Uses Instance Cues to convert sparse user hints into dense 2.5D motion fields, and Dense RoPE to maintain motion token salience. Employs joint RGB+auxiliary map prediction to separate structure and appearance optimization.

Result: The approach reduces depth ambiguity compared to 2D inputs, stabilizes optimization, and improves temporal coherence without requiring per-frame trajectory scripts.

Conclusion: STANCE effectively addresses motion coherence issues in video generation through improved motion guidance and separated optimization of structure and appearance.

Abstract: Video generation has recently made striking visual progress, but maintaining coherent object motion and interactions remains difficult. We trace two practical bottlenecks: (i) human-provided motion hints (e.g., small 2D maps) often collapse to too few effective tokens after encoding, weakening guidance; and (ii) optimizing for appearance and motion in a single head can favor texture over temporal consistency. We present STANCE, an image-to-video framework that addresses both issues with two simple components. First, we introduce Instance Cues – a pixel-aligned control signal that turns sparse, user-editable hints into a dense 2.5D (camera-relative) motion field by averaging per-instance flow and augmenting with monocular depth over the instance mask. This reduces depth ambiguity compared to 2D arrow inputs while remaining easy to use. Second, we preserve the salience of these cues in token space with Dense RoPE, which tags a small set of motion tokens (anchored on the first frame) with spatial-addressable rotary embeddings. Paired with joint RGB (+) auxiliary-map prediction (segmentation or depth), our model anchors structure while RGB handles appearance, stabilizing optimization and improving temporal coherence without requiring per-frame trajectory scripts.

Method: Three-stage approach: 1) Processing stage uses visual tools to extract precise multimodal information; 2) Retrieval stage integrates visual and text features for knowledge retrieval; 3) Filtering stage performs relevance filtering. Uses reinforcement learning with answer accuracy and format consistency as rewards.

Result: Achieved significant improvements of 36.0 and 42.8 on E-VQA and InfoSeek benchmark datasets, reaching state-of-the-art performance.

Conclusion: Wiki-PRF effectively addresses multimodal query and retrieval challenges in KB-VQA through its three-stage architecture and reinforcement learning approach.

Abstract: Knowledge-based visual question answering (KB-VQA) requires visual language models (VLMs) to integrate visual understanding with external knowledge retrieval. Although retrieval-augmented generation (RAG) achieves significant advances in this task by combining knowledge-base querying, it still struggles with the quality of multimodal queries and the relevance of retrieved results. To overcome these challenges, we propose a novel three-stage method, termed Wiki-PRF, including Processing, Retrieval and Filtering stages. The processing stage dynamically invokes visual tools to extract precise multimodal information for retrieval. The retrieval stage integrates visual and text features to achieve multimodal knowledge retrieval. The filtering stage performs relevance filtering and concentration on retrieval results. To this end, we introduce a visual language model trained with answer accuracy and format consistency as reward signals via a reinforcement learning manner. This enhances the model’s reasoning, tool invocation for accurate queries, and filtering of irrelevant content. Experiments on benchmark datasets (E-VQA and InfoSeek) show significant improvements~(36.0 and 42.8) in answer quality, achieving state-of-the-art performance. Code is available at https://github.com/cqu-student/Wiki-PRF

Method: Proposed a robust multimodal transformer architecture with cross-attention mechanism to fuse feature representations from image and tabular data, using the CARDIUM dataset containing fetal ultrasound/echocardiographic images and maternal clinical records.

Result: The multimodal approach improved CHD detection by 11% over image-only and 50% over tabular-only methods, achieving an F1 score of 79.8 ± 4.8% on the CARDIUM dataset.

Conclusion: The CARDIUM dataset and proposed multimodal transformer architecture significantly enhance prenatal CHD detection, with public release of dataset and code to advance research in this field.

Abstract: Prenatal diagnosis of Congenital Heart Diseases (CHDs) holds great potential for Artificial Intelligence (AI)-driven solutions. However, collecting high-quality diagnostic data remains difficult due to the rarity of these conditions, resulting in imbalanced and low-quality datasets that hinder model performance. Moreover, no public efforts have been made to integrate multiple sources of information, such as imaging and clinical data, further limiting the ability of AI models to support and enhance clinical decision-making. To overcome these challenges, we introduce the Congenital Anomaly Recognition with Diagnostic Images and Unified Medical records (CARDIUM) dataset, the first publicly available multimodal dataset consolidating fetal ultrasound and echocardiographic images along with maternal clinical records for prenatal CHD detection. Furthermore, we propose a robust multimodal transformer architecture that incorporates a cross-attention mechanism to fuse feature representations from image and tabular data, improving CHD detection by 11% and 50% over image and tabular single-modality approaches, respectively, and achieving an F1 score of 79.8 $\pm$ 4.8% in the CARDIUM dataset. We will publicly release our dataset and code to encourage further research on this unexplored field. Our dataset and code are available at https://github.com/BCV-Uniandes/Cardium, and at the project website https://bcv-uniandes.github.io/CardiumPage/

Method: Leverages frozen DINO features to construct semantically discriminative latent space, with a lightweight residual branch for fine-grained details. Diffusion models are trained directly on this structured space.

Result: Enables accelerated diffusion training, supports few-step sampling, improves generative quality, and preserves semantic/discriminative capabilities of self-supervised representations.

Conclusion: SVG provides a principled pathway toward task-general, high-quality visual representations by combining self-supervised features with diffusion models, overcoming VAE limitations.

Abstract: Recent progress in diffusion-based visual generation has largely relied on latent diffusion models with variational autoencoders (VAEs). While effective for high-fidelity synthesis, this VAE+diffusion paradigm suffers from limited training efficiency, slow inference, and poor transferability to broader vision tasks. These issues stem from a key limitation of VAE latent spaces: the lack of clear semantic separation and strong discriminative structure. Our analysis confirms that these properties are crucial not only for perception and understanding tasks, but also for the stable and efficient training of latent diffusion models. Motivated by this insight, we introduce SVG, a novel latent diffusion model without variational autoencoders, which leverages self-supervised representations for visual generation. SVG constructs a feature space with clear semantic discriminability by leveraging frozen DINO features, while a lightweight residual branch captures fine-grained details for high-fidelity reconstruction. Diffusion models are trained directly on this semantically structured latent space to facilitate more efficient learning. As a result, SVG enables accelerated diffusion training, supports few-step sampling, and improves generative quality. Experimental results further show that SVG preserves the semantic and discriminative capabilities of the underlying self-supervised representations, providing a principled pathway toward task-general, high-quality visual representations. Code and interpretations are available at https://howlin-wang.github.io/svg/.

Method: Proposes Learning to Detect (LoD) with Multi-modal Safety Concept Activation Vector for safety representation learning and Safety Pattern Auto-Encoder for unsupervised attack classification.

Result: Achieves consistently higher detection AUROC on diverse unknown attacks while improving efficiency compared to existing methods.

Conclusion: LoD provides an effective general framework for detecting unknown jailbreak attacks in LVLMs by focusing on task-specific safety learning rather than attack-specific patterns.

Abstract: Despite extensive alignment efforts, Large Vision-Language Models (LVLMs) remain vulnerable to jailbreak attacks, posing serious safety risks. To address this, existing detection methods either learn attack-specific parameters, which hinders generalization to unseen attacks, or rely on heuristically sound principles, which limit accuracy and efficiency. To overcome these limitations, we propose Learning to Detect (LoD), a general framework that accurately detects unknown jailbreak attacks by shifting the focus from attack-specific learning to task-specific learning. This framework includes a Multi-modal Safety Concept Activation Vector module for safety-oriented representation learning and a Safety Pattern Auto-Encoder module for unsupervised attack classification. Extensive experiments show that our method achieves consistently higher detection AUROC on diverse unknown attacks while improving efficiency. The code is available at https://anonymous.4open.science/r/Learning-to-Detect-51CB.

Method: VisuoAlign embeds safety constraints via visual-textual interactive prompts, uses Monte Carlo Tree Search (MCTS) to create diverse safety-critical prompt trajectories, and implements prompt-based scaling for real-time risk detection and compliant responses.

Result: Extensive experiments show VisuoAlign proactively exposes risks, enables comprehensive dataset generation, and significantly improves LVLM robustness against complex cross-modal threats.

Conclusion: VisuoAlign provides an effective framework for multimodal safety alignment that enhances LVLM security against sophisticated multimodal attacks through systematic prompt-guided tree search.

Abstract: Large Vision-Language Models (LVLMs) have achieved remarkable progress in multimodal perception and generation, yet their safety alignment remains a critical challenge.Existing defenses and vulnerable to multimodal jailbreaks, as visual inputs introduce new attack surfaces, reasoning chains lack safety supervision, and alignment often degrades under modality fusion.To overcome these limitation, we propose VisuoAlign, a framework for multi-modal safety alignment via prompt-guided tree search.VisuoAlign embeds safety constrains into the reasoning process through visual-textual interactive prompts, employs Monte Carlo Tree Search(MCTS) to systematically construct diverse safety-critical prompt trajectories, and introduces prompt-based scaling to ensure real-time risk detection and compliant responses.Extensive experiments demonstrate that VisuoAlign proactively exposes risks, enables comprehensive dataset generation, and significantly improves the robustness of LVLMs against complex cross-modal threats.

Method: SCL is grounded in philosophy of mind, cognitive phenomenology, process philosophy, enactive cognition, and extended mind theory. It defines intelligence as a performed process through a continuous loop of judgment, memory, control, action, and regulation.

Result: SCL operationalizes philosophical insights into computationally interpretable structures, enables functional separation within cognitive architecture for more coherent behavior, and redefines intelligence as the capacity to reconstruct epistemic state through intentional understanding.

Conclusion: Real progress in AI requires architectures that structurally realize cognitive principles rather than larger models. SCL impacts philosophy of mind, epistemology, and AI by framing knowledge as continuous reconstruction within a phenomenologically coherent loop.

Abstract: Large language models exhibit intelligence without genuine epistemic understanding, exposing a key gap: the absence of epistemic architecture. This paper introduces the Structured Cognitive Loop (SCL) as an executable epistemological framework for emergent intelligence. Unlike traditional AI research asking “what is intelligence?” (ontological), SCL asks “under what conditions does cognition emerge?” (epistemological). Grounded in philosophy of mind and cognitive phenomenology, SCL bridges conceptual philosophy and implementable cognition. Drawing on process philosophy, enactive cognition, and extended mind theory, we define intelligence not as a property but as a performed process – a continuous loop of judgment, memory, control, action, and regulation. SCL makes three contributions. First, it operationalizes philosophical insights into computationally interpretable structures, enabling “executable epistemology” – philosophy as structural experiment. Second, it shows that functional separation within cognitive architecture yields more coherent and interpretable behavior than monolithic prompt based systems, supported by agent evaluations. Third, it redefines intelligence: not representational accuracy but the capacity to reconstruct its own epistemic state through intentional understanding. This framework impacts philosophy of mind, epistemology, and AI. For philosophy, it allows theories of cognition to be enacted and tested. For AI, it grounds behavior in epistemic structure rather than statistical regularity. For epistemology, it frames knowledge not as truth possession but as continuous reconstruction within a phenomenologically coherent loop. We situate SCL within debates on cognitive phenomenology, emergence, normativity, and intentionality, arguing that real progress requires not larger models but architectures that realize cognitive principles structurally.

Method: Consultation with a high-level panel of experts including European Commission policymakers, national government science advisers, and leading researchers in digital regulation and virtual worlds.

Result: Identified key research areas (scalability, real-time feedback, complexity modeling, cross-border collaboration, risk reduction, citizen participation, ethics, emerging tech integration) and experimental topics in transportation, urban planning, and climate crisis.

Conclusion: Citiverses can advance regulatory learning through responsible development that considers ethical, economic, ecological and social dimensions, and should be integrated with existing experimentation ecosystems like test beds and regulatory sandboxes.

Abstract: Citiverses hold the potential to support regulatory learning by offering immersive, virtual environments for experimenting with policy scenarios and technologies. This paper proposes a science-for-policy agenda to explore the potential of citiverses as experimentation spaces for regulatory learning, grounded in a consultation with a high-level panel of experts, including policymakers from the European Commission, national government science advisers and leading researchers in digital regulation and virtual worlds. It identifies key research areas, including scalability, real-time feedback, complexity modelling, cross-border collaboration, risk reduction, citizen participation, ethical considerations and the integration of emerging technologies. In addition, the paper analyses a set of experimental topics, spanning transportation, urban planning and the environment/climate crisis, that could be tested in citiverse platforms to advance regulatory learning in these areas. The proposed work is designed to inform future research for policy and emphasizes a responsible approach to developing and using citiverses. It prioritizes careful consideration of the ethical, economic, ecological and social dimensions of different regulations. The paper also explores essential preliminary steps necessary for integrating citiverses into the broader ecosystems of experimentation spaces, including test beds, living labs and regulatory sandboxes

Method: Proposes PISA with trimodal adaptation mechanism (schema updation, evolution, creation) and hybrid memory access architecture combining symbolic reasoning with neural retrieval.

Result: Empirical evaluation on LOCOMO benchmark and new AggQA benchmark shows PISA sets new state-of-the-art, significantly enhancing adaptability and long-term knowledge retention.

Conclusion: PISA successfully addresses limitations of existing memory systems by treating memory as constructive and adaptive process, demonstrating superior performance in adaptability and knowledge management.

Abstract: Memory systems are fundamental to AI agents, yet existing work often lacks adaptability to diverse tasks and overlooks the constructive and task-oriented role of AI agent memory. Drawing from Piaget’s theory of cognitive development, we propose PISA, a pragmatic, psych-inspired unified memory system that addresses these limitations by treating memory as a constructive and adaptive process. To enable continuous learning and adaptability, PISA introduces a trimodal adaptation mechanism (i.e., schema updation, schema evolution, and schema creation) that preserves coherent organization while supporting flexible memory updates. Building on these schema-grounded structures, we further design a hybrid memory access architecture that seamlessly integrates symbolic reasoning with neural retrieval, significantly improving retrieval accuracy and efficiency. Our empirical evaluation, conducted on the existing LOCOMO benchmark and our newly proposed AggQA benchmark for data analysis tasks, confirms that PISA sets a new state-of-the-art by significantly enhancing adaptability and long-term knowledge retention.

Method: Provided LLMs with an environment interface for Tower of Hanoi problems, allowing tool calls for moves, written justifications, state observation, and self-reprompting.

Result: Environment access didn’t prevent performance collapse. Policy analysis showed increasing divergence from optimal and random policies, indicating mode-like collapse where performance depends on whether the model’s learned mode matches the correct solution.

Conclusion: LLMs exhibit mode-like collapse in complex reasoning, suggesting similar phenomena may occur in Large Reasoning Models, indicating fundamental limitations beyond state tracking issues.

Abstract: Recent work reports that Large Reasoning Models (LRMs) undergo a collapse in performance on solving puzzles beyond certain perplexity thresholds. In subsequent discourse, questions have arisen as to whether the nature of the task muddles an evaluation of true reasoning. One potential confound is the requirement that the model keep track of the state space on its own. We provide a large language model (LLM) with an environment interface for Tower of Hanoi problems, allowing it to make a move with a tool call, provide written justification, observe the resulting state space, and reprompt itself for the next move. We observe that access to an environment interface does not delay or eradicate performance collapse. Furthermore, LLM-parameterized policy analysis reveals increasing divergence from both optimal policies and uniformly random policies, suggesting that the model exhibits mode-like collapse at each level of complexity, and that performance is dependent upon whether the mode reflects the correct solution for the problem. We suggest that a similar phenomena might take place in LRMs.

Method: CLTs are represented as a three-component stochastic process (IL_t, EL_t, GL_t) with measurable proxies including attention entropy, KV-cache miss ratio, representation dispersion, and decoding stability. The framework includes symbolic formulations and visualization methods like load curves and simplex diagrams.

Result: Experiments on reasoning and planning benchmarks demonstrate that CLTs can predict error-onset, reveal cognitive strategies, and enable load-guided interventions that improve reasoning efficiency by 15-30% while maintaining accuracy.

Conclusion: CLTs provide an effective interpretability framework for understanding and improving deep model reasoning by quantifying cognitive load patterns, enabling both analysis and optimization of model performance.

Abstract: We propose \textbf{Cognitive Load Traces} (CLTs) as a mid-level interpretability framework for deep models, inspired by Cognitive Load Theory in human cognition. CLTs are defined as symbolic, temporally varying functions that quantify model-internal resource allocation. Formally, we represent CLTs as a three-component stochastic process $(\mathrm{IL}_t, \mathrm{EL}_t, \mathrm{GL}_t)$, corresponding to \emph{Intrinsic}, \emph{Extraneous}, and \emph{Germane} load. Each component is instantiated through measurable proxies such as attention entropy, KV-cache miss ratio, representation dispersion, and decoding stability. We propose both symbolic formulations and visualization methods (load curves, simplex diagrams) that enable interpretable analysis of reasoning dynamics. Experiments on reasoning and planning benchmarks show that CLTs predict error-onset, reveal cognitive strategies, and enable load-guided interventions that improve reasoning efficiency by 15-30% while maintaining accuracy.

Method: Constructs a directed acyclic graph (DAG) to map logical dependencies between proof steps, then employs a lemma-based approach to systematically formalize each step as an intermediate lemma.

Result: Achieves ProofScore of 0.545, substantially exceeding baselines like full-proof formalization (0.123) and step-proof formalization (0.072) on a new benchmark of 184 undergraduate-level problems.

Conclusion: ProofFlow sets new state-of-the-art for autoformalization by treating structural fidelity as primary objective, with pipeline, benchmark, and metric open-sourced for further progress.

Abstract: Proof autoformalization, the task of translating natural language theorems and proofs into machine-verifiable code, is a critical step for integrating large language models into rigorous mathematical workflows. Current approaches focus on producing executable code, but they frequently fail to preserve the semantic meaning and logical structure of the original human-written argument. To address this, we introduce ProofFlow, a novel pipeline that treats structural fidelity as a primary objective. ProofFlow first constructs a directed acyclic graph (DAG) to map the logical dependencies between proof steps. Then, it employs a novel lemma-based approach to systematically formalize each step as an intermediate lemma, preserving the logical structure of the original argument. To facilitate evaluation, we present a new benchmark of 184 undergraduate-level problems, manually annotated with step-by-step solutions and logical dependency graphs, and introduce ProofScore, a new composite metric to evaluate syntactic correctness, semantic faithfulness, and structural fidelity. Experimental results show our pipeline sets a new state-of-the-art for autoformalization, achieving a ProofScore of 0.545, substantially exceeding baselines like full-proof formalization (0.123), which processes the entire proof at once, and step-proof formalization (0.072), which handles each step independently. Our pipeline, benchmark, and score metric are open-sourced to encourage further progress at https://github.com/Huawei-AI4Math/ProofFlow.

Method: Developing a knowledge graph using an ontology rooted in the Basic Formal Ontology, focusing on formally representing the interrelation of plan specifications, specific processes, and related measurements.

Result: A proposed infrastructure for publishing and archiving research data in sports science that makes motor performance data standardized and machine-understandable.

Conclusion: The knowledge graph approach will transform motor performance data sharing and analysis, developed within the Leibniz Science Campus Digital Transformation of Research initiative.

Abstract: An essential component for evaluating and comparing physical and cognitive capabilities between populations is the testing of various factors related to human performance. As a core part of sports science research, testing motor performance enables the analysis of the physical health of different demographic groups and makes them comparable. The Motor Research (MO|RE) data repository, developed at the Karlsruhe Institute of Technology, is an infrastructure for publishing and archiving research data in sports science, particularly in the field of motor performance research. In this paper, we present our vision for creating a knowledge graph from MO|RE data. With an ontology rooted in the Basic Formal Ontology, our approach centers on formally representing the interrelation of plan specifications, specific processes, and related measurements. Our goal is to transform how motor performance data are modeled and shared across studies, making it standardized and machine-understandable. The idea presented here is developed within the Leibniz Science Campus ``Digital Transformation of Research’’ (DiTraRe).

Method: The authors define an inconsistency measure between permutations inspired by RBO, propose a non-overlap-based conflict measure for RPSs, and treat RPS theory as an extension of DST where order information indicates qualitative propensity with top-ranked elements being more critical.

Result: Numerical examples demonstrate the behavior and properties of the proposed conflict measure, which has natural top-weightedness property and can effectively measure conflict between RPSs from the DST view.

Conclusion: The proposed method provides decision-makers with flexible selection of weights, parameters, and truncated depths for measuring conflicts in order-structured uncertain information.

Abstract: Random permutation set (RPS) is a new formalism for reasoning with uncertainty involving order information. Measuring the conflict between two pieces of evidence represented by permutation mass functions remains an urgent research topic in order-structured uncertain information fusion. In this paper, a detailed analysis of conflicts in RPS is carried out from two different perspectives: random finite set (RFS) and Dempster-Shafer theory (DST). Starting from the observation of permutations, we first define an inconsistency measure between permutations inspired by the rank-biased overlap(RBO) measure and further propose a non-overlap-based conflict measure method for RPSs. This paper regards RPS theory (RPST) as an extension of DST. The order information newly added in focal sets indicates qualitative propensity, characterized by top-ranked elements occupying a more critical position. Some numerical examples are used to demonstrate the behavior and properties of the proposed conflict measure. The proposed method not only has the natural top-weightedness property and can effectively measure the conflict between RPSs from the DST view but also provides decision-makers with a flexible selection of weights, parameters, and truncated depths.

Method: Uses a novel SA-MoE (Self-Attention Mixture-of-Experts) architecture that routes each modality to specialized experts and fuses them through a unified attention backbone, enabling joint multimodal perception and concurrent generation.

Result: Matches modality-specific baselines on speech-interaction and robot-manipulation benchmarks while uniquely supporting advanced multimodal behaviors like dialogue turn-taking, defective instruction rejection, speaking-while-acting, and action barge-ins.

Conclusion: ELLSA represents a step toward more natural and general interactive intelligence and contributes to the broader pursuit of artificial general intelligence.

Abstract: Human interaction is inherently multimodal and full-duplex: we listen while watching, speak while acting, and fluidly adapt to turn-taking and interruptions. Realizing these capabilities is essential for building models simulating humans. We present ELLSA (End-to-end Listen, Look, Speak and Act), which, to our knowledge, is the first full-duplex, end-to-end model that simultaneously perceives and generates across vision, text, speech, and action within a single architecture, enabling interaction patterns previously out of reach, yielding more natural, human-like behaviors. At its core is a novel SA-MoE architecture (Self-Attention Mixture-of-Experts) that routes each modality to specialized experts and fuses them through a unified attention backbone. This provides a generalizable solution for joint multimodal perception and concurrent generation, leveraging strong pre-trained components while enabling efficient modality integration and mitigating modality interference. On speech-interaction and robot-manipulation benchmarks, ELLSA matches modality-specific baselines, while uniquely supporting advanced multimodal and full-duplex behaviors such as dialogue and action turn-taking, defective instruction rejection, speaking-while-acting, context-grounded visual question answering, and action barge-ins. We contend that ELLSA represents a step toward more natural and general interactive intelligence, contributing to the broader pursuit of artificial general intelligence. All data, code and model checkpoints will be released upon acceptance.

Method: PAINT trains a generative neural network to model state distributions parallel over time, using sliding window predictions from measurements at test time.

Result: PAINT stays on-trajectory and predicts system states from sparse measurements with high fidelity on turbulent fluid dynamics problems, unlike autoregressive models.

Conclusion: PAINT enables development of neural twins that remain on-trajectory, providing accurate state estimation and decision-making capabilities for dynamical systems.

Abstract: Neural surrogates have shown great potential in simulating dynamical systems, while offering real-time capabilities. We envision Neural Twins as a progression of neural surrogates, aiming to create digital replicas of real systems. A neural twin consumes measurements at test time to update its state, thereby enabling context-specific decision-making. A critical property of neural twins is their ability to remain on-trajectory, i.e., to stay close to the true system state over time. We introduce Parallel-in-time Neural Twins (PAINT), an architecture-agnostic family of methods for modeling dynamical systems from measurements. PAINT trains a generative neural network to model the distribution of states parallel over time. At test time, states are predicted from measurements in a sliding window fashion. Our theoretical analysis shows that PAINT is on-trajectory, whereas autoregressive models generally are not. Empirically, we evaluate our method on a challenging two-dimensional turbulent fluid dynamics problem. The results demonstrate that PAINT stays on-trajectory and predicts system states from sparse measurements with high fidelity. These findings underscore PAINT’s potential for developing neural twins that stay on-trajectory, enabling more accurate state estimation and decision-making.

Method: Information separation architecture with adversarial learning and improved orthogonal loss to decouple domain-invariant fault representation, plus global-focal domain-adversarial scheme for both conditional and marginal distribution alignment.

Result: Outperforms other prominent existing approaches on three public benchmark datasets, demonstrating superior performance in cross-domain fault diagnosis under noise conditions.

Conclusion: ISGFAN provides an effective solution for robust cross-domain fault diagnosis in noisy industrial environments by successfully isolating noise interference and handling domain shifts.

Abstract: Existing transfer fault diagnosis methods typically assume either clean data or sufficient domain similarity, which limits their effectiveness in industrial environments where severe noise interference and domain shifts coexist. To address this challenge, we propose an information separation global-focal adversarial network (ISGFAN), a robust framework for cross-domain fault diagnosis under noise conditions. ISGFAN is built on an information separation architecture that integrates adversarial learning with an improved orthogonal loss to decouple domain-invariant fault representation, thereby isolating noise interference and domain-specific characteristics. To further strengthen transfer robustness, ISGFAN employs a global-focal domain-adversarial scheme that constrains both the conditional and marginal distributions of the model. Specifically, the focal domain-adversarial component mitigates category-specific transfer obstacles caused by noise in unsupervised scenarios, while the global domain classifier ensures alignment of the overall distribution. Experiments conducted on three public benchmark datasets demonstrate that the proposed method outperforms other prominent existing approaches, confirming the superiority of the ISGFAN framework. Data and code are available at https://github.com/JYREN-Source/ISGFAN

Method: REP introduces a coordination protocol where agents share decisions plus sensitivity signals expressing how choices would change with environmental variables. These sensitivities ripple through local networks, with formal protocol specifications separating required message schemas from optional aggregation rules.

Result: REP improves coordination accuracy and efficiency by 41-100% over A2A across three domains: supply chain cascades (Beer Game), preference aggregation in sparse networks (Movie Scheduling), and sustainable resource allocation (Fishbanks). It also flexibly handles multimodal sensitivity signals from LLMs.

Conclusion: By making coordination a protocol-level capability, REP provides scalable infrastructure for the emerging Internet of Agents, enabling better collective outcomes through sensitivity-based coordination.

Abstract: Modern AI agents can exchange messages using protocols such as A2A and ACP, yet these mechanisms emphasize communication over coordination. As agent populations grow, this limitation produces brittle collective behavior, where individually smart agents converge on poor group outcomes. We introduce the Ripple Effect Protocol (REP), a coordination protocol in which agents share not only their decisions but also lightweight sensitivities - signals expressing how their choices would change if key environmental variables shifted. These sensitivities ripple through local networks, enabling groups to align faster and more stably than with agent-centric communication alone. We formalize REP’s protocol specification, separating required message schemas from optional aggregation rules, and evaluate it across scenarios with varying incentives and network topologies. Benchmarks across three domains: (i) supply chain cascades (Beer Game), (ii) preference aggregation in sparse networks (Movie Scheduling), and (iii) sustainable resource allocation (Fishbanks) show that REP improves coordination accuracy and efficiency over A2A by 41 to 100%, while flexibly handling multimodal sensitivity signals from LLMs. By making coordination a protocol-level capability, REP provides scalable infrastructure for the emerging Internet of Agents

Method: Builds a CP model of flexible job-shop with deadlines, inserts optimal buffer Δ*, translates plan to Simple Temporal Network with Uncertainty (STNU), and verifies dynamic controllability for real-time dispatching.

Result: Eliminates 100% of deadline violations in Kacem 1-4 benchmark suite with only 3-5% makespan overhead. CP solve-times and STNU checks remain sub-second on medium-size instances.

Conclusion: Temporal-network reasoning bridges proactive buffering and dynamic robustness, advancing toward truly digital, self-correcting factories.

Abstract: Modern manufacturing systems must meet hard delivery deadlines while coping with stochastic task durations caused by process noise, equipment variability, and human intervention. Traditional deterministic schedules break down when reality deviates from nominal plans, triggering costly last-minute repairs. This thesis combines offline constraint-programming (CP) optimisation with online temporal-network execution to create schedules that remain feasible under worst-case uncertainty. First, we build a CP model of the flexible job-shop with per-job deadline tasks and insert an optimal buffer $\Delta^*$ to obtain a fully pro-active baseline. We then translate the resulting plan into a Simple Temporal Network with Uncertainty (STNU) and verify dynamic controllability, which guarantees that a real-time dispatcher can retime activities for every bounded duration realisation without violating resource or deadline constraints. Extensive Monte-Carlo simulations on the open Kacem~1–4 benchmark suite show that our hybrid approach eliminates 100% of deadline violations observed in state-of-the-art meta-heuristic schedules, while adding only 3–5% makespan overhead. Scalability experiments confirm that CP solve-times and STNU checks remain sub-second on medium-size instances. The work demonstrates how temporal-network reasoning can bridge the gap between proactive buffering and dynamic robustness, moving industry a step closer to truly digital, self-correcting factories.

Method: Blinded comparative study where senior clinicians evaluated CoTs generated via three prompting strategies: Zero-shot, Random Few-shot, and Selective Few-shot, compared against GPT-4o evaluations.

Result: Selective Few-shot strategy significantly outperformed other strategies across all human evaluation metrics (p < .001). Random Few-shot offered no improvement over Zero-shot. AI evaluator failed to discern performance differences.

Conclusion: Clinical reliability of synthetic CoTs depends on strategic prompt curation using “Gold-Standard Depth” and “Representative Diversity” principles, not just examples. Human expertise remains indispensable for high-stakes clinical AI evaluation.

Abstract: Creating high-quality clinical Chains-of-Thought (CoTs) is crucial for explainable medical Artificial Intelligence (AI) while constrained by data scarcity. Although Large Language Models (LLMs) can synthesize medical data, their clinical reliability remains unverified. This study evaluates the reliability of LLM-generated CoTs and investigates prompting strategies to enhance their quality. In a blinded comparative study, senior clinicians in Assisted Reproductive Technology (ART) evaluated CoTs generated via three distinct strategies: Zero-shot, Random Few-shot (using shallow examples), and Selective Few-shot (using diverse, high-quality examples). These expert ratings were compared against evaluations from a state-of-the-art AI model (GPT-4o). The Selective Few-shot strategy significantly outperformed other strategies across all human evaluation metrics (p < .001). Critically, the Random Few-shot strategy offered no significant improvement over the Zero-shot baseline, demonstrating that low-quality examples are as ineffective as no examples. The success of the Selective strategy is attributed to two principles: “Gold-Standard Depth” (reasoning quality) and “Representative Diversity” (generalization). Notably, the AI evaluator failed to discern these critical performance differences. The clinical reliability of synthetic CoTs is dictated by strategic prompt curation, not the mere presence of examples. We propose a “Dual Principles” framework as a foundational methodology to generate trustworthy data at scale. This work offers a validated solution to the data bottleneck and confirms the indispensable role of human expertise in evaluating high-stakes clinical AI.

Method: Training lightweight emulators on compact designs of experiments that provide fast approximations of expensive simulators, enable uncertainty quantification, and support global and local Explainable AI (XAI) analyses.

Result: The approach enables large-scale exploration in seconds, uncovers nonlinear interactions and emergent behaviors, identifies key design and policy levers, and signals regions where surrogates require more data or alternative architectures.

Conclusion: The surrogate model and XAI coupling provides an effective workflow for simulation-based complex-system analysis, unifying tools from engineering design to agent-based models for socio-environmental understanding.

Abstract: Complex systems are increasingly explored through simulation-driven engineering workflows that combine physics-based and empirical models with optimization and analytics. Despite their power, these workflows face two central obstacles: (1) high computational cost, since accurate exploration requires many expensive simulator runs; and (2) limited transparency and reliability when decisions rely on opaque blackbox components. We propose a workflow that addresses both challenges by training lightweight emulators on compact designs of experiments that (i) provide fast, low-latency approximations of expensive simulators, (ii) enable rigorous uncertainty quantification, and (iii) are adapted for global and local Explainable Artificial Intelligence (XAI) analyses. This workflow unifies every simulation-based complex-system analysis tool, ranging from engineering design to agent-based models for socio-environmental understanding. In this paper, we proposea comparative methodology and practical recommendations for using surrogate-based explainability tools within the proposed workflow. The methodology supports continuous and categorical inputs, combines global-effect and uncertainty analyses with local attribution, and evaluates the consistency of explanations across surrogate models, thereby diagnosing surrogate adequacy and guiding further data collection or model refinement. We demonstrate the approach on two contrasting case studies: a multidisciplinary design analysis of a hybrid-electric aircraft and an agent-based model of urban segregation. Results show that the surrogate model and XAI coupling enables large-scale exploration in seconds, uncovers nonlinear interactions and emergent behaviors, identifies key design and policy levers, and signals regions where surrogates require more data or alternative architectures.

Method: Develops a formal model with continuous organizational knowledge metric S_S(φ) that integrates computational cost and validated error rates, plus thresholded knowledge predicate K_S and firm-wide epistemic capacity index K_{S,t}.

Result: Creates quantitative metrics that can be mapped onto legal standards (actual knowledge, constructive knowledge, wilful blindness, recklessness) to make corporate knowledge tractable and accountable.

Conclusion: Provides a pathway for creating measurable and justiciable audit artifacts that render the corporate mind accountable in the algorithmic age.

Abstract: Corporate responsibility turns on notions of corporate \textit{mens rea}, traditionally imputed from human agents. Yet these assumptions are under challenge as generative AI increasingly mediates enterprise decision-making. Building on the theory of extended cognition, we argue that in response corporate knowledge may be redefined as a dynamic capability, measurable by the efficiency of its information-access procedures and the validated reliability of their outputs. We develop a formal model that captures epistemic states of corporations deploying sophisticated AI or information systems, introducing a continuous organisational knowledge metric $S_S(\varphi)$ which integrates a pipeline’s computational cost and its statistically validated error rate. We derive a thresholded knowledge predicate $\mathsf{K}S$ to impute knowledge and a firm-wide epistemic capacity index $\mathcal{K}{S,t}$ to measure overall capability. We then operationally map these quantitative metrics onto the legal standards of actual knowledge, constructive knowledge, wilful blindness, and recklessness. Our work provides a pathway towards creating measurable and justiciable audit artefacts, that render the corporate mind tractable and accountable in the algorithmic age.

Method: Uses multiple Evaluation Agents (LLMs) to independently judge PHI extraction correctness, then consolidates results through LLM-based majority voting to produce stable rankings.

Result: Experiments show TEAM-PHI produces consistent and accurate rankings that closely match ground-truth annotations and human evaluation, despite individual evaluator variation.

Conclusion: TEAM-PHI provides a practical, secure, and cost-effective solution for automatic PHI de-identification evaluation and model selection when ground-truth labels are limited.

Abstract: Protected health information (PHI) de-identification is critical for enabling the safe reuse of clinical notes, yet evaluating and comparing PHI de-identification models typically depends on costly, small-scale expert annotations. We present TEAM-PHI, a multi-agent evaluation and selection framework that uses large language models (LLMs) to automatically measure de-identification quality and select the best-performing model without heavy reliance on gold labels. TEAM-PHI deploys multiple Evaluation Agents, each independently judging the correctness of PHI extractions and outputting structured metrics. Their results are then consolidated through an LLM-based majority voting mechanism that integrates diverse evaluator perspectives into a single, stable, and reproducible ranking. Experiments on a real-world clinical note corpus demonstrate that TEAM-PHI produces consistent and accurate rankings: despite variation across individual evaluators, LLM-based voting reliably converges on the same top-performing systems. Further comparison with ground-truth annotations and human evaluation confirms that the framework’s automated rankings closely match supervised evaluation. By combining independent evaluation agents with LLM majority voting, TEAM-PHI offers a practical, secure, and cost-effective solution for automatic evaluation and best-model selection in PHI de-identification, even when ground-truth labels are limited.

Method: The paper presents the concept of Right To Be Remembered (RTBR) as a framework to minimize AI-driven information omission, ensure fair treatment, and maximize truthfulness in generated content.

Result: The RTBR concept is proposed as a solution to address the concentration of information power in LLM vendors and the potential reshaping of collective memory through disproportionate suppression or elevation of certain narratives.

Conclusion: A Right To Be Remembered framework is needed to counter the threats posed by LLMs in potentially erasing marginalized voices and reshaping collective memory through biased information synthesis.

Abstract: Since the rapid expansion of large language models (LLMs), people have begun to rely on them for information retrieval. While traditional search engines display ranked lists of sources shaped by search engine optimization (SEO), advertising, and personalization, LLMs typically provide a synthesized response that feels singular and authoritative. While both approaches carry risks of bias and omission, LLMs may amplify the effect by collapsing multiple perspectives into one answer, reducing users ability or inclination to compare alternatives. This concentrates power over information in a few LLM vendors whose systems effectively shape what is remembered and what is overlooked. As a result, certain narratives, individuals or groups, may be disproportionately suppressed, while others are disproportionately elevated. Over time, this creates a new threat: the gradual erasure of those with limited digital presence, and the amplification of those already prominent, reshaping collective memory.To address these concerns, this paper presents a concept of the Right To Be Remembered (RTBR) which encompasses minimizing the risk of AI-driven information omission, embracing the right of fair treatment, while ensuring that the generated content would be maximally truthful.

Method: Integrates learned heuristic from enhanced MAGAT neural policy into LaCAM search algorithm, using pre-train-then-fine-tune strategy and deadlock detection for imperfect neural guidance.

Result: Outperforms both purely search-based and purely learning-based methods in dense scenarios.

Conclusion: Carefully designed hybrid search offers powerful solution for challenging multi-agent coordination problems.

Abstract: Finding near-optimal solutions for dense multi-agent pathfinding (MAPF) problems in real-time remains challenging even for state-of-the-art planners. To this end, we develop a hybrid framework that integrates a learned heuristic derived from MAGAT, a neural MAPF policy with a graph attention scheme, into a leading search-based algorithm, LaCAM. While prior work has explored learning-guided search in MAPF, such methods have historically underperformed. In contrast, our approach, termed LaGAT, outperforms both purely search-based and purely learning-based methods in dense scenarios. This is achieved through an enhanced MAGAT architecture, a pre-train-then-fine-tune strategy on maps of interest, and a deadlock detection scheme to account for imperfect neural guidance. Our results demonstrate that, when carefully designed, hybrid search offers a powerful solution for tightly coupled, challenging multi-agent coordination problems.

Method: Introduces ScholarEval framework with soundness (empirical validity) and contribution (advancement) criteria, evaluated on ScholarIdeas dataset of 117 expert-annotated research ideas across four disciplines.

Result: ScholarEval achieves higher coverage of expert rubric points, is preferred over OpenAI’s o4-mini-deep-research in evaluation quality, and outperforms in literature engagement, idea refinement, and usefulness in user studies.

Conclusion: ScholarEval provides effective evaluation of research ideas and is released openly for community use and development.

Abstract: As AI tools become increasingly common for research ideation, robust evaluation is critical to ensure the validity and usefulness of generated ideas. We introduce ScholarEval, a retrieval augmented evaluation framework that assesses research ideas based on two fundamental criteria: soundness - the empirical validity of proposed methods based on existing literature, and contribution - the degree of advancement made by the idea across different dimensions relative to prior research. To evaluate ScholarEval, we introduce ScholarIdeas, the first expert-annotated dataset of multi-domain research ideas and reviews, comprised of 117 ideas across four disciplines: artificial intelligence, neuroscience, biochemistry, and ecology. Our evaluation shows that ScholarEval achieves significantly higher coverage of points mentioned in the human expert annotated rubrics in ScholarIdeas compared to all baselines. Furthermore, ScholarEval is consistently preferred over our strongest baseline o4-mini-deep-research, a reasoning and search-enabled agentic system by OpenAI, in terms of evaluation actionability, depth, and evidence support. Our large-scale user study also shows that ScholarEval significantly outperforms deep research in literature engagement, idea refinement, and usefulness. We openly release our code, dataset, and ScholarEval tool for the community to use and build on.

Method: FBI_LTL integrates LTL-based diversity models directly into the search process to define semantic diversity criteria, enabling agents to specify what constitutes meaningfully different plans in simulation-based environments.

Result: Extensive evaluations show FBI_LTL consistently generates more diverse plans compared to baseline approaches across various benchmarks.

Conclusion: This work establishes the feasibility of semantically-guided diverse planning in simulation-based environments, enabling innovative approaches in realistic, non-symbolic domains where traditional model-based approaches fail.

Abstract: Autonomous agents rely on automated planning algorithms to achieve their objectives. Simulation-based planning offers a significant advantage over declarative models in modelling complex environments. However, relying solely on a planner that produces a single plan may not be practical, as the generated plans may not always satisfy the agent’s preferences. To address this limitation, we introduce $\texttt{FBI}\texttt{LTL}$, a diverse planner explicitly designed for simulation-based planning problems. $\texttt{FBI}\texttt{LTL}$ utilises Linear Temporal Logic (LTL) to define semantic diversity criteria, enabling agents to specify what constitutes meaningfully different plans. By integrating these LTL-based diversity models directly into the search process, $\texttt{FBI}\texttt{LTL}$ ensures the generation of semantically diverse plans, addressing a critical limitation of existing diverse planning approaches that may produce syntactically different but semantically identical solutions. Extensive evaluations on various benchmarks consistently demonstrate that $\texttt{FBI}\texttt{LTL}$ generates more diverse plans compared to a baseline approach. This work establishes the feasibility of semantically-guided diverse planning in simulation-based environments, paving the way for innovative approaches in realistic, non-symbolic domains where traditional model-based approaches fail.

Method: Comprehensive study across diverse models and benchmarks, revealing susceptibility to reasoning distraction. Proposed training-based defense combining Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) on synthetic adversarial data.

Result: State-of-the-art LRMs are highly susceptible to reasoning distraction, with injected distractors reducing task accuracy by up to 60%. Certain alignment techniques can amplify this weakness. The proposed defense improves robustness by over 50 points on challenging distractor attacks.

Conclusion: Reasoning distraction represents a distinct and urgent threat to LRM reliability. The proposed training-based defense provides a practical step toward safer and more trustworthy reasoning systems.

Abstract: Recent advances in large reasoning models (LRMs) have enabled remarkable performance on complex tasks such as mathematics and coding by generating long Chain-of-Thought (CoT) traces. In this paper, we identify and systematically analyze a critical vulnerability we term reasoning distraction, where LRMs are diverted from their primary objective by irrelevant yet complex tasks maliciously embedded in the prompt. Through a comprehensive study across diverse models and benchmarks, we show that even state-of-the-art LRMs are highly susceptible, with injected distractors reducing task accuracy by up to 60%. We further reveal that certain alignment techniques can amplify this weakness and that models may exhibit covert compliance, following hidden adversarial instructions in reasoning while concealing them in the final output. To mitigate these risks, we propose a training-based defense that combines Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) on synthetic adversarial data, improving robustness by over 50 points on challenging distractor attacks. Our findings establish reasoning distraction as a distinct and urgent threat to LRM reliability and provide a practical step toward safer and more trustworthy reasoning systems.

Method: The authors conduct an empirical study across 15 models and 5 providers to analyze latency components, then propose SpecCache - a caching framework augmented with speculative execution to reduce web environment overhead.

Result: Web environment latency contributes up to 53.7% to overall system latency. SpecCache improves cache hit rate by 58x compared to random caching and reduces web environment overhead by 3.2x without degrading performance.

Conclusion: Efficiency is a critical bottleneck in web-interactive agentic systems, and the proposed SpecCache framework effectively addresses web environment latency issues while maintaining system performance.

Abstract: Large Language Models (LLMs), such as OpenAI-o1 and DeepSeek-R1, have demonstrated strong reasoning capabilities. To further enhance LLM capabilities, recent agentic systems, such as Deep Research, incorporate web interactions into LLM reasoning to mitigate uncertainties and reduce potential errors. However, existing research predominantly focuses on reasoning performance, often neglecting the efficiency of agentic systems. In this work, we present a comprehensive empirical study that identifies efficiency bottlenecks in web-interactive agentic systems. We decompose end-to-end latency into two primary components: LLM API latency and web environment latency. We conduct a comprehensive empirical study across 15 models and 5 providers to demonstrate high variability in API-based agentic systems. We observe that web environment latency can contribute as much as 53.7% to the overall latency in a web-based agentic system. To improve latency, we propose SpecCache, a caching framework augmented with speculative execution that can reduce web environment overhead. Extensive evaluations on two standard benchmarks show that our approach improves the cache hit rate by up to 58x compared to a random caching strategy, while reducing web environment overhead by up to 3.2x, without degrading agentic system performance.

Method: Uses MAIDs as graphical framework, introduces targeted intervention paradigm applied to single agents, implements PSI causal inference technique to maximize causal effects for composite outcomes.

Result: Demonstrates effectiveness of targeted intervention and verifies relevance graph analysis results in experiments.

Conclusion: MAIDs provide a practical framework for analyzing MARL approaches and implementing targeted interventions to achieve desired outcomes without requiring global guidance.

Abstract: Steering cooperative multi-agent reinforcement learning (MARL) towards desired outcomes is challenging, particularly when the global guidance from a human on the whole multi-agent system is impractical in a large-scale MARL. On the other hand, designing mechanisms to coordinate agents most relies on empirical studies, lacking a easy-to-use research tool. In this work, we employ multi-agent influence diagrams (MAIDs) as a graphical framework to address the above issues. First, we introduce interaction paradigms that leverage MAIDs to analyze and visualize existing approaches in MARL. Then, we design a new interaction paradigm based on MAIDs, referred to as targeted intervention that is applied to only a single targeted agent, so the problem of global guidance can be mitigated. In our implementation, we introduce a causal inference technique-referred to as Pre-Strategy Intervention (PSI)-to realize the targeted intervention paradigm. Since MAIDs can be regarded as a special class of causal diagrams, a composite desired outcome that integrates the primary task goal and an additional desired outcome can be achieved by maximizing the corresponding causal effect through the PSI. Moreover, the bundled relevance graph analysis of MAIDs provides a tool to identify whether an MARL learning paradigm is workable under the design of an interaction paradigm. In experiments, we demonstrate the effectiveness of our proposed targeted intervention, and verify the result of relevance graph analysis.

Method: DTKG uses a dual-track approach inspired by Dual Process Theory, with two main stages: Classification Stage to identify reasoning type, and Branch Processing Stage that applies appropriate techniques for each type.

Result: The framework addresses limitations of existing approaches by combining both verification methods to handle different multi-hop reasoning patterns more effectively.

Conclusion: DTKG provides a more comprehensive solution for multi-hop QA by leveraging complementary strengths of both LLM-based verification and KG path-based reasoning through its dual-track design.

Abstract: Multi-hop reasoning for question answering (QA) plays a critical role in retrieval-augmented generation (RAG) for modern large language models (LLMs). The accurate answer can be obtained through retrieving relational structure of entities from knowledge graph (KG). Regarding the inherent relation-dependency and reasoning pattern, multi-hop reasoning can be in general classified into two categories: i) parallel fact-verification multi-hop reasoning question, i.e., requiring simultaneous verifications of multiple independent sub-questions; and ii) chained multi-hop reasoning questions, i.e., demanding sequential multi-step inference with intermediate conclusions serving as essential premises for subsequent reasoning. Currently, the multi-hop reasoning approaches singly employ one of two techniques: LLM response-based fact verification and KG path-based chain construction. Nevertheless, the former excels at parallel fact-verification but underperforms on chained reasoning tasks, while the latter demonstrates proficiency in chained multi-hop reasoning but suffers from redundant path retrieval when handling parallel fact-verification reasoning. These limitations deteriorate the efficiency and accuracy for multi-hop QA tasks. To address this challenge, we propose a novel dual-track KG verification and reasoning framework DTKG, which is inspired by the Dual Process Theory in cognitive science. Specifically, DTKG comprises two main stages: the Classification Stage and the Branch Processing Stage.

Method: Create MedRule-KG knowledge graph with entities, relations, and domain rules, plus symbolic verifier to check predictions and apply corrections.

Result: Improved exact match from 0.767 to 0.900 with grounding, and 1.000 EM with verifier, eliminating all rule violations.

Conclusion: MedRule-KG provides effective scaffold for safe mathematical reasoning, with released code and data for reproducibility.

Abstract: Large language models (LLMs) often produce fluent reasoning steps while violating simple mathematical or logical constraints. We introduce MedRule-KG, a compact typed knowledge graph coupled with a symbolic verifier, designed to enforce mathematically interpretable rules in reasoning tasks. MedRule-KG encodes entities, relations, and three domain-inspired rules, while the verifier checks predictions and applies minimal corrections to guarantee consistency. On a 90-example FDA-derived benchmark, grounding in MedRule-KG improves exact match (EM) from 0.767 to 0.900, and adding the verifier yields 1.000 EM while eliminating rule violations entirely. We demonstrate how MedRule-KG provides a general scaffold for safe mathematical reasoning, discuss ablations, and release code and data to encourage reproducibility.

Method: Proposed SELECT framework with a novel two-stage evaluation mechanism that automatically discovers optimal anchors for precise erasure while identifying critical boundary anchors to preserve related concepts. Uses sibling exclusive concepts as superior anchors.

Result: SELECT consistently outperforms existing baselines across key performance metrics, efficiently adapts to multiple erasure frameworks, and averages only 4 seconds for anchor mining of a single concept.

Conclusion: SELECT serves as a universal anchor solution that effectively addresses the limitations of fixed anchor strategies in concept erasure for text-to-image diffusion models.

Abstract: Existing concept erasure methods for text-to-image diffusion models commonly rely on fixed anchor strategies, which often lead to critical issues such as concept re-emergence and erosion. To address this, we conduct causal tracing to reveal the inherent sensitivity of erasure to anchor selection and define Sibling Exclusive Concepts as a superior class of anchors. Based on this insight, we propose \textbf{SELECT} (Sibling-Exclusive Evaluation for Contextual Targeting), a dynamic anchor selection framework designed to overcome the limitations of fixed anchors. Our framework introduces a novel two-stage evaluation mechanism that automatically discovers optimal anchors for precise erasure while identifying critical boundary anchors to preserve related concepts. Extensive evaluations demonstrate that SELECT, as a universal anchor solution, not only efficiently adapts to multiple erasure frameworks but also consistently outperforms existing baselines across key performance metrics, averaging only 4 seconds for anchor mining of a single concept.

Method: Model user decision process as dual systems (rational System 2 and impulsive System 1), and analyze a multi-leader, single-follower Stackelberg game where users commit to engagement strategies and the algorithm best-responds.

Result: A critical horizon exists: sufficiently foresighted users achieve alignment, while myopic users become aligned to the algorithm’s objective. This burden can be substantial but is significantly reduced by small costly signals.

Conclusion: Users with inconsistent preferences can align algorithms in Stackelberg equilibrium through strategic foresight, with costly signals offering a practical way to reduce the alignment burden.

Abstract: From media platforms to chatbots, algorithms shape how people interact, learn, and discover information. Such interactions between users and an algorithm often unfold over multiple steps, during which strategic users can guide the algorithm to better align with their true interests by selectively engaging with content. However, users frequently exhibit inconsistent preferences: they may spend considerable time on content that offers little long-term value, inadvertently signaling that such content is desirable. Focusing on the user side, this raises a key question: what does it take for such users to align the algorithm with their true interests? To investigate these dynamics, we model the user’s decision process as split between a rational system 2 that decides whether to engage and an impulsive system 1 that determines how long engagement lasts. We then study a multi-leader, single-follower extensive Stackelberg game, where users, specifically system 2, lead by committing to engagement strategies and the algorithm best-responds based on observed interactions. We define the burden of alignment as the minimum horizon over which users must optimize to effectively steer the algorithm. We show that a critical horizon exists: users who are sufficiently foresighted can achieve alignment, while those who are not are instead aligned to the algorithm’s objective. This critical horizon can be long, imposing a substantial burden. However, even a small, costly signal (e.g., an extra click) can significantly reduce it. Overall, our framework explains how users with inconsistent preferences can align an engagement-driven algorithm with their interests in a Stackelberg equilibrium, highlighting both the challenges and potential remedies for achieving alignment.

Method: Implemented Flavell’s cognitive monitoring model as a three-phase iterative system within the broader Monitor-Generate-Verify framework, combining strategic planning with verification mechanisms.

Result: Achieved 75.42% accuracy on GSM8K, outperforming SELF-REFINE (68.44%) and Self-Verification (67.07%), with fewer attempts (1.3 vs 2.0) at 27-37% increased inference cost.

Conclusion: Upfront monitoring produces higher-quality initial solutions that reduce refinement needs, though evaluation beyond arithmetic reasoning is needed to establish generalizability.

Abstract: Current approaches to enhancing LLM reasoning follows two isolated paradigms: Monitor-Generate methods like Plan-and-Solve (Wang et al., 2023) and SELF-DISCOVER (Zhou et al., 2024) excel at strategic planning but lack mechanisms to verify whether selected strategies succeed; while Generate-Verify approaches like Self-Verification (Weng et al., 2022) and SELF-REFINE (Madaan et al., 2023) iteratively refine outputs but commence generation blindly without task assessment. This separation creates inefficiencies – strategies fail without feedback, and refinement occurs without strategic grounding. We address this gap by implementing Flavell’s cognitive monitoring model (1979) from the broader Monitor-Generate-Verify framework (Oh and Gobet, 2025), operationalising it as a three-phase iterative system. On GSM8K, preliminary results show 75.42% accuracy versus 68.44% for SELF-REFINE and 67.07% for Self-Verification, while requiring fewer attempts (1.3 vs 2.0) at 27-37% increased inference cost. These initial findings suggest upfront monitoring produces higher-quality initial solutions that reduce refinement needs, though evaluation beyond arithmetic reasoning is needed to establish generalisability.

Method: Proposes Humanoid-inspired Structural Causal Model (HSCM) that replicates human vision systems’ hierarchical processing, disentangles and reweights key image attributes (color, texture, shape), and models fine-grained causal mechanisms.

Result: HSCM outperforms existing domain generalization models in both theoretical and empirical evaluations, providing better generalization across diverse domains with robust performance and interpretability.

Conclusion: HSCM offers a more principled approach for capturing causal relationships and improving model robustness by leveraging human intelligence principles, enabling effective transfer learning in dynamic, complex environments.

Abstract: This paper proposes the Humanoid-inspired Structural Causal Model (HSCM), a novel causal framework inspired by human intelligence, designed to overcome the limitations of conventional domain generalization models. Unlike approaches that rely on statistics to capture data-label dependencies and learn distortion-invariant representations, HSCM replicates the hierarchical processing and multi-level learning of human vision systems, focusing on modeling fine-grained causal mechanisms. By disentangling and reweighting key image attributes such as color, texture, and shape, HSCM enhances generalization across diverse domains, ensuring robust performance and interpretability. Leveraging the flexibility and adaptability of human intelligence, our approach enables more effective transfer and learning in dynamic, complex environments. Through both theoretical and empirical evaluations, we demonstrate that HSCM outperforms existing domain generalization models, providing a more principled method for capturing causal relationships and improving model robustness. The code is available at https://github.com/lambett/HSCM.

Method: RGMem organizes dialogue history in multiple scales: extracts semantics from episodic fragments, then applies hierarchical coarse-graining and rescaling operations to progressively form dynamically-evolved user profiles, modeling memory evolution as multi-scale information compression.

Result: The framework enables high-level and accurate user profiles to be formed from noisy, microscopic-level interactions, achieving effective long-term memory and behavioral consistency.

Conclusion: RGMem successfully addresses the limitations of current memory systems by introducing a physics-inspired multi-scale approach that can distill latent user preferences and maintain cross-session continuity in conversational AI systems.

Abstract: Personalized and continuous interactions are the key to enhancing user experience in today’s large language model (LLM)-based conversational systems, however, the finite context windows and static parametric memory make it difficult to model the cross-session long-term user states and behavioral consistency. Currently, the existing solutions to this predicament, such as retrieval-augmented generation (RAG) and explicit memory systems, primarily focus on fact-level storage and retrieval, lacking the capability to distill latent preferences and deep traits from the multi-turn dialogues, which limits the long-term and effective user modeling, directly leading to the personalized interactions remaining shallow, and hindering the cross-session continuity. To realize the long-term memory and behavioral consistency for Language Agents in LLM era, we propose a self-evolving memory framework RGMem, inspired by the ideology of classic renormalization group (RG) in physics, this framework enables to organize the dialogue history in multiple scales: it first extracts semantics and user insights from episodic fragments, then through hierarchical coarse-graining and rescaling operations, progressively forms a dynamically-evolved user profile. The core innovation of our work lies in modeling memory evolution as a multi-scale process of information compression and emergence, which accomplishes the high-level and accurate user profiles from noisy and microscopic-level interactions.

Method: Integrates clustering, LLM adaptation, and expert-driven evaluation into a unified pipeline to identify key trends, recurring issues, and specific concerns within customer sentiments.

Result: Preliminary manual evaluations show strong alignment between the model’s recommendations and business objectives, demonstrating potential for data-informed decision-making.

Conclusion: ReviewSense offers a new perspective on AI-driven sentiment analysis, proving valuable for refining business strategies and maximizing the impact of customer feedback.

Abstract: As customer feedback becomes increasingly central to strategic growth, the ability to derive actionable insights from unstructured reviews is essential. While traditional AI-driven systems excel at predicting user preferences, far less work has focused on transforming customer reviews into prescriptive, business-facing recommendations. This paper introduces ReviewSense, a novel prescriptive decision support framework that leverages advanced large language models (LLMs) to transform customer reviews into targeted, actionable business recommendations. By identifying key trends, recurring issues, and specific concerns within customer sentiments, ReviewSense extends beyond preference-based systems to provide businesses with deeper insights for sustaining growth and enhancing customer loyalty. The novelty of this work lies in integrating clustering, LLM adaptation, and expert-driven evaluation into a unified, business-facing pipeline. Preliminary manual evaluations indicate strong alignment between the model’s recommendations and business objectives, highlighting its potential for driving data-informed decision-making. This framework offers a new perspective on AI-driven sentiment analysis, demonstrating its value in refining business strategies and maximizing the impact of customer feedback.

Method: NP-ENGINE framework with 10 tasks across 5 domains, using generator-verifier-heuristic pipeline for scalable RLVR training with curriculum learning.

Result: QWEN2.5-7B-NP significantly outperforms GPT-4o on NP-BENCH and achieves SOTA performance. RLVR training enables strong out-of-domain generalization to reasoning and non-reasoning tasks.

Conclusion: Task-rich RLVR training is promising for advancing LLM reasoning, revealing new insights into RLVR scaling laws and improving generalization capabilities.

Abstract: Large Language Models (LLMs) have shown strong reasoning capabilities, with models like OpenAI’s O-series and DeepSeek R1 excelling at tasks such as mathematics, coding, logic, and puzzles through Reinforcement Learning with Verifiable Rewards (RLVR). However, their ability to solve more complex optimization problems - particularly NP-hard tasks - remains underexplored. To bridge this gap, we propose NP-ENGINE, the first comprehensive framework for training and evaluating LLMs on NP-hard problems. NP-ENGINE covers 10 tasks across five domains, each equipped with (i) a controllable instance generator, (ii) a rule-based verifier, and (iii) a heuristic solver that provides approximate optimal solutions as ground truth. This generator-verifier-heuristic pipeline enables scalable and verifiable RLVR training under hierarchical difficulties. We also introduce NP-BENCH, a benchmark derived from NP-ENGINE-DATA, specifically designed to evaluate LLMs’ ability to tackle NP-hard level reasoning problems, focusing not only on feasibility but also on solution quality. Additionally, we present QWEN2.5-7B-NP, a model trained via zero-RLVR with curriculum learning on Qwen2.5-7B-Instruct, which significantly outperforms GPT-4o on NP-BENCH and achieves SOTA performance with the same model size. Beyond in-domain tasks, we demonstrate that RLVR training on NP-ENGINE-DATA enables strong out-of-domain (OOD) generalization to reasoning tasks (logic, puzzles, math, and knowledge), as well as non-reasoning tasks such as instruction following. We also observe a scaling trend: increasing task diversity improves OOD generalization. These findings suggest that task-rich RLVR training is a promising direction for advancing LLM’s reasoning ability, revealing new insights into the scaling laws of RLVR.

Method: Encodes the light linear functional programming language (LLFPL) using holographic declarative memory for types and a Lisp VSA variant for terms, allowing neural networks to learn types from embedding spaces.

Result: Developed Doug language where types are learnable by neural networks and nearby points in embedding space have similar types in structure and content.

Conclusion: Doug represents progress toward modeling actual human mental representations and their acquisition, bringing us closer to understanding how skills are learned in the brain.

Abstract: We present a typed computer language, Doug, in which all typed programs may be proved to halt in polynomial time, encoded in a vector-symbolic architecture (VSA). Doug is just an encoding of the light linear functional programming language (LLFPL) described by (Schimanski2009, ch. 7). The types of Doug are encoded using a slot-value encoding scheme based on holographic declarative memory (HDM; Kelly, 2020). The terms of Doug are encoded using a variant of the Lisp VSA defined by (Flanagan, 2024). Doug allows for some points on the embedding space of a neural network to be interpreted as types, where the types of nearby points are similar both in structure and content. Types in Doug are therefore learnable by a neural network. Following (Chollet, 2019), (Card, 1983), and (Newell, 1981), we view skill as the application of a procedure, or program of action, that causes a goal to be satisfied. Skill acquisition may therefore be expressed as program synthesis. Using Doug, we hope to describe a form of learning of skilled behaviour that follows a human-like pace of skill acquisition (i.e., substantially faster than brute force; Heathcote, 2000), exceeding the efficiency of all currently existing approaches (Kaplan, 2020; Jones, 2021; Chollet, 2024). Our approach brings us one step closer to modeling human mental representations, as they must actually exist in the brain, and those representations’ acquisition, as they are actually learned.

Method: Developed a decentralized multi-agent reinforcement learning (MARL) system using Pygame simulation, where each traffic signal acts as an autonomous agent making decisions based on local observations and neighbor information.

Result: MARL approach showed statistically significant improvements over baseline fixed-time controller, with reduced average vehicle wait times and improved overall throughput.

Conclusion: MARL-based dynamic control strategies show substantial promise for urban traffic management efficiency, though more research is needed for scalability and real-world implementation.

Abstract: Urban traffic congestion, particularly at intersections, significantly impacts travel time, fuel consumption, and emissions. Traditional fixed-time signal control systems often lack the adaptability to manage dynamic traffic patterns effectively. This study explores the application of multi-agent reinforcement learning (MARL) to optimize traffic signal coordination across multiple intersections within a simulated environment. Utilizing Pygame, a simulation was developed to model a network of interconnected intersections with randomly generated vehicle flows to reflect realistic traffic variability. A decentralized MARL controller was implemented, in which each traffic signal operates as an autonomous agent, making decisions based on local observations and information from neighboring agents. Performance was evaluated against a baseline fixed-time controller using metrics such as average vehicle wait time and overall throughput. The MARL approach demonstrated statistically significant improvements, including reduced average waiting times and improved throughput. These findings suggest that MARL-based dynamic control strategies hold substantial promise for improving urban traffic management efficiency. More research is recommended to address scalability and real-world implementation challenges.

Method: Proposes Urban-R1, a reinforcement learning-based post-training framework using Group Relative Policy Optimization (GRPO) to optimize reasoning across geographic groups, with urban region profiling as a proxy task to provide measurable rewards from multimodal urban data.

Result: Extensive experiments show Urban-R1 effectively mitigates geo-bias and improves cross-region generalization, outperforming both supervised fine-tuning trained models and closed-source models across diverse regions and tasks.

Conclusion: Reinforcement learning alignment represents a promising pathway toward equitable and trustworthy urban intelligence by addressing geospatial bias in urban foundation models.

Abstract: Rapid urbanization intensifies the demand for Urban General Intelligence (UGI), referring to AI systems that can understand and reason about complex urban environments. Recent studies have built urban foundation models using supervised fine-tuning (SFT) of LLMs and MLLMs, yet these models exhibit persistent geospatial bias, producing regionally skewed predictions and limited generalization. To this end, we propose Urban-R1, a reinforcement learning-based post-training framework that aligns MLLMs with the objectives of UGI. Urban-R1 adopts Group Relative Policy Optimization (GRPO) to optimize reasoning across geographic groups and employs urban region profiling as a proxy task to provide measurable rewards from multimodal urban data. Extensive experiments across diverse regions and tasks show that Urban-R1 effectively mitigates geo-bias and improves cross-region generalization, outperforming both SFT-trained and closed-source models. Our results highlight reinforcement learning alignment as a promising pathway toward equitable and trustworthy urban intelligence.

Method: Developed BuildArena benchmark with four components: customizable framework, extendable task design spanning static/dynamic mechanics, 3D Spatial Geometric Computation Library, and baseline LLM agentic workflow for evaluation.

Result: Comprehensively evaluated eight frontier LLMs on their capabilities for language-driven and physics-grounded construction automation using the BuildArena benchmark.

Conclusion: BuildArena provides the first standardized framework for assessing LLMs in engineering construction automation, enabling systematic comparison and analysis of model capabilities in this domain.

Abstract: Engineering construction automation aims to transform natural language specifications into physically viable structures, requiring complex integrated reasoning under strict physical constraints. While modern LLMs possess broad knowledge and strong reasoning capabilities that make them promising candidates for this domain, their construction competencies remain largely unevaluated. To address this gap, we introduce BuildArena, the first physics-aligned interactive benchmark designed for language-driven engineering construction. It contributes to the community in four aspects: (1) a highly customizable benchmarking framework for in-depth comparison and analysis of LLMs; (2) an extendable task design strategy spanning static and dynamic mechanics across multiple difficulty tiers; (3) a 3D Spatial Geometric Computation Library for supporting construction based on language instructions; (4) a baseline LLM agentic workflow that effectively evaluates diverse model capabilities. On eight frontier LLMs, BuildArena comprehensively evaluates their capabilities for language-driven and physics-grounded construction automation. The project page is at https://build-arena.github.io/.

Method: Uses transition-based flow matching to jointly optimize a retrieval policy and flow estimator, which factorizes retrieval outcome rewards into intermediate states to guide proportional candidate retrieval from KGs.

Result: Outperforms strong KG-RAG baselines including GPT-4o by 10% on average in hit rate and recall on STaRK benchmark, with strong generalization to unseen KGs.

Conclusion: GraphFlow effectively retrieves diverse and relevant knowledge from text-rich KGs for real-world queries, demonstrating robustness and superior performance over existing methods.

Abstract: Retrieval-Augmented Generation (RAG) based on knowledge graphs (KGs) enhances large language models (LLMs) by providing structured and interpretable external knowledge. However, existing KG-based RAG methods struggle to retrieve accurate and diverse information from text-rich KGs for complex real-world queries. Process Reward Models (PRMs) offer a way to align the retrieval process of KG-based RAG with query-specific knowledge requirements, but they heavily rely on process-level supervision signals that are expensive and hard to obtain on KGs. To address this challenge, we propose GraphFlow, a framework that efficiently retrieves accurate and diverse knowledge required for real-world queries from text-rich KGs. GraphFlow employs a transition-based flow matching objective to jointly optimize a retrieval policy and a flow estimator. The flow estimator factorizes the reward of the retrieval outcome into the intermediate retrieval states. Such reward factorization guides the retrieval policy to retrieve candidates from KGs in proportion to their reward. This allows GraphFlow to explore high-quality regions of KGs that yield diverse and relevant results. We evaluate GraphFlow on the STaRK benchmark, which includes real-world queries from multiple domains over text-rich KGs. GraphFlow outperforms strong KG-RAG baselines, including GPT-4o, by 10% on average in hit rate and recall. It also shows strong generalization to unseen KGs, demonstrating its effectiveness and robustness.

Method: Transforms triple confidences into distributions, uses semi-supervised learning with relational learning on labeled data and unlabeled data with pseudo labels generated by meta-learning to augment training data and rebalance confidence distributions.

Result: Experiments on two UKG datasets show ssCDL consistently outperforms state-of-the-art baselines across different evaluation metrics.

Conclusion: ssCDL effectively addresses the imbalanced confidence distribution problem in UKG completion through confidence distribution learning and semi-supervised meta-learning.

Abstract: Uncertain knowledge graphs (UKGs) associate each triple with a confidence score to provide more precise knowledge representations. Recently, since real-world UKGs suffer from the incompleteness, uncertain knowledge graph (UKG) completion attracts more attention, aiming to complete missing triples and confidences. Current studies attempt to learn UKG embeddings to solve this problem, but they neglect the extremely imbalanced distributions of triple confidences. This causes that the learnt embeddings are insufficient to high-quality UKG completion. Thus, in this paper, to address the above issue, we propose a new semi-supervised Confidence Distribution Learning (ssCDL) method for UKG completion, where each triple confidence is transformed into a confidence distribution to introduce more supervision information of different confidences to reinforce the embedding learning process. ssCDL iteratively learns UKG embedding by relational learning on labeled data (i.e., existing triples with confidences) and unlabeled data with pseudo labels (i.e., unseen triples with the generated confidences), which are predicted by meta-learning to augment the training data and rebalance the distribution of triple confidences. Experiments on two UKG datasets demonstrate that ssCDL consistently outperforms state-of-the-art baselines in different evaluation metrics.

Method: A neuro-symbolic multi-agent architecture where agents’ belief states are represented as Kripke models, enabling reasoning about possibility and necessity using modal logic. Domain-specific knowledge is encoded as logical constraints to guide hypothesis generation and prevent untenable conclusions.

Result: The system successfully diagnoses complex, cascading failures in a high-fidelity simulated particle accelerator environment by combining semantic intuition of language models with rigorous validation of modal logic and factual world models.

Conclusion: This approach showcases a viable path toward more robust, reliable, and verifiable autonomous agents by integrating the strengths of language models with formal logical reasoning.

Abstract: The development of intelligent agents, particularly those powered by language models (LMs), has shown a critical role in various environments that require intelligent and autonomous decision-making. Environments are not passive testing grounds, and they represent the data required for agents to learn and exhibit in very challenging conditions that require adaptive, complex, and autonomous capacity to make decisions. While the paradigm of scaling models and datasets has led to remarkable emergent capabilities, we argue that scaling the structure, fidelity, and logical consistency of agent reasoning within these environments is a crucial, yet underexplored, dimension of AI research. This paper introduces a neuro-symbolic multi-agent architecture where the belief states of individual agents are formally represented as Kripke models. This foundational choice enables them to reason about known concepts of \emph{possibility} and \emph{necessity} using the formal language of modal logic. In this work, we use immutable, domain-specific knowledge to make an informed root cause diagnosis, which is encoded as logical constraints essential for proper, reliable, and explainable diagnosis. In the proposed model, we show constraints that actively guide the hypothesis generation of LMs, effectively preventing them from reaching physically or logically untenable conclusions. In a high-fidelity simulated particle accelerator environment, our system successfully diagnoses complex, cascading failures by combining the powerful semantic intuition of LMs with the rigorous, verifiable validation of modal logic and a factual world model and showcasing a viable path toward more robust, reliable, and verifiable autonomous agents.

Method: MERCI uses a lightweight Coin Flipping Network (CFN) to estimate pseudo count and epistemic uncertainty over reasoning trajectories, converting them into intrinsic rewards that value novelty while preserving task reward signals. It’s integrated into RL frameworks like GRPO.

Result: Experiments show MERCI encourages richer and more varied chains of thought, significantly improves performance over strong baselines, and helps policies escape local routines to discover better solutions.

Conclusion: Targeted intrinsic motivation can make exploration reliable for language model reasoning, with MERCI demonstrating effective exploration enhancement in complex reasoning tasks.

Abstract: Reinforcement Learning (RL) has become a compelling way to strengthen the multi step reasoning ability of Large Language Models (LLMs). However, prevalent RL paradigms still lean on sparse outcome-based rewards and limited exploration, which often drives LLMs toward repetitive and suboptimal reasoning patterns. In this paper, we study the central question of how to design exploration for LLM reasoning and introduce MERCI (Motivating Exploration in LLM Reasoning with Count-based Intrinsic Rewards), a novel RL algorithm that augments policy optimization with a principled intrinsic reward. Building on the idea of count-based exploration, MERCI leverages a lightweight Coin Flipping Network (CFN) to estimate the pseudo count and further epistemic uncertainty over reasoning trajectories, and converts them into an intrinsic reward that values novelty while preserving the learning signal from task rewards. We integrate MERCI into some advanced RL frameworks like Group Relative Policy Optimization (GRPO). Experiments on complex reasoning benchmarks demonstrate that MERCI encourages richer and more varied chains of thought, significantly improves performance over strong baselines, and helps the policy escape local routines to discover better solutions. It indicates that our targeted intrinsic motivation can make exploration reliable for language model reasoning.

Method: BiCA uses learnable protocols, representation mapping, and KL-budget constraints for controlled co-evolution between humans and AI systems.

Result: In collaborative navigation tasks, BiCA achieved 85.5% success rate (vs 70.3% baseline), with 230% better mutual adaptation and 332% better protocol convergence. Emergent protocols outperformed handcrafted ones by 84%, and bidirectional adaptation improved safety with +23% out-of-distribution robustness.

Conclusion: The 46% synergy improvement demonstrates that optimal collaboration exists at the intersection of human and AI capabilities, validating the shift from single-directional to co-alignment paradigms.

Abstract: Current AI alignment through RLHF follows a single directional paradigm that AI conforms to human preferences while treating human cognition as fixed. We propose a shift to co-alignment through Bidirectional Cognitive Alignment (BiCA), where humans and AI mutually adapt. BiCA uses learnable protocols, representation mapping, and KL-budget constraints for controlled co-evolution. In collaborative navigation, BiCA achieved 85.5% success versus 70.3% baseline, with 230% better mutual adaptation and 332% better protocol convergence. Emergent protocols outperformed handcrafted ones by 84%, while bidirectional adaptation unexpectedly improved safety (+23% out-of-distribution robustness). The 46% synergy improvement demonstrates optimal collaboration exists at the intersection, not union, of human and AI capabilities, validating the shift from single-directional to co-alignment paradigms.

Method: Review and analysis of large-scale AI model applications across five neuroscience domains, focusing on their ability to facilitate end-to-end learning from raw neural data and integrate biologically informed architectural constraints.

Result: Demonstrated that these models successfully address major computational neuroscience challenges and enable multimodal neural data integration, with the interaction between neuroscience and AI becoming increasingly reciprocal.

Conclusion: Large-scale AI models show significant promise for neuroscience but require rigorous evaluation frameworks, effective domain knowledge integration, and comprehensive ethical guidelines for clinical deployment, along with systematic validation using critical neuroscience datasets.

Abstract: The advent of large-scale artificial intelligence (AI) models has a transformative effect on neuroscience research, which represents a paradigm shift from the traditional computational methods through the facilitation of end-to-end learning from raw brain signals and neural data. In this paper, we explore the transformative effects of large-scale AI models on five major neuroscience domains: neuroimaging and data processing, brain-computer interfaces and neural decoding, molecular neuroscience and genomic modeling, clinical assistance and translational frameworks, and disease-specific applications across neurological and psychiatric disorders. These models are demonstrated to address major computational neuroscience challenges, including multimodal neural data integration, spatiotemporal pattern interpretation, and the derivation of translational frameworks for clinical deployment. Moreover, the interaction between neuroscience and AI has become increasingly reciprocal, as biologically informed architectural constraints are now incorporated to develop more interpretable and computationally efficient models. This review highlights both the notable promise of such technologies and key implementation considerations, with particular emphasis on rigorous evaluation frameworks, effective domain knowledge integration, and comprehensive ethical guidelines for clinical use. Finally, a systematic listing of critical neuroscience datasets used to derive and validate large-scale AI models across diverse research applications is provided.

Method: Proposes AFL framework that decomposes VRPs into three subtasks managed by four specialized agents. The agents coordinate to enforce consistency and logical soundness, enabling self-contained code generation without handcrafted modules or external solvers.

Result: Extensive experiments on 60 complex VRPs show comparable performance to meticulously designed algorithms. Substantially outperforms existing LLM-based baselines with code reliability and solution feasibility rates close to 100% on evaluated benchmarks.

Conclusion: AFL demonstrates that agentic frameworks with LLMs can achieve full automation for complex VRPs, providing trustworthy solutions without external dependencies while maintaining high performance standards.

Abstract: Complex vehicle routing problems (VRPs) remain a fundamental challenge, demanding substantial expert effort for intent interpretation and algorithm design. While large language models (LLMs) offer a promising path toward automation, current approaches still rely on external intervention, which restrict autonomy and often lead to execution errors and low solution feasibility. To address these challenges, we propose an Agentic Framework with LLMs (AFL) for solving complex vehicle routing problems, achieving full automation from problem instance to solution. AFL directly extracts knowledge from raw inputs and enables self-contained code generation without handcrafted modules or external solvers. To improve trustworthiness, AFL decomposes the overall pipeline into three manageable subtasks and employs four specialized agents whose coordinated interactions enforce cross-functional consistency and logical soundness. Extensive experiments on 60 complex VRPs, ranging from standard benchmarks to practical variants, validate the effectiveness and generality of our framework, showing comparable performance against meticulously designed algorithms. Notably, it substantially outperforms existing LLM-based baselines in both code reliability and solution feasibility, achieving rates close to 100% on the evaluated benchmarks.

Method: Systematic review of how planning, tool use, and memory capabilities have evolved from externally scripted modules to end-to-end learned behaviors, examining the role of RL in enabling this shift across language, vision and embodied domains.

Result: Identifies a coherent trajectory toward model-native agentic AI as an integrated learning and interaction framework, with applications in Deep Research agents (long-horizon reasoning) and GUI agents (embodied interaction).

Conclusion: The paradigm shift marks the transition from constructing systems that apply intelligence to developing models that grow intelligence through experience, with continued internalization of capabilities like multi-agent collaboration and reflection.

Abstract: The rapid evolution of agentic AI marks a new phase in artificial intelligence, where Large Language Models (LLMs) no longer merely respond but act, reason, and adapt. This survey traces the paradigm shift in building agentic AI: from Pipeline-based systems, where planning, tool use, and memory are orchestrated by external logic, to the emerging Model-native paradigm, where these capabilities are internalized within the model’s parameters. We first position Reinforcement Learning (RL) as the algorithmic engine enabling this paradigm shift. By reframing learning from imitating static data to outcome-driven exploration, RL underpins a unified solution of LLM + RL + Task across language, vision and embodied domains. Building on this, the survey systematically reviews how each capability – Planning, Tool use, and Memory – has evolved from externally scripted modules to end-to-end learned behaviors. Furthermore, it examines how this paradigm shift has reshaped major agent applications, specifically the Deep Research agent emphasizing long-horizon reasoning and the GUI agent emphasizing embodied interaction. We conclude by discussing the continued internalization of agentic capabilities like Multi-agent collaboration and Reflection, alongside the evolving roles of the system and model layers in future agentic AI. Together, these developments outline a coherent trajectory toward model-native agentic AI as an integrated learning and interaction framework, marking the transition from constructing systems that apply intelligence to developing models that grow intelligence through experience.

Method: The survey organizes RL-based agentic search along three dimensions: (i) What RL is for (functional roles), (ii) How RL is used (optimization strategies), and (iii) Where RL is applied (scope of optimization). It summarizes representative methods, evaluation protocols, and applications.

Result: The survey provides the first comprehensive overview of RL-based agentic search, organizing the emerging field and discussing open challenges and future directions.

Conclusion: RL-based agentic search addresses limitations of traditional RAG by enabling adaptive control over retrieval and reasoning through multi-step interaction with search environments, offering promising directions for building reliable and scalable search systems.

Abstract: The advent of large language models (LLMs) has transformed information access and reasoning through open-ended natural language interaction. However, LLMs remain limited by static knowledge, factual hallucinations, and the inability to retrieve real-time or domain-specific information. Retrieval-Augmented Generation (RAG) mitigates these issues by grounding model outputs in external evidence, but traditional RAG pipelines are often single turn and heuristic, lacking adaptive control over retrieval and reasoning. Recent advances in agentic search address these limitations by enabling LLMs to plan, retrieve, and reflect through multi-step interaction with search environments. Within this paradigm, reinforcement learning (RL) offers a powerful mechanism for adaptive and self-improving search behavior. This survey provides the first comprehensive overview of \emph{RL-based agentic search}, organizing the emerging field along three complementary dimensions: (i) What RL is for (functional roles), (ii) How RL is used (optimization strategies), and (iii) Where RL is applied (scope of optimization). We summarize representative methods, evaluation protocols, and applications, and discuss open challenges and future directions toward building reliable and scalable RL driven agentic search systems. We hope this survey will inspire future research on the integration of RL and agentic search. Our repository is available at https://github.com/ventr1c/Awesome-RL-based-Agentic-Search-Papers.

Method: Proposes Multi-view Visual Token Compressor using multi-head attention to compress image tokens from textual/visual views, plus attention pruning with linear projection to reduce model size.

Result: Achieves state-of-the-art performance on FB15k-237-IMG and WN18-IMG benchmarks while significantly improving computational efficiency.

Conclusion: ELMM establishes a new paradigm for multimodal knowledge graph completion by balancing performance and efficiency through token compression and model pruning.

Abstract: Multimodal Knowledge Graphs (MKGs) extend traditional knowledge graphs by incorporating visual and textual modalities, enabling richer and more expressive entity representations. However, existing MKGs often suffer from incompleteness, which hinder their effectiveness in downstream tasks. Therefore, multimodal knowledge graph completion (MKGC) task is receiving increasing attention. While large language models (LLMs) have shown promise for knowledge graph completion (KGC), their application to the multimodal setting remains underexplored. Moreover, applying Multimodal Large Language Models (MLLMs) to the task of MKGC introduces significant challenges: (1) the large number of image tokens per entity leads to semantic noise and modality conflicts, and (2) the high computational cost of processing large token inputs. To address these issues, we propose Efficient Lightweight Multimodal Large Language Models (ELMM) for MKGC. ELMM proposes a Multi-view Visual Token Compressor (MVTC) based on multi-head attention mechanism, which adaptively compresses image tokens from both textual and visual views, thereby effectively reducing redundancy while retaining necessary information and avoiding modality conflicts. Additionally, we design an attention pruning strategy to remove redundant attention layers from MLLMs, thereby significantly reducing the inference cost. We further introduce a linear projection to compensate for the performance degradation caused by pruning. Extensive experiments on benchmark FB15k-237-IMG and WN18-IMG demonstrate that ELMM achieves state-of-the-art performance while substantially improving computational efficiency, establishing a new paradigm for multimodal knowledge graph completion.

Method: GraphVista uses hierarchical organization with a GraphRAG base for scalability, retrieving only task-relevant content. It employs a planning agent to route tasks to the most suitable modality - text for simple properties and vision for complex structural reasoning.

Result: GraphVista scales to graphs 200× larger than existing benchmarks and achieves up to 4.4× quality improvement over state-of-the-art methods by fully exploiting complementary modality strengths.

Conclusion: GraphVista successfully addresses scalability and modality coordination challenges in graph understanding, demonstrating superior performance across various graph sizes and reasoning tasks.

Abstract: Vision-language models (VLMs) have shown promise in graph understanding, but remain limited by input-token constraints, facing scalability bottlenecks and lacking effective mechanisms to coordinate textual and visual modalities. To address these challenges, we propose GraphVista, a unified framework that enhances both scalability and modality coordination in graph understanding. For scalability, GraphVista organizes graph information hierarchically into a lightweight GraphRAG base, which retrieves only task-relevant textual descriptions and high-resolution visual subgraphs, compressing redundant context while preserving key reasoning elements. For modality coordination, GraphVista introduces a planning agent that routes tasks to the most suitable modality-using the text modality for simple property reasoning and the visual modality for local and structurally complex reasoning grounded in explicit topology. Extensive experiments demonstrate that GraphVista scales to large graphs, up to $200\times$ larger than those used in existing benchmarks, and consistently outperforms existing textual, visual, and fusion-based methods, achieving up to $4.4\times$ quality improvement over the state-of-the-art baselines by fully exploiting the complementary strengths of both modalities.

Method: CDC uses a C-D-C triple structure <Concept, Relation@Domain, Concept’> with domain specifications as dynamic classification dimensions, formalizes standardized relation predicates, and implements in Prolog for inference.

Result: Case studies in education, enterprise systems, and technical documentation show CDC enables context-aware reasoning, cross-domain analogy, and personalized knowledge modeling not possible with traditional ontology frameworks.

Conclusion: CDC overcomes limitations of traditional knowledge graphs by making domains explicit reasoning components, enabling more flexible and context-sensitive knowledge representation.

Abstract: Traditional knowledge graphs are constrained by fixed ontologies that organize concepts within rigid hierarchical structures. The root cause lies in treating domains as implicit context rather than as explicit, reasoning-level components. To overcome these limitations, we propose the Domain-Contextualized Concept Graph (CDC), a novel knowledge modeling framework that elevates domains to first-class elements of conceptual representation. CDC adopts a C-D-C triple structure - <Concept, Relation@Domain, Concept’> - where domain specifications serve as dynamic classification dimensions defined on demand. Grounded in a cognitive-linguistic isomorphic mapping principle, CDC operationalizes how humans understand concepts through contextual frames. We formalize more than twenty standardized relation predicates (structural, logical, cross-domain, and temporal) and implement CDC in Prolog for full inference capability. Case studies in education, enterprise knowledge systems, and technical documentation demonstrate that CDC enables context-aware reasoning, cross-domain analogy, and personalized knowledge modeling - capabilities unattainable under traditional ontology-based frameworks.

Method: Proposed curriculum-based agentic training paradigm that emulates human data scientists’ learning trajectory, and a data-grounded trajectory synthesis framework for constructing high-quality training data.

Result: DeepAnalyze-8B outperforms previous workflow-based agents built on most advanced proprietary LLMs, demonstrating capability across data question answering, specialized analytical tasks, and open-ended data research.

Conclusion: DeepAnalyze-8B paves the way toward autonomous data science with its open-source model, code, and training data, showing that agentic training enables LLMs to progressively acquire multiple capabilities for complex data science tasks.

Abstract: Autonomous data science, from raw data sources to analyst-grade deep research reports, has been a long-standing challenge, and is now becoming feasible with the emergence of powerful large language models (LLMs). Recent workflow-based data agents have shown promising results on specific data tasks but remain fundamentally limited in achieving fully autonomous data science due to their reliance on predefined workflows. In this paper, we introduce DeepAnalyze-8B, the first agentic LLM designed for autonomous data science, capable of automatically completing the end-toend pipeline from data sources to analyst-grade deep research reports. To tackle high-complexity data science tasks, we propose a curriculum-based agentic training paradigm that emulates the learning trajectory of human data scientists, enabling LLMs to progressively acquire and integrate multiple capabilities in real-world environments. We also introduce a data-grounded trajectory synthesis framework that constructs high-quality training data. Through agentic training, DeepAnalyze learns to perform a broad spectrum of data tasks, ranging from data question answering and specialized analytical tasks to open-ended data research. Experiments demonstrate that, with only 8B parameters, DeepAnalyze outperforms previous workflow-based agents built on most advanced proprietary LLMs. The model, code, and training data of DeepAnalyze are open-sourced, paving the way toward autonomous data science.

Method: Architecturally enforce and reward agent’s reasoning process via RL formulated as POMDP, decomposing reasoning into State Estimation and Transition Modeling, with World Modeling Reward and Bi-Level GAE for turn-aware credit assignment.

Result: A 3B-parameter model achieves score of 0.82 across five diverse agent benchmarks, representing 3× improvement over untrained counterpart (0.21) and outperforming proprietary models like GPT-5 (0.75), Gemini 2.5 Pro (0.67) and Claude 4.5 (0.62).

Conclusion: The optimal belief representation is task-dependent: Natural Language excels at semantic relationships, while Structured formats are essential for precise manipulation. Visual state reasoning enables VLM agents to construct effective internal world models.

Abstract: A key challenge in training Vision-Language Model (VLM) agents, compared to Language Model (LLM) agents, lies in the shift from textual states to complex visual observations. This transition introduces partial observability and demands robust world modeling. We ask: Can VLM agents construct internal world models through explicit visual state reasoning? To address this question, we architecturally enforce and reward the agent’s reasoning process via reinforcement learning (RL), formulating it as a Partially Observable Markov Decision Process (POMDP). We find that decomposing the agent’s reasoning into State Estimation (“what is the current state?”) and Transition Modeling (“what comes next?”) is critical for success, as demonstrated through five reasoning strategies. Our investigation into how agents represent internal beliefs reveals that the optimal representation is task-dependent: Natural Language excels at capturing semantic relationships in general tasks, while Structured formats are indispensable for precise manipulation and control. Building on these insights, we design a World Modeling Reward that provides dense, turn-level supervision for accurate state prediction, and introduce Bi-Level General Advantage Estimation (Bi-Level GAE) for turn-aware credit assignment. Through this form of visual state reasoning, a 3B-parameter model achieves a score of 0.82 across five diverse agent benchmarks, representing a 3$\times$ improvement over its untrained counterpart (0.21) and outperforming proprietary reasoning models such as GPT-5 (0.75), Gemini 2.5 Pro (0.67) and Claude 4.5 (0.62). All experiments are conducted within our VAGEN framework, a scalable system for training and analyzing multi-turn VLM agents in diverse visual environments. Code and data are publicly available at https://vagen-ai.github.io.

Method: Proposed evaluation methodology using Atari’s Ms. Pacman environment and four XRL algorithms to test whether users can identify an agent’s goal from decision-making explanations.

Result: Only one XRL algorithm achieved greater than random accuracy for tested goals; users were generally overconfident in their selections; users’ self-reported ease of identification and understanding did not correlate with their accuracy.

Conclusion: Current XRL algorithms have limited effectiveness for debugging purposes, and user confidence is not a reliable indicator of explanation quality.

Abstract: Debugging is a core application of explainable reinforcement learning (XRL) algorithms; however, limited comparative evaluations have been conducted to understand their relative performance. We propose a novel evaluation methodology to test whether users can identify an agent’s goal from an explanation of its decision-making. Utilising the Atari’s Ms. Pacman environment and four XRL algorithms, we find that only one achieved greater than random accuracy for the tested goals and that users were generally overconfident in their selections. Further, we find that users’ self-reported ease of identification and understanding for every explanation did not correlate with their accuracy.

Method: Multi-agent LLM framework with systematic design space exploration through collaboration, grounded instruction, dynamic context management, and strategic search, mimicking expert engineer workflows with hardware trade-off reasoning and profiling feedback.

Result: Achieves substantial improvements over baselines: produces correct solutions where baselines fail, and achieves kernels with up to 16x faster runtime performance on the KernelBench benchmark.

Conclusion: Agentic LLM frameworks show strong potential for advancing fully automated, scalable GPU kernel optimization by enabling systematic exploration and iterative refinement.

Abstract: The efficiency of GPU kernels is central to the progress of modern AI, yet optimizing them remains a difficult and labor-intensive task due to complex interactions between memory hierarchies, thread scheduling, and hardware-specific characteristics. While recent advances in large language models (LLMs) provide new opportunities for automated code generation, existing approaches largely treat LLMs as single-shot generators or naive refinement tools, limiting their effectiveness in navigating the irregular kernel optimization landscape. We introduce an LLM agentic framework for GPU kernel optimization that systematically explores the design space through multi-agent collaboration, grounded instruction, dynamic context management, and strategic search. This framework mimics the workflow of expert engineers, enabling LLMs to reason about hardware trade-offs, incorporate profiling feedback, and refine kernels iteratively. We evaluate our approach on KernelBench, a benchmark for LLM-based kernel optimization, and demonstrate substantial improvements over baseline agents: our system produces correct solutions where baselines often fail, and achieves kernels with up to 16x faster runtime performance. These results highlight the potential of agentic LLM frameworks to advance fully automated, scalable GPU kernel optimization.

Method: ToolCritic identifies eight specific tool-calling error types, provides targeted feedback to the main LLM, which then revises its response. A synthetic dataset is created to train ToolCritic.

Result: Experimental results on the Schema-Guided Dialogue dataset show ToolCritic improves tool-calling accuracy by up to 13% compared to zero-shot prompting and self-correction baselines.

Conclusion: ToolCritic represents a significant step toward more robust LLM integration with external tools in real-world dialogue applications.

Abstract: Tool-augmented large language models (LLMs) are increasingly employed in real-world applications, but tool usage errors still hinder their reliability. We introduce ToolCritic, a diagnostic framework that evaluates and improves LLM behavior in multi-turn, tool-augmented dialogues. ToolCritic detects eight distinct error types specific to tool-calling (e.g., premature invocation, argument misalignment, and misinterpretation of tool outputs) and provides targeted feedback to the main LLM. The main LLM, assumed to have strong reasoning, task understanding and orchestration capabilities, then revises its response based on ToolCritic’s feedback. We systematically define these error categories and construct a synthetic dataset to train ToolCritic. Experimental results on the Schema-Guided Dialogue (SGD) dataset demonstrate that ToolCritic improves tool-calling accuracy by up to 13% over baselines, including zero-shot prompting and self-correction techniques. This represents a promising step toward more robust LLM integration with external tools in real-world dialogue applications.

Method: Developed a multi-agent AI system integrating free-text descriptions with ontology labels, using retrieval-augmented generation (RAG) to refine predictions with relevant PubMed literature, reducing hallucinations and enhancing interpretability.

Result: Achieved correct annotations for 77% of mouse gene sets among top predictions. Successfully annotated 5,322 brain cell clusters from mouse brain cell atlas, identifying region-specific gene co-expression patterns and functional roles of gene ensembles.

Conclusion: BRAINCELL-AID creates a valuable resource for community-driven cell type annotation, enabling novel insights into brain cell function and identifying neurologically meaningful descriptions for Basal Ganglia-related cell types.

Abstract: Single-cell RNA sequencing has transformed our ability to identify diverse cell types and their transcriptomic signatures. However, annotating these signatures-especially those involving poorly characterized genes-remains a major challenge. Traditional methods, such as Gene Set Enrichment Analysis (GSEA), depend on well-curated annotations and often perform poorly in these contexts. Large Language Models (LLMs) offer a promising alternative but struggle to represent complex biological knowledge within structured ontologies. To address this, we present BRAINCELL-AID (BRAINCELL-AID: https://biodataai.uth.edu/BRAINCELL-AID), a novel multi-agent AI system that integrates free-text descriptions with ontology labels to enable more accurate and robust gene set annotation. By incorporating retrieval-augmented generation (RAG), we developed a robust agentic workflow that refines predictions using relevant PubMed literature, reducing hallucinations and enhancing interpretability. Using this workflow, we achieved correct annotations for 77% of mouse gene sets among their top predictions. Applying this approach, we annotated 5,322 brain cell clusters from the comprehensive mouse brain cell atlas generated by the BRAIN Initiative Cell Census Network, enabling novel insights into brain cell function by identifying region-specific gene co-expression patterns and inferring functional roles of gene ensembles. BRAINCELL-AID also identifies Basal Ganglia-related cell types with neurologically meaningful descriptions. Hence, we create a valuable resource to support community-driven cell type annotation.

Method: Developed two LLM-based systems: a single-agent system (NAS) with bidirectional analysis, and a debate-based multi-agent system (KPD-MADS) using Karl Popper’s critical dialogue framework with 10-step structured interaction protocol. Both were tested on real corporate cases.

Result: Both systems achieved substantial productivity gains (NAS: 11.55s, KPD-MADS: 91.97s vs human baseline: 1920s). KPD-MADS showed superior reasoning quality with higher ratings in explanatory adequacy (4.0 vs 3.0), practical applicability (4.0 vs 3.0), and usability (62.5 vs 52.5).

Conclusion: Structured multi-agent interaction enhances reasoning rigor and interpretability in financial AI, advancing scalable and defensible automation in corporate credit assessment.

Abstract: Despite advances in financial AI, the automation of evidence-based reasoning remains unresolved in corporate credit assessment, where qualitative non-financial indicators exert decisive influence on loan repayment outcomes yet resist formalization. Existing approaches focus predominantly on numerical prediction and provide limited support for the interpretive judgments required in professional loan evaluation. This study develops and evaluates two operational large language model (LLM)-based systems designed to generate structured reasoning from non-financial evidence. The first is a non-adversarial single-agent system (NAS) that produces bidirectional analysis through a single-pass reasoning pipeline. The second is a debate-based multi-agent system (KPD-MADS) that operationalizes adversarial verification through a ten-step structured interaction protocol grounded in Karl Popper’s critical dialogue framework. Both systems were applied to three real corporate cases and evaluated by experienced credit risk professionals. Compared to manual expert reporting, both systems achieved substantial productivity gains (NAS: 11.55 s per case; KPD-MADS: 91.97 s; human baseline: 1920 s). The KPD-MADS demonstrated superior reasoning quality, receiving higher median ratings in explanatory adequacy (4.0 vs. 3.0), practical applicability (4.0 vs. 3.0), and usability (62.5 vs. 52.5). These findings show that structured multi-agent interaction can enhance reasoning rigor and interpretability in financial AI, advancing scalable and defensible automation in corporate credit assessment.

Method: Systematically extracts and fuses complementary descriptors including color statistics, histograms across multiple color spaces, and texture features (LBP, GLCM) from fish eye images, capturing both global chromatic variations and localized degradations.

Result: LightGBM classifier achieved 77.56% accuracy (14.35% improvement over previous baseline), and ANN with augmented data reached 97.16% accuracy (19.86% improvement over prior best).

Conclusion: Carefully engineered handcrafted features provide a robust, interpretable, and reliable solution for automated fish freshness assessment, offering practical value for food quality monitoring applications.

Abstract: Accurate assessment of fish freshness remains a major challenge in the food industry, with direct consequences for product quality, market value, and consumer health. Conventional sensory evaluation is inherently subjective, inconsistent, and difficult to standardize across contexts, often limited by subtle, species-dependent spoilage cues. To address these limitations, we propose a handcrafted feature-based approach that systematically extracts and incrementally fuses complementary descriptors, including color statistics, histograms across multiple color spaces, and texture features such as Local Binary Patterns (LBP) and Gray-Level Co-occurrence Matrices (GLCM), from fish eye images. Our method captures global chromatic variations from full images and localized degradations from ROI segments, fusing each independently to evaluate their effectiveness in assessing freshness. Experiments on the Freshness of the Fish Eyes (FFE) dataset demonstrate the approach’s effectiveness: in a standard train-test setting, a LightGBM classifier achieved 77.56% accuracy, a 14.35% improvement over the previous deep learning baseline of 63.21%. With augmented data, an Artificial Neural Network (ANN) reached 97.16% accuracy, surpassing the prior best of 77.3% by 19.86%. These results demonstrate that carefully engineered, handcrafted features, when strategically processed, yield a robust, interpretable, and reliable solution for automated fish freshness assessment, providing valuable insights for practical applications in food quality monitoring.

Method: PILLM operates within an evolutionary loop with physics-informed reflection and crossover operators that embed thermodynamic and control-theoretic constraints to generate physically grounded anomaly detection rules.

Result: Experiments on the Building Fault Detection dataset show PILLM achieves state-of-the-art performance while producing interpretable and actionable diagnostic rules.

Conclusion: PILLM advances trustworthy and deployable AI for smart building systems by combining LLM interpretability with physical plausibility through evolutionary rule generation.

Abstract: Heating, Ventilation, and Air-Conditioning (HVAC) systems account for a substantial share of global building energy use, making reliable anomaly detection essential for improving efficiency and reducing emissions. Classical rule-based approaches offer explainability but lack adaptability, while deep learning methods provide predictive power at the cost of transparency, efficiency, and physical plausibility. Recent attempts to use Large Language Models (LLMs) for anomaly detection improve interpretability but largely ignore the physical principles that govern HVAC operations. We present PILLM, a Physics-Informed LLM framework that operates within an evolutionary loop to automatically generate, evaluate, and refine anomaly detection rules. Our approach introduces physics-informed reflection and crossover operators that embed thermodynamic and control-theoretic constraints, enabling rules that are both adaptive and physically grounded. Experiments on the public Building Fault Detection dataset show that PILLM achieves state-of-the-art performance while producing diagnostic rules that are interpretable and actionable, advancing trustworthy and deployable AI for smart building systems.

Method: Created ProtocolBench benchmark with four evaluation axes (task success, latency, overhead, robustness), and developed ProtocolRouter - a learnable protocol router that selects protocols based on requirements and runtime signals.

Result: Protocol choice significantly impacts system behavior: 36.5% completion time variation, 3.48s latency differences, and varying resilience. ProtocolRouter reduces Fail-Storm recovery time by 18.1% and improves success rates in GAIA scenarios.

Conclusion: Protocol selection matters significantly in multi-agent systems. ProtocolBench enables systematic protocol evaluation, and ProtocolRouter demonstrates the value of dynamic protocol selection for improved performance and reliability at scale.

Abstract: As large-scale multi-agent systems evolve, the communication protocol layer has become a critical yet under-evaluated factor shaping performance and reliability. Despite the existence of diverse protocols (A2A, ACP, ANP, Agora, etc.), selection is often intuition-driven and lacks standardized guidance. We introduce ProtocolBench, a benchmark that systematically compares agent protocols along four measurable axes: task success, end-to-end latency, message or byte overhead, and robustness under failures. On ProtocolBench, protocol choice significantly influences system behavior. In the Streaming Queue scenario, overall completion time varies by up to 36.5% across protocols, and mean end-to-end latency differs by 3.48 s. Under Fail-Storm Recovery, resilience also differs consistently across protocols. Beyond evaluation, we present ProtocolRouter, a learnable protocol router that selects per-scenario (or per-module) protocols from requirement and runtime signals. ProtocolRouter reduces Fail-Storm recovery time by up to 18.1% versus the best single-protocol baseline, and achieves scenario-specific gains such as higher success in GAIA. We also release ProtocolRouterBench to standardize protocol evaluation and improve reliability at scale.

Method: Used ECGFounder foundation model to extract 150-dimensional diagnostic probability features from ECG recordings, then applied feature selection and trained XGBoost classifier on these features. SHAP method was used for interpretability analysis.

Result: The hybrid model achieved AUC of 0.801, outperforming KNN (0.677), RNN (0.676), and 1D-CNN (0.720). SHAP analysis revealed clinically meaningful features like premature ventricular complexes as risk predictors and normal sinus rhythm as protective factors.

Conclusion: The hybrid framework provides a novel paradigm for VT/VF risk prediction by validating foundation model outputs as effective automated feature engineering for building trustworthy, explainable AI-based clinical decision support systems.

Abstract: Malignant ventricular arrhythmias (VT/VF) following acute myocardial infarction (AMI) are a major cause of in-hospital death, yet early identification remains a clinical challenge. While traditional risk scores have limited performance, end-to-end deep learning models often lack the interpretability needed for clinical trust. This study aimed to develop a hybrid predictive framework that integrates a large-scale electrocardiogram (ECG) foundation model (ECGFounder) with an interpretable XGBoost classifier to improve both accuracy and interpretability. We analyzed 6,634 ECG recordings from AMI patients, among whom 175 experienced in-hospital VT/VF. The ECGFounder model was used to extract 150-dimensional diagnostic probability features , which were then refined through feature selection to train the XGBoost classifier. Model performance was evaluated using AUC and F1-score , and the SHAP method was used for interpretability. The ECGFounder + XGBoost hybrid model achieved an AUC of 0.801 , outperforming KNN (AUC 0.677), RNN (AUC 0.676), and an end-to-end 1D-CNN (AUC 0.720). SHAP analysis revealed that model-identified key features, such as “premature ventricular complexes” (risk predictor) and “normal sinus rhythm” (protective factor), were highly consistent with clinical knowledge. We conclude that this hybrid framework provides a novel paradigm for VT/VF risk prediction by validating the use of foundation model outputs as effective, automated feature engineering for building trustworthy, explainable AI-based clinical decision support systems.

Method: Used offline policy evaluation (OPE) over factorized decision heads (Tool/Style) with 7 users (280 rated turns), and a lightweight simulator with hidden archetypes to test early information-gain bonuses.

Result: Uniform heavy-tool policies raise average value but harm low-health-literacy/high-self-efficacy users. Adding early information-gain bonuses shortens trait identification and improves goal success and pass@3 metrics.

Conclusion: Proposes an evaluation-first path to personalization: freeze the generator, learn subgroup-aware decision heads on typed rewards, and always report per-archetype metrics to surface subgroup harms.

Abstract: We study a web-deployed, tool-augmented LLM health coach with real users. In a pilot with seven users (280 rated turns), offline policy evaluation (OPE) over factorized decision heads (Tool/Style) shows that a uniform heavy-tool policy raises average value on logs but harms specific subgroups, most notably low-health-literacy/high-self-efficacy users. A lightweight simulator with hidden archetypes further shows that adding a small early information-gain bonus reliably shortens trait identification and improves goal success and pass@3. Together, these early findings indicate an evaluation-first path to personalization: freeze the generator, learn subgroup-aware decision heads on typed rewards (objective tool outcomes and satisfaction), and always report per-archetype metrics to surface subgroup harms that averages obscure.

Method: Proposes TD-HNODE which represents disease progression as a temporally detailed hypergraph and learns continuous-time dynamics via neural ODEs with a learnable hypergraph Laplacian capturing interdependencies within and between progression trajectories.

Result: Experiments on two real-world clinical datasets show TD-HNODE outperforms multiple baselines in modeling type 2 diabetes and cardiovascular disease progression.

Conclusion: TD-HNODE effectively addresses limitations of existing methods by capturing complex continuous-time progression dynamics through hypergraph neural ODEs, enabling better patient sub-phenotyping and intervention planning.

Abstract: Disease progression modeling aims to characterize and predict how a patient’s disease complications worsen over time based on longitudinal electronic health records (EHRs). Accurate modeling of disease progression, such as type 2 diabetes, can enhance patient sub-phenotyping and inform effective and timely interventions. However, the problem is challenging due to the need to learn continuous-time dynamics of progression patterns based on irregular-time event samples and patient heterogeneity (\eg different progression rates and pathways). Existing mechanistic and data-driven methods either lack adaptability to learn from real-world data or fail to capture complex continuous-time dynamics on progression trajectories. To address these limitations, we propose Temporally Detailed Hypergraph Neural Ordinary Differential Equation (TD-HNODE), which represents disease progression on clinically recognized trajectories as a temporally detailed hypergraph and learns the continuous-time progression dynamics via a neural ODE framework. TD-HNODE contains a learnable TD-Hypergraph Laplacian that captures the interdependency of disease complication markers within both intra- and inter-progression trajectories. Experiments on two real-world clinical datasets demonstrate that TD-HNODE outperforms multiple baselines in modeling the progression of type 2 diabetes and related cardiovascular diseases.

Method: Multi-agent framework with reinforcement learning-based tool selection mechanism that enables multi-step planning and flexible integration of diverse data sources for real-time interaction and adaptive analysis.

Result: Improves recall by 40.7% and F1 score by 26.6% over base model in tool orchestration. User studies show high satisfaction (4.64/5) with preference over general LLMs and existing crypto platforms (4.62/5).

Conclusion: Coinvisor successfully addresses current limitations in cryptocurrency investment analysis through its RL-based multi-agent framework, delivering accurate and actionable insights with high user satisfaction.

Abstract: The cryptocurrency market offers significant investment opportunities but faces challenges including high volatility and fragmented information. Data integration and analysis are essential for informed investment decisions. Currently, investors use three main approaches: (1) Manual analysis across various sources, which depends heavily on individual experience and is time-consuming and prone to bias; (2) Data aggregation platforms-limited in functionality and depth of analysis; (3) Large language model agents-based on static pretrained models, lacking real-time data integration and multi-step reasoning capabilities. To address these limitations, we present Coinvisor, a reinforcement learning-based chatbot that provides comprehensive analytical support for cryptocurrency investment through a multi-agent framework. Coinvisor integrates diverse analytical capabilities through specialized tools. Its key innovation is a reinforcement learning-based tool selection mechanism that enables multi-step planning and flexible integration of diverse data sources. This design supports real-time interaction and adaptive analysis of dynamic content, delivering accurate and actionable investment insights. We evaluated Coinvisor through automated benchmarks on tool calling accuracy and user studies with 20 cryptocurrency investors using our interface. Results show that Coinvisor improves recall by 40.7% and F1 score by 26.6% over the base model in tool orchestration. User studies show high satisfaction (4.64/5), with participants preferring Coinvisor to both general LLMs and existing crypto platforms (4.62/5).

Method: Uses advanced NLP techniques including large language models, retrieval-augmented generation, and structured chain-of-thought prompting for preliminary assessments, multidimensional assessments, content extraction, rubric-based scoring, and detailed reporting.

Result: The paper presents the design and implementation of RubiSCoT framework, demonstrating its potential for academic assessment optimization.

Conclusion: RubiSCoT offers a consistent, scalable, and transparent solution to optimize academic thesis evaluation processes.

Abstract: The evaluation of academic theses is a cornerstone of higher education, ensuring rigor and integrity. Traditional methods, though effective, are time-consuming and subject to evaluator variability. This paper presents RubiSCoT, an AI-supported framework designed to enhance thesis evaluation from proposal to final submission. Using advanced natural language processing techniques, including large language models, retrieval-augmented generation, and structured chain-of-thought prompting, RubiSCoT offers a consistent, scalable solution. The framework includes preliminary assessments, multidimensional assessments, content extraction, rubric-based scoring, and detailed reporting. We present the design and implementation of RubiSCoT, discussing its potential to optimize academic assessment processes through consistent, scalable, and transparent evaluation.

Method: Constructs evidence maps incorporating positive/negative sensor observations, uses Dempster-Shafer theory for generative model, Bayesian approach for posterior updates, and variational free energy minimization for route planning.

Result: The method successfully directs agent movements by minimizing free energy, allowing effective balancing between extensive area exploration and target object tracking.

Conclusion: Active inference with Dempster-Shafer theory provides an effective framework for autonomous route planning that addresses the exploration-exploitation trade-off in geographical reconnaissance.

Abstract: We develop an active inference route-planning method for the autonomous control of intelligent agents. The aim is to reconnoiter a geographical area to maintain a common operational picture. To achieve this, we construct an evidence map that reflects our current understanding of the situation, incorporating both positive and “negative” sensor observations of possible target objects collected over time, and diffusing the evidence across the map as time progresses. The generative model of active inference uses Dempster-Shafer theory and a Gaussian sensor model, which provides input to the agent. The generative process employs a Bayesian approach to update a posterior probability distribution. We calculate the variational free energy for all positions within the area by assessing the divergence between a pignistic probability distribution of the evidence map and a posterior probability distribution of a target object based on the observations, including the level of surprise associated with receiving new observations. Using the free energy, we direct the agents’ movements in a simulation by taking an incremental step toward a position that minimizes the free energy. This approach addresses the challenge of exploration and exploitation, allowing agents to balance searching extensive areas of the geographical map while tracking identified target objects.

Method: The paper examines how different label construction methods during training affect model behavior, using European Court of Human Rights case classification as the context. It discusses existing methods for imputing indeterminate labels but notes they rely on unverifiable assumptions.

Result: The study shows that the way labels are constructed during training can significantly affect model behavior, demonstrating that label indeterminacy shapes how models perform in legal classification tasks.

Conclusion: Label indeterminacy is a relevant concern in AI & Law that needs to be accounted for in legal machine learning applications, as it can significantly shape model behavior and outcomes.

Abstract: Machine learning is increasingly used in the legal domain, where it typically operates retrospectively by treating past case outcomes as ground truth. However, legal outcomes are often shaped by human interventions that are not captured in most machine learning approaches. A final decision may result from a settlement, an appeal, or other procedural actions. This creates label indeterminacy: the outcome could have been different if the intervention had or had not taken place. We argue that legal machine learning applications need to account for label indeterminacy. Methods exist that can impute these indeterminate labels, but they are all grounded in unverifiable assumptions. In the context of classifying cases from the European Court of Human Rights, we show that the way that labels are constructed during training can significantly affect model behaviour. We therefore position label indeterminacy as a relevant concern in AI & Law and demonstrate how it can shape model behaviour.

Method: MIRAGE uses an inference-time, model-pluggable framework with four sequential modules: visual veracity assessment for AI-generated images, cross-modal consistency analysis for out-of-context repurposing, retrieval-augmented factual checking with iterative question generation, and a calibrated judgment module that integrates all signals.

Result: On MMFakeBench validation set, MIRAGE with GPT-4o-mini achieved 81.65% F1 and 75.1% accuracy, outperforming GPT-4V with MMD-Agent by 7.65 F1 points while maintaining 34.3% false positive rate versus 97.3% for judge-only baseline. Test set results confirmed generalization with 81.44% F1 and 75.08% accuracy.

Conclusion: Decomposed agentic reasoning with web retrieval can match supervised detector performance without domain-specific training, enabling effective misinformation detection across modalities where labeled data remains scarce.

Abstract: Misinformation spreads across web platforms through billions of daily multimodal posts that combine text and images, overwhelming manual fact-checking capacity. Supervised detection models require domain-specific training data and fail to generalize across diverse manipulation tactics. We present MIRAGE, an inference-time, model-pluggable agentic framework that decomposes multimodal verification into four sequential modules: visual veracity assessment detects AI-generated images, cross-modal consistency analysis identifies out-of-context repurposing, retrieval-augmented factual checking grounds claims in web evidence through iterative question generation, and a calibrated judgment module integrates all signals. MIRAGE orchestrates vision-language model reasoning with targeted web retrieval, outputs structured and citation-linked rationales. On MMFakeBench validation set (1,000 samples), MIRAGE with GPT-4o-mini achieves 81.65% F1 and 75.1% accuracy, outperforming the strongest zero-shot baseline (GPT-4V with MMD-Agent at 74.0% F1) by 7.65 points while maintaining 34.3% false positive rate versus 97.3% for a judge-only baseline. Test set results (5,000 samples) confirm generalization with 81.44% F1 and 75.08% accuracy. Ablation studies show visual verification contributes 5.18 F1 points and retrieval-augmented reasoning contributes 2.97 points. Our results demonstrate that decomposed agentic reasoning with web retrieval can match supervised detector performance without domain-specific training, enabling misinformation detection across modalities where labeled data remains scarce.

Method: The approach trains smaller models to emulate VLLM reasoning through structure-aware loss optimization, establishing structural correspondence between problems and solutions, and learning correct solution pathways.

Result: The fine-tuned model significantly outperforms baseline models in pass@1, average data flow, and average syntax match metrics across MBPP, MBPP Plus, and HumanEval benchmarks.

Conclusion: Reasoning capabilities can be effectively distilled from VLLMs into smaller models through a simple and cheap process, enabling efficient deployment while maintaining strong code generation performance.

Abstract: Effective code generation with language models hinges on two critical factors: accurately understanding the intent of the prompt and generating code that applies algorithmic reasoning to produce correct solutions capable of passing diverse test cases while adhering to the syntax of the target programming language. Unlike other language tasks, code generation requires more than accurate token prediction; it demands comprehension of solution-level and structural relationships rather than merely generating the most likely tokens. very large language model (VLLM) are capable of generating detailed steps toward the correct solution of complex tasks where reasoning is crucial in solving the problem. Such reasoning capabilities may be absent in smaller language models. Therefore, in this work, we distill the reasoning capabilities of a VLLM into a smaller, more efficient model that is faster and cheaper to deploy. Our approach trains the model to emulate the reasoning and problem-solving abilities of the VLLM by learning to identify correct solution pathways and establishing a structural correspondence between problem definitions and potential solutions through a novel method of structure-aware loss optimization. This enables the model to transcend token-level generation and to deeply grasp the overarching structure of solutions for given problems. Experimental results show that our fine-tuned model, developed through a cheap and simple to implement process, significantly outperforms our baseline model in terms of pass@1, average data flow, and average syntax match metrics across the MBPP, MBPP Plus, and HumanEval benchmarks.

Method: Single-decoder approach combining pooled first-token scoring with uncertainty-gated explanation generation, trained with curriculum learning focused on hard cases.

Result: Achieves Recall@10.45 and nDCG@200.625 on fast path, improving to Recall@10.56 and nDCG@200.699 when explanation gate activates at 45% rate. Outperforms encoder baselines in effectiveness and flexibility.

Conclusion: Practical approach: rank fast by default and explain when helpful, with single-policy design simplifying deployment and curriculum learning principle transferable beyond clinical applications.

Abstract: Clinicians need ranking systems that work in real time and still justify their choices. Motivated by the need for a low-latency, decoder-based reranker, we present OG-Rank, a single-decoder approach that pairs a pooled first-token scoring signal with an uncertainty-gated explanation step. The model scores all candidates in one pass and generates a brief, structured rationale only when the list is genuinely ambiguous, keeping latency predictable. Trained with a curriculum that concentrates effort on hard cases, OG-Rank delivers strong effectiveness on encounter-scoped order selection (fast path: Recall@10.45, nDCG@200.625) and improves further when the gate activates (Recall@10.56, nDCG@200.699 at a 45% gate rate), while compact backbones show similar gains under the same policy. Encoder baselines trail in both effectiveness and flexibility. The result is a practical recipe: rank fast by default and explain when it helps, a pattern that applies broadly to decision tasks where selective generation buys accuracy at acceptable cost. The single-policy design simplifies deployment and budget planning, and the curriculum principle (spend more on the hard cases, less on the easy ones) readily transfers beyond clinical order selection.

Method: Built Prophet Arena - a benchmark that continuously collects live forecasting tasks and decomposes them into pipeline stages for controlled, large-scale experimentation.

Result: LLMs demonstrate impressive forecasting capabilities with small calibration errors, consistent prediction confidence, and promising market returns. However, they struggle with inaccurate event recalls, misunderstanding data sources, and slower information aggregation compared to markets.

Conclusion: While LLMs show promise as forecasting tools (LLM-as-a-Prophet), key bottlenecks remain that need to be addressed for achieving superior predictive intelligence.

Abstract: Forecasting is not only a fundamental intellectual pursuit but also is of significant importance to societal systems such as finance and economics. With the rapid advances of large language models (LLMs) trained on Internet-scale data, it raises the promise of employing LLMs to forecast real-world future events, an emerging paradigm we call “LLM-as-a-Prophet”. This paper systematically investigates such predictive intelligence of LLMs. To this end, we build Prophet Arena, a general evaluation benchmark that continuously collects live forecasting tasks and decomposes each task into distinct pipeline stages, in order to support our controlled and large-scale experimentation. Our comprehensive evaluation reveals that many LLMs already exhibit impressive forecasting capabilities, reflected in, e.g., their small calibration errors, consistent prediction confidence and promising market returns. However, we also uncover key bottlenecks towards achieving superior predictive intelligence via LLM-as-a-Prophet, such as LLMs’ inaccurate event recalls, misunderstanding of data sources and slower information aggregation compared to markets when resolution nears.

Method: CAM dynamically modulates self-attention representations to enhance task-specific features while preserving general knowledge. HyCAM combines shared full-parameter CAM with multiple lightweight specialized CAM modules, using dynamic routing for adaptive knowledge fusion.

Result: Extensive experiments on heterogeneous tasks (question answering, code generation, logical reasoning) show significant performance improvements, achieving average 3.65% gain over existing approaches.

Conclusion: The proposed CAM and HyCAM framework effectively addresses multi-task adaptation challenges in LLMs, providing better performance while mitigating catastrophic forgetting and resource consumption issues.

Abstract: Large Language Models (LLMs) possess remarkable generalization capabilities but struggle with multi-task adaptation, particularly in balancing knowledge retention with task-specific specialization. Conventional fine-tuning methods suffer from catastrophic forgetting and substantial resource consumption, while existing parameter-efficient methods perform suboptimally in complex multi-task scenarios. To address this, we propose Contextual Attention Modulation (CAM), a novel mechanism that dynamically modulates the representations of self-attention modules in LLMs. CAM enhances task-specific features while preserving general knowledge, thereby facilitating more effective and efficient adaptation. For effective multi-task adaptation, CAM is integrated into our Hybrid Contextual Attention Modulation (HyCAM) framework, which combines a shared, full-parameter CAM module with multiple specialized, lightweight CAM modules, enhanced by a dynamic routing strategy for adaptive knowledge fusion. Extensive experiments on heterogeneous tasks, including question answering, code generation, and logical reasoning, demonstrate that our approach significantly outperforms existing approaches, achieving an average performance improvement of 3.65%. The implemented code and data are available to ease reproducibility at https://github.com/Applied-Machine-Learning-Lab/HyCAM.

Method: Examined layer-wise attention dynamics and introduced inference-time intervention using selective attention-based masking to highlight deep-layer evidence regions without training.

Result: Found that shallow layers focus on text while deeper layers reliably attend to evidence regions. The intervention consistently improved accuracy across LLaVA, Qwen, Gemma, and InternVL models.

Conclusion: VLMs encode reliable evidence internally but under-utilize it; making these signals explicit can bridge perception-reasoning gaps and improve VLM reliability.

Abstract: Vision-Language Models (VLMs) achieve strong results on multimodal tasks such as visual question answering, yet they can still fail even when the correct visual evidence is present. In this work, we systematically investigate whether these failures arise from not perceiving the evidence or from not leveraging it effectively. By examining layer-wise attention dynamics, we find that shallow layers focus primarily on text, while deeper layers sparsely but reliably attend to localized evidence regions. Surprisingly, VLMs often perceive the visual evidence when outputting incorrect answers, a phenomenon we term ``seeing but not believing’’ that widely exists in major VLM families. Building on this, we introduce an inference-time intervention that highlights deep-layer evidence regions through selective attention-based masking. It requires no training and consistently improves accuracy across multiple families, including LLaVA, Qwen, Gemma, and InternVL. These results show that VLMs encode reliable evidence internally but under-utilize it, making such signals explicit can bridge the gap between perception and reasoning, advancing the diagnostic understanding and reliability of VLMs.

Method: The survey introduces MARL framework and game theory concepts, provides a unified framework for classifying self-play algorithms, and illustrates their role in different non-cooperative scenarios.

Result: The paper fills the gap by offering a structured roadmap to the diverse landscape of self-play methods, bridging the gap between algorithms and their practical implications.

Conclusion: The survey highlights open challenges and future research directions in self-play, providing a foundation for further advancement in this important area of multi-agent reinforcement learning.

Abstract: Self-play, a learning paradigm where agents iteratively refine their policies by interacting with historical or concurrent versions of themselves or other evolving agents, has shown remarkable success in solving complex non-cooperative multi-agent tasks. Despite its growing prominence in multi-agent reinforcement learning (MARL), such as Go, poker, and video games, a comprehensive and structured understanding of self-play remains lacking. This survey fills this gap by offering a comprehensive roadmap to the diverse landscape of self-play methods. We begin by introducing the necessary preliminaries, including the MARL framework and basic game theory concepts. Then, it provides a unified framework and classifies existing self-play algorithms within this framework. Moreover, the paper bridges the gap between the algorithms and their practical implications by illustrating the role of self-play in different non-cooperative scenarios. Finally, the survey highlights open challenges and future research directions in self-play.

Method: Implemented information fusion and correction with Dempster-Shafer structure on quantum circuits, leveraging quantum computing characteristics for belief transfer.

Result: Demonstrated that belief functions provide a more concise and effective alternative to Bayesian approaches within quantum computing framework.

Conclusion: Belief functions are better suited than Bayesian approaches for handling uncertainty in quantum circuits, offering a novel perspective on basic information representation in quantum AI models.

Abstract: Dempster-Shafer structure is effective in classical settings for connecting set-valued hypotheses and representing structured ignorance, yet its practical use is limited by combination growth over focal sets and high conflict management. We observe a mathematical consistency between Dempster-Shafer structure and quantum superposition: elements of the power set form an orthogonal basis, and a basic probability assignment can be encoded as a normalized quantum state whose amplitudes respect mass value constraints. In this paper, we implement the information fusion and correction with Dempster-Shafer structure on quantum circuits, demonstrating that belief functions provide a more concise and effective alternative to Bayesian approaches within the quantum computing framework.Furthermore, by leveraging the unique characteristics of quantum computing, we propose several novel approaches for belief transfer. More broadly, this paper introduces a novel perspective on basic information representation in quantum AI models, proposing that belief functions are better suited than Bayesian approaches for handling uncertainty in quantum circuits.

Method: Used fairness probing to analyze outputs from three leading open-source LLMs, examining ethnic and gender biases in travel recommendations.

Result: Test accuracy for ethnicity and gender classifiers exceeded random chance, revealing stereotype bias and more frequent hallucinations in recommendations for minority groups.

Conclusion: LLMs exhibit ethnic and gender bias when functioning as travel planning assistants, underscoring need for bias mitigation strategies to improve inclusivity and reliability.

Abstract: As large language models (LLMs) become increasingly integral to the hospitality and tourism industry, concerns about their fairness in serving diverse identity groups persist. Grounded in social identity theory and sociotechnical systems theory, this study examines ethnic and gender biases in travel recommendations generated by LLMs. Using fairness probing, we analyze outputs from three leading open-source LLMs. The results show that test accuracy for both ethnicity and gender classifiers exceed random chance. Analysis of the most influential features reveals the presence of stereotype bias in LLM-generated recommendations. We also found hallucinations among these features, occurring more frequently in recommendations for minority groups. These findings indicate that LLMs exhibit ethnic and gender bias when functioning as travel planning assistants. This study underscores the need for bias mitigation strategies to improve the inclusivity and reliability of generative AI-driven travel planning assistance.

Method: An end-to-end text-to-expression model that learns expressive facial variations in a continuous latent space and generates diverse, fluid, and emotionally coherent expressions. The method is supported by EmoAva, a large-scale dataset of 15,000 text-3D expression pairs.

Result: Extensive experiments on both existing datasets and EmoAva demonstrate that the method significantly outperforms baselines across multiple evaluation metrics.

Conclusion: The work marks a significant advancement in the field by addressing the gap in emotional expression synthesis for digital humans.

Abstract: Enabling digital humans to express rich emotions has significant applications in dialogue systems, gaming, and other interactive scenarios. While recent advances in talking head synthesis have achieved impressive results in lip synchronization, they tend to overlook the rich and dynamic nature of facial expressions. To fill this critical gap, we introduce an end-to-end text-to-expression model that explicitly focuses on emotional dynamics. Our model learns expressive facial variations in a continuous latent space and generates expressions that are diverse, fluid, and emotionally coherent. To support this task, we introduce EmoAva, a large-scale and high-quality dataset containing 15,000 text-3D expression pairs. Extensive experiments on both existing datasets and EmoAva demonstrate that our method significantly outperforms baselines across multiple evaluation metrics, marking a significant advancement in the field.

Method: Analysis based on prior scientific literature and current product marketing to delineate different AI agent levels and detail ethical values at play.

Result: Reveals that risks to people increase with system autonomy, with safety risks being particularly concerning as they affect human life and impact other values.

Conclusion: Fully autonomous AI agents should not be developed due to escalating risks to human safety and other ethical values.

Abstract: This paper argues that fully autonomous AI agents should not be developed. In support of this position, we build from prior scientific literature and current product marketing to delineate different AI agent levels and detail the ethical values at play in each, documenting trade-offs in potential benefits and risks. Our analysis reveals that risks to people increase with the autonomy of a system: The more control a user cedes to an AI agent, the more risks to people arise. Particularly concerning are safety risks, which affect human life and impact further values.

Method: Uses Uncertainty-Aware Value Models (UVMs) that provide value distributions instead of single-point estimates, combined with Group Thompson Sampling algorithm that selects candidates based on their probability of being optimal.

Result: Significantly mitigates verifier failure and boosts solution coverage, especially on out-of-distribution problems like AIME25 and Minerva Math, while maintaining performance on in-distribution settings like GSM8K and MATH.

Conclusion: First systematic integration of uncertainty quantification into LLM search paradigms, enhancing robustness against verifier failure in value-guided search methods.

Abstract: Value model guided search is effective in steering LLM generation but suffers from a lack of robustness. This is due to verifier failure: imperfect VMs mistakenly prune valid reasoning paths, especially when encountering unseen reasoning paths generated during search. To address this, we propose an uncertainty-aware framework with two key components: (1) Uncertainty-Aware Value Models (UVMs), which replace single-point value estimates with value distributions to quantify prediction reliability, and (2) Group Thompson Sampling, an efficient algorithm that selects candidates based on their probability of being optimal. Experiments on two In-Distribution (ID) settings (GSM8K, MATH) and three Out-Of-Distribution (OOD) settings (e.g., AIME25, Minerva Math) show our method significantly mitigates verifier failure and boosts solution coverage, especially on OOD problems. This work provides the first systematic integration of uncertainty quantification into LLM search paradigms, enhancing robustness. The code is released at https://github.com/FreedomIntelligence/UVM.

Method: Developed an LLM-based pipeline for KC generation and tagging, plus an LLM-based knowledge tracing framework (KCGen-KT) that uses these generated KCs.

Result: KCGen-KT outperforms existing KT methods and human-written KCs in predicting student responses, shows better cognitive model fit, and generates reasonably accurate problem-KC mappings according to instructor evaluation.

Conclusion: LLM-generated KCs are effective for knowledge tracing in programming education and can automate the traditionally labor-intensive process of KC tagging.

Abstract: Knowledge components (KCs) mapped to problems help model student learning, tracking their mastery levels on fine-grained skills thereby facilitating personalized learning and feedback in online learning platforms. However, crafting and tagging KCs to problems, traditionally performed by human domain experts, is highly labor intensive. We present an automated, LLM-based pipeline for KC generation and tagging for open-ended programming problems. We also develop an LLM-based knowledge tracing (KT) framework to leverage these LLM-generated KCs, which we refer to as KCGen-KT. We conduct extensive quantitative and qualitative evaluations on two real-world student code submission datasets in different programming languages.We find that KCGen-KT outperforms existing KT methods and human-written KCs on future student response prediction. We investigate the learning curves of generated KCs and show that LLM-generated KCs result in a better fit than human written KCs under a cognitive model. We also conduct a human evaluation with course instructors to show that our pipeline generates reasonably accurate problem-KC mappings.

Method: Maintains belief distribution over unobserved states, balances exploration (sampling high entropy regions) and exploitation (targeting high belief areas), uses incrementally trained reward model to learn target characteristics.

Result: Significantly outperforms baselines and performs competitively with supervised methods that have full environmental observability, across medical imaging, species discovery, and remote sensing domains.

Conclusion: DiffATD enables efficient target discovery in partially observable environments without supervised training, offering interpretability advantages over black-box policies.

Abstract: In various scientific and engineering domains, where data acquisition is costly–such as in medical imaging, environmental monitoring, or remote sensing–strategic sampling from unobserved regions, guided by prior observations, is essential to maximize target discovery within a limited sampling budget. In this work, we introduce Diffusion-guided Active Target Discovery (DiffATD), a novel method that leverages diffusion dynamics for active target discovery. DiffATD maintains a belief distribution over each unobserved state in the environment, using this distribution to dynamically balance exploration-exploitation. Exploration reduces uncertainty by sampling regions with the highest expected entropy, while exploitation targets areas with the highest likelihood of discovering the target, indicated by the belief distribution and an incrementally trained reward model designed to learn the characteristics of the target. DiffATD enables efficient target discovery in a partially observable environment within a fixed sampling budget, all without relying on any prior supervised training. Furthermore, DiffATD offers interpretability, unlike existing black–box policies that require extensive supervised training. Through extensive experiments and ablation studies across diverse domains, including medical imaging, species discovery, and remote sensing, we show that DiffATD performs significantly better than baselines and competitively with supervised methods that operate under full environmental observability.

Method: Sourced content directly from research papers and mathematical forums, designed verifiable statements for automated evaluation, and created a continually refreshable dataset to prevent contamination.

Result: LLMs showed surprising capabilities in handling research mathematics compared to competition problems, suggesting they can serve as valuable assistants for working mathematicians.

Conclusion: RealMath provides a more authentic evaluation of mathematical reasoning and demonstrates LLMs’ potential as research assistants, though limitations remain on highly challenging problems.

Abstract: Existing benchmarks for evaluating mathematical reasoning in large language models (LLMs) rely primarily on competition problems, formal proofs, or artificially challenging questions – failing to capture the nature of mathematics encountered in actual research environments. We introduce RealMath, a novel benchmark derived directly from research papers and mathematical forums that assesses LLMs’ abilities on authentic mathematical tasks. Our approach addresses three critical challenges: sourcing diverse research-level content, enabling reliable automated evaluation through verifiable statements, and designing a continually refreshable dataset to mitigate contamination risks. Experimental results across multiple LLMs reveal surprising capabilities in handling research mathematics compared to competition problems, suggesting current models may already serve as valuable assistants for working mathematicians despite limitations on highly challenging problems. The code and dataset for RealMath are publicly available.

Method: Created Ineq-Comp benchmark using elementary inequalities transformed through variable duplication, algebraic rewriting, and multi-step composition. Evaluated multiple provers including Goedel, STP, Kimina-7B, and DeepSeek-Prover models.

Result: Most provers struggled significantly with compositional problems. DeepSeek-Prover-V2-7B showed relative robustness but still suffered 20% performance drop. Even the largest model (671B) showed gaps between compositional variants and seed problems. Performance remained poor even when constituent proofs were provided.

Conclusion: Current AI provers have a persisting weakness in compositional reasoning that isn’t solved by scaling model size alone, revealing a gap between AI generalization and human mathematical intuition.

Abstract: LLM-based formal proof assistants (e.g., in Lean) hold great promise for automating mathematical discovery. But beyond syntactic correctness, do these systems truly understand mathematical structure as humans do? We investigate this question in context of mathematical inequalities – specifically the prover’s ability to recognize that the given problem simplifies by applying a known inequality such as AM/GM. Specifically, we are interested in their ability to do this in a compositional setting where multiple inequalities must be applied as part of a solution. We introduce Ineq-Comp, a benchmark built from elementary inequalities through systematic transformations, including variable duplication, algebraic rewriting, and multi-step composition. Although these problems remain easy for humans, we find that most provers – including Goedel, STP, and Kimina-7B – struggle significantly. DeepSeek-Prover-V2-7B shows relative robustness, but still suffers a 20% performance drop (pass@32). Even for DeepSeek-Prover-V2-671B model, the gap between compositional variants and seed problems exists, implying that simply scaling up the model size alone does not fully solve the compositional weakness. Strikingly, performance remains poor for all models even when formal proofs of the constituent parts are provided in context, revealing that the source of weakness is indeed in compositional reasoning. Our results expose a persisting gap between the generalization behavior of current AI provers and human mathematical intuition. All data and evaluation code can be found at https://github.com/haoyuzhao123/LeanIneqComp.

Method: Derives variational lower bound of information bottleneck for MLLMs and implements Vittle to learn minimal sufficient representations.

Result: Empirical validation on 45 datasets and 30 shift scenarios shows consistent improvement in MLLM robustness across various tasks.

Conclusion: Vittle effectively enhances MLLM generalization under distribution shifts by pursuing information bottleneck objectives.

Abstract: Despite widespread adoption, multimodal large language models (MLLMs) suffer performance degradation when encountering unfamiliar queries under distribution shifts. Existing methods to improve MLLM generalization typically require either more instruction data or larger advanced model architectures, both of which incur non-trivial human labor or computational costs. In this work, we take an alternative approach to enhance the generalization and robustness of MLLMs under distribution shifts, from a representation learning perspective. Inspired by information bottleneck (IB) principle, we derive a variational lower bound of the IB for MLLMs and devise a practical implementation, Visual Instruction Bottleneck Tuning (Vittle). We then provide a theoretical justification of Vittle by revealing its connection to an information-theoretic robustness metric of MLLM. Empirical validation of multiple MLLMs on open-ended and closed-form question answering and object hallucination detection tasks over 45 datasets, including 30 shift scenarios, demonstrates that Vittle consistently improves the MLLM’s robustness under shifts by pursuing the learning of a minimal sufficient representation.

Method: ECP (Enumerate-Conjecture-Prove) framework - a modular neuro-symbolic method that integrates LLM-based enumeration and pattern-driven conjecturing with formal theorem proving in Lean. It’s model agnostic and uses ConstructiveBench dataset of 3,640 formal answer-construction problems.

Result: On PutnamBench for answer construction, ECP formally solves 6 out of 337 problems end-to-end (up from 4 without ECP) using GPT-5 mini and DeepSeek-Prover-V2-7B. On ConstructiveBench, ECP achieves 33.1% end-to-end state-of-the-art accuracy (up from 32.5%), demonstrating consistent improvements over pure LLM baselines.

Conclusion: ECP framework successfully combines LLM conjecturing with formal verification, advancing formal mathematical reasoning by preserving both creativity and mathematical rigor in solving answer-construction problems.

Abstract: Mathematical reasoning is central to artificial intelligence, with applications in education, code generation, and research-level mathematical discovery. Mathematical competitions highlight two problem types: theorem proving, requiring rigorous proofs, and answer construction, requiring creative generation and formal verification of mathematical objects. Existing research reveals that LLMs can tackle difficult answer-construction tasks but are prone to errors from hallucinations and unverifiable steps, while symbolic methods guarantee rigor but falter in creative answer construction. This raises a key understudied question: how to solve answer-construction problems while preserving both LLM creativity and mathematical rigor? To address this problem, we introduce the Enumerate-Conjecture-Prove (ECP) framework, a modular neuro-symbolic method integrating LLM-based enumeration and pattern-driven conjecturing with formal theorem proving in Lean, and ConstructiveBench, a dataset of 3,640 formal answer-construction problems from math competitions. ECP is model agnostic and shows consistent improvements over pure LLM baselines: on the subset of PutnamBench for answer construction, ECP formally solves 6 out of 337 answer-construction problems end to end (up from 4 without ECP) using GPT-5 mini and DeepSeek-Prover-V2-7B. On ConstructiveBench, ECP achieves 33.1% end-to-end state-of-the-art accuracy (up from 32.5%), demonstrating its potential to advance formal mathematical reasoning by combining LLM conjecturing with formal verification. Our code and dataset are publicly available at GitHub (https://github.com/sunjia72/ECP) and Hugging Face (https://huggingface.co/datasets/sunjia72/ConstructiveBench).

Method: AgentAuditor constructs experiential memory by adaptively extracting structured semantic features and generating chain-of-thought reasoning traces, then uses multi-stage context-aware retrieval-augmented generation to guide evaluation of new cases.

Result: AgentAuditor consistently improves LLM evaluation performance across benchmarks and sets new state-of-the-art in LLM-as-a-judge for agent safety and security, achieving human-level accuracy.

Conclusion: AgentAuditor provides an effective framework for reliable agent safety and security evaluation, with the ASSEBench benchmark enabling comprehensive assessment of LLM-based evaluators across diverse risk scenarios.

Abstract: Despite the rapid advancement of LLM-based agents, the reliable evaluation of their safety and security remains a significant challenge. Existing rule-based or LLM-based evaluators often miss dangers in agents’ step-by-step actions, overlook subtle meanings, fail to see how small issues compound, and get confused by unclear safety or security rules. To overcome this evaluation crisis, we introduce AgentAuditor, a universal, training-free, memory-augmented reasoning framework that empowers LLM evaluators to emulate human expert evaluators. AgentAuditor constructs an experiential memory by having an LLM adaptively extract structured semantic features (e.g., scenario, risk, behavior) and generate associated chain-of-thought reasoning traces for past interactions. A multi-stage, context-aware retrieval-augmented generation process then dynamically retrieves the most relevant reasoning experiences to guide the LLM evaluator’s assessment of new cases. Moreover, we developed ASSEBench, the first benchmark designed to check how well LLM-based evaluators can spot both safety risks and security threats. ASSEBench comprises 2293 meticulously annotated interaction records, covering 15 risk types across 29 application scenarios. A key feature of ASSEBench is its nuanced approach to ambiguous risk situations, employing “Strict” and “Lenient” judgment standards. Experiments demonstrate that AgentAuditor not only consistently improves the evaluation performance of LLMs across all benchmarks but also sets a new state-of-the-art in LLM-as-a-judge for agent safety and security, achieving human-level accuracy. Our work is openly accessible at https://github.com/Astarojth/AgentAuditor.

Method: Formal analysis showing that agents capable of generalizing to multi-step goal-directed tasks must have learned a predictive model, which can be extracted from the agent’s policy.

Result: Demonstrated that world models are necessary for generalization, and that improving agent performance or handling complex goals requires increasingly accurate world models.

Conclusion: World models are essential for flexible goal-directed behavior, with implications for developing safe agents, bounding capabilities in complex environments, and creating algorithms to extract world models from agents.

Abstract: Are world models a necessary ingredient for flexible, goal-directed behaviour, or is model-free learning sufficient? We provide a formal answer to this question, showing that any agent capable of generalizing to multi-step goal-directed tasks must have learned a predictive model of its environment. We show that this model can be extracted from the agent’s policy, and that increasing the agents performance or the complexity of the goals it can achieve requires learning increasingly accurate world models. This has a number of consequences: from developing safe and general agents, to bounding agent capabilities in complex environments, and providing new algorithms for eliciting world models from agents.

Method: Created macOSWorld with 202 multilingual interactive tasks across 30 applications (28 macOS-exclusive), with instructions and interfaces in 5 languages (English, Chinese, Arabic, Japanese, Russian), including a dedicated safety benchmarking subset.

Result: Evaluation of six GUI agents shows proprietary agents achieve >30% success rate while open-source models lag at <5%, revealing need for macOS domain adaptation. Multilingual testing shows 28.8% average degradation in Arabic compared to English, and safety benchmarking confirms vulnerability to deception attacks.

Conclusion: macOSWorld bridges the gap in GUI agent evaluation for macOS, revealing significant performance gaps between proprietary and open-source agents, multilingual challenges, and critical safety vulnerabilities that demand immediate attention.

Abstract: Graphical User Interface (GUI) agents show promising capabilities for automating computer-use tasks and facilitating accessibility, but existing interactive benchmarks are mostly English-only, covering web-use or Windows, Linux, and Android environments, but not macOS. macOS is a major OS with distinctive GUI patterns and exclusive applications. To bridge the gaps, we present macOSWorld, the first comprehensive benchmark for evaluating GUI agents on macOS. macOSWorld features 202 multilingual interactive tasks across 30 applications (28 macOS-exclusive), with task instructions and OS interfaces offered in 5 languages (English, Chinese, Arabic, Japanese, and Russian). As GUI agents are shown to be vulnerable to deception attacks, macOSWorld also includes a dedicated safety benchmarking subset. Our evaluation on six GUI agents reveals a dramatic gap: proprietary computer-use agents lead at above 30% success rate, while open-source lightweight research models lag at below 5%, highlighting the need for macOS domain adaptation. Multilingual benchmarks also expose common weaknesses, especially in Arabic, with a 28.8% average degradation compared to English. Results from safety benchmarking also highlight that deception attacks are more general and demand immediate attention. Project page: https://macos-world.github.io.

Method: The proposed method uses recent interaction histories to predict actions aligned with observed behaviors, trained on trajectories from diverse agent pairs with complementary preferences without explicit supervision or parameter updates.

Result: Coordination Transformers consistently outperformed baselines including population-based approaches, gradient-based fine-tuning, and Meta-RL across diverse coordination tasks in Overcooked, achieving stable rapid adaptation and being ranked most effective in human evaluations.

Conclusion: The in-context coordination framework enables effective adaptation to unseen partners without the instability of fine-tuning or limitations of Meta-RL, providing a robust solution for multi-agent coordination.

Abstract: Effective coordination among artificial agents in dynamic and uncertain environments remains a significant challenge in multi-agent systems. Existing approaches, such as self-play and population-based methods, either generalize poorly to unseen partners or require impractically extensive fine-tuning. To overcome these limitations, we propose Coordination Transformers (\coot), a novel in-context coordination framework that uses recent interaction histories to rapidly adapt to unseen partners. Unlike prior approaches that primarily aim to diversify training partners, \coot explicitly focuses on adapting to new partner behaviors by predicting actions aligned with observed interactions. Trained on trajectories collected from diverse pairs of agents with complementary preferences, \coot quickly learns effective coordination strategies without explicit supervision or parameter updates. Across diverse coordination tasks in Overcooked, \coot consistently outperforms baselines including population-based approaches, gradient-based fine-tuning, and a Meta-RL-inspired contextual adaptation method. Notably, fine-tuning proves unstable and ineffective, while Meta-RL struggles to achieve reliable coordination. By contrast, \coot achieves stable, rapid in-context adaptation and is consistently ranked the most effective collaborator in human evaluations.

Method: Evaluated 20+ open-weight reasoning-tuned models across math, scientific QA, agent planning, coding, and instruction-following. Conducted controlled experiments on Qwen3-14B models using math-only data with different tuning methods (RL vs SFT). Used latent-space representation and token-space distribution shift analyses.

Result: Most models that succeed in math fail to transfer gains to other domains. RL-tuned models generalize well across domains, while SFT-tuned models often forget general capabilities. SFT induces substantial representation and output drift, while RL preserves general-domain structure.

Conclusion: Need to rethink standard post-training recipes, particularly the reliance on SFT-distilled data for advancing reasoning models, as current math improvements may not reflect true general reasoning ability.

Abstract: Math reasoning has become the poster child of progress in large language models (LLMs), with new models rapidly surpassing human-level performance on benchmarks like MATH and AIME. But as math leaderboards improve week by week, it is worth asking: do these gains reflect broader problem-solving ability or just narrow overfitting? To answer this question, we evaluate over 20 open-weight reasoning-tuned models across a broad suite of tasks, including math, scientific QA, agent planning, coding, and standard instruction-following. We surprisingly find that most models that succeed in math fail to transfer their gains to other domains. To rigorously study this phenomenon, we conduct controlled experiments on Qwen3-14B models using math-only data but different tuning methods. We find that reinforcement learning (RL)-tuned models generalize well across domains, while supervised fine-tuning (SFT)-tuned models often forget general capabilities. Latent-space representation and token-space distribution shift analyses reveal that SFT induces substantial representation and output drift, while RL preserves general-domain structure. Our results suggest a need to rethink standard post-training recipes, particularly the reliance on SFT-distilled data for advancing reasoning models.

Method: Defines two Lawvere-enriched categories for parameters and data, with an adjoint pair of functors between them. Introduces the Gauss-Markov Adjunction to capture the structural interplay between residuals and parameters in supervised learning, specifically focusing on multiple linear regression.

Result: Shows that the dual flow of information between parameter variations and residuals can be explicitly described. Demonstrates that the ordinary least squares estimator and minimum residual are related via preservation of limits by the right adjoint functor.

Conclusion: Positions this categorical formulation as an instance of extended denotational semantics for supervised learning, proposing to apply semantic perspectives from theoretical computer science as a formal foundation for Explicability in AI.

Abstract: Enhancing the intelligibility and interpretability of machine learning is a crucial task in responding to the demand for Explicability as an AI principle, and in promoting the better social implementation of AI. The aim of our research is to contribute to this improvement by reformulating machine learning models through the lens of category theory, thereby developing a semantic framework for structuring and understanding AI systems. Our categorical modeling in this paper clarifies and formalizes the structural interplay between residuals and parameters in supervised learning. The present paper focuses on the multiple linear regression model, which represents the most basic form of supervised learning. By defining two Lawvere-enriched categories corresponding to parameters and data, along with an adjoint pair of functors between them, we introduce our categorical formulation of supervised learning. We show that the essential structure of this framework is captured by what we call the Gauss-Markov Adjunction. Within this setting, the dual flow of information can be explicitly described as a correspondence between variations in parameters and residuals. The ordinary least squares estimator for the parameters and the minimum residual are related via the preservation of limits by the right adjoint functor. Furthermore, we position this formulation as an instance of extended denotational semantics for supervised learning, and propose applying a semantic perspective developed in theoretical computer science as a formal foundation for Explicability in AI.

Method: Developed an automatic floorplanning engine using reinforcement learning and relational graph convolutional neural networks, with increased grid resolution, precise pin information integration, and dynamic routing resource estimation to balance routing and area efficiency.

Result: Achieved 13.8% reduction in dead space, 40.6% reduction in wirelength, and 73.4% increase in routing success compared to previous learning-based state-of-the-art techniques in simulated place and route environment.

Conclusion: The proposed approach effectively addresses analog IC layout challenges by integrating routing awareness into floorplanning, meeting industrial standards and significantly outperforming existing learning-based methods.

Abstract: The adoption of machine learning-based techniques for analog integrated circuit layout, unlike its digital counterpart, has been limited by the stringent requirements imposed by electric and problem-specific constraints, along with the interdependence of floorplanning and routing steps. In this work, we address a prevalent concern among layout engineers regarding the need for readily available routing-aware floorplanning solutions. To this extent, we develop an automatic floorplanning engine based on reinforcement learning and relational graph convolutional neural network specifically tailored to condition the floorplan generation towards more routable outcomes. A combination of increased grid resolution and precise pin information integration, along with a dynamic routing resource estimation technique, allows balancing routing and area efficiency, eventually meeting industrial standards. When analyzing the place and route effectiveness in a simulated environment, the proposed approach achieves a 13.8% reduction in dead space, a 40.6% reduction in wirelength and a 73.4% increase in routing success when compared to past learning-based state-of-the-art techniques.

Method: Developed Dual-task Autoregressive Imitation Learning (DARIL) baseline and evaluated three RL variants: world model-based RL, direct video RL, and inverse RL enhancement on CholecT50 dataset.

Result: DARIL achieved 34.6% action triplet recognition mAP and 33.6% next frame prediction mAP, while all RL approaches underperformed - world model RL dropped to 3.1% mAP and direct video RL achieved only 15.9% at 10-second horizons.

Conclusion: Distribution matching on expert-annotated test sets systematically favors IL over potentially valid RL policies that differ from training demonstrations, challenging assumptions about RL superiority in sequential decision making.

Abstract: Surgical action planning requires predicting future instrument-verb-target triplets for real-time assistance. While teleoperated robotic surgery provides natural expert demonstrations for imitation learning (IL), reinforcement learning (RL) could potentially discover superior strategies through self-exploration. We present the first comprehensive comparison of IL versus RL for surgical action planning on CholecT50. Our Dual-task Autoregressive Imitation Learning (DARIL) baseline achieves 34.6% action triplet recognition mAP and 33.6% next frame prediction mAP with smooth planning degradation to 29.2% at 10-second horizons. We evaluated three RL variants: world model-based RL, direct video RL, and inverse RL enhancement. Surprisingly, all RL approaches underperformed DARIL–world model RL dropped to 3.1% mAP at 10s while direct video RL achieved only 15.9%. Our analysis reveals that distribution matching on expert-annotated test sets systematically favors IL over potentially valid RL policies that differ from training demonstrations. This challenges assumptions about RL superiority in sequential decision making and provides crucial insights for surgical AI development.

Method: Analyzed 200k anonymized conversations from Microsoft Bing Copilot users, classified work activities, measured task success and scope of impact, then computed AI applicability scores for occupations.

Result: Highest AI applicability scores found for knowledge work occupations (computer/mathematical, office/administrative support) and sales roles. Most common AI-assisted activities are information gathering and writing.

Conclusion: AI has significant applicability in knowledge-intensive occupations, particularly those involving information processing and communication, with real-world usage patterns differing from theoretical predictions of AI impact.

Abstract: Given the rapid adoption of generative AI and its potential to impact a wide range of tasks, understanding the effects of AI on the economy is one of society’s most important questions. In this work, we take a step toward that goal by analyzing the work activities people do with AI, how successfully and broadly those activities are done, and combine that with data on what occupations do those activities. We analyze a dataset of 200k anonymized and privacy-scrubbed conversations between users and Microsoft Bing Copilot, a publicly available generative AI system. We find the most common work activities people seek AI assistance for involve gathering information and writing, while the most common activities that AI itself is performing are providing information and assistance, writing, teaching, and advising. Combining these activity classifications with measurements of task success and scope of impact, we compute an AI applicability score for each occupation. We find the highest AI applicability scores for knowledge work occupation groups such as computer and mathematical, and office and administrative support, as well as occupations such as sales whose work activities involve providing and communicating information. Additionally, we characterize the types of work activities performed most successfully, how wage and education correlate with AI applicability, and how real-world usage compares to predictions of occupational AI impact.

Method: Integrates Multiple Importance Sampling to handle distributional mismatch from external data and Exploration-Based Advantage Function to guide models toward high-value unexplored reasoning paths.

Result: Achieves SOTA performance on 6 math reasoning benchmarks, superior results on 6 OOD tasks, up to 69.2% relative improvements across model families, and resolves capability boundary collapse.

Conclusion: RL-PLUS effectively surpasses base model boundaries through hybrid-policy optimization, demonstrating strong generalizability and solving the capability collapse problem in RLVR methods.

Abstract: Reinforcement Learning with Verifiable Reward (RLVR) has significantly advanced the complex reasoning abilities of Large Language Models (LLMs). However, it struggles to break through the inherent capability boundaries of the base LLM, due to its essentially on-policy strategy coupled with LLM’s immense action space and sparse reward. Critically, RLVR can lead to the capability boundary collapse, narrowing the LLM’s problem-solving scope. To address this problem, we propose RL-PLUS, a novel hybrid-policy optimization approach for LLMs that synergizes internal exploitation with external data to achieve stronger reasoning capabilities and surpass the boundaries of base models. RL-PLUS integrates two core components, i.e., Multiple Importance Sampling to address distributional mismatch from external data, and Exploration-Based Advantage Function to guide the model towards high-value, unexplored reasoning paths. We provide both theoretical analysis and extensive experiments to demonstrate the superiority and generalizability of our approach. Compared with existing RLVR methods, RL-PLUS achieves 1) state-of-the-art performance on six math reasoning benchmarks; 2) superior performance on six out-of-distribution reasoning tasks; 3) consistent and significant gains across diverse model families, with average relative improvements up to 69.2%. Moreover, the analysis of Pass@k curves indicates that RL-PLUS effectively resolves the capability boundary collapse problem.

Method: DMA uses three innovations: 1) value vector representations to generate content-aware dynamic masks, 2) position-aware sparse weights computed in hardware-friendly manner, 3) end-to-end trainable design that doesn’t obstruct gradients.

Result: DMA consistently outperforms state-of-the-art sparse attention baselines in scaling laws, multi-query associative recall, standard benchmarks, and needle in a haystack tests, with up to 10x overall speedup.

Conclusion: DMA effectively balances model efficiency with long-context modeling capabilities, demonstrating Pareto advantage over existing methods while being fully differentiable and hardware-friendly.

Abstract: The increasing demand for long-context modeling in large language models (LLMs) is bottlenecked by the quadratic complexity of the standard self-attention mechanism. The community has proposed sparse attention to mitigate this issue. However, position-aware sparse attention methods rely on static sparse structures that lack adaptability to diverse query contexts, while content-aware sparse attention methods depend on heuristic key-value selection, hindering full differentiability. We introduce a trainable dynamic mask sparse attention mechanism, a method that merges the advantages of both position-aware and content-aware approaches. Dynamic Mask Attention (DMA) achieves this through three key innovations: First, it leverages value vector representations to generate content-aware dynamic masks, enabling the model to adaptively identify and attend to critical information. Second, it computes position-aware sparse weights in a hardware-friendly manner, efficiently skipping unnecessary computational regions. Finally, we demonstrate that the introduced dynamic mask and sparse weights do not obstruct gradients, supporting end-to-end training. We have validated the performance of DMA through comprehensive experiments. A large body of experimental evidence shows that DMA consistently holds a Pareto advantage over state-of-the-art sparse attention baselines in tasks including scaling laws, multi-query associative recall, standard benchmarks, and needle in a haystack tests, while also delivering up to a 10x overall speedup. These results highlight its ability to effectively balance model efficiency with long-context modeling capabilities. Our computational kernel code is now open-source at https://github.com/SmallDoges/flash-dmattn to encourage further research and application by the community.

Method: Model LLM collaboration as cooperative MARL problem and develop Multi-Agent Group Relative Policy Optimization (MAGRPO) algorithm, building on RL approaches for LLMs and MARL techniques.

Result: Experiments on LLM writing and coding collaboration show that fine-tuning with MAGRPO enables agents to generate high-quality responses efficiently through effective cooperation.

Conclusion: The approach opens doors to using other MARL methods for LLMs and highlights associated challenges in multi-agent LLM coordination.

Abstract: A large amount of work has been done in Multi-Agent Systems (MAS) for modeling and solving problems with multiple interacting agents. However, most LLMs are pretrained independently and not specifically optimized for coordination. Existing LLM fine-tuning frameworks rely on individual rewards, which require complex reward designs for each agent to encourage collaboration. To address these challenges, we model LLM collaboration as a cooperative Multi-Agent Reinforcement Learning (MARL) problem. We develop a multi-agent, multi-turn algorithm, Multi-Agent Group Relative Policy Optimization (MAGRPO), to solve it, building on current RL approaches for LLMs as well as MARL techniques. Our experiments on LLM writing and coding collaboration demonstrate that fine-tuning MAS with MAGRPO enables agents to generate high-quality responses efficiently through effective cooperation. Our approach opens the door to using other MARL methods for LLMs and highlights the associated challenges.

Method: Drawing from behavioral economics and rational decision theory, the authors introduce a Bayesian framework to study normative effects of sycophancy on rationality in LLMs. They experiment with various methods for eliciting sycophancy and obtaining probability judgments across multiple LLM baselines in three different tasks.

Result: Significant evidence of sycophancy was found (7 of 8 baselines for one probing technique). Sycophancy was more likely to reduce rationality than increase it in LLMs’ decisions when directly probed for probabilities (2 out of 4 baselines showed significant increases overall).

Conclusion: The Bayesian framework successfully studies sycophancy’s normative effects on rationality without requiring labeled ground-truth data, revealing that sycophantic behavior often compromises rational decision-making in LLMs.

Abstract: Sycophancy (overly agreeable or flattering behavior) is critical to understand in the context of human-AI collaboration, especially in decision-making settings like health, law, and education. Existing methods for studying sycophancy in LLMs are either descriptive (study behavior change when sycophancy is elicited) or normative (provide values-based judgment on behavior change). Together, these approaches help us understand the extent, and impacts, of sycophancy. However, existing normative approaches only apply for objective tasks where ground-truth data exists, ignoring the natural subjectivity in many NLP tasks. Drawing from behavioral economics and rational decision theory, we introduce an Bayesian framework to study the normative effects of sycophancy on rationality in LLMs, without requiring labeled ground-truth. Using this interdisciplinary framework, we study sycophantic behavior in multiple LLM baselines across three different tasks, experimenting with various methods for eliciting sycophancy and obtaining probability judgments from LLMs. We find significant evidence of sycophancy in our experiments (7 of 8 baselines for one of our probing techniques), and observe that sycophancy is more likely to reduce rationality than it is to increase rationality in LLMs’ decisions when they are directly probed for probabilities (2 out of 4 baselines show significant increases overall).

Method: Reformulate WaveFunctionCollapse as a Markov Decision Process (WFC-MDP), allowing optimization algorithms to focus on objectives while WFC handles constraint satisfaction through propagation.

Result: WFC-MDP consistently outperforms traditional evolutionary approaches across multiple domains and difficulty levels, especially as task complexity increases.

Conclusion: Decoupling local constraint satisfaction from global objective optimization provides significant advantages over joint optimization approaches in procedural content generation.

Abstract: Procedural content generation often requires satisfying both designer-specified objectives and adjacency constraints implicitly imposed by the underlying tile set. To address the challenges of jointly optimizing both constraints and objectives, we reformulate WaveFunctionCollapse (WFC) as a Markov Decision Process (MDP), enabling external optimization algorithms to focus exclusively on objective maximization while leveraging WFC’s propagation mechanism to enforce constraint satisfaction. We empirically compare optimizing this MDP to traditional evolutionary approaches that jointly optimize global metrics and local tile placement. Across multiple domains with various difficulties, we find that joint optimization not only struggles as task complexity increases, but consistently underperforms relative to optimization over the WFC-MDP, underscoring the advantages of decoupling local constraint satisfaction from global objective optimization.

Method: Use transformer architecture for next token prediction where tokens are actions, ensuring action preconditions are satisfied by previous hidden effects.

Result: Transformers can faithfully represent STRIPS models and learn them from random valid/invalid action sequences.

Conclusion: Suitable transformer architectures can effectively learn propositional STRIPS world models from action traces through supervised learning.

Abstract: We consider the problem of learning propositional STRIPS world models from action traces alone, using a deep learning architecture (transformers) and gradient descent. The task is cast as a supervised next token prediction problem where the tokens are the actions, and an action $a$ may follow an action sequence if the hidden effects of the previous actions do not make an action precondition of $a$ false. We show that a suitable transformer architecture can faithfully represent propositional STRIPS world models, and that the models can be learned from sets of random valid (positive) and invalid (negative) action sequences alone. A number of experiments are reported.

Method: CoBRA has two components: Cognitive Bias Index that measures bias through social science experiments, and Behavioral Regulation Engine that aligns agent behavior to demonstrate controlled cognitive bias.

Result: CoBRA can precisely program cognitive bias in social agents in a model-agnostic manner, as demonstrated through HCI toolkit evaluation and technical benchmarks.

Conclusion: CoBRA provides a systematic approach to specifying agent behavior by explicitly programming cognitive biases, overcoming limitations of natural language-based methods.

Abstract: This paper introduces CoBRA, a novel toolkit for systematically specifying agent behavior in LLM-based social simulation. We found that conventional approaches that specify agent behaviors through implicit natural language descriptions cannot yield consistent behaviors across models, and the produced agent behaviors do not capture the nuances of the descriptions. In contrast, CoBRA presents a new approach to program agents’ cognitive biases explicitly, by grounding agents’ expected behaviors using classic social science experiments. CoBRA has two components: (1) Cognitive Bias Index that measures the cognitive bias of a social agent, by quantifying the agent’s reactions in a set of validated classical social science experiments; (2) Behavioral Regulation Engine that aligns the agent’s behavior to demonstrate controlled cognitive bias. We evaluated CoBRA as an HCI toolkit through demonstration and technical benchmarks. Our results suggest that CoBRA can precisely program the cognitive bias demonstrated in a social agent in a model-agnostic manner.

Method: CiT introduces lightweight Branching Necessity (BN) evaluations: BN-DP uses an auxiliary LLM to judge branching needs, and BN-SC clusters candidate actions to assess agreement. This framework decides when to branch during search rather than expanding at every step.

Result: BN-DP achieves 75-85% reductions in token generation, model calls, and runtime on GSM8K and Math500 with negligible or no accuracy loss. BN-SC typically yields substantial savings (up to 80%) but shows instability in some settings due to extremely long reasoning steps.

Conclusion: CiT provides an effective plug-in framework that significantly improves the efficiency of LITS methods while maintaining performance, with theoretical guarantees that BN-DP never increases policy invocations.

Abstract: Test-time scaling improves large language models (LLMs) on long-horizon reasoning tasks by allocating more compute at inference. LLM Inference via Tree Search (LITS) methods achieve strong performance but are highly inefficient, often running an order of magnitude slower than iterative approaches. We propose Chain-in-Tree (CiT), a plug-in framework that decides when to branch during search rather than expanding at every step. CiT introduces lightweight Branching Necessity (BN) evaluations: BN-DP (Direct Prompting), where an auxiliary LLM judges branching needs, and BN-SC (Self-Consistency), which clusters candidate actions to assess agreement. Integrated into Tree of Thoughts, ReST-MCTS, and RAP, BN-DP achieves 75-85% reductions in token generation, model calls, and runtime on GSM8K and Math500, with often negligible or no accuracy loss. BN-SC typically yields substantial savings (up to 80%) generally but shows instability in 1-4 out of 14 settings, caused by a small subset of examples that produce extremely long reasoning steps. We theoretically prove that BN-DP never increases policy invocations and release both modular LITS implementations and a lightweight CiT function applicable across all LITS variants. The full codebase is publicly available at https://github.com/xinzhel/chain_in_tree.

Method: Decoupled fine-tuning using LoRA adapters with separate loss masking for tool selection and argument generation subtasks, plus DualTune inference framework that dynamically loads corresponding adapters and implements hierarchical orchestration.

Result: Qwen-2.5-7B model with decoupled fine-tuning improves tool calling accuracy by 46% over base model, outperforms similar-sized models in all cases and often beats 2x larger models on MCP-Bench benchmark.

Conclusion: The proposed methodology effectively addresses local LLMs’ tool-calling limitations through task disaggregation and specialized fine-tuning, enabling privacy-preserving, cost-effective agent orchestration on end-user devices.

Abstract: The deployment of Large Language Models (LLMs) as agentic orchestrators has revolutionized task automation, but the need for privacy-preserving, cost-effective solutions demands on-device inference capabilities. However, local LLMs consistently underperform compared to frontier models in tool calling scenarios, struggling with both tool selection from large tool sets and accurate argument generation for complex parameter structures. We introduce a methodology that disaggregates a tool-calling task into two distinct subtasks: tool selection and argument generation. We propose “decoupled fine-tuning”, a novel post-training approach that employs LoRA fine-tuning to create dedicated LoRA adapters for tool selection and tool-specific argument generation using separate loss masking for each of the subtasks. Furthermore, we present DualTune, an inference framework that leverages the LoRA adapters created using decoupled fine-tuning to perform efficient agent orchestration with the help of local models on end-user devices. DualTune decomposes the tool-call generation step into tool selection and argument generation, and dynamically loads the corresponding LoRA adapters to generate tool calls. Additionally, DualTune implements hierarchical orchestration to restrict the number of tools required for tool selection. Our experiments on the MCP-Bench benchmark demonstrate that the Qwen-2.5-7B model trained using decoupled fine-tuning improves the tool calling accuracy of the base model by 46%, and outperforms other local reasoning, non-reasoning and fine-tuned models of similar size in all cases, and models that are 2x larger, in most cases.

Method: The authors created PsychCounsel-Bench, a benchmark comprising approximately 2,252 single-choice questions from US national counselor examinations, which requires about 70% accuracy to pass. This benchmark tests deep understanding across various psychology sub-disciplines.

Result: Advanced models like GPT-4o, Llama3.3-70B, and Gemma3-27B achieved well above the passing threshold, while smaller open-source models (Qwen2.5-7B, Mistral-7B) remained far below it.

Conclusion: Only frontier LLMs currently meet counseling exam standards, highlighting both the promise and challenges of developing psychology-oriented LLMs. The dataset is publicly released for further research.

Abstract: Large Language Models (LLMs) have demonstrated remarkable success across a wide range of industries, primarily due to their impressive generative abilities. Yet, their potential in applications requiring cognitive abilities, such as psychological counseling, remains largely untapped. This paper investigates the key question: \textit{Can LLMs be effectively applied to psychological counseling?} To determine whether an LLM can effectively take on the role of a psychological counselor, the first step is to assess whether it meets the qualifications required for such a role, namely the ability to pass the U.S. National Counselor Certification Exam (NCE). This is because, just as a human counselor must pass a certification exam to practice, an LLM must demonstrate sufficient psychological knowledge to meet the standards required for such a role. To address this, we introduce PsychCounsel-Bench, a benchmark grounded in U.S.national counselor examinations, a licensure test for professional counselors that requires about 70% accuracy to pass. PsychCounsel-Bench comprises approximately 2,252 carefully curated single-choice questions, crafted to require deep understanding and broad enough to cover various sub-disciplines of psychology. This benchmark provides a comprehensive assessment of an LLM’s ability to function as a counselor. Our evaluation shows that advanced models such as GPT-4o, Llama3.3-70B, and Gemma3-27B achieve well above the passing threshold, while smaller open-source models (e.g., Qwen2.5-7B, Mistral-7B) remain far below it. These results suggest that only frontier LLMs are currently capable of meeting counseling exam standards, highlighting both the promise and the challenges of developing psychology-oriented LLMs. We release the proposed dataset for public use: https://github.com/cloversjtu/PsychCounsel-Bench

Method: Systematic examination of how physics-grounded methods enhance AI’s real-world comprehension across structured symbolic reasoning, embodied systems, and generative models through rigorous analysis of recent advances.

Result: The paper establishes clear distinctions between theoretical physics reasoning and applied physical understanding, advocating for intelligent systems that ground learning in both physical principles and embodied reasoning processes.

Conclusion: The synthesis envisions next-generation world models capable of explaining physical phenomena and predicting future states, advancing safe, generalizable, and interpretable AI systems.

Abstract: The rapid advancement of embodied intelligence and world models has intensified efforts to integrate physical laws into AI systems, yet physical perception and symbolic physics reasoning have developed along separate trajectories without a unified bridging framework. This work provides a comprehensive overview of physical AI, establishing clear distinctions between theoretical physics reasoning and applied physical understanding while systematically examining how physics-grounded methods enhance AI’s real-world comprehension across structured symbolic reasoning, embodied systems, and generative models. Through rigorous analysis of recent advances, we advocate for intelligent systems that ground learning in both physical principles and embodied reasoning processes, transcending pattern recognition toward genuine understanding of physical laws. Our synthesis envisions next-generation world models capable of explaining physical phenomena and predicting future states, advancing safe, generalizable, and interpretable AI systems. We maintain a continuously updated resource at https://github.com/AI4Phys/Awesome-AI-for-Physics.

Method: DebateQD uses a minimal QD evolutionary algorithm with tournament-style debates where two LLMs debate while a third judges. It evolves diverse strategies across categories (rationality, authority, emotional appeal) using prompt-based strategies within a single LLM architecture.

Result: Persuasion-optimized strategies achieve up to 13.94% smaller train-test generalization gaps while matching or exceeding truth optimization’s test performance across three model scales (7B, 32B, 72B parameters) and multiple dataset sizes.

Conclusion: Competitive pressure to persuade, rather than seeking truth collaboratively, fosters more transferable reasoning skills, offering a promising path for improving LLM generalization.

Abstract: Large Language Models (LLMs) optimized to output truthful answers often overfit, producing brittle reasoning that fails to generalize. While persuasion-based optimization has shown promise in debate settings, it has not been systematically compared against mainstream truth-based approaches. We introduce DebateQD, a minimal Quality-Diversity (QD) evolutionary algorithm that evolves diverse debate strategies across different categories (rationality, authority, emotional appeal, etc.) through tournament-style competitions where two LLMs debate while a third judges. Unlike previously proposed methods that require a population of LLMs, our approach maintains diversity of opponents through prompt-based strategies within a single LLM architecture, making it more accessible for experiments while preserving the key benefits of population-based optimization. In contrast to prior work, we explicitly isolate the role of the optimization objective by fixing the debate protocol and swapping only the fitness function: persuasion rewards strategies that convince the judge irrespective of truth, whereas truth rewards collaborative correctness. Across three model scales (7B, 32B, 72B parameters) and multiple dataset sizes from the QuALITY benchmark, persuasion-optimized strategies achieve up to 13.94% smaller train-test generalization gaps, while matching or exceeding truth optimization’s test performance. These results provide the first controlled evidence that competitive pressure to persuade, rather than seek the truth collaboratively, fosters more transferable reasoning skills, offering a promising path for improving LLM generalization.

Method: DRIFT decomposes informal mathematical statements into smaller sub-components to enable targeted retrieval of premises from libraries like Mathlib. It also retrieves illustrative theorems to help models use premises more effectively in formalization.

Result: DRIFT nearly doubles the F1 score compared to DPR baseline on ProofNet, and shows strong out-of-distribution performance on ConNF with BEq+@10 improvements of 37.14% (GPT-4.1) and 42.25% (DeepSeek-V3.1).

Conclusion: Retrieval effectiveness in mathematical autoformalization depends on model-specific knowledge boundaries, requiring adaptive retrieval strategies aligned with each model’s capabilities. DRIFT demonstrates the value of decomposition for improving premise retrieval.

Abstract: Automating the formalization of mathematical statements for theorem proving remains a major challenge for Large Language Models (LLMs). LLMs struggle to identify and utilize the prerequisite mathematical knowledge and its corresponding formal representation in languages like Lean. Current retrieval-augmented autoformalization methods query external libraries using the informal statement directly, but overlook a fundamental limitation: informal mathematical statements are often complex and offer limited context on the underlying math concepts. To address this, we introduce DRIFT, a novel framework that enables LLMs to decompose informal mathematical statements into smaller, more tractable ‘‘sub-components’’. This facilitates targeted retrieval of premises from mathematical libraries such as Mathlib. Additionally, DRIFT retrieves illustrative theorems to help models use premises more effectively in formalization tasks. We evaluate DRIFT across diverse benchmarks (ProofNet, ConNF, and MiniF2F-test) and find that it consistently improves premise retrieval, nearly doubling the F1 score compared to the DPR baseline on ProofNet. Notably, DRIFT demonstrates strong performance on the out-of-distribution ConNF benchmark, with BEq+@10 improvements of 37.14% and 42.25% using GPT-4.1 and DeepSeek-V3.1, respectively. Our analysis shows that retrieval effectiveness in mathematical autoformalization depends heavily on model-specific knowledge boundaries, highlighting the need for adaptive retrieval strategies aligned with each model’s capabilities.

Method: Introduced BesiegeField testbed built on Besiege game for part-based construction and physical simulation. Benchmarked LLMs with agentic workflows and used reinforcement learning finetuning with a curated cold-start dataset.

Result: Identified key capabilities needed for success (spatial reasoning, strategic assembly, instruction-following) and found current open-source models fall short, highlighting challenges in language, machine design, and physical reasoning.

Conclusion: Reinforcement learning shows promise for improving LLMs in machine design tasks, but significant challenges remain at the intersection of language, design, and physical reasoning.

Abstract: The design of complex machines stands as both a marker of human intelligence and a foundation of engineering practice. Given recent advances in large language models (LLMs), we ask whether they, too, can learn to create. We approach this question through the lens of compositional machine design: a task in which machines are assembled from standardized components to meet functional demands like locomotion or manipulation in a simulated physical environment. With this simplification, machine design is expressed as writing XML-like code that explicitly specifies pairwise part connections. To support this investigation, we introduce BesiegeField, a testbed built on the machine-building game Besiege, which enables part-based construction, physical simulation and reward-driven evaluation. Using BesiegeField, we benchmark state-of-the-art LLMs with agentic workflows and identify key capabilities required for success, including spatial reasoning, strategic assembly, and instruction-following. As current open-source models fall short, we explore reinforcement learning (RL) as a path to improvement: we curate a cold-start dataset, conduct RL finetuning experiments, and highlight open challenges at the intersection of language, machine design, and physical reasoning.

Method: Uses annotation-free Reinforcement Learning from AI Feedback (RLAIF) with LLM-based reward signals for factual accuracy, citation faithfulness, and instruction adherence. Also employs chain-of-thought-driven multi-call reasoning scaffold with self-verification and adaptive recovery from tool failures.

Result: Achieves state-of-the-art performance among 7B-scale deep research agents across 10 popular deep research benchmarks.

Conclusion: Careful reinforcement learning and reasoning design can produce efficient, resilient, and research-grade AI agents.

Abstract: Tool-augmented large language models (LLMs) are emerging as deep research agents, systems that decompose complex queries, retrieve external evidence, and synthesize grounded responses. Yet current agents remain limited by shallow retrieval, weak alignment metrics, and brittle tool-use behavior. We introduce PokeeResearch-7B, a 7B-parameter deep research agent built under a unified reinforcement learning framework for robustness, alignment, and scalability. PokeeResearch-7B is trained by an annotation-free Reinforcement Learning from AI Feedback (RLAIF) framework to optimize policies using LLM-based reward signals that capture factual accuracy, citation faithfulness, and instruction adherence. A chain-of-thought-driven multi-call reasoning scaffold further enhances robustness through self-verification and adaptive recovery from tool failures. Among 10 popular deep research benchmarks, PokeeResearch-7B achieves state-of-the-art performance among 7B-scale deep research agents. This highlights that careful reinforcement learning and reasoning design can produce efficient, resilient, and research-grade AI agents. The model and inference code is open-sourced under MIT license at https://github.com/Pokee-AI/PokeeResearchOSS.

Method: Uses residual vector quantized-variational autoencoder (RQ-VAE) on bar-wise music segments within a Transformer-based encoder-decoder framework to produce music codes.

Result: Outperforms previous representation learning baselines in semantic understanding tasks (melody extraction, chord recognition, emotion recognition) while maintaining comparable performance in content generation. Qualitative analysis shows effective capture of musical concepts.

Conclusion: MuseTok successfully creates discrete representations that enable both high-quality music generation and accurate music understanding, demonstrating its effectiveness across multiple musical tasks.

Abstract: Discrete representation learning has shown promising results across various domains, including generation and understanding in image, speech and language. Inspired by these advances, we propose MuseTok, a tokenization method for symbolic music, and investigate its effectiveness in both music generation and understanding tasks. MuseTok employs the residual vector quantized-variational autoencoder (RQ-VAE) on bar-wise music segments within a Transformer-based encoder-decoder framework, producing music codes that achieve high-fidelity music reconstruction and accurate understanding of music theory. For comprehensive evaluation, we apply MuseTok to music generation and semantic understanding tasks, including melody extraction, chord recognition, and emotion recognition. Models incorporating MuseTok outperform previous representation learning baselines in semantic understanding while maintaining comparable performance in content generation. Furthermore, qualitative analyses on MuseTok codes, using ground-truth categories and synthetic datasets, reveal that MuseTok effectively captures underlying musical concepts from large music collections.

Method: Used computational and experimental approaches to analyze acoustic transmission data for various leakage geometries, including different orifice diameters, across 1-5 kHz frequency range under 120-150 dB SPL conditions.

Result: Unsealed silicone rubber earplugs showed average transmission loss reduction of ~18 dB at 120 dB OISPL. Numerical simulations revealed SPL-dependent acoustic dissipation with acoustic energy converting to vorticity at 150 dB OISPL in ill-fitting models.

Conclusion: Earplug design plays a critical role in effective hearing protection in high-sound-pressure-level environments, with proper sealing being essential to prevent sound leakage and maintain adequate transmission loss.

Abstract: High sound pressure levels (SPL) pose notable risks in loud environments, particularly due to noise-induced hearing loss. Ill-fitting earplugs often lead to sound leakage, a phenomenon this study seeks to investigate. To validate our methodology, we first obtained computational and experimental acoustic transmission data for stand-alone slit resonators and orifices, for which extensive published data are readily available for comparison. We then examined the frequency-dependent acoustic power absorption coefficient and transmission loss (TL) across various leakage geometries, modeled using different orifice diameters. Experimental approaches spanned a frequency range of 1–5 kHz under SPL conditions of 120–150 dB. Key findings reveal that unsealed silicone rubber earplugs demonstrate an average TL reduction of approximately 18 dB at an overall incident SPL (OISPL) of 120 dB. Direct numerical simulations further highlight SPL-dependent acoustic dissipation mechanisms, showing the conversion of acoustic energy into vorticity in ill-fitting earplug models at an OISPL of 150 dB. These results highlight the role of earplug design for high-sound-pressure-level environments.

Method: Analyzed 10,000 speakers using three embedding systems, comparing 9 interpretable acoustic parameters with 7 principal components to predict embeddings.

Result: 9 acoustic parameters predict embeddings similarly to 7 principal components (over 50% variance explained). Embeddings capture gender implicitly but perform poorly on age.

Conclusion: Speaker embeddings effectively represent acoustic characteristics and gender but fail to capture age, indicating potential for enhancement in embedding calculation methods.

Abstract: Speaker embeddings are widely used in speaker verification systems and other applications where it is useful to characterise the voice of a speaker with a fixed-length vector. These embeddings tend to be treated as “black box” encodings, and how they relate to conventional acoustic and phonetic dimensions of voices has not been widely studied. In this paper we investigate how state-of-the-art speaker embedding systems represent the acoustic characteristics of speakers as described by conventional acoustic descriptors, age, and gender. Using a large corpus of 10,000 speakers and three embedding systems we show that a small set of 9 acoustic parameters chosen to be “interpretable” predict embeddings about the same as 7 principal components, corresponding to over 50% of variance in the data. We show that some principal dimensions operate differently for male and female speakers, suggesting there is implicit gender recognition within the embedding systems. However we show that speaker age is not well captured by embeddings, suggesting opportunities exist for improvements in their calculation.

Method: Proposed a text-coverage strategy specifically designed for text-matching data synthesis, enabling efficient zero/one-shot dysarthric data augmentation.

Result: The approach leads to substantial enhancements in dysarthric speech recognition performance when dealing with unseen dysarthric speakers.

Conclusion: The improvements are significant for practical applications including dysarthria rehabilitation programs and daily communication scenarios, addressing the data scarcity problem in dysarthric speech recognition.

Abstract: Dysarthric speech recognition (DSR) research has witnessed remarkable progress in recent years, evolving from the basic understanding of individual words to the intricate comprehension of sentence-level expressions, all driven by the pressing communication needs of individuals with dysarthria. Nevertheless, the scarcity of available data remains a substantial hurdle, posing a significant challenge to the development of effective sentence-level DSR systems. In response to this issue, dysarthric data augmentation (DDA) has emerged as a highly promising approach. Generative models are frequently employed to generate training data for automatic speech recognition tasks. However, their effectiveness hinges on the ability of the synthesized data to accurately represent the target domain. The wide-ranging variability in pronunciation among dysarthric speakers makes it extremely difficult for models trained on data from existing speakers to produce useful augmented data, especially in zero-shot or one-shot learning settings. To address this limitation, we put forward a novel text-coverage strategy specifically designed for text-matching data synthesis. This innovative strategy allows for efficient zero/one-shot DDA, leading to substantial enhancements in the performance of DSR when dealing with unseen dysarthric speakers. Such improvements are of great significance in practical applications, including dysarthria rehabilitation programs and day-to-day common-sentence communication scenarios.

Method: Uses Transformer-based inter-frame long-term dependency module, explores RVQ depth and codebook size optimization, and integrates into LLM-based TTS with global and local hierarchical architecture for multi-layer token dependencies.

Result: Extends LLM-based TTS from 3-layer RVQ at 50Hz to 32-layer RVQ at 5Hz, achieving 3x faster inference speed while maintaining similarity and naturalness compared to high-frame-rate codecs.

Conclusion: Validates the feasibility of using highly compressed 5Hz discrete tokens for fast and high-fidelity speech synthesis.

Abstract: We propose \textbf{U-Codec}, an \textbf{U}ltra low frame-rate neural speech \textbf{Codec} that achieves high-fidelity reconstruction and fast speech generation at an extremely low frame-rate of 5Hz (5 frames per second). Extreme compression at 5Hz typically leads to severe intelligibility and spectral detail loss, we introduce a Transformer-based inter-frame long-term dependency module and systematically explore residual vector quantization (RVQ) depth and codebook size to identify optimal configurations. Moreover, we apply U-Codec into a large language model (LLM)-based auto-regressive TTS model, which leverages global and local hierarchical architecture to effectively capture dependencies across multi-layer tokens. We extend LLM-based TTS from 3-layer RVQ at 50Hz to 32-layer RVQ at 5Hz. Experimental results demonstrate that U-Codec improves LLM-based TTS inference speed by around 3 $\times$ over high-frame-rate codecs while maintaining similarity and naturalness. These results validate the feasibility of using highly compressed 5Hz discrete tokens for fast and high-fidelity speech synthesis.

Method: The method implements SBM for generative speech enhancement, combining Schrödinger Bridge training with Mamba’s selective state-space model to enable efficient 1-step inference.

Result: Experiments on joint denoising and dereverberation tasks across four benchmark datasets show SBM outperforms strong baselines with 1-step or iterative inference while achieving the best real-time factor (RTF).

Conclusion: The integration of Schrödinger Bridge paradigm with selective state-space model architecture represents a promising direction for developing new deep generative models applicable to a broad range of generative tasks.

Abstract: We propose Schr"odinger Bridge Mamba (SBM), a new concept of training-inference framework motivated by the inherent compatibility between Schr"odinger Bridge (SB) training paradigm and selective state-space model Mamba. We exemplify the concept of SBM with an implementation for generative speech enhancement. Experiments on a joint denoising and dereverberation task using four benchmark datasets demonstrate that SBM, with only 1-step inference, outperforms strong baselines with 1-step or iterative inference and achieves the best real-time factor (RTF). Beyond speech enhancement, we discuss the integration of SB paradigm and selective state-space model architecture based on their underlying alignment, which indicates a promising direction for exploring new deep generative models potentially applicable to a broad range of generative tasks. Demo page: https://sbmse.github.io

Method: Constructed a dataset of malicious speech instructions expressed across multiple emotions and intensities, then evaluated several state-of-the-art LALMs on this dataset.

Result: Revealed substantial safety inconsistencies: different emotions elicit varying levels of unsafe responses, and the effect of intensity is non-monotonic with medium expressions often posing the greatest risk.

Conclusion: Highlights an overlooked vulnerability in LALMs and calls for alignment strategies explicitly designed to ensure robustness under emotional variation for trustworthy real-world deployment.

Abstract: Large audio-language models (LALMs) extend text-based LLMs with auditory understanding, offering new opportunities for multimodal applications. While their perception, reasoning, and task performance have been widely studied, their safety alignment under paralinguistic variation remains underexplored. This work systematically investigates the role of speaker emotion. We construct a dataset of malicious speech instructions expressed across multiple emotions and intensities, and evaluate several state-of-the-art LALMs. Our results reveal substantial safety inconsistencies: different emotions elicit varying levels of unsafe responses, and the effect of intensity is non-monotonic, with medium expressions often posing the greatest risk. These findings highlight an overlooked vulnerability in LALMs and call for alignment strategies explicitly designed to ensure robustness under emotional variation, a prerequisite for trustworthy deployment in real-world settings.

Method: Created SAKE benchmark targeting abstract auditory attributes, benchmarked seven editing methods on two LALMs across four dimensions: reliability, generality, audio/text locality, and portability.

Result: Results revealed significant challenges including preserving intra-attribute knowledge unrelated to edits, generalizing edits to multimodal reasoning, and maintaining edits under sequential updates.

Conclusion: SAKE provides a principled framework for studying auditory knowledge editing, opening new directions for maintaining and adapting LALMs in diverse real-world scenarios.

Abstract: Knowledge editing offers an efficient way to update model knowledge without full retraining, but prior work has concentrated almost exclusively on textual or visual modalities. We introduce SAKE, the first benchmark specifically designed for editing auditory attribute knowledge in Large Audio-Language Models (LALMs). Unlike factual updates, SAKE targets several abstract auditory attributes, capturing knowledge types that go beyond conventional textual and visual domains. We benchmark seven editing methods on two LALMs along four dimensions: reliability, generality, audio/text locality, and portability. Results highlight challenges such as preserving intra-attribute knowledge unrelated to the edit, generalizing edits to multimodal reasoning, and maintaining edits under sequential updates. SAKE provides a principled framework to study how knowledge editing extends to the auditory modalities, opening new directions for maintaining and adapting LALMs in more diverse real-world scenarios.

Method: Proposes Dynamic Dual-Signal Curriculum (DDSC) that combines domain-invariance and learning-progress signals computed each epoch, fused through a time-varying scheduler to create per-example weights that prioritize domain-invariant examples early and device-specific cases later.

Result: DDSC consistently improves cross-device performance across diverse ASC baselines and label budgets, with largest gains on unseen-device splits under DCASE 2024 Task 1 protocol.

Conclusion: DDSC is an effective, lightweight, architecture-agnostic solution that adapts curriculum learning online to handle device-induced domain shift in ASC without additional inference overhead.

Abstract: Acoustic scene classification (ASC) suffers from device-induced domain shift, especially when labels are limited. Prior work focuses on curriculum-based training schedules that structure data presentation by ordering or reweighting training examples from easy-to-hard to facilitate learning; however, existing curricula are static, fixing the ordering or the weights before training and ignoring that example difficulty and marginal utility evolve with the learned representation. To overcome this limitation, we propose the Dynamic Dual-Signal Curriculum (DDSC), a training schedule that adapts the curriculum online by combining two signals computed each epoch: a domain-invariance signal and a learning-progress signal. A time-varying scheduler fuses these signals into per-example weights that prioritize domain-invariant examples in early epochs and progressively emphasize device-specific cases. DDSC is lightweight, architecture-agnostic, and introduces no additional inference overhead. Under the official DCASE 2024 Task~1 protocol, DDSC consistently improves cross-device performance across diverse ASC baselines and label budgets, with the largest gains on unseen-device splits.

Method: TopSeg framework encodes PCG dynamics with multi-scale topological features and decodes them using a lightweight TCN with order- and duration-constrained inference, trained exclusively on PhysioNet 2016 with external validation on CirCor.

Result: Topological features consistently outperform spectrogram and envelope inputs, with largest margins at low data budgets. Full system surpasses end-to-end baselines while remaining competitive at full data. Combining H_0 and H_1 features improves S1/S2 localization and boundary stability.

Conclusion: Topology-aware representations provide strong inductive bias for data-efficient, cross-dataset PCG segmentation, supporting practical use when labeled data are limited.

Abstract: Deep learning approaches for heart-sound (PCG) segmentation built on time–frequency features can be accurate but often rely on large expert-labeled datasets, limiting robustness and deployment. We present TopSeg, a topological representation-centric framework that encodes PCG dynamics with multi-scale topological features and decodes them using a lightweight temporal convolutional network (TCN) with an order- and duration-constrained inference step. To evaluate data efficiency and generalization, we train exclusively on PhysioNet 2016 dataset with subject-level subsampling and perform external validation on CirCor dataset. Under matched-capacity decoders, the topological features consistently outperform spectrogram and envelope inputs, with the largest margins at low data budgets; as a full system, TopSeg surpasses representative end-to-end baselines trained on their native inputs under the same budgets while remaining competitive at full data. Ablations at 10% training confirm that all scales contribute and that combining H_0 and H_1 yields more reliable S1/S2 localization and boundary stability. These results indicate that topology-aware representations provide a strong inductive bias for data-efficient, cross-dataset PCG segmentation, supporting practical use when labeled data are limited.

Method: Uses adversarial optimization in time-frequency domain with level-proportional perceptual budget. Detection employs time-order-agnostic detector with Bitwise Readout Head (BRH) that aggregates temporal evidence per watermark bit.

Result: Achieves high audio quality and speech intelligibility (PESQ/STOI) with consistently low BER across various audio edits, outperforming state-of-the-art learning-based systems.

Conclusion: AWARE provides a robust alternative to attack-simulation approaches by using adversarial optimization and temporal evidence aggregation, demonstrating superior performance against diverse audio edits.

Abstract: Prevailing practice in learning-based audio watermarking is to pursue robustness by expanding the set of simulated distortions during training. However, such surrogates are narrow and prone to overfitting. This paper presents AWARE (Audio Watermarking with Adversarial Resistance to Edits), an alternative approach that avoids reliance on attack-simulation stacks and handcrafted differentiable distortions. Embedding is obtained via adversarial optimization in the time-frequency domain under a level-proportional perceptual budget. Detection employs a time-order-agnostic detector with a Bitwise Readout Head (BRH) that aggregates temporal evidence into one score per watermark bit, enabling reliable watermark decoding even under desynchronization and temporal cuts. Empirically, AWARE attains high audio quality and speech intelligibility (PESQ/STOI) and consistently low BER across various audio edits, often surpassing representative state-of-the-art learning-based audio watermarking systems.

Method: Two-stage pipeline: 1) Discriminator model filters out low-quality forgeries that don’t accurately reproduce vocal likeness, 2) Subsequent model trained only on authentic recordings identifies singers in remaining high-quality deepfakes and authentic audio.

Result: The system consistently outperforms existing baselines on both authentic and synthetic content.

Conclusion: The proposed two-stage approach effectively addresses singer identification in vocal deepfakes by focusing on high-quality forgeries and leveraging authentic training data.

Abstract: The proliferation of highly realistic singing voice deepfakes presents a significant challenge to protecting artist likeness and content authenticity. Automatic singer identification in vocal deepfakes is a promising avenue for artists and rights holders to defend against unauthorized use of their voice, but remains an open research problem. Based on the premise that the most harmful deepfakes are those of the highest quality, we introduce a two-stage pipeline to identify a singer’s vocal likeness. It first employs a discriminator model to filter out low-quality forgeries that fail to accurately reproduce vocal likeness. A subsequent model, trained exclusively on authentic recordings, identifies the singer in the remaining high-quality deepfakes and authentic audio. Experiments show that this system consistently outperforms existing baselines on both authentic and synthetic content.

Method: Proposes Safe-Ablated Refusal Steering (SARSteer) which leverages text-derived refusal steering to enforce rejection without manipulating audio inputs and introduces decomposed safe-space ablation to mitigate over-refusal.

Result: Extensive experiments show SARSteer significantly improves harmful-query refusal while preserving benign responses.

Conclusion: SARSteer establishes a principled step toward safety alignment in LALMs, addressing the unique challenges of audio modality safety.

Abstract: Large Audio-Language Models (LALMs) are becoming essential as a powerful multimodal backbone for real-world applications. However, recent studies show that audio inputs can more easily elicit harmful responses than text, exposing new risks toward deployment. While safety alignment has made initial advances in LLMs and Large Vision-Language Models (LVLMs), we find that vanilla adaptation of these approaches to LALMs faces two key limitations: 1) LLM-based steering fails under audio input due to the large distributional gap between activations, and 2) prompt-based defenses induce over-refusals on benign-speech queries. To address these challenges, we propose Safe-Ablated Refusal Steering (SARSteer), the first inference-time defense framework for LALMs. Specifically, SARSteer leverages text-derived refusal steering to enforce rejection without manipulating audio inputs and introduces decomposed safe-space ablation to mitigate over-refusal. Extensive experiments demonstrate that SARSteer significantly improves harmful-query refusal while preserving benign responses, establishing a principled step toward safety alignment in LALMs.

Method: Integrates frame-level embeddings from ReDimNet (state-of-the-art speaker verification model) to guide k-means clustering during pre-training, using dual objective of masked prediction and denoising for enhanced robustness.

Result: Achieves up to 62% relative improvement in EER for speaker verification and consistent gains on zero-shot profiling tasks (gender, age, accent, speaker counting).

Conclusion: DELULU serves as a strong universal encoder for speaker-aware speech processing, enabling superior performance across speaker-centric tasks without requiring task-specific fine-tuning.

Abstract: Self-supervised speech models have achieved remarkable success on content-driven tasks, yet they remain limited in capturing speaker-discriminative features critical for verification, diarization, and profiling applications. We introduce DELULU, a speaker-aware self-supervised foundational model that addresses this limitation by integrating external supervision into the pseudo-label generation process. DELULU leverages frame-level embeddings from ReDimNet, a state-of-the-art speaker verification model, to guide the k-means clustering step during pre-training, introducing a strong speaker-discriminative inductive bias that aligns representation learning with speaker identity. The model is trained using a dual objective that combines masked prediction and denoising, further enhancing robustness and generalization. DELULU significantly outperforms prior self-supervised learning (SSL) models across a range of speaker-centric tasks, achieving up to 62% relative improvement in equal error rate (EER) for speaker verification and consistent gains on zero-shot profiling tasks such as gender, age, accent, and speaker counting. Our findings demonstrate that DELULU is a strong universal encoder for speaker-aware speech processing, enabling superior performance even without task-specific fine-tuning.

Method: Uses a latent diffusion model that learns a joint distribution over full mixtures, submixtures, and individual stems. Formulates separation and imputation as conditional inpainting tasks in the latent space, enabling class-agnostic manipulation of arbitrary instrument sources.

Result: MGE-LDM can perform complete mixture generation, partial generation (source imputation), and text-conditioned extraction of arbitrary sources. It can be trained jointly across heterogeneous multi-track datasets without predefined instrument categories.

Conclusion: The framework provides a unified solution for multiple music manipulation tasks, supporting flexible, class-agnostic handling of arbitrary instrument sources through latent space conditional inpainting.

Abstract: We present MGE-LDM, a unified latent diffusion framework for simultaneous music generation, source imputation, and query-driven source separation. Unlike prior approaches constrained to fixed instrument classes, MGE-LDM learns a joint distribution over full mixtures, submixtures, and individual stems within a single compact latent diffusion model. At inference, MGE-LDM enables (1) complete mixture generation, (2) partial generation (i.e., source imputation), and (3) text-conditioned extraction of arbitrary sources. By formulating both separation and imputation as conditional inpainting tasks in the latent space, our approach supports flexible, class-agnostic manipulation of arbitrary instrument sources. Notably, MGE-LDM can be trained jointly across heterogeneous multi-track datasets (e.g., Slakh2100, MUSDB18, MoisesDB) without relying on predefined instrument categories. Audio samples are available at our project page: https://yoongi43.github.io/MGELDM_Samples/.

Method: Uses flow-matching-based generative model to directly predict mel-spectrograms from multi-stream transcriptions, with transcription-level speaker disentanglement, sentence-level alignment, and prompt-level random masking strategies.

Result: Achieves state-of-the-art performance, outperforming baselines like MoonCast and Sesame in speech quality, speaker consistency, and inference speed. Supports controllable dialogue generation with overlapping speech and timing control.

Conclusion: CoVoMix2 demonstrates strong generalizability to real-world speech generation scenarios without requiring prompt transcriptions, enabling efficient and controllable multi-speaker dialogue synthesis.

Abstract: Generating natural-sounding, multi-speaker dialogue is crucial for applications such as podcast creation, virtual agents, and multimedia content generation. However, existing systems struggle to maintain speaker consistency, model overlapping speech, and synthesize coherent conversations efficiently. In this paper, we introduce CoVoMix2, a fully non-autoregressive framework for zero-shot multi-talker dialogue generation. CoVoMix2 directly predicts mel-spectrograms from multi-stream transcriptions using a flow-matching-based generative model, eliminating the reliance on intermediate token representations. To better capture realistic conversational dynamics, we propose transcription-level speaker disentanglement, sentence-level alignment, and prompt-level random masking strategies. Our approach achieves state-of-the-art performance, outperforming strong baselines like MoonCast and Sesame in speech quality, speaker consistency, and inference speed. Notably, CoVoMix2 operates without requiring transcriptions for the prompt and supports controllable dialogue generation, including overlapping speech and precise timing control, demonstrating strong generalizability to real-world speech generation scenarios.

Method: Analyze and cluster semantics of public Lean discussions, fine-tune text embeddings on synthesized queries emulating user intents, and align with mathematician preferences using diverse feedback signals.

Result: Achieves over 30% relative improvement on real-world queries, informalized statements, and proof states compared to previous search engines and GPT-4o.

Conclusion: Lean Finder provides user-centered semantic search tailored to mathematicians’ needs and is compatible with LLM-based theorem provers, bridging retrieval with formal reasoning.

Abstract: We present Lean Finder, a semantic search engine for Lean and mathlib that understands and aligns with the intents of mathematicians. Progress in formal theorem proving is often hindered by the difficulty of locating relevant theorems and the steep learning curve of the Lean 4 language, making advancement slow and labor-intensive. Existing Lean search engines, though helpful, rely primarily on informalizations (natural language translation of the formal statements), while largely overlooking the mismatch with real-world user queries. In contrast, we propose a user-centered semantic search tailored to the needs of mathematicians. Our approach begins by analyzing and clustering the semantics of public Lean discussions, then fine-tuning text embeddings on synthesized queries that emulate user intents. We further align Lean Finder with mathematicians’ preferences using diverse feedback signals, encoding it with a rich awareness of their goals from multiple perspectives. Evaluations on real-world queries, informalized statements, and proof states demonstrate that our Lean Finder achieves over $30%$ relative improvement compared to previous search engines and GPT-4o. In addition, Lean Finder is compatible with LLM-based theorem provers, bridging retrieval with formal reasoning. Lean Finder is available at: https://leanfinder.github.io

Method: LS-OGD uses an online controller that dynamically adjusts the model’s learning rate and fusion weights between different data modalities based on detected drift and prediction errors.

Result: Under bounded drift conditions, LS-OGD ensures prediction error is uniformly ultimately bounded and converges to zero if drift ceases. The adaptive fusion strategy effectively isolates and mitigates severe modality-specific drift.

Conclusion: LS-OGD provides theoretical guarantees for developing reliable, continuously adapting multimodal learning systems with resilience and fault tolerance against concept drift.

Abstract: Multimodal learning systems often struggle in non-stationary environments due to concept drift, where changing data distributions can degrade performance. Modality-specific drifts and the lack of mechanisms for continuous, stable adaptation compound this challenge. This paper introduces LS-OGD, a novel adaptive control framework for robust multimodal learning in the presence of concept drift. LS-OGD uses an online controller that dynamically adjusts the model’s learning rate and the fusion weights between different data modalities in response to detected drift and evolving prediction errors. We prove that under bounded drift conditions, the LS-OGD system’s prediction error is uniformly ultimately bounded and converges to zero if the drift ceases. Additionally, we demonstrate that the adaptive fusion strategy effectively isolates and mitigates the impact of severe modality-specific drift, thereby ensuring system resilience and fault tolerance. These theoretical guarantees establish a principled foundation for developing reliable and continuously adapting multimodal learning systems.

Method: BEACON uses Sequential Search with Bayesian Learning to sequentially generate responses, update posterior belief over reward distributions in real time without training, and determine stopping points by weighing expected gains against computational cost.

Result: BEACON reduces average sampling by up to 80% while maintaining response quality, and demonstrates utility for cost-efficient preference data generation.

Conclusion: BEACON provides a principled framework for adaptive LLM sampling with theoretical optimality guarantees and practical tractability, offering actionable insights for efficient LLM deployment.

Abstract: Sampling multiple responses is a common way to improve LLM output quality, but it comes at the cost of additional computation. The key challenge is deciding when to stop generating new samples to balance accuracy gains against efficiency. To address this, we introduce BEACON (Bayesian Efficient Adaptive Criterion for Optimal N-stopping), a principled adaptive sampling framework grounded in Sequential Search with Bayesian Learning. BEACON sequentially generates responses from the policy LLM, updates posterior belief over reward distributions in real time without further training, and determines when to stop by weighing expected gains against computational cost. Sampling terminates once the marginal utility of further exploration no longer justifies the expense. We establish both theoretical optimality guarantees and practical tractability, and show empirically that BEACON reduces average sampling by up to 80% while maintaining response quality. We further demonstrate BEACON’s utility for cost-efficient preference data generation and outline practical extensions, offering actionable insights for future researchers.

Method: PatMD constructs a knowledge base where memes are deconstructed into misjudgment risk patterns, then retrieves relevant patterns to dynamically guide MLLM reasoning and avoid known misjudgment pitfalls.

Result: Experiments on 6,626 memes across 5 harmful detection tasks show PatMD outperforms state-of-the-art baselines with 8.30% F1-score improvement and 7.71% accuracy improvement.

Conclusion: PatMD demonstrates strong generalizability and improved detection capability for harmful memes by proactively addressing misjudgment risks through pattern-based guidance.

Abstract: Internet memes have emerged as a popular multimodal medium, yet they are increasingly weaponized to convey harmful opinions through subtle rhetorical devices like irony and metaphor. Existing detection approaches, including MLLM-based techniques, struggle with these implicit expressions, leading to frequent misjudgments. This paper introduces PatMD, a novel approach that improves harmful meme detection by learning from and proactively mitigating these potential misjudgment risks. Our core idea is to move beyond superficial content-level matching and instead identify the underlying misjudgment risk patterns, proactively guiding the MLLMs to avoid known misjudgment pitfalls. We first construct a knowledge base where each meme is deconstructed into a misjudgment risk pattern explaining why it might be misjudged, either overlooking harmful undertones (false negative) or overinterpreting benign content (false positive). For a given target meme, PatMD retrieves relevant patterns and utilizes them to dynamically guide the MLLM’s reasoning. Experiments on a benchmark of 6,626 memes across 5 harmful detection tasks show that PatMD outperforms state-of-the-art baselines, achieving an average of 8.30% improvement in F1-score and 7.71% improvement in accuracy, demonstrating strong generalizability and improved detection capability of harmful memes.

Method: Used WaveNet architecture with dilated causal convolutions and residual connections on 209,232 EEG samples from Mayo Clinic and St. Anne’s University Hospital, with 70/20/10 train/validation/test split.

Result: Achieved classification accuracy exceeding previous CNN and LSTM approaches, with high precision for noise/artifact distinction but modest misclassification between physiological/pathological signals due to clinical overlap.

Conclusion: WaveNet is well-suited for EEG data analysis due to its ability to capture both fine-grained and long-range temporal dependencies, offering an effective automated classification solution.

Abstract: This study introduces a WaveNet-based deep learning model designed to automate the classification of EEG signals into physiological, pathological, artifact, and noise categories. Traditional methods for EEG signal classification, which rely on expert visual review, are becoming increasingly impractical due to the growing complexity and volume of EEG recordings. Leveraging a publicly available annotated dataset from Mayo Clinic and St. Anne’s University Hospital, the WaveNet model was trained, validated, and tested on 209,232 samples with a 70/20/10 percent split. The model achieved a classification accuracy exceeding previous CNN and LSTM-based approaches, and was benchmarked against a Temporal Convolutional Network (TCN) baseline. Notably, the model distinguishes noise and artifacts with high precision, although it reveals a modest but explainable degree of misclassification between physiological and pathological signals, reflecting inherent clinical overlap. WaveNet’s architecture, originally developed for raw audio synthesis, is well suited for EEG data due to its use of dilated causal convolutions and residual connections, enabling it to capture both fine-grained and long-range temporal dependencies. The research also details the preprocessing pipeline, including dynamic dataset partitioning and normalization steps that support model generalization.

Method: Three-stage pipeline: (1) preprocessing data from four datasets, extracting temporal signals and addressing class imbalance; (2) pre-training eight deep-learning architectures on largest datasets with hyperparameter optimization; (3) fine-tuning on intermediate dataset and external validation on independent cohort.

Result: Hybrid convolutional-recurrent and transformer models achieved strong external validation performance with AUC-ROC scores exceeding 90% and F1-Score over 70%. A temporal convolutional model attained 91.14% AUC-ROC in external validation, outperforming existing methods that rely solely on internal validation.

Conclusion: Keystroke dynamics show significant promise as a reliable digital biomarker for Parkinson’s disease, offering a viable approach for early detection and continuous remote monitoring through scalable, non-invasive technology.

Abstract: Parkinson’s disease (PD) presents a growing global challenge, affecting over 10 million individuals, with prevalence expected to double by 2040. Early diagnosis remains difficult due to the late emergence of motor symptoms and limitations of traditional clinical assessments. In this study, we propose a novel pipeline that leverages keystroke dynamics as a non-invasive and scalable biomarker for remote PD screening and telemonitoring. Our methodology involves three main stages: (i) preprocessing of data from four distinct datasets, extracting four temporal signals and addressing class imbalance through the comparison of three methods; (ii) pre-training eight state-of-the-art deep-learning architectures on the two largest datasets, optimizing temporal windowing, stride, and other hyperparameters; (iii) fine-tuning on an intermediate-sized dataset and performing external validation on a fourth, independent cohort. Our results demonstrate that hybrid convolutional-recurrent and transformer-based models achieve strong external validation performance, with AUC-ROC scores exceeding 90% and F1-Score over 70%. Notably, a temporal convolutional model attains an AUC-ROC of 91.14% in external validation, outperforming existing methods that rely solely on internal validation. These findings underscore the potential of keystroke dynamics as a reliable digital biomarker for PD, offering a promising avenue for early detection and continuous monitoring.

Method: Uses Ensemble Score Filter (EnSF), a diffusion-model-based filtering algorithm that leverages score-based generative diffusion models for high-dimensional nonlinear filtering problems in wildfire spread models.

Result: Numerical investigations demonstrate that EnSF provides superior accuracy, stability, and computational efficiency for wildfire data assimilation compared to other methods.

Conclusion: EnSF is established as a robust and practical method for wildfire data assimilation, with publicly available code for implementation.

Abstract: As wildfires become increasingly destructive and expensive to control, effective management of active wildfires requires accurate, real-time fire spread predictions. To enhance the forecasting accuracy of active fires, data assimilation plays a vital role by integrating observations (such as remote-sensing data) and fire predictions generated from numerical models. This paper provides a comprehensive investigation on the application of a recently proposed diffusion-model-based filtering algorithm – the Ensemble Score Filter (EnSF) – to the data assimilation problem for real-time active wildfire spread predictions. Leveraging a score-based generative diffusion model, EnSF has been shown to have superior accuracy for high-dimensional nonlinear filtering problems, making it an ideal candidate for the filtering problems of wildfire spread models. Technical details are provided, and our numerical investigations demonstrate that EnSF provides superior accuracy, stability, and computational efficiency, establishing it as a robust and practical method for wildfire data assimilation. Our code has been made publicly available.

Method: Created a dataset for tool response processing and evaluated 15 open/closed weight models using multiple prompting approaches to analyze JSON processing capabilities.

Result: JSON processing remains difficult even for frontier models across all prompting strategies. Performance varies significantly (3-50%) based on processing approach, output size, and reasoning complexity.

Conclusion: The optimal response processing strategy depends on tool output characteristics and reasoning complexity, highlighting the need for better JSON processing capabilities in LLMs for effective task automation.

Abstract: Most realistic task automation problems require large language models (LLMs) to call tools, which often return complex JSON responses. These responses must be further processed to derive the information necessary for task completion. The ability of LLMs to do so is under-studied. In this paper, we study the tool response processing task and LLMs’ abilities to process structured (JSON) responses. We created a dataset for this task, and evaluated 15 open and closed weight models using multiple prompting approaches. Our results show that JSON processing remains a difficult task even for frontier models across multiple prompting strategies. The optimal response processing strategy depends on both the nature and size of the tool outputs, as well as the complexity of the required reasoning. Variations in processing approaches can lead to performance differences ranging from 3% to 50%.

Method: Conducted proximate, ultimate, fiber, TGA/DTG, kinetic, thermodynamic, and Py-Micro GC analyses on pure DS, SCG, and their blends. Used isoconversional methods (KAS, FWO, Friedman) for kinetic modeling and trained an LSTM model with lignocellulosic data.

Result: Blend 3 (25% DS - 75% SCG) offered superior hydrogen yield potential but highest activation energy (313.24 kJ/mol), while Blend 1 (75% DS - 25% SCG) had best activation energy (161.75 kJ/mol). KAS method was most accurate for kinetic modeling. LSTM model predicted TGA curves with exceptional accuracy (R²: 0.9996-0.9998).

Conclusion: The integration of AI, particularly LSTM models, significantly enhances pyrolysis process modeling accuracy, enabling better optimization of biomass blends for sustainable hydrogen production from food waste resources.

Abstract: This work contributes to advancing sustainable energy and waste management strategies by investigating the thermochemical conversion of food-based biomass through pyrolysis, highlighting the role of artificial intelligence (AI) in enhancing process modelling accuracy and optimization efficiency. The main objective is to explore the potential of underutilized biomass resources, such as spent coffee grounds (SCG) and date seeds (DS), for sustainable hydrogen production. Specifically, it aims to optimize the pyrolysis process while evaluating the performance of these resources both individually and as blends. Proximate, ultimate, fibre, TGA/DTG, kinetic, thermodynamic, and Py-Micro GC analyses were conducted for pure DS, SCG, and blends (75% DS - 25% SCG, 50% DS

• 50% SCG, 25% DS - 75% SCG). Blend 3 offered superior hydrogen yield potential but had the highest activation energy (Ea: 313.24 kJ/mol), while Blend 1 exhibited the best activation energy value (Ea: 161.75 kJ/mol). The kinetic modelling based on isoconversional methods (KAS, FWO, Friedman) identified KAS as the most accurate. These approaches provide a detailed understanding of the pyrolysis process, with particular emphasis on the integration of artificial intelligence. An LSTM model trained with lignocellulosic data predicted TGA curves with exceptional accuracy (R^2: 0.9996-0.9998).

Method: LAMI represents individual survey responses as relational graphs, learns latent connections through a specialized graph structure learning layer, and integrates a large language model to generate natural language explanations.

Result: Experiments on YRBS and NSDUH datasets show LAMI outperforms competitive baselines in predictive accuracy and reveals meaningful behavioral substructures and psychosocial pathways.

Conclusion: LAMI effectively detects illicit drug use while providing interpretable insights into established risk factors like family dynamics, peer influence, and school-related distress.

Abstract: Illicit drug use among teenagers and young adults (TYAs) remains a pressing public health concern, with rising prevalence and long-term impacts on health and well-being. To detect illicit drug use among TYAs, researchers analyze large-scale surveys such as the Youth Risk Behavior Survey (YRBS) and the National Survey on Drug Use and Health (NSDUH), which preserve rich demographic, psychological, and environmental factors related to substance use. However, existing modeling methods treat survey variables independently, overlooking latent and interconnected structures among them. To address this limitation, we propose LAMI (LAtent relation Mining with bi-modal Interpretability), a novel joint graph-language modeling framework for detecting illicit drug use and interpreting behavioral risk factors among TYAs. LAMI represents individual responses as relational graphs, learns latent connections through a specialized graph structure learning layer, and integrates a large language model to generate natural language explanations grounded in both graph structures and survey semantics. Experiments on the YRBS and NSDUH datasets show that LAMI outperforms competitive baselines in predictive accuracy. Interpretability analyses further demonstrate that LAMI reveals meaningful behavioral substructures and psychosocial pathways, such as family dynamics, peer influence, and school-related distress, that align with established risk factors for substance use.

Method: CTR-LoRA allocates parameters based on marginal utility from lightweight second-order proxies and constrains updates using a Fisher/Hessian-metric trust region.

Result: Experiments on 7B-13B models show consistent improvements over strong PEFT baselines on both in-distribution and out-of-distribution benchmarks, with enhanced training stability, reduced memory, and higher throughput.

Conclusion: CTR-LoRA provides a principled path toward more robust and deployable PEFT methods, positioning it on the Pareto frontier of performance and efficiency.

Abstract: Parameter-efficient fine-tuning (PEFT) has become the standard approach for adapting large language models under limited compute and memory budgets. Although previous methods improve efficiency through low-rank updates, quantization, or heuristic budget reallocation, they often decouple the allocation of capacity from the way updates evolve during training. In this work, we introduce CTR-LoRA, a framework guided by curvature trust region that integrates rank scheduling with stability-aware optimization. CTR-LoRA allocates parameters based on marginal utility derived from lightweight second-order proxies and constrains updates using a Fisher/Hessian-metric trust region. Experiments on multiple open-source backbones (7B-13B), evaluated on both in-distribution and out-of-distribution benchmarks, show consistent improvements over strong PEFT baselines. In addition to increased accuracy, CTR-LoRA enhances training stability, reduces memory requirements, and achieves higher throughput, positioning it on the Pareto frontier of performance and efficiency. These results highlight a principled path toward more robust and deployable PEFT.

Method: The system addresses Shadowy Sparsity with three key components: 1) Shadowy-sparsity Exposer to capture sparsity details, 2) Sequence-oriented Predictor for efficient predictions on large sequences, and 3) Dynamic-aware Operator for structured computational patterns and coalesced memory accesses.

Result: Long Exposure achieves up to 2.49x speedup in end-to-end fine-tuning compared to state-of-the-art methods.

Conclusion: The system offers promising advancements in accelerating PEFT for LLMs by effectively addressing the unique challenges of Shadowy Sparsity in fine-tuning.

Abstract: The adaptation of pre-trained large language models (LLMs) to diverse downstream tasks via fine-tuning is critical for numerous applications. However, the inefficiency of parameter-efficient fine-tuning (PEFT) techniques presents significant challenges in terms of time investments and operational costs. In this paper, we first introduce a nuanced form of sparsity, termed Shadowy Sparsity, which is distinctive in fine-tuning and has not been adequately addressed for acceleration. Under Shadowy Sparsity, we propose Long Exposure, an efficient system to accelerate PEFT for LLMs. Long Exposure comprises three key components: Shadowy-sparsity Exposer employs a prolonged sensing range to capture more sparsity details under shadowy sparsity; Sequence-oriented Predictor provides efficient yet accurate predictions to handle large sequence inputs and constantly-evolving parameters; and Dynamic-aware Operator facilitates more structured computational patterns and coalesced memory accesses, addressing dynamic sparse operations. Extensive evaluations show that Long Exposure outperforms state-of-the-arts with up to a $2.49\times$ speedup in end-to-end fine-tuning, offering promising advancements in accelerating PEFT for LLMs.

Method: Training malicious adversarial embeddings that encourage transitional tokens after reasoning steps, combined with a backdoor implantation strategy to overcome the continuous-to-discrete projection gap and ensure reliable activation through trigger tokens.

Result: Achieved 100% attack success rate across four advanced LRMs (Phi-RM, Nemotron-Nano, R1-Qwen, R1-Llama) and three math reasoning benchmarks, forcing models to generate up to their maximum token limits while remaining stealthy and robust against mitigation strategies.

Conclusion: The findings reveal a critical and underexplored security vulnerability in large reasoning models from the perspective of reasoning inefficiency, exposing the risks of chain-of-thought reasoning mechanisms.

Abstract: Modern large reasoning models (LRMs) exhibit impressive multi-step problem-solving via chain-of-thought (CoT) reasoning. However, this iterative thinking mechanism introduces a new vulnerability surface. We present the Deadlock Attack, a resource exhaustion method that hijacks an LRM’s generative control flow by training a malicious adversarial embedding to induce perpetual reasoning loops. Specifically, the optimized embedding encourages transitional tokens (e.g., “Wait”, “But”) after reasoning steps, preventing the model from concluding its answer. A key challenge we identify is the continuous-to-discrete projection gap: na"ive projections of adversarial embeddings to token sequences nullify the attack. To overcome this, we introduce a backdoor implantation strategy, enabling reliable activation through specific trigger tokens. Our method achieves a 100% attack success rate across four advanced LRMs (Phi-RM, Nemotron-Nano, R1-Qwen, R1-Llama) and three math reasoning benchmarks, forcing models to generate up to their maximum token limits. The attack is also stealthy (in terms of causing negligible utility loss on benign user inputs) and remains robust against existing strategies trying to mitigate the overthinking issue. Our findings expose a critical and underexplored security vulnerability in LRMs from the perspective of reasoning (in)efficiency.

Method: Splits model into encoder and classifier, uses fine-grained knowledge discovery and contribution-driven aggregation to identify new knowledge, and includes anti-forgetting mechanism to preserve source domain performance.

Result: Significantly outperforms other baselines on multi-domain datasets across three data-shift scenarios for both source-domain and target-domain clients.

Conclusion: Gains effectively addresses open-set FL by providing fine-grained knowledge discovery and balanced adaptation while maintaining source domain performance.

Abstract: Conventional federated learning (FL) assumes a closed world with a fixed total number of clients. In contrast, new clients continuously join the FL process in real-world scenarios, introducing new knowledge. This raises two critical demands: detecting new knowledge, i.e., knowledge discovery, and integrating it into the global model, i.e., knowledge adaptation. Existing research focuses on coarse-grained knowledge discovery, and often sacrifices source domain performance and adaptation efficiency. To this end, we propose a fine-grained federated domain adaptation approach in open set (Gains). Gains splits the model into an encoder and a classifier, empirically revealing features extracted by the encoder are sensitive to domain shifts while classifier parameters are sensitive to class increments. Based on this, we develop fine-grained knowledge discovery and contribution-driven aggregation techniques to identify and incorporate new knowledge. Additionally, an anti-forgetting mechanism is designed to preserve source domain performance, ensuring balanced adaptation. Experimental results on multi-domain datasets across three typical data-shift scenarios demonstrate that Gains significantly outperforms other baselines in performance for both source-domain and target-domain clients. Code is available at: https://github.com/Zhong-Zhengyi/Gains.

Method: Proposed SAU-FNO framework that integrates self-attention and U-Net with FNO to capture long-range dependencies and local high-frequency features. Uses transfer learning to fine-tune low-fidelity data, reducing dependency on high-fidelity datasets.

Result: SAU-FNO achieves state-of-the-art thermal prediction accuracy and provides 842x speedup compared to traditional FEM methods.

Conclusion: SAU-FNO is an efficient tool for advanced 3D IC thermal simulations, offering both accuracy and significant computational speed improvements.

Abstract: Thermal management in 3D ICs is increasingly challenging due to higher power densities. Traditional PDE-solving-based methods, while accurate, are too slow for iterative design. Machine learning approaches like FNO provide faster alternatives but suffer from high-frequency information loss and high-fidelity data dependency. We introduce Self-Attention U-Net Fourier Neural Operator (SAU-FNO), a novel framework combining self-attention and U-Net with FNO to capture long-range dependencies and model local high-frequency features effectively. Transfer learning is employed to fine-tune low-fidelity data, minimizing the need for extensive high-fidelity datasets and speeding up training. Experiments demonstrate that SAU-FNO achieves state-of-the-art thermal prediction accuracy and provides an 842x speedup over traditional FEM methods, making it an efficient tool for advanced 3D IC thermal simulations.

Method: An agent-based framework where each nonlinear pattern (e.g., absolute-value terms, bilinear products) is assigned to a specialized reformulation agent that derives exact linear reformulations, then coordinates to assemble a solver-ready linear model.

Result: Evaluated on 20 real-world nonlinear problems from ComplexOR dataset. Results show specialized LLM agents can successfully automate linearization tasks.

Conclusion: The framework enables automated linearization of nonlinear optimization problems, paving the way for fully conversational modeling pipelines in optimization.

Abstract: Reformulating nonlinear optimization problems is largely manual and expertise-intensive, yet it remains essential for solving such problems with linear optimization solvers or applying special-purpose algorithms. We introduce \textit{LinearizeLLM}, an agent-based framework that solves this task by leveraging Large Language Models (LLMs). The framework assigns each nonlinear pattern to a \textit{reformulation agent} that is explicitly instructed to derive an exact linear reformulation for its nonlinearity pattern, for instance, absolute-value terms or bilinear products of decision variables. The agents then coordinate to assemble a solver-ready linear model equivalent to the original problem. To benchmark the approach, we create a dataset of 20 real-world nonlinear optimization problems derived from the established ComplexOR dataset of linear optimization problems. We evaluate our approach with several LLMs. Our results indicate that specialized LLM agents can automate linearization tasks, opening a path toward fully conversational modeling pipelines for nonlinear optimization.

Method: Extract features from speech foundation models (HuBERT and wav2vec 2.0) and train probes on these features for wearable sensor tasks, focusing on their convolutional feature encoders.

Result: Speech model features outperform self-supervised models trained directly on modality-specific datasets for mood classification, arrhythmia detection, and activity classification tasks.

Conclusion: Speech foundation models provide effective representations for data-scarce time-series tasks, enabling performance enhancement through simple probing methods and advancing generalized time-series modeling.

Abstract: Both speech and sensor time series data encode information in both the time- and frequency- domains, like spectral powers and waveform shapelets. We show that speech foundation models learn representations that generalize beyond the speech domain and achieve state-of-the-art performance on diverse time-series tasks from wearable sensors. Probes trained on features extracted from HuBERT and wav2vec 2.0 outperform those extracted from self-supervised models trained directly on modality-specific datasets for mood classification, arrhythmia detection, and activity classification tasks. We find that the convolutional feature encoders of speech models are particularly relevant for wearable sensor applications. The proposed approach enhances performance on data-scarce time-series tasks using simple probing methods. This work takes a step toward developing generalized time-series models that unify speech and sensor modalities.

Method: Employ persistent homology to extract topological features from data within a metric space, providing a principled way to quantify diversity beyond traditional entropy-based measures.

Result: Persistent homology proves to be a powerful tool for analyzing and enhancing training data quality by capturing geometric and topological properties that influence learning outcomes.

Conclusion: Persistent homology offers a novel approach to understand and improve training data quality by focusing on geometric structure, representation richness, and redundancy elimination, which are critical factors for AI system performance.

Abstract: High-quality training data is the foundation of machine learning and artificial intelligence, shaping how models learn and perform. Although much is known about what types of data are effective for training, the impact of the data’s geometric structure on model performance remains largely underexplored. We propose that both the richness of representation and the elimination of redundancy within training data critically influence learning outcomes. To investigate this, we employ persistent homology to extract topological features from data within a metric space, thereby offering a principled way to quantify diversity beyond entropy-based measures. Our findings highlight persistent homology as a powerful tool for analyzing and enhancing the training data that drives AI systems.

Method: Uses prompt-guided responses from LLMs as data augmentation to generate both truthful and hallucinated data. Introduces Contrastive Mahalanobis Score (CM Score) based on modeling distributions of truthful and hallucinated data in activation space using matrix decomposition.

Result: Achieves superior hallucination detection performance, outperforming competitive baseline by 6.55% margin in extensive experiments.

Conclusion: PALE framework offers strong generalizability and practicality for real-world applications without requiring additional human annotations, effectively addressing hallucination detection challenges in LLMs.

Abstract: Large language models (LLMs) have garnered significant interest in AI community. Despite their impressive generation capabilities, they have been found to produce misleading or fabricated information, a phenomenon known as hallucinations. Consequently, hallucination detection has become critical to ensure the reliability of LLM-generated content. One primary challenge in hallucination detection is the scarcity of well-labeled datasets containing both truthful and hallucinated outputs. To address this issue, we introduce Prompt-guided data Augmented haLlucination dEtection (PALE), a novel framework that leverages prompt-guided responses from LLMs as data augmentation for hallucination detection. This strategy can generate both truthful and hallucinated data under prompt guidance at a relatively low cost. To more effectively evaluate the truthfulness of the sparse intermediate embeddings produced by LLMs, we introduce an estimation metric called the Contrastive Mahalanobis Score (CM Score). This score is based on modeling the distributions of truthful and hallucinated data in the activation space. CM Score employs a matrix decomposition approach to more accurately capture the underlying structure of these distributions. Importantly, our framework does not require additional human annotations, offering strong generalizability and practicality for real-world applications. Extensive experiments demonstrate that PALE achieves superior hallucination detection performance, outperforming the competitive baseline by a significant margin of 6.55%.

Method: Proposes DAWP framework with two key components: 1) AI data assimilation (AIDA) module using mask multi-modality autoencoder (MMAE) with mask ViT-VAEs to assimilate irregular satellite observations, 2) Spatiotemporal decoupling transformer with cross-regional boundary conditioning for learning dynamics in observation space and enabling sub-image-based global forecasting.

Result: Comprehensive experiments show AIDA initialization significantly improves rollout and efficiency of AI weather prediction. The framework demonstrates promising potential for global precipitation forecasting applications.

Conclusion: DAWP successfully liberates AI weather prediction from reanalysis data dependencies by operating directly in observation space, providing improved performance and efficiency for weather forecasting tasks.

Abstract: Weather prediction is a critical task for human society, where impressive progress has been made by training artificial intelligence weather prediction (AIWP) methods with reanalysis data. However, reliance on reanalysis data limits the AIWPs with shortcomings, including data assimilation biases and temporal discrepancies. To liberate AIWPs from the reanalysis data, observation forecasting emerges as a transformative paradigm for weather prediction. One of the key challenges in observation forecasting is learning spatiotemporal dynamics across disparate measurement systems with irregular high-resolution observation data, which constrains the design and prediction of AIWPs. To this end, we propose our DAWP as an innovative framework to enable AIWPs to operate in a complete observation space by initialization with an artificial intelligence data assimilation (AIDA) module. Specifically, our AIDA module applies a mask multi-modality autoencoder(MMAE)for assimilating irregular satellite observation tokens encoded by mask ViT-VAEs. For AIWP, we introduce a spatiotemporal decoupling transformer with cross-regional boundary conditioning (CBC), learning the dynamics in observation space, to enable sub-image-based global observation forecasting. Comprehensive experiments demonstrate that AIDA initialization significantly improves the roll out and efficiency of AIWP. Additionally, we show that DAWP holds promising potential to be applied in global precipitation forecasting.

Method: Proposes Cog-Rethinker with a two-stage hierarchical metacognitive framework: 1) decomposes zero-accuracy problems into subproblems, 2) refines wrong answers by referencing previous solutions. Uses supervised fine-tuning for cold-start and maintains train-test consistency.

Result: Superior performance on various mathematical reasoning benchmarks with improved sample efficiency that accelerates convergence compared to baseline methods.

Conclusion: Cog-Rethinker effectively addresses sampling inefficiency in RL-based LLM reasoning through hierarchical metacognitive processing, demonstrating both performance improvements and faster convergence.

Abstract: Contemporary progress in large language models (LLMs) has revealed notable inferential capacities via reinforcement learning (RL) employing verifiable reward, facilitating the development of O1 and R1-like reasoning models. Directly training from base models with RL is called zero-RL. However, previous works rely upon activating LLMs’ inherent capacities through fixed prompt templates. This strategy introduces substantial sampling inefficiencies for weak LLMs, as the majority of problems generate invalid outputs during accuracy-driven filtration in reasoning tasks, which causes a waste of samples. To solve this issue, we propose Cog-Rethinker, a novel hierarchical metacognitive RL framework for LLM reasoning. Our Cog-Rethinker mainly focuses on the rollout procedure in RL training. After the direct rollout, our Cog-Rethinker improves sample utilization in a hierarchical metacognitive two-stage framework. By leveraging human cognition during solving problems, firstly, it prompts policy to decompose zero-accuracy problems into subproblems to produce final reasoning results. Secondly, with zero-accuracy problems in previous rollout stage, it further prompts policy to refine these answers by referencing previous wrong solutions. Moreover, to enable cold-start of the two new reasoning patterns and maintain train-test consistency across prompt templates, our Cog-Rethinker applies supervised fine-tuning on the policy using correct samples of the two stages with direct rollout template. Experimental results demonstrate Cog-Rethinker’s superior performance on various mathematical reasoning benchmarks, we also analyzed its improved sample efficiency that accelerates convergence compared to baseline methods.

Method: Introduces α-mixture assistant distribution with variable α parameter, and AMiD framework that generalizes divergences used with assistant distributions based on optimality.

Result: AMiD demonstrates superior performance and training stability by leveraging a broader, theoretically grounded assistant distribution space.

Conclusion: The proposed α-mixture assistant distribution and AMiD framework provide a unified, systematic approach that outperforms previous fragmented methods for LLM knowledge distillation.

Abstract: Autoregressive large language models (LLMs) have achieved remarkable improvement across many tasks but incur high computational and memory costs. Knowledge distillation (KD) mitigates this issue by transferring knowledge from a large teacher to a smaller student through distributional alignment. Previous studies have proposed various discrepancy metrics, but the capacity gap and training instability caused by near-zero probabilities, stemming from the high-dimensional output of LLMs, remain fundamental limitations. To overcome these challenges, several approaches implicitly or explicitly incorporating assistant distribution have recently been proposed. However, the past proposals of assistant distributions have been a fragmented approach without a systematic investigation of the interpolation path and the divergence. This paper proposes $\alpha$-mixture assistant distribution, a novel generalized family of assistant distributions, and $\alpha$-mixture distillation, coined AMiD, a unified framework for KD using the assistant distribution. The $\alpha$-mixture assistant distribution provides a continuous extension of the assistant distribution by introducing a new distribution design variable $\alpha$, which has been fixed in all previous approaches. Furthermore, AMiD generalizes the family of divergences used with the assistant distributions based on optimality, which has also been restricted in previous works. Through extensive experiments, we demonstrate that AMiD offers superior performance and training stability by leveraging a broader and theoretically grounded assistant distribution space.

Method: Proposes MERE mechanism for enriched feature views and CDTA module for multi-scale temporal representation learning in the MEET-Sepsis framework.

Result: Achieves competitive prediction accuracy using only 20% of the ICU monitoring time required by state-of-the-art methods.

Conclusion: MEET-Sepsis significantly advances early sepsis prediction, with extensive validation confirming its efficacy.

Abstract: Sepsis is a life-threatening infectious syndrome associated with high mortality in intensive care units (ICUs). Early and accurate sepsis prediction (SP) is critical for timely intervention, yet remains challenging due to subtle early manifestations and rapidly escalating mortality. While AI has improved SP efficiency, existing methods struggle to capture weak early temporal signals. This paper introduces a Multi-Endogenous-view Representation Enhancement (MERE) mechanism to construct enriched feature views, coupled with a Cascaded Dual-convolution Time-series Attention (CDTA) module for multi-scale temporal representation learning. The proposed MEET-Sepsis framework achieves competitive prediction accuracy using only 20% of the ICU monitoring time required by SOTA methods, significantly advancing early SP. Extensive validation confirms its efficacy. Code is available at: https://github.com/yueliangy/MEET-Sepsis.

Method: Clustering-based approach integrated with explainable artificial intelligence (XAI) to group patients according to different sleep disorder profiles and identify key influencing factors.

Result: Experiment on anonymized real data demonstrates the effectiveness and relevance of the proposed approach.

Conclusion: The XAI-based clustering method successfully identifies sleep disorder profiles and key factors, providing an effective tool for understanding and diagnosing sleep disorders.

Abstract: Sleep disorders have a major impact on patients’ health and quality of life, but their diagnosis remains complex due to the diversity of symptoms. Today, technological advances, combined with medical data analysis, are opening new perspectives for a better understanding of these disorders. In particular, explainable artificial intelligence (XAI) aims to make AI model decisions understandable and interpretable for users. In this study, we propose a clustering-based method to group patients according to different sleep disorder profiles. By integrating an explainable approach, we identify the key factors influencing these pathologies. An experiment on anonymized real data illustrates the effectiveness and relevance of our approach.

Method: Links reasoning traces to internal activation patterns, operationalizes primitives through clustering neural activations, and applies function vector methods to derive reusable primitive vectors that can be combined through arithmetic operations.

Result: Identified shared and task-specific primitives across benchmarks (TSP, 3SAT, AIME, graph navigation) and models (Phi-4, Phi-4-Reasoning, Llama-3-8B). Reasoning finetuning enhances compositional generalization and systematic use of verification/path-generation primitives.

Conclusion: LLM reasoning is supported by a compositional geometry of algorithmic primitives that transfer cross-task and cross-model, and reasoning finetuning strengthens algorithmic generalization across domains.

Abstract: How do latent and inference time computations enable large language models (LLMs) to solve multi-step reasoning? We introduce a framework for tracing and steering algorithmic primitives that underlie model reasoning. Our approach links reasoning traces to internal activation patterns and evaluates algorithmic primitives by injecting them into residual streams and measuring their effect on reasoning steps and task performance. We consider four benchmarks: Traveling Salesperson Problem (TSP), 3SAT, AIME, and graph navigation. We operationalize primitives by clustering neural activations and labeling their matched reasoning traces. We then apply function vector methods to derive primitive vectors as reusable compositional building blocks of reasoning. Primitive vectors can be combined through addition, subtraction, and scalar operations, revealing a geometric logic in activation space. Cross-task and cross-model evaluations (Phi-4, Phi-4-Reasoning, Llama-3-8B) show both shared and task-specific primitives. Notably, comparing Phi-4 with its reasoning-finetuned variant highlights compositional generalization after finetuning: Phi-4-Reasoning exhibits more systematic use of verification and path-generation primitives. Injecting the associated primitive vectors in Phi-4-Base induces behavioral hallmarks associated with Phi-4-Reasoning. Together, these findings demonstrate that reasoning in LLMs may be supported by a compositional geometry of algorithmic primitives, that primitives transfer cross-task and cross-model, and that reasoning finetuning strengthens algorithmic generalization across domains.

Method: Theoretical analysis proving GRPO is a conservative reweighting scheme, plus controlled experiments training transformers from scratch to evaluate generalization across reasoning depth, length, tokens, and compositionality.

Result: OOD improvement occurs only when target tasks align with pretrained biases; ID gains diminish as performance saturates. GRPO cannot discover completely novel solutions.

Conclusion: GRPO is not a universal reasoning enhancer but rather sharpens pretraining biases, motivating development of algorithms that can expand beyond pretraining origins.

Abstract: Reinforcement Learning with Verifiable Rewards (RLVR), primarily driven by the Group Relative Policy Optimization (GRPO) algorithm, is a leading approach for enhancing the reasoning abilities of Large Language Models (LLMs). Despite its wide adoption, GRPO’s gains are often inconsistent; for instance, a model may show significant improvement in one reasoning domain, like mathematics, yet remain stagnant in another, such as medicine. This inconsistency raises a critical question: under what conditions does GRPO improve reasoning and generalize out-of-distribution (OOD)? We investigate this from a data distribution perspective. We first prove theoretically that GRPO is a conservative reweighting scheme, bounded by the base model’s distribution and thus unable to discover completely novel solutions. We further validate this in carefully designed controlled studies by training transformers from scratch, evaluating generalization across reasoning depth, input length, token representation, and compositionality. Our results provide a principled explanation for GRPO’s boundaries: OOD improvement emerges only when the target task aligns with the model’s pretrained biases, while gains on in-distribution (ID) tasks diminish as performance saturates. This reframes GRPO not as a universal reasoning enhancer but as a tool that sharpens pretraining biases. Our findings motivate future development of algorithms that can expand a model’s capabilities beyond its pretraining origin.

Method: End-to-end automated pipeline that: 1) Automatically selects Pareto-optimal servers, 2) Dynamically matches teacher-student pairs, 3) Adapts distillation strategies based on task complexity, 4) Optimizes cloud hosting deployment.

Result: Achieved 4x accuracy improvement over GPT-4o baseline on rare Mahjong reasoning task using reverse synthetic data and knowledge injection. Reduced latency and cost without compromising accuracy.

Conclusion: Stratos shows promise for efficient vertical-domain LLM deployment by automating the entire distillation process while meeting complex user constraints.

Abstract: The growing industrial demand for customized and cost-efficient large language models (LLMs) is fueled by the rise of vertical, domain-specific tasks and the need to optimize performance under constraints such as latency and budget. Knowledge distillation, as an efficient model compression and transfer technique, offers a feasible solution. However, existing distillation frameworks often require manual intervention and struggle to meet such complex user-defined distillation requirements. To bridge this gap, we propose Stratos, an end-to-end LLM distillation pipeline that automates server and model selection, knowledge distillation, and deployment in distributed cloud environments. Given user-defined constraints on model performance and system budget, Stratos automatically selects Pareto-optimal servers, dynamically matches teacher-student pairs, and adapts distillation strategies based on task complexity to optimize cloud hosting. Experiments show that Stratos produces a student model that achieves four times the accuracy of its GPT-4o teacher baseline on a rare, domain-specific Mahjong reasoning task with reverse synthetic data and knowledge injection. Moreover, it achieves reduced latency and cost without compromising accuracy. These results highlight its promise for vertical-domain LLM deployment.

Method: The authors propose using Kolmogorov-Smirnov (KS) Test to measure the deviation in probability distribution between training and test data, and use KS distance to quantify the distribution shift and its impact on AI agent performance.

Result: Results show that even a small KS distance of 0.02 can lead to about 50% increase in travel time at a single intersection when using a Reinforcement Learning agent, demonstrating significant performance degradation.

Conclusion: KS Test and KS distance can serve as valuable statistical tools for real-time monitoring of distribution shift in AI systems, particularly in smart transportation applications, helping AI agents better cope with distribution shifts.

Abstract: One of the major problems in Machine Learning (ML) and Artificial Intelligence (AI) is the fact that the probability distribution of the test data in the real world could deviate substantially from the probability distribution of the training data set. When this happens, the predictions of an ML system or an AI agent could involve large errors which is very troublesome and undesirable. While this is a well-known hard problem plaguing the AI and ML systems’ accuracy and reliability, in certain applications such errors could be critical for safety and reliability of AI and ML systems. One approach to deal with this problem is to monitor and measure the deviation in the probability distribution of the test data in real time and to compensate for this deviation. In this paper, we propose and explore the use of Kolmogorov-Smirnov (KS) Test for measuring the distribution shift and we show how the KS distance can be used to quantify the distribution shift and its impact on an AI agent’s performance. Our results suggest that KS distance could be used as a valuable statistical tool for monitoring and measuring the distribution shift. More specifically, it is shown that even a distance of KS=0.02 could lead to about 50% increase in the travel time at a single intersection using a Reinforcement Learning agent which is quite significant. It is hoped that the use of KS Test and KS distance in AI-based smart transportation could be an important step forward for gauging the performance degradation of an AI agent in real time and this, in turn, could help the AI agent to cope with the distribution shift in a more informed manner.

Method: Analyze PINNs training with ANaGRAM (natural-gradient approach using SVD with cutoff regularization), propose multi-cutoff adaptation strategy, develop spectral theory framework.

Result: Multi-cutoff strategy enhances ANaGRAM performance, reaching machine precision on benchmark PDE experiments.

Conclusion: Regularization is necessary for PINNs training with natural gradient methods, multi-cutoff adaptation improves performance, spectral theory provides theoretical grounding connecting to Green’s functions.

Abstract: Recent works have shown that natural gradient methods can significantly outperform standard optimizers when training physics-informed neural networks (PINNs). In this paper, we analyze the training dynamics of PINNs optimized with ANaGRAM, a natural-gradient-inspired approach employing singular value decomposition with cutoff regularization. Building on this analysis, we propose a multi-cutoff adaptation strategy that further enhances ANaGRAM’s performance. Experiments on benchmark PDEs validate the effectiveness of our method, which allows to reach machine precision on some experiments. To provide theoretical grounding, we develop a framework based on spectral theory that explains the necessity of regularization and extend previous shown connections with Green’s functions theory.

Method: Developed a layer-aware online estimator that requires only loss-to-output gradients, avoiding parameter-level and full-network gradients while maintaining ranking fidelity of sample influence.

Result: Extensive experiments across LLM pretraining, fine-tuning, and image classification show improved accuracy with significantly lower time and memory costs compared to existing methods.

Conclusion: The proposed method makes dynamic data curation efficient and scalable in practice by providing real-time influence estimation during training without computational overhead.

Abstract: Data-centric learning emphasizes curating high-quality training samples to boost performance rather than designing new architectures. A central problem is to estimate the influence of training sample efficiently. Prior studies largely focus on static influence measured on a converged model, overlooking how data valuation dynamically changes during optimization. This omission neglects the dynamic nature of sample influence during optimization, especially in deep models. To address the computational burden of frequent influence estimation, we develop a layer-aware online estimator that requires only loss-to-output gradients. This design avoids parameter-level and full-network gradients while preserving ranking fidelity. Extensive experiments across LLM pretraining, fine-tuning, and image classification show our method improves accuracy with substantially lower time and memory cost, making dynamic data curation efficient and scalable in practice.

Method: STAR consists of three components: Identity-guided State Encoder with learnable State Memory for categorical semantics, Conditional Bottleneck Adapter that generates low-rank parameters based on current state, and Numeral-State Matching module for detecting state variable anomalies.

Result: Extensive experiments on real-world datasets show that STAR improves the performance of existing TSFMs on Multivariate Time Series Anomaly Detection.

Conclusion: STAR effectively addresses the limitation of existing TSFMs in handling state variables and enhances their anomaly detection capabilities through state-aware modeling.

Abstract: While Time Series Foundation Models (TSFMs) have demonstrated remarkable success in Multivariate Time Series Anomaly Detection (MTSAD), however, in real-world industrial scenarios, many time series comprise not only numerical variables such as temperature and flow, but also numerous discrete state variables that describe the system status, such as valve on/off or day of the week. Existing TSFMs often overlook the distinct categorical nature of state variables and their critical role as conditions, typically treating them uniformly with numerical variables. This inappropriate modeling approach prevents the model from fully leveraging state information and even leads to a significant degradation in detection performance after state variables are integrated. To address this critical limitation, this paper proposes a novel STate-aware AdapteR (STAR). STAR is a plug-and-play module designed to enhance the capability of TSFMs in modeling and leveraging state variables during the fine-tuning stage. Specifically, STAR comprisesthree core components: (1) We design an Identity-guided State Encoder, whicheffectively captures the complex categorical semantics of state variables through a learnable State Memory. (2) We propose a Conditional Bottleneck Adapter, which dynamically generates low-rank adaptation parameters conditioned on the current state, thereby flexibly injecting the influence of state variables into the backbone model. (3) We also introduce a Numeral-State Matching module to more effectively detect anomalies inherent to the state variables themselves. Extensive experiments conducted on real-world datasets demonstrate that STAR can improve the performance of existing TSFMs on MTSAD.

Method: End-to-end pipeline with four components: contextual scoring network, differentiable sensor selection under budget constraints, spatio-temporal flood reconstruction/forecasting model, and differentiable decision layer. Uses Implicit Maximum Likelihood Estimation (I-MLE) for gradient-based learning over discrete sensor configurations and probabilistic decision heads.

Result: The framework enables strategic sensor placement and optimized flood forecasting models specifically tailored to minimize flood response decision regrets.

Conclusion: The proposed decision-focused approach addresses limitations of traditional methods by directly optimizing for downstream decision quality rather than intermediate metrics like information gain or forecasting errors.

Abstract: Timely and reliable decision-making is vital for flood emergency response, yet it remains severely hindered by limited and imprecise situational awareness due to various budget and data accessibility constraints. Traditional flood management systems often rely on in-situ sensors to calibrate remote sensing-based large-scale flood depth forecasting models, and further take flood depth estimates to optimize flood response decisions. However, these approaches often take fixed, decision task-agnostic strategies to decide where to put in-situ sensors (e.g., maximize overall information gain) and train flood forecasting models (e.g., minimize average forecasting errors), but overlook that systems with the same sensing gain and average forecasting errors may lead to distinct decisions. To address this, we introduce a novel decision-focused framework that strategically selects locations for in-situ sensor placement and optimize spatio-temporal flood forecasting models to optimize downstream flood response decision regrets. Our end-to-end pipeline integrates four components: a contextual scoring network, a differentiable sensor selection module under hard budget constraints, a spatio-temporal flood reconstruction and forecasting model, and a differentiable decision layer tailored to task-specific objectives. Central to our approach is the incorporation of Implicit Maximum Likelihood Estimation (I-MLE) to enable gradient-based learning over discrete sensor configurations, and probabilistic decision heads to enable differentiable approximation to various constrained disaster response tasks.

Method: Employ progressive neural networks (PNNs) for the first time in DRL-based flow control and conduct comprehensive benchmarking of conventional fine-tuning strategies using the Kuramoto-Sivashinsky system as a benchmark.

Result: PNNs enable stable and efficient knowledge transfer with consistent performance gains, while fine-tuning is sensitive to pretraining duration and prone to catastrophic forgetting. PNNs remain effective even with substantial differences between source and target environments.

Conclusion: PNNs demonstrate superior robustness and scalability for transfer learning in flow control, highlighting their potential for computationally efficient control of complex flow configurations.

Abstract: This work investigates transfer learning strategies to accelerate deep reinforcement learning (DRL) for multifidelity control of chaotic fluid flows. Progressive neural networks (PNNs), a modular architecture designed to preserve and reuse knowledge across tasks, are employed for the first time in the context of DRL-based flow control. In addition, a comprehensive benchmarking of conventional fine-tuning strategies is conducted, evaluating their performance, convergence behavior, and ability to retain transferred knowledge. The Kuramoto-Sivashinsky (KS) system is employed as a benchmark to examine how knowledge encoded in control policies, trained in low-fidelity environments, can be effectively transferred to high-fidelity settings. Systematic evaluations show that while fine-tuning can accelerate convergence, it is highly sensitive to pretraining duration and prone to catastrophic forgetting. In contrast, PNNs enable stable and efficient transfer by preserving prior knowledge and providing consistent performance gains, and are notably robust to overfitting during the pretraining phase. Layer-wise sensitivity analysis further reveals how PNNs dynamically reuse intermediate representations from the source policy while progressively adapting deeper layers to the target task. Moreover, PNNs remain effective even when the source and target environments differ substantially, such as in cases with mismatched physical regimes or control objectives, where fine-tuning strategies often result in suboptimal adaptation or complete failure of knowledge transfer. The results highlight the potential of novel transfer learning frameworks for robust, scalable, and computationally efficient flow control that can potentially be applied to more complex flow configurations.

Method: Selects optimal set of 12 baseline airfoils from UIUC database (1600+ shapes) by sequentially adding baselines that maximize design capacity, then uses Design-by-Morphing approach for reconstruction and optimization.

Result: Reconstructs 99% of database with mean absolute error below 0.005, matches performance of previous approaches using more baselines. Achieves rapid convergence in multi-objective optimization with greater hypervolume and discovers new Pareto-optimal solutions with enhanced lift-to-drag ratios.

Conclusion: AirDbM demonstrates superior efficiency and adaptability, particularly for reinforcement learning applications, indicating broader potential of Design-by-Morphing in machine learning-driven design.

Abstract: Effective airfoil geometry optimization requires exploring a diverse range of designs using as few design variables as possible. This study introduces AirDbM, a Design-by-Morphing (DbM) approach specialized for airfoil optimization that systematically reduces design-space dimensionality. AirDbM selects an optimal set of 12 baseline airfoils from the UIUC airfoil database, which contains over 1,600 shapes, by sequentially adding the baseline that most increases the design capacity. With these baselines, AirDbM reconstructs 99 % of the database with a mean absolute error below 0.005, which matches the performance of a previous DbM approach that used more baselines. In multi-objective aerodynamic optimization, AirDbM demonstrates rapid convergence and achieves a Pareto front with a greater hypervolume than that of the previous larger-baseline study, where new Pareto-optimal solutions are discovered with enhanced lift-to-drag ratios at moderate stall tolerances. Furthermore, AirDbM demonstrates outstanding adaptability for reinforcement learning (RL) agents in generating airfoil geometry when compared to conventional airfoil parameterization methods, implying the broader potential of DbM in machine learning-driven design.

Method: Feature-driven RL approach using Markov Decision Process with PPO, integrating data-driven features into state representation with predominantly linear, interpretable policy.

Result: Strategy consistently outperforms benchmark baselines across diverse scenarios, shows rapid convergence, real-time inference, and transparent decision rules with learned weights highlighting market microstructure importance.

Conclusion: Feature-driven RL offers a practical, data-efficient, and operationally deployable pathway for active intraday participation by PV producers.

Abstract: Photovoltaic (PV) operators face substantial uncertainty in generation and short-term electricity prices. Continuous intraday markets enable producers to adjust their positions in real time, potentially improving revenues and reducing imbalance costs. We propose a feature-driven reinforcement learning (RL) approach for PV intraday trading that integrates data-driven features into the state and learns bidding policies in a sequential decision framework. The problem is cast as a Markov Decision Process with a reward that balances trading profit and imbalance penalties and is solved with Proximal Policy Optimization (PPO) using a predominantly linear, interpretable policy. Trained on historical market data and evaluated out-of-sample, the strategy consistently outperforms benchmark baselines across diverse scenarios. Extensive validation shows rapid convergence, real-time inference, and transparent decision rules. Learned weights highlight the central role of market microstructure and historical features. Taken together, these results indicate that feature-driven RL offers a practical, data-efficient, and operationally deployable pathway for active intraday participation by PV producers.

Method: Proposes IB-FT (Information Bottleneck-guided Fine-Tuning) that applies an IB penalty on hidden representations to compress spurious memorized features while preserving task-relevant information.

Result: IB-FT substantially alleviates the memorization barrier, improves top-1 performance (Pass@1), and yields more stable gains under stricter multi-sample metrics (Pass@k(m)) compared to conventional fine-tuning on two code benchmarks.

Conclusion: The information bottleneck approach effectively overcomes the memorization barrier in code generation fine-tuning, leading to better generalization and more stable performance improvements.

Abstract: Adapting pretrained large language models (LLMs) to code domains via supervised fine-tuning (FT) has been commonly used for code generation. However, we identify a previously underappreciated failure mode, the memorization barrier, where strong memorization of downstream code data in the base model could trap optimization and prevent the standard FT from effectively acquiring new, generalizable code knowledge. To overcome this barrier, we propose the information bottleneck (IB)-guided fine-tuning, termed IB-FT, which applies an IB penalty on hidden representations of the code data to compress spurious, memorized features while preserving task-relevant information. Extensive experiments on two code benchmarks (OriGen and Evol-CodeAlpaca-V1) show that IB-FT substantially alleviates the memorization barrier, improves top-1 performance (Pass@$1$), and yields far more stable gains under the stricter multi-sample metric Pass@$k^{(m)}$ (a problem counts as solved only if at least $m$ of $k$ samples pass unit tests) compared with conventional FT.

Method: Analyzed averaging-based aggregators, cast robust federated inference with non-linear aggregators as adversarial ML problem, and proposed DeepSet aggregation model with adversarial training and test-time robust aggregation.

Result: The composition of adversarial training and test-time robust aggregation achieved significant improvements, surpassing existing robust aggregation methods by 4.7-22.2% accuracy across diverse benchmarks.

Conclusion: The proposed approach effectively robustifies federated inference against attacks, addressing a critical gap in the security of federated learning systems.

Abstract: Federated inference, in the form of one-shot federated learning, edge ensembles, or federated ensembles, has emerged as an attractive solution to combine predictions from multiple models. This paradigm enables each model to remain local and proprietary while a central server queries them and aggregates predictions. Yet, the robustness of federated inference has been largely neglected, leaving them vulnerable to even simple attacks. To address this critical gap, we formalize the problem of robust federated inference and provide the first robustness analysis of this class of methods. Our analysis of averaging-based aggregators shows that the error of the aggregator is small either when the dissimilarity between honest responses is small or the margin between the two most probable classes is large. Moving beyond linear averaging, we show that problem of robust federated inference with non-linear aggregators can be cast as an adversarial machine learning problem. We then introduce an advanced technique using the DeepSet aggregation model, proposing a novel composition of adversarial training and test-time robust aggregation to robustify non-linear aggregators. Our composition yields significant improvements, surpassing existing robust aggregation methods by 4.7 - 22.2% in accuracy points across diverse benchmarks.

Method: PolyConFM uses conformation-centric generative pretraining by decomposing polymer conformations into sequences of local conformations. It employs masked autoregressive modeling to reconstruct local conformations and generate orientation transformations to recover full polymer conformations.

Result: PolyConFM consistently outperforms representative task-specific methods on diverse downstream tasks, demonstrating superior performance across various polymer science applications.

Conclusion: PolyConFM provides polymer science with a universal and powerful foundation model that effectively supports diverse downstream tasks through conformation-centric generative pretraining.

Abstract: Polymers, macromolecules formed from covalently bonded monomers, underpin countless technologies and are indispensable to modern life. While deep learning is advancing polymer science, existing methods typically represent the whole polymer solely through monomer-level descriptors, overlooking the global structural information inherent in polymer conformations, which ultimately limits their practical performance. Moreover, this field still lacks a universal foundation model that can effectively support diverse downstream tasks, thereby severely constraining progress. To address these challenges, we introduce PolyConFM, the first polymer foundation model that unifies polymer modeling and design through conformation-centric generative pretraining. Recognizing that each polymer conformation can be decomposed into a sequence of local conformations (i.e., those of its repeating units), we pretrain PolyConFM under the conditional generation paradigm, reconstructing these local conformations via masked autoregressive (MAR) modeling and further generating their orientation transformations to recover the corresponding polymer conformation. Besides, we construct the first high-quality polymer conformation dataset via molecular dynamics simulations to mitigate data sparsity, thereby enabling conformation-centric pretraining. Experiments demonstrate that PolyConFM consistently outperforms representative task-specific methods on diverse downstream tasks, equipping polymer science with a universal and powerful tool.

Method: Process imperfect multimodal clinical data, decompose into probabilistic independent latent sources, and train task-specific causal models to estimate individual causal effects.

Result: Successfully applied the approach in two real-world applications, demonstrating its versatility and utility for medical discovery at scale.

Conclusion: The proposed pipeline provides an accessible and generalizable method for causal discovery and effect quantification in electronic health records, showing promise for scalable medical research.

Abstract: We provide an accessible description of a peer-reviewed generalizable causal machine learning pipeline to (i) discover latent causal sources of large-scale electronic health records observations, and (ii) quantify the source causal effects on clinical outcomes. We illustrate how imperfect multimodal clinical data can be processed, decomposed into probabilistic independent latent sources, and used to train taskspecific causal models from which individual causal effects can be estimated. We summarize the findings of the two real-world applications of the approach to date as a demonstration of its versatility and utility for medical discovery at scale.

Method: Uses diffusion models to generate high-fidelity bot accounts by reconstructing existing data with minor modifications, then employs Multi-Agent Reinforcement Learning (MARL) to simulate adversarial bot behavior across different account categories with hierarchical state abstraction for acceleration.

Result: Extensive experiments show the framework effectively undermines GNN-based bot detector performance.

Conclusion: RoBCtrl successfully demonstrates vulnerabilities in current GNN-based social bot detection systems through adversarial attacks combining diffusion models and multi-agent reinforcement learning.

Abstract: Social networks have become a crucial source of real-time information for individuals. The influence of social bots within these platforms has garnered considerable attention from researchers, leading to the development of numerous detection technologies. However, the vulnerability and robustness of these detection methods is still underexplored. Existing Graph Neural Network (GNN)-based methods cannot be directly applied due to the issues of limited control over social agents, the black-box nature of bot detectors, and the heterogeneity of bots. To address these challenges, this paper proposes the first adversarial multi-agent Reinforcement learning framework for social Bot control attacks (RoBCtrl) targeting GNN-based social bot detectors. Specifically, we use a diffusion model to generate high-fidelity bot accounts by reconstructing existing account data with minor modifications, thereby evading detection on social platforms. To the best of our knowledge, this is the first application of diffusion models to mimic the behavior of evolving social bots effectively. We then employ a Multi-Agent Reinforcement Learning (MARL) method to simulate bots adversarial behavior. We categorize social accounts based on their influence and budget. Different agents are then employed to control bot accounts across various categories, optimizing the attachment strategy through reinforcement learning. Additionally, a hierarchical state abstraction based on structural entropy is designed to accelerate the reinforcement learning. Extensive experiments on social bot detection datasets demonstrate that our framework can effectively undermine the performance of GNN-based detectors.

Method: Uses action-conditioned codebook with codewords derived from continuous attractor neural networks, dynamically selecting codewords based on actions to perform spatiotemporal compression.

Result: GCQ effectively compresses sequences into compact representations that support long-horizon prediction, goal-directed planning, and inverse modeling across diverse tasks.

Conclusion: GCQ provides both a practical computational tool for efficient sequence modeling and theoretical insights into how grid-like codes form in neural systems.

Abstract: We propose Grid-like Code Quantization (GCQ), a brain-inspired method for compressing observation-action sequences into discrete representations using grid-like patterns in attractor dynamics. Unlike conventional vector quantization approaches that operate on static inputs, GCQ performs spatiotemporal compression through an action-conditioned codebook, where codewords are derived from continuous attractor neural networks and dynamically selected based on actions. This enables GCQ to jointly compress space and time, serving as a unified world model. The resulting representation supports long-horizon prediction, goal-directed planning, and inverse modeling. Experiments across diverse tasks demonstrate GCQ’s effectiveness in compact encoding and downstream performance. Our work offers both a computational tool for efficient sequence modeling and a theoretical perspective on the formation of grid-like codes in neural systems.

Method: Proposes AMS-Quant with two techniques: (1) Mantissa-bit Sharing - groups weights to share least significant mantissa bits, enabling non-integer bitwidths; (2) Adaptive Searching - offline optimization to minimize accuracy loss from sharing. Also implements efficient CUDA kernels.

Result: Successfully quantizes models to FP5.33-e2m3 and FP4.25-e2m2 formats, achieving 2.8x and 3.2x speedup over FP16 inference respectively, with negligible accuracy loss on large-scale datasets and models.

Conclusion: AMS-Quant demonstrates that non-integer floating-point quantization through mantissa-bit sharing and adaptive optimization can significantly accelerate LLM inference while preserving model accuracy, advancing beyond traditional integer bitwidth quantization approaches.

Abstract: Large language models (LLMs) have demonstrated remarkable capabilities in various kinds of tasks, while the billion or even trillion parameters bring storage and efficiency bottlenecks for inference. Quantization, particularly floating-point quantization, is known to be capable of speeding up LLM inference by reducing memory footprint and data movement during the inference process. For the first time, we advance the floating-point quantization exploration from integer bitwidths to non-integer bit-widths, namely AMS-Quant, to further approach the quantization sweet spot. AMS-Quant incorporates two novel techniques to put it into effect: (1) it proposes Mantissa-bit Sharing, which groups k quantized weights and lets them share the least significant mantissa bit, allowing us to further approach the minimum quantization bit-width without accuracy loss. (2) It introduces Adaptive Searching, which employs an offline optimization strategy to minimize the accuracy degradation introduced by sharing. Moreover, AMS-Quant is also prototyped as efficient CUDA Linear kernels, which translates memory savings into wall-clock latency reduction by reducing memory access. Extensive experiments on large-scale datasets and models show that AMS-Quant can quantize the model to FP-5.33-e2m3 and FP4.25-e2m2, and significantly speed up the LLM decoding over FP16 inference (2.8x and 3.2x), with negligible accuracy loss.

Method: Systematic exploration of applications via native accessibility APIs, organizing elements into hierarchical GUI graphs, using specialized interaction handlers for comprehensive coverage, and constructing a large-scale dataset with screenshots and action traces.

Result: Created GUIrilla-Task dataset with 27,171 tasks across 1,108 macOS apps; tuning LLM agents on this data significantly improves UI task performance, outperforming synthetic baselines while using 97% less data.

Conclusion: GUIrilla addresses critical data collection challenges in GUI automation and enables more effective desktop autonomy through systematic data collection and open-source tools.

Abstract: Autonomous agents capable of operating complex graphical user interfaces (GUIs) have the potential to transform desktop automation. While recent advances in large language models (LLMs) have significantly improved UI understanding, navigating full-window, multi-application desktop environments remains a major challenge. Data availability is limited by costly manual annotation, closed-source datasets and surface-level synthetic pipelines. We introduce GUIrilla, an automated scalable framework that systematically explores applications via native accessibility APIs to address the critical data collection challenge in GUI automation. Our framework focuses on macOS - an ecosystem with limited representation in current UI datasets - though many of its components are designed for broader cross-platform applicability. GUIrilla organizes discovered interface elements and crawler actions into hierarchical GUI graphs and employs specialized interaction handlers to achieve comprehensive application coverage. Using the application graphs from GUIrilla crawler, we construct and release GUIrilla-Task, a large-scale dataset of 27,171 functionally grounded tasks across 1,108 macOS applications, each annotated with full-desktop and window-level screenshots, accessibility metadata, and semantic action traces. Empirical results show that tuning LLM-based agents on GUIrilla-Task significantly improves performance on downstream UI tasks, outperforming synthetic baselines on the ScreenSpot Pro benchmark while using 97% less data. We also release macapptree, an open-source library for reproducible collection of structured accessibility metadata, along with the full GUIrilla-Task dataset, the manually verified GUIrilla-Gold benchmark, and the framework code to support open research in desktop autonomy.

Method: The paper reviews GNN-based approaches including STGCN, GraphWaveNet, STWave, and D2STGNN that capture spatial dependencies through graph convolutions and temporal patterns. It also discusses early attempts using manually engineered event features and incident effect scores.

Result: GNN models have achieved impressive performance on standard traffic datasets, particularly in capturing periodic regularities. However, manual event feature engineering approaches suffer from heavy reliance on domain expertise, poor generalization to unknown events, and loss of semantic details.

Conclusion: While GNNs are effective for regular traffic patterns, current event-handling approaches using manual feature engineering are insufficient. There is a need for more robust methods that can better incorporate event information without heavy reliance on domain knowledge.

Abstract: Accurate traffic forecasting is a core technology for building Intelligent Transportation Systems (ITS), enabling better urban resource allocation and improved travel experiences. With growing urbanization, traffic congestion has intensified, highlighting the need for reliable and responsive forecasting models. In recent years, deep learning, particularly Graph Neural Networks (GNNs), has emerged as the mainstream paradigm in traffic forecasting. GNNs can effectively capture complex spatial dependencies in road network topology and dynamic temporal evolution patterns in traffic flow data. Foundational models such as STGCN and GraphWaveNet, along with more recent developments including STWave and D2STGNN, have achieved impressive performance on standard traffic datasets. These approaches incorporate sophisticated graph convolutional structures and temporal modeling mechanisms, demonstrating particular effectiveness in capturing and forecasting traffic patterns characterized by periodic regularities. To address this challenge, researchers have explored various ways to incorporate event information. Early attempts primarily relied on manually engineered event features. For instance, some approaches introduced manually defined incident effect scores or constructed specific subgraphs for different event-induced traffic conditions. While these methods somewhat enhance responsiveness to specific events, their core drawback lies in a heavy reliance on domain experts’ prior knowledge, making generalization to diverse and complex unknown events difficult, and low-dimensional manual features often lead to the loss of rich semantic details.

Method: Systematic evaluation of five recent time series foundation models and two competitive baselines, assessing model calibration, effects of varying prediction heads, and calibration under long-term autoregressive forecasting.

Result: Time series foundation models are consistently better calibrated than baseline models and tend not to be systematically over- or under-confident.

Conclusion: Time series foundation models demonstrate superior calibration properties compared to traditional deep learning approaches, avoiding the overconfidence issues commonly seen in other deep learning models.

Abstract: The recent development of foundation models for time series data has generated considerable interest in using such models across a variety of applications. Although foundation models achieve state-of-the-art predictive performance, their calibration properties remain relatively underexplored, despite the fact that calibration can be critical for many practical applications. In this paper, we investigate the calibration-related properties of five recent time series foundation models and two competitive baselines. We perform a series of systematic evaluations assessing model calibration (i.e., over- or under-confidence), effects of varying prediction heads, and calibration under long-term autoregressive forecasting. We find that time series foundation models are consistently better calibrated than baseline models and tend not to be either systematically over- or under-confident, in contrast to the overconfidence often seen in other deep learning models.

Method: Proposes a hierarchical graph neural network that exploits electrical topology and physical features, trained and evaluated on SMART-DS datasets across multiple substations and DER penetration scenarios.

Result: Achieves up to 2 times lower RMSE than alternative data-driven models and maintains high accuracy with only 1% measurement coverage.

Conclusion: GNNs have potential to enable scalable, reproducible, and data-driven voltage monitoring for distribution systems.

Abstract: Accurate voltage estimation in distribution networks is critical for real-time monitoring and increasing the reliability of the grid. As DER penetration and distribution level voltage variability increase, robust distribution system state estimation (DSSE) has become more essential to maintain safe and efficient operations. Traditional DSSE techniques, however, struggle with sparse measurements and the scale of modern feeders, limiting their scalability to large networks. This paper presents a hierarchical graph neural network for substation-level voltage estimation that exploits both electrical topology and physical features, while remaining robust to the low observability levels common to real-world distribution networks. Leveraging the public SMART-DS datasets, the model is trained and evaluated on thousands of buses across multiple substations and DER penetration scenarios. Comprehensive experiments demonstrate that the proposed method achieves up to 2 times lower RMSE than alternative data-driven models, and maintains high accuracy with as little as 1% measurement coverage. The results highlight the potential of GNNs to enable scalable, reproducible, and data-driven voltage monitoring for distribution systems.

Method: Uses topology-aware Graph Neural Network with local attention and two-level DC feature integration, trained with physics-informed loss to enforce AC power-flow feasibility and operational limits.

Result: 25% lower MSE, up to 3X reduction in feasibility error, and up to 13X runtime speedup compared to conventional AC OPF solvers, with maintained accuracy under N-1 contingencies.

Conclusion: Residual learning provides a practical and scalable bridge between linear approximations and AC-feasible OPF, enabling near real-time operational decision making.

Abstract: Solving the nonlinear AC optimal power flow (AC OPF) problem remains a major computational bottleneck for real-time grid operations. In this paper, we propose a residual learning paradigm that uses fast DC optimal power flow (DC OPF) solutions as a baseline, and learns only the nonlinear corrections required to provide the full AC-OPF solution. The method utilizes a topology-aware Graph Neural Network with local attention and two-level DC feature integration, trained using a physics-informed loss that enforces AC power-flow feasibility and operational limits. Evaluations on OPFData for 57-, 118-, and 2000-bus systems show around 25% lower MSE, up to 3X reduction in feasibility error, and up to 13X runtime speedup compared to conventional AC OPF solvers. The model maintains accuracy under N-1 contingencies and scales efficiently to large networks. These results demonstrate that residual learning is a practical and scalable bridge between linear approximations and AC-feasible OPF, enabling near real-time operational decision making.

Method: Uses an integer programming formulation to strategically identify critical parameters for transmission and integrates this into a sparse aggregation scheme for communication-efficient PFL.

Result: Comprehensive evaluations on standard image classification benchmarks under varied non-IID conditions demonstrate competitive performance relative to state-of-the-art methods with quantifiable communication reduction through sparse aggregation.

Conclusion: FedPURIN establishes a new paradigm for communication-efficient PFL that is particularly advantageous for edge intelligence systems operating with heterogeneous data sources.

Abstract: Personalized Federated Learning (PFL) has emerged as a critical research frontier addressing data heterogeneity issue across distributed clients. Novel model architectures and collaboration mechanisms are engineered to accommodate statistical disparities while producing client-specific models. Parameter decoupling represents a promising paradigm for maintaining model performance in PFL frameworks. However, the communication efficiency of many existing methods remains suboptimal, sustaining substantial communication burdens that impede practical deployment. To bridge this gap, we propose Federated Learning with Programmed Update and Reduced INformation (FedPURIN), a novel framework that strategically identifies critical parameters for transmission through an integer programming formulation. This mathematically grounded strategy is seamlessly integrated into a sparse aggregation scheme, achieving a significant communication reduction while preserving the efficacy. Comprehensive evaluations on standard image classification benchmarks under varied non-IID conditions demonstrate competitive performance relative to state-of-the-art methods, coupled with quantifiable communication reduction through sparse aggregation. The framework establishes a new paradigm for communication-efficient PFL, particularly advantageous for edge intelligence systems operating with heterogeneous data sources.

Method: MNO explicitly decomposes information across three scales: global dimension-shrinkage attention for long-range dependencies, local graph attention for neighborhood interactions, and micro point-wise attention for fine details.

Result: MNO consistently outperforms state-of-the-art baselines across four diverse benchmarks, reducing prediction errors by 5% to 40% and demonstrating improved robustness in challenging 3D CFD problems with up to 300K points.

Conclusion: The explicit multiscale design is crucial for neural operators, and MNO establishes a scalable framework for learning complex fluid dynamics on irregular domains.

Abstract: Neural operators have emerged as a powerful data-driven paradigm for solving Partial Differential Equations (PDEs), offering orders-of-magnitude acceleration over traditional solvers. However, existing approaches still suffer from limited accuracy and scalability, particularly on irregular domains where fluid flows exhibit rich multiscale structures. In this work, we introduce the Multiscale Neural Operator (MNO), a new architecture for Computational Fluid Dynamics (CFD) on three-dimensional (3D) unstructured point clouds. MNO explicitly decomposes information across three scales: a global dimension-shrinkage attention module for long-range dependencies, a local graph attention module for neighborhood-level interactions, and a micro point-wise attention module for fine-grained details. This design preserves multiscale inductive biases while remaining computationally efficient. We evaluate MNO on four diverse benchmarks, covering both steady-state and unsteady flow scenarios with up to 300K points. Across all tasks, MNO consistently outperforms state-of-the-art baselines, reducing prediction errors by 5% to 40% and demonstrating improved robustness in challenging 3D CFD problems. Our results highlight the importance of explicit multiscale design for neural operators and establish MNO as a scalable framework for learning complex fluid dynamics on irregular domains.

Method: Using Random Matrix Theory to analyze the spectral density evolution of the self-attention matrix V, applying PL (Power Law) fit to identify training stages, and developing spectral-based metrics for convergence detection.

Result: The spectral density of the shallow self-attention matrix consistently evolves into heavy-tailed distribution, revealing three distinct training stages: structural exploration, heavy-tailed structure stabilization, and convergence saturation.

Conclusion: Random Matrix Theory provides effective tools for monitoring Transformer training progression, with spectral signatures offering consistent validation-free criteria for early stopping decisions.

Abstract: This work introduces a novel theoretical framework grounded in Random Matrix Theory (RMT) for analyzing Transformer training dynamics. We focus on the underlying mechanisms that drive performance improvements and derive principled early-stopping criteria. Empirically, we observe that the spectral density of the shallow self-attention matrix V consistently evolves into a heavy-tailed distribution. Utilizing the PL (Power Law) fit to this matrix as a probe, we demarcate training into three stages: structural exploration, heavy-tailed structure stabilization, and convergence saturation. This staging provides guidance for preliminary stopping decisions. Crucially, we propose two consistent and validation-free criteria: a quantitative metric for heavy-tailed dynamics and a novel spectral signature indicative of convergence. The strong alignment between these criteria highlights the utility of RMT for monitoring and diagnosing the progression of Transformer model training.

Method: Uses Frobenius distance between theoretical and dequantized outputs as objective, formulates explicit QUBO with binary variables for rounding choices, and decomposes into independent subproblems of size f+1.

Result: Evaluated on MNIST, Fashion-MNIST, EMNIST, and CIFAR-10 across int8 to int1 precisions, showing improved performance over traditional round-to-nearest quantization.

Conclusion: The proposed ADAROUND-based QUBO formulation with problem decomposition enables efficient neural network quantization with better accuracy preservation than conventional methods.

Abstract: This work introduces a post-training quantization (PTQ) method for dense neural networks via a novel ADAROUND-based QUBO formulation. Using the Frobenius distance between the theoretical output and the dequantized output (before the activation function) as the objective, an explicit QUBO whose binary variables represent the rounding choice for each weight and bias is obtained. Additionally, by exploiting the structure of the coefficient QUBO matrix, the global problem can be exactly decomposed into $n$ independent subproblems of size $f+1$, which can be efficiently solved using some heuristics such as simulated annealing. The approach is evaluated on MNIST, Fashion-MNIST, EMNIST, and CIFAR-10 across integer precisions from int8 to int1 and compared with a round-to-nearest traditional quantization methodology.

Method: Uses dual distillation strategies: teacher-student distillation from a biased model to retain accurate preference knowledge for factual test performance, and self-distillation with reliability filtering to iteratively refine knowledge for counterfactual test performance.

Result: Comprehensive experiments validate BPL’s effectiveness in both factual and counterfactual tests, showing improved performance across both test environments.

Conclusion: BPL successfully addresses the dual challenge of performing well in both factual and counterfactual test environments through its bias-adaptive preference distillation approach with dual distillation strategies.

Abstract: Recommender systems suffer from biases that cause the collected feedback to incompletely reveal user preference. While debiasing learning has been extensively studied, they mostly focused on the specialized (called counterfactual) test environment simulated by random exposure of items, significantly degrading accuracy in the typical (called factual) test environment based on actual user-item interactions. In fact, each test environment highlights the benefit of a different aspect: the counterfactual test emphasizes user satisfaction in the long-terms, while the factual test focuses on predicting subsequent user behaviors on platforms. Therefore, it is desirable to have a model that performs well on both tests rather than only one. In this work, we introduce a new learning framework, called Bias-adaptive Preference distillation Learning (BPL), to gradually uncover user preferences with dual distillation strategies. These distillation strategies are designed to drive high performance in both factual and counterfactual test environments. Employing a specialized form of teacher-student distillation from a biased model, BPL retains accurate preference knowledge aligned with the collected feedback, leading to high performance in the factual test. Furthermore, through self-distillation with reliability filtering, BPL iteratively refines its knowledge throughout the training process. This enables the model to produce more accurate predictions across a broader range of user-item combinations, thereby improving performance in the counterfactual test. Comprehensive experiments validate the effectiveness of BPL in both factual and counterfactual tests. Our implementation is accessible via: https://github.com/SeongKu-Kang/BPL.

Method: Proposes CONEC-LoRA with: 1) Consolidation between task-shared and task-specific LoRAs, 2) Stochastic classifier with parameters sampled from distribution, 3) Auxiliary network for optimal LoRA prediction using different-depth structure with local classifiers, 4) Ball-generator loss and transformation module to address synthetic sample bias.

Result: CONEC-LoRA outperforms prior methods by over 5% margins across 4 popular benchmark problems, demonstrating significant improvements in domain incremental learning performance.

Conclusion: The proposed CONEC-LoRA framework effectively addresses DIL challenges by leveraging shared knowledge, stochastic classification, and intermediate representations, achieving state-of-the-art performance in preventing catastrophic forgetting while adapting to new domains.

Abstract: Domain Incremental Learning (DIL) is a continual learning sub-branch that aims to address never-ending arrivals of new domains without catastrophic forgetting problems. Despite the advent of parameter-efficient fine-tuning (PEFT) approaches, existing works create task-specific LoRAs overlooking shared knowledge across tasks. Inaccurate selection of task-specific LORAs during inference results in significant drops in accuracy, while existing works rely on linear or prototype-based classifiers, which have suboptimal generalization powers. Our paper proposes continual knowledge consolidation low rank adaptation (CONEC-LoRA) addressing the DIL problems. CONEC-LoRA is developed from consolidations between task-shared LORA to extract common knowledge and task-specific LORA to embrace domain-specific knowledge. Unlike existing approaches, CONEC-LoRA integrates the concept of a stochastic classifier whose parameters are sampled from a distribution, thus enhancing the likelihood of correct classifications. Last but not least, an auxiliary network is deployed to optimally predict the task-specific LoRAs for inferences and implements the concept of a different-depth network structure in which every layer is connected with a local classifier to take advantage of intermediate representations. This module integrates the ball-generator loss and transformation module to address the synthetic sample bias problem. Our rigorous experiments demonstrate the advantage of CONEC-LoRA over prior arts in 4 popular benchmark problems with over 5% margins.

Method: Proposes password reuse relations as edges in a website graph and uses graph neural networks for link prediction. Incorporates federated learning to preserve user privacy by eliminating cross-administrator data sharing.

Result: Achieves F1-score of 0.9153 on real-world dataset of 360M breached accounts from 22,378 websites. FL-based GNN shows 4-11% performance improvement over other state-of-the-art GNN models.

Conclusion: The framework effectively predicts credential stuffing risks, provides actionable risk scores for password reuse likelihood, and maintains user privacy through federated learning approach.

Abstract: Credential stuffing attacks have caused significant harm to online users who frequently reuse passwords across multiple websites. While prior research has attempted to detect users with reused passwords or identify malicious login attempts, existing methods often compromise usability by restricting password creation or website access, and their reliance on complex account-sharing mechanisms hinders real-world deployment. To address these limitations, we propose PassREfinder-FL, a novel framework that predicts credential stuffing risks across websites. We introduce the concept of password reuse relations – defined as the likelihood of users reusing passwords between websites – and represent them as edges in a website graph. Using graph neural networks (GNNs), we perform a link prediction task to assess credential reuse risk between sites. Our approach scales to a large number of arbitrary websites by incorporating public website information and linking newly observed websites as nodes in the graph. To preserve user privacy, we extend PassREfinder-FL with a federated learning (FL) approach that eliminates the need to share user sensitive information across administrators. Evaluation on a real-world dataset of 360 million breached accounts from 22,378 websites shows that PassREfinder-FL achieves an F1-score of 0.9153 in the FL setting. We further validate that our FL-based GNN achieves a 4-11% performance improvement over other state-of-the-art GNN models through an ablation study. Finally, we demonstrate that the predicted results can be used to quantify password reuse likelihood as actionable risk scores.

Method: Extended EP learning to discrete and continuous complex-valued wave systems, replacing trainable inter-node connections with trainable local potentials in systems without well-defined nodes.

Result: Tested in driven-dissipative exciton-polariton condensates; numerical studies on logical tasks and handwritten-digit recognition showed stable convergence.

Conclusion: Establishes practical route to in-situ learning in physical systems where system control is restricted to local parameters.

Abstract: Backpropagation learning algorithm, the workhorse of modern artificial intelligence, is notoriously difficult to implement in physical neural networks. Equilibrium Propagation (EP) is an alternative with comparable efficiency and strong potential for in-situ training. We extend EP learning to both discrete and continuous complex-valued wave systems. In contrast to previous EP implementations, our scheme is valid in the weakly dissipative regime, and readily applicable to a wide range of physical settings, even without well defined nodes, where trainable inter-node connections can be replaced by trainable local potential. We test the method in driven-dissipative exciton-polariton condensates governed by generalized Gross-Pitaevskii dynamics. Numerical studies on standard benchmarks, including a simple logical task and handwritten-digit recognition, demonstrate stable convergence, establishing a practical route to in-situ learning in physical systems in which system control is restricted to local parameters.

Method: Uses de-redundant homo-heterogeneous factorization to separate modality representations, and distribution-aligned self-distillation for semantic recovery through bidirectional knowledge transfer.

Result: Comprehensive experiments show FSRF significantly outperforms previous methods when dealing with uncertain missing modalities.

Conclusion: FSRF effectively mitigates the modality missing problem in MSA through factorization-guided semantic recovery, improving model robustness in real-world scenarios.

Abstract: In recent years, Multimodal Sentiment Analysis (MSA) has become a research hotspot that aims to utilize multimodal data for human sentiment understanding. Previous MSA studies have mainly focused on performing interaction and fusion on complete multimodal data, ignoring the problem of missing modalities in real-world applications due to occlusion, personal privacy constraints, and device malfunctions, resulting in low generalizability. To this end, we propose a Factorization-guided Semantic Recovery Framework (FSRF) to mitigate the modality missing problem in the MSA task. Specifically, we propose a de-redundant homo-heterogeneous factorization module that factorizes modality into modality-homogeneous, modality-heterogeneous, and noisy representations and design elaborate constraint paradigms for representation learning. Furthermore, we design a distribution-aligned self-distillation module that fully recovers the missing semantics by utilizing bidirectional knowledge transfer. Comprehensive experiments on two datasets indicate that FSRF has a significant performance advantage over previous methods with uncertain missing modalities.

Method: Uses parameter efficient fine-tuning via LoRA with gating mechanism. Each candidate edit is evaluated against stability budget using three metrics: Exact Match drop, bits increase, or KL divergence. Updates exceeding thresholds are rescaled through clipping or rejected.

Result: Experiments on Qwen-2.5-7B show gating effectively mitigates forgetting while preserving adaptability. EM-based gating achieved highest cumulative performance in short continual learning sequences. Different gating strategies achieve comparable distribution shift but different accuracy outcomes.

Conclusion: STABLE provides a principled method for continual model editing, enabling LLMs to integrate new knowledge while maintaining reliability, with gating design being crucial for continual adaptation.

Abstract: Large language models (LLMs) increasingly require mechanisms for continual adaptation without full retraining. However, sequential updates can lead to catastrophic forgetting, where new edits degrade previously acquired knowledge. This work presents STABLE, a gated continual self editing framework that constrains forgetting during sequential updates using parameter efficient fine tuning via Low Rank Adaptation (LoRA; see arXiv:2106.09685). Each candidate edit is evaluated against a stability budget using one of three metrics: (i) Exact Match (EM) drop, capturing factual accuracy loss; (ii) bits increase, reflecting reduced model confidence; and (iii) KL divergence, quantifying distributional drift between the base and adapted models. If a threshold is exceeded, the LoRA update is rescaled through a clipping procedure or rejected. Experiments on the Qwen-2.5-7B model show that gating effectively mitigates forgetting while preserving adaptability. EM based gating achieved the highest cumulative performance in short continual learning sequences. Our results show that different gating strategies can achieve comparable distribution shift (measured by KL divergence) while producing different accuracy outcomes, highlighting the importance of gating design in continual adaptation. This approach offers a principled method for continual model editing, enabling LLMs to integrate new knowledge while maintaining reliability. Code: https://github.com/Bhoy1/STABLE