Method: Constructed dataset with 140,750 street view images and 984,024 social media posts, developed reaction index using object detection and NLP, analyzed sentiment using regression, image segmentation, and word frequency based on land-use distribution.

Result: Perception showed shift toward evenly distributed positive sentiment, while opinion showed more extreme changes. Significant disparities found between perception and opinion sentiments, with relationships to dense buildings and pedestrian presence.

Conclusion: The study provides insights into sentiment inconsistencies before/after pandemic, offering directions for environmental management and urban renewal strategies.

Abstract: The ascension of social media platforms has transformed our understanding of urban environments, giving rise to nuanced variations in sentiment reaction embedded within human perception and opinion, and challenging existing multidimensional sentiment analysis approaches in urban studies. This study presents novel methodologies for identifying and elucidating sentiment inconsistency, constructing a dataset encompassing 140,750 Baidu and Tencent Street view images to measure perceptions, and 984,024 Weibo social media text posts to measure opinions. A reaction index is developed, integrating object detection and natural language processing techniques to classify sentiment in Beijing Second Ring for 2016 and 2022. Classified sentiment reaction is analysed and visualized using regression analysis, image segmentation, and word frequency based on land-use distribution to discern underlying factors. The perception affective reaction trend map reveals a shift toward more evenly distributed positive sentiment, while the opinion affective reaction trend map shows more extreme changes. Our mismatch map indicates significant disparities between the sentiments of human perception and opinion of urban areas over the years. Changes in sentiment reactions have significant relationships with elements such as dense buildings and pedestrian presence. Our inconsistent maps present perception and opinion sentiments before and after the pandemic and offer potential explanations and directions for environmental management, in formulating strategies for urban renewal.

Method: Built HaystackCraft on the full English Wikipedia hyperlink network with multi-hop questions, evaluating heterogeneous retrieval strategies (sparse, dense, hybrid, graph-based) and extending to dynamic agentic settings with query refinement, reflection, and stopping decisions.

Result: Experiments with 15 models showed: (1) stronger dense retrievers create harder distractors but graph-based reranking helps; (2) even advanced models suffer cascading failures from self-generated distractors and struggle with early stopping in agentic tests.

Conclusion: HaystackCraft reveals persistent challenges in agentic long-context reasoning and serves as a valuable testbed for future progress in robust long-context LLM development.

Abstract: Modern long-context large language models (LLMs) perform well on synthetic “needle-in-a-haystack” (NIAH) benchmarks, but such tests overlook how noisy contexts arise from biased retrieval and agentic workflows. We argue that haystack engineering is necessary to construct noisy long contexts that faithfully capture key real-world factors – distraction from heterogeneous biased retrievers and cascading errors in agentic workflows – to test models' long-context robustness. We instantiate it through HaystackCraft, a new NIAH benchmark built on the full English Wikipedia hyperlink network with multi-hop questions. HaystackCraft evaluates how heterogeneous retrieval strategies (e.g., sparse, dense, hybrid, and graph-based) affect distractor composition, haystack ordering, and downstream LLM performance. HaystackCraft further extends NIAH to dynamic, LLM-dependent settings that simulate agentic operations, where models refine queries, reflect on their past reasonings, and decide when to stop. Experiments with 15 long-context models show that (1) while stronger dense retrievers can introduce more challenging distractors, graph-based reranking simultaneously improves retrieval effectiveness and mitigates more harmful distractors; (2) in agentic tests, even advanced models like Gemini 2.5 Pro and GPT-5 suffer cascading failures from self-generated distractors or struggle to perform early stops. These results highlight persistent challenges in agentic long-context reasoning and establish HaystackCraft as a valuable testbed for future progress.

Method: Empirical comparison of in-context lemmatization using LLMs against encoder-only supervised approaches fine-tuned out-of-domain and cross-lingual methods, across 12 languages with varying morphological complexity.

Result: LLMs reach state-of-the-art results for most languages by directly generating lemmas in-context without fine-tuning, using just a few examples, while encoders remain competitive when fine-tuned on gold data.

Conclusion: Current LLMs demonstrate superior performance in contextual lemmatization compared to traditional supervised methods, especially in cross-domain scenarios where training data is unavailable.

Abstract: Lemmatization is the task of transforming all words in a given text to their dictionary forms. While large language models (LLMs) have demonstrated their ability to achieve competitive results across a wide range of NLP tasks, there is no prior evidence of how effective they are in the contextual lemmatization task. In this paper, we empirically investigate the capacity of the latest generation of LLMs to perform in-context lemmatization, comparing it to the traditional fully supervised approach. In particular, we consider the setting in which supervised training data is not available for a target domain or language, comparing (i) encoder-only supervised approaches, fine-tuned out-of-domain, and (ii) cross-lingual methods, against direct in-context lemma generation with LLMs. Our experimental investigation across 12 languages of different morphological complexity finds that, while encoders remain competitive in out-of-domain settings when fine-tuned on gold data, current LLMs reach state-of-the-art results for most languages by directly generating lemmas in-context without prior fine-tuning, provided just with a few examples. Data and code available upon publication: https://github.com/oltoporkov/lemma-dilemma

Method: Uses LLMs’ in-context learning with detailed examples in prompts; also fine-tunes smaller LLMs like Llama 3 on word-pair examples from reference and ASR predictions to predict appropriate penalties.

Result: Hindi LASER scores achieved 94% correlation with human annotations using Gemini 2.5 Pro. The approach also worked effectively for other Indian languages (Marathi, Kannada, Malayalam). Fine-tuned Llama 3 achieved 89% accuracy in penalty prediction.

Conclusion: LLM-based evaluation methods like LASER provide more semantically-aware alternatives to traditional ASR metrics, with high correlation to human judgment and cross-lingual applicability.

Abstract: Standard ASR evaluation metrics like Word Error Rate (WER) tend to unfairly penalize morphological and syntactic nuances that do not significantly alter sentence semantics. We introduce an LLM-based scoring rubric LASER that leverages state-of-the-art LLMs’ in-context learning abilities to learn from prompts with detailed examples. Hindi LASER scores using Gemini 2.5 Pro achieved a very high correlation score of 94% with human annotations. Hindi examples in the prompt were also effective in analyzing errors in other Indian languages such as Marathi, Kannada and Malayalam. We also demonstrate how a smaller LLM like Llama 3 can be finetuned on word-pair examples derived from reference and ASR predictions to predict what kind of penalty should be applied with close to 89% accuracy.

Method: Comprehensive study using keypoint distance-based, embedding-based, and back-translation-based metrics, evaluated through automatic meta-evaluation of sign-level retrieval and human correlation study of text-to-pose translation.

Result: Shows tradeoffs between different metrics in different scenarios, providing insights into metric selection for sign language evaluation.

Conclusion: The findings and open-source pose-evaluation toolkit offer a practical and reproducible approach for developing and evaluating sign language systems.

Abstract: We present a comprehensive study on meaningfully evaluating sign language utterances in the form of human skeletal poses. The study covers keypoint distance-based, embedding-based, and back-translation-based metrics. We show tradeoffs between different metrics in different scenarios through automatic meta-evaluation of sign-level retrieval and a human correlation study of text-to-pose translation across different sign languages. Our findings and the open-source pose-evaluation toolkit provide a practical and reproducible way of developing and evaluating sign language translation or generation systems.

Method: Curated CLIMATEMEMES dataset with 1,184 memes from 47 subreddits, annotated with stance and media frames. Evaluated LLaVA-NeXT and Molmo models on stance detection and media frame detection tasks, testing various setups including human captions, synthetic captions, and OCR.

Result: Human captions consistently enhanced performance. VLMs performed well on stance detection but struggled with frame detection, where LLMs outperformed VLMs. Synthetic captions and human-corrected OCR provided occasional improvements.

Conclusion: The study reveals VLMs’ limitations in handling nuanced frames and stance expressions in climate change memes, highlighting the need for improved multimodal understanding of complex framing patterns in internet content.

Abstract: Media framing refers to the emphasis on specific aspects of perceived reality to shape how an issue is defined and understood. Its primary purpose is to shape public perceptions often in alignment with the authors’ opinions and stances. However, the interaction between stance and media frame remains largely unexplored. In this work, we apply an interdisciplinary approach to conceptualize and computationally explore this interaction with internet memes on climate change. We curate CLIMATEMEMES, the first dataset of climate-change memes annotated with both stance and media frames, inspired by research in communication science. CLIMATEMEMES includes 1,184 memes sourced from 47 subreddits, enabling analysis of frame prominence over time and communities, and sheds light on the framing preferences of different stance holders. We propose two meme understanding tasks: stance detection and media frame detection. We evaluate LLaVA-NeXT and Molmo in various setups, and report the corresponding results on their LLM backbone. Human captions consistently enhance performance. Synthetic captions and human-corrected OCR also help occasionally. Our findings highlight that VLMs perform well on stance, but struggle on frames, where LLMs outperform VLMs. Finally, we analyze VLMs' limitations in handling nuanced frames and stance expressions on climate change internet memes.

Method: Rubric and anchor guided chain of thought prompting approach that mirrors human coder training, using the Global Populism Database to guide LLM reasoning.

Result: LLM achieves classification accuracy on par with expert human coders in identifying populist content across multiple proprietary and open weight models.

Conclusion: Domain-specific prompting strategy enables LLMs to effectively navigate nuanced, context-sensitive aspects of populism classification.

Abstract: Measuring the ideational content of populism remains a challenge. Traditional strategies based on textual analysis have been critical for building the field’s foundations and providing a valid, objective indicator of populist framing. Yet these approaches are costly, time consuming, and difficult to scale across languages, contexts, and large corpora. Here we present the results from a rubric and anchor guided chain of thought (CoT) prompting approach that mirrors human coder training. By leveraging the Global Populism Database (GPD), a comprehensive dataset of global leaders’ speeches annotated for degrees of populism, we replicate the process used to train human coders by prompting the LLM with an adapted version of the same documentation to guide the model’s reasoning. We then test multiple proprietary and open weight models by replicating scores in the GPD. Our findings reveal that this domain specific prompting strategy enables the LLM to achieve classification accuracy on par with expert human coders, demonstrating its ability to navigate the nuanced, context sensitive aspects of populism.

Method: Used CoT fine-tuning for reasoning in text space, introduced entropy-based question completeness metric for early reasoning, and applied DPO on preference data.

Result: 2.4x average accuracy improvement on spoken reasoning tasks, 4% accuracy gain on ARC-Easy under equivalent latency, and 70% latency reduction without accuracy loss.

Conclusion: Combining CoT fine-tuning with early reasoning initiation and DPO optimization enables speech LLMs to achieve both high accuracy and low latency for practical voice interactions.

Abstract: Recent advances in speech large language models (speech LLMs) have enabled seamless spoken interactions, but these systems still struggle with complex reasoning tasks. Previously, chain-of-thought (CoT) prompting or fine-tuning has been to shown to significantly improve the reasoning abilities of text-based LLMs. In this work, we investigate the effect of CoT fine-tuning for multi-stream speech LLMs, demonstrating that reasoning in text space improves the accuracy of speech LLMs by 2.4x, on average, over a suite of spoken reasoning tasks. Beyond accuracy, the latency of the spoken response is a crucial factor for interacting with voice-based agents. Inspired by the human behavior of “thinking while listening,” we propose methods to reduce the additional latency from reasoning by allowing the model to start reasoning before the user query has ended. To achieve this, we introduce an entropy-based metric, “question completeness,” which acts as an indicator to guide the model on the optimal time to start reasoning. This method provides greater control over the accuracy-latency trade-off compared with heuristic-based approaches and, under equivalent latency conditions, yields a 4% accuracy gain on ARC-Easy. Finally, we use Direct Preference Optimization (DPO) on preference data created using rejection sampling to push the accuracy-latency pareto frontier further, resulting in a 70% reduction in latency without loss in accuracy.

Method: Four-stage framework: formulates MAS prompt optimization as MAP inference, solves with language-guided max-product belief propagation, uses topology-aware refinement with execution feedback and downstream blames to iteratively update agent prompts.

Result: Achieves state-of-the-art performance across various benchmarks, consistently surpassing manually engineered baselines and recent automated alternatives.

Conclusion: MAPRO provides a principled approach for multi-agent prompt optimization and delivers guidelines for building more reliable multi-agent systems.

Abstract: Large language models (LLMs) have demonstrated remarkable capabilities across diverse tasks, and LLM-based agents further extend these abilities to various practical workflows. While recent progress shows that multi-agent systems (MAS) can outperform single agents by coordinating specialized roles, designing effective MAS remains difficult due to prompt sensitivity and the compounded instability MAS creates. To cope with the challenge, recent efforts in automated prompt design have reduced manual effort. However, multi-agent prompt optimization remains largely unexplored. Challenges like exponentially expanding search space and ambiguous credit assignment together make systematic design intractable without principled methods. Therefore, we introduce M}ulti-Agent PRompt Optimization (MAPRO), a four-stage framework that first formulates MAS prompt optimization as a Maximum a Posteriori (MAP) inference problem and solves it using a language-guided variant of max-product belief propagation algorithm. To address credit assignment and updates the system iteratively, MAPRO employs a topology-aware refinement mechanism that integrates execution feedback and downstream blames to selectively update agent prompts. Through this process, MAPRO progressively converges to a coordinated set of agent-specific prompt policies. Across benchmarks in various tasks, MAPRO achieves state-of-the-art performance, consistently surpassing manually engineered baselines and recent automated alternatives. Beyond performance, our MAP-based formulation also delivers general guidelines for building more reliable and principled multi-agent systems in the future

Method: Proposes AsyncSpade with two core components: (1) a light-weight temporal-regressive module that predicts next-token query state from recent queries, enabling training-free sparsity; (2) an asynchronous disaggregated framework that decouples KV cache filtering from the auto-regressive decoding loop.

Result: Achieves over 20% reduction in time-per-output-token compared to state-of-the-art baseline (Quest) and at least 50% reduction compared to full attention on Qwen3-8B and Qwen3-32B models, while matching or surpassing accuracy on various TTS benchmarks.

Conclusion: AsyncSpade successfully eliminates sequential dependence without sacrificing model performance, delivering theoretical optimal time-per-output-token by fully overlapping KV-cache operations with the inference pipeline.

Abstract: Test-time scaling (TTS) boosts LLM reasoning via long chain-of-thought (CoT), but the linear KV-cache growth amplifies the memory-bound bottleneck of LLM decoding. Query-aware page-level sparse decoding can achieve state-of-the-art performance under constrained FLOPs budgets, but is limited by both sequential-dependent page filtering and coarse-grained token selection, hampering serving efficiency and model performance on TTS tasks under high concurrency and long CoT scenarios (consuming even higher runtime than the forward pipeline itself). In this paper, we first find that the current-step query state can be accurately approximated in a unified manner from a short window of recent queries, enabling training-free query-aware sparsity without waiting in the decoding loop. We propose AsyncSpade, an asynchronous framework for efficient TTS built on two core components: (1) a novel light-weight temporal-regressive module that predicts the next-token query state; (2) an asynchronous and disaggregated framework that decouples the KV cache filtering from the auto-regressive decoding loop, overlapping the token-level KV selection with the forward inference computation through asynchronism. To our knowledge, AsyncSpade is the first to eliminate the sequential dependence without sacrificing model performance. We validate the effectiveness of AsyncSpade on common LLM serving setups with an A100 node, where AsyncSpade fully overlaps KV-cache operations with the inference pipeline, achieving theoretical optimal time-per-output-token (TPOT). Specifically, AsyncSpade delivers over 20% reduction on TPOT compared to SoTA baseline (i.e. Quest) and at least 50% TPOT reduction compared to full attention on Qwen3-8B and Qwen3-32B models, while matching or surpassing their accuracy on various TTS benchmarks (AIME-24/25, GPQA-Diamond, MATH-500).

Method: Proposed a multi-agent framework inspired by human team science to examine structure, diversity, and interaction dynamics, evaluating performance across CommonsenseQA, StrategyQA, Social IQa, and Latent Implicit Hate tasks.

Result: Flat teams performed better than hierarchical ones, diversity had nuanced impact, agents were overconfident about team performance, and post-task reflections showed appreciation for collaboration but challenges in integration and conversational coordination.

Conclusion: The study provides insights into LLM-powered multi-agent team dynamics, highlighting the importance of structure and revealing both collaborative benefits and integration challenges in agent interactions.

Abstract: Multi-Agent Systems (MAS) with Large Language Model (LLM)-powered agents are gaining attention, yet fewer studies explore their team dynamics. Inspired by human team science, we propose a multi-agent framework to examine core aspects of team science: structure, diversity, and interaction dynamics. We evaluate team performance across four tasks: CommonsenseQA, StrategyQA, Social IQa, and Latent Implicit Hate, spanning commonsense and social reasoning. Our results show that flat teams tend to perform better than hierarchical ones, while diversity has a nuanced impact. Interviews suggest agents are overconfident about their team performance, yet post-task reflections reveal both appreciation for collaboration and challenges in integration, including limited conversational coordination.

Method: Propose thought templates that structure evidence combination and guide multi-hop inference, with iterative update strategy using natural-language feedback to refine templates.

Result: Consistent gains over strong baselines across diverse benchmarks and LCLM families in both retrieval-based and retrieval-free settings; optimized templates can be distilled into smaller models.

Conclusion: Thought templates provide effective structured reasoning for multi-hop inference in long-context models, with broad applicability and transparent reasoning reuse.

Abstract: Recent Long-Context Language Models (LCLMs) can process hundreds of thousands of tokens in a single prompt, enabling new opportunities for knowledge-intensive multi-hop reasoning by integrating large sets of retrieved documents or, in some cases, directly all necessary information. However, simply feeding more documents into the context window fails to capture how evidence should be connected. We address this gap with thought templates, which recast reasoning as reusable thought caches, derived from prior problem solving traces, structuring how evidence is combined and guiding multi-hop inference with factual documents. To keep these templates effective, we propose an update strategy that iteratively refines templates derived from training data through natural-language feedback. Across diverse benchmarks and LCLM families, our approach delivers consistent gains over strong baselines in both retrieval-based and retrieval-free settings. Furthermore, we show that optimized templates can be distilled into smaller open-source models, demonstrating its broad applicability and transparent reasoning reuse. We refer to our framework as Thought Template Augmented LCLMs (ToTAL).

Method: Created a fully reproducible benchmark with standardized text normalization, evaluating systems across multilingual and long-form tracks using both WER and inverse real-time factor (RTFx).

Result: Conformer encoders with LLM decoders achieve best WER but are slower, while CTC/TDT decoders offer better efficiency. Whisper fine-tuning improves English accuracy but reduces multilingual coverage.

Conclusion: The benchmark enables fair accuracy-efficiency comparisons and provides open-source tools for transparent, extensible ASR evaluation.

Abstract: Despite rapid progress, ASR evaluation remains saturated with short-form English, and efficiency is rarely reported. We present the Open ASR Leaderboard, a fully reproducible benchmark and interactive leaderboard comparing 60+ open-source and proprietary systems across 11 datasets, including dedicated multilingual and long-form tracks. We standardize text normalization and report both word error rate (WER) and inverse real-time factor (RTFx), enabling fair accuracy-efficiency comparisons. For English transcription, Conformer encoders paired with LLM decoders achieve the best average WER but are slower, while CTC and TDT decoders deliver much better RTFx, making them attractive for long-form and offline use. Whisper-derived encoders fine-tuned for English improve accuracy but often trade off multilingual coverage. All code and dataset loaders are open-sourced to support transparent, extensible evaluation.

Method: Developed a sequence-to-sequence model trained across all available datasets, including two new datasets created by the authors, to handle varied domains of text.

Result: Achieved chrF++ scores of 87.91 (Farsi→Tajik) and 92.28 (Tajik→Farsi), with Normalized CER scores of 0.05 and 0.04 respectively, setting comprehensive benchmarks.

Conclusion: The model provides clearer understanding of the transliteration task’s difficulty and offers a versatile solution for real-world usage across different text domains.

Abstract: As a digraphic language, the Persian language utilizes two written standards: Perso-Arabic in Afghanistan and Iran, and Tajik-Cyrillic in Tajikistan. Despite the significant similarity between the dialects of each country, script differences prevent simple one-to-one mapping, hindering written communication and interaction between Tajikistan and its Persian-speaking ``siblings’’. To overcome this, previously-published efforts have investigated machine transliteration models to convert between the two scripts. Unfortunately, most efforts did not use datasets other than those they created, limiting these models to certain domains of text such as archaic poetry or word lists. A truly usable transliteration system must be capable of handling varied domains, meaning that suck models lack the versatility required for real-world usage. The contrast in domain between data also obscures the task’s true difficulty. We present a new state-of-the-art sequence-to-sequence model for Tajik-Farsi transliteration trained across all available datasets, and present two datasets of our own. Our results across domains provide clearer understanding of the task, and set comprehensive comparable leading benchmarks. Overall, our model achieves chrF++ and Normalized CER scores of 87.91 and 0.05 from Farsi to Tajik and 92.28 and 0.04 from Tajik to Farsi. Our model, data, and code are available at https://anonymous.4open.science/r/ParsTranslit-FB30/.

Method: OWL uses three innovations: (1) LSTM-based drafter conditioned only on last-token state for length generalization, (2) special [SPEC] token in verifier for richer drafter representations, and (3) hybrid algorithm combining tree and non-tree decoding methods.

Result: OWL achieves about 5x higher acceptance length than EAGLE3 on long-context inputs, addressing the severe degradation where EAGLE3 even slows down generation speed by 0.81x.

Conclusion: The paper releases LongSpecBench benchmark and OWL model with all code and datasets to advance future research in long-context speculative decoding.

Abstract: Speculative decoding promises faster inference for large language models (LLMs), yet existing methods fail to generalize to real-world settings. Benchmarks typically assume short contexts (e.g., 2K tokens), whereas practical workloads involve long contexts. We find current approaches degrade severely with long contexts; for instance, EAGLE3 even slows down the generation speed by 0.81x. We address these limitations by releasing a new long-context benchmark (LongSpecBench) and introducing a novel model (OWL). OWL achieves about 5x higher acceptance length than EAGLE3 on long-context inputs through three innovations: (1) an LSTM-based drafter conditioned only on the last-token state, making it generalize to various lengths, (2) a special token [SPEC] in the verifier that produces richer representation for drafter, and (3) a hybrid algorithm combining both tree and non-tree decoding methods. We release all code and datasets to advance future research.

Method: Two approaches: (1) multi-criteria prompting that combines separate evaluation criteria into single queries, and (2) domain-adaptive transfer learning fine-tuning a 2B-parameter LVLM on synthetic judgments in chart datasets to create ChartJudge.

Result: Multi-criteria prompting exposes robustness gaps causing huge performance drops in 7B models. ChartJudge effectively transfers knowledge between datasets and becomes a more specialized model. Fine-grained analysis reveals trade-offs between model size, prompt design, and transferability.

Conclusion: The approaches enable scalable, low-cost evaluation for chart reasoning tasks, providing actionable insights into balancing model size, prompt design, and transferability for resource-constrained environments.

Abstract: Large Vision-Language Models (LVLMs) with only 7B parameters have shown promise as automated judges in chart comprehension tasks. However, tiny models (<=2B parameters) still perform poorly as judges, limiting their real-world use in resource-constrained settings. To address this, we propose two approaches to ensure cost-efficient evaluation: (i) multi-criteria prompting, which combines separate evaluation criteria into a single query, and (ii) domain-adaptive transfer learning, in which we fine-tune a 2B-parameter LVLM on synthetic judgments in a chart dataset to create the ChartJudge. Experiments show that multi-criteria prompting exposes robustness gaps, which led to a huge drop in performance for 7B models, including specialized LVLM judges like LLaVA-Critic. In addition, we find that our tiny LVLM (ChartJudge) can effectively transfer knowledge from one dataset to another to make it a more specialized model. Our fine-grained analysis across chart types and query complexities offers actionable insights into trade-offs between model size, prompt design, and transferability, enabling scalable, low-cost evaluation for chart reasoning tasks. Our code and the data will be made publicly available.

Method: Propose a multi-task pre-finetuning framework based on task-primary LoRA modules, enabling a single shared encoder backbone with modular adapters to avoid conflicting optimization signals from naive multi-task pre-finetuning.

Result: Achieves performance comparable to individual pre-finetuning, with average improvements of +0.8% for NER and +8.8% for text classification across 21 downstream tasks.

Conclusion: The proposed method effectively enables versatile mobile NLP applications by providing efficient adaptation capabilities while meeting practical deployment constraints.

Abstract: Deploying natural language processing (NLP) models on mobile platforms requires models that can adapt across diverse applications while remaining efficient in memory and computation. We investigate pre-finetuning strategies to enhance the adaptability of lightweight BERT-like encoders for two fundamental NLP task families: named entity recognition (NER) and text classification. While pre-finetuning improves downstream performance for each task family individually, we find that na"ive multi-task pre-finetuning introduces conflicting optimization signals that degrade overall performance. To address this, we propose a simple yet effective multi-task pre-finetuning framework based on task-primary LoRA modules, which enables a single shared encoder backbone with modular adapters. Our approach achieves performance comparable to individual pre-finetuning while meeting practical deployment constraint. Experiments on 21 downstream tasks show average improvements of +0.8% for NER and +8.8% for text classification, demonstrating the effectiveness of our method for versatile mobile NLP applications.

Method: Created ReasonMed dataset through multi-agent generation, verification, and refinement using an EMD pipeline. Error Refiner corrects error-prone steps identified by verifier. Integrated detailed CoT reasoning with concise answer summaries for training.

Result: ReasonMed-7B surpasses prior best sub-10B models by 4.17% and exceeds LLaMA3.1-70B on PubMedQA by 4.60%. ReasonMed-14B remains highly competitive, showing consistent scaling potential.

Conclusion: ReasonMed bridges the gap in medical reasoning evaluation and demonstrates effective training strategies for medical reasoning models with strong performance and scaling potential.

Abstract: Reasoning-based large language models have excelled in mathematics and programming, yet their potential in knowledge-intensive medical question answering remains underexplored and insufficiently validated in clinical contexts. To bridge this gap, we introduce ReasonMed, the largest medical reasoning dataset to date, comprising 370k high-quality examples distilled from 1.75 million initial reasoning paths generated by complementary LLMs and curated through a cost-efficient easy-medium-difficult (EMD) pipeline. ReasonMed is built through a multi-agent generation, verification, and refinement process, in which an Error Refiner improves reasoning paths by correcting error-prone steps identified by a verifier. Using ReasonMed, we investigate effective strategies for training medical reasoning models and find that integrating detailed CoT reasoning with concise answer summaries yields the most robust fine-tuning results. Models trained on ReasonMed set a new benchmark: ReasonMed-7B surpasses the prior best sub-10B models by 4.17% and even exceeds LLaMA3.1-70B on PubMedQA by 4.60%. When scaled to ReasonMed-14B, it remains highly competitive, underscoring consistent scaling potential. The codes and datasets are available at https://github.com/YuSun-Work/ReasonMed.

Method: Computational linguistic analysis of three corpora: COVID-19 false narratives (n=7588), general COVID-19 content (n=10700), and Monkeypox-related posts (n=5787), examining readability, rhetorical markers, and persuasive language use.

Result: COVID-19 misinformation had lower readability scores, contained over twice the frequency of fear-related/persuasive terms, and showed minimal exclamation marks compared to other datasets. Misinformation employs complex rhetorical style with emotional cues.

Conclusion: Linguistic patterns can aid misinformation detection and inform public health messaging strategies, though limitations include reliance on traditional readability indices and static analysis. Future research needs longitudinal designs and broader emotion lexicons.

Abstract: This study conducts a computational linguistic analysis of pandemic-related online discourse to examine how language distinguishes health misinformation from factual communication. Drawing on three corpora: COVID-19 false narratives (n = 7588), general COVID-19 content (n = 10700), and Monkeypox-related posts (n = 5787), we identify significant differences in readability, rhetorical markers, and persuasive language use. COVID-19 misinformation exhibited markedly lower readability scores and contained over twice the frequency of fear-related or persuasive terms compared to the other datasets. It also showed minimal use of exclamation marks, contrasting with the more emotive style of Monkeypox content. These patterns suggest that misinformation employs a deliberately complex rhetorical style embedded with emotional cues, a combination that may enhance its perceived credibility. Our findings contribute to the growing body of work on digital health misinformation by highlighting linguistic indicators that may aid detection efforts. They also inform public health messaging strategies and theoretical models of crisis communication in networked media environments. At the same time, the study acknowledges limitations, including reliance on traditional readability indices, use of a deliberately narrow persuasive lexicon, and reliance on static aggregate analysis. Future research should therefore incorporate longitudinal designs, broader emotion lexicons, and platform-sensitive approaches to strengthen robustness.

Method: Modular approach with agentic phonology creation, sentence translation to morphosyntactic markup, lexicon construction from translated corpus, orthography design using existing scripts, and grammar handbook generation.

Result: System can create constructed languages and translate sentences, with varying capabilities across different LLMs and linguistic specifications - better performance on common patterns than rare ones.

Conclusion: The system demonstrates LLMs’ linguistic knowledge and potential for language creation, with future potential for high-to-low-resource language translation despite current limitations.

Abstract: We present a system that uses LLMs as a tool in the development of Constructed Languages. The system is modular in that one first creates a target phonology for the language using an agentic approach that refines its output at each step with commentary feedback on its previous attempt. Next, a set of sentences is ’translated’ from their English original into a morphosyntactic markup that reflects the word order and morphosyntactic feature specifications of the desired target language, with affixes represented as morphosyntactic feature bundles. From this translated corpus, a lexicon is constructed using the phonological model and the set of morphemes (stems and affixes) extracted from the ’translated’ sentences. The system is then instructed to provide an orthography for the language, using an existing script such as Latin or Cyrillic. Finally, the system writes a brief grammatical handbook of the language. The system can also translate further sentences into the target language. Our goal is twofold. First, we hope that these tools will be fun to use for creating artificially constructed languages. Second, we are interested in exploring what LLMs ‘know’ about language-not what they know about any particular language or linguistic phenomenon, but how much they know about and understand language and linguistic concepts. As we shall see, there is a fairly wide gulf in capabilities both among different LLMs and among different linguistic specifications, with it being notably easier for systems to deal with more common patterns than rarer ones. An additional avenue that we explore is the application of our approach to translating from high-resource into low-resource languages. While the results so far are mostly negative, we provide some evidence that an improved version of the present system could afford some real gains in such tasks. https://github.com/SakanaAI/IASC

Method: Used representational similarity analysis to correlate geometric structure of input/output embeddings in Pythia 12B and OLMo 7B with semantic, syntactic, and frequency-based metrics throughout training.

Result: 1) Vocabulary embeddings quickly converge to high correlations with semantic/syntactic features during training; 2) High-frequency and function words converge faster than lexical/low-frequency words, which retain some alignment with random initialization bias.

Conclusion: The findings reveal distinct roles for word frequency and function in embedding evolution, motivating deeper study of how vocabulary geometry evolution facilitates specific capability gains during training.

Abstract: Large language models (LLMs) work by manipulating the geometry of input embedding vectors over multiple layers. Here, we ask: how are the input vocabulary representations of language models structured, and how and when does this structure evolve over training? To answer this question, we use representational similarity analysis, running a suite of experiments that correlate the geometric structure of the input embeddings and output embeddings of two open-source models (Pythia 12B and OLMo 7B) with semantic, syntactic, and frequency-based metrics over the course of training. Our key findings are as follows: 1) During training, the vocabulary embedding geometry quickly converges to high correlations with a suite of semantic and syntactic features; 2) Embeddings of high-frequency and function words (e.g., “the,” “of”) converge to their final vectors faster than lexical and low-frequency words, which retain some alignment with the bias in their random initializations. These findings help map the dynamic trajectory by which input embeddings organize around linguistic structure, revealing distinct roles for word frequency and function. Our findings motivate a deeper study of how the evolution of vocabulary geometry may facilitate specific capability gains during model training.

Method: The authors use lightweight interventions including prompt engineering and small-scale fine-tuning, introduce clinical code verification as a standalone task and pipeline component, and release an expert-annotated outpatient benchmark to address dataset limitations.

Result: The approach shows that verification is an effective step toward improving LLM-based medical coding, with hierarchical misalignments accounting for a substantial portion of LLM failures.

Conclusion: Clinical code verification combined with lightweight interventions can significantly improve LLM accuracy in medical coding without the computational overhead of search-based methods.

Abstract: Accurate clinical coding is essential for healthcare documentation, billing, and decision-making. While prior work shows that off-the-shelf LLMs struggle with this task, evaluations based on exact match metrics often overlook errors where predicted codes are hierarchically close but incorrect. Our analysis reveals that such hierarchical misalignments account for a substantial portion of LLM failures. We show that lightweight interventions, including prompt engineering and small-scale fine-tuning, can improve accuracy without the computational overhead of search-based methods. To address hierarchically near-miss errors, we introduce clinical code verification as both a standalone task and a pipeline component. To mitigate the limitations in existing datasets, such as incomplete evidence and inpatient bias in MIMIC, we release an expert double-annotated benchmark of outpatient clinical notes with ICD-10 codes. Our results highlight verification as an effective and reliable step toward improving LLM-based medical coding.

Method: Created a novel dataset extending Spider and BIRD text-to-SQL datasets with PostgreSQL role-based policies, then compared three approaches: zero/few-shot prompting, two-step generator-verifier pipeline, and LoRA fine-tuned models.

Result: Explicit verification (two-step framework) improved refusal precision and reduced false permits, while fine-tuning achieved better balance between safety and utility. Longer and more complex policies consistently reduced reliability across all systems.

Conclusion: The study demonstrates that explicit verification is most effective for refusal precision, while fine-tuning provides the best safety-utility balance, with both approaches being challenged by complex policies.

Abstract: Access control is a cornerstone of secure computing, yet large language models often blur role boundaries by producing unrestricted responses. We study role-conditioned refusals, focusing on the LLM’s ability to adhere to access control policies by answering when authorized and refusing when not. To evaluate this behavior, we created a novel dataset that extends the Spider and BIRD text-to-SQL datasets, both of which have been modified with realistic PostgreSQL role-based policies at the table and column levels. We compare three designs: (i) zero or few-shot prompting, (ii) a two-step generator-verifier pipeline that checks SQL against policy, and (iii) LoRA fine-tuned models that learn permission awareness directly. Across multiple model families, explicit verification (the two-step framework) improves refusal precision and lowers false permits. At the same time, fine-tuning achieves a stronger balance between safety and utility (i.e., when considering execution accuracy). Longer and more complex policies consistently reduce the reliability of all systems. We release RBAC-augmented datasets and code.

Method: Built entirely in-house using ILMU (closed-source LLM) with four specialized agents (Guardrails, Intent, Payment, FAQ) using task-specific LoRA adapters, hosted within bank infrastructure with deterministic guardrails and human-in-the-loop confirmation.

Result: Successfully demonstrated that regulator-approved natural-language interfaces can reliably support core financial operations under strict governance, representing Banking Done Right.

Conclusion: The framework proves that conversational AI can serve as the primary banking interface for core transactions when properly secured with defense-in-depth security measures and regulatory compliance.

Abstract: This paper presents Ryt AI, an LLM-native agentic framework that powers Ryt Bank to enable customers to execute core financial transactions through natural language conversation. This represents the first global regulator-approved deployment worldwide where conversational AI functions as the primary banking interface, in contrast to prior assistants that have been limited to advisory or support roles. Built entirely in-house, Ryt AI is powered by ILMU, a closed-source LLM developed internally, and replaces rigid multi-screen workflows with a single dialogue orchestrated by four LLM-powered agents (Guardrails, Intent, Payment, and FAQ). Each agent attaches a task-specific LoRA adapter to ILMU, which is hosted within the bank’s infrastructure to ensure consistent behavior with minimal overhead. Deterministic guardrails, human-in-the-loop confirmation, and a stateless audit architecture provide defense-in-depth for security and compliance. The result is Banking Done Right: demonstrating that regulator-approved natural-language interfaces can reliably support core financial operations under strict governance.

Method: Formulates cache eviction as layer-wise structured pruning using Optimal Brain Damage theory, deriving closed-form saliency scores that measure perturbation in attention outputs when pruning tokens, considering isolated keys, values, and joint key-value pairs.

Result: Experiments on LLaMA and Qwen models show that replacing heuristic scores with OBCache’s output-aware scores consistently improves long-context accuracy across various tasks.

Conclusion: OBCache provides a principled, output-aware approach to KV cache eviction that outperforms existing heuristic methods by better quantifying token saliency’s true impact on attention outputs.

Abstract: Large language models (LLMs) with extended context windows enable powerful downstream applications but impose significant memory overhead, as caching all key-value (KV) states scales linearly with sequence length and batch size. Existing cache eviction methods address this by exploiting attention sparsity, yet they typically rank tokens heuristically using accumulated attention weights without considering their true impact on attention outputs. We propose Optimal Brain Cache (OBCache), a principled framework that formulates cache eviction as a layer-wise structured pruning problem. Building upon the Optimal Brain Damage (OBD) theory, OBCache quantifies token saliency by measuring the perturbation in attention outputs induced by pruning tokens, with closed-form scores derived for isolated keys, isolated values, and joint key-value pairs. Our scores account not only for attention weights but also for information from value states and attention outputs, thereby enhancing existing eviction strategies with output-aware signals. Experiments on LLaMA and Qwen models demonstrate that replacing the heuristic scores in existing works, which estimate token saliency across different query positions, with OBCache’s output-aware scores consistently improves long-context accuracy.

Method: Compare natural language inference (NLI) metrics with token probability (TP) metrics for measuring social bias in language models, analyzing their correlation and behavior differences.

Result: NLI and TP bias evaluation behave substantially differently with very low correlation; NLI metrics detect more “underdebiased” cases but are more brittle and sensitive to wording than TP approaches.

Conclusion: Neither token probability nor natural language inference is a “better” bias metric in all cases; recommend combining TP, NLI, and downstream bias evaluations for comprehensive assessment.

Abstract: Measurement of social bias in language models is typically by token probability (TP) metrics, which are broadly applicable but have been criticized for their distance from real-world langugage model use cases and harms. In this work, we test natural language inference (NLI) as a more realistic alternative bias metric. We show that, curiously, NLI and TP bias evaluation behave substantially differently, with very low correlation among different NLI metrics and between NLI and TP metrics. We find that NLI metrics are more likely to detect “underdebiased” cases. However, NLI metrics seem to be more brittle and sensitive to wording of counterstereotypical sentences than TP approaches. We conclude that neither token probability nor natural language inference is a “better” bias metric in all cases, and we recommend a combination of TP, NLI, and downstream bias evaluations to ensure comprehensive evaluation of language models. Content Warning: This paper contains examples of anti-LGBTQ+ stereotypes.

Method: Generate diverse value tradeoff scenarios where models must choose between competing legitimate principles, then evaluate responses from twelve frontier LLMs using value classification scores to measure behavioral disagreement.

Result: Identified over 70,000 cases of significant behavioral divergence across models, which strongly predicts underlying specification problems. Found direct contradictions, interpretive ambiguities, misalignment cases, and false-positive refusals in current model specs.

Conclusion: The stress-testing methodology effectively reveals fundamental problems in model specifications, including principle contradictions and ambiguities, providing insights for improving AI constitutions and behavioral guidelines.

Abstract: Large language models (LLMs) are increasingly trained from AI constitutions and model specifications that establish behavioral guidelines and ethical principles. However, these specifications face critical challenges, including internal conflicts between principles and insufficient coverage of nuanced scenarios. We present a systematic methodology for stress-testing model character specifications, automatically identifying numerous cases of principle contradictions and interpretive ambiguities in current model specs. We stress test current model specs by generating scenarios that force explicit tradeoffs between competing value-based principles. Using a comprehensive taxonomy we generate diverse value tradeoff scenarios where models must choose between pairs of legitimate principles that cannot be simultaneously satisfied. We evaluate responses from twelve frontier LLMs across major providers (Anthropic, OpenAI, Google, xAI) and measure behavioral disagreement through value classification scores. Among these scenarios, we identify over 70,000 cases exhibiting significant behavioral divergence. Empirically, we show this high divergence in model behavior strongly predicts underlying problems in model specifications. Through qualitative analysis, we provide numerous example issues in current model specs such as direct contradiction and interpretive ambiguities of several principles. Additionally, our generated dataset also reveals both clear misalignment cases and false-positive refusals across all of the frontier models we study. Lastly, we also provide value prioritization patterns and differences of these models.

Method: The authors propose a unified taxonomy organizing methods into two paradigms: LLMs as Oracles (direct cellular modeling) and LLMs as Agents (orchestrating complex scientific tasks). They identify and review three core tasks: cellular representation, perturbation prediction, and gene regulation inference.

Result: The review systematically organizes existing virtual cell modeling approaches, analyzing associated models, datasets, evaluation benchmarks, and identifying critical challenges in scalability, generalizability, and interpretability.

Conclusion: LLMs show significant potential for virtual cell modeling in cellular biology, but face key challenges that need to be addressed for broader adoption and effectiveness in biological research.

Abstract: Large language models (LLMs) are transforming cellular biology by enabling the development of “virtual cells”–computational systems that represent, predict, and reason about cellular states and behaviors. This work provides a comprehensive review of LLMs for virtual cell modeling. We propose a unified taxonomy that organizes existing methods into two paradigms: LLMs as Oracles, for direct cellular modeling, and LLMs as Agents, for orchestrating complex scientific tasks. We identify three core tasks–cellular representation, perturbation prediction, and gene regulation inference–and review their associated models, datasets, evaluation benchmarks, as well as the critical challenges in scalability, generalizability, and interpretability.

Method: Model hate speech generation as a causal graph involving contextual environment, creator motivation, target, and style. Use causal representation learning to disentangle hate speech into interpretable latent factors and control confounders. Enable counterfactual reasoning by intervening on style in latent space.

Result: CADET demonstrates superior performance in comprehensive experiments, showing improved detection of hate speech across varying forms and platforms.

Conclusion: The framework highlights the potential of causal priors in advancing generalizable hate speech detection by isolating genuine hate intent from superficial linguistic patterns.

Abstract: The proliferation of online hate speech poses a significant threat to the harmony of the web. While explicit hate is easily recognized through overt slurs, implicit hate speech is often conveyed through sarcasm, irony, stereotypes, or coded language – making it harder to detect. Existing hate speech detection models, which predominantly rely on surface-level linguistic cues, fail to generalize effectively across diverse stylistic variations. Moreover, hate speech spread on different platforms often targets distinct groups and adopts unique styles, potentially inducing spurious correlations between them and labels, further challenging current detection approaches. Motivated by these observations, we hypothesize that the generation of hate speech can be modeled as a causal graph involving key factors: contextual environment, creator motivation, target, and style. Guided by this graph, we propose CADET, a causal representation learning framework that disentangles hate speech into interpretable latent factors and then controls confounders, thereby isolating genuine hate intent from superficial linguistic cues. Furthermore, CADET allows counterfactual reasoning by intervening on style within the latent space, naturally guiding the model to robustly identify hate speech in varying forms. CADET demonstrates superior performance in comprehensive experiments, highlighting the potential of causal priors in advancing generalizable hate speech detection.

Method: Builds two complementary memory components: behavioral memory for specific user actions and cognitive memory for long-term preferences, both integrating temporal and semantic information at different abstraction levels.

Result: Experiments on the Language Model Personalization (LaMP) benchmark validate the efficacy of MemWeaver framework.

Conclusion: MemWeaver provides a unified user representation that allows LLMs to reason over both concrete behaviors and abstract traits, enabling deeper personalization.

Abstract: The primary form of user-internet engagement is shifting from leveraging implicit feedback signals, such as browsing and clicks, to harnessing the rich explicit feedback provided by textual interactive behaviors. This shift unlocks a rich source of user textual history, presenting a profound opportunity for a deeper form of personalization. However, prevailing approaches offer only a shallow form of personalization, as they treat user history as a flat list of texts for retrieval and fail to model the rich temporal and semantic structures reflecting dynamic nature of user interests. In this work, we propose \textbf{MemWeaver}, a framework that weaves the user’s entire textual history into a hierarchical memory to power deeply personalized generation. The core innovation of our memory lies in its ability to capture both the temporal evolution of interests and the semantic relationships between different activities. To achieve this, MemWeaver builds two complementary memory components that both integrate temporal and semantic information, but at different levels of abstraction: behavioral memory, which captures specific user actions, and cognitive memory, which represents long-term preferences. This dual-component memory serves as a unified representation of the user, allowing large language models (LLMs) to reason over both concrete behaviors and abstracted traits. Experiments on the Language Model Personalization (LaMP) benchmark validate the efficacy of MemWeaver. Our code is available\footnote{https://github.com/fishsure/MemWeaver}.

Method: Decomposes complex questions into ordered sub-questions, retrieves relevant triples from graph, dynamically expands graph when insufficient, and aggregates triples into traceable graph memory.

Result: Achieves consistent and significant improvements on three multi-hop QA benchmarks, especially in Exact Match scores.

Conclusion: SubQRAG effectively addresses Graph RAG’s limitations by enhancing reasoning depth through sub-question decomposition and dynamic graph expansion.

Abstract: Graph Retrieval-Augmented Generation (Graph RAG) effectively builds a knowledge graph (KG) to connect disparate facts across a large document corpus. However, this broad-view approach often lacks the deep structured reasoning needed for complex multi-hop question answering (QA), leading to incomplete evidence and error accumulation. To address these limitations, we propose SubQRAG, a sub-question-driven framework that enhances reasoning depth. SubQRAG decomposes a complex question into an ordered chain of verifiable sub-questions. For each sub-question, it retrieves relevant triples from the graph. When the existing graph is insufficient, the system dynamically expands it by extracting new triples from source documents in real time. All triples used in the reasoning process are aggregated into a “graph memory,” forming a structured and traceable evidence path for final answer generation. Experiments on three multi-hop QA benchmarks demonstrate that SubQRAG achieves consistent and significant improvements, especially in Exact Match scores.

Method: Proposed framework with Knowledge-level Grouped Mixture of Experts (KL-GMoE) to model shared knowledge and Iterative Entity Reranking (IER) to enhance utilization.

Result: Achieved improvements of 5.47%, 3.27%, and 1.01% in Hits@1, Hits@3, and Hits@10 metrics respectively compared to SOTA MKGC method on 5-language dataset.

Conclusion: The framework effectively leverages multilingual shared knowledge and reveals knowledge sharing properties in unseen/unbalanced language settings.

Abstract: Large language models (LLMs) based Multilingual Knowledge Graph Completion (MKGC) aim to predict missing facts by leveraging LLMs’ multilingual understanding capabilities, improving the completeness of multilingual knowledge graphs (KGs). However, existing MKGC research underutilizes the multilingual capabilities of LLMs and ignores the shareability of cross-lingual knowledge. In this paper, we propose a novel MKGC framework that leverages multilingual shared knowledge to significantly enhance performance through two components: Knowledge-level Grouped Mixture of Experts (KL-GMoE) and Iterative Entity Reranking (IER). KL-GMoE efficiently models shared knowledge, while IER significantly enhances its utilization. To evaluate our framework, we constructed a mKG dataset containing 5 languages and conducted comprehensive comparative experiments with existing state-of-the-art (SOTA) MKGC method. The experimental results demonstrate that our framework achieves improvements of 5.47%, 3.27%, and 1.01% in the Hits@1, Hits@3, and Hits@10 metrics, respectively, compared with SOTA MKGC method. Further experimental analysis revealed the properties of knowledge sharing in settings of unseen and unbalanced languages. We have released the dataset and code for our work on https://github.com/gaoxiaofei07/KL-GMoE.

Method: ToolExpander uses Dynamic Multi-Round Hard Sampling (replacing challenging samples with demonstrations) and Self-Exemplifying Thinking (removing KL divergence with adjusted clipping coefficients and minimal reward for autonomous example generation).

Result: Significantly enhances tool-using capabilities in LLMs, especially weaker small-scale models, improving both training stability and overall performance.

Conclusion: ToolExpander effectively addresses GRPO limitations through innovative sampling and reward mechanisms, enabling better tool-oriented reinforcement learning for resource-constrained LLMs.

Abstract: Training Large Language Models (LLMs) with Group Relative Policy Optimization (GRPO) encounters a significant challenge: models often fail to produce accurate responses, particularly in small-scale architectures. This limitation not only diminishes performance improvements and undermines the potential of GRPO but also frequently leads to mid-training collapse, adversely affecting stability and final efficacy. To address these issues, we propose ToolExpander, a novel framework that advances tool-oriented reinforcement learning for resource-constrained LLMs through two key innovations:(1) Dynamic Multi-Round Hard Sampling, which dynamically substitutes challenging samples(those without correct outputs over 10 rollouts) with high-quality few-shot demonstrations during training, coupled with an exponential learning rate decay strategy to mitigate oscillations;(2) Self-Exemplifying Thinking, an enhanced GRPO framework that eliminates KL divergence and incorporates adjusted clipping coefficients, encouraging models to autonomously generate and analyze few-shot examples via a minimal additional reward (0.01).Experimental results demonstrate that ToolExpander significantly enhances tool-using capabilities in LLMs, especially in weaker small-scale models, improving both training stability and overall performance.

Method: Contrastive Rubric Generation (CRG) derives hard rules and principles by contrasting preferred and rejected responses, with reliability improved via rejection sampling to remove noisy rubrics.

Result: Rubric-RM surpasses size-matched baselines by 6.8% across multiple reward-modeling benchmarks, with gains transferring to policy models on instruction-following and biomedical tasks.

Conclusion: Rubrics provide scalable alignment signals that narrow the gap between human evaluation and automated reward modeling, enabling a principle-driven paradigm for LLM alignment.

Abstract: Reward modeling lies at the core of reinforcement learning from human feedback (RLHF), yet most existing reward models rely on scalar or pairwise judgments that fail to capture the multifaceted nature of human preferences. Recent studies have explored rubrics-as-rewards (RaR) that uses structured natural language criteria that capture multiple dimensions of response quality. However, producing rubrics that are both reliable and scalable remains a key challenge. In this work, we introduce OpenRubrics, a diverse, large-scale collection of (prompt, rubric) pairs for training rubric-generation and rubric-based reward models. To elicit discriminative and comprehensive evaluation signals, we introduce Contrastive Rubric Generation (CRG), which derives both hard rules (explicit constraints) and principles (implicit qualities) by contrasting preferred and rejected responses. We further improve reliability by enforcing preference-label consistency via rejection sampling to remove noisy rubrics. Across multiple reward-modeling benchmarks, our rubric-based reward model, Rubric-RM, surpasses strong size-matched baselines by 6.8%. These gains transfer to policy models on instruction-following and biomedical benchmarks. Our results show that rubrics provide scalable alignment signals that narrow the gap between costly human evaluation and automated reward modeling, enabling a new principle-driven paradigm for LLM alignment.

Method: Introduced two uncertainty-inspired sampling strategies (Monte Carlo Dropout and Additive Gaussian Noise) and designed a Latent Reward Model trained with step-wise contrastive objective to score and guide latent reasoning trajectories.

Result: Both sampling strategies scale effectively with compute and show distinct exploration dynamics, while the LatentRM enables effective trajectory selection. Extensive experiments and visualization analyses demonstrate the effectiveness of the approach.

Conclusion: This work opens a new direction for scalable inference in continuous spaces by enabling parallel test-time scaling for latent reasoning models.

Abstract: Parallel test-time scaling (TTS) is a pivotal approach for enhancing large language models (LLMs), typically by sampling multiple token-based chains-of-thought in parallel and aggregating outcomes through voting or search. Recent advances in latent reasoning, where intermediate reasoning unfolds in continuous vector spaces, offer a more efficient alternative to explicit Chain-of-Thought, yet whether such latent models can similarly benefit from parallel TTS remains open, mainly due to the absence of sampling mechanisms in continuous space, and the lack of probabilistic signals for advanced trajectory aggregation. \ This work enables parallel TTS for latent reasoning models by addressing the above issues. For sampling, we introduce two uncertainty-inspired stochastic strategies: Monte Carlo Dropout and Additive Gaussian Noise. For aggregation, we design a Latent Reward Model (LatentRM) trained with step-wise contrastive objective to score and guide latent reasoning. Extensive experiments and visualization analyses show that both sampling strategies scale effectively with compute and exhibit distinct exploration dynamics, while LatentRM enables effective trajectory selection. Together, our explorations open a new direction for scalable inference in continuous spaces. Code released at https://github.com/YRYangang/LatentTTS.

Method: Test reasoning LLMs on MCQs with full inputs and choices-only inputs, analyze reasoning traces for faithfulness, and examine how reasoning length affects choices-only success.

Result: Reasoning boosts accuracy in both full and choices-only settings (50% of time). Choices-only success is unaffected by reasoning length, and faithful traces show LLMs infer missing questions rather than using problematic shortcuts.

Conclusion: Partial-input success isn’t always a flaw; reasoning traces can distinguish problematic data from legitimate reasoning strategies like question inference.

Abstract: Large language models (LLMs) now give reasoning before answering, excelling in tasks like multiple-choice question answering (MCQA). Yet, a concern is that LLMs do not solve MCQs as intended, as work finds LLMs sans reasoning succeed in MCQA without using the question, i.e., choices-only. Such partial-input success is often deemed problematic, but reasoning traces could reveal if these strategies are truly shallow in choices-only settings. To study these strategies, reasoning LLMs solve MCQs in full and choices-only inputs; test-time reasoning often boosts accuracy on full and in choices-only half the time. While possibly due to shallow shortcuts, choices-only success is barely affected by the length of reasoning traces, and after finding traces pass faithfulness tests, we show they use less problematic strategies like inferring missing questions. In all, we challenge claims that partial-input success is always a flaw, so we discuss how reasoning traces could separate problematic data from less problematic reasoning.

Method: Generates task-specific tools, clusters them into semantic topics, then uses a multi-agent framework with code and reviewing agents to consolidate scattered functionalities into versatile aggregated tools.

Result: Significantly improves tool retrieval accuracy and overall reasoning performance across multiple tasks, with enhanced scalability as tool numbers increase.

Conclusion: The approach successfully transforms numerous question-specific tools into a smaller set of powerful aggregated tools without functionality loss, addressing scalability issues in tool-augmented reasoning.

Abstract: Large Language Models (LLMs) equipped with external tools have demonstrated enhanced performance on complex reasoning tasks. The widespread adoption of this tool-augmented reasoning is hindered by the scarcity of domain-specific tools. For instance, in domains such as physics question answering, suitable and specialized tools are often missing. Recent work has explored automating tool creation by extracting reusable functions from Chain-of-Thought (CoT) reasoning traces; however, these approaches face a critical scalability bottleneck. As the number of generated tools grows, storing them in an unstructured collection leads to significant retrieval challenges, including an expanding search space and ambiguity between function-related tools. To address this, we propose a systematic approach to automatically refactor an unstructured collection of tools into a structured tool library. Our system first generates discrete, task-specific tools and clusters them into semantically coherent topics. Within each cluster, we introduce a multi-agent framework to consolidate scattered functionalities: a code agent refactors code to extract shared logic and creates versatile, aggregated tools, while a reviewing agent ensures that these aggregated tools maintain the complete functional capabilities of the original set. This process transforms numerous question-specific tools into a smaller set of powerful, aggregated tools without loss of functionality. Experimental results demonstrate that our approach significantly improves tool retrieval accuracy and overall reasoning performance across multiple reasoning tasks. Furthermore, our method shows enhanced scalability compared with baselines as the number of question-specific increases.

Method: Introduces Rubric Reward Model (RRM) - a process-oriented reward function that evaluates entire reasoning trajectories against problem-specific rubrics, providing fine-grained calibrated rewards (0-1) that penalize logical flaws and encourage rigorous deduction.

Result: RRM-based training consistently outperforms outcome-only supervision across four math benchmarks, boosting Verified Pass@1024 on AIME2024 from 26.7% to 62.6% and reducing Miracle Steps by 71%.

Conclusion: Rewarding the solution process is crucial for building models that are not only more accurate but also more reliable, as process-oriented evaluation mitigates reward hacking and improves reasoning quality.

Abstract: Large language models for mathematical reasoning are typically trained with outcome-based rewards, which credit only the final answer. In our experiments, we observe that this paradigm is highly susceptible to reward hacking, leading to a substantial overestimation of a model’s reasoning ability. This is evidenced by a high incidence of false positives - solutions that reach the correct final answer through an unsound reasoning process. Through a systematic analysis with human verification, we establish a taxonomy of these failure modes, identifying patterns like Miracle Steps - abrupt jumps to a correct output without a valid preceding derivation. Probing experiments suggest a strong association between these Miracle Steps and memorization, where the model appears to recall the answer directly rather than deriving it. To mitigate this systemic issue, we introduce the Rubric Reward Model (RRM), a process-oriented reward function that evaluates the entire reasoning trajectory against problem-specific rubrics. The generative RRM provides fine-grained, calibrated rewards (0-1) that explicitly penalize logical flaws and encourage rigorous deduction. When integrated into a reinforcement learning pipeline, RRM-based training consistently outperforms outcome-only supervision across four math benchmarks. Notably, it boosts Verified Pass@1024 on AIME2024 from 26.7% to 62.6% and reduces the incidence of Miracle Steps by 71%. Our work demonstrates that rewarding the solution process is crucial for building models that are not only more accurate but also more reliable.

Method: Proposes disentangling refusal-related features from hallucination features using sparse autoencoders, and preserving refusal behavior during fine-tuning through subspace orthogonalization to prevent unintentional suppression of factual knowledge.

Result: Evaluation on commonsense reasoning tasks and harmful benchmarks (AdvBench and StrongReject) shows the approach successfully preserves refusal behavior and task utility while mitigating hallucinations, effectively addressing the truthfulness-safety trade-off.

Conclusion: The proposed method successfully disentangles refusal and hallucination features, enabling improvement of LLM truthfulness without compromising safety alignment, providing a solution to the critical trade-off between these two important capabilities.

Abstract: Hallucination in large language models (LLMs) has been widely studied in recent years, with progress in both detection and mitigation aimed at improving truthfulness. Yet, a critical side effect remains largely overlooked: enhancing truthfulness can negatively impact safety alignment. In this paper, we investigate this trade-off and show that increasing factual accuracy often comes at the cost of weakened refusal behavior. Our analysis reveals that this arises from overlapping components in the model that simultaneously encode hallucination and refusal information, leading alignment methods to suppress factual knowledge unintentionally. We further examine how fine-tuning on benign datasets, even when curated for safety, can degrade alignment for the same reason. To address this, we propose a method that disentangles refusal-related features from hallucination features using sparse autoencoders, and preserves refusal behavior during fine-tuning through subspace orthogonalization. This approach prevents hallucinations from increasing while maintaining safety alignment.We evaluate our method on commonsense reasoning tasks and harmful benchmarks (AdvBench and StrongReject). Results demonstrate that our approach preserves refusal behavior and task utility, mitigating the trade-off between truthfulness and safety.

Method: Constructs an instance relation learning network with label knowledge propagation to learn interaction relations between instances with class information, propagating label knowledge between support and query sets. Uses dual relation-enhanced loss to optimize support- and query-level relation strength.

Result: Outperforms strong baselines by an average of 9.54% AUC and 11.19% Macro-F1 in 1-shot scenarios.

Conclusion: The proposed end-to-end multi-label joint learning method with instance relation learning and label knowledge propagation effectively addresses error propagation in few-shot multi-label intent detection.

Abstract: Few-shot Multi-label Intent Detection (MID) is crucial for dialogue systems, aiming to detect multiple intents of utterances in low-resource dialogue domains. Previous studies focus on a two-stage pipeline. They first learn representations of utterances with multiple labels and then use a threshold-based strategy to identify multi-label results. However, these methods rely on representation classification and ignore instance relations, leading to error propagation. To solve the above issues, we propose a multi-label joint learning method for few-shot MID in an end-to-end manner, which constructs an instance relation learning network with label knowledge propagation to eliminate error propagation. Concretely, we learn the interaction relations between instances with class information to propagate label knowledge between a few labeled (support set) and unlabeled (query set) instances. With label knowledge propagation, the relation strength between instances directly indicates whether two utterances belong to the same intent for multi-label prediction. Besides, a dual relation-enhanced loss is developed to optimize support- and query-level relation strength to improve performance. Experiments show that we outperform strong baselines by an average of 9.54% AUC and 11.19% Macro-F1 in 1-shot scenarios.

Method: Formalized drift as turn-wise KL divergence between test model and reference model predictions. Proposed a recurrence model interpreting drift evolution as bounded stochastic process with restoring forces and controllable interventions. Tested on synthetic rewriting tasks and realistic user-agent simulations using τ-Bench.

Result: Experiments revealed stable, noise-limited equilibria rather than runaway degradation. Simple reminder interventions reliably reduced divergence in line with theoretical predictions.

Conclusion: Multi-turn drift can be understood as a controllable equilibrium phenomenon rather than inevitable decay, providing foundation for studying and mitigating context drift in extended interactions.

Abstract: Large Language Models (LLMs) excel at single-turn tasks such as instruction following and summarization, yet real-world deployments require sustained multi-turn interactions where user goals and conversational context persist and evolve. A recurring challenge in this setting is context drift: the gradual divergence of a model’s outputs from goal-consistent behavior across turns. Unlike single-turn errors, drift unfolds temporally and is poorly captured by static evaluation metrics. In this work, we present a study of context drift in multi-turn interactions and propose a simple dynamical framework to interpret its behavior. We formalize drift as the turn-wise KL divergence between the token-level predictive distributions of the test model and a goal-consistent reference model, and propose a recurrence model that interprets its evolution as a bounded stochastic process with restoring forces and controllable interventions. We instantiate this framework in both synthetic long-horizon rewriting tasks and realistic user-agent simulations such as in $\tau$-Bench, measuring drift for several open-weight LLMs that are used as user simulators. Our experiments consistently reveal stable, noise-limited equilibria rather than runaway degradation, and demonstrate that simple reminder interventions reliably reduce divergence in line with theoretical predictions. Together, these results suggest that multi-turn drift can be understood as a controllable equilibrium phenomenon rather than as inevitable decay, providing a foundation for studying and mitigating context drift in extended interactions.

Method: Update pruned parameters under rotation constraint to preserve output geometry, combined with variance-aware importance scoring to prioritize dimensions with large variance.

Result: Consistently better perplexity on WikiText-2 and higher task accuracy on multiple language understanding benchmarks compared to existing baselines.

Conclusion: The proposed method effectively compensates pruning errors while retaining important components in a geometry-preserving manner.

Abstract: In this paper, we propose a rotation-constrained compensation method to address the errors introduced by structured pruning of large language models (LLMs). LLMs are trained on massive datasets and accumulate rich semantic knowledge in their representation space. In contrast, pruning is typically carried out with only a small amount of calibration data, which makes output mismatches unavoidable. Although direct least-squares fitting can reduce such errors, it tends to overfit to the limited calibration set, destructively modifying pretrained weights. To overcome this difficulty, we update the pruned parameters under a rotation constraint. This constrained update preserves the geometry of output representations (i.e., norms and inner products) and simultaneously re-aligns the pruned subspace with the original outputs. Furthermore, in rotation-constrained compensation, removing components that strongly contribute to the principal directions of the output makes error recovery difficult. Since input dimensions with large variance strongly affect these principal directions, we design a variance-aware importance score that ensures such dimensions are preferentially kept in the pruned model. By combining this scoring rule with rotation-constrained updates, the proposed method effectively compensates errors while retaining the components likely to be more important in a geometry-preserving manner. In the experiments, we apply the proposed method to LLaMA-7B and evaluate it on WikiText-2 and multiple language understanding benchmarks. The results demonstrate consistently better perplexity and task accuracy compared with existing baselines.

Method: Categorized 58 foundation and agentic models into five families (foundation, text-bridge, spatial, multimodal, epigenomic, and agentic), mapped them to eight analytical tasks, and evaluated using over 40 public datasets across 10 domain dimensions.

Result: Provided the first integrated view of language-driven single-cell intelligence, analyzing benchmark suitability, data diversity, and ethical/scalability constraints while evaluating models across biological grounding, multi-omics alignment, fairness, privacy, and explainability.

Conclusion: Outlined open challenges in interpretability, standardization, and trustworthy model development for language-driven single-cell intelligence.

Abstract: Large language models (LLMs) and emerging agentic frameworks are beginning to transform single-cell biology by enabling natural-language reasoning, generative annotation, and multimodal data integration. However, progress remains fragmented across data modalities, architectures, and evaluation standards. LLM4Cell presents the first unified survey of 58 foundation and agentic models developed for single-cell research, spanning RNA, ATAC, multi-omic, and spatial modalities. We categorize these methods into five families-foundation, text-bridge, spatial, multimodal, epigenomic, and agentic-and map them to eight key analytical tasks including annotation, trajectory and perturbation modeling, and drug-response prediction. Drawing on over 40 public datasets, we analyze benchmark suitability, data diversity, and ethical or scalability constraints, and evaluate models across 10 domain dimensions covering biological grounding, multi-omics alignment, fairness, privacy, and explainability. By linking datasets, models, and evaluation domains, LLM4Cell provides the first integrated view of language-driven single-cell intelligence and outlines open challenges in interpretability, standardization, and trustworthy model development.

Method: HiPRAG incorporates fine-grained, knowledge-grounded process rewards into RL training by decomposing reasoning trajectories into discrete steps and applying hierarchical rewards based on optimal search/non-search step proportions, alongside outcome and format rewards.

Result: Experiments on Qwen2.5 and Llama-3.2 models across seven QA benchmarks achieved average accuracies of 65.4% (3B) and 67.2% (7B), while reducing over-search rate to 2.3% and lowering under-search rate, demonstrating good generalizability across RL algorithms and model types.

Conclusion: HiPRAG demonstrates the importance of fine-grained process control in RL for improving reasoning efficiency and optimality in search agents, showing that optimizing the reasoning process itself is crucial beyond just final outcomes.

Abstract: Agentic RAG is a powerful technique for incorporating external information that LLMs lack, enabling better problem solving and question answering. However, suboptimal search behaviors exist widely, such as over-search (retrieving information already known) and under-search (failing to search when necessary), which leads to unnecessary overhead and unreliable outputs. Current training methods, which typically rely on outcome-based rewards in a RL framework, lack the fine-grained control needed to address these inefficiencies. To overcome this, we introduce Hierarchical Process Rewards for Efficient agentic RAG (HiPRAG), a training methodology that incorporates a fine-grained, knowledge-grounded process reward into the RL training. Our approach evaluates the necessity of each search decision on-the-fly by decomposing the agent’s reasoning trajectory into discrete, parsable steps. We then apply a hierarchical reward function that provides an additional bonus based on the proportion of optimal search and non-search steps, on top of commonly used outcome and format rewards. Experiments on the Qwen2.5 and Llama-3.2 models across seven diverse QA benchmarks show that our method achieves average accuracies of 65.4% (3B) and 67.2% (7B). This is accomplished while improving search efficiency, reducing the over-search rate to just 2.3% and concurrently lowering the under-search rate. These results demonstrate the efficacy of optimizing the reasoning process itself, not just the final outcome. Further experiments and analysis demonstrate that HiPRAG shows good generalizability across a wide range of RL algorithms, model families, sizes, and types. This work demonstrates the importance and potential of fine-grained control through RL, for improving the efficiency and optimality of reasoning for search agents.

Method: GTD formulates topology synthesis as an iterative construction process using conditional discrete graph diffusion models, guided by a lightweight proxy model that predicts multi-objective rewards for real-time optimization.

Result: GTD generates highly task-adaptive, sparse, and efficient communication topologies that significantly outperform existing methods in LLM agent collaboration across multiple benchmarks.

Conclusion: The iterative guided synthesis process enables GTD to better navigate complex design trade-offs and generate optimal communication topologies for multi-agent LLM systems.

Abstract: The efficiency of multi-agent systems driven by large language models (LLMs) largely hinges on their communication topology. However, designing an optimal topology is a non-trivial challenge, as it requires balancing competing objectives such as task performance, communication cost, and robustness. Existing frameworks often rely on static or hand-crafted topologies, which inherently fail to adapt to diverse task requirements, leading to either excessive token consumption for simple problems or performance bottlenecks for complex ones. To address this challenge, we introduce a novel generative framework called \textit{Guided Topology Diffusion (GTD)}. Inspired by conditional discrete graph diffusion models, GTD formulates topology synthesis as an iterative construction process. At each step, the generation is steered by a lightweight proxy model that predicts multi-objective rewards (e.g., accuracy, utility, cost), enabling real-time, gradient-free optimization towards task-adaptive topologies. This iterative, guided synthesis process distinguishes GTD from single-step generative frameworks, enabling it to better navigate complex design trade-offs. We validated GTD across multiple benchmarks, and experiments show that this framework can generate highly task-adaptive, sparse, and efficient communication topologies, significantly outperforming existing methods in LLM agent collaboration.

Method: Proposed MGR-CSC framework that unifies semantically equivalent multilingual keywords into shared atoms to align semantics and compresses identifier space, with dynamic multi-step constrained decoding strategy during retrieval.

Result: Achieved 6.83% improvement on mMarco100k and 4.77% on mNQ320k in retrieval accuracy, while reducing document identifiers length by 74.51% and 78.2% respectively.

Conclusion: MGR-CSC effectively addresses cross-lingual alignment challenges and enhances decoding efficiency through semantic compression and identifier space reduction.

Abstract: Generative Information Retrieval is an emerging retrieval paradigm that exhibits remarkable performance in monolingual scenarios.However, applying these methods to multilingual retrieval still encounters two primary challenges, cross-lingual identifier misalignment and identifier inflation. To address these limitations, we propose Multilingual Generative Retrieval via Cross-lingual Semantic Compression (MGR-CSC), a novel framework that unifies semantically equivalent multilingual keywords into shared atoms to align semantics and compresses the identifier space, and we propose a dynamic multi-step constrained decoding strategy during retrieval. MGR-CSC improves cross-lingual alignment by assigning consistent identifiers and enhances decoding efficiency by reducing redundancy. Experiments demonstrate that MGR-CSC achieves outstanding retrieval accuracy, improving by 6.83% on mMarco100k and 4.77% on mNQ320k, while reducing document identifiers length by 74.51% and 78.2%, respectively.

Method: AdaSwitch dynamically combines on-policy and off-policy generation at token level, allowing students to explore their own predictions and selectively integrate teacher guidance based on real-time quality assessment.

Result: Experiments on three datasets with two teacher-student LLM pairs show that AdaSwitch consistently improves accuracy with acceptable additional overhead.

Conclusion: AdaSwitch offers a practical and effective method for distilling small language models by simultaneously preserving consistency and maintaining supervision quality.

Abstract: Small language models (SLMs) are crucial for applications with strict latency and computational constraints, yet achieving high performance remains challenging. Knowledge distillation (KD) can transfer capabilities from large teacher models, but existing methods involve trade-offs: off-policy distillation provides high-quality supervision but introduces a training-inference mismatch, while on-policy approaches maintain consistency but rely on low-quality student outputs. To address these issues, we propose AdaSwitch, a novel approach that dynamically combines on-policy and off-policy generation at the token level. AdaSwitch allows the student to first explore its own predictions and then selectively integrate teacher guidance based on real-time quality assessment. This approach simultaneously preserves consistency and maintains supervision quality. Experiments on three datasets with two teacher-student LLM pairs demonstrate that AdaSwitch consistently improves accuracy, offering a practical and effective method for distilling SLMs with acceptable additional overhead.

Method: Proposed a hybrid bias detection pipeline combining rule-based heuristics, semantic similarity filtering, and LLM-based validation, with evaluation across 24 bidirectional language pairs using multiple metrics.

Result: Created a high-quality, bias-annotated dataset of 1,439 translation-reference pairs based on human evaluations, with code and dataset publicly available on GitHub.

Conclusion: Translation Tangles provides a comprehensive framework to assess and improve translation quality and fairness in LLMs, addressing critical gaps in current evaluation methods.

Abstract: The rise of Large Language Models (LLMs) has redefined Machine Translation (MT), enabling context-aware and fluent translations across hundreds of languages and textual domains. Despite their remarkable capabilities, LLMs often exhibit uneven performance across language families and specialized domains. Moreover, recent evidence reveals that these models can encode and amplify different biases present in their training data, posing serious concerns for fairness, especially in low-resource languages. To address these gaps, we introduce Translation Tangles, a unified framework and dataset for evaluating the translation quality and fairness of open-source LLMs. Our approach benchmarks 24 bidirectional language pairs across multiple domains using different metrics. We further propose a hybrid bias detection pipeline that integrates rule-based heuristics, semantic similarity filtering, and LLM-based validation. We also introduce a high-quality, bias-annotated dataset based on human evaluations of 1,439 translation-reference pairs. The code and dataset are accessible on GitHub: https://github.com/faiyazabdullah/TranslationTangles

Method: Developed TRACE analyzer that: 1) decomposes thought process into minimally complete sub-thoughts, 2) infers discourse relationships to construct granular thought progression graphs, 3) identifies common thinking patterns for similar queries.

Result: Identified two major overthinking patterns in open-weight models: Explorer and Late Landing. Found that long-thinking models are 5-20x slower on simple tasks with no substantial accuracy improvements. Over-verification and over-exploration are the primary drivers of overthinking.

Conclusion: Proposed a utility-based definition of overthinking that moves beyond length-based metrics, providing more insightful understanding of LLMs’ thought progression and practical guidelines for principled overthinking management.

Abstract: Models employing long chain-of-thought (CoT) reasoning have shown superior performance on complex reasoning tasks. Yet, this capability introduces a critical and often overlooked inefficiency – overthinking – models often engage in unnecessarily extensive reasoning even for simple queries, incurring significant computations without accuracy improvements. While prior work has explored solutions to mitigate overthinking, a fundamental gap remains in our understanding of its underlying causes. Most existing analyses are limited to superficial, profiling-based observations, failing to delve into LLMs’ inner workings. This study introduces a systematic, fine-grained analyzer of LLMs’ thought process to bridge the gap, TRACE. We first benchmark the overthinking issue, confirming that long-thinking models are five to twenty times slower on simple tasks with no substantial gains. We then use TRACE to first decompose the thought process into minimally complete sub-thoughts. Next, by inferring discourse relationships among sub-thoughts, we construct granular thought progression graphs and subsequently identify common thinking patterns for topically similar queries. Our analysis reveals two major patterns for open-weight thinking models – Explorer and Late Landing. This finding provides evidence that over-verification and over-exploration are the primary drivers of overthinking in LLMs. Grounded in thought structures, we propose a utility-based definition of overthinking, which moves beyond length-based metrics. This revised definition offers a more insightful understanding of LLMs’ thought progression, as well as practical guidelines for principled overthinking management.

Method: Proposed Chain of Recognition (CoR) to enhance understanding and Keyword Highlighting (KH) to guide generation, along with data construction and training approaches to improve language alignment capabilities.

Result: Improved knowledge accuracy from 25.14% to 46.13%, open-ended understanding rate from 64.5% to 86.5%, and significantly reduced pronunciation errors in secondary language.

Conclusion: The proposed methods effectively address language alignment issues in speech-to-speech models for code-switching scenarios, with CS3-Bench serving as a valuable benchmark for future research.

Abstract: The advancement of multimodal large language models has accelerated the development of speech-to-speech interaction systems. While natural monolingual interaction has been achieved, we find existing models exhibit deficiencies in language alignment. In our proposed Code-Switching Speech-to-Speech Benchmark (CS3-Bench), experiments on 7 mainstream models demonstrate a relative performance drop of up to 66% in knowledge-intensive question answering and varying degrees of misunderstanding in open-ended conversations. Starting from a model with severe performance deterioration, we propose both data constructions and training approaches to improve the language alignment capabilities, specifically employing Chain of Recognition (CoR) to enhance understanding and Keyword Highlighting (KH) to guide generation. Our approach improves the knowledge accuracy from 25.14% to 46.13%, with open-ended understanding rate from 64.5% to 86.5%, and significantly reduces pronunciation errors in the secondary language. CS3-Bench is available at https://huggingface.co/datasets/VocalNet/CS3-Bench.

Method: Leverages implicit rewards through log-likelihood ratios, connects them with Contrastive Decoding, and proposes ConG framework that uses contrastive decoding between pre- and post-alignment weak models to generate higher-quality samples.

Result: Empirical results across different model families show consistent improvements in capability transfer, denoising, and robustness, substantially mitigating limitations of traditional weak-to-strong methods.

Conclusion: ConG has potential to advance weak-to-strong generalization and provides a promising pathway toward AGI through more reliable capability transfer and improved robustness.

Abstract: Weak-to-strong generalization provides a promising paradigm for scaling large language models (LLMs) by training stronger models on samples from aligned weaker ones, without requiring human feedback or explicit reward modeling. However, its robustness and generalization are hindered by the noise and biases in weak-model outputs, which limit its applicability in practice. To address this challenge, we leverage implicit rewards, which approximate explicit rewards through log-likelihood ratios, and reveal their structural equivalence with Contrastive Decoding (CD), a decoding strategy shown to reduce noise in LLM generation. Building on this connection, we propose Contrastive Weak-to-Strong Generalization (ConG), a framework that employs contrastive decoding between pre- and post-alignment weak models to generate higher-quality samples. This approach enables more reliable capability transfer, denoising, and improved robustness, substantially mitigating the limitations of traditional weak-to-strong methods. Empirical results across different model families confirm consistent improvements, demonstrating the generality and effectiveness of ConG. Taken together, our findings highlight the potential of ConG to advance weak-to-strong generalization and provide a promising pathway toward AGI.

Method: Compared standard-to-dialect transfer in three settings: text models, speech models, and cascaded systems (speech transcribed then processed by text models) for German and German dialects in intent/topic classification.

Result: Speech-only setup provides best results on dialect data; text-only setup works best on standard data; cascaded systems perform relatively well on dialect data when transcription generates normalized, standard-like output.

Conclusion: Speech models are more effective for dialect processing than text models, and cascaded systems can be viable for dialects when using normalized transcriptions.

Abstract: Research on cross-dialectal transfer from a standard to a non-standard dialect variety has typically focused on text data. However, dialects are primarily spoken, and non-standard spellings are known to cause issues in text processing. We compare standard-to-dialect transfer in three settings: text models, speech models, and cascaded systems where speech first gets automatically transcribed and then further processed by a text model. In our experiments, we focus on German and multiple German dialects in the context of written and spoken intent and topic classification. To that end, we release the first dialectal audio intent classification dataset. We find that the speech-only setup provides the best results on the dialect data while the text-only setup works best on the standard data. While the cascaded systems lag behind the text-only models for German, they perform relatively well on the dialectal data if the transcription system generates normalized, standard-like output.

Method: MCBench uses string-matching NLP metrics with step-by-step instructions, providing parallel reference code for objective evaluation. It includes three evaluative metrics and three benchmark variants to measure detailed instruction understanding.

Result: MCBench serves as an effective and objective tool for evaluating cutting-edge LLMs’ capabilities in maintaining accurate step-by-step execution, instruction adherence, numerical computation, and long-range consistency.

Conclusion: The benchmark provides a deterministic and code-verifiable evaluation method that systematically tests LLMs’ ability to follow complex instructions and maintain consistency in intermediate results.

Abstract: Recent frontier-level LLMs have saturated many previously difficult benchmarks, leaving little room for further differentiation. This progress highlights the need for challenging benchmarks that provide objective verification. In this paper, we introduce MCBench, a benchmark designed to evaluate whether LLMs can execute string-matching NLP metrics by strictly following step-by-step instructions. Unlike prior benchmarks that depend on subjective judgments or general reasoning, MCBench offers an objective, deterministic and codeverifiable evaluation. This setup allows us to systematically test whether LLMs can maintain accurate step-by-step execution, including instruction adherence, numerical computation, and long-range consistency in handling intermediate results. To ensure objective evaluation of these abilities, we provide a parallel reference code that can evaluate the accuracy of LLM output. We provide three evaluative metrics and three benchmark variants designed to measure the detailed instruction understanding capability of LLMs. Our analyses show that MCBench serves as an effective and objective tool for evaluating the capabilities of cutting-edge LLMs.

Method: ACE (Attribution-Controlled Knowledge Editing) leverages neuron-level attribution to identify and edit critical query-value pathways, where implicit subjects function as query neurons that activate corresponding value neurons across transformer layers to accumulate information.

Result: ACE empirically outperforms state-of-the-art methods by 9.44% on GPT-J and 37.46% on Qwen3-8B, with analysis revealing fine-grained activation patterns and showing that semantic interpretability of value neurons is orchestrated by query-driven accumulation.

Conclusion: The findings establish a new pathway for advancing knowledge editing capabilities based on principled understanding of internal reasoning mechanisms, providing a mechanistically grounded solution for multi-hop knowledge editing.

Abstract: Large Language Models (LLMs) require efficient knowledge editing (KE) to update factual information, yet existing methods exhibit significant performance decay in multi-hop factual recall. This failure is particularly acute when edits involve intermediate implicit subjects within reasoning chains. Through causal analysis, we reveal that this limitation stems from an oversight of how chained knowledge is dynamically represented and utilized at the neuron level. We discover that during multi hop reasoning, implicit subjects function as query neurons, which sequentially activate corresponding value neurons across transformer layers to accumulate information toward the final answer, a dynamic prior KE work has overlooked. Guided by this insight, we propose ACE: Attribution-Controlled Knowledge Editing for Multi-hop Factual Recall, a framework that leverages neuron-level attribution to identify and edit these critical query-value (Q-V) pathways. ACE provides a mechanistically grounded solution for multi-hop KE, empirically outperforming state-of-the-art methods by 9.44% on GPT-J and 37.46% on Qwen3-8B. Our analysis further reveals more fine-grained activation patterns in Qwen3 and demonstrates that the semantic interpretability of value neurons is orchestrated by query-driven accumulation. These findings establish a new pathway for advancing KE capabilities based on the principled understanding of internal reasoning mechanisms.

Method: Integrates four complementary modules: basic text matching, LLM-based key knowledge point comparison, pseudo-question generation from student answers, and human-like evaluation simulation identifying content/non-content strengths/weaknesses.

Result: Extensive experiments show consistent outperformance over traditional and LLM-based baselines across multiple grading metrics. Successfully deployed in real-world training and certification exams at a major e-commerce enterprise.

Conclusion: The proposed framework provides effective and generalizable automatic grading for diverse subjective questions, demonstrating practical utility in real-world educational settings.

Abstract: Automatic grading of subjective questions remains a significant challenge in examination assessment due to the diversity in question formats and the open-ended nature of student responses. Existing works primarily focus on a specific type of subjective question and lack the generality to support comprehensive exams that contain diverse question types. In this paper, we propose a unified Large Language Model (LLM)-enhanced auto-grading framework that provides human-like evaluation for all types of subjective questions across various domains. Our framework integrates four complementary modules to holistically evaluate student answers. In addition to a basic text matching module that provides a foundational assessment of content similarity, we leverage the powerful reasoning and generative capabilities of LLMs to: (1) compare key knowledge points extracted from both student and reference answers, (2) generate a pseudo-question from the student answer to assess its relevance to the original question, and (3) simulate human evaluation by identifying content-related and non-content strengths and weaknesses. Extensive experiments on both general-purpose and domain-specific datasets show that our framework consistently outperforms traditional and LLM-based baselines across multiple grading metrics. Moreover, the proposed system has been successfully deployed in real-world training and certification exams at a major e-commerce enterprise.

Method: StepER employs step-wise supervision to align with evolving information and reasoning demands across stages, and incorporates difficulty-aware training to progressively optimize learning by prioritizing suitable steps. It’s adaptable to various multi-step retrieval-augmented language models.

Result: Extensive experiments show StepER outperforms prior methods on multi-hop QA benchmarks, with an 8B model achieving performance comparable to a 70B teacher model.

Conclusion: StepER effectively enhances reasoning ability in multi-step retrieval-augmented language models through step-wise knowledge distillation, enabling smaller models to achieve performance comparable to much larger teacher models.

Abstract: Answering complex real-world questions requires step-by-step retrieval and integration of relevant information to generate well-grounded responses. However, existing knowledge distillation methods overlook the need for different reasoning abilities at different steps, hindering transfer in multi-step retrieval-augmented frameworks. To address this, we propose Stepwise Knowledge Distillation for Enhancing Reasoning Ability in Multi-Step Retrieval-Augmented Language Models (StepER). StepER employs step-wise supervision to align with evolving information and reasoning demands across stages. Additionally, it incorporates difficulty-aware training to progressively optimize learning by prioritizing suitable steps. Our method is adaptable to various multi-step retrieval-augmented language models, including those that use retrieval queries for reasoning paths or decomposed questions. Extensive experiments show that StepER outperforms prior methods on multi-hop QA benchmarks, with an 8B model achieving performance comparable to a 70B teacher model.

Method: Three automated evaluation strategies: (1) NLI-based method decomposing texts into atomic statements, (2) Q&A-based approach comparing responses across sources, and (3) end-to-end method directly identifying missing content using LLMs.

Result: The simple end-to-end approach demonstrated surprising effectiveness compared to more complex methods, though with reduced robustness, interpretability and result granularity.

Conclusion: End-to-end LLM methods are effective for detecting missing information in LLM outputs, but trade-offs exist in robustness and interpretability compared to more complex approaches.

Abstract: Despite demonstrating remarkable performance across a wide range of tasks, large language models (LLMs) have also been found to frequently produce outputs that are incomplete or selectively omit key information. In sensitive domains, such omissions can result in significant harm comparable to that posed by factual inaccuracies, including hallucinations. In this study, we address the challenge of evaluating the comprehensiveness of LLM-generated texts, focusing on the detection of missing information or underrepresented viewpoints. We investigate three automated evaluation strategies: (1) an NLI-based method that decomposes texts into atomic statements and uses natural language inference (NLI) to identify missing links, (2) a Q&A-based approach that extracts question-answer pairs and compares responses across sources, and (3) an end-to-end method that directly identifies missing content using LLMs. Our experiments demonstrate the surprising effectiveness of the simple end-to-end approach compared to more complex methods, though at the cost of reduced robustness, interpretability and result granularity. We further assess the comprehensiveness of responses from several popular open-weight LLMs when answering user queries based on multiple sources.

Method: Used LLMs for three main tasks: enriching historical dictionaries with modern word forms and meanings, using vision-enabled LLMs for text recognition on Fraktur script sources, and employing overlapping tiling with multiple LLMs for digitizing dictionary sections.

Result: Claude 3.7 Sonnet provided accurate meanings and modern equivalents for 81% of headword entries with sufficient context. Zero-shot text recognition successfully identified and structured 41% of headword entries into error-free JSON. Overlapping tiling method used for digitizing 1780 grammar dictionary.

Conclusion: LLMs have significant potential for saving time and financial resources in historical dictionary research, even for minor languages like Estonian.

Abstract: This article presents research conducted at the Institute of the Estonian Language between 2022 and 2025 on the application of large language models (LLMs) to the study of 17th and 18th century Estonian dictionaries. The authors address three main areas: enriching historical dictionaries with modern word forms and meanings; using vision-enabled LLMs to perform text recognition on sources printed in Gothic script (Fraktur); and preparing for the creation of a unified, cross-source dataset. Initial experiments with J. Gutslaff’s 1648 dictionary indicate that LLMs have significant potential for semi-automatic enrichment of dictionary information. When provided with sufficient context, Claude 3.7 Sonnet accurately provided meanings and modern equivalents for 81% of headword entries. In a text recognition experiment with A. T. Helle’s 1732 dictionary, a zero-shot method successfully identified and structured 41% of headword entries into error-free JSON-formatted output. For digitising the Estonian-German dictionary section of A. W. Hupel’s 1780 grammar, overlapping tiling of scanned image files is employed, with one LLM being used for text recognition and a second for merging the structured output. These findings demonstrate that even for minor languages LLMs have a significant potential for saving time and financial resources.

Method: A²Search uses an automated pipeline with trajectory sampling and evidence verification to detect ambiguous questions and gather alternative answers. The model is optimized with RL using a specially designed AnsF1 reward that accommodates multiple answers.

Result: A²Search achieves state-of-the-art performance across eight open-domain QA benchmarks. A²Search-7B yields average AnsF1@1 score of 48.4% across four multi-hop benchmarks, outperforming larger models like ReSearch-32B (46.2%).

Conclusion: A²Search effectively resolves ambiguity and generalizes across benchmarks, demonstrating that embracing ambiguity is essential for building more reliable QA systems.

Abstract: Recent advances in Large Language Models (LLMs) and Reinforcement Learning (RL) have led to strong performance in open-domain question answering (QA). However, existing models still struggle with questions that admit multiple valid answers. Standard QA benchmarks, which typically assume a single gold answer, overlook this reality and thus produce inappropriate training signals. Existing attempts to handle ambiguity often rely on costly manual annotation, which is difficult to scale to multi-hop datasets such as HotpotQA and MuSiQue. In this paper, we present A$^2$Search, an annotation-free, end-to-end training framework to recognize and handle ambiguity. At its core is an automated pipeline that detects ambiguous questions and gathers alternative answers via trajectory sampling and evidence verification. The model is then optimized with RL using a carefully designed $\mathrm{AnsF1}$ reward, which naturally accommodates multiple answers. Experiments on eight open-domain QA benchmarks demonstrate that A$^2$Search achieves new state-of-the-art performance. With only a single rollout, A$^2$Search-7B yields an average $\mathrm{AnsF1}@1$ score of $48.4%$ across four multi-hop benchmarks, outperforming all strong baselines, including the substantially larger ReSearch-32B ($46.2%$). Extensive analyses further show that A$^2$Search resolves ambiguity and generalizes across benchmarks, highlighting that embracing ambiguity is essential for building more reliable QA systems. Our code, data, and model weights can be found at https://github.com/zfj1998/A2Search

Method: LightReasoner operates in two stages: (1) sampling stage that pinpoints critical reasoning moments through expert-amateur contrast between LLM and SLM, and (2) fine-tuning stage that aligns the expert model with these distilled examples.

Result: Across seven mathematical benchmarks, LightReasoner improves accuracy by up to 28.1%, while reducing time consumption by 90%, sampled problems by 80%, and tuned token usage by 99%, all without relying on ground-truth labels.

Conclusion: By turning weaker SLMs into effective teaching signals, LightReasoner offers a scalable and resource-efficient approach for advancing LLM reasoning capabilities.

Abstract: Large language models (LLMs) have demonstrated remarkable progress in reasoning, often through supervised fine-tuning (SFT). However, SFT is resource-intensive, relying on large curated datasets, rejection-sampled demonstrations, and uniform optimization across all tokens, even though only a fraction carry meaningful learning value. In this work, we explore a counterintuitive idea: can smaller language models (SLMs) teach larger language models (LLMs) by revealing high-value reasoning moments that reflect the latter’s unique strength? We propose LightReasoner, a novel framework that leverages the behavioral divergence between a stronger expert model (LLM) and a weaker amateur model (SLM). LightReasoner operates in two stages: (1) a sampling stage that pinpoints critical reasoning moments and constructs supervision examples capturing the expert’s advantage through expert-amateur contrast, and (2) a fine-tuning stage that aligns the expert model with these distilled examples, amplifying its reasoning strengths. Across seven mathematical benchmarks, LightReasoner improves accuracy by up to 28.1%, while reducing time consumption by 90%, sampled problems by 80%, and tuned token usage by 99%, all without relying on ground-truth labels. By turning weaker SLMs into effective teaching signals, LightReasoner offers a scalable and resource-efficient approach for advancing LLM reasoning. Code is available at: https://github.com/HKUDS/LightReasoner

Method: Propose an adaptive world model-enhanced reasoning mechanism that constructs dynamic textual world models to track entity states and temporal sequences. It monitors reasoning trajectories for confusion indicators and intervenes by providing clear world state descriptions.

Result: Evaluations on three social benchmarks show significant accuracy improvements (e.g., +10% in Hi-ToM) while reducing computational costs by up to 33.8% token reduction.

Conclusion: The adaptive world model mechanism offers a simple yet effective solution for deploying LLMs in social contexts by helping models navigate cognitive dilemmas and distinguish between external events and internal beliefs.

Abstract: While large language models (LLMs) excel in mathematical and code reasoning, we observe they struggle with social reasoning tasks, exhibiting cognitive confusion, logical inconsistencies, and conflation between objective world states and subjective belief states. Through deteiled analysis of DeepSeek-R1’s reasoning trajectories, we find that LLMs frequently encounter reasoning impasses and tend to output contradictory terms like “tricky” and “confused” when processing scenarios with multiple participants and timelines, leading to erroneous reasoning or infinite loops. The core issue is their inability to disentangle objective reality from agents’ subjective beliefs. To address this, we propose an adaptive world model-enhanced reasoning mechanism that constructs a dynamic textual world model to track entity states and temporal sequences. It dynamically monitors reasoning trajectories for confusion indicators and promptly intervenes by providing clear world state descriptions, helping models navigate through cognitive dilemmas. The mechanism mimics how humans use implicit world models to distinguish between external events and internal beliefs. Evaluations on three social benchmarks demonstrate significant improvements in accuracy (e.g., +10% in Hi-ToM) while reducing computational costs (up to 33.8% token reduction), offering a simple yet effective solution for deploying LLMs in social contexts.

Method: Two-stage pipeline: Stage 1 uses context filtering, category-specific prompt optimization via DSPy’s MIPROv2 and SIMBA, and caption candidate selection; Stage 2 applies few-shot prompting with profile figures for stylistic refinement.

Result: Category-specific prompts improved ROUGE-1 recall by +8.3% with limited precision loss (-2.8%) and BLEU-4 reduction (-10.9%). Profile-informed stylistic refinement yielded 40-48% gains in BLEU scores and 25-27% in ROUGE metrics.

Conclusion: Combining contextual understanding with author-specific stylistic adaptation can generate captions that are both scientifically accurate and stylistically faithful to the source paper.

Abstract: Scientific figure captions require both accuracy and stylistic consistency to convey visual information. Here, we present a domain-specific caption generation system for the 3rd SciCap Challenge that integrates figure-related textual context with author-specific writing styles using the LaMP-Cap dataset. Our approach uses a two-stage pipeline: Stage 1 combines context filtering, category-specific prompt optimization via DSPy’s MIPROv2 and SIMBA, and caption candidate selection; Stage 2 applies few-shot prompting with profile figures for stylistic refinement. Our experiments demonstrate that category-specific prompts outperform both zero-shot and general optimized approaches, improving ROUGE-1 recall by +8.3% while limiting precision loss to -2.8% and BLEU-4 reduction to -10.9%. Profile-informed stylistic refinement yields 40–48% gains in BLEU scores and 25–27% in ROUGE. Overall, our system demonstrates that combining contextual understanding with author-specific stylistic adaptation can generate captions that are both scientifically accurate and stylistically faithful to the source paper.

Method: Proposes MUSE framework with hierarchical Memory Module that organizes experience levels, enables autonomous reflection after sub-tasks, and converts raw trajectories into structured experience for integration.

Result: Achieves new SOTA performance on TAC benchmark using lightweight Gemini-2.5 Flash model, shows superior task completion with experience accumulation, and demonstrates strong generalization with zero-shot improvement on new tasks.

Conclusion: MUSE establishes a new paradigm for AI agents capable of real-world productivity task automation through continuous learning and self-evolution.

Abstract: Large Language Models have demonstrated remarkable capabilities across diverse domains, yet significant challenges persist when deploying them as AI agents for real-world long-horizon tasks. Existing LLM agents suffer from a critical limitation: they are test-time static and cannot learn from experience, lacking the ability to accumulate knowledge and continuously improve on the job. To address this challenge, we propose MUSE, a novel agent framework that introduces an experience-driven, self-evolving system centered around a hierarchical Memory Module. MUSE organizes diverse levels of experience and leverages them to plan and execute long-horizon tasks across multiple applications. After each sub-task execution, the agent autonomously reflects on its trajectory, converting the raw trajectory into structured experience and integrating it back into the Memory Module. This mechanism enables the agent to evolve beyond its static pretrained parameters, fostering continuous learning and self-evolution. We evaluate MUSE on the long-horizon productivity benchmark TAC. It achieves new SOTA performance by a significant margin using only a lightweight Gemini-2.5 Flash model. Sufficient Experiments demonstrate that as the agent autonomously accumulates experience, it exhibits increasingly superior task completion capabilities, as well as robust continuous learning and self-evolution capabilities. Moreover, the accumulated experience from MUSE exhibits strong generalization properties, enabling zero-shot improvement on new tasks. MUSE establishes a new paradigm for AI agents capable of real-world productivity task automation.

Method: Used both automatic evaluation and MQM-based human evaluation to compare ChatGPT’s translation performance with simple vs enhanced prompts, sentence-level vs document-level translation, and ChatGPT-3.5 vs ChatGPT-4.

Result: Document-level translation performed better than sentence-level. No clear advantage for enhanced prompts over simple ones. ChatGPT-3.5 preferred in automatic evaluation but tradeoff exists between accuracy (ChatGPT-3.5) and fluency (ChatGPT-4). ChatGPT yields competitive results against commercial translation systems.

Conclusion: ChatGPT is a viable option for Japanese-English translation, with document-level approach being more effective, though there are tradeoffs between different versions in terms of accuracy vs fluency.

Abstract: This study investigates ChatGPT for Japanese-English translation, exploring simple and enhanced prompts and comparing against commercially available translation engines. Performing both automatic and MQM-based human evaluations, we found that document-level translation outperforms sentence-level translation for ChatGPT. On the other hand, we were not able to determine if enhanced prompts performed better than simple prompts in our experiments. We also discovered that ChatGPT-3.5 was preferred by automatic evaluation, but a tradeoff exists between accuracy (ChatGPT-3.5) and fluency (ChatGPT-4). Lastly, ChatGPT yields competitive results against two widely-known translation systems.

Method: Constructed a global grid of queries at 1° resolution land points asking for mean July 2-m air temperature 1991-2020, validated responses against ERA5 reanalysis data.

Result: LLMs encode non-trivial climate structure with RMSE of 3-6°C and biases of ±1°C, but show spatially coherent errors in mountains and high latitudes. Performance degrades sharply above 1500m elevation. Geographic context reduces errors by 27% on average.

Conclusion: While LLMs capture present-day climate distributions, they fail to reproduce spatial patterns of temperature change essential for understanding climate dynamics, highlighting limitations in representing regional climate shifts.

Abstract: Large language models (LLMs) are increasingly deployed for climate-related applications, where understanding internal climatological knowledge is crucial for reliability and misinformation risk assessment. Despite growing adoption, the capacity of LLMs to recall climate normals from parametric knowledge remains largely uncharacterized. We investigate the capacity of contemporary LLMs to recall climate normals without external retrieval, focusing on a prototypical query: mean July 2-m air temperature 1991-2020 at specified locations. We construct a global grid of queries at 1{\deg} resolution land points, providing coordinates and location descriptors, and validate responses against ERA5 reanalysis. Results show that LLMs encode non-trivial climate structure, capturing latitudinal and topographic patterns, with root-mean-square errors of 3-6 {\deg}C and biases of $\pm$1 {\deg}C. However, spatially coherent errors remain, particularly in mountains and high latitudes. Performance degrades sharply above 1500 m, where RMSE reaches 5-13 {\deg}C compared to 2-4 {\deg}C at lower elevations. We find that including geographic context (country, city, region) reduces errors by 27% on average, with larger models being most sensitive to location descriptors. While models capture the global mean magnitude of observed warming between 1950-1974 and 2000-2024, they fail to reproduce spatial patterns of temperature change, which directly relate to assessing climate change. This limitation highlights that while LLMs may capture present-day climate distributions, they struggle to represent the regional and local expression of long-term shifts in temperature essential for understanding climate dynamics. Our evaluation framework provides a reproducible benchmark for quantifying parametric climate knowledge in LLMs and complements existing climate communication assessments.

Method: The survey systematically examines the full PRM loop: generating process data, building PRMs, and using them for test-time scaling and reinforcement learning. It covers applications in math, code, text, multimodal reasoning, robotics, and agents.

Result: The paper summarizes design spaces, applications across multiple domains, and emerging benchmarks for PRMs, providing a comprehensive framework for understanding process-level reasoning evaluation.

Conclusion: The survey aims to clarify PRM design spaces, reveal open challenges, and guide future research toward fine-grained, robust reasoning alignment through process-level evaluation.

Abstract: Although Large Language Models (LLMs) exhibit advanced reasoning ability, conventional alignment remains largely dominated by outcome reward models (ORMs) that judge only final answers. Process Reward Models(PRMs) address this gap by evaluating and guiding reasoning at the step or trajectory level. This survey provides a systematic overview of PRMs through the full loop: how to generate process data, build PRMs, and use PRMs for test-time scaling and reinforcement learning. We summarize applications across math, code, text, multimodal reasoning, robotics, and agents, and review emerging benchmarks. Our goal is to clarify design spaces, reveal open challenges, and guide future research toward fine-grained, robust reasoning alignment.

Method: FedDTRE uses trustworthiness scores of both global and local models on a fairness-oriented evaluation dataset to dynamically regulate the global model’s contribution during local updates, rather than directly replacing local models with the global model.

Result: Experimental results show that FedDTRE improves dialogue model performance and enhances the quality of dialogue generation.

Conclusion: FedDTRE effectively addresses the challenges of overfitting and global information forgetting in federated dialogue systems, leading to improved performance and generation quality.

Abstract: With the rapid development of artificial intelligence, dialogue systems have become a prominent form of human-computer interaction. However, traditional centralized or fully local training approaches face challenges in balancing privacy preservation and personalization due to data privacy concerns and heterogeneous device capabilities. Federated learning, as a representative distributed paradigm, offers a promising solution. However, existing methods often suffer from overfitting under limited client data and tend to forget global information after multiple training rounds, leading to poor generalization. To address these issues, we propose FedDTRE, a Federated adaptive aggregation strategy for Dialogue generation based on Trustworthiness Evaluation. Instead of directly replacing local models with the global model, FedDTRE leverages trustworthiness scores of both global and local models on a fairness-oriented evaluation dataset to dynamically regulate the global model’s contribution during local updates. Experimental results demonstrate that FedDTRE can improve dialogue model performance and enhance the quality of dialogue generation.

Method: Collected 3,000 human plausibility judgments and 13,600 LLM judgments on multiple-choice commonsense questions, testing the effect of PRO and CON rationales generated by LLMs.

Result: Human plausibility ratings increased with PRO rationales and decreased with CON rationales, showing LLM arguments are convincing. LLMs showed similar patterns of influence.

Conclusion: LLMs can significantly influence human plausibility judgments even in commonsense domains, demonstrating both a novel research application and raising concerns about LLMs’ potential to shape human beliefs.

Abstract: We investigate the degree to which human plausibility judgments of multiple-choice commonsense benchmark answers are subject to influence by (im)plausibility arguments for or against an answer, in particular, using rationales generated by LLMs. We collect 3,000 plausibility judgments from humans and another 13,600 judgments from LLMs. Overall, we observe increases and decreases in mean human plausibility ratings in the presence of LLM-generated PRO and CON rationales, respectively, suggesting that, on the whole, human judges find these rationales convincing. Experiments with LLMs reveal similar patterns of influence. Our findings demonstrate a novel use of LLMs for studying aspects of human cognition, while also raising practical concerns that, even in domains where humans are ``experts’’ (i.e., common sense), LLMs have the potential to exert considerable influence on people’s beliefs.

Method: Self-play setup across three diverse dialogue games, analyzing trade-offs between performance and cost, language consistency of reasoning processes, and strategic adaptation in commercial and open-weight LLMs across three languages.

Result: Reasoning improves GPT-5’s negotiation performance by 31.4% but increases computational cost by nearly 400%. Open-weight models switch to English for internal reasoning even when negotiating in other languages, while commercial models maintain language consistency.

Conclusion: Reasoning significantly enhances negotiation outcomes but at high computational cost, with important multilingual distinctions in reasoning language patterns that affect explainability.

Abstract: Negotiation is a fundamental challenge for AI agents, as it requires an ability to reason strategically, model opponents, and balance cooperation with competition. We conduct the first comprehensive study systematically evaluating the effect of (LLM-)reasoning on the negotiation abilities of both commercial and open-weight LLMs, and do this across three languages. Using a self-play setup across three diverse dialogue games, we analyse trade-offs between performance and cost, the language consistency of reasoning processes, and the nature of strategic adaptation exhibited by models. Our findings show that enabling reasoning-that is, scaling test time compute-significantly improves negotiation outcomes by enhancing collaboration and helping models overcome task complexities, but comes at a substantial computational cost: reasoning improves GPT-5’s performance by 31.4 % while increasing its cost by nearly 400 %. Most critically, we uncover a significant multilingual reasoning distinction: open-weight models consistently switch to English for their internal reasoning steps, even when negotiating in German or Italian (and thus possibly impacting potential explainability gains through the disclosure of reasoning traces), while leading commercial models maintain language consistency between their reasoning and final output.

Method: Established a theoretical framework for lossless vocabulary reduction that efficiently converts auto-regressive language models to models with arbitrarily small vocabularies while maintaining accuracy.

Result: The method enables language models with different tokenizations to cooperate efficiently through their maximal common vocabulary.

Conclusion: Lossless vocabulary reduction provides a practical solution for enabling cooperation between diverse language models by creating compatible vocabulary spaces without sacrificing performance.

Abstract: Tokenization – the process of decomposing a given text into a sequence of subwords called tokens – is one of the key components in the development of language models. Particularly, auto-regressive language models generate texts token by token, i.e., by predicting the next-token distribution given the previous ones, and thus tokenization directly affects their efficiency in text generation. Since each language model has their own vocabulary as a set of possible tokens, they struggle to cooperate with each other at the level of next-token distributions such as model ensemble. In this paper, we establish a theoretical framework of lossless vocabulary reduction, which efficiently converts a given auto-regressive language model into the one with an arbitrarily small vocabulary without any loss in accuracy. As an application, we demonstrate that language models with different tokenization can cooperate with each other efficiently through their maximal common vocabulary.

Method: Used 1,143 Instagram posts to compare GPT-5-nano and Gemini-2.5-flash-lite under three prompting strategies with controlled legal knowledge. Developed taxonomy of reasoning errors and analyzed LLM explanations annotated by law-trained students.

Result: Both models performed strongly in classifying sponsored content (F1 up to 0.93), but performance dropped over 10 points on ambiguous cases. Common errors included citation omissions (28.57%), unclear references (20.71%), with hidden ads having highest miscue rate (28.57%). Adding regulatory text improved explanation quality but not detection accuracy.

Conclusion: The paper contributes to regulatory compliance technology by providing a taxonomy of LLM legal reasoning errors, an original dataset of annotated explanations, and combined evaluation strategies to support advertising regulatory bodies in automating moderation with legal foundation.

Abstract: The rise of influencer marketing has blurred boundaries between organic content and sponsored content, making the enforcement of legal rules relating to transparency challenging. Effective regulation requires applying legal knowledge with a clear purpose and reason, yet current detection methods of undisclosed sponsored content generally lack legal grounding or operate as opaque “black boxes”. Using 1,143 Instagram posts, we compare gpt-5-nano and gemini-2.5-flash-lite under three prompting strategies with controlled levels of legal knowledge provided. Both models perform strongly in classifying content as sponsored or not (F1 up to 0.93), though performance drops by over 10 points on ambiguous cases. We further develop a taxonomy of reasoning errors, showing frequent citation omissions (28.57%), unclear references (20.71%), and hidden ads exhibiting the highest miscue rate (28.57%). While adding regulatory text to the prompt improves explanation quality, it does not consistently improve detection accuracy. The contribution of this paper is threefold. First, it makes a novel addition to regulatory compliance technology by providing a taxonomy of common errors in LLM-generated legal reasoning to evaluate whether automated moderation is not only accurate but also legally robust, thereby advancing the transparent detection of influencer marketing content. Second, it features an original dataset of LLM explanations annotated by two students who were trained in influencer marketing law. Third, it combines quantitative and qualitative evaluation strategies for LLM explanations and critically reflects on how these findings can support advertising regulatory bodies in automating moderation processes on a solid legal foundation.

Method: Two algorithms: CLoVE for generating verifiable concept-based contrastive local explanations, and GloVE for condensing local rules into global policies through iterative clustering, summarization and verification.

Result: Extracted global policies are highly faithful to LLM-as-a-Judge decisions, robust to text perturbations and adversarial attacks, and users showed good understanding and satisfaction with the policies.

Conclusion: The proposed approach successfully extracts interpretable global policies from LLM-as-a-Judge systems, providing transparency into their decision-making processes.

Abstract: Using LLMs to evaluate text, that is, LLM-as-a-judge, is increasingly being used at scale to augment or even replace human annotations. As such, it is imperative that we understand the potential biases and risks of doing so. In this work, we propose an approach for extracting high-level concept-based global policies from LLM-as-a-Judge. Our approach consists of two algorithms:

1. CLoVE (Contrastive Local Verifiable Explanations), which generates verifiable, concept-based, contrastive local explanations and 2) GloVE (Global Verifiable Explanations), which uses iterative clustering, summarization and verification to condense local rules into a global policy. We evaluate GloVE on seven standard benchmarking datasets for content harm detection. We find that the extracted global policies are highly faithful to decisions of the LLM-as-a-Judge. Additionally, we evaluated the robustness of global policies to text perturbations and adversarial attacks. Finally, we conducted a user study to evaluate user understanding and satisfaction with global policies.

Method: Multi-agent collaborative framework simulating client-server polling mechanism to optimize judgment models unsupervisedly through group interactions.

Result: Outperforms supervised models trained on annotated data, improves performance across different client agents, and effectively mitigates judgment preference bias.

Conclusion: Genii provides an effective unsupervised solution for reducing judgment preference bias in LLM evaluators, demonstrating superior performance without requiring human-labeled data.

Abstract: Large Language Models (LLMs) as automatic evaluators, commonly referred to as LLM-as-a-Judge, have also attracted growing attention. This approach plays a vital role in aligning LLMs with human judgments, providing accurate and reliable assessments. However, LLM-based judgment models often exhibit judgment preference bias during the evaluation phase, tending to favor responses generated by themselves, undermining the reliability of their judgments. This paper introduces the Group-Based Polling Optimization (Genii), an unsupervised multi-agent collaborative optimization framework that mitigates the inherent judgment preference bias of judgment models. Specifically, Genii integrates various LLM-based judgment models into a multi-agent system and simulates the interactive client-server polling mechanism to optimize each client agent unsupervisedly. Our experiments demonstrate that Genii outperforms supervised models trained on annotated judgment data, while requiring no human-labeled annotations. Genii consistently improves performance across different client agents during the polling, even when weaker models act as server agents. Further analysis reveals that Genii effectively mitigates judgment preference bias of LLM-based judgment models, demonstrating its effectiveness. All codes are available at https://github.com/NEUIR/Genii.

Method: Extract question-answer pairs from historical customer-agent conversations to automatically build knowledge bases. Fine-tune lightweight LLM (LLaMA-3.1-8B) on internal company data.

Result: Achieves above 90% accuracy in answering information-seeking questions across 20 companies. Outperforms larger closed-source LLMs and eliminates cold-start gap in contact centers.

Conclusion: The system enables immediate deployment of RAG-powered chatbots by automatically building knowledge bases from existing conversation data, demonstrating state-of-the-art performance with lightweight models.

Abstract: The utilization of conversational AI systems by leveraging Retrieval Augmented Generation (RAG) techniques to solve customer problems has been on the rise with the rapid progress of Large Language Models (LLMs). However, the absence of a company-specific dedicated knowledge base is a major barrier to the integration of conversational AI systems in contact centers. To this end, we introduce AI Knowledge Assist, a system that extracts knowledge in the form of question-answer (QA) pairs from historical customer-agent conversations to automatically build a knowledge base. Fine-tuning a lightweight LLM on internal data demonstrates state-of-the-art performance, outperforming larger closed-source LLMs. More specifically, empirical evaluation on 20 companies demonstrates that the proposed AI Knowledge Assist system that leverages the LLaMA-3.1-8B model eliminates the cold-start gap in contact centers by achieving above 90% accuracy in answering information-seeking questions. This enables immediate deployment of RAG-powered chatbots.

Method: Generates diverse task instructions and responses via reading comprehension on conversation transcripts, unlike conventional next-token prediction approaches.

Result: Significantly improves zero-shot generalization across business conversational tasks including meeting summarization, action item generation, and call purpose identification.

Conclusion: First work to apply instruction pre-training on business conversational data, providing insights for industries to leverage proprietary datasets for domain adaptation.

Abstract: The rapid advancements in Large Language Models (LLMs) have enabled their adoption in real-world industrial scenarios for various natural language processing tasks. However, the high inference cost of large-scale LLMs makes their deployment impractical, necessitating the use of smaller models. Despite their efficiency, smaller LLMs lack robust zero-shot instruction-following capabilities across diverse domains, limiting their adaptability to dynamic user requirements. Traditional fine-tuning approaches exacerbate this issue by inducing catastrophic forgetting, reducing the model’s generalization ability for unseen tasks. In this paper, we propose Domain Adaptive Continual Instruction Pre-Training via Reading Comprehension (DACIP-RC), a continual pre-training technique that enhances smaller LLMs’ domain adaptability for business conversational tasks. Unlike conventional pre-training approaches that rely on next-token prediction, DACIP-RC generates diverse task instructions and responses via reading comprehension on conversation transcripts, enabling better instruction generalization. Our empirical evaluations demonstrate that DACIP-RC significantly improves zero-shot generalization across a wide range of business conversational tasks, including meeting summarization, action item generation, and call purpose identification. To the best of our knowledge, this is the first work to apply instruction pre-training on business conversational data, providing insights into how industries can leverage proprietary datasets for domain adaptation.

Method: Created XSB and MS-XSB benchmarks to identify refusal triggers, then used post-hoc explanation methods to deploy three model-agnostic approaches: ignore-word instructions, prompt rephrasing, and attention steering.

Result: The benchmarks reveal exaggerated refusals persist across diverse LLMs, especially in multi-turn scenarios. The proposed methods substantially improve compliance on safe prompts while maintaining safety protections.

Conclusion: The study establishes a reproducible framework for diagnosing and mitigating exaggerated refusals, providing practical pathways to safer and more helpful LLM deployments.

Abstract: Large language models (LLMs) frequently produce false refusals, declining benign requests that contain terms resembling unsafe queries. We address this challenge by introducing two comprehensive benchmarks: the Exaggerated Safety Benchmark (XSB) for single-turn prompts, annotated with “Focus” keywords that identify refusal-inducing triggers, and the Multi-turn Scenario-based Exaggerated Safety Benchmark (MS-XSB), which systematically evaluates refusal calibration in realistic, context-rich dialog settings. Our benchmarks reveal that exaggerated refusals persist across diverse recent LLMs and are especially pronounced in complex, multi-turn scenarios. To mitigate these failures, we leverage post-hoc explanation methods to identify refusal triggers and deploy three lightweight, model-agnostic approaches, ignore-word instructions, prompt rephrasing, and attention steering, at inference time, all without retraining or parameter access. Experiments on four instruction-tuned Llama models demonstrate that these strategies substantially improve compliance on safe prompts while maintaining robust safety protections. Our findings establish a reproducible framework for diagnosing and mitigating exaggerated refusals, highlighting practical pathways to safer and more helpful LLM deployments.

Method: Uses reinforcement learning framework augmented with length-aware optimization. Integrates vision understanding for multimodal applications and executable code into reasoning to reduce token cost while preserving performance.

Result: ARM2 achieves performance on par with traditional reasoning models trained with GRPO while reducing token usage by over 70% on average.

Conclusion: ARM2 provides an effective solution to the over-thinking problem in Large Reasoning Models through adaptive reasoning that balances performance and efficiency across multiple formats.

Abstract: Large Reasoning Models (LRMs) often suffer from the ``over-thinking’' problem, generating unnecessarily long reasoning on simple tasks. Some strategies have been proposed to mitigate this issue, such as length penalties or routing mechanisms, but they are typically heuristic and task-specific, lacking a general framework for adaptive reasoning. In this paper, we present ARM2, a unified model that adaptively balances reasoning performance and efficiency across multiple formats through a reinforcement learning framework augmented with length-aware optimization. Beyond conventional natural language inference, ARM2 integrates vision understanding, extending its applicability to multimodal. Moreover, ARM2 integrates executable code into reasoning, enabling substantial reductions in token cost while preserving task performance compared to long CoT. Experiments demonstrate that ARM2 achieves performance on par with traditional reasoning models trained with GRPO, while reducing token usage by over 70% on average. We further conduct extensive analyses to validate the effectiveness of ARM2 and the soundness of its design.

Method: Developed MetricalARGS taxonomy with four task dimensions (Analysis, Retrieval, Generation, Support) and applied it to Telugu poetry as a case study.

Result: Created the first comprehensive taxonomy for poetry-related NLP tasks focused on metrical poetry evaluation in LLMs.

Conclusion: MetricalARGS provides a framework for understanding LLM capabilities and limitations through the lens of metrical poetry across multiple dimensions.

Abstract: Prior NLP work studying poetry has focused primarily on automatic poem generation and summarization. Many languages have well-studied traditions of poetic meter which enforce constraints on a poem in terms of syllable and phoneme patterns. Such advanced literary forms offer opportunities for probing deeper reasoning and language understanding in Large Language Models (LLMs) and their ability to follow strict pre-requisites and rules. In this paper, we introduce MetricalARGS, the first taxonomy of poetry-related NLP tasks designed to evaluate LLMs on metrical poetry across four dimensions: Analysis, Retrieval, Generation, and Support. We discuss how these tasks relate to existing NLP tasks, addressing questions around datasets and evaluation metrics. Taking Telugu as our example language, we illustrate how the taxonomy can be used in practice. MetricalARGS highlights the broader possibilities for understanding the capabilities and limitations of today’s LLMs through the lens of metrical poetry.

Method: Leverages group relative semantic advantage instead of numerical ones, iteratively distilling experiential knowledge as token priors during multi-epoch learning on minimal ground-truth data.

Result: Significantly improves out-of-domain performance on mathematical reasoning and web searching tasks, outperforming fine-tuned small LLMs with marginal training data and cost.

Conclusion: Training-Free GRPO provides a cost-effective solution for enhancing LLM agent performance in specialized domains without parameter updates.

Abstract: Recent advances in Large Language Model (LLM) agents have demonstrated their promising general capabilities. However, their performance in specialized real-world domains often degrades due to challenges in effectively integrating external tools and specific prompting strategies. While methods like agentic reinforcement learning have been proposed to address this, they typically rely on costly parameter updates, for example, through a process that uses Supervised Fine-Tuning (SFT) followed by a Reinforcement Learning (RL) phase with Group Relative Policy Optimization (GRPO) to alter the output distribution. However, we argue that LLMs can achieve a similar effect on the output distribution by learning experiential knowledge as a token prior, which is a far more lightweight approach that not only addresses practical data scarcity but also avoids the common issue of overfitting. To this end, we propose Training-Free Group Relative Policy Optimization (Training-Free GRPO), a cost-effective solution that enhances LLM agent performance without any parameter updates. Our method leverages the group relative semantic advantage instead of numerical ones within each group of rollouts, iteratively distilling high-quality experiential knowledge during multi-epoch learning on a minimal ground-truth data. Such knowledge serves as the learned token prior, which is seamlessly integrated during LLM API calls to guide model behavior. Experiments on mathematical reasoning and web searching tasks demonstrate that Training-Free GRPO, when applied to DeepSeek-V3.1-Terminus, significantly improves out-of-domain performance. With just a few dozen training samples, Training-Free GRPO outperforms fine-tuned small LLMs with marginal training data and cost.

Method: Proposed function token hypothesis, conducted bipartite graph analysis to show feature activation patterns, and performed case studies on how function tokens activate predictive features from context.

Result: Experimental evidence shows that a small number of function tokens activate the majority of features, and training loss is dominated by predicting content tokens following function tokens, forcing function tokens to select predictive features.

Conclusion: Function tokens play a crucial role in LLM memory mechanisms by activating predictive features during inference and driving feature learning during pre-training through next-token prediction tasks.

Abstract: The remarkable success of large language models (LLMs) stems from their ability to consolidate vast amounts of knowledge into the memory during pre-training and to retrieve it from the memory during inference, enabling advanced capabilities such as knowledge memorization, instruction-following and reasoning. However, the mechanisms of memory retrieval and consolidation in LLMs remain poorly understood. In this paper, we propose the function token hypothesis to explain the workings of LLMs: During inference, function tokens activate the most predictive features from context and govern next token prediction (memory retrieval). During pre-training, predicting the next tokens (usually content tokens) that follow function tokens increases the number of learned features of LLMs and updates the model parameters (memory consolidation). Function tokens here roughly correspond to function words in linguistics, including punctuation marks, articles, prepositions, and conjunctions, in contrast to content tokens. We provide extensive experimental evidence supporting this hypothesis. Using bipartite graph analysis, we show that a small number of function tokens activate the majority of features. Case studies further reveal how function tokens activate the most predictive features from context to direct next token prediction. We also find that during pre-training, the training loss is dominated by predicting the next content tokens following function tokens, which forces the function tokens to select the most predictive features from context.

Method: Finetune open-sourced LLMs on misaligned completions across diverse domains, test in downstream combined finetuning settings, and simulate human-AI interactions with both benign and biased users.

Result: LLMs show broadly misaligned behavior in dishonesty; introducing 1% misalignment data decreases honest behavior over 20%; with only 10% biased user population, assistant LLMs can be misaligned unintentionally to exacerbate dishonesty.

Conclusion: Emergent misalignment extends to dishonesty and deception in high-stakes scenarios, and this risk arises through direct finetuning, downstream mixture tasks, and practical human-AI interactions.

Abstract: Previous research has shown that LLMs finetuned on malicious or incorrect completions within narrow domains (e.g., insecure code or incorrect medical advice) can become broadly misaligned to exhibit harmful behaviors, which is called emergent misalignment. In this work, we investigate whether this phenomenon can extend beyond safety behaviors to a broader spectrum of dishonesty and deception under high-stakes scenarios (e.g., lying under pressure and deceptive behavior). To explore this, we finetune open-sourced LLMs on misaligned completions across diverse domains. Experimental results demonstrate that LLMs show broadly misaligned behavior in dishonesty. Additionally, we further explore this phenomenon in a downstream combined finetuning setting, and find that introducing as little as 1% of misalignment data into a standard downstream task is sufficient to decrease honest behavior over 20%. Furthermore, we consider a more practical human-AI interaction environment where we simulate both benign and biased users to interact with the assistant LLM. Notably, we find that the assistant can be misaligned unintentionally to exacerbate its dishonesty with only 10% biased user population. In summary, we extend the study of emergent misalignment to the domain of dishonesty and deception under high-stakes scenarios, and demonstrate that this risk arises not only through direct finetuning, but also in downstream mixture tasks and practical human-AI interactions.

Method: Created dataset with 10,000 annotated tweets each in English and Arabic, plus 30,000 translated tweets in Spanish, French, and Italian. Fine-tuned pre-trained transformer models for sentiment classification and evaluated compatibility with ChatGPT.

Result: Successfully developed comprehensive sentiment analysis dataset and models, enabling detailed analysis of emotional evolution across languages, countries, and topics during COVID-19 waves.

Conclusion: SenWave provides a valuable resource for fine-grained sentiment analysis of complex events, fostering more nuanced understanding and research innovations in NLP, with demonstrated robustness and versatility across applications.

Abstract: The global impact of the COVID-19 pandemic has highlighted the need for a comprehensive understanding of public sentiment and reactions. Despite the availability of numerous public datasets on COVID-19, some reaching volumes of up to 100 billion data points, challenges persist regarding the availability of labeled data and the presence of coarse-grained or inappropriate sentiment labels. In this paper, we introduce SenWave, a novel fine-grained multi-language sentiment analysis dataset specifically designed for analyzing COVID-19 tweets, featuring ten sentiment categories across five languages. The dataset comprises 10,000 annotated tweets each in English and Arabic, along with 30,000 translated tweets in Spanish, French, and Italian, derived from English tweets. Additionally, it includes over 105 million unlabeled tweets collected during various COVID-19 waves. To enable accurate fine-grained sentiment classification, we fine-tuned pre-trained transformer-based language models using the labeled tweets. Our study provides an in-depth analysis of the evolving emotional landscape across languages, countries, and topics, revealing significant insights over time. Furthermore, we assess the compatibility of our dataset with ChatGPT, demonstrating its robustness and versatility in various applications. Our dataset and accompanying code are publicly accessible on the repository\footnote{https://github.com/gitdevqiang/SenWave}. We anticipate that this work will foster further exploration into fine-grained sentiment analysis for complex events within the NLP community, promoting more nuanced understanding and research innovations.

Method: Developed a new dataset by augmenting the Causal News Corpus with concausal statements based on rigorous annotation guidelines derived from literature review on causal reasoning with incomplete knowledge.

Result: Achieved substantial inter-annotator agreement (Cohen’s κ=0.74). Models trained without concausal relationships misclassify them as procausal, but training with the new dataset enables transformers to effectively distinguish between pro- and concausality.

Conclusion: Integrating concausal statements is crucial for accurate causality extraction, as it prevents misclassification and enables models to properly distinguish between supporting and refuting causal relationships.

Abstract: Research on causality extraction from text has so far almost entirely neglected counterclaims. Existing causality extraction datasets focus solely on “procausal” claims, i.e., statements that support a relationship. “Concausal” claims, i.e., statements that refute a relationship, are entirely ignored or even accidentally annotated as procausal. We address this shortcoming by developing a new dataset that integrates concausality. Based on an extensive literature review, we first show that concausality is an integral part of causal reasoning on incomplete knowledge. We operationalize this theory in the form of a rigorous guideline for annotation and then augment the Causal News Corpus with concausal statements, obtaining a substantial inter-annotator agreement of Cohen’s $\kappa=0.74$. To demonstrate the importance of integrating concausal statements, we show that models trained without concausal relationships tend to misclassify these as procausal instead. Based on our new dataset, this mistake can be mitigated, enabling transformers to effectively distinguish pro- and concausality.

Method: Multi-agent RL framework with Dynamic Improvement Reward (DIR) that jointly trains conversation and feedback agents. Feedback agent provides suggestions to improve safety and helpfulness, and DIR evolves based on how well conversation agent incorporates feedback.

Result: Significantly reduces unsafe responses (from 39.0% to 4.6% on WildJailbreak) and overrefusals (from 45.3% to 9.9% on OR-Bench) across five datasets while preserving helpfulness and low latency on safe queries.

Conclusion: WaltzRL advances the Pareto front between helpfulness and harmlessness by enabling co-evolution of conversation and feedback agents, enhancing LLM safety without degrading general capabilities.

Abstract: Harnessing the power of LLMs requires a delicate dance between being helpful and harmless. This creates a fundamental tension between two competing challenges: vulnerability to adversarial attacks that elicit unsafe content, and a tendency for overrefusal on benign but sensitive prompts. Current approaches often navigate this dance with safeguard models that completely reject any content that contains unsafe portions. This approach cuts the music entirely-it may exacerbate overrefusals and fails to provide nuanced guidance for queries it refuses. To teach models a more coordinated choreography, we propose WaltzRL, a novel multi-agent reinforcement learning framework that formulates safety alignment as a collaborative, positive-sum game. WaltzRL jointly trains a conversation agent and a feedback agent, where the latter is incentivized to provide useful suggestions that improve the safety and helpfulness of the conversation agent’s responses. At the core of WaltzRL is a Dynamic Improvement Reward (DIR) that evolves over time based on how well the conversation agent incorporates the feedback. At inference time, unsafe or overrefusing responses from the conversation agent are improved rather than discarded. The feedback agent is deployed together with the conversation agent and only engages adaptively when needed, preserving helpfulness and low latency on safe queries. Our experiments, conducted across five diverse datasets, demonstrate that WaltzRL significantly reduces both unsafe responses (e.g., from 39.0% to 4.6% on WildJailbreak) and overrefusals (from 45.3% to 9.9% on OR-Bench) compared to various baselines. By enabling the conversation and feedback agents to co-evolve and adaptively apply feedback, WaltzRL enhances LLM safety without degrading general capabilities, thereby advancing the Pareto front between helpfulness and harmlessness.

Method: Use contrastive decoding to generate synthetic corpora by sampling from the relative difference between good and bad models trained on the same 100M-word corpus, then mix synthetic data with original training data.

Result: Training on mixed synthetic and real data improves language modeling performance and downstream tasks. Contrastive decoding synthetic data helps reasoning tasks, while traditional sampling synthetic data helps surface-level linguistic tasks.

Conclusion: Synthetic data generation, particularly using contrastive decoding, is a viable approach to enhance LLM training and performance across different types of tasks.

Abstract: Large language models (LLMs) are trained on huge amounts of textual data, and concerns have been raised that the limits of such data may soon be reached. A potential solution is to train on synthetic data sampled from LLMs. In this work, we build on this idea and investigate the benefits of contrastive decoding for generating synthetic corpora. In a controlled setting, we experiment with sampling corpora using the relative difference between a good and bad model trained on the same original corpus of 100 million words. By amplifying the signal from a model that has better performance, we create a synthetic corpus and mix it with the original training data. Our findings show that training on a mixture of synthesized and real data improves performance on the language modeling objective and a range of downstream tasks. In particular, we see that training with a mix of synthetic data from contrastive decoding benefits tasks that require more reasoning skills, while synthetic data from traditional sampling helps more on tasks dependent on surface level linguistic capabilities.

Method: Uses reverse construction to generate complex question-answer pairs from web sources, implements dynamic context management with sliding windows, and applies reinforcement learning on Qwen3-32B without external summarization models.

Result: DeepMiner-32B achieves 33.5% accuracy on BrowseComp-en (20% improvement over previous best), shows consistent improvements on multiple benchmarks, and enables sustained interactions of nearly 100 turns within standard 32k context length.

Conclusion: DeepMiner effectively addresses context limitations in multi-turn interaction systems and demonstrates substantial advancements in reasoning capabilities for long-horizon agent interactions.

Abstract: While recent advances in reasoning models have demonstrated cognitive behaviors through reinforcement learning, existing approaches struggle to invoke deep reasoning capabilities in multi-turn agents with long-horizon interactions. We propose DeepMiner, a novel framework that elicits such abilities by introducing high-difficulty training tasks and dynamic context window. DeepMiner presents a reverse construction method to generate complex but verifiable question-answer pairs from authentic web sources, which ensures the challenge and reliability of training data while injecting cognitive capabilities into multi-turn reasoning scenarios. We further design an elegant yet effective dynamic context management strategy for both training and inference, utilizing sliding window mechanisms while eliminating the dependency on external summarization models, thereby efficiently empowering the model to handle continuously expanding long-horizon contexts. Through reinforcement learning on Qwen3-32B, we develop DeepMiner-32B, which achieves substantial performance improvements across multiple search agent benchmarks. DeepMiner attains 33.5% accuracy on BrowseComp-en, surpassing the previous best open-source agent by almost 20 percentage points, and demonstrates consistent improvements on BrowseComp-zh, XBench-DeepSearch, and GAIA. Notably, our dynamic context management enables sustained interactions of nearly 100 turns within standard 32k context length, effectively addressing the context limitations that constrain existing multi-turn interaction systems.

Method: Conducted neuron-level analysis using gradient-based scoring with additional filtering to identify culture-general and culture-specific neurons in LLMs.

Result: Identified neurons concentrated in shallow to middle MLP layers; suppressing them degrades cultural benchmark performance by up to 30% while NLU performance remains unaffected; culture-specific neurons support related cultures; NLU training can diminish cultural understanding.

Conclusion: Findings provide insights into LLM internal mechanisms and offer practical guidance for model training and engineering to improve cultural understanding.

Abstract: As large language models (LLMs) are increasingly deployed worldwide, ensuring their fair and comprehensive cultural understanding is important. However, LLMs exhibit cultural bias and limited awareness of underrepresented cultures, while the mechanisms underlying their cultural understanding remain underexplored. To fill this gap, we conduct a neuron-level analysis to identify neurons that drive cultural behavior, introducing a gradient-based scoring method with additional filtering for precise refinement. We identify both culture-general neurons contributing to cultural understanding regardless of cultures, and culture-specific neurons tied to an individual culture. These neurons account for less than 1% of all neurons and are concentrated in shallow to middle MLP layers. We validate their role by showing that suppressing them substantially degrades performance on cultural benchmarks (by up to 30%), while performance on general natural language understanding (NLU) benchmarks remains largely unaffected. Moreover, we show that culture-specific neurons support knowledge of not only the target culture, but also related cultures. Finally, we demonstrate that training on NLU benchmarks can diminish models’ cultural understanding when we update modules containing many culture-general neurons. These findings provide insights into the internal mechanisms of LLMs and offer practical guidance for model training and engineering. Our code is available at https://github.com/ynklab/CULNIG

Method: Two-stage framework: (1) persona-guided adversarial instruction generation, and (2) reflection loop to iteratively refine low-quality prompts. Includes a verifier to assess prompt harmfulness without querying target models.

Result: Built AutoRed-Medium and AutoRed-Hard datasets. Achieved higher attack success rates and better generalization than existing baselines when evaluating eight state-of-the-art LLMs.

Conclusion: Demonstrates limitations of seed-based approaches and shows potential of free-form red teaming for comprehensive LLM safety evaluation. Will open source datasets.

Abstract: The safety of Large Language Models (LLMs) is crucial for the development of trustworthy AI applications. Existing red teaming methods often rely on seed instructions, which limits the semantic diversity of the synthesized adversarial prompts. We propose AutoRed, a free-form adversarial prompt generation framework that removes the need for seed instructions. AutoRed operates in two stages: (1) persona-guided adversarial instruction generation, and (2) a reflection loop to iteratively refine low-quality prompts. To improve efficiency, we introduce a verifier to assess prompt harmfulness without querying the target models. Using AutoRed, we build two red teaming datasets – AutoRed-Medium and AutoRed-Hard – and evaluate eight state-of-the-art LLMs. AutoRed achieves higher attack success rates and better generalization than existing baselines. Our results highlight the limitations of seed-based approaches and demonstrate the potential of free-form red teaming for LLM safety evaluation. We will open source our datasets in the near future.

Method: Two-stage voting architecture: Stage 1 uses lightweight BERT for high-confidence explicit cases; Stage 2 escalates ambiguous inputs to either (i) multi-perspective LLM voting for implicit ideation, or (ii) feature-based ML ensemble using psychologically grounded indicators extracted via prompt-engineered LLMs.

Result: Outperforms single-model baselines on two datasets - achieves 98.0% F1 on explicit cases, 99.7% on implicit cases, reduces cross-domain gap below 2%, and significantly lowers LLM computational costs.

Conclusion: The proposed framework effectively balances efficiency and robustness for suicide risk detection, being among the first to operationalize LLM-extracted psychological features as structured vectors, demonstrating strong performance on both explicit and implicit suicidal ideation.

Abstract: Suicide rates have risen worldwide in recent years, underscoring the urgent need for proactive prevention strategies. Social media provides valuable signals, as many at-risk individuals - who often avoid formal help due to stigma - choose instead to share their distress online. Yet detecting implicit suicidal ideation, conveyed indirectly through metaphor, sarcasm, or subtle emotional cues, remains highly challenging. Lightweight models like BERT handle explicit signals but fail on subtle implicit ones, while large language models (LLMs) capture nuance at prohibitive computational cost. To address this gap, we propose a two-stage voting architecture that balances efficiency and robustness. In Stage 1, a lightweight BERT classifier rapidly resolves high-confidence explicit cases. In Stage 2, ambiguous inputs are escalated to either (i) a multi-perspective LLM voting framework to maximize recall on implicit ideation, or (ii) a feature-based ML ensemble guided by psychologically grounded indicators extracted via prompt-engineered LLMs for efficiency and interpretability. To the best of our knowledge, this is among the first works to operationalize LLM-extracted psychological features as structured vectors for suicide risk detection. On two complementary datasets - explicit-dominant Reddit and implicit-only DeepSuiMind - our framework outperforms single-model baselines, achieving 98.0% F1 on explicit cases, 99.7% on implicit ones, and reducing the cross-domain gap below 2%, while significantly lowering LLM cost.

Method: Developed an optimization algorithm to enumerate label sets with varying semantic relevance, then performed ICL experiments with varying numbers of demonstrations for each label set across different model sizes.

Result: Learning occurs regardless of label set quality, but learning efficiency depends on both label set quality and model size. The relative quality of label sets is maintained throughout learning, confirming their orthogonality.

Conclusion: Representation and learning in ICL have independent effects - representation determines baseline performance while learning capacity improves on top of this baseline, revealing a previously underexplored aspect of ICL.

Abstract: In-context learning (ICL) is the ability of a large language model (LLM) to learn a new task from a few demonstrations presented as part of the context. Past studies have attributed a large portion of the success of ICL to the way these in-context demonstrations are represented, particularly to how labels are represented in classification tasks. On the other hand, observations of the learning capacity of ICL (i.e., the extent to which more in-context demonstrations can lead to higher performance) have been mixed, and ICL is often thought to occur only under specific conditions. The interaction between these two aspects in ICL, representation and learning, has not been studied in depth until now. We hypothesize that they are largely independent of one another, such that the representation of demonstrations determines the baseline accuracy of ICL, while learning from additional demonstrations improves only on top of this baseline. We validate this hypothesis by developing an optimization algorithm that can enumerate a spectrum of possible label sets (representations) varying in semantic relevance. We then perform ICL with varying numbers of in-context demonstrations for each of these label sets. We observed that learning happens regardless of the quality of the label set itself, although its efficiency, measured by the slope of improvement over in-context demonstrations, is conditioned on both the label set quality and the parameter count of the underlying language model. Despite the emergence of learning, the relative quality (accuracy) of the choice of a label set (representation) is largely maintained throughout learning, confirming our hypothesis and implying their orthogonality. Our work reveals a previously underexplored aspect of ICL: the independent effects of learning from demonstrations and their representations on ICL performance.

Method: Comprehensive study across different LLM families, sizes, and prompting strategies using linear mixed-effects models and ANOVA tests to analyze conditional acceptability judgments.

Result: LLMs are sensitive to both conditional probability and semantic relevance, but to varying degrees depending on architecture and prompting style. Larger models don’t necessarily align more closely with human judgments.

Conclusion: While LLMs incorporate probabilistic and semantic cues in conditional acceptability judgments, they do so less consistently than humans, suggesting limitations in their reasoning alignment with human cognition.

Abstract: Conditional acceptability refers to how plausible a conditional statement is perceived to be. It plays an important role in communication and reasoning, as it influences how individuals interpret implications, assess arguments, and make decisions based on hypothetical scenarios. When humans evaluate how acceptable a conditional “If A, then B” is, their judgments are influenced by two main factors: the $\textit{conditional probability}$ of $B$ given $A$, and the $\textit{semantic relevance}$ of the antecedent $A$ given the consequent $B$ (i.e., whether $A$ meaningfully supports $B$). While prior work has examined how large language models (LLMs) draw inferences about conditional statements, it remains unclear how these models judge the $\textit{acceptability}$ of such statements. To address this gap, we present a comprehensive study of LLMs’ conditional acceptability judgments across different model families, sizes, and prompting strategies. Using linear mixed-effects models and ANOVA tests, we find that models are sensitive to both conditional probability and semantic relevance-though to varying degrees depending on architecture and prompting style. A comparison with human data reveals that while LLMs incorporate probabilistic and semantic cues, they do so less consistently than humans. Notably, larger models do not necessarily align more closely with human judgments.

Method: Developed Co⁴ machine with single layer, two heads, and 8M parameters using O(N) computational complexity, compared against GPT-2 (124M, 12 layers) and GPT-BERT (30M, 12 layers) with O(N²) complexity using BabyLM Challenge evaluation pipeline.

Result: Co⁴ achieved superior training efficiency (orders-of-magnitude better on 10M tokens), outperformed GPT-2 on 5/7 zero-shot metrics and 6/7 fine-tuning tasks, and beat GPT-BERT on 4/7 metrics in both cases - all in just 2 epochs vs 10 epochs for baselines.

Conclusion: Results suggest the need to rethink current deep learning paradigms and scaling laws, as smaller, more efficient models can achieve better performance than larger, computationally expensive alternatives.

Abstract: We show that a tiny Co$^4$ machine(Adeel,2025) with a single layer, two heads, and 8M parameters, operating at an approximate cost of $O(N)$ (where $N$ is the number of input tokens), outpaces the BabyLM Challenge baselines GPT-2 (124M, 12 layers, $O(N^2))$ and GPT-BERT (30M, 12 layers, $O(N^2))$ in just two epochs, while both are trained for ten. Co$^4$ achieves orders-of-magnitude greater training efficiency on 10M tokens, demonstrating highly sample efficient pretraining. Using the BabyLM challenge evaluation pipeline across complex benchmarks, Co$^4$ exhibits strong zero-shot and fine-tuning performance on SuperGLUE tasks. Specifically, Co$^4$ outperforms GPT-2 on 5 out of 7 zero-shot metrics and 6 out of 7 fine-tuning tasks, and GPT-BERT on 4 out of 7 metrics in both cases. These results suggest the need to rethink prevailing deep learning paradigms and associated scaling laws.

Method: Two-stage training pipeline: 1) Adaptive Cold-Start stage curates data with reasoning traces proportional to problem difficulty; 2) Adaptive Entropy Policy Optimization (AEPO) uses high window-entropy tokens as exploration triggers and hierarchical entropy reward with dynamic KL control to optimize exploration.

Result: ARES achieves superior performance and reasoning efficiency across diverse mathematical, logical, and multimodal benchmarks, while closing the gap to leading commercial systems with significantly lower inference costs.

Conclusion: The proposed adaptive reasoning framework effectively balances exploration effort based on task difficulty, improving both performance and efficiency across various reasoning tasks.

Abstract: Recent advances in multimodal large reasoning models (MLRMs) have substantially improved their ability to solve complex textual and visual tasks. However, these models tend to overthink on simple problems, producing unnecessarily lengthy reasoning traces, while under-exploring on challenging ones, leading to missed solutions. To address this imbalance, we propose ARES, a unified open-source framework for adaptive reasoning that dynamically allocates exploration effort based on task difficulty. Our approach is motivated by two key empirical findings: (i) while single-token entropy is noisy, high window-entropy (HWE) tokens (token-level entropies averaged under a sliding window) can reliably capture reasoning-critical moments; and (ii) reducing HWE usage benefits easy problems, while increasing it is essential for solving hard ones. Building on these insights, ARES introduces a two-stage training pipeline. In the Adaptive Cold-Start stage, we curate multimodal and textual data paired with reasoning traces of length proportional to problem difficulty, equipping the model with initial difficulty awareness. In the second stage, we develop Adaptive Entropy Policy Optimization (AEPO), which uses HWE tokens as exploration triggers to decide when to explore, and a hierarchical entropy reward with dynamic KL control to decide how much to explore. Extensive experiments demonstrate that ARES achieves superior performance and reasoning efficiency across diverse mathematical, logical, and multimodal benchmarks, while closing the gap to leading commercial systems under significantly lower inference costs.

Method: Extended LEWIDI benchmark to four datasets (paraphrase identification, irony detection, sarcasm detection, natural language inference) with categorical and ordinal judgments. Used two evaluation paradigms: soft-label (predicting population-level distributions) and perspectivist (predicting individual annotator interpretations) with new metrics beyond standard cross-entropy.

Result: The task attracted diverse participation and provided insights into strengths and limitations of methods for modeling variation. Results strengthened LEWIDI as a framework and provided new resources, benchmarks, and findings.

Conclusion: LEWIDI’s third edition successfully expanded the benchmark, introduced novel evaluation approaches, and generated valuable insights for developing disagreement-aware AI technologies.

Abstract: Many researchers have reached the conclusion that AI models should be trained to be aware of the possibility of variation and disagreement in human judgments, and evaluated as per their ability to recognize such variation. The LEWIDI series of shared tasks on Learning With Disagreements was established to promote this approach to training and evaluating AI models, by making suitable datasets more accessible and by developing evaluation methods. The third edition of the task builds on this goal by extending the LEWIDI benchmark to four datasets spanning paraphrase identification, irony detection, sarcasm detection, and natural language inference, with labeling schemes that include not only categorical judgments as in previous editions, but ordinal judgments as well. Another novelty is that we adopt two complementary paradigms to evaluate disagreement-aware systems: the soft-label approach, in which models predict population-level distributions of judgments, and the perspectivist approach, in which models predict the interpretations of individual annotators. Crucially, we moved beyond standard metrics such as cross-entropy, and tested new evaluation metrics for the two paradigms. The task attracted diverse participation, and the results provide insights into the strengths and limitations of methods to modeling variation. Together, these contributions strengthen LEWIDI as a framework and provide new resources, benchmarks, and findings to support the development of disagreement-aware technologies.

Method: DeepPrune uses a specialized judge model trained with focal loss and oversampling to predict answer equivalence from partial reasoning traces (0.87 AUROC), combined with an online greedy clustering algorithm that dynamically prunes redundant paths while preserving answer diversity.

Result: Evaluations across AIME 2024, AIME 2025, and GPQA benchmarks show DeepPrune achieves over 80% token reduction compared to conventional consensus sampling while maintaining competitive accuracy within 3 percentage points.

Conclusion: DeepPrune establishes a new standard for efficient parallel reasoning, making high-performance reasoning more computationally efficient by addressing the critical efficiency bottleneck of inter-trace redundancy.

Abstract: Parallel scaling has emerged as a powerful paradigm to enhance reasoning capabilities in large language models (LLMs) by generating multiple Chain-of-Thought (CoT) traces simultaneously. However, this approach introduces significant computational inefficiency due to inter-trace redundancy – our analysis reveals that over 80% of parallel reasoning traces yield identical final answers, representing substantial wasted computation. To address this critical efficiency bottleneck, we propose DeepPrune, a novel framework that enables efficient parallel scaling through dynamic pruning. Our method features a specialized judge model trained with focal loss and oversampling techniques to accurately predict answer equivalence from partial reasoning traces which realizes 0.87 AUROC on equivalence prediction, combined with an online greedy clustering algorithm that dynamically prunes redundant paths while preserving answer diversity. Comprehensive evaluations across three challenging benchmarks (AIME 2024, AIME 2025, and GPQA) and multiple reasoning models demonstrate that DeepPrune achieves remarkable token reduction by over 80% compared to conventional consensus sampling on most cases, while maintaining competitive accuracy within 3 percentage points. Our work establishes a new standard for efficient parallel reasoning, making high-performance reasoning more efficient. Our code and data are here: https://deepprune.github.io/

Method: Add new word embeddings and train with concept examples, then use self-verbalization to get model explanations and plug-in evaluation to validate.

Result: Neologisms successfully control concepts like flattery, incorrect answers, and text length; models can self-verbalize meanings; discovered machine-only synonyms.

Conclusion: Neologism learning is effective for controlling and understanding LLMs, enabling complex concept manipulation and revealing machine-specific word relationships.

Abstract: Humans invent new words when there is a rising demand for a new useful concept (e.g., doomscrolling). We explore and validate a similar idea in our communication with LLMs: introducing new words to better understand and control the models, expanding on the recently introduced neologism learning. This method introduces a new word by adding a new word embedding and training with examples that exhibit the concept with no other changes in model parameters. We show that adding a new word allows for control of concepts such as flattery, incorrect answers, text length, as well as more complex concepts in AxBench. We discover that neologisms can also further our understanding of the model via self-verbalization: models can describe what each new word means to them in natural language, like explaining that a word that represents a concept of incorrect answers means ``a lack of complete, coherent, or meaningful answers…’’ To validate self-verbalizations, we introduce plug-in evaluation: we insert the verbalization into the context of a model and measure whether it controls the target concept. In some self-verbalizations, we find machine-only synonyms: words that seem unrelated to humans but cause similar behavior in machines. Finally, we show how neologism learning can jointly learn multiple concepts in multiple words.

Method: Proposes a framework that integrates Monte Carlo Tree Search (MCTS) with a proxy prompt evaluator to effectively explore the prompt space while reducing evaluation costs.

Result: Experiments show the approach achieves higher classification accuracy and efficiency than baseline methods under constrained computation budgets.

Conclusion: The proposed MCTS-based framework with proxy evaluation provides an effective and efficient solution for prompt optimization in legal NLP tasks.

Abstract: Prompt optimization aims to systematically refine prompts to enhance a language model’s performance on specific tasks. Fairness detection in Terms of Service (ToS) clauses is a challenging legal NLP task that demands carefully crafted prompts to ensure reliable results. However, existing prompt optimization methods are often computationally expensive due to inefficient search strategies and costly prompt candidate scoring. In this paper, we propose a framework that combines Monte Carlo Tree Search (MCTS) with a proxy prompt evaluator to more effectively explore the prompt space while reducing evaluation costs. Experiments demonstrate that our approach achieves higher classification accuracy and efficiency than baseline methods under a constrained computation budget.

Method: Propose RLKV framework using reinforcement learning to optimize relationship between each head’s cache usage and reasoning quality, identifying critical heads for chain-of-thought consistency while compressing others with constant KV cache.

Result: Only small fraction of attention heads is essential for reasoning, enabling 20-50% cache reduction with near lossless performance compared to uncompressed results.

Conclusion: RLKV successfully identifies reasoning-critical heads and achieves efficient KV cache compression while preserving reasoning model performance.

Abstract: Reasoning large language models exhibit complex reasoning behaviors through the extended chain-of-thought generation, creating unprecedented Key-Value (KV) cache overhead during the decoding phase. Existing KV cache compression methods underperform on reasoning models: token-dropping methods break reasoning integrity by discarding critical information, while head-reallocating methods mistakenly compress reasoning-critical heads since they are designed for retrieval tasks, resulting in significant performance degradation as compression rates increase. We hypothesize that KV heads exhibit functional heterogeneity in reasoning models-some heads are critical for chain-of-thought consistency while others are compressible. To validate and exploit this insight, we propose RLKV, a novel reasoning-critical head identification framework, which uses reinforcement learning to directly optimize the relationship between each head’s cache usage and reasoning quality. As RLKV produces rewards from actual generated samples during training, it naturally identifies heads relevant to reasoning behaviors. We then allocate full KV cache to these heads while applying compressed constant KV cache to others for efficient inference. Our experiments reveal that only a small fraction of attention heads is essential for reasoning, enabling our KV compression approach to outperform baseline methods while achieving 20-50% cache reduction with near lossless performance compared to uncompressed results.

Method: CoMAS uses multi-agent systems where agents generate intrinsic rewards from discussion dynamics, employs LLM-as-a-judge mechanism to formulate rewards, and optimizes policies through reinforcement learning in a decentralized manner.

Result: CoMAS consistently outperforms untrained agents and achieves state-of-the-art performance across most evaluation settings, with promising scalability as agent number and diversity increase.

Conclusion: CoMAS establishes a novel and effective paradigm for self-evolution in LLM-based agents through collaborative learning from inter-agent interactions.

Abstract: Self-evolution is a central research topic in enabling large language model (LLM)-based agents to continually improve their capabilities after pretraining. Recent research has witnessed a transition from reinforcement learning (RL)-free to RL-based methods. Current RL-based methods either rely on dense external reward signals or extract intrinsic reward signals from LLMs themselves. However, these approaches diverge from the self-evolution mechanisms observed in human intelligence, where individuals learn and improve through mutual discussion and collaboration. In this work, we introduce Co-Evolving Multi-Agent Systems (CoMAS), a novel framework that enables agents to improve autonomously by learning from inter-agent interactions without external supervision. CoMAS generates intrinsic rewards from rich discussion dynamics, employs an LLM-as-a-judge mechanism to formulate these rewards, and optimizes each agent’s policy through RL, thereby enabling decentralized and scalable co-evolution. Experimental results demonstrate that CoMAS consistently outperforms untrained agents and achieves state-of-the-art performance across most evaluation settings. Ablation studies confirm the necessity of interaction-based reward signals and reveal promising scalability as the number and diversity of agents increase. These findings establish CoMAS as a novel and effective paradigm for self-evolution in LLM-based agents.

Method: ArenaBencher infers core abilities of test cases, generates candidate question-answer pairs preserving original objectives, verifies correctness with LLM judges, and aggregates feedback from multiple models to select candidates that expose shared weaknesses through iterative in-context demonstrations.

Result: The framework produces verified, diverse, and fair updates that uncover new failure modes, increase difficulty while preserving test objective alignment, and improve model separability across math, commonsense reasoning, and safety domains.

Conclusion: ArenaBencher provides a scalable path to continuously evolve benchmarks in step with the rapid progress of foundation models, addressing data leakage issues and enabling more valid model comparisons.

Abstract: Benchmarks are central to measuring the capabilities of large language models and guiding model development, yet widespread data leakage from pretraining corpora undermines their validity. Models can match memorized content rather than demonstrate true generalization, which inflates scores, distorts cross-model comparisons, and misrepresents progress. We introduce ArenaBencher, a model-agnostic framework for automatic benchmark evolution that updates test cases while preserving comparability. Given an existing benchmark and a diverse pool of models to be evaluated, ArenaBencher infers the core ability of each test case, generates candidate question-answer pairs that preserve the original objective, verifies correctness and intent with an LLM as a judge, and aggregates feedback from multiple models to select candidates that expose shared weaknesses. The process runs iteratively with in-context demonstrations that steer generation toward more challenging and diagnostic cases. We apply ArenaBencher to math problem solving, commonsense reasoning, and safety domains and show that it produces verified, diverse, and fair updates that uncover new failure modes, increase difficulty while preserving test objective alignment, and improve model separability. The framework provides a scalable path to continuously evolve benchmarks in step with the rapid progress of foundation models.

Method: Extensive experiments with various LLMs and prompting techniques on four financial tabular QA datasets (TATQA, FinQA, ConvFinQA, Multihiertt), focusing on sensitivity to table complexity and performance with increasing arithmetic reasoning steps.

Result: The study provides insights into LLMs’ capabilities and limitations in handling complex mathematical scenarios for semi-structured tables, with the novel prompting technique matching or outperforming other baselines.

Conclusion: The research offers a nuanced understanding of LLMs’ abilities for mathematical reasoning with semi-structured documents and introduces an effective prompting technique for such tasks.

Abstract: Large Language Models (LLMs), excel in natural language understanding, but their capability for complex mathematical reasoning with an amalgamation of structured tables and unstructured text is uncertain. This study explores LLMs' mathematical reasoning on four financial tabular question-answering datasets: TATQA, FinQA, ConvFinQA, and Multihiertt. Through extensive experiments with various models and prompting techniques, we assess how LLMs adapt to complex tables and mathematical tasks. We focus on sensitivity to table complexity and performance variations with an increasing number of arithmetic reasoning steps. The results provide insights into LLMs’ capabilities and limitations in handling complex mathematical scenarios for semi-structured tables. Ultimately, we introduce a novel prompting technique tailored to semi-structured documents, matching or outperforming other baselines in performance while providing a nuanced understanding of LLMs abilities for such a task.

Method: A two-stage constructivist cognitive modeling process: (1) knowledge assimilation to align new information with parametric memory, and (2) thought accommodation to adapt internal reasoning for consistent outputs.

Result: Achieved 10% improvement over strong baselines on various question-answering benchmarks, with effective integration of external knowledge leading to more accurate responses and improved self-consistency.

Conclusion: ThinkNote successfully enhances LLMs’ external knowledge utilization through constructivist learning principles, demonstrating significant performance gains and more reliable outputs.

Abstract: Large Language Models (LLMs) have demonstrated strong performance across a wide range of NLP tasks. However, they often exhibit suboptimal behaviors and inconsistencies when exposed to unfamiliar external information, underscoring their limitations in effectively leveraging such knowledge. Inspired by constructivist learning theory, we propose ThinkNote, a novel framework that enhances the external knowledge utilization of LLMs through a two-stage constructivist cognitive modeling process. Specifically, ThinkNote performs knowledge assimilation to align new information with the model’s parametric memory, forming a coherent internal representation. It then applies thought accommodation to adapt internal reasoning, thereby promoting more consistent and reliable outputs. Extensive experimental results demonstrate that ThinkNote achieves a 10% improvement over strong baseline methods on various question-answering benchmarks. Further analysis indicates that ThinkNote effectively integrates and utilizes external knowledge to help LLMs generate accurate responses and improves their self-consistency. All data and codes are available at https://github.com/OpenMatch/ThinkNote.

Method: Uses LLMs to annotate user texts against medical diagnostic criteria and summarize historical posts into temporal mood courses, then trains a Gradient Boosting Tree classifier with these medically-informed features, and generates explanations from symptom annotations.

Result: Extensive experiments validate the framework’s effectiveness and interpretability, demonstrating accurate depression detection with explainable predictions.

Conclusion: DORIS shows potential as a supportive clinical tool for early depression detection using social media data with medical knowledge integration and explainable AI.

Abstract: Limited access to mental healthcare resources hinders timely depression diagnosis, leading to detrimental outcomes. Social media platforms present a valuable data source for early detection, yet this task faces two significant challenges: 1) the need for medical knowledge to distinguish clinical depression from transient mood changes, and 2) the dual requirement for high accuracy and model explainability. To address this, we propose DORIS, a framework that leverages Large Language Models (LLMs). To integrate medical knowledge, DORIS utilizes LLMs to annotate user texts against established medical diagnostic criteria and to summarize historical posts into temporal mood courses. These medically-informed features are then used to train an accurate Gradient Boosting Tree (GBT) classifier. Explainability is achieved by generating justifications for predictions based on the LLM-derived symptom annotations and mood course analyses. Extensive experimental results validate the effectiveness as well as interpretability of our method, highlighting its potential as a supportive clinical tool.

Method: ETR uses: 1) affinity-based routing with Grouped Average Pooling to reduce complexity while maintaining orthogonality, 2) bidirectional selection mechanism with cosine similarity, and 3) adaptive capacity strategy that adjusts expert bounds dynamically.

Result: ETR reduces expert capacity lower bound by up to 40%, achieves 5.4%-46.6% improvements in training efficiency, and shows 9.7%-14.5% performance gains across multiple benchmarks.

Conclusion: ETR provides a theoretically-grounded solution that fundamentally improves MoE architectures by enabling adaptive coordination between routing strategies, achieving simultaneous improvements in efficiency and performance.

Abstract: Mixture-of-Experts (MoE) architectures enable efficient scaling of large language models by activating only a subset of parameters per input. However, existing MoE models suffer from two critical limitations: (1) inefficient token-to-expert routing that causes excessive communication overhead, and (2) expert homogenization that leads to redundant computations. Current approaches address these challenges separately, failing to achieve simultaneous improvements in both training efficiency and model performance. We present Expert-Token Resonance (ETR), a theoretically-grounded bidirectional routing mechanism that fundamentally reimagines expert-token interactions in MoE architectures. Our key insight is that optimal routing requires adaptive coordination between token-choice routing (TCR) during early training phases and expert-choice routing (ECR) in later stages. We prove that this dynamic approach maximizes training success rate (the probability of correct token-expert assignments) while reducing the expert capacity lower bound by up to 40%. ETR incorporates three technical innovations: (1) an affinity-based routing architecture using Grouped Average Pooling (GrAP) that reduces computational complexity from O(d^2) to O(d^2/D) while maintaining orthogonality to prevent expert homogenization; (2) a bidirectional selection mechanism that enables both tokens and experts to actively participate in the routing process based on cosine similarity scores; and (3) an adaptive capacity strategy that dynamically adjusts expert bounds based on training progress, eliminating communication bubbles in All-to-All operations. Extensive experiments on Ascend NPU clusters demonstrate that ETR achieves 5.4%-46.6% improvements in end-to-end training efficiency compared to baseline MoE implementations, with 9.7%-14.5% performance gains across GDAD, GPQA, HumanEval, and TeleQnA benchmarks.

Method: Developed a standardized benchmark dataset combining structured and unstructured knowledge across diverse domains, and created PruningRAG framework with multi-granularity pruning strategies to optimize relevant information integration while minimizing misleading context.

Result: PruningRAG consistently improves performance across various existing RAG variants, demonstrating robustness and broad applicability. The framework effectively handles multi-source knowledge integration.

Conclusion: The standardized dataset and PruningRAG framework advance RAG research by addressing multi-source knowledge integration challenges. The resources are publicly available to support future research in the RAG community.

Abstract: Retrieval-augmented generation (RAG) is increasingly recognized as an effective approach to mitigating the hallucination of large language models (LLMs) through the integration of external knowledge. While numerous efforts, most studies focus on a single type of external knowledge source. However, in real-world applications, most situations involve diverse knowledge from various sources, yet this area has been less explored. The main dilemma is the lack of a suitable dataset containing multiple knowledge sources and pre-exploration of the associated issues. To address these challenges, we standardize a benchmark dataset that combines structured and unstructured knowledge across diverse and complementary domains. Based on this dataset, we further develop a plug-and-play RAG framework, \textbf{PruningRAG}, whose main characteristic is the use of multi-granularity pruning strategies to optimize the integration of relevant information while minimizing misleading context. It consistently improves performance across various existing RAG variants, demonstrating its robustness and broad applicability. Building upon the standardized dataset and PruningRAG, we also report a series of experimental results, as well as insightful findings. Our dataset and code are publicly available\footnote{https://github.com/USTCAGI/PruningRAG}, with the aim of advancing future research in the RAG community.

Method: Uses QK dot products to measure KV cache criticality at token-level, implements per-head soft voting mechanism to select critical tokens, and employs Selection Cache with Paged Dot Product Kernel for efficient implementation.

Result: Achieves up to 23.84× speedup in attention computation and 2.28× acceleration in end-to-end latency while maintaining superior performance compared to state-of-the-art methods.

Conclusion: TokenSelect provides an efficient and accurate solution for long-context inference in LLMs, overcoming key limitations of current approaches through dynamic KV cache selection.

Abstract: Rapid advances in Large Language Models (LLMs) have spurred demand for processing extended context sequences in contemporary applications. However, this progress faces two challenges: performance degradation due to sequence lengths out-of-distribution, and excessively long inference times caused by the quadratic computational complexity of attention. These issues limit LLMs in long-context scenarios. In this paper, we propose Dynamic Token-Level KV Cache Selection (TokenSelect), a training-free method for efficient and accurate long-context inference. TokenSelect builds upon the observation of non-contiguous attention sparsity, using QK dot products to measure per-head KV Cache criticality at token-level. By per-head soft voting mechanism, TokenSelect selectively involves a few critical KV cache tokens in attention calculation without sacrificing accuracy. To further accelerate TokenSelect, we design the Selection Cache based on observations of consecutive Query similarity and implemented the efficient Paged Dot Product Kernel, significantly reducing the selection overhead. A comprehensive evaluation of TokenSelect demonstrates up to $23.84\times$ speedup in attention computation and up to $2.28\times$ acceleration in end-to-end latency, while providing superior performance compared to state-of-the-art long-context inference methods.

Method: Constructs a feature tree from raw data and refines it iteratively to extract hierarchical code features from high-level abstractions. By adjusting depth and breadth of sampled subtrees, it provides precise control over generated code complexity.

Result: Fine-tuned base models to create EpiCoder series, achieving state-of-the-art performance on multiple benchmarks at both function and file levels. Shows significant potential for repository-level code data synthesis.

Conclusion: The feature tree-based framework enables generation of more complex and diverse code data, with demonstrated effectiveness across function, file, and repository levels, representing a significant advancement in code synthesis capabilities.

Abstract: Existing methods for code generation use code snippets as seed data, restricting the complexity and diversity of the synthesized data. In this paper, we introduce a novel feature tree-based synthesis framework, which revolves around hierarchical code features derived from high-level abstractions of code. The feature tree is constructed from raw data and refined iteratively to increase the quantity and diversity of the extracted features, which captures and recognizes more complex patterns and relationships within the code. By adjusting the depth and breadth of the sampled subtrees, our framework provides precise control over the complexity of the generated code, enabling functionalities that range from function-level operations to multi-file scenarios. We fine-tuned widely-used base models to obtain EpiCoder series, achieving state-of-the-art performance on multiple benchmarks at both the function and file levels. In particular, empirical evidence indicates that our approach shows significant potential in the synthesizing of repository-level code data. Our code and data are publicly available at https://github.com/microsoft/EpiCoder.

Method: The Med-R^2 framework follows the Evidence-Based Medicine (EBM) process, efficiently integrating retrieval mechanisms with evidence selection and reasoning processes to enhance LLM problem-solving in healthcare.

Result: Med-R^2 achieves 13.27% improvement over vanilla RAG methods and 4.55% enhancement compared to fine-tuning strategies. LLaMA3.1-70B + Med-R^2 surpasses GPT-4o, Claude3.5-Sonnet and DeepSeek-V3 by 1.05%, 6.14% and 1.91% respectively.

Conclusion: Med-R^2 effectively enhances LLM capabilities in the medical domain by integrating retrieval with evidence-based reasoning, providing a trustworthy LLM physician framework without requiring additional training costs.

Abstract: Large Language Models (LLMs) have exhibited remarkable capabilities in clinical scenarios. Despite their potential, existing works face challenges when applying LLMs to medical settings. Strategies relying on training with medical datasets are highly cost-intensive and may suffer from outdated training data. Leveraging external knowledge bases is a suitable alternative, yet it faces obstacles such as limited retrieval precision and poor effectiveness in answer extraction. These issues collectively prevent LLMs from demonstrating the expected level of proficiency in mastering medical expertise. To address these challenges, we introduce Med-R^2, a novel LLM physician framework that adheres to the Evidence-Based Medicine (EBM) process, efficiently integrating retrieval mechanisms as well as the selection and reasoning processes of evidence, thereby enhancing the problem-solving capabilities of LLMs in healthcare scenarios and fostering a trustworthy LLM physician. Our comprehensive experiments indicate that Med-R^2 achieves a 13.27% improvement over vanilla RAG methods and even a 4.55% enhancement compared to fine-tuning strategies, without incurring additional training costs. Furthermore, we find that our LLaMA3.1-70B + Med-R$^2$ surpasses frontier models, including GPT-4o, Claude3.5-Sonnet and DeepSeek-V3 by 1.05%, 6.14% and 1.91%. Med-R$^2$ effectively enhances the capabilities of LLMs in the medical domain.

Method: Systematic analysis of fine-tuning with LLM-generated data vs ground truth data, examining token perplexity in data sequences across domains, and implementing high perplexity token masking in ground truth data.

Result: LLM-generated data fine-tuning improves target task performance while better preserving non-target task performance. Masking high perplexity tokens in ground truth data achieves similar robustness benefits.

Conclusion: Reducing high perplexity tokens is key to mitigating catastrophic forgetting in LLM fine-tuning, providing insights for developing more robust fine-tuning strategies across different model scales.

Abstract: Maintaining consistent model performance across domains is a fundamental challenge in machine learning. While recent work has explored using LLM-generated data for fine-tuning, its impact on cross-domain generalization remains poorly understood. This paper presents a systematic analysis revealing that fine-tuning with LLM-generated data not only improves target task performance but also reduces non-target task degradation compared to fine-tuning with ground truth data. Through analyzing the data sequence in tasks of various domains, we demonstrate that this enhancement of non-target task robustness stems from the reduction of high perplexity tokens found in LLM-generated sequences. Following our findings, we showed that masking high perplexity tokens in ground truth training data achieves similar non-target task performance preservation, comparable to using LLM-generated data. Extensive experiments across different model families and scales, including Gemma 2 IT 2B, Llama 3 8B Instruct, and three additional models, agree with our findings. To the best of our knowledge, this is the first work to provide an empirical explanation based on token perplexity reduction to mitigate catastrophic forgetting in LLMs after fine-tuning, offering valuable insights for developing more robust fine-tuning strategies.

Method: Proposes Cross-Lingual Semantic Discrimination (CLSD) task that uses parallel sentences and LLMs to generate adversarial distractors, measuring embedding models’ ranking performance on German-French news domain datasets.

Result: Models fine-tuned for retrieval benefit from English pivoting, while bitext mining models excel in direct cross-lingual settings. Embedding models show different sensitivity to linguistic perturbations.

Conclusion: CLSD provides an effective lightweight evaluation framework for cross-lingual semantic search models, revealing model-specific performance patterns and sensitivity to linguistic variations.

Abstract: The evaluation of cross-lingual semantic search models is often limited to existing datasets from tasks such as information retrieval and semantic textual similarity. We introduce Cross-Lingual Semantic Discrimination (CLSD), a lightweight evaluation task that requires only parallel sentences and a Large Language Model (LLM) to generate adversarial distractors. CLSD measures an embedding model’s ability to rank the true parallel sentence above semantically misleading but lexically similar alternatives. As a case study, we construct CLSD datasets for German–French in the news domain. Our experiments show that models fine-tuned for retrieval tasks benefit from pivoting through English, whereas bitext mining models perform best in direct cross-lingual settings. A fine-grained similarity analysis further reveals that embedding models differ in their sensitivity to linguistic perturbations. We release our code and datasets under AGPL-3.0: https://github.com/impresso/cross_lingual_semantic_discrimination

Method: CompSelect uses a MinMax optimization framework with three components: (1) clue extractor that extracts potential clues using answer-containing sentences, (2) reranker trained with LLM feedback to prioritize effective clues, and (3) truncator that identifies the minimum sufficient clues for answering questions.

Result: Experiments on three QA datasets show CompSelect improves QA performance by ~11% and reduces Total Latency by ~17% and Online Latency by ~67% compared to baselines on LLaMA3 and Qwen3. It also demonstrates robustness to unreliable retrieval and generalization across scenarios.

Conclusion: CompSelect offers a scalable and cost-efficient solution for web-scale RAG applications by optimizing retrieval specifically for LLM reasoning needs through compact clue selection and organization.

Abstract: Current RAG retrievers are designed primarily for human readers, emphasizing complete, readable, and coherent paragraphs. However, LLMs benefit more from precise, compact, and well-structured input, which enhances reasoning quality and efficiency. Existing methods often rely on reranking or summarization to identify key sentences, but may suffer from semantic breaks and unfaithfulness. Thus, efficiently extracting and organizing answer-relevant clues from large-scale documents while reducing LLM reasoning costs remains a challenge for RAG. Inspired by Occam’s razor, we frame LLM-centric retrieval as a MinMax optimization: maximizing the extraction of potential clues and reranking them for well-organization, while minimizing reasoning costs by truncating to the smallest sufficient clues set. In this paper, we propose CompSelect, a Compact clue Selection mechanism for LLM-centric RAG, consisting of a clue extractor, a reranker, and a truncator. (1) The clue extractor first uses answer-containing sentences as fine-tuning targets, aiming to extract sufficient potential clues; (2) The reranker is trained to prioritize effective clues based on real LLM feedback; (3) The truncator uses the truncated text containing the minimum sufficient clues for answering the question as fine-tuning targets, thereby enabling efficient RAG reasoning. Experiments on three QA datasets show that CompSelect improves QA performance by approximately 11% and reduces Total Latency and Online Latency by approximately 17% and 67% compared to various baseline methods on both LLaMA3 and Qwen3. Further analysis confirms its robustness to unreliable retrieval and generalization across different scenarios, offering a scalable and cost-efficient solution for web-scale RAG applications.

Method: MoM employs multiple independent memory states with a router network that directs input tokens to specific memory states, serving as a general framework compatible with various memory update mechanisms in linear models.

Result: MoM outperforms existing linear sequence models on downstream language tasks, especially recall-intensive tasks, and achieves performance comparable to Transformer models while maintaining linear complexity during training and constant complexity during inference.

Conclusion: MoM provides an effective solution to enhance memory capacity in linear sequence models without sacrificing computational efficiency, making it particularly suitable for recall-intensive tasks.

Abstract: Linear sequence modeling methods, such as linear attention, state space modeling, and linear RNNs, offer significant efficiency improvements by reducing the complexity of training and inference. However, these methods typically compress the entire input sequence into a single fixed-size memory state, which leads to suboptimal performance on recall-intensive tasks. To address this limitation, we introduce a novel architecture called Mixture-of-Memories (MoM). MoM utilizes multiple independent memory states, with a router network directing input tokens to specific memory states. This approach greatly enhances the overall memory capacity while minimizing memory interference. MoM serves as a general framework that can be seamlessly combined with diverse memory update mechanisms across linear models. As a result, MoM performs exceptionally well on recall-intensive tasks, surpassing existing linear sequence modeling techniques. Despite incorporating multiple memory states, the computation of each memory state remains linear in complexity, allowing MoM to retain the linear-complexity advantage during training, while constant-complexity during inference. Our experimental results show that MoM outperforms current linear sequence models on downstream language tasks, particularly recall-intensive tasks, and even achieves performance comparable to Transformer models. The code is released at https://github.com/OpenSparseLLMs/MoM and is also released as a part of https://github.com/OpenSparseLLMs/Linear-MoE.

Method: Created StripCipher benchmark with human-annotated dataset and three subtasks: visual narrative comprehension, contextual frame prediction, and temporal narrative reordering. Evaluated 16 state-of-the-art LMMs including GPT-4o and Qwen2.5VL.

Result: LMMs show significant performance gaps compared to humans, especially in reordering tasks where GPT-4o achieved only 23.93% accuracy (56.07% lower than human performance). Input format of images was identified as a key factor affecting performance.

Conclusion: Fundamental challenges remain in developing LMMs for sequential understanding, with current models struggling particularly with temporal reasoning and reordering tasks that require understanding narrative flow.

Abstract: Large Multimodal Models (LMMs) have achieved remarkable success across various visual-language tasks. However, existing benchmarks predominantly focus on single-image understanding, leaving the analysis of image sequences largely unexplored. To address this limitation, we introduce StripCipher, a comprehensive benchmark designed to evaluate capabilities of LMMs to comprehend and reason over sequential images. StripCipher comprises a human-annotated dataset and three challenging subtasks: visual narrative comprehension, contextual frame prediction, and temporal narrative reordering. Our evaluation of 16 state-of-the-art LMMs, including GPT-4o and Qwen2.5VL, reveals a significant performance gap compared to human capabilities, particularly in tasks that require reordering shuffled sequential images. For instance, GPT-4o achieves only 23.93% accuracy in the reordering subtask, which is 56.07% lower than human performance. Further quantitative analysis discuss several factors, such as input format of images, affecting the performance of LLMs in sequential understanding, underscoring the fundamental challenges that remain in the development of LMMs.

Method: Proposed PerMU, a probability perturbation-based unlearning paradigm that simulates adversarial unlearning samples to eliminate fact-related tokens from the logit distribution by collectively reducing probabilities of all answer-associated tokens.

Result: PerMU delivers up to 50.40% improvement in unlearning vanilla target data while maintaining 40.73% boost in forgetting implicit knowledge across diverse datasets (TOFU, Harry Potter, ZsRE, WMDP, MUSE) and models (1.3B to 13B scale).

Conclusion: The study demonstrates the importance of generalized implicit knowledge forgetting and shows that PerMU effectively addresses this challenge through probability perturbation, significantly improving unlearning performance for both explicit and implicit knowledge.

Abstract: In this paper, we investigate knowledge forgetting in large language models with a focus on its generalisation, ensuring that models forget not only specific training samples but also related implicit knowledge. To this end, we begin by identifying a broader unlearning scope that includes both target data and logically associated samples, including rephrased, subject-replaced, relation-reversed, and one-hop reasoned data. We then conduct a rigorous evaluation of 15 state-of-the-art methods across three datasets, revealing that unlearned models still recall paraphrased answers and retain target facts in their intermediate layers. This motivates us to take a preliminary step toward more generalised implicit knowledge forgetting by proposing PerMU, a novel probability perturbation-based unlearning paradigm. PerMU simulates adversarial unlearning samples to eliminate fact-related tokens from the logit distribution, collectively reducing the probabilities of all answer-associated tokens. Experiments are conducted on a diverse range of datasets, including TOFU, Harry Potter, ZsRE, WMDP, and MUSE, using models ranging from 1.3B to 13B in scale. The results demonstrate that PerMU delivers up to a 50.40% improvement in unlearning vanilla target data while maintaining a 40.73% boost in forgetting implicit knowledge. Our code can be found in https://github.com/MaybeLizzy/PERMU.

Method: Proposed a novel prompt-based evaluation scheme validated through a human benchmark dataset, integrated LLMs into existing ArgSum systems, developed two new LLM-based ArgSum systems, and benchmarked against prior methods.

Result: LLMs substantially improve both generation and evaluation of argument summaries, achieving state-of-the-art results. Qwen-3-32B performed best among four tested LLMs, even surpassing GPT-4o despite having fewer parameters.

Conclusion: The integration of LLMs advances the field of Argument Summarization by providing superior performance in both generation and evaluation tasks, with Qwen-3-32B emerging as the most effective model tested.

Abstract: Large Language Models (LLMs) have revolutionized various Natural Language Generation (NLG) tasks, including Argument Summarization (ArgSum), a key subfield of Argument Mining. This paper investigates the integration of state-of-the-art LLMs into ArgSum systems and their evaluation. In particular, we propose a novel prompt-based evaluation scheme, and validate it through a novel human benchmark dataset. Our work makes three main contributions: (i) the integration of LLMs into existing ArgSum systems, (ii) the development of two new LLM-based ArgSum systems, benchmarked against prior methods, and (iii) the introduction of an advanced LLM-based evaluation scheme. We demonstrate that the use of LLMs substantially improves both the generation and evaluation of argument summaries, achieving state-of-the-art results and advancing the field of ArgSum. We also show that among the four LLMs integrated in (i) and (ii), Qwen-3-32B, despite having the fewest parameters, performs best, even surpassing GPT-4o.

Method: Adapted LLaMA-3.1-8B model and trained on 45.3B tokens across four languages using translated instruction datasets, automatically constructed and manually verified Kazakhstan-specific instruction dataset, and Kazakh-specific safety data.

Result: Demonstrates strong knowledge and reasoning abilities in Kazakh, significantly outperforming existing open Kazakh and multilingual models of similar scale while achieving competitive performance in English.

Conclusion: Released as an open-weight model to support research and real-world applications for Kazakh speakers, with detailed documentation of training, alignment, and evaluation processes.

Abstract: Llama-3.1-Sherkala-8B-Chat, or Sherkala-Chat (8B) for short, is a state-of-the-art instruction-tuned open generative large language model (LLM) designed for Kazakh. Sherkala-Chat (8B) aims to enhance the inclusivity of LLM advancements for Kazakh speakers. Adapted from the LLaMA-3.1-8B model, Sherkala-Chat (8B) is trained on 45.3B tokens across Kazakh, English, Russian, and Turkish. With 8 billion parameters, it demonstrates strong knowledge and reasoning abilities in Kazakh, significantly outper-forming existing open Kazakh and multilingual models of similar scale while achieving competitive performance in English. To ensure effective and responsible alignment, we leverage translated instruction datasets, a Kazakhstan-specific instruction dataset that is automatically constructed and manually verified, and Kazakh-specific safety data. We release Sherkala-Chat (8B) as an open-weight model, along with a detailed description of its training, alignment, and evaluation, to support research and real-world applications for Kazakh speakers.

Method: EKBM framework integrates fast and slow reasoning systems: fast-thinking model generates confidence-labeled responses, and uncertain predictions trigger slow refinement model. Uses hybrid training pipeline to enhance self-awareness without degrading task performance.

Result: Evaluations on dialogue state tracking show EKBM achieves superior model reliability over uncertainty-based baselines. Refinement substantially boosts accuracy while maintaining low computational overhead.

Conclusion: EKBM establishes a scalable paradigm for deploying reliable LLMs in error-sensitive applications, effectively balancing accuracy and practical utility.

Abstract: Large language models (LLMs) are prone to hallucination stemming from misaligned self-awareness, particularly when processing queries exceeding their knowledge boundaries. While existing mitigation strategies employ uncertainty estimation or query rejection mechanisms, they suffer from computational efficiency and sacrificed helpfulness. To address these issues, we propose the Explicit Knowledge Boundary Modeling (EKBM) framework, integrating fast and slow reasoning systems to harmonize reliability and usability. The framework first employs a fast-thinking model to generate confidence-labeled responses, enabling immediate utilization of high-confidence outputs, whereas uncertain predictions trigger a slow refinement model for accuracy improvement. To align model behavior with our proposed object, we propose a hybrid training pipeline, enhancing self-awareness without degrading task performance. Evaluations on dialogue state tracking tasks demonstrate that EKBM achieves superior model reliability over uncertainty-based baselines. Further analysis reveals that refinement substantially boosts accuracy while maintaining low computational overhead. The framework establishes a scalable paradigm for deploying reliable LLMs in error-sensitive applications, effectively balancing accuracy and practical utility.

Method: Proposes Target-DPO framework that explicitly locates error regions and aligns corresponding tokens via tailored DPO algorithm. Uses CodeFlow dataset with iteratively refined samples capturing error corrections.

Result: Code LLMs equipped with Target-DPO achieve significant performance gains in code generation and improve on challenging tasks like BigCodeBench, yielding fewer errors.

Conclusion: Target-DPO effectively enhances Code LLMs by providing granular error-correction learning through explicit error region localization and token-level alignment.

Abstract: Preference learning extends the performance of Code LLMs beyond traditional supervised fine-tuning by leveraging relative quality comparisons. In existing approaches, a set of n candidate solutions is evaluated based on test case success rates, with the candidate demonstrating a higher pass rate being labeled as positive and its counterpart with a lower pass rate as negative. However, because this approach aligns entire failing code blocks rather than pinpointing specific errors, it lacks the granularity necessary to capture meaningful error-correction relationships. As a result, the model is unable to learn more informative error-correction patterns. To address these issues, we propose Target-DPO, a new preference alignment framework that mimics human iterative debugging to refine Code LLMs. Target-DPO explicitly locates error regions and aligns the corresponding tokens via a tailored DPO algorithm. To facilitate it, we introduce the CodeFlow dataset, where samples are iteratively refined until passing tests, with modifications capturing error corrections. Extensive experiments show that a diverse suite of Code LLMs equipped with Target-DPO achieves significant performance gains in code generation and improves on challenging tasks like BigCodeBench. In-depth analysis reveals that Target-DPO yields fewer errors. Code, model and datasets are in: https://github.com/JieWu02/Target-DPO.

Method: Proposed spatiotemporal-aware order planning module with external tools for precise reasoning, cost-effective dialogue system with multi-type repliers and cost-aware LLM configurations, and continual fine-tuning scheme using real-world interactions and simulated dialogues.

Result: Achieved 93% accuracy in order planning and 92% in response generation during real-world deployment. Offline experiments showed improvements of up to 70.23% in order planning and 321.27% in response generation compared to state-of-the-art frameworks, with latency reductions of 0.72× to 5.47×.

Conclusion: DiMA establishes itself as an effective, efficient, and intelligent mobile assistant for ride-hailing services, demonstrating superior performance in real-world deployment and significant improvements over existing frameworks.

Abstract: On-demand ride-hailing services like DiDi, Uber, and Lyft have transformed urban transportation, offering unmatched convenience and flexibility. In this paper, we introduce DiMA, an LLM-powered ride-hailing assistant deployed in DiDi Chuxing. Its goal is to provide seamless ride-hailing services and beyond through a natural and efficient conversational interface under dynamic and complex spatiotemporal urban contexts. To achieve this, we propose a spatiotemporal-aware order planning module that leverages external tools for precise spatiotemporal reasoning and progressive order planning. Additionally, we develop a cost-effective dialogue system that integrates multi-type dialog repliers with cost-aware LLM configurations to handle diverse conversation goals and trade-off response quality and latency. Furthermore, we introduce a continual fine-tuning scheme that utilizes real-world interactions and simulated dialogues to align the assistant’s behavior with human preferred decision-making processes. Since its deployment in the DiDi application, DiMA has demonstrated exceptional performance, achieving 93% accuracy in order planning and 92% in response generation during real-world interactions. Offline experiments further validate DiMA capabilities, showing improvements of up to 70.23% in order planning and 321.27% in response generation compared to three state-of-the-art agent frameworks, while reducing latency by $0.72\times$ to $5.47\times$. These results establish DiMA as an effective, efficient, and intelligent mobile assistant for ride-hailing services. Our project is released at https://github.com/usail-hkust/DiMA and we also release the MCP service (https://mcp.didichuxing.com/api) to foster the ride-hailing research community.

Method: Proposed two innovations: context-efficient architecture with hierarchical section search to reduce token consumption, and scenario-based dialogue generation methodology to distill tool-using capabilities from larger models.

Result: Experimental evaluations show the fine-tuned 8B parameter model substantially outperforms both untuned models and baseline approaches in terms of DB faithfulness and preference.

Conclusion: The work offers valuable insights for researchers developing domain-specific language agents under practical constraints.

Abstract: Language agents powered by large language models (LLMs) face significant deployment challenges in resource-constrained environments, particularly for specialized domains and less-common languages. This paper presents Tox-chat, a Korean chemical toxicity information agent devised within these limitations. We propose two key innovations: a context-efficient architecture that reduces token consumption through hierarchical section search, and a scenario-based dialogue generation methodology that effectively distills tool-using capabilities from larger models. Experimental evaluations demonstrate that our fine-tuned 8B parameter model substantially outperforms both untuned models and baseline approaches, in terms of DB faithfulness and preference. Our work offers valuable insights for researchers developing domain-specific language agents under practical constraints.

Method: Proposed UniEDU, a unified language and vision assistant designed as a single model for multiple educational tasks, optimized for computational efficiency and real-world deployment.

Result: Achieved approximately 300% increase in efficiency while maintaining competitive performance with minimal degradation compared to fully fine-tuned models. The model demonstrates strong generalization across multiple educational tasks.

Conclusion: UniEDU represents a significant advancement toward versatile AI systems for education, offering a unified solution that adapts to diverse learning environments with high efficiency and practical deployment capabilities.

Abstract: Education materials for K-12 students often consist of multiple modalities, such as text and images, posing challenges for models to fully understand nuanced information in these materials. In this paper, we propose a unified language and vision assistant UniEDU designed for various educational applications, including knowledge recommendation, knowledge tracing, time cost prediction, and user answer prediction, all within a single model. Unlike conventional task-specific models, UniEDU offers a unified solution that excels across multiple educational tasks while maintaining strong generalization capabilities. Its adaptability makes it well-suited for real-world deployment in diverse learning environments. Furthermore, UniEDU is optimized for industry-scale deployment by significantly reducing computational overhead-achieving approximately a 300% increase in efficiency-while maintaining competitive performance with minimal degradation compared to fully fine-tuned models. This work represents a significant step toward creating versatile AI systems tailored to the evolving demands of education.

Method: Incorporates a plug-in router and adapter to enable dynamic layer skipping in Transformer models, allowing adaptive computation based on token characteristics.

Result: Achieves skipping of 8 out of 32 layers in Llama-3-8B while maintaining full benchmark performance, showing computational demands vary significantly by token type (repetitive tokens need fewer layers, computational/high-uncertainty tokens need more).

Conclusion: FlexiDepth demonstrates effective adaptive computation but doesn’t achieve wall-clock speedup due to varied skipping patterns and I/O overhead. The method and dataset are open-sourced to advance practical speedup research.

Abstract: Various layer-skipping methods have been proposed to accelerate token generation in large language models (LLMs). However, limited attention has been paid to a fundamental question: How do computational demands vary across the generation of different tokens? In this work, we introduce FlexiDepth, a method that dynamically adjusts the number of Transformer layers used in text generation. By incorporating a plug-in router and adapter, FlexiDepth enables adaptive computation in LLMs without modifying their original parameters. Applied to Llama-3-8B, it skips 8 out of 32 layers while maintaining full benchmark performance. Our experiments reveal that computational demands in LLMs significantly vary based on token type. Specifically, generating repetitive tokens or fixed phrases requires fewer layers, whereas producing tokens involving computation or high uncertainty requires more layers. Despite the computational savings, FlexiDepth does not yet achieve wall-clock speedup due to varied skipping patterns and I/O overhead. To inspire future work and advance research on practical speedup, we open-sourced FlexiDepth and a dataset documenting its layer allocation patterns.

Method: Created CQBench using multi-character conversation stories based on World Value Survey and GlobalOpinions data, with automatic validation pipeline achieving 94.5% human agreement. Designed three tasks: attitude detection, value selection, and value extraction.

Result: Frontier models like o1 reach human-level performance in value selection (0.809 F1) but struggle with nuanced attitude detection (0.622 F1). Fine-tuning smaller models on 500 examples improves performance by over 10%, sometimes outperforming larger models.

Conclusion: CQBench reveals current challenges in LLMs’ cultural intelligence and provides practical pathways for enhancing cross-cultural reasoning abilities.

Abstract: Cultural Intelligence (CQ) refers to the ability to understand unfamiliar cultural contexts, a crucial skill for large language models (LLMs) to effectively engage with globally diverse users. Existing studies often focus on explicitly stated cultural norms, but fail to capture the subtle, implicit values that are common in daily conversation. To address this gap, we introduce CQBench, a benchmark specifically designed to assess LLMs’ capability to infer implicit cultural values from natural conversational contexts. CQBench consists of multi character conversation based stories using values from the World Value Survey and the GlobalOpinions, with topics including ethical, religious, social, etc. Our automatic dataset construction pipeline integrates rigorous validation procedures (incorporation, consistency, and implicitness checks), achieving a 94.5% human model agreement in the final validation. To leverage CQBench data, we design three tasks of increasing complexity: attitude detection, value selection, and value extraction. These tasks evaluate whether models can detect attitude and recognize values embedded within natural dialogues rather than relying on explicit cultural knowledge. We find that while frontier models like o1 reach human level performance in value selection (0.809 F1), they still fall short in nuanced attitude detection (0.622 F1). Notably, finetuning a smaller LLaMA-3.2-3B on only 500 culturally rich examples improves performance by over 10%, even outperforming o3-mini in some cases. Using CQ-Bench, we provide insights into the current challenges in LLMs’ CQ research and suggest practical pathways for enhancing LLMs’ cross-cultural reasoning abilities.

Method: Leverages topological divergence metric to quantify structural properties of graphs from attention matrices, examining divergence between prompt and response subgraphs.

Result: Achieves state-of-the-art or competitive results on question answering and summarization benchmarks with minimal annotated data and computational resources.

Conclusion: Analyzing topological structure of attention matrices serves as an efficient and robust indicator of factual reliability in LLMs.

Abstract: Hallucination, i.e., generating factually incorrect content, remains a critical challenge for large language models (LLMs). We introduce TOHA, a TOpology-based HAllucination detector in the RAG setting, which leverages a topological divergence metric to quantify the structural properties of graphs induced by attention matrices. Examining the topological divergence between prompt and response subgraphs reveals consistent patterns: higher divergence values in specific attention heads correlate with hallucinated outputs, independent of the dataset. Extensive experiments - including evaluation on question answering and summarization tasks - show that our approach achieves state-of-the-art or competitive results on several benchmarks while requiring minimal annotated data and computational resources. Our findings suggest that analyzing the topological structure of attention matrices can serve as an efficient and robust indicator of factual reliability in LLMs.

Method: A hybrid approach that combines efficient embedding-based clustering with LLM-based prompting, striking a balance between scalability and semantic precision. This avoids the computational burden of LLM-heavy methods like iterative tree construction.

Result: The method achieves superior quality-speed trade-offs compared to LLM-heavy approaches. The hierarchies capture different dimensions of research contributions and reflect the interdisciplinary nature of modern science. Evaluation shows improved interpretability and effective navigation by LLM-based agents to locate target papers.

Conclusion: SCIENCE HIERARCHOGRAPHY offers an alternative pathway for exploring scientific literature beyond traditional search methods, providing insights into which fields are well-explored versus under-explored.

Abstract: Scientific knowledge is growing rapidly, making it difficult to track progress and high-level conceptual links across broad disciplines. While tools like citation networks and search engines help retrieve related papers, they lack the abstraction needed to capture the needed to represent the density and structure of activity across subfields. We motivate SCIENCE HIERARCHOGRAPHY, the goal of organizing scientific literature into a high-quality hierarchical structure that spans multiple levels of abstraction – from broad domains to specific studies. Such a representation can provide insights into which fields are well-explored and which are under-explored. To achieve this goal, we develop a hybrid approach that combines efficient embedding-based clustering with LLM-based prompting, striking a balance between scalability and semantic precision. Compared to LLM-heavy methods like iterative tree construction, our approach achieves superior quality-speed trade-offs. Our hierarchies capture different dimensions of research contributions, reflecting the interdisciplinary and multifaceted nature of modern science. We evaluate its utility by measuring how effectively an LLM-based agent can navigate the hierarchy to locate target papers. Results show that our method improves interpretability and offers an alternative pathway for exploring scientific literature beyond traditional search methods. Code, data and demo are available: https://github.com/JHU-CLSP/science-hierarchography

Method: Fine-tuned gte-Qwen2-1.5B-instruct model using triplet loss on T-Embed dataset, which contains diverse telecom concepts, standards, and operational scenarios (75% of dataset released publicly).

Result: T-VEC outperforms MPNet, BGE, Jina and E5 on a custom benchmark of 1500 query-passage pairs from IETF RFCs and vendor manuals, showing superior domain grounding and semantic precision. Embedding visualizations confirm tight clustering of telecom concepts.

Conclusion: T-VEC enables semantically faithful NLP applications in telecom domain and is released publicly along with its tokenizer to support continued research in domain-specific representation learning.

Abstract: The specialized vocabulary and nuanced concepts of the telecommunications industry pose persistent challenges for standard Natural Language Processing (NLP) models. Generic embedding models often struggle to represent telecom-specific semantics, limiting their utility in retrieval and downstream tasks. We present T-VEC (Telecom Vectorization Model), a domain-adapted embedding model fine-tuned from the gte-Qwen2-1.5B-instruct backbone using a triplet loss objective. Fine-tuning was performed on T-Embed, a high-quality, large-scale dataset covering diverse telecom concepts, standards, and operational scenarios. Although T-Embed contains some proprietary material and cannot be fully released, we open source 75% of the dataset to support continued research in domain-specific representation learning. On a custom benchmark comprising 1500 query-passage pairs from IETF RFCs and vendor manuals, T-VEC surpasses MPNet, BGE, Jina and E5, demonstrating superior domain grounding and semantic precision in telecom-specific retrieval. Embedding visualizations further showcase tight clustering of telecom-relevant concepts. We release T-VEC and its tokenizer to support semantically faithful NLP applications within the telecom domain.

Method: Conducted large-scale empirical evaluation of 6 diverse hallucination detection metrics across 4 datasets, 37 language models from 5 families, and 5 decoding methods.

Result: Metrics often fail to align with human judgments, take myopic views of hallucinations, and show inconsistent gains with parameter scaling. LLM-based evaluation (GPT-4) performs best overall, and mode-seeking decoding reduces hallucinations in knowledge-grounded settings.

Conclusion: Current hallucination evaluation has concerning gaps, highlighting need for more robust metrics to understand/quantify hallucinations and better mitigation strategies.

Abstract: Hallucinations pose a significant obstacle to the reliability and widespread adoption of language models, yet their accurate measurement remains a persistent challenge. While many task- and domain-specific metrics have been proposed to assess faithfulness and factuality concerns, the robustness and generalization of these metrics are still untested. In this paper, we conduct a large-scale empirical evaluation of 6 diverse sets of hallucination detection metrics across 4 datasets, 37 language models from 5 families, and 5 decoding methods. Our extensive investigation reveals concerning gaps in current hallucination evaluation: metrics often fail to align with human judgments, take an overtly myopic view of the problem, and show inconsistent gains with parameter scaling. Encouragingly, LLM-based evaluation, particularly with GPT-4, yields the best overall results, and mode-seeking decoding methods seem to reduce hallucinations, especially in knowledge-grounded settings. These findings underscore the need for more robust metrics to understand and quantify hallucinations, and better strategies to mitigate them.

Method: AUGMENT uses linguistically informed rules to generate controlled paraphrases, with quality checks for instruction adherence, semantic similarity, and realism to ensure reliable and meaningful auditing.

Result: Case studies on BBQ and MMLU datasets show that controlled paraphrases uncover systematic weaknesses that remain hidden under unconstrained variation.

Conclusion: AUGMENT provides a valuable framework for reliable LLM auditing by generating user-grounded, controlled paraphrases that reveal systematic model weaknesses.

Abstract: Large language models (LLMs) are highly sensitive to subtle changes in prompt phrasing, posing challenges for reliable auditing. Prior methods often apply unconstrained prompt paraphrasing, which risk missing linguistic and demographic factors that shape authentic user interactions. We introduce AUGMENT (Automated User-Grounded Modeling and Evaluation of Natural Language Transformations), a framework for generating controlled paraphrases, grounded in user behaviors. AUGMENT leverages linguistically informed rules and enforces quality through checks on instruction adherence, semantic similarity, and realism, ensuring paraphrases are both reliable and meaningful for auditing. Through case studies on the BBQ and MMLU datasets, we show that controlled paraphrases uncover systematic weaknesses that remain obscured under unconstrained variation. These results highlight the value of the AUGMENT framework for reliable auditing.

Method: Developed Hakim embedding model with RetroMAE-based architecture for retrieval tasks, introduced three new datasets (Corpesia, Pairsia-sup, Pairsia-unsup) for supervised and unsupervised training, and proposed a new BERT-based baseline model.

Result: Achieved 8.5% performance improvement over existing approaches on FaMTEB benchmark, with consistent higher accuracy across various Persian NLP tasks. RetroMAE-based model proved particularly effective for textual information retrieval.

Conclusion: These contributions establish a new foundation for advancing Persian language understanding, particularly for applications in chatbots and retrieval-augmented generation systems.

Abstract: Recent advancements in text embedding have significantly improved natural language understanding across many languages, yet Persian remains notably underrepresented in large-scale embedding research. In this paper, we present Hakim, a novel state-of-the-art Persian text embedding model that achieves a 8.5% performance improvement over existing approaches on the FaMTEB benchmark, outperforming all previously developed Persian language models. As part of this work, we introduce three new datasets - Corpesia, Pairsia-sup, and Pairsia-unsup - to support supervised and unsupervised training scenarios. Additionally, Hakim is designed for applications in chatbots and retrieval-augmented generation (RAG) systems, particularly addressing retrieval tasks that require incorporating message history within these systems. We also propose a new baseline model built on the BERT architecture. Our language model consistently achieves higher accuracy across various Persian NLP tasks, while the RetroMAE-based model proves particularly effective for textual information retrieval applications. Together, these contributions establish a new foundation for advancing Persian language understanding.

Method: Leverages logical expression translation to convert harmful natural language prompts into formal logical expressions, exploiting the distributional gap between alignment data and logic-based inputs.

Result: Demonstrated effectiveness across a multilingual jailbreak dataset spanning three languages and various evaluation settings.

Conclusion: LogiBreak successfully circumvents LLM safety systems by preserving semantic intent while evading safety constraints through logical expression translation.

Abstract: Despite substantial advancements in aligning large language models (LLMs) with human values, current safety mechanisms remain susceptible to jailbreak attacks. We hypothesize that this vulnerability stems from distributional discrepancies between alignment-oriented prompts and malicious prompts. To investigate this, we introduce LogiBreak, a novel and universal black-box jailbreak method that leverages logical expression translation to circumvent LLM safety systems. By converting harmful natural language prompts into formal logical expressions, LogiBreak exploits the distributional gap between alignment data and logic-based inputs, preserving the underlying semantic intent and readability while evading safety constraints. We evaluate LogiBreak on a multilingual jailbreak dataset spanning three languages, demonstrating its effectiveness across various evaluation settings and linguistic contexts.

Method: End-to-end multi-turn RL framework that learns from online web interactions by asynchronously generating diverse trajectories guided by binary task success rewards.

Result: Boosted task success rates from 6.1% to 33.9% for Qwen-2.5-3B and from 8.5% to 44.8% for Llama-3.1-8B on WebArena-Lite, outperforming state-of-the-art methods including OpenAI o3.

Conclusion: The framework demonstrates effectiveness of thinking-based prompting and test-time scaling, with warm-up training and chain-of-thought reasoning providing important insights for web agent development.

Abstract: While reinforcement learning (RL) has demonstrated remarkable success in enhancing large language models (LLMs), it has primarily focused on single-turn tasks such as solving math problems. Training effective web agents for multi-turn interactions remains challenging due to the complexity of long-horizon decision-making across dynamic web interfaces. In this work, we present WebAgent-R1, a simple yet effective end-to-end multi-turn RL framework for training web agents. It learns directly from online interactions with web environments by asynchronously generating diverse trajectories, entirely guided by binary rewards depending on task success. Experiments on the WebArena-Lite benchmark demonstrate the effectiveness of WebAgent-R1, boosting the task success rate of Qwen-2.5-3B from 6.1% to 33.9% and Llama-3.1-8B from 8.5% to 44.8%, significantly outperforming existing state-of-the-art methods and strong proprietary models such as OpenAI o3. In-depth analyses reveal the effectiveness of the thinking-based prompting strategy and test-time scaling through increased interactions for web tasks. We further investigate different RL initialization policies by introducing two variants, namely WebAgent-R1-Zero and WebAgent-R1-CoT, which highlight the importance of the warm-up training stage (i.e., behavior cloning) and provide insights on incorporating long chain-of-thought (CoT) reasoning in web agents.

Method: Introduced UNCLE benchmark covering 5 domains with 1,000+ entities, paired short- and long-form QA items, and new metrics. Explored prompt-based and training-based improvement methods.

Result: Current models fail to convey uncertainty appropriately in long-form generation. Training-based methods yielded greater performance gains than prompt-based approaches.

Conclusion: UNCLE provides a comprehensive framework for evaluating uncertainty expression, highlighting alignment gaps between short- and long-form responses as promising future research direction.

Abstract: Large Language Models (LLMs) are prone to hallucination, particularly in long-form generations. A promising direction to mitigate hallucination is to teach LLMs to express uncertainty explicitly when they lack sufficient knowledge. However, existing work lacks direct and fair evaluation of LLMs' ability to express uncertainty effectively in long-form generation. To address this gap, we first introduce UNCLE, a benchmark designed to evaluate uncertainty expression in both long- and short-form question answering (QA). UNCLE covers five domains and includes more than 1,000 entities, each with paired short- and long-form QA items. Our dataset is the first to directly link short- and long-form QA through aligned questions and gold-standard answers. Along with UNCLE, we propose a suite of new metrics to assess the models' capabilities to selectively express uncertainty. We then demonstrate that current models fail to convey uncertainty appropriately in long-form generation. We further explore both prompt-based and training-based methods to improve models’ performance, with the training-based methods yielding greater gains. Further analysis of alignment gaps between short- and long-form uncertainty expression highlights promising directions for future research using UNCLE.

Method: The authors propose β-GRPO, a reinforcement learning-based training method that incorporates confidence thresholds to reward high-certainty search decisions, addressing the link between search inefficiencies and model uncertainty.

Result: Experiments on seven QA benchmarks show β-GRPO enables a 3B model to achieve better agentic RAG ability, outperforming other baselines with 4% higher average exact match score. One model could have avoided searching in 27.7% of its search steps.

Conclusion: The study demonstrates that addressing model uncertainty through confidence-based reinforcement learning can significantly improve the efficiency and accuracy of Agentic RAG systems.

Abstract: Agentic Retrieval-Augmented Generation (RAG) systems enhance Large Language Models (LLMs) by enabling dynamic, multi-step reasoning and information retrieval. However, these systems often exhibit sub-optimal search behaviors like over-search (retrieving redundant information) and under-search (failing to retrieve necessary information), which hinder efficiency and reliability. This work formally defines and quantifies these behaviors, revealing their prevalence across multiple QA datasets and agentic RAG systems (e.g., one model could have avoided searching in 27.7% of its search steps). Furthermore, we demonstrate a crucial link between these inefficiencies and the models' uncertainty regarding their own knowledge boundaries, where response accuracy correlates with model’s uncertainty in its search decisions. To address this, we propose $\beta$-GRPO, a reinforcement learning-based training method that incorporates confidence threshold to reward high-certainty search decisions. Experiments on seven QA benchmarks show that $\beta$-GRPO enable a 3B model with better agentic RAG ability, outperforming other strong baselines with a 4% higher average exact match score.

Method: SEA formulates inference as iterative optimization on an energy function over actions in continuous space, directly adapting original responses via gradient-based sampling.

Result: SEA outperforms the second-best baseline with relative improvements of up to 77.51% on AdvBench and 16.36% on MATH.

Conclusion: SEA provides an effective alternative to discrete search methods for inference-time alignment by leveraging continuous optimization in latent space.

Abstract: Aligning large language models with human feedback at inference time has received increasing attention due to its flexibility. Existing methods rely on generating multiple responses from the base policy for search using a reward model, which can be considered as searching in a discrete response space. However, these methods struggle to explore informative candidates when the base policy is weak or the candidate set is small, resulting in limited effectiveness. In this paper, to address this problem, we propose Simple Energy Adaptation ($\textbf{SEA}$), a simple yet effective algorithm for inference-time alignment. In contrast to expensive search over the discrete space, SEA directly adapts original responses from the base policy toward the optimal one via gradient-based sampling in continuous latent space. Specifically, SEA formulates inference as an iterative optimization procedure on an energy function over actions in the continuous space defined by the optimal policy, enabling simple and effective alignment. For instance, despite its simplicity, SEA outperforms the second-best baseline with a relative improvement of up to $ \textbf{77.51%}$ on AdvBench and $\textbf{16.36%}$ on MATH. Our code is publicly available at https://github.com/yuanyige/sea

Method: Created a benchmark with 68,000 question/SQL query/answer triples from templates, grounded in a harmonized BigQuery knowledge base integrating gene-disease associations, omics data, and drug records. Evaluated various LLMs across different prompting strategies.

Result: GPT-o3-mini achieved 59.0% execution accuracy, while the custom multi-step agent BMSQL reached 62.6%, both significantly below the expert baseline of 90.0%.

Conclusion: BiomedSQL provides a foundation for advancing text-to-SQL systems that can support scientific discovery through robust reasoning over structured biomedical knowledge bases.

Abstract: Biomedical researchers increasingly rely on large-scale structured databases for complex analytical tasks. However, current text-to-SQL systems often struggle to map qualitative scientific questions into executable SQL, particularly when implicit domain reasoning is required. We introduce BiomedSQL, the first benchmark explicitly designed to evaluate scientific reasoning in text-to-SQL generation over a real-world biomedical knowledge base. BiomedSQL comprises 68,000 question/SQL query/answer triples generated from templates and grounded in a harmonized BigQuery knowledge base that integrates gene-disease associations, causal inference from omics data, and drug approval records. Each question requires models to infer domain-specific criteria, such as genome-wide significance thresholds, effect directionality, or trial phase filtering, rather than rely on syntactic translation alone. We evaluate a range of open- and closed-source LLMs across prompting strategies and interaction paradigms. Our results reveal a substantial performance gap: GPT-o3-mini achieves 59.0% execution accuracy, while our custom multi-step agent, BMSQL, reaches 62.6%, both well below the expert baseline of 90.0%. BiomedSQL provides a new foundation for advancing text-to-SQL systems capable of supporting scientific discovery through robust reasoning over structured biomedical knowledge bases. Our dataset is publicly available at https://huggingface.co/datasets/NIH-CARD/BiomedSQL, and our code is open-source at https://github.com/NIH-CARD/biomedsql.

Method: Combines linguistics expert knowledge to curate variety-specific features and transformation guidelines with LLM-based transformations to ensure linguistic validity and scalability, transforming six benchmark datasets into 38 English varieties.

Result: Significant performance disparities with accuracy decreasing by up to 46.3% on non-standard varieties, highlighting the importance of comprehensive linguistic robustness evaluation.

Conclusion: The framework enables extensive evaluation of LLM linguistic robustness across diverse English varieties, with each construction validated through statistical testing and expert consultation to ensure linguistic validity.

Abstract: Large Language Models (LLMs) are predominantly evaluated on Standard American English (SAE), often overlooking the diversity of global English varieties. This narrow focus may raise fairness concerns as degraded performance on non-standard varieties can lead to unequal benefits for users worldwide. Therefore, it is critical to extensively evaluate the linguistic robustness of LLMs on multiple non-standard English varieties. We introduce Trans-EnV, a framework that automatically transforms SAE datasets into multiple English varieties to evaluate the linguistic robustness. Our framework combines (1) linguistics expert knowledge to curate variety-specific features and transformation guidelines from linguistic literature and corpora, and (2) LLM-based transformations to ensure both linguistic validity and scalability. Using Trans-EnV, we transform six benchmark datasets into 38 English varieties and evaluate seven state-of-the-art LLMs. Our results reveal significant performance disparities, with accuracy decreasing by up to 46.3% on non-standard varieties. These findings highlight the importance of comprehensive linguistic robustness evaluation across diverse English varieties. Each construction of Trans-EnV was validated through rigorous statistical testing and consultation with a researcher in the field of second language acquisition, ensuring its linguistic validity. Our code and datasets are publicly available at https://github.com/jiyounglee-0523/TransEnV and https://huggingface.co/collections/jiyounglee0523/transenv-681eadb3c0c8cf363b363fb1.

Method: Two training-free techniques: FreeCache (KV approximation caching that reuses stable KV projections across denoising steps) and Guided Diffusion (using a lightweight pretrained autoregressive model to supervise token unmasking to reduce denoising iterations).

Result: Achieved average 12.14x end-to-end speedup across various tasks with negligible accuracy degradation, making diffusion language models achieve comparable and even faster latency than autoregressive models for the first time.

Conclusion: The work successfully enables scaling up diffusion language models to broader applications by solving their key inference efficiency limitations.

Abstract: Diffusion language models offer parallel token generation and inherent bidirectionality, promising more efficient and powerful sequence modeling compared to autoregressive approaches. However, state-of-the-art diffusion models (e.g., Dream 7B, LLaDA 8B) suffer from slow inference. While they match the quality of similarly sized autoregressive (AR) models (e.g., Qwen2.5 7B, Llama3 8B), their iterative denoising requires multiple full-sequence forward passes, resulting in high computational costs and latency, particularly for long input prompts and long-context scenarios. Furthermore, parallel token generation introduces token incoherence problems, and current sampling heuristics suffer from significant quality drops with decreasing denoising steps. We address these limitations with two training-free techniques. First, we propose FreeCache, a Key-Value (KV) approximation caching technique that reuses stable KV projections across denoising steps, effectively reducing the computational cost of DLM inference. Second, we introduce Guided Diffusion, a training-free method that uses a lightweight pretrained autoregressive model to supervise token unmasking, dramatically reducing the total number of denoising iterations without sacrificing quality. We conduct extensive evaluations on open-source reasoning benchmarks, and our combined methods deliver an average of 12.14x end-to-end speedup across various tasks with negligible accuracy degradation. For the first time, diffusion language models achieve a comparable and even faster latency as the widely adopted autoregressive models. Our work successfully paved the way for scaling up the diffusion language model to a broader scope of applications across different domains.

Method: Proposed FlowNIB, a dynamic and scalable method for estimating mutual information during training that addresses limitations of classical IB approaches. Also developed a generalized framework for measuring representational complexity.

Result: Bidirectional models retain more mutual information and exhibit higher effective dimensionality than unidirectional models. Bidirectional representations are strictly more informative under mild conditions, as validated through extensive experiments across multiple models and tasks.

Conclusion: The work provides a principled explanation for the effectiveness of bidirectional architectures and introduces a practical tool (FlowNIB) for analyzing information flow in deep language models.

Abstract: Bidirectional language models have better context understanding and perform better than unidirectional models on natural language understanding tasks, yet the theoretical reasons behind this advantage remain unclear. In this work, we investigate this disparity through the lens of the Information Bottleneck (IB) principle, which formalizes a trade-off between compressing input information and preserving task-relevant content. We propose FlowNIB, a dynamic and scalable method for estimating mutual information during training that addresses key limitations of classical IB approaches, including computational intractability and fixed trade-off schedules. Theoretically, we show that bidirectional models retain more mutual information and exhibit higher effective dimensionality than unidirectional models. To support this, we present a generalized framework for measuring representational complexity and prove that bidirectional representations are strictly more informative under mild conditions. We further validate our findings through extensive experiments across multiple models and tasks using FlowNIB, revealing how information is encoded and compressed throughout training. Together, our work provides a principled explanation for the effectiveness of bidirectional architectures and introduces a practical tool for analyzing information flow in deep language models.

Method: Employed causal tracing to systematically analyze how pretrained causal transformers process idioms, localizing three key mechanisms in the model architecture.

Result: Identified three mechanisms: early layers retrieve figurative interpretations while suppressing literal ones; context is leveraged from earliest layers with refinement in later layers; parallel pathways maintain both interpretations with figurative prioritized in intermediate pathway and literal favored in direct route.

Conclusion: The study provides mechanistic evidence for idiom comprehension in autoregressive transformers, revealing how they handle the ambiguity between literal and figurative interpretations through specialized processing pathways.

Abstract: Idioms present a unique challenge for language models due to their non-compositional figurative interpretations, which often strongly diverge from the idiom’s literal interpretation. In this paper, we employ causal tracing to systematically analyze how pretrained causal transformers deal with this ambiguity. We localize three mechanisms: (i) Early sublayers and specific attention heads retrieve an idiom’s figurative interpretation, while suppressing its literal interpretation. (ii) When disambiguating context precedes the idiom, the model leverages it from the earliest layer and later layers refine the interpretation if the context conflicts with the retrieved interpretation. (iii) Then, selective, competing pathways carry both interpretations: an intermediate pathway prioritizes the figurative interpretation and a parallel direct route favors the literal interpretation, ensuring that both readings remain available. Our findings provide mechanistic evidence for idiom comprehension in autoregressive transformers.

Method: Introduces FineLogic framework with three evaluation dimensions, fine-tunes LLMs on four supervision styles (one natural language and three symbolic variants), and conducts probing analysis.

Result: Natural language supervision excels at generalization to out-of-distribution and long-chain problems, while symbolic supervision produces structurally sound, atomic reasoning steps. Fine-tuning primarily refines step-by-step generation rather than early answer convergence.

Conclusion: The framework provides a more rigorous lens for evaluating and improving logical reasoning in LLMs, revealing key trade-offs between different supervision formats.

Abstract: Logical reasoning is a core capability for large language models (LLMs), yet existing benchmarks that rely solely on final-answer accuracy fail to capture the quality of the reasoning process. To address this, we introduce FineLogic, a fine-grained evaluation framework that assesses logical reasoning across three dimensions: overall accuracy, stepwise soundness, and representation-level probing. Leveraging this framework, we conduct a comprehensive study on how different supervision formats in fine-tuning shape reasoning abilities. We fine-tune LLMs on four supervision styles: one in natural language and three symbolic variants. We find a key trade-off: natural language supervision excels at generalization to out-of-distribution and long-chain problems, whereas symbolic supervision is superior at instilling structurally sound, atomic reasoning steps. Furthermore, our probing analysis indicates that fine-tuning primarily refines the model’s step-by-step generation process, rather than improving its ability to converge on an answer early. Together, our framework and analysis provide a more rigorous lens for evaluating and improving logical reasoning in LLMs. The code is available at https://github.com/YujunZhou/FineLogic.

Method: Evaluated VLMs and LLMs on over 14,000 handwritten answers from Indonesian grade-4 classrooms covering Mathematics and English aligned with local curriculum, focusing on grading and generating personalized Indonesian feedback using rubric-based evaluation.

Result: VLMs struggled with handwriting recognition, leading to error propagation in LLM grading. However, LLM feedback remained pedagogically useful despite imperfect visual inputs, though limitations were found in personalization and contextual relevance.

Conclusion: Current VLMs face significant challenges with real-world handwriting recognition in educational contexts, but LLMs can still provide valuable pedagogical feedback even with imperfect visual inputs, highlighting both limitations and potential for AI in underrepresented classroom assessment.

Abstract: Despite rapid progress in vision-language and large language models (VLMs and LLMs), their effectiveness for AI-driven educational assessment in real-world, underrepresented classrooms remains largely unexplored. We evaluate state-of-the-art VLMs and LLMs on over 14K handwritten answers from grade-4 classrooms in Indonesia, covering Mathematics and English aligned with the local national curriculum. Unlike prior work on clean digital text, our dataset features naturally curly, diverse handwriting from real classrooms, posing realistic visual and linguistic challenges. Assessment tasks include grading and generating personalized Indonesian feedback guided by rubric-based evaluation. Results show that the VLM struggles with handwriting recognition, causing error propagation in LLM grading, yet LLM feedback remains pedagogically useful despite imperfect visual inputs, revealing limits in personalization and contextual relevance.

Method: The study evaluates three key aspects: LLM effectiveness in generating reviews compared to humans, impact of adversarial attacks on LLM-generated reviews, and analysis of challenges and mitigation strategies for LLM-based review systems.

Result: The evaluation reveals significant vulnerabilities where text manipulations can distort LLM assessments, compromising the reliability of automated peer review.

Conclusion: Addressing adversarial risks is crucial to ensure AI strengthens rather than compromises the integrity of scholarly communication, highlighting the need for robust LLM-based review systems.

Abstract: Peer review is essential for maintaining academic quality, but the increasing volume of submissions places a significant burden on reviewers. Large language models (LLMs) offer potential assistance in this process, yet their susceptibility to textual adversarial attacks raises reliability concerns. This paper investigates the robustness of LLMs used as automated reviewers in the presence of such attacks. We focus on three key questions: (1) The effectiveness of LLMs in generating reviews compared to human reviewers. (2) The impact of adversarial attacks on the reliability of LLM-generated reviews. (3) Challenges and potential mitigation strategies for LLM-based review. Our evaluation reveals significant vulnerabilities, as text manipulations can distort LLM assessments. We offer a comprehensive evaluation of LLM performance in automated peer reviewing and analyze its robustness against adversarial attacks. Our findings emphasize the importance of addressing adversarial risks to ensure AI strengthens, rather than compromises, the integrity of scholarly communication.

Method: Proposes Inference-Time Language Control (ITLC) - a novel method using latent injection to enable precise cross-lingual language control and mitigate language confusion.

Result: Empirically confirmed representation alignment in middle layers, demonstrated ITLC’s strong cross-lingual control capabilities while preserving semantic integrity, and showed effectiveness in alleviating cross-lingual language confusion.

Conclusion: Advances understanding of representation alignment in LLMs and introduces a practical solution for enhancing both monolingual and cross-lingual performance by addressing language confusion issues.

Abstract: Large Language Models (LLMs) have demonstrated remarkable generalization capabilities across tasks and languages, revolutionizing natural language processing. This paper investigates the naturally emerging representation alignment in LLMs, particularly in the middle layers, and its implications for disentangling language-specific and language-agnostic information. We empirically confirm the existence of this alignment, analyze its behavior in comparison to explicitly designed alignment models, and demonstrate its potential for language-specific manipulation without semantic degradation. Building on these findings, we propose Inference-Time Language Control (ITLC), a novel method that leverages latent injection to enable precise cross-lingual language control and mitigate language confusion in LLMs. Our experiments highlight ITLC’s strong cross-lingual control capabilities while preserving semantic integrity in target languages. Furthermore, we demonstrate its effectiveness in alleviating the cross-lingual language confusion problem, which persists even in current large-scale LLMs, leading to inconsistent language generation. This work advances our understanding of representation alignment in LLMs and introduces a practical solution for enhancing their monolingual and cross-lingual performance.

Method: Introduces PeopleProfiles - a large-scale, multi-level dataset of claim support annotations on biographical Wikipedia articles. Analyzes claim support across lead sections and article bodies, and examines citation practices.

Result: Found that ~22% of claims in Wikipedia lead sections are unsupported by the article body; ~30% of claims in the article body are unsupported by publicly accessible sources; real-world Wikipedia citation practices often differ from documented standards.

Conclusion: Complex evidence retrieval remains a challenge for modern reasoning rerankers, highlighting ongoing issues with Wikipedia’s grounding reliability despite its importance as an NLP resource.

Abstract: Wikipedia is a critical resource for modern NLP, serving as a rich repository of up-to-date and citation-backed information on a wide variety of subjects. The reliability of Wikipedia – its groundedness in its cited sources – is vital to this purpose. This work analyzes both how grounded Wikipedia is and how readily fine-grained grounding evidence can be retrieved. To this end, we introduce PeopleProfiles – a large-scale, multi-level dataset of claim support annotations on biographical Wikipedia articles. We show that: (1) ~22% of claims in Wikipedia lead sections are unsupported by the article body; (2) ~30% of claims in the article body are unsupported by their publicly accessible sources; and (3) real-world Wikipedia citation practices often differ from documented standards. Finally, we show that complex evidence retrieval remains a challenge – even for recent reasoning rerankers.

Method: Conducted experiments on Visual Question Answering (VQA) tasks using GPT-4o and Qwen2-VL, analyzing how prompt biases affect measured epistemic and aleatoric uncertainty across varying bias-free confidence levels.

Result: Mitigating prompt-introduced bias improves uncertainty quantification in GPT-4o. All biases have greater effects on both uncertainties when bias-free model confidence is lower. Lower bias-free confidence leads to greater bias-induced underestimation of epistemic uncertainty, causing overconfident estimates, while having no significant effect on aleatoric uncertainty direction.

Conclusion: The distinct effects of bias on epistemic vs. aleatoric uncertainty deepen understanding of bias mitigation for uncertainty quantification and can inform development of more advanced techniques.

Abstract: With the growing adoption of Large Language Models (LLMs) for open-ended tasks, accurately assessing epistemic uncertainty, which reflects a model’s lack of knowledge, has become crucial to ensuring reliable outcomes. However, quantifying epistemic uncertainty in such tasks is challenging due to the presence of aleatoric uncertainty, which arises from multiple valid answers. While bias can introduce noise into epistemic uncertainty estimation, it may also reduce noise from aleatoric uncertainty. To investigate this trade-off, we conduct experiments on Visual Question Answering (VQA) tasks and find that mitigating prompt-introduced bias improves uncertainty quantification in GPT-4o. Building on prior work showing that LLMs tend to copy input information when model confidence is low, we further analyze how these prompt biases affect measured epistemic and aleatoric uncertainty across varying bias-free confidence levels with GPT-4o and Qwen2-VL. We find that all considered biases have greater effects in both uncertainties when bias-free model confidence is lower. Moreover, lower bias-free model confidence is associated with greater bias-induced underestimation of epistemic uncertainty, resulting in overconfident estimates, whereas it has no significant effect on the direction of bias effect in aleatoric uncertainty estimation. These distinct effects deepen our understanding of bias mitigation for uncertainty quantification and potentially inform the development of more advanced techniques.

Method: Conducted a meta-analysis of existing data on downstream scaling laws to examine when predictable scaling occurs and how experimental settings affect scaling behavior.

Result: Predictable scaling only occurs in 39% of cases, and seemingly minor changes to experimental settings can completely alter scaling behavior.

Conclusion: Scaling laws require understanding the conditions under which they succeed, and modeling must account for cases where scaling deviates from linear trends rather than assuming universal linear scaling.

Abstract: Downstream scaling laws aim to predict task performance at larger scales from the model’s performance at smaller scales. Whether such prediction should be possible is unclear: some works discover clear linear scaling trends after simple transformations of the performance metric, whereas others point out fundamental challenges to downstream scaling laws, such as emergence and inverse scaling. In this work, we conduct a meta-analysis of existing data on downstream scaling laws, and we find that predictable scaling only occurs in a minority of cases: 39% of the time. Moreover, seemingly benign changes to the experimental setting can completely change the scaling behavior. Our analysis underscores the need to understand the conditions under which scaling laws succeed. To accurately model the relationship between pretraining loss and task performance, we must embrace the cases in which scaling behavior deviates from linear trends.

Method: Rigorous benchmarking using a dataset of over 1,000 health questions, assessing models across honesty, helpfulness, and harmlessness metrics.

Result: AlpaCare-13B achieves highest accuracy (91.7%) and harmlessness (0.92); BioMistral-7B-DARE shows improved safety (0.90) despite smaller size; few-shot prompting boosts accuracy from 78% to 85%; all models struggle with complex queries.

Conclusion: There are trade-offs between factual reliability and safety in medical LLMs, with domain-specific tuning improving safety and few-shot prompting enhancing accuracy, but challenges remain for complex clinical questions.

Abstract: Large Language Models (LLMs) hold significant promise for transforming digital health by enabling automated medical question answering. However, ensuring these models meet critical industry standards for factual accuracy, usefulness, and safety remains a challenge, especially for open-source solutions. We present a rigorous benchmarking framework using a dataset of over 1,000 health questions. We assess model performance across honesty, helpfulness, and harmlessness. Our results highlight trade-offs between factual reliability and safety among evaluated models – Mistral-7B, BioMistral-7B-DARE, and AlpaCare-13B. AlpaCare-13B achieves the highest accuracy (91.7%) and harmlessness (0.92), while domain-specific tuning in BioMistral-7B-DARE boosts safety (0.90) despite its smaller scale. Few-shot prompting improves accuracy from 78% to 85%, and all models show reduced helpfulness on complex queries, highlighting ongoing challenges in clinical QA.

Method: Developed RPP (ranking metrics per PetaFLOP) and QPP (queries per PetaFLOP) metrics, along with an interpretable FLOPs estimator to calculate FLOPs without running experiments.

Result: Conducted comprehensive experiments evaluating various LLM-based rerankers with different architectures to study efficiency-effectiveness tradeoffs.

Conclusion: The proposed metrics provide better evaluation of LLM-based reranker efficiency and bring attention to efficiency-effectiveness tradeoffs in the research community.

Abstract: Large Language Models (LLMs) have recently been applied to reranking tasks in information retrieval, achieving strong performance. However, their high computational demands often hinder practical deployment. Existing studies evaluate the efficiency of LLM-based rerankers using proxy metrics such as latency, the number of forward passes, input tokens, and output tokens. However, these metrics depend on hardware and running-time choices (\eg parallel or not, batch size, etc), and often fail to account for model size, making it difficult to interpret and obscuring the evaluation of the efficiency-effectiveness tradeoff. To address this issue, we propose \ours\footnote{https://github.com/zhiyuanpeng/EER-FLOPs.} for LLM-based rerankers: RPP (ranking metrics per PetaFLOP), measuring how much ranking quality (e.g., NDCG or MRR) a method achieves per PetaFLOP, and QPP (queries per PetaFLOP), measuring how many queries can be processed per PetaFLOP. Accompanied by the new metrics, an interpretable FLOPs estimator is developed to estimate the FLOPs of an LLM-based reranker even without running any experiments. Based on the proposed metrics, we conduct comprehensive experiments to evaluate a wide range of LLM-based rerankers with different architectures, studying the efficiency-effectiveness trade-off and bringing this issue to the attention of the research community.

Method: Developed a framework to study poisoning in LLMs, tested multiple triggers with high embedding similarity, and proposed a post hoc recovery method using layer-wise weight difference analysis for selective retraining.

Result: Multiple distinct backdoor triggers can coexist in a single model without interference, and poisoned triggers remain robust even with token substitutions or long token spans, revealing broader vulnerability.

Conclusion: LLMs have persistent multi-trigger poisoning vulnerabilities, but the proposed selective retraining defense can effectively remove trigger behavior with minimal parameter updates.

Abstract: Recent studies have shown that Large Language Models (LLMs) are vulnerable to data poisoning attacks, where malicious training examples embed hidden behaviours triggered by specific input patterns. However, most existing works assume a phrase and focus on the attack’s effectiveness, offering limited understanding of trigger mechanisms and how multiple triggers interact within the model. In this paper, we present a framework for studying poisoning in LLMs. We show that multiple distinct backdoor triggers can coexist within a single model without interfering with each other, enabling adversaries to embed several triggers concurrently. Using multiple triggers with high embedding similarity, we demonstrate that poisoned triggers can achieve robust activation even when tokens are substituted or separated by long token spans. Our findings expose a broader and more persistent vulnerability surface in LLMs. To mitigate this threat, we propose a post hoc recovery method that selectively retrains specific model components based on a layer-wise weight difference analysis. Our method effectively removes the trigger behaviour with minimal parameter updates, presenting a practical and efficient defence against multi-trigger poisoning.

Method: Identified separate “harmfulness” and “refusal” directions in LLMs’ internal representations. Used causal interventions by steering along these directions to analyze how jailbreak methods work and how adversarial finetuning affects internal beliefs.

Result: Found that jailbreak methods reduce refusal signals without changing internal harmfulness beliefs. Adversarial finetuning has minimal impact on internal harmfulness understanding. Created “Latent Guard” that performs comparably to dedicated safeguard models like Llama Guard 3 8B.

Conclusion: LLMs’ internal understanding of harmfulness is more robust than their refusal decisions, providing a new perspective for AI safety research and enabling intrinsic safeguards resistant to finetuning attacks.

Abstract: LLMs are trained to refuse harmful instructions, but do they truly understand harmfulness beyond just refusing? Prior work has shown that LLMs’ refusal behaviors can be mediated by a one-dimensional subspace, i.e., a refusal direction. In this work, we identify a new dimension to analyze safety mechanisms in LLMs, i.e., harmfulness, which is encoded internally as a separate concept from refusal. There exists a harmfulness direction that is distinct from the refusal direction. As causal evidence, steering along the harmfulness direction can lead LLMs to interpret harmless instructions as harmful, but steering along the refusal direction tends to elicit refusal responses directly without reversing the model’s judgment on harmfulness. Furthermore, using our identified harmfulness concept, we find that certain jailbreak methods work by reducing the refusal signals without reversing the model’s internal belief of harmfulness. We also find that adversarially finetuning models to accept harmful instructions has minimal impact on the model’s internal belief of harmfulness. These insights lead to a practical safety application: The model’s latent harmfulness representation can serve as an intrinsic safeguard (Latent Guard) for detecting unsafe inputs and reducing over-refusals that is robust to finetuning attacks. For instance, our Latent Guard achieves performance comparable to or better than Llama Guard 3 8B, a dedicated finetuned safeguard model, across different jailbreak methods. Our findings suggest that LLMs’ internal understanding of harmfulness is more robust than their refusal decision to diverse input instructions, offering a new perspective to study AI safety.

Method: Analyze keystroke and gaze data to identify behavioral patterns, then organize them into a multi-layered hierarchical BTSS structure representing embedded processing layers.

Result: Developed a BTSS framework that can serve as basis for computational translation agents to simulate temporal dynamics of affect, behavior, and cognition during translation.

Conclusion: The BTSS provides a comprehensive model connecting physical translation behavior with cognitive and affective processes, enabling computational simulation of human translation production.

Abstract: The paper introduces a novel behavioural translation style space (BTSS) that describes possible behavioural translation patterns. The suggested BTSS is organized as a hierarchical structure that entails various embedded processing layers. We posit that observable translation behaviour - i.e. eye and finger movements - is fundamental when executing the physical act of translation but it is caused and shaped by higher-order cognitive processes and affective translation states. We analyse records of keystrokes and gaze data as indicators of the hidden mental processing structure and organize the behavioural patterns as a multi-layered embedded BTSS. We develop a perspective in which the BTSS serves as the basis for a computational translation agent to simulate the temporal dynamics of affect, behavioural routines and cognition during human translation production.

Method: Develop byte-level LMs with learnable tokenizers that include a submodule predicting boundaries between input byte sequences, encoding them into variable-length segments. Propose FLEXITOKENS - a simplified training objective that enables greater flexibility during adaptation compared to existing tokenizer-free methods that enforce fixed compression rates.

Result: Across multiple multilingual benchmarks, morphologically diverse tasks, and domains, FLEXITOKENS consistently reduces token over-fragmentation and achieves up to 10% improvements on downstream task performance compared to subword and other gradient-based tokenizers.

Conclusion: FLEXITOKENS provides a more flexible approach to tokenization that adapts better to new data distributions, addressing the limitations of rigid subword tokenizers and improving model performance across diverse linguistic contexts.

Abstract: Language models (LMs) are challenging to adapt to new data distributions by simple finetuning. This is due to the rigidity of their subword tokenizers, which typically remain unchanged during adaptation. This inflexibility often leads to inefficient tokenization, causing overfragmentation of out-of-distribution domains, unseen languages, or scripts. In this work, we develop byte-level LMs with learnable tokenizers to make tokenization adaptive. Our models include a submodule that learns to predict boundaries between the input byte sequence, encoding it into variable-length segments. Existing tokenizer-free methods train this boundary predictor using an auxiliary loss that enforces a fixed compression rate across the training corpus, introducing a new kind of rigidity. We propose FLEXITOKENS, a simplified training objective that enables significantly greater flexibility during adaptation. Evaluating across multiple multilingual benchmarks, morphologically diverse tasks, and domains, we demonstrate that FLEXITOKENS consistently reduces token over-fragmentation and achieves up to 10% improvements on downstream task performance compared to subword and other gradient-based tokenizers. Code and data for our experiments will be released at https://github.com/owos/flexitokens

Method: Systematically distill real user feedback from over 21,000 clinical encounters into structured checklists that are interpretable and enforceable by LLM-based evaluators.

Result: The feedback-derived checklist outperforms baseline approaches in coverage, diversity, and predictive power for human ratings. It’s robust to quality-degrading perturbations and aligns well with clinician preferences.

Conclusion: The checklist provides a practical evaluation methodology for flagging substandard AI-generated clinical notes in offline research settings.

Abstract: AI-generated clinical notes are increasingly used in healthcare, but evaluating their quality remains a challenge due to high subjectivity and limited scalability of expert review. Existing automated metrics often fail to align with real-world physician preferences. To address this, we propose a pipeline that systematically distills real user feedback into structured checklists for note evaluation. These checklists are designed to be interpretable, grounded in human feedback, and enforceable by LLM-based evaluators. Using deidentified data from over 21,000 clinical encounters (prepared in accordance with the HIPAA safe harbor standard) from a deployed AI medical scribe system, we show that our feedback-derived checklist outperforms a baseline approach in our offline evaluations in coverage, diversity, and predictive power for human ratings. Extensive experiments confirm the checklist’s robustness to quality-degrading perturbations, significant alignment with clinician preferences, and practical value as an evaluation methodology. In offline research settings, our checklist offers a practical tool for flagging notes that may fall short of our defined quality standards.

Method: Used small poisoned budgets during instruction tuning that paired dialectal prompts (AAVE and Southern dialect) with toxic/stereotyped completions, then evaluated across multiple model families and scales using multi-metric audit combining classifier-based toxicity and LLM-as-a-judge.

Result: Poisoned exposure elevated toxicity and stereotype expression for dialectal inputs (especially AAVE), while Standard American English remained lower but not immune. Poisoned models showed emergent jailbreaking despite no explicit slurs, indicating weakened alignment rather than memorization.

Conclusion: Need for dialect-aware evaluation, content-level stereotype auditing, and training protocols that explicitly decouple style from toxicity to prevent bias amplification through style-based contamination.

Abstract: Style-conditioned data poisoning is identified as a covert vector for amplifying sociolinguistic bias in large language models. Using small poisoned budgets that pair dialectal prompts – principally African American Vernacular English (AAVE) and a Southern dialect – with toxic or stereotyped completions during instruction tuning, this work probes whether linguistic style can act as a latent trigger for harmful behavior. Across multiple model families and scales, poisoned exposure elevates toxicity and stereotype expression for dialectal inputs – most consistently for AAVE – while Standard American English remains comparatively lower yet not immune. A multi-metric audit combining classifier-based toxicity with an LLM-as-a-judge reveals stereotype-laden content even when lexical toxicity appears muted, indicating that conventional detectors under-estimate sociolinguistic harms. Additionally, poisoned models exhibit emergent jailbreaking despite the absence of explicit slurs in the poison, suggesting weakened alignment rather than memorization. These findings underscore the need for dialect-aware evaluation, content-level stereotype auditing, and training protocols that explicitly decouple style from toxicity to prevent bias amplification through seemingly minor, style-based contamination.

Method: Flora constructs long contexts by arbitrarily assembling short instructions based on categories and using long-context meta-instructions for response generation, without requiring LLMs or human intervention.

Result: Experiments on Llama3-8B-Instruct and QwQ-32B show Flora-enhanced models excel in three long-context benchmarks while maintaining strong short-context task performance.

Conclusion: Flora provides an effective solution for improving long-context capabilities in LLMs with minimal impact on short-context performance, offering arbitrary length scaling and rich diversity.

Abstract: Effectively handling long contexts is challenging for Large Language Models (LLMs) due to the rarity of long texts, high computational demands, and substantial forgetting of short-context abilities. Recent approaches have attempted to construct long contexts for instruction tuning, but these methods often require LLMs or human interventions, which are both costly and limited in length and diversity. Also, the drop in short-context performances of present long-context LLMs remains significant. In this paper, we introduce Flora, an effortless (human/LLM-free) long-context construction strategy. Flora can markedly enhance the long-context performance of LLMs by arbitrarily assembling short instructions based on categories and instructing LLMs to generate responses based on long-context meta-instructions. This enables Flora to produce contexts of arbitrary length and scale with rich diversity, while only slightly compromising short-context performance. Experiments on Llama3-8B-Instruct and QwQ-32B show that LLMs enhanced by Flora excel in three long-context benchmarks while maintaining strong performances in short-context tasks. Our data-construction code is available at \href{https://github.com/txchen-USTC/Flora}{https://github.com/txchen-USTC/Flora}.

Method: Developed a knowledge graph-based framework that generates varied and subtle conflicts between similar contexts using explicit relational structures of KGs.

Result: Both open-source and proprietary LLMs struggle with conflict detection, especially in multi-hop reasoning, and often fail to identify exact sources of contradictions.

Conclusion: Provides foundation for improving LLMs’ ability to integrate diverse and sometimes conflicting information through in-depth analyses of knowledge conflict handling.

Abstract: Knowledge conflict often arises in retrieval-augmented generation (RAG) systems, where retrieved documents may be inconsistent with one another or contradict the model’s parametric knowledge. Existing benchmarks for investigating the phenomenon have notable limitations, including a narrow focus on the question answering setup, heavy reliance on entity substitution techniques, and a restricted range of conflict types. To address these issues, we propose a knowledge graph (KG)-based framework that generates varied and subtle conflicts between two similar yet distinct contexts, while ensuring interpretability through the explicit relational structure of KGs. Experimental results on our benchmark, MAGIC, provide intriguing insights into the inner workings of LLMs regarding knowledge conflict: both open-source and proprietary models struggle with conflict detection – especially when multi-hop reasoning is required – and often fail to pinpoint the exact source of contradictions. Finally, we present in-depth analyses that serve as a foundation for improving LLMs in integrating diverse, sometimes even conflicting, information.

Method: Progressive reinforcement learning framework that first develops logical reasoning ability, then adaptively optimizes retrieval to align with knowledge corpus characteristics, and finally conducts joint optimization of retrieval-reasoning coordination.

Result: LLaMA3.1-8B-Instruct + Med-R^3 surpasses GPT-4o-mini by 3.93% at comparable parameter scale, while Qwen2.5-14B with Med-R^3 shows 13.53% gain.

Conclusion: Med-R^3 effectively addresses the limitations of existing approaches by enabling joint optimization of retrieval and reasoning through progressive reinforcement learning, achieving superior performance in medical reasoning tasks.

Abstract: In medical scenarios, effectively retrieving external knowledge and leveraging it for rigorous logical reasoning is of significant importance. Despite their potential, existing work has predominantly focused on enhancing either retrieval or reasoning capabilities of the models in isolation, with little attention given to their joint optimization, which leads to limited coordination between the two processes. Additionally, current methods rely heavily on supervised fine-tuning (SFT), which can cause models to memorize existing problem-solving pathways, thereby restricting their generalization ability when confronted with novel problem contexts. Furthermore, while some studies have explored to improve retrieval-augmented reasoning in general domains via reinforcement learning, their reward function designs do not adequately capture the specific demands of the medical domain. To address these challenges, we introduce Med-R$^3$, a Medical Retrieval-augmented Reasoning framework driven by progressive Reinforcement learning. In this framework, we first develop the model’s ability to perform logical reasoning over medical problems. Subsequently, on the basis of this foundation, we adaptively optimize the retrieval capability to better align with the characteristics of knowledge corpus and external information utilization throughout the reasoning process. Finally, we conduct joint optimization of the model’s retrieval and reasoning coordination. Extensive experiments indicate that Med-R$^3$ could achieve state-of-the-art performances, with LLaMA3.1-8B-Instruct + Med-R$^3$ surpassing closed-sourced GPT-4o-mini by 3.93% at a comparable parameter scale, while Qwen2.5-14B augmented with Med-R$^3$ shows a more substantial gain of 13.53%.

Method: Developed CoCoA-zero as a multi-agent RAG framework for conditional knowledge induction and reasoning, then created CoCoA with long-chain training strategy that synthesizes extended multi-agent reasoning trajectories to fine-tune LLMs for better knowledge integration.

Result: Experimental results show CoCoA’s superiority in open-domain QA and multi-hop QA tasks, demonstrating enhanced capability to integrate and leverage both parametric and retrieved knowledge.

Conclusion: The proposed Collaborative Chain-of-Agents framework effectively addresses the synergy limitations in current RAG methods by enabling explicit integration and joint leveraging of parametric and retrieved knowledge through multi-agent reasoning and specialized training.

Abstract: Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs), especially for knowledge-intensive tasks. Despite its advantages, current RAG methods often struggle to fully exploit knowledge during generation. In particular, the synergy between the model’s internal parametric knowledge and external retrieved knowledge remains limited. Retrieved contents may sometimes mislead generation, while certain generated content can guide the model toward more accurate outputs. In this work, we propose Collaborative Chain-of-Agents, a framework designed to enhance explicitly synergy over both parametric and retrieved knowledge. Specifically, we first introduce CoCoA-zero, a multi-agent RAG framework that first performs conditional knowledge induction and then reasons answers. Building on this, we develop CoCoA, a long-chain training strategy that synthesizes extended multi-agent reasoning trajectories from CoCoA-zero to fine-tune the LLM. This strategy enhances the model’s capability to explicitly integrate and jointly leverage parametric and retrieved knowledge. Experimental results demonstrate the superiority of CoCoA in open-domain QA and multi-hop QA.

Method: User-centric, actor-level pipeline combining discourse-aware analysis with metrics for sentiment, syntactic agency, and quotation styles for fine-grained auditing and exclusion-based balancing.

Result: Applied to taz2024full corpus (1980-2024), the pipeline created a more gender-balanced dataset while preserving core dynamics, though subtler biases in sentiment and framing remained.

Conclusion: Structural asymmetries can be reduced through systematic filtering, and tools are released to support further research in discourse-based fairness auditing and equitable corpus construction.

Abstract: Language corpora are the foundation of most natural language processing research, yet they often reproduce structural inequalities. One such inequality is gender discrimination in how actors are represented, which can distort analyses and perpetuate discriminatory outcomes. This paper introduces a user-centric, actor-level pipeline for detecting and mitigating gender discrimination in large-scale text corpora. By combining discourse-aware analysis with metrics for sentiment, syntactic agency, and quotation styles, our method enables both fine-grained auditing and exclusion-based balancing. Applied to the taz2024full corpus of German newspaper articles (1980-2024), the pipeline yields a more gender-balanced dataset while preserving core dynamics of the source material. Our findings show that structural asymmetries can be reduced through systematic filtering, though subtler biases in sentiment and framing remain. We release the tools and reports to support further research in discourse-based fairness auditing and equitable corpus construction.

Method: Systematically investigates four model development stages (scaling, pretraining, post-training, inference) across nine non-English languages, comparing En-CoT (English reasoning) and Target-CoT (target-language reasoning) settings.

Result: Scaling improves En-CoT but Target-CoT lags, especially for multi-step reasoning. Specialized reasoning pretraining helps En-CoT but hurts Target-CoT, while multilingual pretraining helps both. Fine-tuning on translated English traces outperforms model-distilled traces. Language-specific failure modes and efficiency disparities exist.

Conclusion: Multilingual reasoning capabilities require careful consideration across development stages, with translated data showing promise for addressing data scarcity in non-English reasoning.

Abstract: While large reasoning models have shown remarkable ability to generate long chains-of-thought (CoTs) in English, we still lack understanding of how these long-form reasoning abilities transfer to the vast majority of the world’s languages. In this work, we systematically investigate four key stages of model development–scaling, pretraining, post-training, and inference–to understand how long CoT capabilities extend beyond English. We compare two reasoning settings across nine non-English target languages: En-CoT, where models process target-language inputs, but reason in English; and Target-CoT, where models both process inputs and generate long CoTs in the target language. We find that scaling reasoning model size improves multilingual task performance in En-CoT, but Target-CoT performance lags behind. This gap widens for tasks requiring long, multi-step CoTs such as mathematical reasoning. Shifting to pretraining, we find that adding a specialized reasoning stage enhances En-CoT performance but degrades Target-CoT, whereas broad multilingual pretraining improves both modes simultaneously. Given the scarcity of high-quality reasoning traces in languages other than English, we explore synthetic data curation approaches for post-training. We demonstrate that fine-tuning on reasoning traces automatically translated from gold English traces outperforms fine-tuning on target-language traces distilled from large reasoning models. Finally, we report disparities in inference efficiency between languages and uncover language-specific failure modes in CoTs. We release models, datasets, and code to foster further research.

Method: Created ObjexMT benchmark where models must output a one-sentence base objective and self-reported confidence from multi-turn transcripts. Evaluation uses semantic similarity scoring thresholded on 300 calibration items, with metacognition assessed via expected calibration error, Brier score, Wrong@High-Confidence metrics, and risk-coverage curves.

Result: Across six models tested on three datasets, kimi-k2 achieved highest objective-extraction accuracy (0.612), while claude-sonnet-4 showed best selective risk and calibration. Performance varied widely (16-82% accuracy), and high-confidence errors were substantial (Wrong@0.90 ranged from 14.9% to 47.7%).

Conclusion: LLM judges often misinfer implicit objectives, making it advisable to expose objectives explicitly or gate decisions by confidence. ObjexMT provides an actionable test for evaluating LLM judge qualification in objective extraction tasks.

Abstract: LLM-as-a-Judge (LLMaaJ) enables scalable evaluation, yet we lack a decisive test of a judge’s qualification: can it recover the hidden objective of a conversation and know when that inference is reliable? Large language models degrade with irrelevant or lengthy context, and multi-turn jailbreaks can scatter goals across turns. We present ObjexMT, a benchmark for objective extraction and metacognition. Given a multi-turn transcript, a model must output a one-sentence base objective and a self-reported confidence. Accuracy is scored by semantic similarity to gold objectives, then thresholded once on 300 calibration items ($\tau^\star = 0.66$; $F_1@\tau^\star = 0.891$). Metacognition is assessed with expected calibration error, Brier score, Wrong@High-Confidence (0.80 / 0.90 / 0.95), and risk–coverage curves. Across six models (gpt-4.1, claude-sonnet-4, Qwen3-235B-A22B-FP8, kimi-k2, deepseek-v3.1, gemini-2.5-flash) evaluated on SafeMTData_Attack600, SafeMTData_1K, and MHJ, kimi-k2 achieves the highest objective-extraction accuracy (0.612; 95% CI [0.594, 0.630]), while claude-sonnet-4 (0.603) and deepseek-v3.1 (0.599) are statistically tied. claude-sonnet-4 offers the best selective risk and calibration (AURC 0.242; ECE 0.206; Brier 0.254). Performance varies sharply across datasets (16–82% accuracy), showing that automated obfuscation imposes challenges beyond model choice. High-confidence errors remain: Wrong@0.90 ranges from 14.9% (claude-sonnet-4) to 47.7% (Qwen3-235B-A22B-FP8). ObjexMT therefore supplies an actionable test for LLM judges: when objectives are implicit, judges often misinfer them; exposing objectives or gating decisions by confidence is advisable. All experimental data are in the Supplementary Material and at https://github.com/hyunjun1121/ObjexMT_dataset.

Method: Introduces Adaptive Originality Filtering (AOF) - a prompting strategy that enforces novelty and cultural fidelity via semantic rejection, and proposes RiddleScore metric combining novelty, diversity, fluency, and answer alignment.

Result: AOF improves Distinct-2 (0.915 in Japanese), reduces Self-BLEU (0.177), and raises RiddleScore (up to +57.1% in Arabic). Human evaluations confirm gains in fluency, creativity, and cultural fit, though improvements vary across languages.

Conclusion: Semantic filtering with composite evaluation offers a lightweight path to culturally rich generation without fine-tuning, and while focused on riddles, the method may apply to broader creative tasks.

Abstract: Language models are increasingly tested on multilingual creativity, demanding culturally grounded, abstract generations. Standard prompting methods often produce repetitive or shallow outputs. We introduce Adaptive Originality Filtering (AOF), a prompting strategy that enforces novelty and cultural fidelity via semantic rejection. To assess quality, we propose RiddleScore, a metric combining novelty, diversity, fluency, and answer alignment. AOF improves Distinct-2 (0.915 in Japanese), reduces Self-BLEU (0.177), and raises RiddleScore (up to +57.1% in Arabic). Human evaluations confirm fluency, creativity, and cultural fit gains. However, improvements vary: Arabic shows greater RiddleScore gains than Distinct-2; Japanese sees similar changes. Though focused on riddles, our method may apply to broader creative tasks. Overall, semantic filtering with composite evaluation offers a lightweight path to culturally rich generation without fine-tuning.

Method: Non-linear hashing functions optimize query/key embeddings, trained with Bradley-Terry ranking loss on 16GB GPUs in 8 hours, with specialized CUDA kernels for fast bitwise operations.

Result: 5x shorter hash codes, 512K token hashing in <100μs on A100, 3x higher end-to-end throughput, and improved retrieval precision.

Conclusion: Spotlight Attention efficiently reduces KV cache burden through optimized non-linear hashing, enabling faster LLM inference with maintained performance.

Abstract: Reducing the key-value (KV) cache burden in Large Language Models (LLMs) significantly accelerates inference. Dynamically selecting critical KV caches during decoding helps maintain performance. Existing methods use random linear hashing to identify important tokens, but this approach is inefficient due to the orthogonal distribution of queries and keys within two narrow cones in LLMs. We introduce Spotlight Attention, a novel method that employs non-linear hashing functions to optimize the embedding distribution of queries and keys, enhancing coding efficiency and robustness. We also developed a lightweight, stable training framework using a Bradley-Terry ranking-based loss, enabling optimization of the non-linear hashing module on GPUs with 16GB memory in 8 hours. Experimental results show that Spotlight Attention drastically improves retrieval precision while shortening the length of the hash code at least 5$\times$ compared to traditional linear hashing. Finally, we exploit the computational advantages of bitwise operations by implementing specialized CUDA kernels, achieving hashing retrieval for 512K tokens in under 100$\mu$s on a single A100 GPU, with end-to-end throughput up to 3$\times$ higher than vanilla decoding.

Method: A closed-loop optimization system with: (1) self-referential diagnostic module using tri-axial model signals (loss patterns, embedding clusters, self-alignment scores), (2) adaptive optimization engine that transforms suboptimal samples while preserving semantics, (3) continuous evolution with model capability through dynamic learning principles.

Result: Experiments show Middo consistently enhances seed data quality and boosts LLM performance with 7.15% average accuracy improvement while maintaining original dataset scale.

Conclusion: Establishes a new paradigm for sustainable LLM training through dynamic human-AI co-evolution of data and models.

Abstract: Supervised Fine-Tuning (SFT) Large Language Models (LLM) fundamentally rely on high-quality training data. While data selection and data synthesis are two common strategies to improve data quality, existing approaches often face limitations in static dataset curation that fail to adapt to evolving model capabilities. In this paper, we introduce Middo, a self-evolving Model-informed dynamic data optimization framework that uses model-aware data selection and context-preserving data refinement. Unlike conventional one-off filtering/synthesis methods, our framework establishes a closed-loop optimization system: (1) A self-referential diagnostic module proactively identifies suboptimal samples through tri-axial model signals - loss patterns (complexity), embedding cluster dynamics (diversity), and self-alignment scores (quality); (2) An adaptive optimization engine then transforms suboptimal samples into pedagogically valuable training points while preserving semantic integrity; (3) This optimization process continuously evolves with model capability through dynamic learning principles. Experiments on multiple benchmarks demonstrate that our Middo consistently enhances the quality of seed data and boosts LLM’s performance with improving accuracy by 7.15% on average while maintaining the original dataset scale. This work establishes a new paradigm for sustainable LLM training through dynamic human-AI co-evolution of data and models. Our datasets, models, and code are publicly available at https://github.com/Word2VecT/Middo.

Method: Developed an evaluation framework grounded in legal fact and reasoning inclusion, analyzing how LLMs respond to prompts conditioned on different legal roles when summarizing judicial decisions, including tests with balancing instructions.

Result: Models show selective inclusion patterns that reflect role-consistent perspectives even with balancing instructions, demonstrating systematic alignment with stakeholder positions in legal contexts.

Conclusion: The findings highlight the need for role-aware evaluation of LLM summarization behavior in legal settings, as models may infer user roles from context and exhibit motivated reasoning without explicit instructions, posing risks in high-stakes applications.

Abstract: Large Language Models (LLMs) are increasingly used to generate user-tailored summaries, adapting outputs to specific stakeholders. In legal contexts, this raises important questions about motivated reasoning – how models strategically frame information to align with a stakeholder’s position within the legal system. Building on theories of legal realism and recent trends in legal practice, we investigate how LLMs respond to prompts conditioned on different legal roles (e.g., judges, prosecutors, attorneys) when summarizing judicial decisions. We introduce an evaluation framework grounded in legal fact and reasoning inclusion, also considering favorability towards stakeholders. Our results show that even when prompts include balancing instructions, models exhibit selective inclusion patterns that reflect role-consistent perspectives. These findings raise broader concerns about how similar alignment may emerge as LLMs begin to infer user roles from prior interactions or context, even without explicit role instructions. Our results underscore the need for role-aware evaluation of LLM summarization behavior in high-stakes legal settings.

Method: Adds a new training stage before contrastive learning using bidirectional generative reconstruction tasks: EBQ2D (Embedding-Based Query-to-Document) and EBD2Q (Embedding-Based Document-to-Query), which interleave to anchor the [EOS] embedding and reconstruct Query-Document pairs.

Result: The additional training stage significantly improves LLM performance on the Massive Text Embedding Benchmark (MTEB), achieving new state-of-the-art results across different LLM base models and scales.

Conclusion: Enriching final token embeddings through bidirectional generative reconstruction tasks before contrastive learning effectively improves LLM-based text embedding performance for retrieval and re-ranking tasks.

Abstract: Large language models (LLMs) have increasingly been explored as powerful text embedders. Existing LLM-based text embedding approaches often leverage the embedding of the final token, typically a reserved special token such as [EOS]. However, these tokens have not been intentionally trained to capture the semantics of the whole context, limiting their capacity as text embeddings, especially for retrieval and re-ranking tasks. We propose to add a new training stage before contrastive learning to enrich the semantics of the final token embedding. This stage employs bidirectional generative reconstruction tasks, namely EBQ2D (Embedding-Based Query-to-Document) and EBD2Q (Embedding-Based Document-to-Query), which interleave to anchor the [EOS] embedding and reconstruct either side of Query-Document pairs. Experimental results demonstrate that our additional training stage significantly improves LLM performance on the Massive Text Embedding Benchmark (MTEB), achieving new state-of-the-art results across different LLM base models and scales.

Method: Automated framework using language-model-guided evolution with smart sampling from 12 sources and LLM-as-judge evaluation inspired by StrongREJECT.

Result: Achieved 44.8% overall success (103/230) on GPT-4.1, discovered two new template families, and found structural gains transfer across models but vary by target.

Conclusion: Structure-level search is a reproducible route to stronger single-turn probes, highlighting the importance of threshold calibration and cross-model evaluation.

Abstract: Multi-turn-to-single-turn (M2S) compresses iterative red-teaming into one structured prompt, but prior work relied on a handful of manually written templates. We present X-Teaming Evolutionary M2S, an automated framework that discovers and optimizes M2S templates through language-model-guided evolution. The system pairs smart sampling from 12 sources with an LLM-as-judge inspired by StrongREJECT and records fully auditable logs. Maintaining selection pressure by setting the success threshold to $\theta = 0.70$, we obtain five evolutionary generations, two new template families, and 44.8% overall success (103/230) on GPT-4.1. A balanced cross-model panel of 2,500 trials (judge fixed) shows that structural gains transfer but vary by target; two models score zero at the same threshold. We also find a positive coupling between prompt length and score, motivating length-aware judging. Our results demonstrate that structure-level search is a reproducible route to stronger single-turn probes and underscore the importance of threshold calibration and cross-model evaluation. Code, configurations, and artifacts are available at https://github.com/hyunjun1121/M2S-x-teaming.

Method: The authors examine research applying RL to LLMs and LRMs for reasoning abilities, analyzing foundational components, core problems, training resources, and downstream applications since DeepSeek-R1’s release.

Result: The survey identifies future opportunities and directions for RL in reasoning models, providing a comprehensive review of the rapidly evolving domain.

Conclusion: This review aims to promote future research on RL for broader reasoning models, addressing scalability challenges in computational resources, algorithm design, training data, and infrastructure.

Abstract: In this paper, we survey recent advances in Reinforcement Learning (RL) for reasoning with Large Language Models (LLMs). RL has achieved remarkable success in advancing the frontier of LLM capabilities, particularly in addressing complex logical tasks such as mathematics and coding. As a result, RL has emerged as a foundational methodology for transforming LLMs into LRMs. With the rapid progress of the field, further scaling of RL for LRMs now faces foundational challenges not only in computational resources but also in algorithm design, training data, and infrastructure. To this end, it is timely to revisit the development of this domain, reassess its trajectory, and explore strategies to enhance the scalability of RL toward Artificial SuperIntelligence (ASI). In particular, we examine research applying RL to LLMs and LRMs for reasoning abilities, especially since the release of DeepSeek-R1, including foundational components, core problems, training resources, and downstream applications, to identify future opportunities and directions for this rapidly evolving area. We hope this review will promote future research on RL for broader reasoning models. Github: https://github.com/TsinghuaC3I/Awesome-RL-for-LRMs

Method: Proposed HiCBench with manually annotated multi-level chunking points and synthesized evidence-dense QA pairs. Also introduced HiChunk framework using fine-tuned LLMs for multi-level document structuring combined with Auto-Merge retrieval algorithm.

Result: HiCBench effectively evaluates different chunking methods across the entire RAG pipeline. HiChunk achieves better chunking quality within reasonable time consumption, enhancing overall RAG system performance.

Conclusion: The proposed HiCBench benchmark and HiChunk framework successfully address document chunking evaluation challenges and improve RAG system performance through better chunking quality and retrieval algorithms.

Abstract: Retrieval-Augmented Generation (RAG) enhances the response capabilities of language models by integrating external knowledge sources. However, document chunking as an important part of RAG system often lacks effective evaluation tools. This paper first analyzes why existing RAG evaluation benchmarks are inadequate for assessing document chunking quality, specifically due to evidence sparsity. Based on this conclusion, we propose HiCBench, which includes manually annotated multi-level document chunking points, synthesized evidence-dense quetion answer(QA) pairs, and their corresponding evidence sources. Additionally, we introduce the HiChunk framework, a multi-level document structuring framework based on fine-tuned LLMs, combined with the Auto-Merge retrieval algorithm to improve retrieval quality. Experiments demonstrate that HiCBench effectively evaluates the impact of different chunking methods across the entire RAG pipeline. Moreover, HiChunk achieves better chunking quality within reasonable time consumption, thereby enhancing the overall performance of RAG systems.

Method: Student generates training trajectories, teacher corrects only the earliest error, producing data matched to student’s ability. Student is fine-tuned on corrected trajectories, then short-horizon reinforcement learning starts from verified prefix preceding the earliest error with target rewards at that step.

Result: On 12 challenging benchmarks, a 7B-parameter student distilled with SCoRe matches the agentic performance of a 72B-parameter teacher.

Conclusion: SCoRe enables effective knowledge distillation for LLM agents by addressing compounding error issues through student-centered training and targeted error correction, allowing smaller models to achieve performance comparable to much larger models.

Abstract: Large Language Model agents excel at solving complex tasks through iterative reasoning and tool use, but typically depend on ultra-large, costly backbones. Existing distillation approaches train smaller students to imitate full teacher trajectories, yet reasoning and knowledge gaps between the teacher and student can cause compounding errors. We propose SCoRe, a student-centered framework in which the student generates training trajectories and the teacher corrects only the earliest error, producing training data matched to the student’s ability and exposing specific weaknesses. The student is first fine-tuned on corrected trajectories. Subsequently, short-horizon reinforcement learning starts from the verified prefix preceding the earliest error, with target rewards assigned at that step. This design encourages autonomous problem-solving beyond imitation and enhances training stability. On 12 challenging benchmarks, a 7B-parameter student distilled with SCoRe matches the agentic performance of a 72B-parameter teacher.

Method: Proposes a DNA-inspired perspective using a repair-based process. DNA-DetectLLM constructs ideal AI-generated sequences, iteratively repairs non-optimal tokens, and quantifies cumulative repair effort as an interpretable detection signal.

Result: Achieves state-of-the-art detection performance with 5.55% AUROC and 2.08% F1 score improvements across multiple benchmark datasets. Shows strong robustness against adversarial attacks and varying input lengths.

Conclusion: The DNA-inspired repair approach provides an effective and interpretable method for AI-generated text detection, addressing the challenges posed by overlapping feature distributions between human and AI-generated content.

Abstract: The rapid advancement of large language models (LLMs) has blurred the line between AI-generated and human-written text. This progress brings societal risks such as misinformation, authorship ambiguity, and intellectual property concerns, highlighting the urgent need for reliable AI-generated text detection methods. However, recent advances in generative language modeling have resulted in significant overlap between the feature distributions of human-written and AI-generated text, blurring classification boundaries and making accurate detection increasingly challenging. To address the above challenges, we propose a DNA-inspired perspective, leveraging a repair-based process to directly and interpretably capture the intrinsic differences between human-written and AI-generated text. Building on this perspective, we introduce DNA-DetectLLM, a zero-shot detection method for distinguishing AI-generated and human-written text. The method constructs an ideal AI-generated sequence for each input, iteratively repairs non-optimal tokens, and quantifies the cumulative repair effort as an interpretable detection signal. Empirical evaluations demonstrate that our method achieves state-of-the-art detection performance and exhibits strong robustness against various adversarial attacks and input lengths. Specifically, DNA-DetectLLM achieves relative improvements of 5.55% in AUROC and 2.08% in F1 score across multiple public benchmark datasets. Code and data are available at https://github.com/Xiaoweizhu57/DNA-DetectLLM.

Method: Used human evaluation (N=38) and automated LLM-as-a-judge assessment (Gemini 2.0 Flash) to compare LLM-generated vs human-authored responses in multi-turn professional training simulations.

Result: Human evaluation showed significant degradation in LLM response quality across turns (naturalness, context maintenance, overall quality), while human responses improved. Participants consistently preferred human-authored dialogue. LLM-as-a-judge evaluation validated human judgements with strong alignment.

Conclusion: The study provides a multi-turn benchmark exposing LLM degradation in knowledge-grounded role-play dialogues and a validated hybrid evaluation framework for reliable LLM integration in training simulations.

Abstract: Evaluating large language models (LLMs) in long-form, knowledge-grounded role-play dialogues remains challenging. This study compares LLM-generated and human-authored responses in multi-turn professional training simulations through human evaluation ($N=38$) and automated LLM-as-a-judge assessment. Human evaluation revealed significant degradation in LLM-generated response quality across turns, particularly in naturalness, context maintenance and overall quality, while human-authored responses progressively improved. In line with this finding, participants also indicated a consistent preference for human-authored dialogue. These human judgements were validated by our automated LLM-as-a-judge evaluation, where Gemini 2.0 Flash achieved strong alignment with human evaluators on both zero-shot pairwise preference and stochastic 6-shot construct ratings, confirming the widening quality gap between LLM and human responses over time. Our work contributes a multi-turn benchmark exposing LLM degradation in knowledge-grounded role-play dialogues and provides a validated hybrid evaluation framework to guide the reliable integration of LLMs in training simulations.

Method: SubSpec constructs a highly aligned draft model by generating low-bit quantized substitute layers from offloaded target LLM portions. It shares remaining GPU-resident layers and KV-Cache to reduce memory overhead and enhance alignment, enabling speculative decoding without additional training.

Result: Achieved 9.1x speedup for Qwen2.5 7B on MT-Bench with 8GB VRAM limit, and average 12.5x speedup for Qwen2.5 32B on popular generation benchmarks with 24GB VRAM limit.

Conclusion: SubSpec provides a plug-and-play, lossless, and training-free solution that significantly accelerates parameter offloading for large language models while maintaining model quality through high alignment between draft and target models.

Abstract: The immense model sizes of large language models (LLMs) challenge deployment on memory-limited consumer GPUs. Although model compression and parameter offloading are common strategies to address memory limitations, compression can degrade quality, and offloading maintains quality but suffers from slow inference. Speculative decoding presents a promising avenue to accelerate parameter offloading, utilizing a fast draft model to propose multiple draft tokens, which are then verified by the target LLM in parallel with a single forward pass. This method reduces the time-consuming data transfers in forward passes that involve offloaded weight transfers. Existing methods often rely on pretrained weights of the same family, but require additional training to align with custom-trained models. Moreover, approaches that involve draft model training usually yield only modest speedups. This limitation arises from insufficient alignment with the target model, preventing higher token acceptance lengths. To address these challenges and achieve greater speedups, we propose SubSpec, a plug-and-play method to accelerate parameter offloading that is lossless and training-free. SubSpec constructs a highly aligned draft model by generating low-bit quantized substitute layers from offloaded target LLM portions. Additionally, our method shares the remaining GPU-resident layers and the KV-Cache, further reducing memory overhead and enhance alignment. SubSpec achieves a high average acceptance length, delivering 9.1x speedup for Qwen2.5 7B on MT-Bench (8GB VRAM limit) and an average of 12.5x speedup for Qwen2.5 32B on popular generation benchmarks (24GB VRAM limit).

Method: QoNext identifies experiential factors, conducts controlled experiments with human ratings under varied configurations, builds a QoE-oriented database, and trains predictive models to estimate user experience from system parameters.

Result: QoNext enables proactive and fine-grained evaluation of foundation models and provides actionable guidance for optimizing foundation models in practical services.

Conclusion: The QoNext framework successfully bridges the gap in foundation model evaluation by focusing on user experience, offering a more comprehensive assessment approach that accounts for interaction quality alongside response quality.

Abstract: Existing evaluations of foundation models, including recent human-centric approaches, fail to capture what truly matters: user’s experience during interaction. Current methods treat evaluation as a matter of output correctness alone, overlooking that user satisfaction emerges from the interplay between response quality and interaction, which limits their ability to account for the mechanisms underlying user experience. To address this gap, we introduce QoNext, the first framework that adapts Quality of Experience (QoE) principles from networking and multimedia to the assessment of foundation models. QoNext identifies experiential factors that shape user experience and incorporates them into controlled experiments, where human ratings are collected under varied configurations. From these studies we construct a QoE-oriented database and train predictive models that estimate perceived user experience from measurable system parameters. Our results demonstrate that QoNext not only enables proactive and fine-grained evaluation but also provides actionable guidance for productized services of optimizing foundation models in practice.

Method: Uses a fine-grained, hybrid-granularity quantization strategy for end-to-end FP8 training that integrates continual pre-training and supervised fine-tuning while maintaining numerical fidelity.

Result: The FP8 recipe is remarkably stable and essentially lossless, achieving performance on par with BF16 baseline across reasoning benchmarks, with 22% reduction in training time, 14% decrease in peak memory usage, and 19% increase in throughput.

Conclusion: FP8 is established as a practical and robust alternative to BF16 for large-scale model training, with code release to democratize access to efficient LLM training.

Abstract: The immense computational cost of training Large Language Models (LLMs) presents a major barrier to innovation. While FP8 training offers a promising solution with significant theoretical efficiency gains, its widespread adoption has been hindered by the lack of a comprehensive, open-source training recipe. To bridge this gap, we introduce an end-to-end FP8 training recipe that seamlessly integrates continual pre-training and supervised fine-tuning. Our methodology employs a fine-grained, hybrid-granularity quantization strategy to maintain numerical fidelity while maximizing computational efficiency. Through extensive experiments, including the continue pre-training of models on a 160B-token corpus, we demonstrate that our recipe is not only remarkably stable but also essentially lossless, achieving performance on par with the BF16 baseline across a suite of reasoning benchmarks. Crucially, this is achieved with substantial efficiency improvements, including up to a 22% reduction in training time, a 14% decrease in peak memory usage, and a 19% increase in throughput. Our results establish FP8 as a practical and robust alternative to BF16, and we will release the accompanying code to further democratize large-scale model training.

Method: PARL-MT combines Progress Awareness Generation (PAG) pipeline for automatic dataset construction and Progress Awareness-Guided Reinforcement Learning (PAG-RL) algorithm that integrates progress awareness into RL training to reduce redundancy and improve action-task alignment.

Result: Empirical results on two public benchmarks show that PARL-MT significantly outperforms existing methods, demonstrating the effectiveness of progress awareness in multi-turn function calling.

Conclusion: Progress awareness is crucial for enabling robust and efficient multi-turn function calling in LLMs, and PARL-MT provides an effective framework for incorporating this capability.

Abstract: Large language models (LLMs) have achieved impressive success in single-turn function calling, yet real-world applications such as travel planning or multi-stage data analysis typically unfold across multi-turn conversations. In these settings, LLMs must not only issue accurate function calls at each step but also maintain progress awareness, the ability to summarize past interactions and plan future actions to ensure coherent, long-horizon task execution. Existing approaches, however, either reduce multi-turn training to isolated single-turn samples, which neglects task-level planning, or employ end-to-end reinforcement learning (RL) that struggles with redundancy and lacks explicit integration of progress awareness. To overcome these limitations, we introduce PARL-MT, a framework that explicitly incorporates progress awareness into LLM training for multi-turn function calling. PARL-MT combines (i) a Progress Awareness Generation (PAG) pipeline, which automatically constructs datasets coupling conversation summaries with future task planning, and (ii) a Progress Awareness-Guided Reinforcement Learning (PAG-RL) algorithm, which integrates progress awareness into RL training to reduce contextual redundancy and improve alignment between local actions and global task completion. Empirical results on two public benchmarks demonstrate that PARL-MT significantly outperforms existing methods, highlighting the effectiveness of progress awareness in enabling robust and efficient multi-turn function calling.

Method: Introduces KG-R1, an agentic KG-RAG framework using reinforcement learning where a single agent interacts with KGs as its environment, learning to retrieve information at each step and incorporating it into reasoning and generation through end-to-end RL optimization.

Result: In KGQA benchmarks, KG-R1 with Qwen-2.5-3B achieved higher answer accuracy with fewer generation tokens than prior multi-module methods using larger models, and demonstrated plug-and-play capability by maintaining strong accuracy on new KGs without modification.

Conclusion: KG-R1 is a promising KG-RAG framework for real-world deployment due to its efficiency, transferability, and plug-and-play capabilities.

Abstract: Knowledge-graph retrieval-augmented generation (KG-RAG) couples large language models (LLMs) with structured, verifiable knowledge graphs (KGs) to reduce hallucinations and expose reasoning traces. However, many KG-RAG systems compose multiple LLM modules (e.g planning, reasoning, and responding), inflating inference cost and binding behavior to a specific target KG. To address this, we introduce KG-R1, an agentic KG retrieval-augmented generation (KG-RAG) framework through reinforcement learning (RL). KG-R1 utilizes a single agent that interacts with KGs as its environment, learning to retrieve at each step and incorporating the retrieved information into its reasoning and generation. The process is optimized through end-to-end RL. In controlled experiments across Knowledge-Graph Question Answering (KGQA) benchmarks, our method demonstrates both efficiency and transferability: Using Qwen-2.5-3B, KG-R1 improves answer accuracy with fewer generation tokens than prior multi-module workflow methods that use larger foundation or fine-tuned models. Furthermore, KG-R1 enables plug and play: after training, it maintains strong accuracy on new KGs without modification. These properties make KG-R1 a promising KG-RAG framework for real-world deployment. Our code is publicly available at https://github.com/Jinyeop3110/KG-R1.

Method: Extracted and linked evidence from text, tables, and figures from PDF pages, then generated multimodal QA pairs spanning factual retrieval, comparison, summarization, and logical reasoning queries with expert validation.

Result: Multimodal text-image fusion RAG systems consistently outperform unimodal and jointly multimodal embedding-based retrieval, showing neither text nor images alone are sufficient and current multimodal embeddings remain inadequate.

Conclusion: The benchmark enables apples-to-apples comparison across retrieval paradigms, reveals when visual context complements text, uncovers systematic failure modes, and provides guidance for developing more robust MM-RAG pipelines.

Abstract: Multimodal retrieval-augmented generation (MM-RAG) is a key approach for applying large language models (LLMs) and agents to real-world knowledge bases, yet current evaluations are fragmented, focusing on either text or images in isolation or on simplified multimodal setups that fail to capture document-centric multimodal use cases. In this paper, we introduce UniDoc-Bench, the first large-scale, realistic benchmark for MM-RAG built from 70k real-world PDF pages across eight domains. Our pipeline extracts and links evidence from text, tables, and figures, then generates 1,600 multimodal QA pairs spanning factual retrieval, comparison, summarization, and logical reasoning queries. To ensure reliability, 20% of QA pairs are validated by multiple annotators and expert adjudication. UniDoc-Bench supports apples-to-apples comparison across four paradigms: (1) text-only, (2) image-only, (3) multimodal text-image fusion, and (4) multimodal joint retrieval – under a unified protocol with standardized candidate pools, prompts, and evaluation metrics. Our experiments show that multimodal text-image fusion RAG systems consistently outperform both unimodal and jointly multimodal embedding-based retrieval, indicating that neither text nor images alone are sufficient and that current multimodal embeddings remain inadequate. Beyond benchmarking, our analysis reveals when and how visual context complements textual evidence, uncovers systematic failure modes, and offers actionable guidance for developing more robust MM-RAG pipelines.

Method: Proposed RL4HS framework using reinforcement learning with span-level reward function, building on Group Relative Policy Optimization and introducing Class-Aware Policy Optimization to address reward imbalance issues.

Result: Experiments on RAGTruth benchmark across summarization, question answering, and data-to-text tasks show RL4HS surpasses pretrained reasoning models and supervised fine-tuning.

Conclusion: Reinforcement learning with span-level rewards is necessary for effectively detecting hallucination spans in LLM outputs.

Abstract: Large language models (LLMs) often generate hallucinations – unsupported content that undermines reliability. While most prior works frame hallucination detection as a binary task, many real-world applications require identifying hallucinated spans, which is a multi-step decision making process. This naturally raises the question of whether explicit reasoning can help the complex task of detecting hallucination spans. To answer this question, we first evaluate pretrained models with and without Chain-of-Thought (CoT) reasoning, and show that CoT reasoning has the potential to generate at least one correct answer when sampled multiple times. Motivated by this, we propose RL4HS, a reinforcement learning framework that incentivizes reasoning with a span-level reward function. RL4HS builds on Group Relative Policy Optimization and introduces Class-Aware Policy Optimization to mitigate reward imbalance issue. Experiments on the RAGTruth benchmark (summarization, question answering, data-to-text) show that RL4HS surpasses pretrained reasoning models and supervised fine-tuning, demonstrating the necessity of reinforcement learning with span-level rewards for detecting hallucination spans.

Method: AMPO adaptively leverages multiple teacher models only when the on-policy model fails, using a ‘guidance-on-demand’ approach with comprehension-based selection to learn from most understandable reasoning paths.

Result: AMPO outperforms GRPO baseline by 4.3% on mathematical reasoning and 12.2% on out-of-distribution tasks, significantly boosts Pass@k performance, and enables more diverse exploration with comparable results to single powerful teacher approaches.

Conclusion: AMPO provides a more efficient and scalable path to superior reasoning and generalizability in LLMs through adaptive multi-teacher guidance that balances exploration and exploitation.

Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) is a promising paradigm for enhancing the reasoning ability in Large Language Models (LLMs). However, prevailing methods primarily rely on self-exploration or a single off-policy teacher to elicit long chain-of-thought (LongCoT) reasoning, which may introduce intrinsic model biases and restrict exploration, ultimately limiting reasoning diversity and performance. Drawing inspiration from multi-teacher strategies in knowledge distillation, we introduce Adaptive Multi-Guidance Policy Optimization (AMPO), a novel framework that adaptively leverages guidance from multiple proficient teacher models, but only when the on-policy model fails to generate correct solutions. This “guidance-on-demand” approach expands exploration while preserving the value of self-discovery. Moreover, AMPO incorporates a comprehension-based selection mechanism, prompting the student to learn from the reasoning paths that it is most likely to comprehend, thus balancing broad exploration with effective exploitation. Extensive experiments show AMPO substantially outperforms a strong baseline (GRPO), with a 4.3% improvement on mathematical reasoning tasks and 12.2% on out-of-distribution tasks, while significantly boosting Pass@k performance and enabling more diverse exploration. Notably, using four peer-sized teachers, our method achieves comparable results to approaches that leverage a single, more powerful teacher (e.g., DeepSeek-R1) with more data. These results demonstrate a more efficient and scalable path to superior reasoning and generalizability. Our code is available at https://github.com/SII-Enigma/AMPO.

Method: Proposed Sub-CP framework leverages submodular objectives to control block diversity, supporting flexible selection strategies where each block can range from globally diverse to locally coherent, enabling fine-grained control over semantic structure.

Result: Extensive experiments across diverse tasks on multiple datasets show that Sub-CP consistently improves performance across different model scales.

Conclusion: Sub-CP provides an effective solution for optimizing context selection in in-context learning by addressing information redundancy and under-representation issues through controlled block diversity.

Abstract: In-context learning (ICL) enables efficient few-shot learning in large language models (LLMs) without training, but suffers from the quadratic input complexity of transformers, limiting the maximum number of exemplars. While various efficient ICL approaches partition the context into blocks to process (e.g., ensembling, compression, cross-attention), they often ignore the information redundancy or under-representation caused by different partition strategies, leading to suboptimal performance. To tackle this problem, we propose Sub-CP, a block-aware context selection framework that leverages submodular objectives to control block diversity. Sub-CP supports a flexible spectrum of selection strategies, allowing each block to range from globally diverse to locally coherent. This allows fine-grained control over semantic structure while enabling precomputation. Extensive experiments across diverse tasks on multiple datasets show that Sub-CP consistently improves performance across model scales.

Method: A strictly causal thinking approach that incrementally reasons while listening, using only past audio without lookahead. Reasoning is amortized during listening windows with no added latency when switching to speaking.

Result: Experiments show consistent improvements in response quality through both objective metrics and human evaluations. The method also handles conversational dynamics effectively and achieves competitive full-duplex interaction performance.

Conclusion: Chronological Thinking successfully bridges the gap between human conversational behavior and current full-duplex systems by enabling continuous, real-time reasoning during listening phases while maintaining low latency.

Abstract: Recent advances in spoken dialogue language models (SDLMs) reflect growing interest in shifting from turn-based to full-duplex systems, where the models continuously perceive user speech streams while generating responses. This simultaneous listening and speaking design enables real-time interaction and the agent can handle dynamic conversational behaviors like user barge-in. However, during the listening phase, existing systems keep the agent idle by repeatedly predicting the silence token, which departs from human behavior: we usually engage in lightweight thinking during conversation rather than remaining absent-minded. Inspired by this, we propose Chronological Thinking, a on-the-fly conversational thinking mechanism that aims to improve response quality in full-duplex SDLMs. Specifically, chronological thinking presents a paradigm shift from conventional LLM thinking approaches, such as Chain-of-Thought, purpose-built for streaming acoustic input. (1) Strictly causal: the agent reasons incrementally while listening, updating internal hypotheses only from past audio with no lookahead. (2) No additional latency: reasoning is amortized during the listening window; once the user stops speaking, the agent halts thinking and begins speaking without further delay. Experiments demonstrate the effectiveness of chronological thinking through both objective metrics and human evaluations show consistent improvements in response quality. Furthermore, chronological thinking robustly handles conversational dynamics and attains competitive performance on full-duplex interaction metrics.

Method: Created SimulatorArena benchmark with 909 annotated human-LLM conversations on math tutoring and document creation tasks, evaluating simulators on message matching and rating alignment.

Result: Profile-conditioned simulators achieved Spearman’s ρ of 0.7 on both tasks, closely aligning with human judgments, and were used to benchmark 18 assistants including latest LLMs.

Conclusion: Profile-conditioned simulators provide practical, scalable alternative to human evaluation for assessing LLM performance in interactive applications.

Abstract: Large language models (LLMs) are increasingly used in interactive applications, and human evaluation remains the gold standard for assessing their performance in multi-turn conversations. Since human studies are costly, time-consuming, and hard to reproduce, recent work explores using LLMs to simulate users for automatic assistant evaluation. However, there is no benchmark or systematic study to evaluate whether these simulated users are reliable stand-ins for real users. To address this, we introduce SimulatorArena, a benchmark of 909 annotated human-LLM conversations on two interactive tasks – math tutoring and document creation. SimulatorArena evaluates simulators based on how closely their messages match human behavior and how well their assistant ratings align with human judgments. Experiments on various simulator methods show that simulators conditioned on user profiles, capturing traits like background and message styles, align closely with human judgments. They reach Spearman’s $\rho$ of 0.7 on both tasks, providing a practical, scalable alternative to human evaluation. Using the best simulator for each task, we benchmark 18 assistants, including the latest LLMs such as GPT-5, Claude 4.1 Opus, and Gemini 2.5 Pro.

Method: Used continual pre-training with large-scale unlabeled business conversation data to adapt LLMs for conversational summarization tasks.

Result: Continual pre-training achieved substantial gains in both in-domain and out-of-domain summarization benchmarks while maintaining generalization and robustness.

Conclusion: Continual pre-training is an effective self-supervised approach for adapting LLMs to specialized summarization tasks, with practical guidelines for industrial applications.

Abstract: Large language models (LLMs) have achieved impressive performance in text summarization, yet their performance often falls short when applied to specialized domains that differ from their original pre-training distribution. While fine-tuning can improve summarization quality, it typically relies on costly and scarce high-quality labeled data. In this work, we explore continual pre-training as a scalable, self-supervised approach to adapt LLMs for downstream summarization tasks, particularly in the context of noisy real-world conversation transcripts. We conduct extensive experiments using large-scale, unlabeled business conversation data to investigate whether continual pre-training enhances model capabilities in conversational summarization. Our results demonstrate that continual pre-training yields substantial gains in both in-domain and out-of-domain summarization benchmarks, while maintaining strong generalization and robustness. We also analyze the effects of data selection strategies, providing practical guidelines for applying continual pre-training in summarization-focused industrial applications.

Method: Proposes Distributional Semantics Tracing (DST) - a unified framework integrating interpretability techniques to create causal maps of model reasoning. Uses dual-process theory to analyze conflicts between fast associative pathways (System 1) and slow contextual pathways (System 2). Identifies a specific ‘commitment layer’ where hallucinations become irreversible.

Result: Found strong negative correlation (ρ = -0.863) between contextual pathway coherence and hallucination rates. Identified predictable failure modes like ‘Reasoning Shortcut Hijacks’ where the fast associative pathway overrides the slower contextual reasoning.

Conclusion: Hallucinations are predictable consequences of internal semantic weaknesses in Transformer architecture, occurring due to conflicts between competing computational pathways. The framework provides a mechanistic account of how, when, and why hallucinations occur.

Abstract: Large Language Models (LLMs) are prone to hallucination, the generation of plausible yet factually incorrect statements. This work investigates the intrinsic, architectural origins of this failure mode through three primary contributions. First, to enable the reliable tracing of internal semantic failures, we propose Distributional Semantics Tracing (DST), a unified framework that integrates established interpretability techniques to produce a causal map of a model’s reasoning, treating meaning as a function of context (distributional semantics). Second, we pinpoint the model’s layer at which a hallucination becomes inevitable, identifying a specific commitment layer where a model’s internal representations irreversibly diverge from factuality. Third, we identify the underlying mechanism for these failures. We observe a conflict between distinct computational pathways, which we interpret using the lens of dual-process theory: a fast, heuristic associative pathway (akin to System 1) and a slow, deliberate, contextual pathway (akin to System 2), leading to predictable failure modes such as Reasoning Shortcut Hijacks. Our framework’s ability to quantify the coherence of the contextual pathway reveals a strong negative correlation ($\rho = -0.863$) with hallucination rates, implying that these failures are predictable consequences of internal semantic weakness. The result is a mechanistic account of how, when, and why hallucinations occur within the Transformer architecture.

Method: The paper presents taxonomies of hallucination types and analyzes root causes across the LLM development lifecycle. It introduces structured taxonomies for detection approaches and mitigation strategies, and reviews evaluation benchmarks and metrics.

Result: The survey provides a comprehensive framework for understanding, detecting, and mitigating hallucinations in LLMs, analyzing strengths and limitations of current approaches.

Conclusion: The paper outlines key open challenges and promising future research directions to develop more truthful and trustworthy LLMs, establishing a foundation for addressing hallucination issues.

Abstract: Large language models (LLMs) have transformed natural language processing, achieving remarkable performance across diverse tasks. However, their impressive fluency often comes at the cost of producing false or fabricated information, a phenomenon known as hallucination. Hallucination refers to the generation of content by an LLM that is fluent and syntactically correct but factually inaccurate or unsupported by external evidence. Hallucinations undermine the reliability and trustworthiness of LLMs, especially in domains requiring factual accuracy. This survey provides a comprehensive review of research on hallucination in LLMs, with a focus on causes, detection, and mitigation. We first present a taxonomy of hallucination types and analyze their root causes across the entire LLM development lifecycle, from data collection and architecture design to inference. We further examine how hallucinations emerge in key natural language generation tasks. Building on this foundation, we introduce a structured taxonomy of detection approaches and another taxonomy of mitigation strategies. We also analyze the strengths and limitations of current detection and mitigation approaches and review existing evaluation benchmarks and metrics used to quantify LLMs hallucinations. Finally, we outline key open challenges and promising directions for future research, providing a foundation for the development of more truthful and trustworthy LLMs.

Method: Combines fine-tuned Mixtral-8x7B model for continuous note generation with DeBERTa-based classifier to filter trivial content. Uses agent edits to refine online notes and inform offline model retraining, creating a feedback loop.

Result: Achieved 3% reduction in case handling time compared to bulk summarization (up to 9% reduction in highly complex cases) with high agent satisfaction ratings in production deployment.

Conclusion: Incremental summarization with continuous feedback effectively enhances summary quality and agent productivity at scale in customer support environments.

Abstract: We introduce an incremental summarization system for customer support agents that intelligently determines when to generate concise bullet notes during conversations, reducing agents’ context-switching effort and redundant review. Our approach combines a fine-tuned Mixtral-8x7B model for continuous note generation with a DeBERTa-based classifier to filter trivial content. Agent edits refine the online notes generation and regularly inform offline model retraining, closing the agent edits feedback loop. Deployed in production, our system achieved a 3% reduction in case handling time compared to bulk summarization (with reductions of up to 9% in highly complex cases), alongside high agent satisfaction ratings from surveys. These results demonstrate that incremental summarization with continuous feedback effectively enhances summary quality and agent productivity at scale.

Method: Systematic study using miniGPTKBs (domain-specific subcrawls) with four variations (seed, language, randomness, model) across three domains (history, entertainment, finance), analyzing termination, reproducibility, and robustness.

Result: High termination rates (model-dependent), mixed reproducibility, and robustness that varies by perturbation type: high for seeds and temperature, lower for languages and models.

Conclusion: LLM knowledge materialization can reliably surface core knowledge but reveals important limitations, suggesting the approach works for extracting fundamental knowledge while highlighting areas needing improvement.

Abstract: Large Language Models (LLMs) encode substantial factual knowledge, yet measuring and systematizing this knowledge remains challenging. Converting it into structured format, for example through recursive extraction approaches such as the GPTKB methodology (Hu et al., 2025b), is still underexplored. Key open questions include whether such extraction can terminate, whether its outputs are reproducible, and how robust they are to variations. We systematically study LLM knowledge materialization using miniGPTKBs (domain-specific, tractable subcrawls), analyzing termination, reproducibility, and robustness across three categories of metrics: yield, lexical similarity, and semantic similarity. We experiment with four variations (seed, language, randomness, model) and three illustrative domains (from history, entertainment, and finance). Our findings show (i) high termination rates, though model-dependent; (ii) mixed reproducibility; and (iii) robustness that varies by perturbation type: high for seeds and temperature, lower for languages and models. These results suggest that LLM knowledge materialization can reliably surface core knowledge, while also revealing important limitations.

Method: Multi-agent architecture with ToolMaker (generates tool interfaces), AutoAgent (produces think-act-observe sequences), and ToolActor (simulates responses). Training involves Stage-1 SFT for trace grammar and Stage-2 GRPO with composite trace reward.

Result: Competitive Exact Match scores on four multi-hop QA benchmarks (HotpotQA, MuSiQue, 2WikiMultiHopQA, Bamboogle) compared to live-API systems, with strong performance on reasoning-intensive tasks.

Conclusion: Effective tool reasoning can be learned from structured traces without live interactions, addressing scalability and reliability issues of API-dependent systems.

Abstract: Tool-augmented language models have demonstrated strong capabilities, but their reliance on live API access creates scalability and reliability challenges during training and deployment. We propose MTR, a simulation-first training framework for tool-augmented reasoning. Instead of relying on live APIs, MTR learns from complete ReAct traces with schema-validated, simulated observations. Our approach operates through a multi-agent architecture where a ToolMaker generates task-specific, OpenAI-compatible tool interfaces, an AutoAgent produces structured think-act-observe sequences, and a ToolActor simulates realistic responses. Training proceeds in two stages: Stage-1 Supervised Fine-Tuning (SFT) teaches ’trace grammar’ from complete reasoning sequences; Stage-2 Group Relative Policy Optimization (GRPO) optimizes strategy with a composite trace reward that balances answer correctness and internal consistency. Across four multi-hop QA benchmarks (HotpotQA, MuSiQue, 2WikiMultiHopQA, Bamboogle), MTR attains competitive Exact Match (EM) scores to live-API systems and excels on reasoning-intensive tasks, suggesting that effective tool reasoning can be learned from structured traces without live interactions.

Method: The authors unify existing frameworks and introduce a learnable parameter λ that adaptively controls token-level weighting during optimization, allowing the model to learn its own token preferences.

Result: λ-GRPO improves average accuracy by +1.9%, +1.0%, and +1.7% compared to GRPO on Qwen2.5 models with 1.5B, 3B, and 7B parameters respectively across multiple mathematical reasoning benchmarks.

Conclusion: Learning token preferences through adaptive weighting is an effective and practical approach that improves reasoning capabilities without requiring training data modifications or additional computational resources.

Abstract: Reinforcement Learning with Human Feedback (RLHF) has been the dominant approach for improving the reasoning capabilities of Large Language Models (LLMs). Recently, Reinforcement Learning with Verifiable Rewards (RLVR) has simplified this paradigm by replacing the reward and value models with rule-based verifiers. A prominent example is Group Relative Policy Optimization (GRPO). However, GRPO inherently suffers from a length bias, since the same advantage is uniformly assigned to all tokens of a response. As a result, longer responses distribute the reward over more tokens and thus contribute disproportionately to gradient updates. Several variants, such as DAPO and Dr. GRPO, modify the token-level aggregation of the loss, yet these methods remain heuristic and offer limited interpretability regarding their implicit token preferences. In this work, we explore the possibility of allowing the model to learn its own token preference during optimization. We unify existing frameworks under a single formulation and introduce a learnable parameter $\lambda$ that adaptively controls token-level weighting. We use $\lambda$-GRPO to denote our method, and we find that $\lambda$-GRPO achieves consistent improvements over vanilla GRPO and DAPO on multiple mathematical reasoning benchmarks. On Qwen2.5 models with 1.5B, 3B, and 7B parameters, $\lambda$-GRPO improves average accuracy by $+1.9%$, $+1.0%$, and $+1.7%$ compared to GRPO, respectively. Importantly, these gains come without any modifications to the training data or additional computational cost, highlighting the effectiveness and practicality of learning token preferences.

Method: Used GPTKB v1.5, a recursively elicited set of 100 million beliefs from GPT-4.1, to conduct comprehensive analysis of the model’s factual knowledge.

Result: GPT-4.1’s factual knowledge significantly differs from established knowledge bases, has lower accuracy than previous benchmarks suggested, and suffers from major issues with inconsistency, ambiguity, and hallucinations.

Conclusion: The findings reveal substantial challenges in LLM factual knowledge and highlight important research opportunities for improving factual accuracy and reliability in large language models.

Abstract: LLMs are remarkable artifacts that have revolutionized a range of NLP and AI tasks. A significant contributor is their factual knowledge, which, to date, remains poorly understood, and is usually analyzed from biased samples. In this paper, we take a deep tour into the factual knowledge (or beliefs) of a frontier LLM, based on GPTKB v1.5 (Hu et al., 2025a), a recursively elicited set of 100 million beliefs of one of the strongest currently available frontier LLMs, GPT-4.1. We find that the models’ factual knowledge differs quite significantly from established knowledge bases, and that its accuracy is significantly lower than indicated by previous benchmarks. We also find that inconsistency, ambiguity and hallucinations are major issues, shedding light on future research opportunities concerning factual LLM knowledge.

Method: Constructed benchmark from causal relationships identified in top economics/finance journals using rigorous methods (instrumental variables, difference-in-differences, regression discontinuity), covering 40,379 items across 5 task types in multiple domains.

Result: Best model achieved only 57.6% accuracy; model scale doesn’t consistently improve performance; even advanced reasoning models struggle with basic causal relationship identification.

Conclusion: There’s a critical gap between current LLM capabilities and the demands of reliable causal reasoning required for high-stakes real-world applications.

Abstract: Causal reasoning is fundamental for Large Language Models (LLMs) to understand genuine cause-and-effect relationships beyond pattern matching. Existing benchmarks suffer from critical limitations such as reliance on synthetic data and narrow domain coverage. We introduce a novel benchmark constructed from casually identified relationships extracted from top-tier economics and finance journals, drawing on rigorous methodologies including instrumental variables, difference-in-differences, and regression discontinuity designs. Our benchmark comprises 40,379 evaluation items covering five task types across domains such as health, environment, technology, law, and culture. Experimental results on eight state-of-the-art LLMs reveal substantial limitations, with the best model achieving only 57.6% accuracy. Moreover, model scale does not consistently translate to superior performance, and even advanced reasoning models struggle with fundamental causal relationship identification. These findings underscore a critical gap between current LLM capabilities and demands of reliable causal reasoning in high-stakes applications.

Method: The survey classifies recent advances by architecture, training strategy, and evaluation methodology, analyzing how LLMs have reshaped CSW modeling through comprehensive review of 308 studies spanning 5 research areas, 12 NLP tasks, 30+ datasets, and 80+ languages.

Result: The analysis outlines how LLMs have reshaped CSW modeling and identifies persistent challenges in multilingual NLP, providing a comprehensive overview of the current state of CSW-aware LLM research.

Conclusion: The paper concludes with a roadmap emphasizing the need for inclusive datasets, fair evaluation, and linguistically grounded models to achieve truly multilingual intelligence, with all resources maintained at a curated GitHub repository.

Abstract: Code-switching (CSW), the alternation of languages and scripts within a single utterance, remains a fundamental challenge for multiling ual NLP, even amidst the rapid advances of large language models (LLMs). Most LLMs still struggle with mixed-language inputs, limited CSW datasets, and evaluation biases, hindering deployment in multilingual societies. This survey provides the first comprehensive analysis of CSW-aware LLM research, reviewing 308 studies spanning five research areas, 12 NLP tasks, 30+ datasets, and 80+ languages. We classify recent advances by architecture, training strategy, and evaluation methodology, outlining how LLMs have reshaped CSW modeling and what challenges persist. The paper concludes with a roadmap emphasizing the need for inclusive datasets, fair evaluation, and linguistically grounded models to achieve truly multilingual intelligence. A curated collection of all resources is maintained at https://github.com/lingo-iitgn/awesome-code-mixing/.

Method: HERO integrates verifier signals with reward-model scores through stratified normalization (bounding reward scores within verifier groups) and variance-aware weighting (emphasizing challenging prompts where dense signals matter most).

Result: HERO consistently outperforms RM-only and verifier-only baselines across diverse mathematical reasoning benchmarks, with strong gains on both verifiable and hard-to-verify tasks.

Conclusion: Hybrid reward design retains verifier stability while leveraging reward model nuance to advance reasoning capabilities in large language models.

Abstract: Post-training for reasoning of large language models (LLMs) increasingly relies on verifiable rewards: deterministic checkers that provide 0-1 correctness signals. While reliable, such binary feedback is brittle–many tasks admit partially correct or alternative answers that verifiers under-credit, and the resulting all-or-nothing supervision limits learning. Reward models offer richer, continuous feedback, which can serve as a complementary supervisory signal to verifiers. We introduce HERO (Hybrid Ensemble Reward Optimization), a reinforcement learning framework that integrates verifier signals with reward-model scores in a structured way. HERO employs stratified normalization to bound reward-model scores within verifier-defined groups, preserving correctness while refining quality distinctions, and variance-aware weighting to emphasize challenging prompts where dense signals matter most. Across diverse mathematical reasoning benchmarks, HERO consistently outperforms RM-only and verifier-only baselines, with strong gains on both verifiable and hard-to-verify tasks. Our results show that hybrid reward design retains the stability of verifiers while leveraging the nuance of reward models to advance reasoning.

Method: Mechanistic analysis of moral self-correction through multi-round interactions, examining how self-correction instructions activate moral concepts and reduce model uncertainty.

Result: Intrinsic self-correction exhibits performance convergence - consistently injected instructions activate moral concepts that stabilize over rounds, leading to converged performance as model uncertainty decreases.

Conclusion: Moral self-correction demonstrates strong potential with the desirable property of converged performance, showing that activated moral concepts drive the convergence behavior in LLM self-correction.

Abstract: Large Language Models (LLMs) are able to improve their responses when instructed to do so, a capability known as self-correction. When instructions provide only a general and abstract goal without specific details about potential issues in the response, LLMs must rely on their internal knowledge to improve response quality, a process referred to as intrinsic self-correction. The empirical success of intrinsic self-correction is evident in various applications, but how and why it is effective remains unknown. Focusing on moral self-correction in LLMs, we reveal a key characteristic of intrinsic self-correction: performance convergence through multi-round interactions; and provide a mechanistic analysis of this convergence behavior. Based on our experimental results and analysis, we uncover the underlying mechanism of convergence: consistently injected self-correction instructions activate moral concepts that reduce model uncertainty, leading to converged performance as the activated moral concepts stabilize over successive rounds. This paper demonstrates the strong potential of moral self-correction by showing that it exhibits a desirable property of converged performance.

Method: Created CORGI benchmark with synthetic databases inspired by real enterprises (Doordash, Airbnb, Lululemon) and questions across four complexity levels: descriptive, explanatory, predictive, and recommendational.

Result: LLM performance drops significantly on high-level questions, struggling with accurate predictions and actionable plans. CORGI is about 21% more difficult than BIRD benchmark based on execution success rate.

Conclusion: There’s a significant gap between current LLM capabilities and real-world business intelligence needs, highlighting the need for benchmarks that test multi-level, multi-step agentic intelligence.

Abstract: In the business domain, where data-driven decision making is crucial, text-to-SQL is fundamental for easy natural language access to structured data. While recent LLMs have achieved strong performance in code generation, existing text-to-SQL benchmarks remain focused on factual retrieval of past records. We introduce CORGI, a new benchmark specifically designed for real-world business contexts. CORGI is composed of synthetic databases inspired by enterprises such as Doordash, Airbnb, and Lululemon. It provides questions across four increasingly complex categories of business queries: descriptive, explanatory, predictive, and recommendational. This challenge calls for causal reasoning, temporal forecasting, and strategic recommendation, reflecting multi-level and multi-step agentic intelligence. We find that LLM performance drops on high-level questions, struggling to make accurate predictions and offer actionable plans. Based on execution success rate, the CORGI benchmark is about 21% more difficult than the BIRD benchmark. This highlights the gap between popular LLMs and the need for real-world business intelligence. We release a public dataset and evaluation framework, and a website for public submissions.

Method: Combines I3D-based feature extraction with specialized feature transformation network and audio generation pipeline, using novel Non-Maximal Suppression algorithm for temporal detection of signs in non-grammatic continuous sequences.

Result: Achieved Top-1 accuracies of 72.01% on ASL-Citizen-1500 and 78.67% on WLASL-100 datasets, with audio quality metrics (PESQ: 2.67, STOI: 0.73) indicating intelligible speech output.

Conclusion: The proposed end-to-end framework successfully translates isolated sign sequences directly to speech with competitive performance and intelligible audio quality, providing immediate communication benefits.

Abstract: Sign language to spoken language audio translation is important to connect the hearing- and speech-challenged humans with others. We consider sign language videos with isolated sign sequences rather than continuous grammatical signing. Such videos are useful in educational applications and sign prompt interfaces. Towards this, we propose IsoSignVid2Aud, a novel end-to-end framework that translates sign language videos with a sequence of possibly non-grammatic continuous signs to speech without requiring intermediate text representation, providing immediate communication benefits while avoiding the latency and cascading errors inherent in multi-stage translation systems. Our approach combines an I3D-based feature extraction module with a specialized feature transformation network and an audio generation pipeline, utilizing a novel Non-Maximal Suppression (NMS) algorithm for the temporal detection of signs in non-grammatic continuous sequences. Experimental results demonstrate competitive performance on ASL-Citizen-1500 and WLASL-100 datasets with Top-1 accuracies of 72.01% and 78.67%, respectively, and audio quality metrics (PESQ: 2.67, STOI: 0.73) indicating intelligible speech output. Code is available at: https://github.com/BheeshmSharma/IsoSignVid2Aud_AIMLsystems-2025.

Method: Uses RT-DETR with multi-scale feature fusion, uncertainty-minimizing query selection, and smart weighting between synthetic and real training samples, plus data augmentation for class balancing.

Result: Developed a real-time maritime detection pipeline that maintains performance under practical limits and handles extreme lighting/sea conditions, with quantified module contributions.

Conclusion: The system preserves DETR’s end-to-end set prediction while allowing speed-accuracy tradeoffs, delivering robust maritime detection with verified module effectiveness.

Abstract: Maritime object detection faces essential challenges due to the small target size and limitations of labeled real RGB data. This paper will present a real-time object detection system based on RT-DETR, enhanced by employing augmented synthetic images while strictly evaluating on real data. This study employs RT-DETR for the maritime environment by combining multi-scale feature fusion, uncertainty-minimizing query selection, and smart weight between synthetic and real training samples. The fusion module in DETR enhances the detection of small, low-contrast vessels, query selection focuses on the most reliable proposals, and the weighting strategy helps reduce the visual gap between synthetic and real domains. This design preserves DETR’s refined end-to-end set prediction while allowing users to adjust between speed and accuracy at inference time. Data augmentation techniques were also used to balance the different classes of the dataset to improve the robustness and accuracy of the model. Regarding this study, a full Python robust maritime detection pipeline is delivered that maintains real-time performance even under practical limits. It also verifies how each module contributes, and how the system handles failures in extreme lighting or sea conditions. This study also includes a component analysis to quantify the contribution of each architectural module and explore its interactions.

Method: Integrates and optimizes new parameters guided by layout-attention objective, uses parametric memory mechanism for streaming video generation with historical optimization contexts, supports insert/read/update/delete operations.

Result: Achieves better cross-modal alignment for compositional video generation, shows powerful transferability and generalization, disentangles compositional world knowledge.

Conclusion: TTOM is an effective, practical, scalable, and efficient framework for compositional video generation that works on the fly without retraining.

Abstract: Video Foundation Models (VFMs) exhibit remarkable visual generation performance, but struggle in compositional scenarios (e.g., motion, numeracy, and spatial relation). In this work, we introduce Test-Time Optimization and Memorization (TTOM), a training-free framework that aligns VFM outputs with spatiotemporal layouts during inference for better text-image alignment. Rather than direct intervention to latents or attention per-sample in existing work, we integrate and optimize new parameters guided by a general layout-attention objective. Furthermore, we formulate video generation within a streaming setting, and maintain historical optimization contexts with a parametric memory mechanism that supports flexible operations, such as insert, read, update, and delete. Notably, we found that TTOM disentangles compositional world knowledge, showing powerful transferability and generalization. Experimental results on the T2V-CompBench and Vbench benchmarks establish TTOM as an effective, practical, scalable, and efficient framework to achieve cross-modal alignment for compositional video generation on the fly.

Method: Created DynamicEval benchmark with curated prompts emphasizing dynamic camera motion. Proposed two new metrics: (1) background scene consistency metric using object error maps to correct Vbench motion smoothness failures, and (2) foreground object consistency metric that tracks points and neighbors within object instances.

Result: The proposed metrics achieve stronger correlations with human preferences at both video level and model level, with improvements of more than 2 percentage points compared to existing methods. Extensive experiments across 3k videos from 10 T2V models with 45k human annotations validate the approach.

Conclusion: DynamicEval establishes a more comprehensive benchmark for evaluating T2V models under dynamic camera motion, addressing key limitations in existing evaluation frameworks and providing better alignment with human judgment through improved background and foreground consistency metrics.

Abstract: Existing text-to-video (T2V) evaluation benchmarks, such as VBench and EvalCrafter, suffer from two limitations. (i) While the emphasis is on subject-centric prompts or static camera scenes, camera motion essential for producing cinematic shots and existing metrics under dynamic motion are largely unexplored. (ii) These benchmarks typically aggregate video-level scores into a single model-level score for ranking generative models. Such aggregation, however, overlook video-level evaluation, which is vital to selecting the better video among the candidate videos generated for a given prompt. To address these gaps, we introduce DynamicEval, a benchmark consisting of systematically curated prompts emphasizing dynamic camera motion, paired with 45k human annotations on video pairs from 3k videos generated by ten T2V models. DynamicEval evaluates two key dimensions of video quality: background scene consistency and foreground object consistency. For background scene consistency, we obtain the interpretable error maps based on the Vbench motion smoothness metric. We observe that while the Vbench motion smoothness metric shows promising alignment with human judgments, it fails in two cases: occlusions/disocclusions arising from camera and foreground object movements. Building on this, we propose a new background consistency metric that leverages object error maps to correct two failure cases in a principled manner. Our second innovation is the introduction of a foreground consistency metric that tracks points and their neighbors within each object instance to assess object fidelity. Extensive experiments demonstrate that our proposed metrics achieve stronger correlations with human preferences at both the video level and the model level (an improvement of more than 2% points), establishing DynamicEval as a more comprehensive benchmark for evaluating T2V models under dynamic camera motion.

Method: RISP incorporates restarting inertia mechanism into RED framework, allowing score-based image priors while accelerating convergence. It’s analyzed through continuous-time dynamical systems and connected to heavy-ball ODE.

Result: RISP achieves faster stationary-point convergence rate than RED without requiring convexity of image prior. Experiments show fast convergence with high-quality reconstructions across various imaging inverse problems.

Conclusion: RISP successfully bridges the gap between convergence acceleration and reconstruction quality, providing both fast convergence and high-quality image recovery in ill-posed imaging inverse problems.

Abstract: Fast convergence and high-quality image recovery are two essential features of algorithms for solving ill-posed imaging inverse problems. Existing methods, such as regularization by denoising (RED), often focus on designing sophisticated image priors to improve reconstruction quality, while leaving convergence acceleration to heuristics. To bridge the gap, we propose Restarted Inertia with Score-based Priors (RISP) as a principled extension of RED. RISP incorporates a restarting inertia for fast convergence, while still allowing score-based image priors for high-quality reconstruction. We prove that RISP attains a faster stationary-point convergence rate than RED, without requiring the convexity of the image prior. We further derive and analyze the associated continuous-time dynamical system, offering insight into the connection between RISP and the heavy-ball ordinary differential equation (ODE). Experiments across a range of imaging inverse problems demonstrate that RISP enables fast convergence while achieving high-quality reconstructions.

Method: Proposed Temporally Conditioned Attention Sharpening (TCAS) - an attention enhancement method that constructs an enhancement objective based on attention distinctions to improve the model’s temporal resolution capability.

Result: TCAS significantly enhances temporal logic consistency in Video-LLMs, improves temporal discriminability of attention heads, and achieves performance improvements in general video temporal grounding tasks.

Conclusion: Temporal logic consistency is a bottleneck in video temporal understanding, and enhancing consistency through TCAS drives significant progress in video temporal understanding capabilities.

Abstract: Large language models (LLMs) often generate self-contradictory outputs, which severely impacts their reliability and hinders their adoption in practical applications. In video-language models (Video-LLMs), this phenomenon recently draws the attention of researchers. Specifically, these models fail to provide logically consistent responses to rephrased questions based on their grounding outputs. However, the underlying causes of this phenomenon remain underexplored. In this work, we adopt an interpretability-driven approach to analyze, statistically summarize, and intervention the potential factors of the phenomenon. We find that one of the primary reasons for the inconsistency in responses lies in the inability of cross-modal attention heads to effectively distinguish video tokens across different timestamps. To address this, we propose an attention enhancement method called Temporally Conditioned Attention Sharpening (TCAS), which constructs an enhancement objective based on attention distinctions to enhance the model’s temporal resolution capability, thereby improving its temporal understanding logic consistency. Experimental results demonstrate that our method significantly enhances the temporal logic consistency of Video-LLMs. Further interpretability analyses reveal that our method indeed improves the temporal discriminability of attention heads, validating our conclusions. Additionally, our method achieves performance improvements in general video temporal grounding tasks, highlighting that temporal logic consistency is a bottleneck in temporal understanding. By enhancing consistency, our method drives significant progress in video temporal understanding.

Method: Uses Image Purification strategy to generate structurally aligned uLDCT-NDCT pairs, combined with Frequency-domain Flow Matching model to preserve anatomical structure integrity.

Result: IP strategy significantly enhances multiple denoising models’ performance on uLDCT task. FFM model with IP achieves SOTA results in anatomical structure preservation.

Conclusion: Provides effective solution to data mismatch problem in real-world uLDCT denoising, with code and dataset publicly available.

Abstract: Ultra-low dose CT (uLDCT) significantly reduces radiation exposure but introduces severe noise and artifacts. It also leads to substantial spatial misalignment between uLDCT and normal dose CT (NDCT) image pairs. This poses challenges for directly applying existing denoising networks trained on synthetic noise or aligned data. To address this core challenge in uLDCT denoising, this paper proposes an innovative denoising framework based on an Image Purification (IP) strategy. First, we construct a real clinical uLDCT lung dataset. Then, we propose an Image Purification strategy that generates structurally aligned uLDCT-NDCT image pairs, providing a high-quality data foundation for network training. Building upon this, we propose a Frequency-domain Flow Matching (FFM) model, which works synergistically with the IP strategy to excellently preserve the anatomical structure integrity of denoised images. Experiments on the real clinical dataset demonstrate that our IP strategy significantly enhances the performance of multiple mainstream denoising models on the uLDCT task. Notably, our proposed FFM model combined with the IP strategy achieves state-of-the-art (SOTA) results in anatomical structure preservation. This study provides an effective solution to the data mismatch problem in real-world uLDCT denoising. Code and dataset are available at https://github.com/MonkeyDadLufy/flow-matching.

Method: Formulates PRVR as multiple instance learning problem and proposes MS-SL++ network that jointly learns clip-scale and frame-scale similarities to determine partial relevance between video-query pairs.

Result: Extensive experiments on TVshow Retrieval, ActivityNet-Captions and Charades-STA datasets demonstrate the viability of the proposed method.

Conclusion: The study successfully addresses the PRVR problem and shows that learning multi-scale similarities effectively handles partial relevance in video retrieval.

Abstract: In current text-to-video retrieval (T2VR), videos to be retrieved have been properly trimmed so that a correspondence between the videos and ad-hoc textual queries naturally exists. Note in practice that videos circulated on the Internet and social media platforms, while being relatively short, are typically rich in their content. Often, multiple scenes / actions / events are shown in a single video, leading to a more challenging T2VR setting wherein only part of the video content is relevant w.r.t. a given query. This paper presents a first study on this setting which we term Partially Relevant Video Retrieval (PRVR). Considering that a video typically consists of multiple moments, a video is regarded as partially relevant w.r.t. to a given query if it contains a query-related moment. We formulate the PRVR task as a multiple instance learning problem, and propose a Multi-Scale Similarity Learning (MS-SL++) network that jointly learns both clip-scale and frame-scale similarities to determine the partial relevance between video-query pairs. Extensive experiments on three diverse video-text datasets (TVshow Retrieval, ActivityNet-Captions and Charades-STA) demonstrate the viability of the proposed method.

Method: D2RA projects watermarked images onto natural priors across complementary representations to suppress watermark signals while preserving visual fidelity, without requiring access to the underlying model or training.

Result: Experiments show D2RA consistently reduces watermark detectability across diverse watermarking schemes, demonstrating fundamental weaknesses in current designs.

Conclusion: The study reveals that current semantic watermarking methods have significant vulnerabilities that can be exploited by simple, resource-efficient attacks like D2RA.

Abstract: The growing use of generative models has intensified the need for watermarking methods that ensure content attribution and provenance. While recent semantic watermarking schemes improve robustness by embedding signals in latent or frequency representations, we show they remain vulnerable even under resource-constrained adversarial settings. We present D2RA, a training-free, single-image attack that removes or weakens watermarks without access to the underlying model. By projecting watermarked images onto natural priors across complementary representations, D2RA suppresses watermark signals while preserving visual fidelity. Experiments across diverse watermarking schemes demonstrate that our approach consistently reduces watermark detectability, revealing fundamental weaknesses in current designs. Our code is available at https://github.com/Pragati-Meshram/DAWN.

Method: Uses pretrained video diffusion backbones with low-rank style adapters in self-attention layers, trains on synthetic clips from paired images with shared augmentations, and introduces Context-Style Classifier-Free Guidance (CS-CFG) for independent text and video conditioning.

Result: Achieves temporally coherent, style-faithful, and content-preserving video translations, outperforming existing baselines both qualitatively and quantitatively across benchmarks.

Conclusion: PickStyle effectively bridges the gap between static image supervision and dynamic video style transfer through adapter-based specialization and novel guidance factorization.

Abstract: We address the task of video style transfer with diffusion models, where the goal is to preserve the context of an input video while rendering it in a target style specified by a text prompt. A major challenge is the lack of paired video data for supervision. We propose PickStyle, a video-to-video style transfer framework that augments pretrained video diffusion backbones with style adapters and benefits from paired still image data with source-style correspondences for training. PickStyle inserts low-rank adapters into the self-attention layers of conditioning modules, enabling efficient specialization for motion-style transfer while maintaining strong alignment between video content and style. To bridge the gap between static image supervision and dynamic video, we construct synthetic training clips from paired images by applying shared augmentations that simulate camera motion, ensuring temporal priors are preserved. In addition, we introduce Context-Style Classifier-Free Guidance (CS-CFG), a novel factorization of classifier-free guidance into independent text (style) and video (context) directions. CS-CFG ensures that context is preserved in generated video while the style is effectively transferred. Experiments across benchmarks show that our approach achieves temporally coherent, style-faithful, and content-preserving video translations, outperforming existing baselines both qualitatively and quantitatively.

Method: TRAVL combines a balanced training dataset with a trajectory-aware attention module to improve motion encoding and discrimination in VLMs. ImplausiBench provides 300 videos (150 real, 150 generated) that remove linguistic biases and isolate visual-temporal understanding.

Result: The proposed framework offers improved capability for detecting physical plausibility violations compared to existing VLMs, with performance evaluated using both human judgments and LLM-as-judge metrics.

Conclusion: TRAVL and ImplausiBench provide a unified framework for probing and improving physical plausibility in multimodal models, addressing a challenging aspect of visual-temporal understanding that has been underexplored.

Abstract: Despite impressive visual fidelity, modern video generative models frequently produce sequences that violate intuitive physical laws, such as objects floating, teleporting, or morphing in ways that defy causality. While humans can easily detect such implausibilities, there remains no robust method for quantitatively assessing physical realism in video. In this work, we explore whether Video-Language Models (VLMs) can be trained to serve as reliable judges of physical plausibility. We find that existing VLMs struggle to identify physics violations, exposing fundamental limitations in their temporal and causal reasoning. To address this, we introduce TRAVL, a fine-tuning recipe that combines a balanced training dataset with a trajectory-aware attention module to improve motion encoding and discrimination in VLMs. To evaluate physical reasoning more rigorously, we propose ImplausiBench, a benchmark of 300 videos (150 real, 150 generated) that removes linguistic biases and isolates visual-temporal understanding. Performance is reported both with gold-standard human judgments and stricter LLM-as-judge metrics. Together, TRAVL and ImplausiBench offer a unified framework for probing and improving physical plausibility in multimodal models, shedding light on a challenging and underexplored aspect of visual-temporal understanding.

Method: Proposed PaperTalker, a multi-agent framework that integrates slide generation with layout refinement using novel tree search visual choice, cursor grounding, subtitling, speech synthesis, and talking-head rendering, while parallelizing slide-wise generation for efficiency.

Result: Experiments on the Paper2Video benchmark show that the presentation videos produced by PaperTalker are more faithful and informative than existing baselines.

Conclusion: The work establishes a practical step toward automated and ready-to-use academic video generation, with the dataset, agent, and code made publicly available.

Abstract: Academic presentation videos have become an essential medium for research communication, yet producing them remains highly labor-intensive, often requiring hours of slide design, recording, and editing for a short 2 to 10 minutes video. Unlike natural video, presentation video generation involves distinctive challenges: inputs from research papers, dense multi-modal information (text, figures, tables), and the need to coordinate multiple aligned channels such as slides, subtitles, speech, and human talker. To address these challenges, we introduce Paper2Video, the first benchmark of 101 research papers paired with author-created presentation videos, slides, and speaker metadata. We further design four tailored evaluation metrics–Meta Similarity, PresentArena, PresentQuiz, and IP Memory–to measure how videos convey the paper’s information to the audience. Building on this foundation, we propose PaperTalker, the first multi-agent framework for academic presentation video generation. It integrates slide generation with effective layout refinement by a novel effective tree search visual choice, cursor grounding, subtitling, speech synthesis, and talking-head rendering, while parallelizing slide-wise generation for efficiency. Experiments on Paper2Video demonstrate that the presentation videos produced by our approach are more faithful and informative than existing baselines, establishing a practical step toward automated and ready-to-use academic video generation. Our dataset, agent, and code are available at https://github.com/showlab/Paper2Video.

Method: S3FN uses LLMs to generate comprehensive textual descriptions for each class, embeds them using pre-trained text encoders (BERT/RoBERTa), and fuses these semantic embeddings with spectral-spatial data for better feature-label alignment.

Result: Significant performance boost demonstrated on three HSI benchmark datasets: Hyperspectral Wood, HyperspectralBlueberries, and DeepHS-Fruit.

Conclusion: The approach shows synergy between textual semantics and spectral-spatial data, paving the way for semantically augmented HSI classification models.

Abstract: Hyperspectral imaging (HSI) classification is a critical tool with widespread applications across diverse fields such as agriculture, environmental monitoring, medicine, and materials science. Due to the limited availability of high-quality training samples and the high dimensionality of spectral data, HSI classification models are prone to overfitting and often face challenges in balancing accuracy and computational complexity. Furthermore, most of HSI classification models are monomodal, where it solely relies on spectral-spatial data to learn decision boundaries in the high dimensional embedding space. To address this, we propose a general-purpose Semantic Spectral-Spatial Fusion Network (S3FN) that uses contextual, class specific textual descriptions to complement the training of an HSI classification model. Specifically, S3FN leverages LLMs to generate comprehensive textual descriptions for each class label that captures their unique characteristics and spectral behaviors. These descriptions are then embedded into a vector space using a pre-trained text encoder such as BERT or RoBERTa to extract meaningful label semantics which in turn leads to a better feature-label alignment for improved classification performance. To demonstrate the effectiveness of our approach, we evaluate our model on three diverse HSI benchmark datasets - Hyperspectral Wood, HyperspectralBlueberries, and DeepHS-Fruit and report significant performance boost. Our results highlight the synergy between textual semantics and spectral-spatial data, paving the way for further advancements in semantically augmented HSI classification models. Codes are be available in: https://github.com/milab-nsu/S3FN

Method: Cross-Modal Attention Guided Unlearning (CAGUL) uses external modules to encode unlearning information in visual tokens of low importance for relevant queries, leveraging cross-modal attention patterns.

Result: CAGUL performs better or on par with finetuning-based baselines while preventing information leakage and retaining reference model behavior, without parameter changes or retraining costs.

Conclusion: CAGUL provides a practical and effective unlearning solution for VLMs that is lightweight, efficient, and maintains model performance while addressing privacy concerns.

Abstract: Vision-Language Models (VLMs) have demonstrated immense capabilities in multi-modal understanding and inference tasks such as Visual Question Answering (VQA), which requires models to infer outputs based on visual and textual context simultaneously. Such inference abilities of large-scale pretrained models are often attributed to the massive scale of pre-training data collected across several domains. However, the models may memorize private and/or sensitive information during training and regurgitate it in inference. Recently, machine unlearning has been leveraged to address the leakage of private data in LLMs. VLMs add a layer of complexity to this process, as the visual context in the query may also contain sensitive information in addition to the text. To address this issue, we explore unlearning for vision-language models, specifically for the VQA task. We explore the role of visual tokens for output generation in VLMs using cross-modal attention and utilize it to formulate Cross-Modal Attention Guided Unlearning (CAGUL), a lightweight and efficient VLM unlearning framework. In contrast to computationally expensive model finetuning methods, CAGUL utilizes external modules to encode unlearning information in visual tokens of low importance for relevant queries. We find that the transformed visual tokens not only prevent leakage but also retain reference model behavior. Experimental results show that our method performs better or on par with finetuning-based baselines without altering the pre-trained model parameters or incurring retraining costs, making it a practical and effective unlearning solution for VLMs.

Method: Uses YOLOv9 model trained on V2-V10 growth stages with two processing modes: mosaic image patches and homography-aligned video frames, followed by row and range segmentation for precise counting.

Result: Strong agreement with manual counts (R^2=0.616 for mosaics, R^2=0.906 for raw frames), processing 83 frames in 60.63 seconds including inference and post-processing.

Conclusion: MaSC is an effective, scalable, low-cost tool for automated maize stand counting suitable for both research and production environments.

Abstract: Accurate maize stand counts are essential for crop management and research, informing yield prediction, planting density optimization, and early detection of germination issues. Manual counting is labor-intensive, slow, and error-prone, especially across large or variable fields. We present MaizeStandCounting (MaSC), a robust algorithm for automated maize seedling stand counting from RGB imagery captured by low-cost UAVs and processed on affordable hardware. MaSC operates in two modes: (1) mosaic images divided into patches, and (2) raw video frames aligned using homography matrices. Both modes use a lightweight YOLOv9 model trained to detect maize seedlings from V2-V10 growth stages. MaSC distinguishes maize from weeds and other vegetation, then performs row and range segmentation based on the spatial distribution of detections to produce precise row-wise stand counts. Evaluation against in-field manual counts from our 2024 summer nursery showed strong agreement with ground truth (R^2= 0.616 for mosaics, R^2 = 0.906 for raw frames). MaSC processed 83 full-resolution frames in 60.63 s, including inference and post-processing, highlighting its potential for real-time operation. These results demonstrate MaSC’s effectiveness as a scalable, low-cost, and accurate tool for automated maize stand counting in both research and production environments.

Method: QCN builds on producing fewer capsules to create a faster network, and is further enhanced by employing a more powerful decoder instead of the default decoder.

Result: QCN achieves 5x faster inference on MNIST, F-MNIST, SVHN and Cifar-10 datasets with only marginal loss in accuracy compared to CapsNet.

Conclusion: QCN serves as a fast alternative to CapsNet and can be a starting point for developing CapsNet for fast real-time applications.

Abstract: The basic computational unit in Capsule Network (CapsNet) is a capsule (vs. neurons in Convolutional Neural Networks (CNNs)). A capsule is a set of neurons, which form a vector. CapsNet is used for supervised classification of data and has achieved state-of-the-art accuracy on MNIST digit recognition dataset, outperforming conventional CNNs in detecting overlapping digits. Moreover, CapsNet shows higher robustness towards affine transformation when compared to CNNs for MNIST datasets. One of the drawbacks of CapsNet, however, is slow training and testing. This can be a bottleneck for applications that require a fast network, especially during inference. In this work, we introduce Quick-CapsNet (QCN) as a fast alternative to CapsNet, which can be a starting point to develop CapsNet for fast real-time applications. QCN builds on producing a fewer number of capsules, which results in a faster network. QCN achieves this at the cost of marginal loss in accuracy. Inference is 5x faster on MNIST, F-MNIST, SVHN and Cifar-10 datasets. We also further enhanced QCN by employing a more powerful decoder instead of the default decoder to further improve QCN.

Method: Uses adaptive predictor-corrector guidance with conditional RF update followed by weighted conditional correction that interpolates between conditional and unconditional velocity fields.

Result: Proven to maintain marginal consistency and bounded trajectories near data manifold, ensuring stability across guidance strengths. Outperforms standard CFG on benchmark datasets including MS-COCO, LAION-Aesthetic, and T2I-CompBench.

Conclusion: Rectified-CFG++ successfully couples deterministic efficiency of rectified flows with geometry-aware conditioning, providing stable and improved text-to-image generation across various large-scale models.

Abstract: Classifier-free guidance (CFG) is the workhorse for steering large diffusion models toward text-conditioned targets, yet its native application to rectified flow (RF) based models provokes severe off-manifold drift, yielding visual artifacts, text misalignment, and brittle behaviour. We present Rectified-CFG++, an adaptive predictor-corrector guidance that couples the deterministic efficiency of rectified flows with a geometry-aware conditioning rule. Each inference step first executes a conditional RF update that anchors the sample near the learned transport path, then applies a weighted conditional correction that interpolates between conditional and unconditional velocity fields. We prove that the resulting velocity field is marginally consistent and that its trajectories remain within a bounded tubular neighbourhood of the data manifold, ensuring stability across a wide range of guidance strengths. Extensive experiments on large-scale text-to-image models (Flux, Stable Diffusion 3/3.5, Lumina) show that Rectified-CFG++ consistently outperforms standard CFG on benchmark datasets such as MS-COCO, LAION-Aesthetic, and T2I-CompBench. Project page: https://rectified-cfgpp.github.io/

Method: Constructed PIT-QMM, a novel LMM capable of consuming text, images, and point clouds end-to-end to predict quality scores for No-Reference Point Cloud Quality Assessment.

Result: Extensive experiments show the method outperforms state-of-the-art by significant margins on popular benchmarks with fewer training iterations. The framework also enables distortion localization and identification.

Conclusion: PIT-QMM paves a new way forward for model explainability and interactivity in point cloud quality assessment, demonstrating the effectiveness of multimodal approaches for 3D asset evaluation.

Abstract: Large Multimodal Models (LMMs) have recently enabled considerable advances in the realm of image and video quality assessment, but this progress has yet to be fully explored in the domain of 3D assets. We are interested in using these models to conduct No-Reference Point Cloud Quality Assessment (NR-PCQA), where the aim is to automatically evaluate the perceptual quality of a point cloud in absence of a reference. We begin with the observation that different modalities of data - text descriptions, 2D projections, and 3D point cloud views - provide complementary information about point cloud quality. We then construct PIT-QMM, a novel LMM for NR-PCQA that is capable of consuming text, images and point clouds end-to-end to predict quality scores. Extensive experimentation shows that our proposed method outperforms the state-of-the-art by significant margins on popular benchmarks with fewer training iterations. We also demonstrate that our framework enables distortion localization and identification, which paves a new way forward for model explainability and interactivity. Code and datasets are available at https://www.github.com/shngt/pit-qmm.

Method: Dual-Stream Alignment Network (DSA Net) with Temporal Context block using cross-attention and Quantum-based Action-Guided Modulation. Uses Dual-Stream Alignment Loss with relational consistency, cross-level contrastive, and cycle-consistency reconstruction components.

Result: Achieves state-of-the-art performance on GTEA, Breakfast, 50Salads, and EgoProcel datasets. Extensive ablation studies demonstrate effectiveness of each component.

Conclusion: The proposed dual-stream approach with quantum-classical fusion and feature alignment significantly improves action segmentation performance, representing the first hybrid quantum-classical framework for this task.

Abstract: Action segmentation is a challenging yet active research area that involves identifying when and where specific actions occur in continuous video streams. Most existing work has focused on single-stream approaches that model the spatio-temporal aspects of frame sequences. However, recent research has shifted toward two-stream methods that learn action-wise features to enhance action segmentation performance. In this work, we propose the Dual-Stream Alignment Network (DSA Net) and investigate the impact of incorporating a second stream of learned action features to guide segmentation by capturing both action and action-transition cues. Communication between the two streams is facilitated by a Temporal Context (TC) block, which fuses complementary information using cross-attention and Quantum-based Action-Guided Modulation (Q-ActGM), enhancing the expressive power of the fused features. To the best of our knowledge, this is the first study to introduce a hybrid quantum-classical machine learning framework for action segmentation. Our primary objective is for the two streams (frame-wise and action-wise) to learn a shared feature space through feature alignment. This is encouraged by the proposed Dual-Stream Alignment Loss, which comprises three components: relational consistency, cross-level contrastive, and cycle-consistency reconstruction losses. Following prior work, we evaluate DSA Net on several diverse benchmark datasets: GTEA, Breakfast, 50Salads, and EgoProcel. We further demonstrate the effectiveness of each component through extensive ablation studies. Notably, DSA Net achieves state-of-the-art performance, significantly outperforming existing

Method: Use image-based clothing transfer on first frame only, then use pose and mask information to guide video generation model to synthesize remaining frames sequentially under content control of edited first frame.

Result: Achieves superior parameter efficiency and computational efficiency while maintaining leading performance under constraints.

Conclusion: OIE provides an efficient virtual try-on strategy that avoids complex dual-branch architectures and achieves good performance with reduced computational requirements.

Abstract: Video virtual try-on aims to replace the clothing of a person in a video with a target garment. Current dual-branch architectures have achieved significant success in diffusion models based on the U-Net; however, adapting them to diffusion models built upon the Diffusion Transformer remains challenging. Initially, introducing latent space features from the garment reference branch requires adding or modifying the backbone network, leading to a large number of trainable parameters. Subsequently, the latent space features of garments lack inherent temporal characteristics and thus require additional learning. To address these challenges, we propose a novel approach, OIE (Once is Enough), a virtual try-on strategy based on first-frame clothing replacement: specifically, we employ an image-based clothing transfer model to replace the clothing in the initial frame, and then, under the content control of the edited first frame, utilize pose and mask information to guide the temporal prior of the video generation model in synthesizing the remaining frames sequentially. Experiments show that our method achieves superior parameter efficiency and computational efficiency while still maintaining leading performance under these constraints.

Method: Created the BRIGHT dataset - a multimodal dataset using very-high-resolution optical and SAR imagery (0.3-1 meter spatial resolution) covering five types of natural disasters and two types of man-made disasters across 14 regions worldwide, with focus on developing countries.

Result: Tested seven advanced AI models trained with BRIGHT dataset, validating transferability and robustness. The dataset is publicly available and serves as the official dataset for the 2025 IEEE GRSS Data Fusion Contest.

Conclusion: BRIGHT enables the development of robust multimodal AI models for building damage assessment that can operate in all weather conditions and at any time, addressing critical limitations of optical-only solutions and supporting rapid disaster response globally.

Abstract: Disaster events occur around the world and cause significant damage to human life and property. Earth observation (EO) data enables rapid and comprehensive building damage assessment (BDA), an essential capability in the aftermath of a disaster to reduce human casualties and to inform disaster relief efforts. Recent research focuses on the development of AI models to achieve accurate mapping of unseen disaster events, mostly using optical EO data. However, solutions based on optical data are limited to clear skies and daylight hours, preventing a prompt response to disasters. Integrating multimodal (MM) EO data, particularly the combination of optical and SAR imagery, makes it possible to provide all-weather, day-and-night disaster responses. Despite this potential, the development of robust multimodal AI models has been constrained by the lack of suitable benchmark datasets. In this paper, we present a BDA dataset using veRy-hIGH-resoluTion optical and SAR imagery (BRIGHT) to support AI-based all-weather disaster response. To the best of our knowledge, BRIGHT is the first open-access, globally distributed, event-diverse MM dataset specifically curated to support AI-based disaster response. It covers five types of natural disasters and two types of man-made disasters across 14 regions worldwide, with a particular focus on developing countries where external assistance is most needed. The optical and SAR imagery in BRIGHT, with a spatial resolution between 0.3-1 meters, provides detailed representations of individual buildings, making it ideal for precise BDA. In our experiments, we have tested seven advanced AI models trained with our BRIGHT to validate the transferability and robustness. The dataset and code are available at https://github.com/ChenHongruixuan/BRIGHT. BRIGHT also serves as the official dataset for the 2025 IEEE GRSS Data Fusion Contest.

Method: Using IP-Adapter’s automatically generated masks on a second pass to mask image tokens, restricting them to the subject area so text prompts can attend to the background.

Result: The method produces images that accurately depict the subject while matching text prompts, showing high prompt and source image alignment compared to other test time personalization methods.

Conclusion: Masking image tokens to the subject area enables better text prompt control over backgrounds, improving personalization in diffusion models.

Abstract: Personalizing diffusion models allows users to generate new images that incorporate a given subject, allowing more control than a text prompt. These models often suffer somewhat when they end up just recreating the subject image, and ignoring the text prompt. We observe that one popular method for personalization, the IP-Adapter automatically generates masks that we definitively segment the subject from the background during inference. We propose to use this automatically generated mask on a second pass to mask the image tokens, thus restricting them to the subject, not the background, allowing the text prompt to attend to the rest of the image. For text prompts describing locations and places, this produces images that accurately depict the subject while definitively matching the prompt. We compare our method to a few other test time personalization methods, and find our method displays high prompt and source image alignment.

Method: MAViS employs a multi-agent system with specialized agents operating under the 3E Principle (Explore, Examine, Enhance) across multiple stages. It includes Script Writing Guidelines to optimize compatibility between scripts and generative tools.

Result: MAViS achieves state-of-the-art performance in assistive capability, visual quality, and video expressiveness. It enables rapid production of high-quality, complete long-sequence videos from brief idea descriptions.

Conclusion: MAViS is the only framework that provides multimodal design output (videos with narratives and background music) and offers scalability with diverse generative models and tools for efficient long-sequence video storytelling.

Abstract: Despite recent advances, long-sequence video generation frameworks still suffer from significant limitations: poor assistive capability, suboptimal visual quality, and limited expressiveness. To mitigate these limitations, we propose MAViS, a multi-agent collaborative framework designed to assist in long-sequence video storytelling by efficiently translating ideas into visual narratives. MAViS orchestrates specialized agents across multiple stages, including script writing, shot designing, character modeling, keyframe generation, video animation, and audio generation. In each stage, agents operate under the 3E Principle – Explore, Examine, and Enhance – to ensure the completeness of intermediate outputs. Considering the capability limitations of current generative models, we propose the Script Writing Guidelines to optimize compatibility between scripts and generative tools. Experimental results demonstrate that MAViS achieves state-of-the-art performance in assistive capability, visual quality, and video expressiveness. Its modular framework further enables scalability with diverse generative models and tools. With just a brief idea description, MAViS enables users to rapidly explore diverse visual storytelling and creative directions for sequential video generation by efficiently producing high-quality, complete long-sequence videos. To the best of our knowledge, MAViS is the only framework that provides multimodal design output – videos with narratives and background music.

Method: Compared standard Transformer-based method, small VLM (Phi3.5-vision), large pretrained VLM (Gemini), and extended Transformer processing multiple images on Crello dataset.

Result: Standard Transformer-based models generally outperform VLM-based approaches, particularly when incorporating richer appearance information. All methods struggle with very small text or densely populated layouts.

Conclusion: Task-specific architectures provide benefits for automated layout design, suggesting avenues for further improvement despite current limitations with small text and dense layouts.

Abstract: In layout design for advertisements and web pages, balancing visual appeal and communication efficiency is crucial. This study examines automated text box placement in incomplete layouts, comparing a standard Transformer-based method, a small Vision and Language Model (Phi3.5-vision), a large pretrained VLM (Gemini), and an extended Transformer that processes multiple images. Evaluations on the Crello dataset show the standard Transformer-based models generally outperform VLM-based approaches, particularly when incorporating richer appearance information. However, all methods face challenges with very small text or densely populated layouts. These findings highlight the benefits of task-specific architectures and suggest avenues for further improvement in automated layout design.

Method: Introduces Feature-Enhanced Residual Module (FERM) with three sequential blocks for feature extraction, suppression, and deformation estimation, plus dual-stage Threshold-Controlled Iterative (TCI) strategy for adaptive iteration determination.

Result: TCIP outperforms SOTA registration networks on three brain MRI and one abdomen CT datasets in accuracy while maintaining comparable speed and compact parameter size.

Conclusion: FERM and TCI components are effective and generalizable, as validated by integration with existing networks and ablation studies.

Abstract: Although pyramid networks have demonstrated superior performance in deformable medical image registration, their decoder architectures are inherently prone to propagating and accumulating anatomical structure misalignments. Moreover, most existing models do not adaptively determine the number of iterations for optimization under varying deformation requirements across images, resulting in either premature termination or excessive iterations that degrades registration accuracy. To effectively mitigate the accumulation of anatomical misalignments, we propose the Feature-Enhanced Residual Module (FERM) as the core component of each decoding layer in the pyramid network. FERM comprises three sequential blocks that extract anatomical semantic features, learn to suppress irrelevant features, and estimate the final deformation field, respectively. To adaptively determine the number of iterations for varying images, we propose the dual-stage Threshold-Controlled Iterative (TCI) strategy. In the first stage, TCI assesses registration stability and with asserted stability, it continues with the second stage to evaluate convergence. We coin the model that integrates FERM and TCI as Threshold-Controlled Iterative Pyramid (TCIP). Extensive experiments on three public brain MRI datasets and one abdomen CT dataset demonstrate that TCIP outperforms the state-of-the-art (SOTA) registration networks in terms of accuracy, while maintaining comparable inference speed and a compact model parameter size. Finally, we assess the generalizability of FERM and TCI by integrating them with existing registration networks and further conduct ablation studies to validate the effectiveness of these two proposed methods.

Method: Uses variational inference to approximate composed distribution, leverages multiple video generation backbones, employs step-wise KL divergence minimization over annealed distributions, and introduces context-conditioned factorization to reduce solution space modes.

Result: Produces samples with improved controllability, diversity, and 3D consistency compared to prior works.

Conclusion: The method successfully addresses the need for flexible control granularity in video synthesis while maintaining sample quality and diversity.

Abstract: Many video workflows benefit from a mixture of user controls with varying granularity, from exact 4D object trajectories and camera paths to coarse text prompts, while existing video generative models are typically trained for fixed input formats. We develop a video synthesis method that addresses this need and generates samples with high controllability for specified elements while maintaining diversity for under-specified ones. We cast the task as variational inference to approximate a composed distribution, leveraging multiple video generation backbones to account for all task constraints collectively. To address the optimization challenge, we break down the problem into step-wise KL divergence minimization over an annealed sequence of distributions, and further propose a context-conditioned factorization technique that reduces modes in the solution space to circumvent local optima. Experiments suggest that our method produces samples with improved controllability, diversity, and 3D consistency compared to prior works.

Method: Hybrid approach integrating BYOL with CNNs, using multiple DL models (ResNet-50, DenseNet-121, etc.) and introducing a new lightweight CNN model (BYOL-IMNet) with four custom-designed blocks for thermal image classification.

Result: BYOL-IMNet achieves 99.89% test accuracy with 5.7 ms inference time per image, outperforming state-of-the-art models.

Conclusion: The CNN-BYOL hybrid method shows promising performance for accurate fault detection in induction motors, providing a robust solution for industrial online monitoring.

Abstract: Induction motors (IMs) are indispensable in industrial and daily life, but they are susceptible to various faults that can lead to overheating, wasted energy consumption, and service failure. Early detection of faults is essential to protect the motor and prolong its lifespan. This paper presents a hybrid method that integrates BYOL with CNNs for classifying thermal images of induction motors for fault detection. The thermal dataset used in this work includes different operating states of the motor, such as normal operation, overload, and faults. We employed multiple deep learning (DL) models for the BYOL technique, ranging from popular architectures such as ResNet-50, DenseNet-121, DenseNet-169, EfficientNetB0, VGG16, and MobileNetV2. Additionally, we introduced a new high-performance yet lightweight CNN model named BYOL-IMNet, which comprises four custom-designed blocks tailored for fault classification in thermal images. Our experimental results demonstrate that the proposed BYOL-IMNet achieves 99.89% test accuracy and an inference time of 5.7 ms per image, outperforming state-of-the-art models. This study highlights the promising performance of the CNN-BYOL hybrid method in enhancing accuracy for detecting faults in induction motors, offering a robust methodology for online monitoring in industrial settings.

Method: Proposes a weak-to-strong framework with two branches: a strong center-based branch and a weaker pairwise-based branch. Uses mutual learning and a novel mixture-of-hash-experts module for cross-branch interaction.

Result: Extensive experiments show MLH consistently outperforms state-of-the-art hashing methods across multiple benchmark datasets.

Conclusion: The mutual learning framework effectively combines the strengths of both center-based and pairwise-based approaches, achieving superior hashing performance by leveraging both global and local similarity information.

Abstract: Deep hashing has been widely adopted for large-scale image retrieval, with numerous strategies proposed to optimize hash function learning. Pairwise-based methods are effective in learning hash functions that preserve local similarity relationships, whereas center-based methods typically achieve superior performance by more effectively capturing global data distributions. However, the strength of center-based methods in modeling global structures often comes at the expense of underutilizing important local similarity information. To address this limitation, we propose Mutual Learning for Hashing (MLH), a novel weak-to-strong framework that enhances a center-based hashing branch by transferring knowledge from a weaker pairwise-based branch. MLH consists of two branches: a strong center-based branch and a weaker pairwise-based branch. Through an iterative mutual learning process, the center-based branch leverages local similarity cues learned by the pairwise-based branch. Furthermore, inspired by the mixture-of-experts paradigm, we introduce a novel mixture-of-hash-experts module that enables effective cross-branch interaction, further enhancing the performance of both branches. Extensive experiments demonstrate that MLH consistently outperforms state-of-the-art hashing methods across multiple benchmark datasets.

Method: Proposes Repainter framework integrating spatial-matting trajectory refinement with Group Relative Policy Optimization (GRPO), modulating attention mechanisms to emphasize background context and using a composite reward mechanism balancing global, local, and semantic constraints.

Result: Extensive experiments show Repainter significantly outperforms state-of-the-art methods, especially in challenging scenes with intricate compositions. Also introduces EcomPaint-100K dataset and EcomPaint-Bench benchmark.

Conclusion: Repainter effectively addresses e-commerce image inpainting challenges, reducing visual artifacts and unwanted object insertion while providing a standardized evaluation framework for the domain.

Abstract: In web data, product images are central to boosting user engagement and advertising efficacy on e-commerce platforms, yet the intrusive elements such as watermarks and promotional text remain major obstacles to delivering clear and appealing product visuals. Although diffusion-based inpainting methods have advanced, they still face challenges in commercial settings due to unreliable object removal and limited domain-specific adaptation. To tackle these challenges, we propose Repainter, a reinforcement learning framework that integrates spatial-matting trajectory refinement with Group Relative Policy Optimization (GRPO). Our approach modulates attention mechanisms to emphasize background context, generating higher-reward samples and reducing unwanted object insertion. We also introduce a composite reward mechanism that balances global, local, and semantic constraints, effectively reducing visual artifacts and reward hacking. Additionally, we contribute EcomPaint-100K, a high-quality, large-scale e-commerce inpainting dataset, and a standardized benchmark EcomPaint-Bench for fair evaluation. Extensive experiments demonstrate that Repainter significantly outperforms state-of-the-art methods, especially in challenging scenes with intricate compositions. We will release our code and weights upon acceptance.

Method: Jointly fine-tune multiview generative model (good at 2D details) and 3D native generative model (good at structural consistency) using pixel-aligned 2D-3D synchronization attention. Add feature injection mechanism to lift fine details from 2D images onto 3D shapes.

Result: Achieves robust and photo-realistic 3D human reconstruction even for challenging poses. Outperforms baseline methods in both geometric accuracy and visual fidelity.

Conclusion: The integration of complementary 2D and 3D generative models represents a promising direction for future 3D generation, enabling high-quality clothed human mesh reconstruction from single-view images.

Abstract: Photorealistic 3D full-body human reconstruction from a single image is a critical yet challenging task for applications in films and video games due to inherent ambiguities and severe self-occlusions. While recent approaches leverage SMPL estimation and SMPL-conditioned image generative models to hallucinate novel views, they suffer from inaccurate 3D priors estimated from SMPL meshes and have difficulty in handling difficult human poses and reconstructing fine details. In this paper, we propose SyncHuman, a novel framework that combines 2D multiview generative model and 3D native generative model for the first time, enabling high-quality clothed human mesh reconstruction from single-view images even under challenging human poses. Multiview generative model excels at capturing fine 2D details but struggles with structural consistency, whereas 3D native generative model generates coarse yet structurally consistent 3D shapes. By integrating the complementary strengths of these two approaches, we develop a more effective generation framework. Specifically, we first jointly fine-tune the multiview generative model and the 3D native generative model with proposed pixel-aligned 2D-3D synchronization attention to produce geometrically aligned 3D shapes and 2D multiview images. To further improve details, we introduce a feature injection mechanism that lifts fine details from 2D multiview images onto the aligned 3D shapes, enabling accurate and high-fidelity reconstruction. Extensive experiments demonstrate that SyncHuman achieves robust and photo-realistic 3D human reconstruction, even for images with challenging poses. Our method outperforms baseline methods in geometric accuracy and visual fidelity, demonstrating a promising direction for future 3D generation models.

Method: Uses Surface Octahedral Probes (SOPs) for efficient lighting/occlusion storage and querying instead of ray tracing. For lighting estimation, focuses on environment lighting at object placement by capturing 360° radiance fields and fine-tuning diffusion models.

Result: Achieves 2x speedup in reconstruction, real-time shadow computation, and 28 FPS rendering. Provides visually harmonious results with vivid shadows and only 36 seconds for editing.

Conclusion: ComGS enables high-quality, real-time 3D object-scene composition with efficient relightable reconstruction and simplified lighting estimation, overcoming limitations of existing Gaussian-based methods.

Abstract: Gaussian Splatting (GS) enables immersive rendering, but realistic 3D object-scene composition remains challenging. Baked appearance and shadow information in GS radiance fields cause inconsistencies when combining objects and scenes. Addressing this requires relightable object reconstruction and scene lighting estimation. For relightable object reconstruction, existing Gaussian-based inverse rendering methods often rely on ray tracing, leading to low efficiency. We introduce Surface Octahedral Probes (SOPs), which store lighting and occlusion information and allow efficient 3D querying via interpolation, avoiding expensive ray tracing. SOPs provide at least a 2x speedup in reconstruction and enable real-time shadow computation in Gaussian scenes. For lighting estimation, existing Gaussian-based inverse rendering methods struggle to model intricate light transport and often fail in complex scenes, while learning-based methods predict lighting from a single image and are viewpoint-sensitive. We observe that 3D object-scene composition primarily concerns the object’s appearance and nearby shadows. Thus, we simplify the challenging task of full scene lighting estimation by focusing on the environment lighting at the object’s placement. Specifically, we capture a 360 degrees reconstructed radiance field of the scene at the location and fine-tune a diffusion model to complete the lighting. Building on these advances, we propose ComGS, a novel 3D object-scene composition framework. Our method achieves high-quality, real-time rendering at around 28 FPS, produces visually harmonious results with vivid shadows, and requires only 36 seconds for editing. Code and dataset are available at https://nju-3dv.github.io/projects/ComGS/.

Method: A two-stage framework: 1) exposure correction via ratio map to balance dynamic range, 2) brightness-aware RAW denoiser to enhance detail recovery in dark regions. Uses only single short-exposure RAW input to avoid ghosting and motion blur. Created a 9-stop bracketing pipeline to synthesize realistic UHDR training data.

Result: UltraLED significantly outperforms existing single-frame approaches in UHDR reconstruction, effectively recovering details in dark regions while preserving highlight information from the short-exposure RAW image.

Conclusion: Single short-exposure RAW images can effectively reconstruct UHDR scenes when processed through the proposed two-stage UltraLED framework, providing a robust solution that avoids ghosting artifacts common in multi-exposure methods.

Abstract: Ultra-high dynamic range (UHDR) scenes exhibit significant exposure disparities between bright and dark regions. Such conditions are commonly encountered in nighttime scenes with light sources. Even with standard exposure settings, a bimodal intensity distribution with boundary peaks often emerges, making it difficult to preserve both highlight and shadow details simultaneously. RGB-based bracketing methods can capture details at both ends using short-long exposure pairs, but are susceptible to misalignment and ghosting artifacts. We found that a short-exposure image already retains sufficient highlight detail. The main challenge of UHDR reconstruction lies in denoising and recovering information in dark regions. In comparison to the RGB images, RAW images, thanks to their higher bit depth and more predictable noise characteristics, offer greater potential for addressing this challenge. This raises a key question: can we learn to see everything in UHDR scenes using only a single short-exposure RAW image? In this study, we rely solely on a single short-exposure frame, which inherently avoids ghosting and motion blur, making it particularly robust in dynamic scenes. To achieve that, we introduce UltraLED, a two-stage framework that performs exposure correction via a ratio map to balance dynamic range, followed by a brightness-aware RAW denoiser to enhance detail recovery in dark regions. To support this setting, we design a 9-stop bracketing pipeline to synthesize realistic UHDR images and contribute a corresponding dataset based on diverse scenes, using only the shortest exposure as input for reconstruction. Extensive experiments show that UltraLED significantly outperforms existing single-frame approaches. Our code and dataset are made publicly available at https://srameo.github.io/projects/ultraled.

Method: Uses event motion priors to guide deformation field optimization. First extracts motion priors from event streams using LoCM unsupervised fine-tuning, then builds event-Gaussian motion correspondence through geometry-aware data association with motion decomposition and inter-frame pseudo-label strategies.

Result: Outperforms existing image and event-based approaches across synthetic and real scenes, effectively optimizing dynamic 3DGS with event data assistance.

Conclusion: The proposed framework successfully combines RGB and event modalities to reconstruct Dynamic 3DGS, demonstrating that event streams provide valuable deterministic constraints for handling large inter-frame motions.

Abstract: Reconstructing Dynamic 3D Gaussian Splatting (3DGS) from low-framerate RGB videos is challenging. This is because large inter-frame motions will increase the uncertainty of the solution space. For example, one pixel in the first frame might have more choices to reach the corresponding pixel in the second frame. Event cameras can asynchronously capture rapid visual changes and are robust to motion blur, but they do not provide color information. Intuitively, the event stream can provide deterministic constraints for the inter-frame large motion by the event trajectories. Hence, combining low-temporal-resolution images with high-framerate event streams can address this challenge. However, it is challenging to jointly optimize Dynamic 3DGS using both RGB and event modalities due to the significant discrepancy between these two data modalities. This paper introduces a novel framework that jointly optimizes dynamic 3DGS from the two modalities. The key idea is to adopt event motion priors to guide the optimization of the deformation fields. First, we extract the motion priors encoded in event streams by using the proposed LoCM unsupervised fine-tuning framework to adapt an event flow estimator to a certain unseen scene. Then, we present the geometry-aware data association method to build the event-Gaussian motion correspondence, which is the primary foundation of the pipeline, accompanied by two useful strategies, namely motion decomposition and inter-frame pseudo-label. Extensive experiments show that our method outperforms existing image and event-based approaches across synthetic and real scenes and prove that our method can effectively optimize dynamic 3DGS with the help of event data.

Method: Evaluated three deep learning models (UNet, ResUNet, AttUNet) on BraTS2020 dataset using Adam optimizer, with XAI techniques including Grad-CAM and attention-based visualization for model interpretability.

Result: ResUNet outperformed other models in final testing with highest Dice and Jaccard similarity scores, accuracy, recall, and F1 scores. XAI techniques provided insights into model focus areas and attention mechanisms.

Conclusion: ResUNet is recommended as the best-performing model for automated brain tumor segmentation in future clinical assessments, with XAI successfully enhancing model interpretability.

Abstract: The current study investigated the use of Explainable Artificial Intelligence (XAI) to improve the accuracy of brain tumor segmentation in MRI images, with the goal of assisting physicians in clinical decision-making. The study focused on applying UNet models for brain tumor segmentation and using the XAI techniques of Gradient-weighted Class Activation Mapping (Grad-CAM) and attention-based visualization to enhance the understanding of these models. Three deep learning models - UNet, Residual UNet (ResUNet), and Attention UNet (AttUNet) - were evaluated to identify the best-performing model. XAI was employed with the aims of clarifying model decisions and increasing physicians’ trust in these models. We compared the performance of two UNet variants (ResUNet and AttUNet) with the conventional UNet in segmenting brain tumors from the BraTS2020 public dataset and analyzed model predictions with Grad-CAM and attention-based visualization. Using the latest computer hardware, we trained and validated each model using the Adam optimizer and assessed their performance with respect to: (i) training, validation, and inference times, (ii) segmentation similarity coefficients and loss functions, and (iii) classification performance. Notably, during the final testing phase, ResUNet outperformed the other models with respect to Dice and Jaccard similarity scores, as well as accuracy, recall, and F1 scores. Grad-CAM provided visuospatial insights into the tumor subregions each UNet model focused on while attention-based visualization provided valuable insights into the working mechanisms of AttUNet’s attention modules. These results demonstrated ResUNet as the best-performing model and we conclude by recommending its use for automated brain tumor segmentation in future clinical assessments. Our source code and checkpoint are available at https://github.com/ethanong98/MultiModel-XAI-Brats2020

Method: Introduces GTR-Bench, a benchmark for geographic temporal reasoning of moving targets in large-scale camera networks, requiring perspective switches between maps and videos, joint reasoning across multiple non-overlapping videos, and inference over unobserved spatial-temporal regions.

Result: Evaluation of over 10 popular VLMs shows even the best model (Gemini-2.5-Pro) achieves only 34.9% accuracy, significantly lagging behind human performance (78.61%). Analysis reveals three main deficiencies: imbalanced spatial-temporal context utilization, weak temporal forecasting, and poor map-video comprehension/alignment.

Conclusion: GTR-Bench reveals significant gaps in current VLMs’ geo-temporal reasoning capabilities and provides valuable insights for future research in spatial-temporal intelligence, particularly for applications requiring joint reasoning across maps and multiple video streams.

Abstract: Recently spatial-temporal intelligence of Visual-Language Models (VLMs) has attracted much attention due to its importance for Autonomous Driving, Embodied AI and General Artificial Intelligence. Existing spatial-temporal benchmarks mainly focus on egocentric perspective reasoning with images/video context, or geographic perspective reasoning with graphics context (eg. a map), thus fail to assess VLMs’ geographic spatial-temporal intelligence with both images/video and graphics context, which is important for areas like traffic management and emergency response. To address the gaps, we introduce Geo-Temporal Reasoning benchmark (GTR-Bench), a novel challenge for geographic temporal reasoning of moving targets in a large-scale camera network. GTR-Bench is more challenging as it requires multiple perspective switches between maps and videos, joint reasoning across multiple videos with non-overlapping fields of view, and inference over spatial-temporal regions that are unobserved by any video context. Evaluations of more than 10 popular VLMs on GTR-Bench demonstrate that even the best proprietary model, Gemini-2.5-Pro (34.9%), significantly lags behind human performance (78.61%) on geo-temporal reasoning. Moreover, our comprehensive analysis on GTR-Bench reveals three primary deficiencies of current models for geo-temporal reasoning. (1) VLMs’ reasoning is impaired by an imbalanced utilization of spatial-temporal context. (2) VLMs are weak in temporal forecasting, which leads to worse performance on temporal-emphasized tasks than on spatial-emphasized tasks. (3) VLMs lack the proficiency to comprehend or align the map data with multi-view video inputs. We believe GTR-Bench offers valuable insights and opens up new opportunities for research and applications in spatial-temporal intelligence. Benchmark and code will be released at https://github.com/X-Luffy/GTR-Bench.

Method: Introduces MM-COF motion representation that combines optical flow from both micro-expression phases, and FMANet neural network with learnable modules for dual-phase analysis and magnitude modulation to adaptively fuse motion cues.

Result: Experimental evaluations on MMEW, SMIC, CASME-II, and SAMM datasets show that the proposed MM-COF representation and FMANet outperform existing methods.

Conclusion: The learnable dual-phase framework demonstrates significant potential for advancing micro-expression recognition by capturing comprehensive motion dynamics from both phases of micro-expressions.

Abstract: Facial micro-expressions, characterized by their subtle and brief nature, are valuable indicators of genuine emotions. Despite their significance in psychology, security, and behavioral analysis, micro-expression recognition remains challenging due to the difficulty of capturing subtle facial movements. Optical flow has been widely employed as an input modality for this task due to its effectiveness. However, most existing methods compute optical flow only between the onset and apex frames, thereby overlooking essential motion information in the apex-to-offset phase. To address this limitation, we first introduce a comprehensive motion representation, termed Magnitude-Modulated Combined Optical Flow (MM-COF), which integrates motion dynamics from both micro-expression phases into a unified descriptor suitable for direct use in recognition networks. Building upon this principle, we then propose FMANet, a novel end-to-end neural network architecture that internalizes the dual-phase analysis and magnitude modulation into learnable modules. This allows the network to adaptively fuse motion cues and focus on salient facial regions for classification. Experimental evaluations on the MMEW, SMIC, CASME-II, and SAMM datasets, widely recognized as standard benchmarks, demonstrate that our proposed MM-COF representation and FMANet outperforms existing methods, underscoring the potential of a learnable, dual-phase framework in advancing micro-expression recognition.

Method: Uses a shared feature encoder with task-specific decoders for layout estimation, depth estimation, and background segmentation, plus room geometry-based background depth resolving and segmentation-guided fusion strategies.

Result: Achieves significantly superior performance compared to current open-source methods on Stanford2D3D, Matterport3D and Structured3D datasets.

Conclusion: Integrating room geometry constraints through layout prediction and background segmentation effectively improves depth estimation for 360° indoor panoramas.

Abstract: Predicting spherical pixel depth from monocular $360^{\circ}$ indoor panoramas is critical for many vision applications. However, existing methods focus on pixel-level accuracy, causing oversmoothed room corners and noise sensitivity. In this paper, we propose a depth estimation framework based on room geometry constraints, which extracts room geometry information through layout prediction and integrates those information into the depth estimation process through background segmentation mechanism. At the model level, our framework comprises a shared feature encoder followed by task-specific decoders for layout estimation, depth estimation, and background segmentation. The shared encoder extracts multi-scale features, which are subsequently processed by individual decoders to generate initial predictions: a depth map, a room layout map, and a background segmentation map. Furthermore, our framework incorporates two strategies: a room geometry-based background depth resolving strategy and a background-segmentation-guided fusion mechanism. The proposed room-geometry-based background depth resolving strategy leverages the room layout and the depth decoder’s output to generate the corresponding background depth map. Then, a background-segmentation-guided fusion strategy derives fusion weights for the background and coarse depth maps from the segmentation decoder’s predictions. Extensive experimental results on the Stanford2D3D, Matterport3D and Structured3D datasets show that our proposed methods can achieve significantly superior performance than current open-source methods. Our code is available at https://github.com/emiyaning/RGCNet.

Method: Proposed ACAVP with complementary transformations: affine transformation for task-specific prompt regions while preserving original image information, and color transformation for emphasizing task-relevant features. Also introduced TrivialAugment as effective data augmentation to mitigate overfitting.

Result: Achieves state-of-the-art accuracy among VP methods, surpasses linear probing in average accuracy, and shows superior robustness to distribution shifts across twelve diverse image classification datasets with two model architectures. TrivialAugment improves existing VP methods by up to 12 percentage points.

Conclusion: ACAVP addresses key limitations of visual prompting through enhanced transformations and effective data augmentation, achieving improved performance while maintaining the computational efficiency benefits of parameter-efficient fine-tuning.

Abstract: Visual prompting (VP) has emerged as a promising parameter-efficient fine-tuning approach for adapting pre-trained vision models to downstream tasks without modifying model parameters. Despite offering advantages like negligible computational overhead and compatibility with black-box models, conventional VP methods typically achieve lower accuracy than other adaptation approaches. Our analysis reveals two critical limitations: the restricted expressivity of simple additive transformation and a tendency toward overfitting when the parameter count increases. To address these challenges, we propose ACAVP (Affine, Color, and Additive Visual Prompting), which enhances VP’s expressive power by introducing complementary transformation operations: affine transformation for creating task-specific prompt regions while preserving original image information, and color transformation for emphasizing task-relevant visual features. Additionally, we identify that overfitting is a critical issue in VP training and introduce TrivialAugment as an effective data augmentation, which not only benefits our approach but also significantly improves existing VP methods, with performance gains of up to 12 percentage points on certain datasets. This demonstrates that appropriate data augmentation is universally beneficial for VP training. Extensive experiments across twelve diverse image classification datasets with two different model architectures demonstrate that ACAVP achieves state-of-the-art accuracy among VP methods, surpasses linear probing in average accuracy, and exhibits superior robustness to distribution shifts, all while maintaining minimal computational overhead during inference.

Method: MMHOI-Net is an end-to-end transformer-based neural network that uses a structured dual-patch representation for modeling objects and their interactions, combined with action recognition to enhance interaction prediction.

Result: The approach achieves state-of-the-art performance on MMHOI and CORE4D datasets, excelling in both accuracy and reconstruction quality for multi-human object interaction modeling.

Conclusion: MMHOI dataset and MMHOI-Net framework provide a comprehensive testbed and effective solution for next-generation human-object interaction research, successfully addressing complex multi-human multi-object scenarios.

Abstract: Real-world scenes often feature multiple humans interacting with multiple objects in ways that are causal, goal-oriented, or cooperative. Yet existing 3D human-object interaction (HOI) benchmarks consider only a fraction of these complex interactions. To close this gap, we present MMHOI – a large-scale, Multi-human Multi-object Interaction dataset consisting of images from 12 everyday scenarios. MMHOI offers complete 3D shape and pose annotations for every person and object, along with labels for 78 action categories and 14 interaction-specific body parts, providing a comprehensive testbed for next-generation HOI research. Building on MMHOI, we present MMHOI-Net, an end-to-end transformer-based neural network for jointly estimating human-object 3D geometries, their interactions, and associated actions. A key innovation in our framework is a structured dual-patch representation for modeling objects and their interactions, combined with action recognition to enhance the interaction prediction. Experiments on MMHOI and the recently proposed CORE4D datasets demonstrate that our approach achieves state-of-the-art performance in multi-HOI modeling, excelling in both accuracy and reconstruction quality.

Method: PrismGS integrates two regularizers: 1) Pyramidal multi-scale supervision that enforces consistency by supervising rendering against a pre-filtered image pyramid to learn anti-aliased representations, and 2) Explicit size regularization that imposes a physically-grounded lower bound on 3D Gaussian dimensions to prevent degenerate primitives.

Result: Extensive experiments on MatrixCity, Mill-19, and UrbanScene3D show state-of-the-art performance with significant PSNR gains around 1.5 dB against CityGaussian, while maintaining superior quality and robustness under demanding 4K rendering.

Conclusion: PrismGS is a plug-and-play framework that improves 3DGS rendering fidelity in large urban scenes by addressing aliasing artifacts through physically-grounded regularization, achieving better performance while maintaining compatibility with existing pipelines.

Abstract: 3D Gaussian Splatting (3DGS) has recently enabled real-time photorealistic rendering in compact scenes, but scaling to large urban environments introduces severe aliasing artifacts and optimization instability, especially under high-resolution (e.g., 4K) rendering. These artifacts, manifesting as flickering textures and jagged edges, arise from the mismatch between Gaussian primitives and the multi-scale nature of urban geometry. While existing ``divide-and-conquer’’ pipelines address scalability, they fail to resolve this fidelity gap. In this paper, we propose PrismGS, a physically-grounded regularization framework that improves the intrinsic rendering behavior of 3D Gaussians. PrismGS integrates two synergistic regularizers. The first is pyramidal multi-scale supervision, which enforces consistency by supervising the rendering against a pre-filtered image pyramid. This compels the model to learn an inherently anti-aliased representation that remains coherent across different viewing scales, directly mitigating flickering textures. This is complemented by an explicit size regularization that imposes a physically-grounded lower bound on the dimensions of the 3D Gaussians. This prevents the formation of degenerate, view-dependent primitives, leading to more stable and plausible geometric surfaces and reducing jagged edges. Our method is plug-and-play and compatible with existing pipelines. Extensive experiments on MatrixCity, Mill-19, and UrbanScene3D demonstrate that PrismGS achieves state-of-the-art performance, yielding significant PSNR gains around 1.5 dB against CityGaussian, while maintaining its superior quality and robustness under demanding 4K rendering.

Method: AlignGS performs end-to-end optimization of geometry and semantics by distilling priors from 2D foundation models and using novel semantic-to-geometry guidance mechanisms including depth consistency and multi-faceted normal regularization.

Result: Extensive evaluations show state-of-the-art performance in novel view synthesis with superior geometric accuracy, producing more coherent and complete 3D models from limited input views.

Conclusion: Leveraging semantic priors as geometric regularizers enables more robust 3D reconstruction from sparse views, validating the effectiveness of active semantic guidance in geometric reconstruction.

Abstract: The demand for semantically rich 3D models of indoor scenes is rapidly growing, driven by applications in augmented reality, virtual reality, and robotics. However, creating them from sparse views remains a challenge due to geometric ambiguity. Existing methods often treat semantics as a passive feature painted on an already-formed, and potentially flawed, geometry. We posit that for robust sparse-view reconstruction, semantic understanding instead be an active, guiding force. This paper introduces AlignGS, a novel framework that actualizes this vision by pioneering a synergistic, end-to-end optimization of geometry and semantics. Our method distills rich priors from 2D foundation models and uses them to directly regularize the 3D representation through a set of novel semantic-to-geometry guidance mechanisms, including depth consistency and multi-faceted normal regularization. Extensive evaluations on standard benchmarks demonstrate that our approach achieves state-of-the-art results in novel view synthesis and produces reconstructions with superior geometric accuracy. The results validate that leveraging semantic priors as a geometric regularizer leads to more coherent and complete 3D models from limited input views. Our code is avaliable at https://github.com/MediaX-SJTU/AlignGS .

Method: Used self-supervised learning on the EmbryoNet dataset containing zebrafish embryo phenotypes from chemical compounds targeting different developmental processes.

Result: Learned representations effectively distinguish between modes-of-action of different compounds.

Conclusion: Self-supervised learning is suitable for toxicity testing automation and can be integrated into physical testing devices like TOXBOX.

Abstract: High-throughput toxicity testing offers a fast and cost-effective way to test large amounts of compounds. A key component for such systems is the automated evaluation via machine learning models. In this paper, we address critical challenges in this domain and demonstrate how representations learned via self-supervised learning can effectively identify toxicant-induced changes. We provide a proof-of-concept that utilizes the publicly available EmbryoNet dataset, which contains ten zebrafish embryo phenotypes elicited by various chemical compounds targeting different processes in early embryonic development. Our analysis shows that the learned representations using self-supervised learning are suitable for effectively distinguishing between the modes-of-action of different compounds. Finally, we discuss the integration of machine learning models in a physical toxicity testing device in the context of the TOXBOX project.

Method: Proposes a unified cylinder lifting method with: (1) Unified Cylinder Camera Modeling (UCCM) to handle different camera configurations with adjustable parameters, (2) Hybrid representation with Cylinder Plane Feature Group (CPFG) modules to lift 2D image features to 3D space.

Result: Achieves state-of-the-art performance under different evaluation settings and demonstrates zero-shot generalization to other driving scenes.

Conclusion: XYZCylinder effectively addresses generalization and accuracy limitations in driving scene reconstruction through unified camera modeling and improved feature lifting techniques.

Abstract: Recently, more attention has been paid to feedforward reconstruction paradigms, which mainly learn a fixed view transformation implicitly and reconstruct the scene with a single representation. However, their generalization capability and reconstruction accuracy are still limited while reconstructing driving scenes, which results from two aspects: (1) The fixed view transformation fails when the camera configuration changes, limiting the generalization capability across different driving scenes equipped with different camera configurations. (2) The small overlapping regions between sparse views of the $360^\circ$ panorama and the complexity of driving scenes increase the learning difficulty, reducing the reconstruction accuracy. To handle these difficulties, we propose \textbf{XYZCylinder}, a feedforward model based on a unified cylinder lifting method which involves camera modeling and feature lifting. Specifically, to improve the generalization capability, we design a Unified Cylinder Camera Modeling (UCCM) strategy, which avoids the learning of viewpoint-dependent spatial correspondence and unifies different camera configurations with adjustable parameters. To improve the reconstruction accuracy, we propose a hybrid representation with several dedicated modules based on newly designed Cylinder Plane Feature Group (CPFG) to lift 2D image features to 3D space. Experimental results show that XYZCylinder achieves state-of-the-art performance under different evaluation settings, and can be generalized to other driving scenes in a zero-shot manner. Project page: \href{https://yuyuyu223.github.io/XYZCYlinder-projectpage/}{here}.

Method: Proposes a retrieve-then-compress strategy using Visual Memory Retriever (VMR) to select key clips and Compression Group Relative Policy Optimization (C-GRPO) to distill reasoning ability from teacher to student model.

Result: Achieves near-baseline accuracy using only one frame’s tokens, reducing visual tokens by 95%, GPU memory by 72%, and latency by 23.9% on six video benchmarks.

Conclusion: MARC demonstrates potential for efficient, real-time video understanding in resource-constrained settings like video QA, surveillance, and autonomous driving.

Abstract: The rapid progress of large language models (LLMs) has laid the foundation for multimodal models. However, visual language models (VLMs) still face heavy computational costs when extended from images to videos due to high frame rates and long durations. Token compression is a promising solution, yet most existing training-free methods cause information loss and performance degradation. To overcome this, we propose \textbf{Memory-Augmented Reinforcement Learning-based Token Compression (MARC)}, which integrates structured retrieval and RL-based distillation. MARC adopts a \textit{retrieve-then-compress} strategy using a \textbf{Visual Memory Retriever (VMR)} to select key clips and a \textbf{Compression Group Relative Policy Optimization (C-GRPO)} framework to distil reasoning ability from a teacher to a student model. Experiments on six video benchmarks show that MARC achieves near-baseline accuracy using only one frame’s tokens – reducing visual tokens by \textbf{95%}, GPU memory by \textbf{72%}, and latency by \textbf{23.9%}. This demonstrates its potential for efficient, real-time video understanding in resource-constrained settings such as video QA, surveillance, and autonomous driving.

Method: Proposed ASBench framework with four evaluation dimensions: (1) generalization across datasets and pipelines, (2) ratio of synthetic to real data, (3) correlation between synthesis image metrics and detection performance, and (4) strategies for hybrid anomaly synthesis methods.

Result: Extensive experiments revealed limitations in current anomaly synthesis methods and provided actionable insights for future research directions.

Conclusion: ASBench serves as a comprehensive benchmarking tool that addresses critical gaps in anomaly synthesis evaluation and guides future development in this field.

Abstract: Anomaly detection plays a pivotal role in manufacturing quality control, yet its application is constrained by limited abnormal samples and high manual annotation costs. While anomaly synthesis offers a promising solution, existing studies predominantly treat anomaly synthesis as an auxiliary component within anomaly detection frameworks, lacking systematic evaluation of anomaly synthesis algorithms. Current research also overlook crucial factors specific to anomaly synthesis, such as decoupling its impact from detection, quantitative analysis of synthetic data and adaptability across different scenarios. To address these limitations, we propose ASBench, the first comprehensive benchmarking framework dedicated to evaluating anomaly synthesis methods. Our framework introduces four critical evaluation dimensions: (i) the generalization performance across different datasets and pipelines (ii) the ratio of synthetic to real data (iii) the correlation between intrinsic metrics of synthesis images and anomaly detection performance metrics , and (iv) strategies for hybrid anomaly synthesis methods. Through extensive experiments, ASBench not only reveals limitations in current anomaly synthesis methods but also provides actionable insights for future research directions in anomaly synthesis

Method: Fine-tune a spatial-temporal VAE with auxiliary 4D reconstruction task to enhance 3D structure and temporal dynamics encoding, then integrate this VAE into video diffusion process to improve generation quality.

Result: The model achieves substantial improvements in FID and FVD metrics, and the jointly-trained Gaussian Splatting Decoder effectively reconstructs dynamic scenes with valuable geometric information.

Conclusion: CVD-STORM successfully generates long-term, multi-view videos with 4D reconstruction capabilities, providing comprehensive scene understanding for autonomous driving applications.

Abstract: Generative models have been widely applied to world modeling for environment simulation and future state prediction. With advancements in autonomous driving, there is a growing demand not only for high-fidelity video generation under various controls, but also for producing diverse and meaningful information such as depth estimation. To address this, we propose CVD-STORM, a cross-view video diffusion model utilizing a spatial-temporal reconstruction Variational Autoencoder (VAE) that generates long-term, multi-view videos with 4D reconstruction capabilities under various control inputs. Our approach first fine-tunes the VAE with an auxiliary 4D reconstruction task, enhancing its ability to encode 3D structures and temporal dynamics. Subsequently, we integrate this VAE into the video diffusion process to significantly improve generation quality. Experimental results demonstrate that our model achieves substantial improvements in both FID and FVD metrics. Additionally, the jointly-trained Gaussian Splatting Decoder effectively reconstructs dynamic scenes, providing valuable geometric information for comprehensive scene understanding.

Method: Created AnimeText dataset with 735K images and 4.2M annotated text blocks, featuring hierarchical annotations and hard negative samples specifically designed for anime scenarios.

Result: Cross-dataset evaluations show models trained on AnimeText achieve superior performance in anime text detection tasks compared to models trained on existing datasets.

Conclusion: AnimeText effectively addresses the unique challenges of text detection in anime scenes and enables better performance for anime-specific text detection tasks.

Abstract: Current text detection datasets primarily target natural or document scenes, where text typically appear in regular font and shapes, monotonous colors, and orderly layouts. The text usually arranged along straight or curved lines. However, these characteristics differ significantly from anime scenes, where text is often diverse in style, irregularly arranged, and easily confused with complex visual elements such as symbols and decorative patterns. Text in anime scene also includes a large number of handwritten and stylized fonts. Motivated by this gap, we introduce AnimeText, a large-scale dataset containing 735K images and 4.2M annotated text blocks. It features hierarchical annotations and hard negative samples tailored for anime scenarios. %Cross-dataset evaluations using state-of-the-art methods demonstrate that models trained on AnimeText achieve superior performance in anime text detection tasks compared to existing datasets. To evaluate the robustness of AnimeText in complex anime scenes, we conducted cross-dataset benchmarking using state-of-the-art text detection methods. Experimental results demonstrate that models trained on AnimeText outperform those trained on existing datasets in anime scene text detection tasks. AnimeText on HuggingFace: https://huggingface.co/datasets/deepghs/AnimeText

Method: Proposed SimCast with short-to-long term knowledge distillation and weighted MSE loss to prioritize heavy rainfall regions. Then integrated SimCast into CasCast, a diffusion-based framework that combines deterministic predictions with probabilistic modeling strengths.

Result: Achieved mean CSI scores of 0.452 on SEVIR, 0.474 on HKO-7, and 0.361 on MeteoNet, significantly outperforming existing approaches without additional inference overhead.

Conclusion: The proposed framework effectively improves precipitation nowcasting performance by combining knowledge distillation techniques with diffusion-based probabilistic modeling, addressing key limitations of deterministic approaches while maintaining computational efficiency.

Abstract: Precipitation nowcasting predicts future radar sequences based on current observations, which is a highly challenging task driven by the inherent complexity of the Earth system. Accurate nowcasting is of utmost importance for addressing various societal needs, including disaster management, agriculture, transportation, and energy optimization. As a complementary to existing non-autoregressive nowcasting approaches, we investigate the impact of prediction horizons on nowcasting models and propose SimCast, a novel training pipeline featuring a short-to-long term knowledge distillation technique coupled with a weighted MSE loss to prioritize heavy rainfall regions. Improved nowcasting predictions can be obtained without introducing additional overhead during inference. As SimCast generates deterministic predictions, we further integrate it into a diffusion-based framework named CasCast, leveraging the strengths from probabilistic models to overcome limitations such as blurriness and distribution shift in deterministic outputs. Extensive experimental results on three benchmark datasets validate the effectiveness of the proposed framework, achieving mean CSI scores of 0.452 on SEVIR, 0.474 on HKO-7, and 0.361 on MeteoNet, which outperforms existing approaches by a significant margin.

Method: Two-stage approach: Stage One introduces LH-VAE with visual semantic constraints and progressive degradation perturbations; Stage Two jointly trains the refined VAE with restoration model using High-Frequency Low-Rank Adaptation (HF-LoRA) with alternating optimization and selective gradient propagation.

Result: Achieves state-of-the-art performance across UHD and standard-resolution tasks, effectively balancing efficiency, perceptual quality, and reconstruction accuracy with tunable fidelity-perception trade-offs.

Conclusion: Latent Harmony successfully redefines VAEs for UHD restoration by jointly optimizing latent space regularization and high-frequency reconstruction, offering a flexible and effective solution for high-quality image restoration.

Abstract: Ultra-High Definition (UHD) image restoration faces a trade-off between computational efficiency and high-frequency detail retention. While Variational Autoencoders (VAEs) improve efficiency via latent-space processing, their Gaussian constraint often discards degradation-specific high-frequency information, hurting reconstruction fidelity. To overcome this, we propose Latent Harmony, a two-stage framework that redefines VAEs for UHD restoration by jointly regularizing the latent space and enforcing high-frequency-aware reconstruction.In Stage One, we introduce LH-VAE, which enhances semantic robustness through visual semantic constraints and progressive degradation perturbations, while latent equivariance strengthens high-frequency reconstruction.Stage Two jointly trains this refined VAE with a restoration model using High-Frequency Low-Rank Adaptation (HF-LoRA): an encoder LoRA guided by a fidelity-oriented high-frequency alignment loss to recover authentic details, and a decoder LoRA driven by a perception-oriented loss to synthesize realistic textures. Both LoRA modules are trained via alternating optimization with selective gradient propagation to preserve the pretrained latent structure.At inference, a tunable parameter {\alpha} enables flexible fidelity-perception trade-offs.Experiments show Latent Harmony achieves state-of-the-art performance across UHD and standard-resolution tasks, effectively balancing efficiency, perceptual quality, and reconstruction accuracy.

Method: Explored and compared the effectiveness of object tags (denoting concrete entities) and abstract tags (capturing higher-level contextual information) for image privacy classification under different tag budget constraints.

Result: Abstract tags are more effective for privacy classification when the tag budget is limited, while object-related information becomes as useful when a larger number of tags per image is available.

Conclusion: The findings provide guidance for developing more accurate image privacy classifiers by considering the role of tag types and quantity, with abstract tags being preferable for limited budgets and both types being valuable when more tags are available.

Abstract: Object tags denote concrete entities and are central to many computer vision tasks, whereas abstract tags capture higher-level information, which is relevant for tasks that require a contextual, potentially subjective scene understanding. Object and abstract tags extracted from images also facilitate interpretability. In this paper, we explore which type of tags is more suitable for the context-dependent and inherently subjective task of image privacy. While object tags are generally used for privacy classification, we show that abstract tags are more effective when the tag budget is limited. Conversely, when a larger number of tags per image is available, object-related information is as useful. We believe that these findings will guide future research in developing more accurate image privacy classifiers, informed by the role of tag types and quantity.

Method: Comparative analysis between state-of-the-art SwinIR Transformer model and a standard lightweight CNN for simultaneous high-frequency detail restoration and noise suppression in low-light images.

Result: CNN achieved 37.4 dB PSNR vs SwinIR’s 39.03 dB, but converged in only 10 epochs (vs 132 for SwinIR) and was 55x smaller in model size.

Conclusion: Lightweight CNNs offer compelling near state-of-the-art performance with significantly lower computational overhead, making them suitable for resource-constrained real-world applications.

Abstract: The simultaneous restoration of high-frequency details and suppression of severe noise in low-light imagery presents a significant and persistent challenge in computer vision. While large-scale Transformer models like SwinIR have set the state of the art in performance, their high computational cost can be a barrier for practical applications. This paper investigates the critical trade-off between performance and efficiency by comparing the state-of-the-art SwinIR model against a standard, lightweight Convolutional Neural Network (CNN) on this challenging task. Our experimental results reveal a nuanced but important finding. While the Transformer-based SwinIR model achieves a higher peak performance, with a Peak Signal-to-Noise Ratio (PSNR) of 39.03 dB, the lightweight CNN delivers a surprisingly competitive PSNR of 37.4 dB. Crucially, the CNN reached this performance after converging in only 10 epochs of training, whereas the more complex SwinIR model required 132 epochs. This efficiency is further underscored by the model’s size; the CNN is over 55 times smaller than SwinIR. This work demonstrates that a standard CNN can provide a near state-of-the-art result with significantly lower computational overhead, presenting a compelling case for its use in real-world scenarios where resource constraints are a primary concern.

Method: Proposes GraphEnet - a Graph Neural Network that utilizes the sparse nature of event camera output with an intermediate line-based event representation. The architecture incorporates offset vector learning with confidence-based pooling.

Result: This is the first work that applies Graph Neural Networks to event data for Human Pose Estimation. The method achieves high-frequency 2D human pose estimation of a single person.

Conclusion: GraphEnet successfully demonstrates the application of Graph Neural Networks to event camera data for human pose estimation, providing a novel approach that leverages the advantages of event cameras while maintaining open-source availability.

Abstract: Human Pose Estimation is a crucial module in human-machine interaction applications and, especially since the rise in deep learning technology, robust methods are available to consumers using RGB cameras and commercial GPUs. On the other hand, event-based cameras have gained popularity in the vision research community for their low latency and low energy advantages that make them ideal for applications where those resources are constrained like portable electronics and mobile robots. In this work we propose a Graph Neural Network, GraphEnet, that leverages the sparse nature of event camera output, with an intermediate line based event representation, to estimate 2D Human Pose of a single person at a high frequency. The architecture incorporates a novel offset vector learning paradigm with confidence based pooling to estimate the human pose. This is the first work that applies Graph Neural Networks to event data for Human Pose Estimation. The code is open-source at https://github.com/event-driven-robotics/GraphEnet-NeVi-ICCV2025.

Method: Introduces CIR-CoT, an end-to-end retrieval-oriented MLLM that generates explicit reasoning chains before retrieval. Created structured CoT annotations (caption, reasoning, conclusion) and fine-tuned the model to produce this structured output, then encodes retrieval intent into embeddings.

Result: Achieves highly competitive performance on in-domain datasets (FashionIQ, CIRR) and demonstrates remarkable generalization on out-of-domain CIRCO dataset.

Conclusion: CIR-CoT establishes a new path toward more effective and trustworthy retrieval systems by integrating explicit reasoning for better accuracy and transparency.

Abstract: Composed Image Retrieval (CIR), which aims to find a target image from a reference image and a modification text, presents the core challenge of performing unified reasoning across visual and semantic modalities. While current approaches based on Vision-Language Models (VLMs, e.g., CLIP) and more recent Multimodal Large Language Models (MLLMs, e.g., Qwen-VL) have shown progress, they predominantly function as ``black boxes." This inherent opacity not only prevents users from understanding the retrieval rationale but also restricts the models’ ability to follow complex, fine-grained instructions. To overcome these limitations, we introduce CIR-CoT, the first end-to-end retrieval-oriented MLLM designed to integrate explicit Chain-of-Thought (CoT) reasoning. By compelling the model to first generate an interpretable reasoning chain, CIR-CoT enhances its ability to capture crucial cross-modal interactions, leading to more accurate retrieval while making its decision process transparent. Since existing datasets like FashionIQ and CIRR lack the necessary reasoning data, a key contribution of our work is the creation of structured CoT annotations using a three-stage process involving a caption, reasoning, and conclusion. Our model is then fine-tuned to produce this structured output before encoding its final retrieval intent into a dedicated embedding. Comprehensive experiments show that CIR-CoT achieves highly competitive performance on in-domain datasets (FashionIQ, CIRR) and demonstrates remarkable generalization on the out-of-domain CIRCO dataset, establishing a new path toward more effective and trustworthy retrieval systems.

Method: Proposes RayFusion, a ray-based fusion approach that leverages ray occupancy information from multiple collaborators to reduce redundancy and false positive predictions along camera rays.

Result: Comprehensive experiments show the method consistently outperforms existing state-of-the-art models and substantially advances collaborative visual perception performance.

Conclusion: RayFusion effectively enhances camera-based collaborative perception systems by mitigating depth estimation ambiguity through ray-based fusion of occupancy information.

Abstract: Collaborative visual perception methods have gained widespread attention in the autonomous driving community in recent years due to their ability to address sensor limitation problems. However, the absence of explicit depth information often makes it difficult for camera-based perception systems, e.g., 3D object detection, to generate accurate predictions. To alleviate the ambiguity in depth estimation, we propose RayFusion, a ray-based fusion method for collaborative visual perception. Using ray occupancy information from collaborators, RayFusion reduces redundancy and false positive predictions along camera rays, enhancing the detection performance of purely camera-based collaborative perception systems. Comprehensive experiments show that our method consistently outperforms existing state-of-the-art models, substantially advancing the performance of collaborative visual perception. The code is available at https://github.com/wangsh0111/RayFusion.

Method: Two-stage approach: 1) Discriminative Dual Prompt Tuning (DDPT) generates pseudo weak masks from slice-level labels; 2) Segmentation network with region-aware spatial attention using fixed location-based random embeddings to focus on anomalous regions.

Result: Achieves state-of-the-art performance on BraTS20, BraTS21, BraTS23, and MSD datasets, using less than 8 million parameters with significant computational complexity reduction.

Conclusion: RASALoRE provides an effective and efficient solution for weakly supervised brain anomaly detection, outperforming existing methods while maintaining low computational requirements.

Abstract: Weakly Supervised Anomaly detection (WSAD) in brain MRI scans is an important challenge useful to obtain quick and accurate detection of brain anomalies when precise pixel-level anomaly annotations are unavailable and only weak labels (e.g., slice-level) are available. In this work, we propose RASALoRE: Region Aware Spatial Attention with Location-based Random Embeddings, a novel two-stage WSAD framework. In the first stage, we introduce a Discriminative Dual Prompt Tuning (DDPT) mechanism that generates high-quality pseudo weak masks based on slice-level labels, serving as coarse localization cues. In the second stage, we propose a segmentation network with a region-aware spatial attention mechanism that relies on fixed location-based random embeddings. This design enables the model to effectively focus on anomalous regions. Our approach achieves state-of-the-art anomaly detection performance, significantly outperforming existing WSAD methods while utilizing less than 8 million parameters. Extensive evaluations on the BraTS20, BraTS21, BraTS23, and MSD datasets demonstrate a substantial performance improvement coupled with a significant reduction in computational complexity. Code is available at: https://github.com/BheeshmSharma/RASALoRE-BMVC-2025/.

Method: The framework uses two main modules: a visual critic that identifies differences between input and reference images, and a code generator that produces executable codes. It progressively enhances images in multi-step retouching similar to human workflow.

Result: Experiments show the approach generalizes well across diverse retouching styles and enables interpretable, controllable adjustments tailored to user intent through natural language interaction.

Conclusion: RetouchLLM provides a training-free, white-box solution for image retouching that offers interpretability, controllability, and adaptability without requiring large-scale paired training data.

Abstract: Image retouching not only enhances visual quality but also serves as a means of expressing personal preferences and emotions. However, existing learning-based approaches require large-scale paired data and operate as black boxes, making the retouching process opaque and limiting their adaptability to handle diverse, user- or image-specific adjustments. In this work, we propose RetouchLLM, a training-free white-box image retouching system, which requires no training data and performs interpretable, code-based retouching directly on high-resolution images. Our framework progressively enhances the image in a manner similar to how humans perform multi-step retouching, allowing exploration of diverse adjustment paths. It comprises of two main modules: a visual critic that identifies differences between the input and reference images, and a code generator that produces executable codes. Experiments demonstrate that our approach generalizes well across diverse retouching styles, while natural language-based user interaction enables interpretable and controllable adjustments tailored to user intent.

Method: Modified ResNet with additional branches for hierarchical classification, using hierarchy-penalty maps to prevent incoherent transitions. The architecture trains layers progressively from general to specific classes, exploiting class-hierarchy labels for efficient training.

Result: Experimental results on Amazonian Forest Sentinel-2 data show effective generalization across hierarchical levels and accurate micro-class classification on new target areas, with better representation of minority classes.

Conclusion: The hierarchical approach successfully improves classification accuracy at detailed levels while maintaining generalization, demonstrating the value of class-driven hierarchical modeling for time series image classification.

Abstract: This work presents a multitemporal class-driven hierarchical Residual Neural Network (ResNet) designed for modelling the classification of Time Series (TS) of multispectral images at different semantical class levels. The architecture consists of a modification of the ResNet where we introduce additional branches to perform the classification at the different hierarchy levels and leverage on hierarchy-penalty maps to discourage incoherent hierarchical transitions within the classification. In this way, we improve the discrimination capabilities of classes at different levels of semantic details and train a modular architecture that can be used as a backbone network for introducing new specific classes and additional tasks considering limited training samples available. We exploit the class-hierarchy labels to train efficiently the different layers of the architecture, allowing the first layers to train faster on the first levels of the hierarchy modeling general classes (i.e., the macro-classes) and the intermediate classes, while using the last ones to discriminate more specific classes (i.e., the micro-classes). In this way, the targets are constrained in following the hierarchy defined, improving the classification of classes at the most detailed level. The proposed modular network has intrinsic adaptation capability that can be obtained through fine tuning. The experimental results, obtained on two tiles of the Amazonian Forest on 12 monthly composites of Sentinel 2 images acquired during 2019, demonstrate the effectiveness of the hierarchical approach in both generalizing over different hierarchical levels and learning discriminant features for an accurate classification at the micro-class level on a new target area, with a better representation of the minoritarian classes.

Method: Created RedFace dataset using 9 commercial online platforms to generate deepfakes, employing bespoke algorithms to capture diverse real-world manipulation techniques and simulate black-box scenarios.

Result: Extensive experiments (cross-domain, intra-domain, social network simulations) show limited practicality of existing deepfake detection schemes against real-world applications, with detailed analysis revealing performance gaps compared to conventional datasets.

Conclusion: RedFace bridges the gap between academic evaluations and real-world deepfake threats, demonstrating the need for detection methods that can handle commercial deepfake technologies actually used in practice.

Abstract: Deepfakes, leveraging advanced AIGC (Artificial Intelligence-Generated Content) techniques, create hyper-realistic synthetic images and videos of human faces, posing a significant threat to the authenticity of social media. While this real-world threat is increasingly prevalent, existing academic evaluations and benchmarks for detecting deepfake forgery often fall short to achieve effective application for their lack of specificity, limited deepfake diversity, restricted manipulation techniques.To address these limitations, we introduce RedFace (Real-world-oriented Deepfake Face), a specialized facial deepfake dataset, comprising over 60,000 forged images and 1,000 manipulated videos derived from authentic facial features, to bridge the gap between academic evaluations and real-world necessity. Unlike prior benchmarks, which typically rely on academic methods to generate deepfakes, RedFace utilizes 9 commercial online platforms to integrate the latest deepfake technologies found “in the wild”, effectively simulating real-world black-box scenarios.Moreover, RedFace’s deepfakes are synthesized using bespoke algorithms, allowing it to capture diverse and evolving methods used by real-world deepfake creators. Extensive experimental results on RedFace (including cross-domain, intra-domain, and real-world social network dissemination simulations) verify the limited practicality of existing deepfake detection schemes against real-world applications. We further perform a detailed analysis of the RedFace dataset, elucidating the reason of its impact on detection performance compared to conventional datasets. Our dataset is available at: https://github.com/kikyou-220/RedFace.

Method: Uses Normalized Spatiotemporal Gradient (NSG) statistic to quantify ratio of spatial probability gradients to temporal density changes. Leverages pre-trained diffusion models for NSG estimation with spatial gradient approximation and motion-aware temporal modeling without complex motion decomposition.

Result: NSG-VD outperforms state-of-the-art baselines by 16.00% in Recall and 10.75% in F1-Score. Derived theoretical upper bound showing generated videos exhibit amplified NSG feature discrepancies due to distributional shifts.

Conclusion: Proposed physics-driven detection paradigm effectively identifies AI-generated videos by capturing deviations from natural video dynamics through NSG features, with superior performance over existing methods.

Abstract: AI-generated videos have achieved near-perfect visual realism (e.g., Sora), urgently necessitating reliable detection mechanisms. However, detecting such videos faces significant challenges in modeling high-dimensional spatiotemporal dynamics and identifying subtle anomalies that violate physical laws. In this paper, we propose a physics-driven AI-generated video detection paradigm based on probability flow conservation principles. Specifically, we propose a statistic called Normalized Spatiotemporal Gradient (NSG), which quantifies the ratio of spatial probability gradients to temporal density changes, explicitly capturing deviations from natural video dynamics. Leveraging pre-trained diffusion models, we develop an NSG estimator through spatial gradients approximation and motion-aware temporal modeling without complex motion decomposition while preserving physical constraints. Building on this, we propose an NSG-based video detection method (NSG-VD) that computes the Maximum Mean Discrepancy (MMD) between NSG features of the test and real videos as a detection metric. Last, we derive an upper bound of NSG feature distances between real and generated videos, proving that generated videos exhibit amplified discrepancies due to distributional shifts. Extensive experiments confirm that NSG-VD outperforms state-of-the-art baselines by 16.00% in Recall and 10.75% in F1-Score, validating the superior performance of NSG-VD. The source code is available at https://github.com/ZSHsh98/NSG-VD.

Method: Proposes DarkHash with a shadow backdoor attack framework using dual-semantic guidance. Fine-tunes only specific layers of victim models using surrogate datasets, and uses topological alignment loss to optimize both individual and neighboring poisoned samples toward target samples.

Result: Experimental results on four image datasets, five model architectures, and two hashing methods show DarkHash outperforms existing state-of-the-art backdoor attack methods with high effectiveness. It also withstands existing mainstream backdoor defense methods.

Conclusion: DarkHash successfully demonstrates the feasibility of data-free backdoor attacks on deep hashing models, achieving strong attack performance while maintaining original retrieval accuracy and being resistant to current defense methods.

Abstract: Benefiting from its superior feature learning capabilities and efficiency, deep hashing has achieved remarkable success in large-scale image retrieval. Recent studies have demonstrated the vulnerability of deep hashing models to backdoor attacks. Although these studies have shown promising attack results, they rely on access to the training dataset to implant the backdoor. In the real world, obtaining such data (e.g., identity information) is often prohibited due to privacy protection and intellectual property concerns. Embedding backdoors into deep hashing models without access to the training data, while maintaining retrieval accuracy for the original task, presents a novel and challenging problem. In this paper, we propose DarkHash, the first data-free backdoor attack against deep hashing. Specifically, we design a novel shadow backdoor attack framework with dual-semantic guidance. It embeds backdoor functionality and maintains original retrieval accuracy by fine-tuning only specific layers of the victim model using a surrogate dataset. We consider leveraging the relationship between individual samples and their neighbors to enhance backdoor attacks during training. By designing a topological alignment loss, we optimize both individual and neighboring poisoned samples toward the target sample, further enhancing the attack capability. Experimental results on four image datasets, five model architectures, and two hashing methods demonstrate the high effectiveness of DarkHash, outperforming existing state-of-the-art backdoor attack methods. Defense experiments show that DarkHash can withstand existing mainstream backdoor defense methods.

Method: Jointly fit two 3DGS models - one to RGB images and one to initial segmentation maps - enforcing multiview consistency through shared geometry to synthesize pose-diverse training data with minimal post-processing.

Result: Fine-tuning on the refined dataset significantly improves accuracy on challenging head poses while maintaining strong performance on standard views, achieving superior results compared to state-of-the-art methods.

Conclusion: The method offers a scalable and effective solution for improving face parsing robustness in real-world settings without requiring ground-truth 3D annotations and using only a small set of initial images.

Abstract: Accurate face parsing under extreme viewing angles remains a significant challenge due to limited labeled data in such poses. Manual annotation is costly and often impractical at scale. We propose a novel label refinement pipeline that leverages 3D Gaussian Splatting (3DGS) to generate accurate segmentation masks from noisy multiview predictions. By jointly fitting two 3DGS models, one to RGB images and one to their initial segmentation maps, our method enforces multiview consistency through shared geometry, enabling the synthesis of pose-diverse training data with only minimal post-processing. Fine-tuning a face parsing model on this refined dataset significantly improves accuracy on challenging head poses, while maintaining strong performance on standard views. Extensive experiments, including human evaluations, demonstrate that our approach achieves superior results compared to state-of-the-art methods, despite requiring no ground-truth 3D annotations and using only a small set of initial images. Our method offers a scalable and effective solution for improving face parsing robustness in real-world settings.

Method: Random windowing augmentation that exploits the HU distribution in CT images to encourage robustness to contrast-enhancement variations and handle challenging images with poor contrast or timing.

Result: Significantly increases model performance on challenging CT images, outperforms state-of-the-art alternatives in liver tumor segmentation across multiple datasets.

Conclusion: CT-specific augmentation methods like Random windowing are essential for achieving good generalization in medical image analysis, as standard intensity augmentations are unsuitable for CT modality.

Abstract: Contrast-enhanced Computed Tomography (CT) is important for diagnosis and treatment planning for various medical conditions. Deep learning (DL) based segmentation models may enable automated medical image analysis for detecting and delineating tumors in CT images, thereby reducing clinicians’ workload. Achieving generalization capabilities in limited data domains, such as radiology, requires modern DL models to be trained with image augmentation. However, naively applying augmentation methods developed for natural images to CT scans often disregards the nature of the CT modality, where the intensities measure Hounsfield Units (HU) and have important physical meaning. This paper challenges the use of such intensity augmentations for CT imaging and shows that they may lead to artifacts and poor generalization. To mitigate this, we propose a CT-specific augmentation technique, called Random windowing, that exploits the available HU distribution of intensities in CT images. Random windowing encourages robustness to contrast-enhancement and significantly increases model performance on challenging images with poor contrast or timing. We perform ablations and analysis of our method on multiple datasets, and compare to, and outperform, state-of-the-art alternatives, while focusing on the challenge of liver tumor segmentation.

Method: Fine-tunes a base image-to-video model for basic motion control, then enhances it via reinforcement learning with trajectory-based rewards. Uses Self-Rollout mechanism to preserve Markov property and selective stochasticity in denoising steps for training acceleration.

Result: Achieves high visual fidelity and precise motion alignment with only 1.3B parameters, significantly reducing latency compared to state-of-the-art motion-controllable video diffusion models.

Conclusion: AR-Drag successfully enables real-time motion-controllable video generation with improved quality and reduced latency through RL-enhanced autoregressive diffusion approach.

Abstract: Real-time motion-controllable video generation remains challenging due to the inherent latency of bidirectional diffusion models and the lack of effective autoregressive (AR) approaches. Existing AR video diffusion models are limited to simple control signals or text-to-video generation, and often suffer from quality degradation and motion artifacts in few-step generation. To address these challenges, we propose AR-Drag, the first RL-enhanced few-step AR video diffusion model for real-time image-to-video generation with diverse motion control. We first fine-tune a base I2V model to support basic motion control, then further improve it via reinforcement learning with a trajectory-based reward model. Our design preserves the Markov property through a Self-Rollout mechanism and accelerates training by selectively introducing stochasticity in denoising steps. Extensive experiments demonstrate that AR-Drag achieves high visual fidelity and precise motion alignment, significantly reducing latency compared with state-of-the-art motion-controllable VDMs, while using only 1.3B parameters. Additional visualizations can be found on our project page: https://kesenzhao.github.io/AR-Drag.github.io/.

Method: The framework explores condition injection strategies, training schemes, and data mixture techniques within a latent video diffusion model, with distinct data construction and condition utilization methods to enable precise use of all condition types.

Result: UniMMVSR significantly outperforms existing methods, producing videos with superior detail and higher conformity to multi-modal conditions, and enables multi-modal guided generation of 4K video when combined with a base model.

Conclusion: The study presents the first unified generative video super-resolution framework that successfully incorporates hybrid-modal conditions, achieving superior performance and enabling previously unattainable 4K video generation capabilities.

Abstract: Cascaded video super-resolution has emerged as a promising technique for decoupling the computational burden associated with generating high-resolution videos using large foundation models. Existing studies, however, are largely confined to text-to-video tasks and fail to leverage additional generative conditions beyond text, which are crucial for ensuring fidelity in multi-modal video generation. We address this limitation by presenting UniMMVSR, the first unified generative video super-resolution framework to incorporate hybrid-modal conditions, including text, images, and videos. We conduct a comprehensive exploration of condition injection strategies, training schemes, and data mixture techniques within a latent video diffusion model. A key challenge was designing distinct data construction and condition utilization methods to enable the model to precisely utilize all condition types, given their varied correlations with the target video. Our experiments demonstrate that UniMMVSR significantly outperforms existing methods, producing videos with superior detail and a higher degree of conformity to multi-modal conditions. We also validate the feasibility of combining UniMMVSR with a base model to achieve multi-modal guided generation of 4K video, a feat previously unattainable with existing techniques.

Method: Proposes MURE framework with interleaved text-image CoT reasoning, where textual descriptions are followed by visual cues like positional masks. Introduces Multimodal Deep Confidence (MMDC) paradigm that explores multiple visual reasoning paths and prunes low-quality branches using reward model scores.

Result: Method decomposes complex editing tasks into interdependent sub-tasks, achieving greater precision at each stage. Created CoT-Edit-14K dataset with 14K editing examples. Shows significant improvements across three image editing benchmarks.

Conclusion: MURE successfully shifts visual editing from text-based reasoning to multimodal reasoning, enabling more precise and high-fidelity image editing results through interleaved text-image chains and confidence-based path selection.

Abstract: Image editing with natural language has gained significant popularity, yet existing methods struggle with intricate object intersections and fine-grained spatial relationships due to the lack of an explicit reasoning process. While Chain-of-Thought (CoT) has been explored to enhance reasoning, purely textual CoT or CoT augmented with coordinate information is fundamentally limited in its ability to represent intricate visual layouts and lacks the necessary visual cues to guide the generation of fine-grained, pixel-level details. To address these challenges, we propose Multimodal Reasoning Edit (MURE), a novel framework that shifts the visual editing process from purely text-based reasoning to a series of interleaved textual and visual rationales. Our framework performs image editing using a natively multimodal, interleaved text-image CoT. This approach generates a step-by-step chain of reasoning where a textual description is followed by a corresponding visual cue, such as a positional mask that defined intended edited regions or a representation of new content. Furthermore, to mitigate the hallucination phenomenon of large language models, we introduce Multimodal Deep Confidence (MMDC) reasoning paradigm. This paradigm explores a tree of visual reasoning paths at each step. By pruning low-quality branches using a deep confidence score from a reward model, it ensures the model consistently follows a high-quality trajectory towards the final edited result. The proposed method decomposes complex editing tasks into interdependent sub-tasks, achieving greater precision at each stage and yielding high-fidelity edited results. We define the formulation for interleaved text-image chains and release the first CoT-Edit-14K dataset, comprising 14K high-quality editing examples. Extensive experiments show that our method yields significant improvements across three image editing benchmarks.

Method: Proposed a bootstrapping algorithm that iteratively re-aligns training samples by progressively reducing variance and recovering the alignment assumption, with convergence guarantees for arbitrary compact groups.

Result: The method consistently outperforms equivariant and canonicalization baselines on four FGVC benchmarks while performing on par with augmentation-based approaches.

Conclusion: The bootstrapping approach provides an effective alternative to existing methods for handling geometric biases in fine-grained visual classification tasks.

Abstract: Fine-grained visual classification (FGVC) tasks, such as insect and bird identification, demand sensitivity to subtle visual cues while remaining robust to spatial transformations. A key challenge is handling geometric biases and noise, such as different orientations and scales of objects. Existing remedies rely on heavy data augmentation, which demands powerful models, or on equivariant architectures, which constrain expressivity and add cost. Canonicalization offers an alternative by shielding such biases from the downstream model. In practice, such functions are often obtained using canonicalization priors, which assume aligned training data. Unfortunately, real-world datasets never fulfill this assumption, causing the obtained canonicalizer to be brittle. We propose a bootstrapping algorithm that iteratively re-aligns training samples by progressively reducing variance and recovering the alignment assumption. We establish convergence guarantees under mild conditions for arbitrary compact groups, and show on four FGVC benchmarks that our method consistently outperforms equivariant, and canonicalization baselines while performing on par with augmentation.

Method: The framework treats object dragging as image reconstruction with two synergistic branches: moving-reconstruction branch uses energy-based gradient guidance and refined cross-attention maps for accurate relocation, and text-driven editing branch shares gradient signals for consistent transformations. It also employs DDPM inversion and prior information injection to preserve object structure.

Result: Extensive experiments show that InstructUDrag enables flexible, high-fidelity image editing with precision in object relocation and semantic control over image content.

Conclusion: The proposed framework successfully combines text instructions with object dragging to achieve simultaneous precise object positioning and semantic editing capabilities.

Abstract: Text-to-image diffusion models have shown great potential for image editing, with techniques such as text-based and object-dragging methods emerging as key approaches. However, each of these methods has inherent limitations: text-based methods struggle with precise object positioning, while object dragging methods are confined to static relocation. To address these issues, we propose InstructUDrag, a diffusion-based framework that combines text instructions with object dragging, enabling simultaneous object dragging and text-based image editing. Our framework treats object dragging as an image reconstruction process, divided into two synergistic branches. The moving-reconstruction branch utilizes energy-based gradient guidance to move objects accurately, refining cross-attention maps to enhance relocation precision. The text-driven editing branch shares gradient signals with the reconstruction branch, ensuring consistent transformations and allowing fine-grained control over object attributes. We also employ DDPM inversion and inject prior information into noise maps to preserve the structure of moved objects. Extensive experiments demonstrate that InstructUDrag facilitates flexible, high-fidelity image editing, offering both precision in object relocation and semantic control over image content.

Method: Three-stage approach: 1) Self-Learning Stage divides dual-human text into individual texts using LLM for individual-level alignment; 2) Adaptive Adjustment Stage predicts interaction distance and models interactions dynamically using graph network; 3) Teacher-Guided Refinement Stage uses overall text features to refine motion at overall level.

Result: Extensive evaluations on dual-human motion datasets demonstrate that FineDual outperforms existing approaches and effectively models dynamic hierarchical human interaction.

Conclusion: FineDual successfully generates fine-grained and high-quality dual-human motion by explicitly modeling the dynamic hierarchical nature of human interactions.

Abstract: Human interaction is inherently dynamic and hierarchical, where the dynamic refers to the motion changes with distance, and the hierarchy is from individual to inter-individual and ultimately to overall motion. Exploiting these properties is vital for dual-human motion generation, while existing methods almost model human interaction temporally invariantly, ignoring distance and hierarchy. To address it, we propose a fine-grained dual-human motion generation method, namely FineDual, a tri-stage method to model the dynamic hierarchical interaction from individual to inter-individual. The first stage, Self-Learning Stage, divides the dual-human overall text into individual texts through a Large Language Model, aligning text features and motion features at the individual level. The second stage, Adaptive Adjustment Stage, predicts interaction distance by an interaction distance predictor, modeling human interactions dynamically at the inter-individual level by an interaction-aware graph network. The last stage, Teacher-Guided Refinement Stage, utilizes overall text features as guidance to refine motion features at the overall level, generating fine-grained and high-quality dual-human motion. Extensive quantitative and qualitative evaluations on dual-human motion datasets demonstrate that our proposed FineDual outperforms existing approaches, effectively modeling dynamic hierarchical human interaction.

Method: AdaGC framework with gradient calibration mechanism combined with Mixup and dual EMA module for robust pseudo-label generation. Uses adaptive triggering based on training dynamics after warm-up stage.

Result: Achieves state-of-the-art performance on two benchmark RS datasets under two distinct label noise types while maintaining strong robustness across diverse settings.

Conclusion: AdaGC effectively bridges the gap in SPML research for remote sensing by providing a generalizable framework that mitigates overfitting to label noise through adaptive gradient calibration.

Abstract: Multi-label classification (MLC) offers a more comprehensive semantic understanding of Remote Sensing (RS) imagery compared to traditional single-label classification (SLC). However, obtaining complete annotations for MLC is particularly challenging due to the complexity and high cost of the labeling process. As a practical alternative, single-positive multi-label learning (SPML) has emerged, where each image is annotated with only one relevant label, and the model is expected to recover the full set of labels. While scalable, SPML introduces significant supervision ambiguity, demanding specialized solutions for model training. Although various SPML methods have been proposed in the computer vision domain, research in the RS context remains limited. To bridge this gap, we propose Adaptive Gradient Calibration (AdaGC), a novel and generalizable SPML framework tailored to RS imagery. AdaGC adopts a gradient calibration (GC) mechanism combined with Mixup and a dual exponential moving average (EMA) module for robust pseudo-label generation. To maximize AdaGC’s effectiveness, we introduce a simple yet theoretically grounded indicator to adaptively trigger GC after an initial warm-up stage based on training dynamics, thereby guaranteeing the effectiveness of GC in mitigating overfitting to label noise. Extensive experiments on two benchmark RS datasets under two distinct label noise types demonstrate that AdaGC achieves state-of-the-art (SOTA) performance while maintaining strong robustness across diverse settings.

Method: Proposes null-text-null frequency-aware diffusion model that decomposes semantics consistency into frequency band consistencies. Divides denoising into early (high-level noise) and late (low-level noise) stages, using mid-frequency band as guidance for null-text denoising of low-frequency band, followed by text-guided denoising.

Result: Extensive experiments show NTN-Diff outperforms state-of-the-art diffusion models for text-guided image inpainting.

Conclusion: NTN-Diff successfully addresses both preservation of unmasked regions and semantics consistency between masked/unmasked regions through frequency-aware decomposition and staged denoising process.

Abstract: Text-guided image inpainting aims at reconstructing the masked regions as per text prompts, where the longstanding challenges lie in the preservation for unmasked regions, while achieving the semantics consistency between unmasked and inpainted masked regions. Previous arts failed to address both of them, always with either of them to be remedied. Such facts, as we observed, stem from the entanglement of the hybrid (e.g., mid-and-low) frequency bands that encode varied image properties, which exhibit different robustness to text prompts during the denoising process. In this paper, we propose a null-text-null frequency-aware diffusion models, dubbed \textbf{NTN-Diff}, for text-guided image inpainting, by decomposing the semantics consistency across masked and unmasked regions into the consistencies as per each frequency band, while preserving the unmasked regions, to circumvent two challenges in a row. Based on the diffusion process, we further divide the denoising process into early (high-level noise) and late (low-level noise) stages, where the mid-and-low frequency bands are disentangled during the denoising process. As observed, the stable mid-frequency band is progressively denoised to be semantically aligned during text-guided denoising process, which, meanwhile, serves as the guidance to the null-text denoising process to denoise low-frequency band for the masked regions, followed by a subsequent text-guided denoising process at late stage, to achieve the semantics consistency for mid-and-low frequency bands across masked and unmasked regions, while preserve the unmasked regions. Extensive experiments validate the superiority of NTN-Diff over the state-of-the-art diffusion models to text-guided diffusion models. Our code can be accessed from https://github.com/htyjers/NTN-Diff.

Method: Proposes ERU framework with three key components: LLM-based data augmentation for training data, depth-map modality for spatial understanding, and depth-aware decision module for robust cue integration.

Result: Experimental results on two datasets show significant performance improvements over existing baselines, achieving more accurate and reliable referent detection.

Conclusion: The proposed framework effectively addresses ambiguity in embodied reference understanding by jointly leveraging multiple modalities and demonstrates superior performance compared to previous approaches.

Abstract: Embodied Reference Understanding requires identifying a target object in a visual scene based on both language instructions and pointing cues. While prior works have shown progress in open-vocabulary object detection, they often fail in ambiguous scenarios where multiple candidate objects exist in the scene. To address these challenges, we propose a novel ERU framework that jointly leverages LLM-based data augmentation, depth-map modality, and a depth-aware decision module. This design enables robust integration of linguistic and embodied cues, improving disambiguation in complex or cluttered environments. Experimental results on two datasets demonstrate that our approach significantly outperforms existing baselines, achieving more accurate and reliable referent detection.

Method: Proposes learning a neural field that predicts optimal exposure value per 3D point, with joint optimization of scene representation and exposure field via a novel neural conditioning mechanism.

Result: Trains faster than prior works and achieves state-of-the-art results on several benchmarks, improving by over 55% over best-performing baselines.

Conclusion: The approach enables accurate view synthesis in high dynamic range scenarios without needing post-processing or multi-exposure captures, producing superior performance on challenging real-world data.

Abstract: Recent advances in neural scene representations have led to unprecedented quality in 3D reconstruction and view synthesis. Despite achieving high-quality results for common benchmarks with curated data, outputs often degrade for data that contain per image variations such as strong exposure changes, present, e.g., in most scenes with indoor and outdoor areas or rooms with windows. In this paper, we introduce Neural Exposure Fields (NExF), a novel technique for robustly reconstructing 3D scenes with high quality and 3D-consistent appearance from challenging real-world captures. In the core, we propose to learn a neural field predicting an optimal exposure value per 3D point, enabling us to optimize exposure along with the neural scene representation. While capture devices such as cameras select optimal exposure per image/pixel, we generalize this concept and perform optimization in 3D instead. This enables accurate view synthesis in high dynamic range scenarios, bypassing the need of post-processing steps or multi-exposure captures. Our contributions include a novel neural representation for exposure prediction, a system for joint optimization of the scene representation and the exposure field via a novel neural conditioning mechanism, and demonstrated superior performance on challenging real-world data. We find that our approach trains faster than prior works and produces state-of-the-art results on several benchmarks improving by over 55% over best-performing baselines.

Method: Uses stacked sparse local attentions with dilated window attention across frames, plus random global key selection and global query interactions to encode global context.

Result: Achieves new state-of-the-art on four benchmarks with 11.3% and 8.3% improvements on MeViS valu and val datasets respectively.

Conclusion: LTCA effectively aggregates global temporal context while maintaining computational efficiency, demonstrating superior performance in referring video segmentation.

Abstract: Referring Video Segmentation (RVOS) aims to segment objects in videos given linguistic expressions. The key to solving RVOS is to extract long-range temporal context information from the interactions of expressions and videos to depict the dynamic attributes of each object. Previous works either adopt attention across all the frames or stack dense local attention to achieve a global view of temporal context. However, they fail to strike a good balance between locality and globality, and the computation complexity significantly increases with the increase of video length. In this paper, we propose an effective long-range temporal context attention (LTCA) mechanism to aggregate global context information into object features. Specifically, we aggregate the global context information from two aspects. Firstly, we stack sparse local attentions to balance the locality and globality. We design a dilated window attention across frames to aggregate local context information and perform such attention in a stack of layers to enable a global view. Further, we enable each query to attend to a small group of keys randomly selected from a global pool to enhance the globality. Secondly, we design a global query to interact with all the other queries to directly encode the global context information. Experiments show our method achieves new state-of-the-art on four referring video segmentation benchmarks. Notably, our method shows an improvement of 11.3% and 8.3% on the MeViS valu and val datasets respectively.

Method: Proposes Cross-Modal Affinity Transfer (CMAT) pre-training to align 3D encoder with 2D semantics, and Cross-modal Affordance Segmentation Transformer (CAST) that integrates multi-modal prompts with CMAT-pretrained features.

Result: Extensive experiments show the framework establishes new state-of-the-art results for 3D affordance segmentation on standard benchmarks.

Conclusion: The semantic-grounded learning paradigm successfully transfers 2D semantic knowledge to 3D domain, enabling precise and prompt-aware affordance segmentation.

Abstract: Affordance segmentation aims to parse 3D objects into functionally distinct parts, bridging recognition and interaction for applications in robotic manipulation, embodied AI, and AR. While recent studies leverage visual or textual prompts to guide this process, they often rely on point cloud encoders as generic feature extractors, overlooking the intrinsic challenges of 3D data such as sparsity, noise, and geometric ambiguity. As a result, 3D features learned in isolation frequently lack clear and semantically consistent functional boundaries. To address this bottleneck, we propose a semantic-grounded learning paradigm that transfers rich semantic knowledge from large-scale 2D Vision Foundation Models (VFMs) into the 3D domain. Specifically, We introduce Cross-Modal Affinity Transfer (CMAT), a pre-training strategy that aligns a 3D encoder with lifted 2D semantics and jointly optimizes reconstruction, affinity, and diversity to yield semantically organized representations. Building on this backbone, we further design the Cross-modal Affordance Segmentation Transformer (CAST), which integrates multi-modal prompts with CMAT-pretrained features to generate precise, prompt-aware segmentation maps. Extensive experiments on standard benchmarks demonstrate that our framework establishes new state-of-the-art results for 3D affordance segmentation.

Method: Proposes selective transfer to automatically identify and replace self-attention layers with linear attention, and introduces anytime distribution matching (ADM) objective to align sample distributions across timesteps efficiently.

Result: Achieves 1.25-2.00x speedup while preserving generation quality, with 4-step distilled model delivering 15.92x latency reduction with minimal visual quality drop.

Conclusion: LinVideo provides an effective data-free post-training solution for accelerating video diffusion models by replacing self-attention with linear attention while maintaining performance.

Abstract: Video diffusion models (DMs) have enabled high-quality video synthesis. However, their computation costs scale quadratically with sequence length because self-attention has quadratic complexity. While linear attention lowers the cost, fully replacing quadratic attention requires expensive pretraining due to the limited expressiveness of linear attention and the complexity of spatiotemporal modeling in video generation. In this paper, we present LinVideo, an efficient data-free post-training framework that replaces a target number of self-attention modules with linear attention while preserving the original model’s performance. First, we observe a significant disparity in the replaceability of different layers. Instead of manual or heuristic choices, we frame layer selection as a binary classification problem and propose selective transfer, which automatically and progressively converts layers to linear attention with minimal performance impact. Additionally, to overcome the ineffectiveness and inefficiency of existing objectives for this transfer process, we introduce an anytime distribution matching (ADM) objective that aligns the distributions of samples across any timestep along the sampling trajectory. This objective is efficient and recovers model performance. Extensive experiments show that our method achieves a 1.25-2.00x speedup while preserving generation quality, and our 4-step distilled model further delivers a 15.92x latency reduction with minimal visual quality drop.

Method: Created DTPQA benchmark with perception-only questions in traffic scenes enriched with distance annotations, excluding reasoning questions. Evaluated several state-of-the-art small VLMs on this benchmark.

Result: Small VLMs significantly underperformed humans (~60% average accuracy for best model vs ~85% human performance). Specific perception tasks like distinguishing left from right remained particularly challenging. Human sample size was relatively small, imposing statistical limitations.

Conclusion: Current small VLMs have substantial limitations in traffic scene perception capabilities, especially for safety-critical automated driving applications, highlighting the need for improved perception-focused models.

Abstract: Vision-Language Models (VLMs) are becoming increasingly powerful, demonstrating strong performance on a variety of tasks that require both visual and textual understanding. Their strong generalisation abilities make them a promising component for automated driving systems, which must handle unexpected corner cases. However, to be trusted in such safety-critical applications, a model must first possess a reliable perception system. Moreover, since critical objects and agents in traffic scenes are often at a distance, we require systems that are not “shortsighted”, i.e., systems with strong perception capabilities at both close (up to 20 meters) and long (30+ meters) range. With this in mind, we introduce Distance-Annotated Traffic Perception Question Answering (DTPQA), the first Visual Question Answering (VQA) benchmark focused solely on perception-based questions in traffic scenes, enriched with distance annotations. By excluding questions that require reasoning, we ensure that model performance reflects perception capabilities alone. Since automated driving hardware has limited processing power and cannot support large VLMs, our study centers on smaller VLMs. More specifically, we evaluate several state-of-the-art (SOTA) small VLMs on DTPQA and show that, despite the simplicity of the questions, these models significantly underperform compared to humans (~60% average accuracy for the best-performing small VLM versus ~85% human performance). However, it is important to note that the human sample size was relatively small, which imposes statistical limitations. We also identify specific perception tasks, such as distinguishing left from right, that remain particularly challenging for these models.

Method: Constructing image filters to simulate low-light or hazy conditions and deriving approximate reverse filters to minimize distortions in enhanced images.

Result: The approach is highly competitive and often surpasses state-of-the-art techniques, especially in handling extremely dark images and enhancing hazy images.

Conclusion: The method provides a simple, efficient solution for image enhancement that can be implemented with just a few lines of MATLAB code while achieving superior performance.

Abstract: We introduce a simple and efficient method to enhance and clarify images. More specifically, we deal with low light image enhancement and clarification of hazy imagery (hazy/foggy images, images containing sand dust, and underwater images). Our method involves constructing an image filter to simulate low-light or hazy conditions and deriving approximate reverse filters to minimize distortions in the enhanced images. Experimental results show that our approach is highly competitive and often surpasses state-of-the-art techniques in handling extremely dark images and in enhancing hazy images. A key advantage of our approach lies in its simplicity: Our method is implementable with just a few lines of MATLAB code.

Method: Proposed guided diffusion model for data augmentation, lightweight transformer network, modified weighted loss function, and optimized cosine variance scheduler for small dataset training.

Result: Outperformed other data augmentation techniques in average and weighted average accuracy for forest classification with 10 forest types using PRISMA satellite hyperspectral images.

Conclusion: The method provides stable training behavior and addresses practical limitations of deep generative models for data augmentation in hyperspectral classification tasks.

Abstract: The introduction of new generation hyperspectral satellite sensors, combined with advancements in deep learning methodologies, has significantly enhanced the ability to discriminate detailed land-cover classes at medium-large scales. However, a significant challenge in deep learning methods is the risk of overfitting when training networks with small labeled datasets. In this work, we propose a data augmentation technique that leverages a guided diffusion model. To effectively train the model with a limited number of labeled samples and to capture complex patterns in the data, we implement a lightweight transformer network. Additionally, we introduce a modified weighted loss function and an optimized cosine variance scheduler, which facilitate fast and effective training on small datasets. We evaluate the effectiveness of the proposed method on a forest classification task with 10 different forest types using hyperspectral images acquired by the PRISMA satellite. The results demonstrate that the proposed method outperforms other data augmentation techniques in both average and weighted average accuracy. The effectiveness of the method is further highlighted by the stable training behavior of the model, which addresses a common limitation in the practical application of deep generative models for data augmentation.

Method: Dual-stream architecture with MLLM for instruction interpretation and MMDiT for video generation, jointly trained across multiple video tasks under a unified multimodal instruction paradigm.

Result: UniVideo matches or surpasses state-of-the-art task-specific baselines in text/image-to-video generation, in-context video generation, and in-context video editing. It demonstrates task composition and generalization to unseen editing instructions.

Conclusion: UniVideo successfully extends unified multimodal modeling to video domain, enabling diverse video generation/editing tasks with strong performance and generalization capabilities, paving the way for future research in unified video AI systems.

Abstract: Unified multimodal models have shown promising results in multimodal content generation and editing but remain largely limited to the image domain. In this work, we present UniVideo, a versatile framework that extends unified modeling to the video domain. UniVideo adopts a dual-stream design, combining a Multimodal Large Language Model (MLLM) for instruction understanding with a Multimodal DiT (MMDiT) for video generation. This design enables accurate interpretation of complex multimodal instructions while preserving visual consistency. Built on this architecture, UniVideo unifies diverse video generation and editing tasks under a single multimodal instruction paradigm and is jointly trained across them. Extensive experiments demonstrate that UniVideo matches or surpasses state-of-the-art task-specific baselines in text/image-to-video generation, in-context video generation and in-context video editing. Notably, the unified design of UniVideo enables two forms of generalization. First, UniVideo supports task composition, such as combining editing with style transfer, by integrating multiple capabilities within a single instruction. Second, even without explicit training on free-form video editing, UniVideo transfers its editing capability from large-scale image editing data to this setting, handling unseen instructions such as green-screening characters or changing materials within a video. Beyond these core capabilities, UniVideo also supports visual-prompt-based video generation, where the MLLM interprets visual prompts and guides the MMDiT during synthesis. To foster future research, we will release our model and code.

Method: Combines LayoutLMv3 for layout detection with GPT-4o using in-context learning to detect and link legend items and their descriptions via bounding box predictions. Uses structured JSON prompts for improved performance.

Result: GPT-4 with structured JSON prompts outperforms baseline, achieving 88% F-1 score and 85% IoU. Experiments reveal how prompt design, example counts, and layout alignment affect performance.

Conclusion: This approach supports scalable, layout-aware legend parsing and improves the indexing and searchability of historical maps across various visual styles.

Abstract: Historical map legends are critical for interpreting cartographic symbols. However, their inconsistent layouts and unstructured formats make automatic extraction challenging. Prior work focuses primarily on segmentation or general optical character recognition (OCR), with few methods effectively matching legend symbols to their corresponding descriptions in a structured manner. We present a method that combines LayoutLMv3 for layout detection with GPT-4o using in-context learning to detect and link legend items and their descriptions via bounding box predictions. Our experiments show that GPT-4 with structured JSON prompts outperforms the baseline, achieving 88% F-1 and 85% IoU, and reveal how prompt design, example counts, and layout alignment affect performance. This approach supports scalable, layout-aware legend parsing and improves the indexing and searchability of historical maps across various visual styles.

Method: Proposes LFC framework with two curricula: easy-to-hard curriculum starts with easy samples and gradually increases difficulty, and source-to-target curriculum ensures smooth domain transition.

Result: Achieves state-of-the-art performance on public cross-domain datasets for fundus segmentation and polyp segmentation, surpassing existing approaches.

Conclusion: The curriculum-based approach effectively enables source-free domain adaptation for medical image analysis while addressing privacy concerns and improving adaptation quality.

Abstract: Recent studies have uncovered a new research line, namely source-free domain adaptation, which adapts a model to target domains without using the source data. Such a setting can address the concerns on data privacy and security issues of medical images. However, current source-free domain adaptation frameworks mainly focus on the pseudo label refinement for target data without the consideration of learning procedure. Indeed, a progressive learning process from source to target domain will benefit the knowledge transfer during model adaptation. To this end, we propose a curriculum-based framework, namely learning from curriculum (LFC), for source-free domain adaptation, which consists of easy-to-hard and source-to-target curricula. Concretely, the former curriculum enables the framework to start learning with `easy’ samples and gradually tune the optimization direction of model adaption by increasing the sample difficulty. While, the latter can stablize the adaptation process, which ensures smooth transfer of the model from the source domain to the target. We evaluate the proposed source-free domain adaptation approach on the public cross-domain datasets for fundus segmentation and polyp segmentation. The extensive experimental results show that our framework surpasses the existing approaches and achieves a new state-of-the-art.

Method: Collected diverse videos across domains, extracted event-level descriptions with temporal causality, rewrote them into prompts, designed binary evaluation questions across 10 dimensions, and created a QA-based evaluation pipeline using vision-language models.

Result: Built VideoVerse benchmark with 300 curated prompts, 815 events, and 793 binary evaluation questions, then systematically evaluated state-of-the-art T2V models.

Conclusion: The benchmark provides in-depth analysis of how far current T2V generators are from true world models by focusing on temporal causality and world knowledge understanding.

Abstract: The recent rapid advancement of Text-to-Video (T2V) generation technologies, which are critical to build ``world models’’, makes the existing benchmarks increasingly insufficient to evaluate state-of-the-art T2V models. First, current evaluation dimensions, such as per-frame aesthetic quality and temporal consistency, are no longer able to differentiate state-of-the-art T2V models. Second, event-level temporal causality, which not only distinguishes video from other modalities but also constitutes a crucial component of world models, is severely underexplored in existing benchmarks. Third, existing benchmarks lack a systematic assessment of world knowledge, which are essential capabilities for building world models. To address these issues, we introduce VideoVerse, a comprehensive benchmark that focuses on evaluating whether a T2V model could understand complex temporal causality and world knowledge in the real world. We collect representative videos across diverse domains (e.g., natural landscapes, sports, indoor scenes, science fiction, chemical and physical experiments) and extract their event-level descriptions with inherent temporal causality, which are then rewritten into text-to-video prompts by independent annotators. For each prompt, we design a suite of binary evaluation questions from the perspective of dynamic and static properties, with a total of ten carefully defined evaluation dimensions. In total, our VideoVerse comprises 300 carefully curated prompts, involving 815 events and 793 binary evaluation questions. Consequently, a human preference aligned QA-based evaluation pipeline is developed by using modern vision-language models. Finally, we perform a systematic evaluation of state-of-the-art open-source and closed-source T2V models on VideoVerse, providing in-depth analysis on how far the current T2V generators are from world models.

Method: Developed a parallelism-compatible FlashAttention-2 JVP kernel for training on large models, and proposed score-regularized continuous-time consistency model (rCM) that incorporates score distillation as a long-skip regularizer to complement the mode-covering forward divergence with mode-seeking reverse divergence.

Result: Validated on models up to 14B parameters and 5-second videos, rCM matches or surpasses DMD2 on quality metrics while offering better diversity, generating high-fidelity samples in 1-4 steps (15-50x speedup) without GAN tuning or extensive hyperparameter searches.

Conclusion: rCM provides a practical and theoretically grounded framework for advancing large-scale diffusion distillation, enabling efficient high-quality generation while maintaining diversity.

Abstract: This work represents the first effort to scale up continuous-time consistency distillation to general application-level image and video diffusion models. Although continuous-time consistency model (sCM) is theoretically principled and empirically powerful for accelerating academic-scale diffusion, its applicability to large-scale text-to-image and video tasks remains unclear due to infrastructure challenges in Jacobian-vector product (JVP) computation and the limitations of standard evaluation benchmarks. We first develop a parallelism-compatible FlashAttention-2 JVP kernel, enabling sCM training on models with over 10 billion parameters and high-dimensional video tasks. Our investigation reveals fundamental quality limitations of sCM in fine-detail generation, which we attribute to error accumulation and the “mode-covering” nature of its forward-divergence objective. To remedy this, we propose the score-regularized continuous-time consistency model (rCM), which incorporates score distillation as a long-skip regularizer. This integration complements sCM with the “mode-seeking” reverse divergence, effectively improving visual quality while maintaining high generation diversity. Validated on large-scale models (Cosmos-Predict2, Wan2.1) up to 14B parameters and 5-second videos, rCM matches or surpasses the state-of-the-art distillation method DMD2 on quality metrics while offering notable advantages in diversity, all without GAN tuning or extensive hyperparameter searches. The distilled models generate high-fidelity samples in only $1\sim4$ steps, accelerating diffusion sampling by $15\times\sim50\times$. These results position rCM as a practical and theoretically grounded framework for advancing large-scale diffusion distillation.

Method: Uses return-guided contrastive learning where similar visual representations are grouped into positives/negatives based on return differences. Contrastive triplets train the attention mechanism to distinguish features relevant to success vs failure.

Result: Achieves up to 2.4x improvement in sample efficiency and can solve tasks that baseline methods fail to learn, demonstrated across manipulation tasks in ManiSkill3 benchmark without modifying underlying algorithm or hyperparameters.

Conclusion: The framework successfully addresses visual RL’s sample inefficiency by focusing attention on task-relevant features using return differences as guidance, enabling more stable and efficient learning.

Abstract: Visual Reinforcement Learning (RL) agents must learn to act based on high-dimensional image data where only a small fraction of the pixels is task-relevant. This forces agents to waste exploration and computational resources on irrelevant features, leading to sample-inefficient and unstable learning. To address this, inspired by human visual foveation, we introduce Gaze on the Prize. This framework augments visual RL with a learnable foveal attention mechanism (Gaze), guided by a self-supervised signal derived from the agent’s experience pursuing higher returns (the Prize). Our key insight is that return differences reveal what matters most: If two similar representations produce different outcomes, their distinguishing features are likely task-relevant, and the gaze should focus on them accordingly. This is realized through return-guided contrastive learning that trains the attention to distinguish between the features relevant to success and failure. We group similar visual representations into positives and negatives based on their return differences and use the resulting labels to construct contrastive triplets. These triplets provide the training signal that teaches the attention mechanism to produce distinguishable representations for states associated with different outcomes. Our method achieves up to 2.4x improvement in sample efficiency and can solve tasks that the baseline fails to learn, demonstrated across a suite of manipulation tasks from the ManiSkill3 benchmark, all without modifying the underlying algorithm or hyperparameters.

Method: Stepwise intensity transformation for grayscale quantization, histogram equalization in RGB/YCrCb spaces, HSV value-channel brightness adjustment, 3x3 convolution kernel sharpening, bidirectional transformation pipeline with unsharp masking/gamma correction/noise amplification, and geometric feature extraction using Canny edge detection, Hough transform, Harris corner detection, and morphological operations.

Result: Achieved 76.10% forward and 74.80% reverse accuracy in bidirectional transformation, 51.50° billiard cue alignment accuracy, and 81.87% similarity for cue isolation against ground truth images.

Conclusion: The framework demonstrates robust and deterministic performance across diverse datasets, showing strong potential for real-time image analysis and computer vision applications.

Abstract: This study introduces a modular framework for spatial image processing, integrating grayscale quantization, color and brightness enhancement, image sharpening, bidirectional transformation pipelines, and geometric feature extraction. A stepwise intensity transformation quantizes grayscale images into eight discrete levels, producing a posterization effect that simplifies representation while preserving structural detail. Color enhancement is achieved via histogram equalization in both RGB and YCrCb color spaces, with the latter improving contrast while maintaining chrominance fidelity. Brightness adjustment is implemented through HSV value-channel manipulation, and image sharpening is performed using a 3 * 3 convolution kernel to enhance high-frequency details. A bidirectional transformation pipeline that integrates unsharp masking, gamma correction, and noise amplification achieved accuracy levels of 76.10% and 74.80% for the forward and reverse processes, respectively. Geometric feature extraction employed Canny edge detection, Hough-based line estimation (e.g., 51.50{\deg} for billiard cue alignment), Harris corner detection, and morphological window localization. Cue isolation further yielded 81.87% similarity against ground truth images. Experimental evaluation across diverse datasets demonstrates robust and deterministic performance, highlighting its potential for real-time image analysis and computer vision.

Method: Harmonizes contextual sub-motion decomposition with tool-augmented reinforcement learning, using hierarchical rewards to balance tool usage efficiency, sub-motion relevance, and structural coherence.

Result: Achieves state-of-the-art performance on multiple datasets (HMDB-51, UCF-101, SSv2, Kinetics-400, Kinetics-600), outperforming existing methods in fine-grained action distinction and cross-modal hallucination reduction.

Conclusion: The framework successfully enables category-specific reasoning capacity and demonstrates excellent robustness and generalization by transitioning from text-centric to visually grounded inference.

Abstract: Multimodal large language models (MLLMs) have demonstrated remarkable potential in bridging visual and textual reasoning, yet their reliance on text-centric priors often limits their ability to disentangle semantically similar actions in open-vocabulary scenarios. To address this, we propose Video-STAR, a framework that harmonizes contextual sub-motion decomposition with tool-augmented reinforcement learning for open-vocabulary action recognition (OVAR). Unlike prior methods that treat actions as monolithic entities, our approach innovatively decomposes actions into discriminative sub-motions for fine-grained matching while dynamically invoking domain-specific tools for cross-modal interleaving, thereby enabling category-specific reasoning capacity and reducing cross-modal hallucination. Moreover, by designing a hierarchical reward that balances tool-usage efficiency, sub-motion relevance, and structural coherence in reasoning, our method autonomously leverages external tools to prioritize sub-motion patterns without explicit supervision, transmitting from text-centric reasoning to visually grounded inference. Extensive evaluations on HMDB-51, UCF-101, SSv2, Kinetics-400, and Kinetics-600 datasets demonstrate our state-of-the-art performance, outperforming existing methods in distinguishing fine-grained actions and handling cross-modal hallucination, validating our excellent robustness and generalization.

Method: Created a video-based benchmark with three tasks: phonological sign-form prediction (handshape, location), transparency (inferring meaning from form), and graded iconicity ratings. Evaluated 13 state-of-the-art VLMs in zero- and few-shot settings on Sign Language of the Netherlands compared to human baselines.

Result: VLMs recover some handshape and location detail but remain below human performance on phonological form prediction; perform far from human baselines on transparency; only top models correlate moderately with human iconicity ratings. Models with better phonological form prediction correlate better with human iconicity judgment.

Conclusion: The findings validate the diagnostic tasks and motivate human-centric signals and embodied learning methods for modeling iconicity and improving visual grounding in multimodal models.

Abstract: Iconicity, the resemblance between linguistic form and meaning, is pervasive in signed languages, offering a natural testbed for visual grounding. For vision-language models (VLMs), the challenge is to recover such essential mappings from dynamic human motion rather than static context. We introduce the \textit{Visual Iconicity Challenge}, a novel video-based benchmark that adapts psycholinguistic measures to evaluate VLMs on three tasks: (i) phonological sign-form prediction (e.g., handshape, location), (ii) transparency (inferring meaning from visual form), and (iii) graded iconicity ratings. We assess $13$ state-of-the-art VLMs in zero- and few-shot settings on Sign Language of the Netherlands and compare them to human baselines. On \textit{phonological form prediction}, VLMs recover some handshape and location detail but remain below human performance; on \textit{transparency}, they are far from human baselines; and only top models correlate moderately with human \textit{iconicity ratings}. Interestingly, \textit{models with stronger phonological form prediction correlate better with human iconicity judgment}, indicating shared sensitivity to visually grounded structure. Our findings validate these diagnostic tasks and motivate human-centric signals and embodied learning methods for modelling iconicity and improving visual grounding in multimodal models.

Method: Conducts comprehensive study on MLLM-diffusion integration, analyzes cooperation between images and videos in unified modeling, shows emergent video editing from image training, and incorporates modality-specific MLLM features to unify image and video editing in a single model.

Result: Extensive experiments demonstrate the method can handle a broad range of image and video editing tasks and achieves state-of-the-art performance, with training on image data leading to emergent video editing capabilities without explicit supervision.

Conclusion: The proposed InstructX framework effectively unifies image and video editing tasks within a single model, alleviating constraints from scarce video training data and achieving superior performance across diverse editing tasks.

Abstract: With recent advances in Multimodal Large Language Models (MLLMs) showing strong visual understanding and reasoning, interest is growing in using them to improve the editing performance of diffusion models. Despite rapid progress, most studies lack an in-depth analysis of MLLM design choices. Moreover, the integration of MLLMs and diffusion models remains an open challenge in some difficult tasks, such as video editing. In this paper, we present InstructX, a unified framework for image and video editing. Specifically, we conduct a comprehensive study on integrating MLLMs and diffusion models for instruction-driven editing across diverse tasks. Building on this study, we analyze the cooperation and distinction between images and videos in unified modeling. (1) We show that training on image data can lead to emergent video editing capabilities without explicit supervision, thereby alleviating the constraints imposed by scarce video training data. (2) By incorporating modality-specific MLLM features, our approach effectively unifies image and video editing tasks within a single model. Extensive experiments demonstrate that our method can handle a broad range of image and video editing tasks and achieves state-of-the-art performance.

Method: Uses three coordinated agents: 1) VLM-driven degradation identifier trained on large-scale degradation benchmark, 2) LLM-powered self-adaptive router that learns restoration strategies from tool usage patterns, 3) VLM-based video quality assessment model trained on Res-VQ dataset for restoration tasks.

Result: Extensive experiments show MoA-VR effectively handles diverse and compound degradations, consistently outperforming existing baselines in both objective metrics and perceptual quality.

Conclusion: The system demonstrates the potential of integrating multimodal intelligence and modular reasoning for general-purpose video restoration, mimicking human professional reasoning processes.

Abstract: Real-world videos often suffer from complex degradations, such as noise, compression artifacts, and low-light distortions, due to diverse acquisition and transmission conditions. Existing restoration methods typically require professional manual selection of specialized models or rely on monolithic architectures that fail to generalize across varying degradations. Inspired by expert experience, we propose MoA-VR, the first \underline{M}ixture-\underline{o}f-\underline{A}gents \underline{V}ideo \underline{R}estoration system that mimics the reasoning and processing procedures of human professionals through three coordinated agents: Degradation Identification, Routing and Restoration, and Restoration Quality Assessment. Specifically, we construct a large-scale and high-resolution video degradation recognition benchmark and build a vision-language model (VLM) driven degradation identifier. We further introduce a self-adaptive router powered by large language models (LLMs), which autonomously learns effective restoration strategies by observing tool usage patterns. To assess intermediate and final processed video quality, we construct the \underline{Res}tored \underline{V}ideo \underline{Q}uality (Res-VQ) dataset and design a dedicated VLM-based video quality assessment (VQA) model tailored for restoration tasks. Extensive experiments demonstrate that MoA-VR effectively handles diverse and compound degradations, consistently outperforming existing baselines in terms of both objective metrics and perceptual quality. These results highlight the potential of integrating multimodal intelligence and modular reasoning in general-purpose video restoration systems.

Method: The authors present qualitative and quantitative analyses of information in ViT sink tokens, and propose both training-free and training-based approaches to better leverage how this information is interpreted by the LLM.

Result: By explicitly utilizing ViT attention sink tokens, the authors demonstrate substantial improvements across a range of LVLMs and visual reasoning tasks.

Conclusion: ViT attention sinks represent untapped potential in enhancing visual reasoning, as they encapsulate high-level semantic concepts that allow LLMs to perform more effective understanding and reasoning when properly utilized.

Abstract: Large Vision Language Models (LVLMs) have recently emerged as powerful architectures capable of understanding and reasoning over both visual and textual information. These models typically rely on two key components: a Vision Transformer (ViT) and a Large Language Model (LLM). ViT encodes visual content into a sequence of image tokens and serves as the perceptual front-end – the eyes of the model. In contrast, the LLM interprets these tokens to perform high-level reasoning, generates responses, and functions as the cognitive core – the brain of the model. However, it remains unclear which visual tokens contribute most significantly to understanding and reasoning, and how effectively these signals are propagated from ViT to the LLM. While most existing works have focused on identifying attention sinks, low-semantic tokens receiving disproportionately high attention, within the LLM, we shift the focus to the vision encoder by identifying a class of high-norm visual tokens from ViT, referred to as ViT attention sinks – a problem that has been rarely studied but is indeed very important for LVLMs. Our findings show that these ViT sinks encapsulate high-level semantic concepts from images, allowing the LLM to perform more effective understanding and reasoning. Despite their importance, these sink tokens are often overlooked in existing LVLM architectures. To explore their contribution, we present both qualitative and quantitative analyses of the information embedded in these sink tokens. We also propose both training-free and training-based approaches to better leverage how this information is interpreted by the LLM, and to what extent. By explicitly utilizing these tokens, we demonstrate substantial improvements across a range of LVLMs and visual reasoning tasks, highlighting the untapped potential of ViT attention sinks in enhancing visual reasoning.

Method: Decomposes point clouds into semantic patches, encodes them with semantic-aware FiLM-conditioned encoders, and uses folding-based decoder guided by latent features and graph node attributes.

Result: Achieves state-of-the-art compression rates (up to 98% size reduction) on SemanticKITTI and nuScenes datasets while preserving structural and semantic fidelity.

Conclusion: The framework enables efficient point cloud transmission for robotics applications while maintaining performance comparable to raw LiDAR data in downstream tasks like pose graph optimization and map merging.

Abstract: Efficient transmission of 3D point cloud data is critical for advanced perception in centralized and decentralized multi-agent robotic systems, especially nowadays with the growing reliance on edge and cloud-based processing. However, the large and complex nature of point clouds creates challenges under bandwidth constraints and intermittent connectivity, often degrading system performance. We propose a deep compression framework based on semantic scene graphs. The method decomposes point clouds into semantically coherent patches and encodes them into compact latent representations with semantic-aware encoders conditioned by Feature-wise Linear Modulation (FiLM). A folding-based decoder, guided by latent features and graph node attributes, enables structurally accurate reconstruction. Experiments on the SemanticKITTI and nuScenes datasets show that the framework achieves state-of-the-art compression rates, reducing data size by up to 98% while preserving both structural and semantic fidelity. In addition, it supports downstream applications such as multi-robot pose graph optimization and map merging, achieving trajectory accuracy and map alignment comparable to those obtained with raw LiDAR scans.

Method: Proposed SliceFine, a PEFT method that updates only selected slices of original weights without adding new parameters, based on the theoretical framework of spectral balance and high task energy in pre-trained networks.

Result: SliceFine matches state-of-the-art PEFT performance across language and vision tasks while significantly improving training speed, memory efficiency, and model compactness.

Conclusion: The work bridges theory and practice by providing a theoretically grounded alternative to existing PEFT techniques, demonstrating that exploiting inherent model redundancy through slice-based fine-tuning is effective and efficient.

Abstract: This paper presents a theoretical framework explaining why fine tuning small, randomly selected subnetworks (slices) within pre trained models can be sufficient for downstream adaptation. We prove that pretrained networks exhibit a universal winning slice property arising from two phenomena: (1) spectral balance the eigenspectra of different weight matrix slices are remarkably similar; and (2) high task energy their backbone representations retain rich, task relevant features. This leads to the Universal Winning Slice Hypothesis, which provides a theoretical foundation for parameter efficient fine tuning (PEFT) in large scale models. Inspired by this, we propose SliceFine, a PEFT method that exploits this inherent redundancy by updating only selected slices of the original weights introducing zero new parameters, unlike adapter-based approaches. Empirically, SliceFine matches the performance of state of the art PEFT methods across language and vision tasks, while significantly improving training speed, memory efficiency, and model compactness. Our work bridges theory and practice, offering a theoretically grounded alternative to existing PEFT techniques.

Method: Uses unified point-based motion representation with segmentation ID, trajectory ID, and optional color channel. Employs efficient sequence-concatenation scheme instead of token concatenation or ControlNet, with annealing training strategy.

Result: Achieves faster convergence, stronger controllability, and more efficient inference while maintaining robustness under unaligned conditions. Supports various applications including motion cloning, drag-based image-to-video, motion interpolation, camera redirection, flexible action control and mesh animations.

Conclusion: FlexTraj enables flexible point trajectory control for image-to-video generation with improved performance and broader application capabilities.

Abstract: We present FlexTraj, a framework for image-to-video generation with flexible point trajectory control. FlexTraj introduces a unified point-based motion representation that encodes each point with a segmentation ID, a temporally consistent trajectory ID, and an optional color channel for appearance cues, enabling both dense and sparse trajectory control. Instead of injecting trajectory conditions into the video generator through token concatenation or ControlNet, FlexTraj employs an efficient sequence-concatenation scheme that achieves faster convergence, stronger controllability, and more efficient inference, while maintaining robustness under unaligned conditions. To train such a unified point trajectory-controlled video generator, FlexTraj adopts an annealing training strategy that gradually reduces reliance on complete supervision and aligned condition. Experimental results demonstrate that FlexTraj enables multi-granularity, alignment-agnostic trajectory control for video generation, supporting various applications such as motion cloning, drag-based image-to-video, motion interpolation, camera redirection, flexible action control and mesh animations.

Method: Three-stage progressive training: (1) spatial perception via object localization, (2) spatial understanding through multi-dimensional tasks, (3) complex reasoning via RL with verifiable rewards, using SpatialLadder-26k dataset.

Result: 23.4% average improvement over base model, surpassing GPT-4o by 20.8% and Gemini-2.0-Flash by 10.1%, with 7.2% improvement on out-of-domain benchmarks.

Conclusion: Progressive training from perception to reasoning is essential for robust spatial intelligence in VLMs.

Abstract: Spatial reasoning remains a fundamental challenge for Vision-Language Models (VLMs), with current approaches struggling to achieve robust performance despite recent advances. We identify that this limitation stems from a critical gap: existing methods attempt to learn spatial reasoning directly without establishing the hierarchical foundations of perception and understanding. To address this challenge, we present a comprehensive methodology for building spatial intelligence progressively. We introduce SpatialLadder-26k, a multimodal dataset containing 26,610 samples spanning object localization, single image, multi-view, and video spatial reasoning tasks, constructed through a standardized pipeline that ensures systematic coverage across modalities. Building on this dataset, we design a three-stage progressive training framework that (1) establishes spatial perception through object localization, (2) develops spatial understanding through multi-dimensional spatial tasks, and (3) strengthens complex reasoning via reinforcement learning with verifiable rewards. This approach yields SpatialLadder, a 3B-parameter model that achieves state-of-the-art performance on spatial reasoning benchmarks, with 23.4% average improvement over the base model, surpassing GPT-4o by 20.8% and Gemini-2.0-Flash by 10.1%. Notably, SpatialLadder maintains strong generalization with 7.2% improvement on out-of-domain benchmarks, demonstrating that progressive training from perception to reasoning is essential for robust spatial intelligence.

Method: Extends a state-of-the-art image editing model to accept an additional scalar edit strength input. Uses a lightweight projector network to map the scalar and edit instruction to coefficients in the model’s modulation space. Trained on a synthesized dataset of image-edit-instruction-strength quadruplets with quality filtering.

Result: Provides unified fine-grained control over edit strength for diverse operations (stylization, attribute, material, background, shape changes) without requiring attribute-specific training. Enables smooth, continuous adjustment from subtle to strong edits.

Conclusion: Kontinuous Kontext successfully addresses the limitation of text-only instruction editing by introducing explicit control over edit strength, offering a more flexible and precise editing experience across various operation types.

Abstract: Instruction-based image editing offers a powerful and intuitive way to manipulate images through natural language. Yet, relying solely on text instructions limits fine-grained control over the extent of edits. We introduce Kontinuous Kontext, an instruction-driven editing model that provides a new dimension of control over edit strength, enabling users to adjust edits gradually from no change to a fully realized result in a smooth and continuous manner. Kontinuous Kontext extends a state-of-the-art image editing model to accept an additional input, a scalar edit strength which is then paired with the edit instruction, enabling explicit control over the extent of the edit. To inject this scalar information, we train a lightweight projector network that maps the input scalar and the edit instruction to coefficients in the model’s modulation space. For training our model, we synthesize a diverse dataset of image-edit-instruction-strength quadruplets using existing generative models, followed by a filtering stage to ensure quality and consistency. Kontinuous Kontext provides a unified approach for fine-grained control over edit strength for instruction driven editing from subtle to strong across diverse operations such as stylization, attribute, material, background, and shape changes, without requiring attribute-specific training.

Method: Created MM-HELIX benchmark with 1,260 challenging tasks requiring iterative thinking, generated MM-HELIX-100K dataset using Step-Elicited Response Generation pipeline, and proposed Adaptive Hybrid Policy Optimization (AHPO) that dynamically combines offline supervision and online optimization.

Result: Applied to Qwen2.5-VL-7B baseline, achieved +18.6% accuracy improvement on MM-HELIX benchmark and +5.7% average performance gain on general mathematical and logic tasks, demonstrating strong generalization.

Conclusion: Reflective reasoning in MLLMs can be effectively learned and generalized, paving the way for developing more capable multimodal language models.

Abstract: While current Multimodal Large Language Models (MLLMs) have demonstrated proficiency in reasoning tasks such as mathematics and logic, their capacity for long-chain reflective reasoning, a prerequisite for solving complex real-world problems, remains largely underexplored. In this work, we first conduct an extensive empirical investigation to evaluate this capability. Leveraging a carefully designed data synthesis engine, we construct MM-HELIX, a multimodal benchmark consisting 1,260 samples of 42 challenging synthetic tasks that require iterative thinking and backtracking. Empirical results on this benchmark reveal that existing MLLMs exhibit significant performance deficits in long-chain reflective reasoning. To address this limitation, we generate post-training data and further explore learning paradigms for exploiting such data. We first develop the Step-Elicited Response Generation pipeline to create MM-HELIX-100K, a large-scale dataset of 100k high-quality, reflective reasoning traces for instruction-tuning stage. Given that standard Reinforcement Learning fails on complex tasks due to sparse reward signals and catastrophic forgetting after Supervised Fine-Tuning, we propose Adaptive Hybrid Policy Optimization (AHPO), a novel training strategy that dynamically unifies offline supervision and online optimization into a single stage. This strategy enables the model to learn from expert data when rewards are sparse and conduct independent exploration once proficient. When applied to the Qwen2.5-VL-7B baseline, our method achieves a +18.6% accuracy improvement on MM-HELIX benchmark and demonstrates strong generalization with a +5.7% average performance gain on general mathematic and logic tasks. Our work demonstrate that reflective reasoning in MLLMs can be effectively learned and generalized, paving the way for developing more capable MLLMs.

Method: Human-AI collaboration framework where a teacher model provides candidate annotations using theoretically-grounded prompting, and human experts validate/correct them.

Result: Models perform worse on norm violations than adherence, worse on Chinese culture than US culture, struggle with non-verbal evidence, and perform worse in formal contexts unlike humans.

Conclusion: There is a critical need for culturally-grounded video language model training, which this benchmark and framework begin to address.

Abstract: As Video Large Language Models (VideoLLMs) are deployed globally, they require understanding of and grounding in the relevant cultural background. To properly assess these models’ cultural awareness, adequate benchmarks are needed. We introduce VideoNorms, a benchmark of over 1000 (video clip, norm) pairs from US and Chinese cultures annotated with socio-cultural norms grounded in speech act theory, norm adherence and violations labels, and verbal and non-verbal evidence. To build VideoNorms, we use a human-AI collaboration framework, where a teacher model using theoretically-grounded prompting provides candidate annotations and a set of trained human experts validate and correct the annotations. We benchmark a variety of open-weight VideoLLMs on the new dataset which highlight several common trends: 1) models performs worse on norm violation than adherence; 2) models perform worse w.r.t Chinese culture compared to the US culture; 3) models have more difficulty in providing non-verbal evidence compared to verbal for the norm adhere/violation label and struggle to identify the exact norm corresponding to a speech-act; and 4) unlike humans, models perform worse in formal, non-humorous contexts. Our findings emphasize the need for culturally-grounded video language model training - a gap our benchmark and framework begin to address.

Method: Uses 3D foundation models for pose estimation and point prediction, coupled with a Gaussian decoder that transforms multi-scale features into structured 3D Gaussians. Implements hierarchical Gaussian representation with LoD-aware rendering strategy.

Result: Achieves interactive performance comparable to SLAM, robustness similar to feed-forward systems, and reconstruction quality close to per-scene optimization on eight diverse indoor and outdoor benchmarks.

Conclusion: Provides a practical path toward on-the-fly digitization of real-world environments with both accurate geometry and high visual fidelity, bridging the gap between efficiency and quality in 3D reconstruction.

Abstract: On-the-fly 3D reconstruction from monocular image sequences is a long-standing challenge in computer vision, critical for applications such as real-to-sim, AR/VR, and robotics. Existing methods face a major tradeoff: per-scene optimization yields high fidelity but is computationally expensive, whereas feed-forward foundation models enable real-time inference but struggle with accuracy and robustness. In this work, we propose ARTDECO, a unified framework that combines the efficiency of feed-forward models with the reliability of SLAM-based pipelines. ARTDECO uses 3D foundation models for pose estimation and point prediction, coupled with a Gaussian decoder that transforms multi-scale features into structured 3D Gaussians. To sustain both fidelity and efficiency at scale, we design a hierarchical Gaussian representation with a LoD-aware rendering strategy, which improves rendering fidelity while reducing redundancy. Experiments on eight diverse indoor and outdoor benchmarks show that ARTDECO delivers interactive performance comparable to SLAM, robustness similar to feed-forward systems, and reconstruction quality close to per-scene optimization, providing a practical path toward on-the-fly digitization of real-world environments with both accurate geometry and high visual fidelity. Explore more demos on our project page: https://city-super.github.io/artdeco/.

Method: 1) Language-conditioned world model that imagines future states for encoding experiences and generating retrieval queries; 2) Hybrid Viewpoint-Level Memory that anchors both observations and behavioral patterns to viewpoints; 3) Experience-augmented navigation model with specialized encoders for retrieved knowledge integration.

Result: Significant improvements across all 10 testing scenarios, with 5.4% SPL gains on IR2R over best memory-persistent baseline, accompanied by 8.3x training speedup and 74% inference memory reduction.

Conclusion: Predictive retrieval of both environmental and behavioral memories enables more effective navigation, with substantial headroom (73.3% vs 93.4% upper bound) indicating the potential of imagination-guided paradigms in VLN.

Abstract: Vision-and-Language Navigation (VLN) requires agents to follow natural language instructions through environments, with memory-persistent variants demanding progressive improvement through accumulated experience. Existing approaches for memory-persistent VLN face critical limitations: they lack effective memory access mechanisms, instead relying on entire memory incorporation or fixed-horizon lookup, and predominantly store only environmental observations while neglecting navigation behavioral patterns that encode valuable decision-making strategies. We present Memoir, which employs imagination as a retrieval mechanism grounded by explicit memory: a world model imagines future navigation states as queries to selectively retrieve relevant environmental observations and behavioral histories. The approach comprises: 1) a language-conditioned world model that imagines future states serving dual purposes: encoding experiences for storage and generating retrieval queries; 2) Hybrid Viewpoint-Level Memory that anchors both observations and behavioral patterns to viewpoints, enabling hybrid retrieval; and 3) an experience-augmented navigation model that integrates retrieved knowledge through specialized encoders. Extensive evaluation across diverse memory-persistent VLN benchmarks with 10 distinctive testing scenarios demonstrates Memoir’s effectiveness: significant improvements across all scenarios, with 5.4% SPL gains on IR2R over the best memory-persistent baseline, accompanied by 8.3x training speedup and 74% inference memory reduction. The results validate that predictive retrieval of both environmental and behavioral memories enables more effective navigation, with analysis indicating substantial headroom (73.3% vs 93.4% upper bound) for this imagination-guided paradigm. Code at https://github.com/xyz9911/Memoir.

Method: VideoCanvas adapts In-Context Conditioning (ICC) with zero new parameters, using a hybrid conditioning strategy: spatial placement via zero-padding and temporal alignment through Temporal RoPE Interpolation to resolve VAE temporal ambiguity.

Result: VideoCanvas significantly outperforms existing conditioning paradigms and establishes new state-of-the-art performance in flexible and unified video generation, as evaluated on the new VideoCanvasBench benchmark.

Conclusion: The proposed framework successfully enables pixel-frame-aware control on frozen backbones and provides a unified solution for various video generation tasks through arbitrary spatio-temporal completion.

Abstract: We introduce the task of arbitrary spatio-temporal video completion, where a video is generated from arbitrary, user-specified patches placed at any spatial location and timestamp, akin to painting on a video canvas. This flexible formulation naturally unifies many existing controllable video generation tasks–including first-frame image-to-video, inpainting, extension, and interpolation–under a single, cohesive paradigm. Realizing this vision, however, faces a fundamental obstacle in modern latent video diffusion models: the temporal ambiguity introduced by causal VAEs, where multiple pixel frames are compressed into a single latent representation, making precise frame-level conditioning structurally difficult. We address this challenge with VideoCanvas, a novel framework that adapts the In-Context Conditioning (ICC) paradigm to this fine-grained control task with zero new parameters. We propose a hybrid conditioning strategy that decouples spatial and temporal control: spatial placement is handled via zero-padding, while temporal alignment is achieved through Temporal RoPE Interpolation, which assigns each condition a continuous fractional position within the latent sequence. This resolves the VAE’s temporal ambiguity and enables pixel-frame-aware control on a frozen backbone. To evaluate this new capability, we develop VideoCanvasBench, the first benchmark for arbitrary spatio-temporal video completion, covering both intra-scene fidelity and inter-scene creativity. Experiments demonstrate that VideoCanvas significantly outperforms existing conditioning paradigms, establishing a new state of the art in flexible and unified video generation.

Method: Created SciVideoBench with 1,000 carefully crafted multiple-choice questions derived from cutting-edge scientific experimental videos, verified by a semi-automatic system. Questions require domain-specific knowledge, spatiotemporal perception, and logical reasoning.

Result: Evaluation revealed significant performance deficits in state-of-the-art LMMs including Gemini 2.5 Pro and Qwen2.5-VL, showing substantial room for improvement in video reasoning capabilities.

Conclusion: SciVideoBench provides valuable insights and clear direction for future LMM development, driving the evolution of truly capable multimodal AI co-scientists for scientific applications.

Abstract: Large Multimodal Models (LMMs) have achieved remarkable progress across various capabilities; however, complex video reasoning in the scientific domain remains a significant and challenging frontier. Current video benchmarks predominantly target general scenarios where perception/recognition is heavily relied on, while with relatively simple reasoning tasks, leading to saturation and thus failing to effectively evaluate advanced multimodal cognitive skills. To address this critical gap, we introduce SciVideoBench, a rigorous benchmark specifically designed to assess advanced video reasoning in scientific contexts. SciVideoBench consists of 1,000 carefully crafted multiple-choice questions derived from cutting-edge scientific experimental videos spanning over 25 specialized academic subjects and verified by a semi-automatic system. Each question demands sophisticated domain-specific knowledge, precise spatiotemporal perception, and intricate logical reasoning, effectively challenging models’ higher-order cognitive abilities. Our evaluation highlights significant performance deficits in state-of-the-art proprietary and open-source LMMs, including Gemini 2.5 Pro and Qwen2.5-VL, indicating substantial room for advancement in video reasoning capabilities. Detailed analyses of critical factors such as reasoning complexity and visual grounding provide valuable insights and clear direction for future developments in LMMs, driving the evolution of truly capable multimodal AI co-scientists. We hope SciVideoBench could fit the interests of the community and help to push the boundary of cutting-edge AI for border science.

Method: Uses DiT architecture with multi-modal controls mapped to common point-based representation. Separates content and motion controls into two branches with stage-wise training strategy.

Result: Enables more dynamic, customizable, and contextually accurate visual narratives through multi-modal controls.

Conclusion: The framework achieves balance between flexibility, ease of use, and precision for fine-grained video interpolation with multi-modal controls.

Abstract: Video inbetweening creates smooth and natural transitions between two image frames, making it an indispensable tool for video editing and long-form video synthesis. Existing works in this domain are unable to generate large, complex, or intricate motions. In particular, they cannot accommodate the versatility of user intents and generally lack fine control over the details of intermediate frames, leading to misalignment with the creative mind. To fill these gaps, we introduce \modelname{}, a video inbetweening framework that allows multi-modal controls, including depth transition and layering, motion trajectories, text prompts, and target regions for movement localization, while achieving a balance between flexibility, ease of use, and precision for fine-grained video interpolation. To achieve this, we adopt the Diffusion Transformer (DiT) architecture as our video generative model, due to its proven capability to generate high-quality long videos. To ensure compatibility between DiT and our multi-modal controls, we map all motion controls into a common sparse and user-friendly point-based representation as the video/noise input. Further, to respect the variety of controls which operate at varying levels of granularity and influence, we separate content controls and motion controls into two branches to encode the required features before guiding the denoising process, resulting in two generators, one for motion and the other for content. Finally, we propose a stage-wise training strategy to ensure that our model learns the multi-modal controls smoothly. Extensive qualitative and quantitative experiments demonstrate that multi-modal controls enable a more dynamic, customizable, and contextually accurate visual narrative.

Method: Proposes ResAD framework that predicts residual deviations from deterministic inertial references rather than direct trajectories, with point-wise normalization to re-weight optimization objectives.

Result: Achieves state-of-the-art PDMS of 88.6 on NAVSIM benchmark using vanilla diffusion policy with only two denoising steps, demonstrating significant learning task simplification and performance improvement.

Conclusion: ResAD effectively addresses trajectory prediction imbalance in autonomous driving by reframing the learning task to focus on causal factors driving deviations from inertial paths, with normalization handling optimization challenges.

Abstract: End-to-end autonomous driving (E2EAD) systems, which learn to predict future trajectories directly from sensor data, are fundamentally challenged by the inherent spatio-temporal imbalance of trajectory data. This imbalance creates a significant optimization burden, causing models to learn spurious correlations instead of causal inference, while also prioritizing uncertain, distant predictions, thereby compromising immediate safety. To address these issues, we propose ResAD, a novel Normalized Residual Trajectory Modeling framework. Instead of predicting the future trajectory directly, our approach reframes the learning task to predict the residual deviation from a deterministic inertial reference. The inertial reference serves as a counterfactual, forcing the model to move beyond simple pattern recognition and instead identify the underlying causal factors (e.g., traffic rules, obstacles) that necessitate deviations from a default, inertially-guided path. To deal with the optimization imbalance caused by uncertain, long-term horizons, ResAD further incorporates Point-wise Normalization of the predicted residual. It re-weights the optimization objective, preventing large-magnitude errors associated with distant, uncertain waypoints from dominating the learning signal. Extensive experiments validate the effectiveness of our framework. On the NAVSIM benchmark, ResAD achieves a state-of-the-art PDMS of 88.6 using a vanilla diffusion policy with only two denoising steps, demonstrating that our approach significantly simplifies the learning task and improves model performance. The code will be released to facilitate further research.

Method: Systematically studied design space and scaling properties of native MLLMs under data constraints, identified optimal meta-architecture, and proposed NaViL with a cost-effective training recipe.

Result: Found positively correlated scaling relationship between visual encoders and LLMs. NaViL achieves competitive performance on 14 multimodal benchmarks compared to existing MLLMs.

Conclusion: The findings provide in-depth insights for future native MLLM research, demonstrating the viability and benefits of end-to-end training over compositional approaches.

Abstract: Compositional training has been the de-facto paradigm in existing Multimodal Large Language Models (MLLMs), where pre-trained vision encoders are connected with pre-trained LLMs through continuous multimodal pre-training. However, the multimodal scaling property of this paradigm remains difficult to explore due to the separated training. In this paper, we focus on the native training of MLLMs in an end-to-end manner and systematically study its design space and scaling property under a practical setting, i.e., data constraint. Through careful study of various choices in MLLM, we obtain the optimal meta-architecture that best balances performance and training cost. After that, we further explore the scaling properties of the native MLLM and indicate the positively correlated scaling relationship between visual encoders and LLMs. Based on these findings, we propose a native MLLM called NaViL, combined with a simple and cost-effective recipe. Experimental results on 14 multimodal benchmarks confirm the competitive performance of NaViL against existing MLLMs. Besides that, our findings and results provide in-depth insights for the future study of native MLLMs.

Method: Proposes D²GS framework with two components: 1) Depth-and-Density Guided Dropout strategy that adaptively masks redundant Gaussians based on density and depth to suppress overfitting, and 2) Distance-Aware Fidelity Enhancement module that improves reconstruction quality in under-fitted far-field areas through targeted supervision.

Result: Extensive experiments on multiple datasets demonstrate significant improvements in both visual quality and robustness under sparse view conditions. Also introduces a new evaluation metric to quantify Gaussian distribution stability.

Conclusion: The proposed D²GS framework effectively addresses the key challenges of 3D Gaussian Splatting in sparse-view settings, providing more stable and higher-quality novel view synthesis.

Abstract: Recent advances in 3D Gaussian Splatting (3DGS) enable real-time, high-fidelity novel view synthesis (NVS) with explicit 3D representations. However, performance degradation and instability remain significant under sparse-view conditions. In this work, we identify two key failure modes under sparse-view conditions: overfitting in regions with excessive Gaussian density near the camera, and underfitting in distant areas with insufficient Gaussian coverage. To address these challenges, we propose a unified framework D$^2$GS, comprising two key components: a Depth-and-Density Guided Dropout strategy that suppresses overfitting by adaptively masking redundant Gaussians based on density and depth, and a Distance-Aware Fidelity Enhancement module that improves reconstruction quality in under-fitted far-field areas through targeted supervision. Moreover, we introduce a new evaluation metric to quantify the stability of learned Gaussian distributions, providing insights into the robustness of the sparse-view 3DGS. Extensive experiments on multiple datasets demonstrate that our method significantly improves both visual quality and robustness under sparse view conditions. The project page can be found at: https://insta360-research-team.github.io/DDGS-website/.

Method: Developed a pipeline that constructs M-TRACE dataset (28.5K tasks, 177K trajectories) for imitation-based tuning, then created MATRIX Agent controller, and further optimized it with Pref-X (11K preference pairs) via step-wise preference learning.

Result: MATRIX consistently surpasses both open- and closed-source VLMs across three benchmarks (Agent-X, GTA, and GAIA), demonstrating scalable and effective multimodal tool use.

Conclusion: The framework enables scalable training of VLM controllers for complex reasoning through automated trajectory synthesis and preference learning, with publicly available data and code.

Abstract: Vision language models (VLMs) are increasingly deployed as controllers with access to external tools for complex reasoning and decision-making, yet their effectiveness remains limited by the scarcity of high-quality multimodal trajectories and the cost of manual annotation. We address this challenge with a vision-centric agent tuning framework that automatically synthesizes multimodal trajectories, generates step-wise preference pairs, and trains a VLM controller for robust tool-use reasoning. Our pipeline first constructs M-TRACE, a large-scale dataset of 28.5K multimodal tasks with 177K verified trajectories, enabling imitation-based trajectory tuning. Building on this, we develop MATRIX Agent, a controller finetuned on M-TRACE for step-wise tool reasoning. To achieve finer alignment, we further introduce Pref-X, a set of 11K automatically generated preference pairs, and optimize MATRIX on it via step-wise preference learning. Across three benchmarks, Agent-X, GTA, and GAIA, MATRIX consistently surpasses both open- and closed-source VLMs, demonstrating scalable and effective multimodal tool use. Our data and code is avaliable at https://github.com/mbzuai-oryx/MATRIX.

Method: Uses recurrent network with rendering error feedback to iteratively update Gaussians without gradients, plus a compact reconstruction model in 16x subsampled space for initialization.

Result: Achieves SOTA performance across various input views, resolutions, and datasets while reducing Gaussians by 16x and improving rendering speed.

Conclusion: ReSplat demonstrates that rendering error feedback enables effective Gaussian refinement, enabling robust generalization and computational efficiency.

Abstract: While feed-forward Gaussian splatting models provide computational efficiency and effectively handle sparse input settings, their performance is fundamentally limited by the reliance on a single forward pass during inference. We propose ReSplat, a feed-forward recurrent Gaussian splatting model that iteratively refines 3D Gaussians without explicitly computing gradients. Our key insight is that the Gaussian splatting rendering error serves as a rich feedback signal, guiding the recurrent network to learn effective Gaussian updates. This feedback signal naturally adapts to unseen data distributions at test time, enabling robust generalization. To initialize the recurrent process, we introduce a compact reconstruction model that operates in a $16 \times$ subsampled space, producing $16 \times$ fewer Gaussians than previous per-pixel Gaussian models. This substantially reduces computational overhead and allows for efficient Gaussian updates. Extensive experiments across varying of input views (2, 8, 16), resolutions ($256 \times 256$ to $540 \times 960$), and datasets (DL3DV and RealEstate10K) demonstrate that our method achieves state-of-the-art performance while significantly reducing the number of Gaussians and improving the rendering speed. Our project page is at https://haofeixu.github.io/resplat/.

Method: Designed smart glasses platform based on GAP9 multi-core RISC-V processor and proposed family of sub-million parameter TinyissimoYOLO networks for object detection.

Result: Achieved 17ms inference latency, 1.59mJ energy per inference, 56ms end-to-end latency (18 FPS), 62.9mW total power consumption, and 9.3 hours continuous runtime on 154mAh battery.

Conclusion: The proposed solution outperforms existing approaches like MCUNet, demonstrating efficient always-on object detection on smart glasses with extended battery life.

Abstract: Smart glasses are rapidly gaining advanced functions thanks to cutting-edge computing technologies, especially accelerated hardware architectures, and tiny Artificial Intelligence (AI) algorithms. However, integrating AI into smart glasses featuring a small form factor and limited battery capacity remains challenging for a satisfactory user experience. To this end, this paper proposes the design of a smart glasses platform for always-on on-device object detection with an all-day battery lifetime. The proposed platform is based on GAP9, a novel multi-core RISC-V processor from Greenwaves Technologies. Additionally, a family of sub-million parameter TinyissimoYOLO networks are proposed. They are benchmarked on established datasets, capable of differentiating up to 80 classes on MS-COCO. Evaluations on the smart glasses prototype demonstrate TinyissimoYOLO’s inference latency of only 17ms and consuming 1.59mJ energy per inference. An end-to-end latency of 56ms is achieved which is equivalent to 18 frames per seconds (FPS) with a total power consumption of 62.9mW. This ensures continuous system runtime of up to 9.3 hours on a 154mAh battery. These results outperform MCUNet (TinyNAS+TinyEngine), which runs a simpler task (image classification) at just 7.3 FPS, while the 18 FPS achieved in this paper even include image-capturing, network inference, and detection post-processing. The algorithm’s code is released open with this paper and can be found here: https://github.com/ETH-PBL/TinyissimoYOLO

Method: I&S-ViT introduces: (1) A shift-uniform-log2 quantizer (SULQ) with shift mechanism and uniform quantization for inclusive domain representation and accurate distribution approximation; (2) A three-stage smooth optimization strategy (SOS) combining channel-wise and layer-wise quantization for stable learning.

Result: Comprehensive evaluations show I&S-ViT’s superiority over existing PTQ methods, particularly in low-bit scenarios. It elevates the performance of 3-bit ViT-B by 50.68%.

Conclusion: I&S-ViT effectively addresses quantization inefficiency and loss landscape issues in ViT PTQ, achieving significant performance improvements and making ViTs more practical for industrial applications.

Abstract: Albeit the scalable performance of vision transformers (ViTs), the dense computational costs (training & inference) undermine their position in industrial applications. Post-training quantization (PTQ), tuning ViTs with a tiny dataset and running in a low-bit format, well addresses the cost issue but unluckily bears more performance drops in lower-bit cases. In this paper, we introduce I&S-ViT, a novel method that regulates the PTQ of ViTs in an inclusive and stable fashion. I&S-ViT first identifies two issues in the PTQ of ViTs: (1) Quantization inefficiency in the prevalent log2 quantizer for post-Softmax activations; (2) Rugged and magnified loss landscape in coarse-grained quantization granularity for post-LayerNorm activations. Then, I&S-ViT addresses these issues by introducing: (1) A novel shift-uniform-log2 quantizer (SULQ) that incorporates a shift mechanism followed by uniform quantization to achieve both an inclusive domain representation and accurate distribution approximation; (2) A three-stage smooth optimization strategy (SOS) that amalgamates the strengths of channel-wise and layer-wise quantization to enable stable learning. Comprehensive evaluations across diverse vision tasks validate I&S-ViT’ superiority over existing PTQ of ViTs methods, particularly in low-bit scenarios. For instance, I&S-ViT elevates the performance of 3-bit ViT-B by an impressive 50.68%.

Method: Uses pyramid-based strategy to extract fragments from word images, trains attention-driven CNN on these fragments to capture multi-level features, and integrates attention mechanism for enhanced feature representation.

Result: The system demonstrates proficiency on three benchmark databases, showing robust performance particularly in scenarios with limited handwriting data access.

Conclusion: The attention-driven CNN with fragment-based training provides an effective solution for writer identification when only limited word images are available, outperforming traditional methods.

Abstract: Writer identification due to its widespread application in various fields has gained popularity over the years. In scenarios where optimum handwriting samples are available, whether they be in the form of a single line, a sentence, or an entire page, writer identification algorithms have demonstrated noteworthy levels of accuracy. However, in scenarios where only a limited number of handwritten samples are available, particularly in the form of word images, there is a significant scope for improvement. In this paper, we propose a writer identification system based on an attention-driven Convolutional Neural Network (CNN). The system is trained utilizing image segments, known as fragments, extracted from word images, employing a pyramid-based strategy. This methodology enables the system to capture a comprehensive representation of the data, encompassing both fine-grained details and coarse features across various levels of abstraction. These extracted fragments serve as the training data for the convolutional network, enabling it to learn a more robust representation compared to traditional convolution-based networks trained on word images. Additionally, the paper explores the integration of an attention mechanism to enhance the representational power of the learned features. The efficacy of the proposed algorithm is evaluated on three benchmark databases, demonstrating its proficiency in writer identification tasks, particularly in scenarios with limited access to handwriting data.

Method: Proposes misleading-learning strategy that populates the latent space with redundant environments, exposing detectors to rich and balanced high-level information for demographic fairness.

Result: Extensive evaluations show superior fairness and generalization compared to state-of-the-art approaches, while maintaining high detection performance.

Conclusion: The misleading-learning framework effectively mitigates demographic bias in Deepfake detection and achieves better fairness and generalization than existing methods.

Abstract: Detecting falsified faces generated by Deepfake technology is essential for safeguarding trust in digital communication and protecting individuals. However, current detectors often suffer from a dual-overfitting: they become overly specialized in both specific forgery fingerprints and particular demographic attributes. Critically, most existing methods overlook the latter issue, which results in poor fairness: faces from certain demographic groups, such as different genders or ethnicities, are consequently more difficult to reliably detect. To address this challenge, we propose a novel strategy called misleading-learning, which populates the latent space with a multitude of redundant environments. By exposing the detector to a sufficiently rich and balanced variety of high-level information for demographic fairness, our approach mitigates demographic bias while maintaining a high detection performance level. We conduct extensive evaluations on fairness, intra-domain detection, cross-domain generalization, and robustness. Experimental results demonstrate that our framework achieves superior fairness and generalization compared to state-of-the-art approaches.

Method: Post-processing technique that cascades activation functions with novel operators to sparsify mean-centered feature vectors, effectively reducing feature variations around the mean.

Result: Achieved new robustness records: 75.28% on CIFAR-10 (from 73.71%), 44.78% on CIFAR-100 (from 42.67%), and 62.12% on ImageNet (from 59.56%) in AutoAttack accuracy.

Conclusion: MeanSparse provides a simple yet effective method to significantly improve adversarial robustness with minimal impact on model utility, establishing new state-of-the-art performance across multiple benchmarks.

Abstract: We present a simple yet effective method to improve the robustness of both Convolutional and attention-based Neural Networks against adversarial examples by post-processing an adversarially trained model. Our technique, MeanSparse, cascades the activation functions of a trained model with novel operators that sparsify mean-centered feature vectors. This is equivalent to reducing feature variations around the mean, and we show that such reduced variations merely affect the model’s utility, yet they strongly attenuate the adversarial perturbations and decrease the attacker’s success rate. Our experiments show that, when applied to the top models in the RobustBench leaderboard, MeanSparse achieves a new robustness record of 75.28% (from 73.71%), 44.78% (from 42.67%) and 62.12% (from 59.56%) on CIFAR-10, CIFAR-100 and ImageNet, respectively, in terms of AutoAttack accuracy. Code is available at https://github.com/SPIN-UMass/MeanSparse

Method: Integrates pre-integration and image-based lighting for fast light calculations, proposes occlusion approximation strategy and progressive training approach to address material lighting disentanglement and geometry reconstruction challenges.

Result: Achieves highly accurate physical properties, significantly enhances realistic relighting of dynamic human avatars, and provides substantial speed advantage over existing methods.

Conclusion: SGIA demonstrates superior performance in relightable dynamic human reconstruction with efficient training and rendering capabilities suitable for entertainment applications.

Abstract: Efficient and accurate reconstruction of a relightable, dynamic clothed human avatar from a monocular video is crucial for the entertainment industry. This paper presents SGIA (Surfel-based Gaussian Inverse Avatar), which introduces efficient training and rendering for relightable dynamic human reconstruction. SGIA advances previous Gaussian Avatar methods by comprehensively modeling Physically-Based Rendering (PBR) properties for clothed human avatars, allowing for the manipulation of avatars into novel poses under diverse lighting conditions. Specifically, our approach integrates pre-integration and image-based lighting for fast light calculations that surpass the performance of existing implicit-based techniques. To address challenges related to material lighting disentanglement and accurate geometry reconstruction, we propose an innovative occlusion approximation strategy and a progressive training approach. Extensive experiments demonstrate that SGIA not only achieves highly accurate physical properties but also significantly enhances the realistic relighting of dynamic human avatars, providing a substantial speed advantage. We exhibit more results in our project page: https://GS-IA.github.io.

Method: Created MINIONS dataset with over 5M frames, 400 minutes of multi-modal data (IMU signals + RGB videos) labeled with joint positions, rotations, and SMPL parameters. Proposed SparseNet framework to capture motion by discovering supplementary features between IMUs and videos.

Result: The framework successfully demonstrates the complementary advantages of inertial and vision sensors, showing promise for consumer-affordable multi-modal motion capture.

Conclusion: MINIONS provides a valuable resource for research and development in consumer motion capture, showcasing the potential of combining monocular cameras with few IMUs for accurate human motion tracking in daily life applications.

Abstract: Human motion capture is the foundation for many computer vision and graphics tasks. While industrial motion capture systems with complex camera arrays or expensive wearable sensors have been widely adopted in movie and game production, consumer-affordable and easy-to-use solutions for personal applications are still far from mature. To utilize a mixture of a monocular camera and very few inertial measurement units (IMUs) for accurate multi-modal human motion capture in daily life, we contribute MINIONS in this paper, a large-scale Motion capture dataset collected from INertial and visION Sensors. MINIONS has several featured properties: 1) large scale of over five million frames and 400 minutes duration; 2) multi-modality data of IMUs signals and RGB videos labeled with joint positions, joint rotations, SMPL parameters, etc.; 3) a diverse set of 146 fine-grained single and interactive actions with textual descriptions. With the proposed MINIONS dataset, we propose a SparseNet framework to capture human motion from IMUs and videos by discovering their supplementary features and exploring the possibilities of consumer-affordable motion capture using a monocular camera and very few IMUs. The experiment results emphasize the unique advantages of inertial and vision sensors, showcasing the promise of consumer-affordable multi-modal motion capture and providing a valuable resource for further research and development.

Method: Derived a white-box decoder (DEPICT) interpreting self-attention as constructing an ideal principal subspace, cross-attention as finding low-rank approximation for orthonormal bases, and dot-product as yielding compact segmentation masks.

Result: DEPICT consistently outperforms Segmenter on ADE20K dataset, is lightweight, and more robust.

Conclusion: The PCA-based interpretation provides theoretical foundations for Transformer decoders in semantic segmentation, enabling principled design and improved performance.

Abstract: State-of-the-art methods for Transformer-based semantic segmentation typically adopt Transformer decoders that are used to extract additional embeddings from image embeddings via cross-attention, refine either or both types of embeddings via self-attention, and project image embeddings onto the additional embeddings via dot-product. Despite their remarkable success, these empirical designs still lack theoretical justifications or interpretations, thus hindering potentially principled improvements. In this paper, we argue that there are fundamental connections between semantic segmentation and compression, especially between the Transformer decoders and Principal Component Analysis (PCA). From such a perspective, we derive a white-box, fully attentional DEcoder for PrIncipled semantiC segemenTation (DEPICT), with the interpretations as follows: 1) the self-attention operator refines image embeddings to construct an ideal principal subspace that aligns with the supervision and retains most information; 2) the cross-attention operator seeks to find a low-rank approximation of the refined image embeddings, which is expected to be a set of orthonormal bases of the principal subspace and corresponds to the predefined classes; 3) the dot-product operation yields compact representation for image embeddings as segmentation masks. Experiments conducted on dataset ADE20K find that DEPICT consistently outperforms its black-box counterpart, Segmenter, and it is light weight and more robust.

Method: Proposes parameterized spine contour representation in latent space with eigen-spine decomposition, latent contour coefficient regression combined with anchor box classification, and an iterative data engine with image self-generation, automatic annotation, and selection.

Result: Achieves state-of-the-art Cobb angle estimation performance on public AASCE2019, private Spinal2023, and generated Spinal-AI2024 datasets. Creates the largest released scoliosis X-ray dataset without privacy leaks.

Conclusion: CurvNet effectively addresses key challenges in automatic Cobb angle measurement through novel contour representation and data generation techniques, providing superior performance and valuable dataset resources.

Abstract: Curvature angle is a quantitative measurement of a curve, in which Cobb angle is customized for spinal curvature. Automatic Cobb angle measurement from X-ray images is crucial for scoliosis screening and diagnosis. However, most existing regression-based and segmentation-based methods struggle with inaccurate spine representations or mask connectivity and fragmentation issues. Besides, landmark-based methods suffer from insufficient training data and annotations. To address these challenges, we propose a novel curvature angle estimation framework named CurvNet including latent contour representation based contour detection and iterative data engine based image self-generation. Specifically, we propose a parameterized spine contour representation in latent space, which enables eigen-spine decomposition and spine contour reconstruction. Latent contour coefficient regression is combined with anchor box classification to solve inaccurate predictions and mask connectivity issues. Moreover, we develop a data engine with image self-generation, automatic annotation, and automatic selection in an iterative manner. By our data engine, we generate a clean dataset named Spinal-AI2024 without privacy leaks, which is the largest released scoliosis X-ray dataset to our knowledge. Extensive experiments on public AASCE2019, our private Spinal2023, and our generated Spinal-AI2024 datasets demonstrate that our method achieves state-of-the-art Cobb angle estimation performance. Our code and Spinal-AI2024 dataset are available at https://github.com/Ernestchenchen/CurvNet and https://github.com/Ernestchenchen/Spinal-AI2024, respectively.

Method: Couples Gaussian-based primitives with neural SDF, uses SDF to guide Gaussians’ spatial distribution during training, employs Gaussians as priors for surface reconstruction at inference, implements scaling strategy for arbitrary-scale scenes, and uses multi-resolution training with monocular geometric cues.

Result: Outperforms prior methods on real-world datasets while maintaining efficiency, achieving high-quality surface reconstruction without memory-intensive Marching Cubes.

Conclusion: MonoGSDF successfully bridges the gap between high-quality rendering and surface reconstruction by integrating Gaussian splatting with SDF representation, enabling efficient and accurate meshing from monocular images.

Abstract: Accurate meshing from monocular images remains a key challenge in 3D vision. While state-of-the-art 3D Gaussian Splatting (3DGS) methods excel at synthesizing photorealistic novel views through rasterization-based rendering, their reliance on sparse, explicit primitives severely limits their ability to recover watertight and topologically consistent 3D surfaces.We introduce MonoGSDF, a novel method that couples Gaussian-based primitives with a neural Signed Distance Field (SDF) for high-quality reconstruction. During training, the SDF guides Gaussians’ spatial distribution, while at inference, Gaussians serve as priors to reconstruct surfaces, eliminating the need for memory-intensive Marching Cubes. To handle arbitrary-scale scenes, we propose a scaling strategy for robust generalization. A multi-resolution training scheme further refines details and monocular geometric cues from off-the-shelf estimators enhance reconstruction quality. Experiments on real-world datasets show MonoGSDF outperforms prior methods while maintaining efficiency.

Method: Uses a two-process framework: (1) Retrieval-based structure aggregation with contrastive learning and Structure Locally Linear Embedding for global structure and spatial landmarks, and (2) Omni-level faithful garment generation with coarse-to-fine texture alignment for pattern and detail fidelity.

Result: Extensive experiments show RAGDiffusion synthesizes structurally and texture-faithful clothing assets with significant performance improvements on challenging real-world datasets.

Conclusion: RAGDiffusion represents a pioneering effort in high-specification faithful generation using RAG to confront intrinsic hallucinations and enhance fidelity in clothing asset generation.

Abstract: Standard clothing asset generation involves restoring forward-facing flat-lay garment images displayed on a clear background by extracting clothing information from diverse real-world contexts, which presents significant challenges due to highly standardized structure sampling distributions and clothing semantic absence in complex scenarios. Existing models have limited spatial perception, often exhibiting structural hallucinations and texture distortion in this high-specification generative task. To address this issue, we propose a novel Retrieval-Augmented Generation (RAG) framework, termed RAGDiffusion, to enhance structure determinacy and mitigate hallucinations by assimilating knowledge from language models and external databases. RAGDiffusion consists of two processes: (1) Retrieval-based structure aggregation, which employs contrastive learning and a Structure Locally Linear Embedding (SLLE) to derive global structure and spatial landmarks, providing both soft and hard guidance to counteract structural ambiguities; and (2) Omni-level faithful garment generation, which introduces a coarse-to-fine texture alignment that ensures fidelity in pattern and detail components within the diffusing. Extensive experiments on challenging real-world datasets demonstrate that RAGDiffusion synthesizes structurally and texture-faithful clothing assets with significant performance improvements, representing a pioneering effort in high-specification faithful generation with RAG to confront intrinsic hallucinations and enhance fidelity.

Method: Episodic Few-Shot Adaptation (EFSA) dynamically adapts pre-trained models by fine-tuning on top-k retrieved candidates and their synthetic captions at test time for each query.

Result: EFSA improves performance across eight diverse visual domains and maintains generalization on an open-domain retrieval pool of over one million images.

Conclusion: Episodic few-shot adaptation shows strong potential for enhancing robustness in open-domain text-to-image retrieval, addressing the challenge of hard negatives.

Abstract: Text-to-image retrieval is a critical task for managing diverse visual content, but common benchmarks for the task rely on small, single-domain datasets that fail to capture real-world complexity. Pre-trained vision-language models tend to perform well with easy negatives but struggle with hard negatives–visually similar yet incorrect images–especially in open-domain scenarios. To address this, we introduce Episodic Few-Shot Adaptation (EFSA), a novel test-time framework that adapts pre-trained models dynamically to a query’s domain by fine-tuning on top-k retrieved candidates and synthetic captions generated for them. EFSA improves performance across diverse domains while preserving generalization, as shown in evaluations on queries from eight highly distinct visual domains and an open-domain retrieval pool of over one million images. Our work highlights the potential of episodic few-shot adaptation to enhance robustness in the critical and understudied task of open-domain text-to-image retrieval.

Method: Integrates pre-trained foundation models with serverless cloud infrastructure via Function-as-a-Service (FaaS), using redshift prediction with AstroMAE as case study.

Result: Achieved inference on 12.6GB dataset in 28s vs 140.8s on HPC GPUs and 1793s on HPC CPUs, with 18.04 bps throughput and near-constant inference times as data scales, processing up to 1TB data.

Conclusion: CAI provides highly scalable, accessible, and cost-effective inference solution for astronomy community, enabling efficient processing of large-scale astronomical data.

Abstract: Large-scale astronomical image data processing and prediction are essential for astronomers, providing crucial insights into celestial objects, the universe’s history, and its evolution. While modern deep learning models offer high predictive accuracy, they often demand substantial computational resources, making them resource-intensive and limiting accessibility. We introduce the Cloud-based Astronomy Inference (CAI) framework to address these challenges. This scalable solution integrates pre-trained foundation models with serverless cloud infrastructure through a Function-as-a-Service (FaaS). CAI enables efficient and scalable inference on astronomical images without extensive hardware. Using a foundation model for redshift prediction as a case study, our extensive experiments cover user devices, HPC (High-Performance Computing) servers, and Cloud. Using redshift prediction with the AstroMAE model demonstrated CAI’s scalability and efficiency, achieving inference on a 12.6 GB dataset in only 28 seconds compared to 140.8 seconds on HPC GPUs and 1793 seconds on HPC CPUs. CAI also achieved significantly higher throughput, reaching 18.04 billion bits per second (bps), and maintained near-constant inference times as data sizes increased, all at minimal computational cost (under $5 per experiment). We also process large-scale data up to 1 TB to show CAI’s effectiveness at scale. CAI thus provides a highly scalable, accessible, and cost-effective inference solution for the astronomy community. The code is accessible at https://github.com/UVA-MLSys/AI-for-Astronomy.

Method: Self-training with dynamic weighting mechanism using time-varying hyperparameter to control domain-specific learning strength and optimize weighted objective function.

Result: Outperforms existing baselines on rotated MNIST, color-shifted MNIST, portrait datasets, and Cover Type dataset; ablation studies confirm importance of dynamic scheduling.

Conclusion: Provides theoretical insights and practical framework for robust gradual domain adaptation with applications in dynamic real-world scenarios.

Abstract: In this paper, we propose a new method called Self-Training with Dynamic Weighting (STDW), which aims to enhance robustness in Gradual Domain Adaptation (GDA) by addressing the challenge of smooth knowledge migration from the source to the target domain. Traditional GDA methods mitigate domain shift through intermediate domains and self-training but often suffer from inefficient knowledge migration or incomplete intermediate data. Our approach introduces a dynamic weighting mechanism that adaptively balances the loss contributions of the source and target domains during training. Specifically, we design an optimization framework governed by a time-varying hyperparameter $\varrho$ (progressing from 0 to 1), which controls the strength of domain-specific learning and ensures stable adaptation. The method leverages self-training to generate pseudo-labels and optimizes a weighted objective function for iterative model updates, maintaining robustness across intermediate domains. Experiments on rotated MNIST, color-shifted MNIST, portrait datasets, and the Cover Type dataset demonstrate that STDW outperforms existing baselines. Ablation studies further validate the critical role of $\varrho$’s dynamic scheduling in achieving progressive adaptation, confirming its effectiveness in reducing domain bias and improving generalization. This work provides both theoretical insights and a practical framework for robust gradual domain adaptation, with potential applications in dynamic real-world scenarios. The code is available at https://github.com/Dramwig/STDW.

Method: Proposes H3DE-Net framework that integrates CNNs for local feature extraction with a lightweight attention mechanism using hierarchical routing strategy to reduce computational cost while maintaining global context modeling, plus multi-scale feature fusion for enhanced accuracy.

Result: Experimental results on public CT dataset show H3DE-Net achieves state-of-the-art performance, significantly improving accuracy and robustness, particularly in scenarios with missing landmarks or complex anatomical variations.

Conclusion: H3DE-Net is the first 3D landmark detection model to integrate lightweight attention mechanism with CNNs, providing an efficient and precise solution for medical image analysis with open-source availability.

Abstract: 3D landmark detection is a critical task in medical image analysis, and accurately detecting anatomical landmarks is essential for subsequent medical imaging tasks. However, mainstream deep learning methods in this field struggle to simultaneously capture fine-grained local features and model global spatial relationships, while maintaining a balance between accuracy and computational efficiency. Local feature extraction requires capturing fine-grained anatomical details, while global modeling requires understanding the spatial relationships within complex anatomical structures. The high-dimensional nature of 3D volume further exacerbates these challenges, as landmarks are sparsely distributed, leading to significant computational costs. Therefore, achieving efficient and precise 3D landmark detection remains a pressing challenge in medical image analysis. In this work, We propose a \textbf{H}ybrid \textbf{3}D \textbf{DE}tection \textbf{Net}(H3DE-Net), a novel framework that combines CNNs for local feature extraction with a lightweight attention mechanism designed to efficiently capture global dependencies in 3D volumetric data. This mechanism employs a hierarchical routing strategy to reduce computational cost while maintaining global context modeling. To our knowledge, H3DE-Net is the first 3D landmark detection model that integrates such a lightweight attention mechanism with CNNs. Additionally, integrating multi-scale feature fusion further enhances detection accuracy and robustness. Experimental results on a public CT dataset demonstrate that H3DE-Net achieves state-of-the-art(SOTA) performance, significantly improving accuracy and robustness, particularly in scenarios with missing landmarks or complex anatomical variations. We aready open-source our project, including code, data and model weights.

Method: Two-stage framework: 1) Selective weight subcloning and layered initialization from Transformer pre-trained models, 2) Adaptive multi-directional knowledge distillation with layer-wise scaling factors to align representations while handling scanning order variations.

Result: TransMamba consistently outperforms baseline approaches across diverse Mamba backbones and downstream tasks (image classification, VQA, text-video retrieval, multimodal reasoning) despite using reduced training data and more compact architectures.

Conclusion: The proposed cross-architecture knowledge transfer paradigm effectively accelerates Mamba training by leveraging Transformer pre-trained knowledge, demonstrating superior performance across uni-modal and multi-modal tasks with resource efficiency.

Abstract: Transformer-based architectures have become the backbone of both uni-modal and multi-modal foundation models, largely due to their scalability via attention mechanisms, resulting in a rich ecosystem of publicly available pre-trained models such as LLaVA, CLIP, and DeiT, etc. In parallel, emerging sub-quadratic architectures like Mamba offer promising efficiency gains by enabling global context modeling with linear complexity. However, training these architectures from scratch remains resource-intensive (e.g., in terms of data and time). Motivated by this challenge, we explore a cross-architecture knowledge transfer paradigm, termed TransMamba, that facilitates the reuse of Transformer pre-trained knowledge. We propose a two-stage framework to accelerate the training of Mamba-based models, ensuring their effectiveness across both uni-modal and multi-modal tasks. The first stage leverages pre-trained Transformer models to initialize critical components of the Mamba architecture. To bridge architectural and dimensional gaps, we develop a selective weight subcloning strategy and a layered initialization scheme that prioritizes the early $n$ layers. Building on this initialization, the second stage introduces an adaptive multi-directional knowledge distillation method. This mechanism employs layer-wise adaptive scaling factors to align Mamba representations with their Transformer counterparts, while accommodating the scanning order variations inherent to multi-modal Mamba architectures. Despite operating with a reduced training dataset and a more compact model architecture, TransMamba consistently outperforms baseline approaches across diverse mamba-based backbones (e.g., PlainMamba, Vmamba, ViM and VideoMamba) and downstream tasks (e.g., image classification, visual question answering, text-video retrieval and multimodal reasoning). All code and implementation details will be released.

Method: Leverage text-to-image diffusion models pre-trained on billions of images, maximize preservation of pre-trained model’s prior knowledge, use pixel-aligned training, and apply classifier-free guidance for improved performance.

Result: DICEPTION effectively tackles diverse perception tasks, achieving performance comparable to SOTA single-task specialist models while using only 0.06% of their data (600K vs 1B images). Can be fine-tuned on as few as 50 images and 1% of parameters for novel tasks.

Conclusion: DICEPTION offers valuable insights and presents a promising direction for developing advanced diffusion-based visual generalist models with substantially lower computational costs than conventional models.

Abstract: This paper’s primary objective is to develop a robust generalist perception model capable of addressing multiple tasks under constraints of computational resources and limited training data. We leverage text-to-image diffusion models pre-trained on billions of images and successfully introduce our DICEPTION, a visual generalist model. Exhaustive evaluations demonstrate that DICEPTION effectively tackles diverse perception tasks, even achieving performance comparable to SOTA single-task specialist models. Specifically, we achieve results on par with SAM-vit-h using only 0.06% of their data (e.g., 600K vs.
1B pixel-level annotated images). We designed comprehensive experiments on architectures and input paradigms, demonstrating that the key to successfully re-purposing a single diffusion model for multiple perception tasks lies in maximizing the preservation of the pre-trained model’s prior knowledge. Consequently, DICEPTION can be trained with substantially lower computational costs than conventional models requiring training from scratch. Furthermore, adapting DICEPTION to novel tasks is highly efficient, necessitating fine-tuning on as few as 50 images and approximately 1% of its parameters. Finally, we demonstrate that a subtle application of classifier-free guidance can improve the model’s performance on depth and normal estimation. We also show that pixel-aligned training, as is characteristic of perception tasks, significantly enhances the model’s ability to preserve fine details. DICEPTION offers valuable insights and presents a promising direction for the development of advanced diffusion-based visual generalist models. Code and Model: https://github.com/aim-uofa/Diception

Method: Proposed spiking attention block (SAB) that optimizes membrane potentials through inferred attention weights to regulate spiking activity for superior feature representation. Bridges temporal and channel dimension modulation and accesses global self-similar patterns in RS imagery for spatial attention.

Result: SpikeSR achieves state-of-the-art performance across various remote sensing benchmarks (AID, DOTA, DIOR) while maintaining high computational efficiency.

Conclusion: The proposed SpikeSR framework successfully applies SNNs to remote sensing super-resolution through innovative spiking attention mechanisms, demonstrating superior performance and efficiency compared to existing methods.

Abstract: Spiking neural networks (SNNs) are emerging as a promising alternative to traditional artificial neural networks (ANNs), offering biological plausibility and energy efficiency. Despite these merits, SNNs are frequently hampered by limited capacity and insufficient representation power, yet remain underexplored in remote sensing super-resolution (SR) tasks. In this paper, we first observe that spiking signals exhibit drastic intensity variations across diverse textures, highlighting an active learning state of the neurons. This observation motivates us to apply SNNs for efficient SR of RSIs. Inspired by the success of attention mechanisms in representing salient information, we devise the spiking attention block (SAB), a concise yet effective component that optimizes membrane potentials through inferred attention weights, which, in turn, regulates spiking activity for superior feature representation. Our key contributions include: 1) we bridge the independent modulation between temporal and channel dimensions, facilitating joint feature correlation learning, and 2) we access the global self-similar patterns in large-scale remote sensing imagery to infer spatial attention weights, incorporating effective priors for realistic and faithful reconstruction. Building upon SAB, we proposed SpikeSR, which achieves state-of-the-art performance across various remote sensing benchmarks such as AID, DOTA, and DIOR, while maintaining high computational efficiency. Code of SpikeSR will be available at https://github.com/XY-boy/SpikeSR.

Method: Leverages pre-trained latent video diffusion model for iterative refinement of coarse Gaussian scene representation, incorporates Fourier-style transfer for texture alignment, and uses semantic uncertainty quantification to guide refinement.

Result: Extensive experiments on RealEstate-10K and KITTI-v2 datasets show more realistic and high-fidelity novel view synthesis compared to state-of-the-art methods.

Conclusion: The proposed approach effectively addresses limitations of single image 3D reconstruction by combining generative priors with uncertainty-guided refinement and style transfer.

Abstract: Reconstructing 3D scenes from a single image is a fundamentally ill-posed task due to the severely under-constrained nature of the problem. Consequently, when the scene is rendered from novel camera views, existing single image to 3D reconstruction methods render incoherent and blurry views. This problem is exacerbated when the unseen regions are far away from the input camera. In this work, we address these inherent limitations in existing single image-to-3D scene feedforward networks. To alleviate the poor performance due to insufficient information beyond the input image’s view, we leverage a strong generative prior in the form of a pre-trained latent video diffusion model, for iterative refinement of a coarse scene represented by optimizable Gaussian parameters. To ensure that the style and texture of the generated images align with that of the input image, we incorporate on-the-fly Fourier-style transfer between the generated images and the input image. Additionally, we design a semantic uncertainty quantification module that calculates the per-pixel entropy and yields uncertainty maps used to guide the refinement process from the most confident pixels while discarding the remaining highly uncertain ones. We conduct extensive experiments on real-world scene datasets, including in-domain RealEstate-10K and out-of-domain KITTI-v2, showing that our approach can provide more realistic and high-fidelity novel view synthesis results compared to existing state-of-the-art methods.

Method: Jointly optimizes sensor poses with neural Gaussian-based scene representation, freezing reliable LiDAR points as anchor Gaussians to preserve global structure, using auxiliary Gaussians to prevent local overfitting, with fully differentiable pipeline and photometric/geometric regularization.

Result: Consistently outperforms existing targetless methods on KITTI-360, Waymo, and FAST-LIVO2 datasets, and surpasses even provided calibrations in rendering quality.

Conclusion: The proposed targetless calibration method provides robust and generalizable calibration that addresses practical deployment challenges and outperforms both existing targetless methods and manual calibrations.

Abstract: Accurate LiDAR-camera calibration is crucial for multi-sensor systems. However, traditional methods often rely on physical targets, which are impractical for real-world deployment. Moreover, even carefully calibrated extrinsics can degrade over time due to sensor drift or external disturbances, necessitating periodic recalibration. To address these challenges, we present a Targetless LiDAR-Camera Calibration (TLC-Calib) that jointly optimizes sensor poses with a neural Gaussian-based scene representation. Reliable LiDAR points are frozen as anchor Gaussians to preserve global structure, while auxiliary Gaussians prevent local overfitting under noisy initialization. Our fully differentiable pipeline with photometric and geometric regularization achieves robust and generalizable calibration, consistently outperforming existing targetless methods on KITTI-360, Waymo, and FAST-LIVO2, and surpassing even the provided calibrations in rendering quality.

Method: Three-stage framework: (1) domain adaptation via self-training with contrastive calibration, (2) knowledge transfer through unified multi-objective loss, (3) student refinement to reduce pseudo-label bias. Uses instance-aware pixel-wise contrastive loss to extract negatives and enforce inter-instance margins.

Result: On Cityscapes and ADE20K, the ~11x smaller student model improved over zero-shot VFM teachers by +8.5 and +7.1 AP, surpassed adapted teachers by +3.4 and +1.5 AP, and outperformed state-of-the-art SSKD methods on both benchmarks.

Conclusion: CAST effectively compresses vision foundation models into efficient instance segmentation experts while maintaining performance, demonstrating the value of semi-supervised knowledge distillation for this task.

Abstract: Instance segmentation demands costly per-pixel annotations and computationally expensive models. We introduce CAST, a semi-supervised knowledge distillation (SSKD) framework that compresses pre-trained vision foundation models (VFM) into compact experts using limited labeled and abundant unlabeled data. CAST unfolds in three stages: (1) domain adaptation of the VFM(s) via self-training with contrastive calibration, (2) knowledge transfer through a unified multi-objective loss, and (3) student refinement to mitigate residual pseudo-label bias. Central to CAST is an \emph{instance-aware pixel-wise contrastive loss} that fuses mask and class scores to extract informative negatives and enforce clear inter-instance margins. By maintaining this contrastive signal across both adaptation and distillation, we align teacher and student embeddings and fully leverage unlabeled images. On Cityscapes and ADE20K, our ~11x smaller student improves over its zero-shot VFM teacher(s) by +8.5 and +7.1 AP, surpasses adapted teacher(s) by +3.4 and +1.5 AP, and further outperforms state-of-the-art SSKD methods on both benchmarks.

Method: Proposes DvD with coordinate-level denoising instead of pixel-level denoising to generate deformation rectification mappings, plus a time-variant condition refinement mechanism to enhance structure preservation.

Result: Achieves state-of-the-art performance with acceptable computational efficiency on multiple metrics across DocUNet, DIR300, and their new AnyPhotoDoc6300 benchmark comprising 6,300 real image pairs.

Conclusion: DvD demonstrates the successful application of diffusion models to document dewarping while preserving document structures, and introduces a comprehensive benchmark for future evaluation.

Abstract: Document dewarping aims to rectify deformations in photographic document images, thus improving text readability, which has attracted much attention and made great progress, but it is still challenging to preserve document structures. Given recent advances in diffusion models, it is natural for us to consider their potential applicability to document dewarping. However, it is far from straightforward to adopt diffusion models in document dewarping due to their unfaithful control on highly complex document images (e.g., 2000$times$3000 resolution). In this paper, we propose DvD, the first generative model to tackle document Dewarping via a Diffusion framework. To be specific, DvD introduces a coordinate-level denoising instead of typical pixel-level denoising, generating a mapping for deformation rectification. In addition, we further propose a time-variant condition refinement mechanism to enhance the preservation of document structures. In experiments, we find that current document dewarping benchmarks can not evaluate dewarping models comprehensively. To this end, we present AnyPhotoDoc6300, a rigorously designed large-scale document dewarping benchmark comprising 6,300 real image pairs across three distinct domains, enabling fine-grained evaluation of dewarping models. Comprehensive experiments demonstrate that our proposed DvD can achieve state-of-the-art performance with acceptable computational efficiency on multiple metrics across various benchmarks, including DocUNet, DIR300, and AnyPhotoDoc6300. The new benchmark and code will be publicly available at https://github.com/hanquansanren/DvD.

Method: Three key innovations: 1) Hierarchical framework with global correction and local refinement, 2) Dual-Branch Local-Refine Network using parametric generation instead of direct prediction, 3) Modified NestUNet with dense blocks for feature fusion. Uses two-stage training with large-scale synthetic data pretraining.

Result: Achieves state-of-the-art SSIM scores: 0.7721 on DocUNet and 0.9480 on RealDAE. Shows remarkable cross-domain adaptability and maintains computational efficiency for high-resolution images.

Conclusion: GL-PGENet provides an efficient and robust solution for multi-degraded color document image enhancement, confirming practical utility in real-world scenarios with superior performance and generalization capabilities.

Abstract: Document Image Enhancement (DIE) serves as a critical component in Document AI systems, where its performance substantially determines the effectiveness of downstream tasks. To address the limitations of existing methods confined to single-degradation restoration or grayscale image processing, we present Global with Local Parametric Generation Enhancement Network (GL-PGENet), a novel architecture designed for multi-degraded color document images, ensuring both efficiency and robustness in real-world scenarios. Our solution incorporates three key innovations: First, a hierarchical enhancement framework that integrates global appearance correction with local refinement, enabling coarse-to-fine quality improvement. Second, a Dual-Branch Local-Refine Network with parametric generation mechanisms that replaces conventional direct prediction, producing enhanced outputs through learned intermediate parametric representations rather than pixel-wise mapping. This approach enhances local consistency while improving model generalization. Finally, a modified NestUNet architecture incorporating dense block to effectively fuse low-level pixel features and high-level semantic features, specifically adapted for document image characteristics. In addition, to enhance generalization performance, we adopt a two-stage training strategy: large-scale pretraining on a synthetic dataset of 500,000+ samples followed by task-specific fine-tuning. Extensive experiments demonstrate the superiority of GL-PGENet, achieving state-of-the-art SSIM scores of 0.7721 on DocUNet and 0.9480 on RealDAE. The model also exhibits remarkable cross-domain adaptability and maintains computational efficiency for high-resolution images without performance degradation, confirming its practical utility in real-world scenarios.

Method: The approach uses bin-based cumulative distribution function (CDF) approximation to enable efficient gradient-based optimization without expensive sorting, achieving O(nK) complexity. It shares a similar reweighting strategy with dual focal loss but offers greater flexibility through choice of confidence score functions.

Result: Empirical evaluations demonstrate that the method achieves competitive calibration performance across a range of datasets and model architectures.

Conclusion: Minimizing calibration error is closely linked to the selective classification paradigm, and optimizing selective risk in low-confidence region naturally leads to improved calibration.

Abstract: Several variants of reweighted risk functionals, such as focal loss, inverse focal loss, and the Area Under the Risk–Coverage Curve (AURC), have been proposed for improving model calibration, yet their theoretical connections to calibration errors remain unclear. In this paper, we revisit a broad class of weighted risk functions commonly used in deep learning and establish a principled connection between calibration error and selective classification. We show that minimizing calibration error is closely linked to the selective classification paradigm and demonstrate that optimizing selective risk in low-confidence region naturally leads to improved calibration. This loss shares a similar reweighting strategy with dual focal loss but offers greater flexibility through the choice of confidence score functions (CSFs). Our approach uses a bin-based cumulative distribution function (CDF) approximation, enabling efficient gradient-based optimization without requiring expensive sorting and achieving $O(nK)$ complexity. Empirical evaluations demonstrate that our method achieves competitive calibration performance across a range of datasets and model architectures.

Method: Proposes MAGREF with two key components: (1) masked guidance using region-aware masking and pixel-wise channel concatenation to preserve appearance features, and (2) subject disentanglement mechanism that injects semantic values from text conditions into corresponding visual regions. Also uses a four-stage data pipeline for training.

Result: Extensive experiments show MAGREF consistently outperforms existing state-of-the-art approaches, achieving scalable, controllable, and high-fidelity any-reference video synthesis.

Conclusion: MAGREF provides an effective solution for any-reference video generation, addressing key challenges and paving the way for improved video synthesis capabilities without requiring architectural changes to pre-trained backbones.

Abstract: We tackle the task of any-reference video generation, which aims to synthesize videos conditioned on arbitrary types and combinations of reference subjects, together with textual prompts. This task faces persistent challenges, including identity inconsistency, entanglement among multiple reference subjects, and copy-paste artifacts. To address these issues, we introduce MAGREF, a unified and effective framework for any-reference video generation. Our approach incorporates masked guidance and a subject disentanglement mechanism, enabling flexible synthesis conditioned on diverse reference images and textual prompts. Specifically, masked guidance employs a region-aware masking mechanism combined with pixel-wise channel concatenation to preserve appearance features of multiple subjects along the channel dimension. This design preserves identity consistency and maintains the capabilities of the pre-trained backbone, without requiring any architectural changes. To mitigate subject confusion, we introduce a subject disentanglement mechanism which injects the semantic values of each subject derived from the text condition into its corresponding visual region. Additionally, we establish a four-stage data pipeline to construct diverse training pairs, effectively alleviating copy-paste artifacts. Extensive experiments on a comprehensive benchmark demonstrate that MAGREF consistently outperforms existing state-of-the-art approaches, paving the way for scalable, controllable, and high-fidelity any-reference video synthesis. Code and model can be found at: https://github.com/MAGREF-Video/MAGREF

Method: Human-curated queries spanning real-world remote sensing applications (urban planning, disaster assessment, environmental monitoring, etc.) grounded in satellite/aerial imagery, using ReAct-style interaction loop with diverse toolset.

Result: Evaluation of open and closed-source LLMs (GPT-4o, Qwen2.5) revealed notable disparities in tool accuracy and planning consistency across models.

Conclusion: ThinkGeo provides the first extensive testbed for evaluating how tool-enabled LLMs handle spatial reasoning in remote sensing, addressing the gap in domain-specific benchmarks.

Abstract: Recent progress in large language models (LLMs) has enabled tool-augmented agents capable of solving complex real-world tasks through step-by-step reasoning. However, existing evaluations often focus on general-purpose or multimodal scenarios, leaving a gap in domain-specific benchmarks that assess tool-use capabilities in complex remote sensing use cases. We present ThinkGeo, an agentic benchmark designed to evaluate LLM-driven agents on remote sensing tasks via structured tool use and multi-step planning. Inspired by tool-interaction paradigms, ThinkGeo includes human-curated queries spanning a wide range of real-world applications such as urban planning, disaster assessment and change analysis, environmental monitoring, transportation analysis, aviation monitoring, recreational infrastructure, and industrial site analysis. Queries are grounded in satellite or aerial imagery, including both optical RGB and SAR data, and require agents to reason through a diverse toolset. We implement a ReAct-style interaction loop and evaluate both open and closed-source LLMs (e.g., GPT-4o, Qwen2.5) on 486 structured agentic tasks with 1,773 expert-verified reasoning steps. The benchmark reports both step-wise execution metrics and final answer correctness. Our analysis reveals notable disparities in tool accuracy and planning consistency across models. ThinkGeo provides the first extensive testbed for evaluating how tool-enabled LLMs handle spatial reasoning in remote sensing.

Method: Zero-shot method using object-centric visual spotlight and motion blur as visual prompts to improve motion perception and decouple object/camera motion cues without training.

Result: MotionSight achieves state-of-the-art open-source performance and competitiveness with commercial models for fine-grained motion understanding.

Conclusion: The study presents a novel zero-shot technique and large-scale dataset that effectively unlock MLLMs’ inherent capabilities for fine-grained video motion understanding.

Abstract: Despite advancements in Multimodal Large Language Models (MLLMs), their proficiency in fine-grained video motion understanding remains critically limited. They often lack inter-frame differencing and tend to average or ignore subtle visual cues. Furthermore, while visual prompting has shown potential in static images, its application to video’s temporal complexities, particularly for fine-grained motion understanding, remains largely unexplored. We investigate whether inherent capability can be unlocked and boost MLLMs’ motion perception and enable distinct visual signatures tailored to decouple object and camera motion cues. In this study, we introduce MotionSight, a novel zero-shot method pioneering object-centric visual spotlight and motion blur as visual prompts to effectively improve fine-grained motion understanding without training. To convert this into valuable data assets, we curated MotionVid-QA, the first large-scale dataset for fine-grained video motion understanding, with hierarchical annotations including SFT and preference data, {\Theta}(40K) video clips and {\Theta}(87K) QAs. Experiments show MotionSight achieves state-of-the-art open-source performance and competitiveness with commercial models. In particular, for fine-grained motion understanding we present a novel zero-shot technique and a large-scale, high-quality dataset. All the code and annotations will be publicly available.

Method: Uses a plug-and-play harmony aware (HA) module that fuses perception semantics while modeling object counts and locations, plus a preference-guided noise selection (PNS) strategy that selects semantically aligned initial noise.

Result: Outperforms prior methods in structural alignment and semantic accuracy using only 200 training images and 10.6M trainable parameters.

Conclusion: IMAGHarmony provides effective control over object quantity and layout in multi-object scene editing while maintaining strong structural consistency.

Abstract: Recent diffusion models have advanced image editing by improving fidelity and controllability across creative and personalized applications. However, multi-object scenes remain challenging, as reliable control over object categories, counts, and spatial layout is difficult to achieve. For that, we first study quantity and layout consistent image editing, abbreviated as QL-Edit, which targets control of object quantity and spatial layout in multi-object scenes. Then, we present IMAGHarmony, a straightforward framework featuring a plug-and-play harmony aware (HA) module that fuses perception semantics while modeling object counts and locations, resulting in accurate edits and strong structural consistency. We further observe that diffusion models are sensitive to the choice of initial noise and tend to prefer certain noise patterns. Based on this finding, we present a preference-guided noise selection (PNS) strategy that selects semantically aligned initial noise through vision and language matching, thereby further improving generation stability and layout consistency in multiple object editing. To support evaluation, we develop HarmonyBench, a comprehensive benchmark that covers a diverse range of quantity and layout control scenarios. Extensive experiments demonstrate that IMAGHarmony outperforms prior methods in both structural alignment and semantic accuracy, utilizing only 200 training images and 10.6M of trainable parameters. Code, models, and data are available at https://github.com/muzishen/IMAGHarmony.

Method: OASIS estimates each sample’s informativeness relative to all previously seen data (beyond batch-level) and minimizes informative redundancy through iterative selection score updates.

Result: Experiments show OASIS achieves comparable performance to full-data training using only 25% of data and outperforms state-of-the-art sampling methods on various large foundation models.

Conclusion: OASIS provides an effective adaptive online sample selection approach for continual instruction tuning that addresses limitations of existing methods and enables efficient real-time adaptation.

Abstract: In continual instruction tuning (CIT) scenarios, where new instruction tuning data continuously arrive in an online streaming manner, training delays from large-scale data significantly hinder real-time adaptation. Data selection can mitigate this overhead, but existing strategies often rely on pretrained reference models, which are impractical in CIT setups since future data are unknown. Recent reference model-free online sample selection methods address this, but typically select a fixed number of samples per batch (e.g., top-k), making them vulnerable to distribution shifts where informativeness varies across batches. To address these limitations, we propose OASIS, an adaptive online sample selection approach for CIT that (1) selects informative samples by estimating each sample’s informativeness relative to all previously seen data, beyond batch-level constraints, and (2) minimizes informative redundancy of selected samples through iterative selection score updates. Experiments on various large foundation models show that OASIS, using only 25 percent of the data, achieves comparable performance to full-data training and outperforms the state-of-the-art sampling methods.

Method: Tested 111 VLMs and 65 humans using photos with manipulated difficulty and variability to characterize gaze inference skills.

Result: 94 VLMs performed no better than random guessing, while humans achieved near-ceiling accuracy. Top-tier VLMs showed above-chance performance that declined with task difficulty but was robust to different prompts and scene objects.

Conclusion: VLMs lack effective gaze inference skills needed for natural human interaction, but show potential as they likely use head orientation (not eye appearance) to infer gaze direction.

Abstract: The ability to infer what others are looking at is a critical component of a theory of mind that underpins natural human-AI interaction. We characterized this skill in 111 Vision Language Models (VLMs) and human participants (N = 65) using photos taken with manipulated difficulty and variability. We found that 94 of the 111 VLMs were not better than random guessing, while humans achieved near-ceiling accuracy. VLMs respond with each choice almost equally frequently. Are they randomly guessing? At least for five top-tier VLMs, their performance was above chance, declined with increasing task difficulty, but barely varied across different prompts and scene objects. These behavioral patterns cannot be explained by considering VLMs as random guessers. Instead, they likely utilize head orientation but not eye appearance to infer gaze direction, such that their performance is imperfect, subject to the task difficulty, but robust to superficial perceptual variations. This suggests that VLMs, lacking effective gaze inference skills, have yet to become technologies that can naturally interact with humans, but the potential remains.

Method: FBG uses a state-dependent coefficient to self-regulate guidance amounts based on need, derived from first principles assuming the learned conditional distribution is linearly corrupted by the unconditional distribution. It relies on feedback of its own predictions about conditional signal informativeness to adapt guidance dynamically.

Result: On ImageNet512x512, FBG significantly outperforms CFG and is competitive with LIG. On Text-To-Image generation, it automatically applies higher guidance for complex prompts than simpler ones and can be combined with existing guidance schemes.

Conclusion: FBG provides a mathematically grounded alternative to CFG that dynamically adapts guidance during inference, challenging the view of guidance as a fixed hyperparameter.

Abstract: While Classifier-Free Guidance (CFG) has become standard for improving sample fidelity in conditional diffusion models, it can harm diversity and induce memorization by applying constant guidance regardless of whether a particular sample needs correction. We propose FeedBack Guidance (FBG), which uses a state-dependent coefficient to self-regulate guidance amounts based on need. Our approach is derived from first principles by assuming the learned conditional distribution is linearly corrupted by the unconditional distribution, contrasting with CFG’s implicit multiplicative assumption. Our scheme relies on feedback of its own predictions about the conditional signal informativeness to adapt guidance dynamically during inference, challenging the view of guidance as a fixed hyperparameter. The approach is benchmarked on ImageNet512x512, where it significantly outperforms Classifier-Free Guidance and is competitive to Limited Interval Guidance (LIG) while benefitting from a strong mathematical framework. On Text-To-Image generation, we demonstrate that, as anticipated, our approach automatically applies higher guidance scales for complex prompts than for simpler ones and that it can be easily combined with existing guidance schemes such as CFG or LIG.

Method: Training a 7B-parameter MLLM via reinforcement learning on simple arcade-like games (e.g., Snake) without seeing worked solutions, equations, or diagrams during RL.

Result: Significantly enhances performance on multimodal math benchmarks (MathVista), multi-discipline questions (MMMU), and 3D spatial reasoning benchmarks (VSI-Bench), outperforms specialist models while preserving general visual capabilities.

Conclusion: Multimodal reasoning can emerge from gameplay, suggesting a promising strategy of using surrogate tasks for RL post-training to develop generalizable reasoning skills.

Abstract: Developing reasoning capabilities in multimodal large language models (MLLMs) remains challenging. Motivated by literature suggesting that gameplay promotes transferable reasoning skills, we propose a novel post-training method, Visual Game Learning (ViGaL), where MLLMs develop generalizable reasoning skills through playing arcade-like games. Specifically, we show that training a 7B-parameter MLLM via reinforcement learning (RL) on simple games like Snake significantly enhances the downstream performance on multimodal math benchmarks like MathVista, on multi-discipline questions like MMMU and on 3D spatial reasoning benchmarks like VSI-Bench, without seeing any worked solutions, equations, or diagrams during RL. Remarkably, our model outperforms specialist models post-trained on benchmark-oriented multimodal reasoning data, while preserving the model’s performance on general visual benchmarks, a challenge where specialist models often fall short. Our findings suggest that multimodal reasoning can emerge from gameplay, pointing to a promising strategy of designing surrogate tasks for RL post-training.

Method: Product of Experts framework using Annealed Importance Sampling to sample from product distribution across experts, enabling training-free inference-time knowledge composition.

Result: Improved controllability in image and video synthesis tasks compared to monolithic methods, with flexible user interfaces for specifying visual generation goals.

Conclusion: The proposed PoE framework effectively combines diverse knowledge sources for enhanced visual generation without requiring training, offering better controllability and user flexibility.

Abstract: Modern neural models capture rich priors and have complementary knowledge over shared data domains, e.g., images and videos. Integrating diverse knowledge from multiple sources – including visual generative models, visual language models, and sources with human-crafted knowledge such as graphics engines and physics simulators – remains under-explored. We propose a Product of Experts (PoE) framework that performs inference-time knowledge composition from heterogeneous models. This training-free approach samples from the product distribution across experts via Annealed Importance Sampling (AIS). Our framework shows practical benefits in image and video synthesis tasks, yielding better controllability than monolithic methods and additionally providing flexible user interfaces for specifying visual generation goals.

Method: VideoExplorer iteratively formulates sub-questions, locates relevant moments, and performs task-oriented, temporally scalable video understanding. It uses a two-stage training pipeline with supervised trajectory initialization followed by trajectory-level preference optimization on a constructed long-video reasoning dataset.

Result: Extensive evaluations on popular long-video understanding benchmarks demonstrate VideoExplorer’s significant advantage over existing baselines, showing robustness, adaptability, and efficiency.

Conclusion: VideoExplorer enables faithful, efficient, and interpretable reasoning for long-video understanding through its iterative “thinking with video” approach and specialized training methodology.

Abstract: Long-video understanding~(LVU) is a challenging problem in computer vision. Existing methods either downsample frames for single-pass reasoning, sacrificing fine-grained details, or depend on textual reasoning over task-agnostic representations, hindering task-specific perception and exploration. In this paper, we propose VideoExplorer, a framework grounded in the principle of ``thinking with video’’, which naturally intertwines planning, temporal grounding, and scalable perception into a coherent reasoning process. Rather than reasoning over a static context, VideoExplorer iteratively formulates sub-questions, locates relevant moments, and performs task-oriented, temporally scalable video understanding until reaching the final answer, enabling faithful, efficient, and interpretable reasoning. To address the lack of LVU training resources, we construct a long-video reasoning dataset using difficulty-adaptive sampling to ensure high-quality trajectories on complex tasks. Building on this dataset, we design a two-stage training pipeline: supervised trajectory initialization followed by trajectory-level preference optimization, encouraging adaptive temporal grounding and iterative information integration guided by downstream rewards. Extensive evaluations on popular long-video understanding and reasoning benchmarks demonstrate VideoExplorer’s significant advantage over existing baselines, highlighting its robustness, adaptability, and efficiency. Our code is made publicly available in this repository(https://github.com/yhy-2000/VideoDeepResearch).

Method: Constructed from smartphone slow-motion videos by using 240 frames over one second, simulating realistic long-exposure blur through frame averaging to create blurry images, with the temporally centered frame serving as the sharp reference.

Result: The dataset contains over 42,000 high-resolution blur-sharp pairs, is 10x larger than widely used datasets, covers 8x more different scenes including indoor/outdoor environments with varying motions, and causes significant performance degradation in state-of-the-art deblurring models.

Conclusion: This dataset serves as a challenging new benchmark that better reflects real-world complexity and diversity, facilitating the development of more robust and generalizable deblurring models.

Abstract: We introduce the largest real-world image deblurring dataset constructed from smartphone slow-motion videos. Using 240 frames captured over one second, we simulate realistic long-exposure blur by averaging frames to produce blurry images, while using the temporally centered frame as the sharp reference. Our dataset contains over 42,000 high-resolution blur-sharp image pairs, making it approximately 10 times larger than widely used datasets, with 8 times the amount of different scenes, including indoor and outdoor environments, with varying object and camera motions. We benchmark multiple state-of-the-art (SOTA) deblurring models on our dataset and observe significant performance degradation, highlighting the complexity and diversity of our benchmark. Our dataset serves as a challenging new benchmark to facilitate robust and generalizable deblurring models.

Method: Created Zebra-CoT dataset with 182,384 samples containing logically coherent interleaved text-image reasoning traces across four task categories. Fine-tuned Anole-7B and Bagel-7B models on this dataset.

Result: Fine-tuning Anole-7B on Zebra-CoT improved test-set accuracy by +12% and yielded up to +13% performance gain on standard VLM benchmarks. Bagel-7B fine-tuning produced high-quality interleaved visual reasoning chains.

Conclusion: Zebra-CoT is effective for developing multimodal reasoning abilities and supports visual CoT development and evaluation. The dataset and models are open-sourced.

Abstract: Humans often use visual aids, for example diagrams or sketches, when solving complex problems. Training multimodal models to do the same, known as Visual Chain of Thought (Visual CoT), is challenging due to: (1) poor off-the-shelf visual CoT performance, which hinders reinforcement learning, and (2) the lack of high-quality visual CoT training data. We introduce $\textbf{Zebra-CoT}$, a diverse large-scale dataset with 182,384 samples, containing logically coherent interleaved text-image reasoning traces. We focus on four categories of tasks where sketching or visual reasoning is especially natural, spanning scientific questions such as geometry, physics, and algorithms; 2D visual reasoning tasks like visual search and jigsaw puzzles; 3D reasoning tasks including 3D multi-hop inference, embodied and robot planning; visual logic problems and strategic games like chess. Fine-tuning the Anole-7B model on the Zebra-CoT training corpus results in an improvement of +12% in our test-set accuracy and yields up to +13% performance gain on standard VLM benchmark evaluations. Fine-tuning Bagel-7B yields a model that generates high-quality interleaved visual reasoning chains, underscoring Zebra-CoT’s effectiveness for developing multimodal reasoning abilities. We open-source our dataset and models to support development and evaluation of visual CoT.

Method: Uses LaViLa video captioner on video segments, incorporates static scene descriptions using LLaVA VLM, and fine-tunes a controllable hybrid captioner that alternates between action and scene captions based on detected scene changes.

Result: Developed a more detailed caption log that expands answerable questions, and improved efficiency by combining action and scene captioning in a single model rather than using separate models.

Conclusion: The hybrid captioning approach with scene change detection creates more comprehensive text-based video memories, enabling better video understanding and query answering through LLMs.

Abstract: Video data, especially long-form video, is extremely dense and high-dimensional. Text-based summaries of video content offer a way to represent query-relevant content in a much more compact manner than raw video. In addition, textual representations are easily ingested by state-of-the-art large language models (LLMs), which enable reasoning over video content to answer complex natural language queries. To solve this issue, we rely on the progressive construction of a text-based memory by a video captioner operating on shorter chunks of the video, where spatio-temporal modeling is computationally feasible. We explore ways to improve the quality of the activity log comprised solely of short video captions. Because the video captions tend to be focused on human actions, and questions may pertain to other information in the scene, we seek to enrich the memory with static scene descriptions using Vision Language Models (VLMs). Our video understanding system relies on the LaViLa video captioner in combination with a LLM to answer questions about videos. We first explored different ways of partitioning the video into meaningful segments such that the textual descriptions more accurately reflect the structure of the video content. Furthermore, we incorporated static scene descriptions into the captioning pipeline using LLaVA VLM, resulting in a more detailed and complete caption log and expanding the space of questions that are answerable from the textual memory. Finally, we have successfully fine-tuned the LaViLa video captioner to produce both action and scene captions, significantly improving the efficiency of the captioning pipeline compared to using separate captioning models for the two tasks. Our model, controllable hybrid captioner, can alternate between different types of captions according to special input tokens that signals scene changes detected in the video.

Method: Proposes AlignCAT with two modules: 1) coarse-grained alignment using category information and global context to mitigate category-inconsistent objects, and 2) fine-grained alignment using descriptive information and word-level text features for attribute consistency. Progressively filters misaligned visual queries and enhances contrastive learning.

Result: Extensive experiments on RefCOCO, RefCOCO+, and RefCOCOg benchmarks verify superiority against existing weakly supervised methods on two VG tasks.

Conclusion: AlignCAT effectively addresses semantic ambiguity in weakly supervised visual grounding through progressive alignment modules that exploit linguistic cues to enhance visual-linguistic alignment.

Abstract: Weakly supervised visual grounding (VG) aims to locate objects in images based on text descriptions. Despite significant progress, existing methods lack strong cross-modal reasoning to distinguish subtle semantic differences in text expressions due to category-based and attribute-based ambiguity. To address these challenges, we introduce AlignCAT, a novel query-based semantic matching framework for weakly supervised VG. To enhance visual-linguistic alignment, we propose a coarse-grained alignment module that utilizes category information and global context, effectively mitigating interference from category-inconsistent objects. Subsequently, a fine-grained alignment module leverages descriptive information and captures word-level text features to achieve attribute consistency. By exploiting linguistic cues to their fullest extent, our proposed AlignCAT progressively filters out misaligned visual queries and enhances contrastive learning efficiency. Extensive experiments on three VG benchmarks, namely RefCOCO, RefCOCO+, and RefCOCOg, verify the superiority of AlignCAT against existing weakly supervised methods on two VG tasks. Our code is available at: https://github.com/I2-Multimedia-Lab/AlignCAT.

Method: Continual pre-training of vision models on large-scale time series with three innovations: vision-model-based filtering, colorized multivariate conversion (encoding series as RGB images), and multi-quantile forecasting with parallel reconstruction heads.

Result: Achieves state-of-the-art performance in both in-distribution and out-of-distribution forecasting, outperforming specialized TSFMs by 6%-44% in MSE reduction and ranking first in GIFT-Eval benchmark across 23 datasets and 7 domains.

Conclusion: With appropriate adaptation, vision models can effectively generalize to time series forecasting, advancing the pursuit of universal time series foundation models.

Abstract: Recent studies have indicated that vision models pre-trained on images can serve as time series foundation models (TSFMs) by reformulating time series forecasting (TSF) as image reconstruction. However, effective cross-modal transfer from vision to time series remains challenging due to three discrepancies: (1) the data-modality gap between structured, bounded image data and unbounded, heterogeneous time series; (2) the multivariate-forecasting gap between fixed RGB-three-channel vision models and time series with arbitrary numbers of variates; and (3) the probabilistic-forecasting gap between the deterministic outputs of vision models and the requirement for uncertainty-aware probabilistic predictions. To bridge these gaps, we propose VisonTS++, a TSFM based on continual pre-training of a vision model on large-scale time series. Our approach introduces three key innovations: (1) vision-model-based filtering to identify high-quality sequences to stabilize pre-training and mitigate modality gap; (2) colorized multivariate conversion, encoding multivariate series as multi-subfigure RGB images to enhance cross-variate modeling; (3) multi-quantile forecasting, using parallel reconstruction heads to generate quantile forecasts without parametric assumptions. Experiments show that VisionTS++ achieves state-of-the-art performance in both in-distribution and out-of-distribution forecasting, outperforming specialized TSFMs by 6%-44% in MSE reduction and ranking first in GIFT-Eval benchmark which comprises 23 datasets across 7 domains. Our work demonstrates that with appropriate adaptation, vision models can effectively generalize to TSF, thus advancing the pursuit of universal TSFMs. Code is available at https://github.com/HALF111/VisionTSpp.

Method: Proposes Q-CLIP with a Shared Cross-Modal Adapter (SCMA) containing few trainable parameters, learnable quality-level prompts to enhance quality sensitivity, and frame-difference-based sampling for better generalization.

Result: Extensive experiments show Q-CLIP achieves excellent performance on multiple VQA datasets while significantly reducing computational costs compared to traditional pretraining approaches.

Conclusion: Q-CLIP demonstrates that VLMs can be effectively adapted for VQA tasks with minimal training requirements, offering a computationally efficient alternative to conventional large-scale pretraining methods.

Abstract: Accurate and efficient Video Quality Assessment (VQA) has long been a key research challenge. Current mainstream VQA methods typically improve performance by pretraining on large-scale classification datasets (e.g., ImageNet, Kinetics-400), followed by fine-tuning on VQA datasets. However, this strategy presents two significant challenges: (1) merely transferring semantic knowledge learned from pretraining is insufficient for VQA, as video quality depends on multiple factors (e.g., semantics, distortion, motion, aesthetics); (2) pretraining on large-scale datasets demands enormous computational resources, often dozens or even hundreds of times greater than training directly on VQA datasets. Recently, Vision-Language Models (VLMs) have shown remarkable generalization capabilities across a wide range of visual tasks, and have begun to demonstrate promising potential in quality assessment. In this work, we propose Q-CLIP, the first fully VLMs-based framework for VQA. Q-CLIP enhances both visual and textual representations through a Shared Cross-Modal Adapter (SCMA), which contains only a minimal number of trainable parameters and is the only component that requires training. This design significantly reduces computational cost. In addition, we introduce a set of five learnable quality-level prompts to guide the VLMs in perceiving subtle quality variations, thereby further enhancing the model’s sensitivity to video quality. Furthermore, we investigate the impact of different frame sampling strategies on VQA performance, and find that frame-difference-based sampling leads to better generalization performance across datasets. Extensive experiments demonstrate that Q-CLIP exhibits excellent performance on several VQA datasets.

Method: Extends InDI concept from 2D to full 3D volumetric approach, implements iterative denoising process directly on CBCT volumes, and uses a large pseudo-CBCT dataset (16,182 volumes) generated from chest CT scans for robust training.

Result: Achieves 12.96 dB PSNR gain over uncorrected scans with only 50 projections, enables 8x radiation exposure reduction, matches 3D U-Net performance on real patient scans, generalizes to new scanner geometries, and receives positive clinical assessment from 11 clinicians.

Conclusion: MInDI-3D successfully demonstrates effective sparse-view CBCT reconstruction with significant radiation dose reduction while maintaining clinical utility, showing promise for safer medical imaging practices.

Abstract: We present MInDI-3D (Medical Inversion by Direct Iteration in 3D), the first 3D conditional diffusion-based model for real-world sparse-view Cone Beam Computed Tomography (CBCT) artefact removal, aiming to reduce imaging radiation exposure. A key contribution is extending the “InDI” concept from 2D to a full 3D volumetric approach for medical images, implementing an iterative denoising process that refines the CBCT volume directly from sparse-view input. A further contribution is the generation of a large pseudo-CBCT dataset (16,182) from chest CT volumes of the CT-RATE public dataset to robustly train MInDI-3D. We performed a comprehensive evaluation, including quantitative metrics, scalability analysis, generalisation tests, and a clinical assessment by 11 clinicians. Our results show MInDI-3D’s effectiveness, achieving a 12.96 (6.10) dB PSNR gain over uncorrected scans with only 50 projections on the CT-RATE pseudo-CBCT (independent real-world) test set and enabling an 8x reduction in imaging radiation exposure. We demonstrate its scalability by showing that performance improves with more training data. Importantly, MInDI-3D matches the performance of a 3D U-Net on real-world scans from 16 cancer patients across distortion and task-based metrics. It also generalises to new CBCT scanner geometries. Clinicians rated our model as sufficient for patient positioning across all anatomical sites and found it preserved lung tumour boundaries well.

Method: Uses reinforcement learning to train an agent that processes real-time multimodal inputs, builds episodic/semantic memories, and performs autonomous multi-turn reasoning with memory retrieval. Evaluated on M3-Bench benchmark with robot-perspective and web-sourced videos.

Result: Outperforms strongest baseline (Gemini-1.5-pro + GPT-4o) by 6.7% on M3-Bench-robot, 7.7% on M3-Bench-web, and 5.3% on VideoMME-long, demonstrating superior memory effectiveness and reasoning capabilities.

Conclusion: The work successfully advances multimodal agents with human-like long-term memory and provides practical design insights, with the framework showing significant improvements in memory-based reasoning tasks across diverse video benchmarks.

Abstract: We introduce M3-Agent, a novel multimodal agent framework equipped with long-term memory. Like humans, M3-Agent can process real-time visual and auditory inputs to build and update episodic and semantic memories, gradually accumulating world knowledge. Its memory is organized in an entity-centric, multimodal manner, enabling deeper and more consistent understanding of the environment. Given an instruction, M3-Agent autonomously performs multi-turn reasoning and retrieves relevant memories to complete tasks. To evaluate memory effectiveness and memory-based reasoning in multimodal agents, we develop M3-Bench, a long-video question answering benchmark comprising 100 newly recorded robot-perspective videos (M3-Bench-robot) and 920 diverse web-sourced videos (M3-Bench-web). We annotate QA pairs designed to test capabilities essential for agent applications, such as person understanding, general knowledge extraction, and cross-modal reasoning. Experimental results show that M3-Agent, trained via reinforcement learning, outperforms the strongest baseline, a prompting agent using Gemini-1.5-pro and GPT-4o, achieving 6.7%, 7.7%, and 5.3% higher accuracy on M3-Bench-robot, M3-Bench-web and VideoMME-long, respectively. Our work advances multimodal agents toward more human-like long-term memory and provides insights for their practical design. Model, code and data are available at https://github.com/bytedance-seed/m3-agent.

Method: Comprehensive benchmark of fusion strategies and normalization schemes, then introduced MAESTRO - a masked autoencoder with optimized fusion mechanisms and spectral prior normalization as self-supervisory signal.

Result: Achieved state-of-the-art performance on tasks relying on multitemporal dynamics across four Earth observation datasets in intra- and cross-dataset settings, while remaining competitive on other tasks.

Conclusion: MAESTRO effectively adapts self-supervised learning for Earth observation data through optimized fusion and spectral prior normalization, demonstrating strong performance particularly on temporal tasks.

Abstract: Self-supervised learning holds great promise for remote sensing, but standard self-supervised methods must be adapted to the unique characteristics of Earth observation data. We take a step in this direction by conducting a comprehensive benchmark of fusion strategies and normalization schemes of reconstruction targets for multimodal, multitemporal, and multispectral Earth observation data. Based on our findings, we introduce MAESTRO, a novel adaptation of the Masked Autoencoder with optimized fusion mechanisms and a normalization scheme that incorporates a spectral prior as a self-supervisory signal. Evaluated on four Earth observation datasets in both intra- and cross-dataset settings, MAESTRO achieves state-of-the-art performance on tasks that strongly rely on multitemporal dynamics, while also remaining competitive on others. Code to reproduce all our experiments is available at https://github.com/ignf/maestro.

Method: Combines Spherical Vision Transformer for 360-degree VR scene processing with LSTM-based temporal encoder for gaze patterns, using multi-modal fusion to integrate spatial features with temporal dynamics.

Result: Achieves median angular error of 3.83 degrees, outperforming traditional saliency-based baselines by 24% with reliable confidence calibration and consistent performance across spatial regions and scene types.

Conclusion: Software-only gaze prediction can effectively support VR foveated rendering, making performance improvements accessible across different VR platforms without additional hardware requirements.

Abstract: Foveated rendering significantly reduces computational demands in virtual reality applications by concentrating rendering quality where users focus their gaze. Current approaches require expensive hardware-based eye tracking systems, limiting widespread adoption due to cost, calibration complexity, and hardware compatibility constraints. This paper presents GazeProphet, a software-only approach for predicting gaze locations in VR environments without requiring dedicated eye tracking hardware. The approach combines a Spherical Vision Transformer for processing 360-degree VR scenes with an LSTM-based temporal encoder that captures gaze sequence patterns. A multi-modal fusion network integrates spatial scene features with temporal gaze dynamics to predict future gaze locations with associated confidence estimates. Experimental evaluation on a comprehensive VR dataset demonstrates that GazeProphet achieves a median angular error of 3.83 degrees, outperforming traditional saliency-based baselines by 24% while providing reliable confidence calibration. The approach maintains consistent performance across different spatial regions and scene types, enabling practical deployment in VR systems without additional hardware requirements. Statistical analysis confirms the significance of improvements across all evaluation metrics. These results show that software-only gaze prediction can work for VR foveated rendering, making this performance boost more accessible to different VR platforms and apps.

Method: Proposes CoMoE module that aggregates similar patch features and assigns them to expert modules for independent modeling, and WeaveNet - a dynamic connection mechanism that enables interaction between global text-level control and fine-grained conditional control.

Result: Extensive experiments on Subjects-200K and MultiGen-20M datasets demonstrate state-of-the-art performance across various conditional image generation tasks.

Conclusion: The proposed UniGen framework effectively mitigates feature entanglement and redundant computation while achieving superior performance in multi-condition scenarios.

Abstract: The image-to-image generation task aims to produce controllable images by leveraging conditional inputs and prompt instructions. However, existing methods often train separate control branches for each type of condition, leading to redundant model structures and inefficient use of computational resources. To address this, we propose a Unified image-to-image Generation (UniGen) framework that supports diverse conditional inputs while enhancing generation efficiency and expressiveness. Specifically, to tackle the widely existing parameter redundancy and computational inefficiency in controllable conditional generation architectures, we propose the Condition Modulated Expert (CoMoE) module. This module aggregates semantically similar patch features and assigns them to dedicated expert modules for visual representation and conditional modeling. By enabling independent modeling of foreground features under different conditions, CoMoE effectively mitigates feature entanglement and redundant computation in multi-condition scenarios. Furthermore, to bridge the information gap between the backbone and control branches, we propose WeaveNet, a dynamic, snake-like connection mechanism that enables effective interaction between global text-level control from the backbone and fine-grained control from conditional branches. Extensive experiments on the Subjects-200K and MultiGen-20M datasets across various conditional image generation tasks demonstrate that our method consistently achieves state-of-the-art performance, validating its advantages in both versatility and effectiveness. The code has been uploaded to https://github.com/gavin-gqzhang/UniGen.

Method: Safe-Control uses data-driven strategies and safety-aware conditions to inject safety control signals into locked T2I models in a patch-like manner. It can create various safety patches that can be merged into a unified patch, with plug-and-play design for adaptability across similar denoising architectures.

Result: Safe-Control reduces the probability of unsafe content generation to 7% compared to approximately 20% for most baseline methods, working effectively across six diverse T2I models while maintaining benign image quality and text alignment.

Conclusion: Safe-Control provides an effective, adaptable plug-and-play safety solution that significantly outperforms existing safety mechanisms in reducing unsafe content generation in T2I models.

Abstract: Despite the advancements in Text-to-Image (T2I) generation models, their potential for misuse or even abuse raises serious safety concerns. Model developers have made tremendous efforts to introduce safety mechanisms that can address these concerns in T2I models. However, the existing safety mechanisms, whether external or internal, either remain susceptible to evasion under distribution shifts or require extensive model-specific adjustments. To address these limitations, we introduce Safe-Control, an innovative plug-and-play safety patch designed to mitigate unsafe content generation in T2I models. Using data-driven strategies and safety-aware conditions, Safe-Control injects safety control signals into the locked T2I model, acting as an update in a patch-like manner. Model developers can also construct various safety patches to meet the evolving safety requirements, which can be flexibly merged into a single, unified patch. Its plug-and-play design further ensures adaptability, making it compatible with other T2I models of similar denoising architecture. We conduct extensive evaluations on six diverse and public T2I models. Empirical results highlight that Safe-Control is effective in reducing unsafe content generation across six diverse T2I models with similar generative architectures, yet it successfully maintains the quality and text alignment of benign images. Compared to seven state-of-the-art safety mechanisms, including both external and internal defenses, Safe-Control significantly outperforms all baselines in reducing unsafe content generation. For example, it reduces the probability of unsafe content generation to 7%, compared to approximately 20% for most baseline methods, under both unsafe prompts and the latest adversarial attacks.

Method: CLIP-DCA identifies and enhances domain awareness within CLIP’s encoders using a separate domain head and synthetically generated diverse domain data, while encouraging domain-invariant classification through disentanglement from domain features.

Result: CLIP-DCA shows significant improvements compared to existing methods, particularly on datasets that are more out-of-distribution, within the challenging evaluation framework.

Conclusion: Enhancing domain awareness is a prerequisite for effective domain-invariant classification in foundation models, and the proposed disentanglement approach outperforms standard domain invariance methods.

Abstract: Evaluating domain generalization (DG) for foundational models like CLIP is challenging, as web-scale pretraining data potentially covers many existing benchmarks. Consequently, current DG evaluation may neither be sufficiently challenging nor adequately test genuinely unseen data scenarios. To better assess the performance of CLIP on DG in-the-wild, a scenario where CLIP encounters challenging unseen data, we consider two approaches: (1) evaluating on 33 diverse datasets with quantified out-of-distribution (OOD) scores after fine-tuning CLIP on ImageNet, and (2) using unlearning to make CLIP `forget’ some domains as an approximation. We observe that CLIP’s performance deteriorates significantly on more OOD datasets. To address this, we present CLIP-DCA (Disentangling Classification from enhanced domain Aware representations). Our approach is motivated by the observation that while standard domain invariance losses aim to make representations domain-invariant, this can be harmful to foundation models by forcing the discarding of domain-aware representations beneficial for generalization. We instead hypothesize that enhancing domain awareness is a prerequisite for effective domain-invariant classification in foundation models. CLIP-DCA identifies and enhances domain awareness within CLIP’s encoders using a separate domain head and synthetically generated diverse domain data. Simultaneously, it encourages domain-invariant classification through disentanglement from the domain features. CLIP-DCA shows significant improvements within this challenging evaluation compared to existing methods, particularly on datasets that are more OOD.

Method: Proposed UnorthoDOS approach uses unorthorectified hyperspectral data from EMIT sensor, bypassing traditional preprocessing. Also trained models on orthorectified data for comparison.

Result: ML models on unorthorectified data perform comparably to orthorectified data. Orthorectified models outperform matched filter baseline (mag1c).

Conclusion: Unorthorectified data enables effective methane detection without preprocessing overhead, supporting rapid onboard satellite detection systems.

Abstract: Methane is a potent greenhouse gas and a major driver of climate change, making its timely detection critical for effective mitigation. Machine learning (ML) deployed onboard satellites can enable rapid detection while reducing downlink costs, supporting faster response systems. Conventional methane detection methods often rely on image processing techniques, such as orthorectification to correct geometric distortions and matched filters to enhance plume signals. We introduce a novel approach that bypasses these preprocessing steps by using \textit{unorthorectified} data (UnorthoDOS). We find that ML models trained on this dataset achieve performance comparable to those trained on orthorectified data. Moreover, we also train models on an orthorectified dataset, showing that they can outperform the matched filter baseline (mag1c). We release model checkpoints and two ML-ready datasets comprising orthorectified and unorthorectified hyperspectral images from the Earth Surface Mineral Dust Source Investigation (EMIT) sensor at https://huggingface.co/datasets/SpaceML/UnorthoDOS , along with code at https://github.com/spaceml-org/plume-hunter.

Method: Two-stage approach: PointAD uses implicit 3D representation via point-pixel correspondence; PointAD+ adds explicit 3D representation with G-aggregation for spatial awareness, hierarchical representation learning with rendering/geometry prompts, and cross-hierarchy contrastive alignment.

Result: Extensive experiments demonstrate superiority in zero-shot 3D anomaly detection across unseen objects with diverse class semantics, achieving holistic understanding of abnormality with plug-and-play RGB integration.

Conclusion: PointAD+ successfully transfers 2D generalization to 3D anomaly detection through comprehensive implicit and explicit representation learning, enabling robust detection across diverse unseen objects.

Abstract: In this paper, we aim to transfer CLIP’s robust 2D generalization capabilities to identify 3D anomalies across unseen objects of highly diverse class semantics. To this end, we propose a unified framework to comprehensively detect and segment 3D anomalies by leveraging both point- and pixel-level information. We first design PointAD, which leverages point-pixel correspondence to represent 3D anomalies through their associated rendering pixel representations. This approach is referred to as implicit 3D representation, as it focuses solely on rendering pixel anomalies but neglects the inherent spatial relationships within point clouds. Then, we propose PointAD+ to further broaden the interpretation of 3D anomalies by introducing explicit 3D representation, emphasizing spatial abnormality to uncover abnormal spatial relationships. Hence, we propose G-aggregation to involve geometry information to enable the aggregated point representations spatially aware. To simultaneously capture rendering and spatial abnormality, PointAD+ proposes hierarchical representation learning, incorporating implicit and explicit anomaly semantics into hierarchical text prompts: rendering prompts for the rendering layer and geometry prompts for the geometry layer. A cross-hierarchy contrastive alignment is further introduced to promote the interaction between the rendering and geometry layers, facilitating mutual anomaly learning. Finally, PointAD+ integrates anomaly semantics from both layers to capture the generalized anomaly semantics. During the test, PointAD+ can integrate RGB information in a plug-and-play manner and further improve its detection performance. Extensive experiments demonstrate the superiority of PointAD+ in ZS 3D anomaly detection across unseen objects with highly diverse class semantics, achieving a holistic understanding of abnormality.

Method: Use long Chain-of-Thought data for Supervised Fine-Tuning, followed by Reinforcement Learning stage.

Result: SFT with long CoT data significantly improves MLLM reasoning, and enables additional performance gains through subsequent RL.

Conclusion: SFT stage with long CoT data is a critical prerequisite for developing reasoning capabilities in lightweight MLLMs.

Abstract: While Reinforcement Learning with Verifiable Rewards has enhanced the reasoning of large-scale language models (LLMs), its efficacy for lightweight multimodal language models (MLLMs) with fewer than seven billion parameters remains underexplored. This paper investigates the role of long Chain-of-Thought (long CoT) data in enhancing the reasoning abilities of such MLLMs. Our findings demonstrate that Supervised Fine-Tuning (SFT) with long CoT data significantly improves MLLM reasoning. Furthermore, we observe that after this initial SFT phase, MLLMs can achieve additional performance gains through a subsequent RL stage. We conclude that a SFT stage with long CoT data is a critical prerequisite for developing the reasoning capabilities of lightweight MLLMs.

Method: Proposed CPS method that reformulates sampling process by drawing inspiration from DDIM to preserve coefficients and eliminate noise artifacts.

Result: CPS eliminates noise artifacts, enables more accurate reward modeling, and leads to faster and more stable convergence for RL optimizers like Flow-GRPO and Dance-GRPO.

Conclusion: The CPS method successfully addresses the noise artifact problem in RL-enhanced Flow Matching, improving reward learning and optimization convergence.

Abstract: Reinforcement Learning (RL) has recently emerged as a powerful technique for improving image and video generation in Diffusion and Flow Matching models, specifically for enhancing output quality and alignment with prompts. A critical step for applying online RL methods on Flow Matching is the introduction of stochasticity into the deterministic framework, commonly realized by Stochastic Differential Equation (SDE). Our investigation reveals a significant drawback to this approach: SDE-based sampling introduces pronounced noise artifacts in the generated images, which we found to be detrimental to the reward learning process. A rigorous theoretical analysis traces the origin of this noise to an excess of stochasticity injected during inference. To address this, we draw inspiration from Denoising Diffusion Implicit Models (DDIM) to reformulate the sampling process. Our proposed method, Coefficients-Preserving Sampling (CPS), eliminates these noise artifacts. This leads to more accurate reward modeling, ultimately enabling faster and more stable convergence for reinforcement learning-based optimizers like Flow-GRPO and Dance-GRPO. Code will be released at https://github.com/IamCreateAI/FlowCPS

Method: ProtoMedX uses a prototype-based architecture that combines DEXA scans of the lumbar spine with patient records in a multi-modal approach, designed to be explainable by design rather than relying on post hoc assessments.

Result: Achieved 87.58% accuracy in vision-only tasks and 89.8% in multi-modal variant using a dataset of 4,160 NHS patients, surpassing existing published methods while providing visually understandable explanations for clinicians.

Conclusion: ProtoMedX demonstrates that multi-modal approaches with built-in explainability can achieve superior performance in bone health classification while meeting the critical need for transparent AI in medical applications.

Abstract: Bone health studies are crucial in medical practice for the early detection and treatment of Osteopenia and Osteoporosis. Clinicians usually make a diagnosis based on densitometry (DEXA scans) and patient history. The applications of AI in this field are ongoing research. Most successful methods rely on deep learning models that use vision alone (DEXA/X-ray imagery) and focus on prediction accuracy, while explainability is often disregarded and left to post hoc assessments of input contributions. We propose ProtoMedX, a multi-modal (multimodal) model that uses both DEXA scans of the lumbar spine and patient records. ProtoMedX’s prototype-based architecture is explainable by design, which is crucial for medical applications, especially in the context of the upcoming EU AI Act, as it allows explicit analysis of model decisions, including incorrect ones. ProtoMedX demonstrates state-of-the-art performance in bone health classification while also providing explanations that can be visually understood by clinicians. Using a dataset of 4,160 real NHS patients, the proposed ProtoMedX achieves 87.58% accuracy in vision-only tasks and 89.8% in its multi-modal variant, both surpassing existing published methods.

Method: Uses diffusion model conditioned on visual stimuli with novel Corresponding Positional Embedding (CPE) to align spatial gaze information with patch-based semantic features, leveraging raw eye-tracking data instead of processed scanpaths.

Result: Achieves state-of-the-art performance in scanpath generation and enables first-time generation of continuous eye movement trajectories, producing high-quality realistic patterns despite small dataset training.

Conclusion: DiffEye successfully models the inherent variability in human gaze behavior and generates outputs that more accurately reflect human visual attention distribution, representing a significant advancement in eye movement modeling.

Abstract: Numerous models have been developed for scanpath and saliency prediction, which are typically trained on scanpaths, which model eye movement as a sequence of discrete fixation points connected by saccades, while the rich information contained in the raw trajectories is often discarded. Moreover, most existing approaches fail to capture the variability observed among human subjects viewing the same image. They generally predict a single scanpath of fixed, pre-defined length, which conflicts with the inherent diversity and stochastic nature of real-world visual attention. To address these challenges, we propose DiffEye, a diffusion-based training framework designed to model continuous and diverse eye movement trajectories during free viewing of natural images. Our method builds on a diffusion model conditioned on visual stimuli and introduces a novel component, namely Corresponding Positional Embedding (CPE), which aligns spatial gaze information with the patch-based semantic features of the visual input. By leveraging raw eye-tracking trajectories rather than relying on scanpaths, DiffEye captures the inherent variability in human gaze behavior and generates high-quality, realistic eye movement patterns, despite being trained on a comparatively small dataset. The generated trajectories can also be converted into scanpaths and saliency maps, resulting in outputs that more accurately reflect the distribution of human visual attention. DiffEye is the first method to tackle this task on natural images using a diffusion model while fully leveraging the richness of raw eye-tracking data. Our extensive evaluation shows that DiffEye not only achieves state-of-the-art performance in scanpath generation but also enables, for the first time, the generation of continuous eye movement trajectories. Project webpage: https://diff-eye.github.io/

Method: Built on convolutional vision transformer backbone with three key components: (1) component-wise convolution for joint processing of scalar/vector channels, (2) inter-field cross-attention for selective information propagation between fields, (3) axial attention to reduce computational burden while maintaining expressivity. Pretrained on diverse PDE datasets and evaluated with full fine-tuning and LoRA adapters.

Result: MORPH outperforms models trained from scratch in both zero-shot and full-shot generalization, matching or surpassing strong baselines and state-of-the-art models across extensive evaluations. The model demonstrates effective transfer learning to downstream prediction tasks.

Conclusion: MORPH presents a flexible and powerful backbone for learning from heterogeneous scientific observations, charting a path toward scalable and data-efficient scientific machine learning. The model successfully handles multimodal PDE data and enables efficient transfer learning.

Abstract: We introduce MORPH, a shape-agnostic, autoregressive foundation model for partial differential equations (PDEs). MORPH is built on a convolutional vision transformer backbone that seamlessly handles heterogeneous spatiotemporal datasets of varying data dimensionality (1D–3D) at different resolutions, multiple fields with mixed scalar and vector components. The architecture combines (i) component-wise convolution, which jointly processes scalar and vector channels to capture local interactions, (ii) inter-field cross-attention, which models and selectively propagates information between different physical fields, (iii) axial attentions, which factorizes full spatiotemporal self-attention along individual spatial and temporal axes to reduce computational burden while retaining expressivity. We pretrain multiple model variants on a diverse collection of heterogeneous PDE datasets and evaluate transfer to a range of downstream prediction tasks. Using both full-model fine-tuning and parameter-efficient low-rank adapters (LoRA), MORPH outperforms models trained from scratch in both zero-shot and full-shot generalization. Across extensive evaluations, MORPH matches or surpasses strong baselines and recent state-of-the-art models. Collectively, these capabilities present a flexible and powerful backbone for learning from heterogeneous and multimodal nature of scientific observations, charting a path toward scalable and data-efficient scientific machine learning. The source code, datasets, and models are publicly available at https://github.com/lanl/MORPH.

Method: ProDA uses Spatio-temporal Scene Graphs (SSGs) and introduces Dynamic Prompt Module (DPM) to guide a Graph Parsing Neural Network (GPNN) in generating action-specific representations. It features a video-adapted GPNN that aggregates information using dynamic weights.

Result: Experiments in video action recognition demonstrate the effectiveness of ProDA when compared with state-of-the-art methods.

Conclusion: The proposed ProDA framework successfully addresses the challenge of multi-action recognition by disentangling specified actions from complex scenes, showing improved performance over existing methods.

Abstract: Action recognition is a fundamental task in video understanding. Existing methods typically extract unified features to process all actions in one video, which makes it challenging to model the interactions between different objects in multi-action scenarios. To alleviate this issue, we explore disentangling any specified actions from complex scenes as an effective solution. In this paper, we propose Prompt-guided Disentangled Representation for Action Recognition (ProDA), a novel framework that disentangles any specified actions from a multi-action scene. ProDA leverages Spatio-temporal Scene Graphs (SSGs) and introduces Dynamic Prompt Module (DPM) to guide a Graph Parsing Neural Network (GPNN) in generating action-specific representations. Furthermore, we design a video-adapted GPNN that aggregates information using dynamic weights. Experiments in video action recognition demonstrate the effectiveness of our approach when compared with the state-of-the-art methods. Our code can be found in https://github.com/iamsnaping/ProDA.git

Method: Uses large-scale curated datasets (85M pretraining and 22M instruction datasets), an efficient training framework with offline parallel data packing, and achieves training within a $16,000 budget.

Result: LLaVA-OneVision-1.5-8B outperforms Qwen2.5-VL-7B on 18 of 27 benchmarks, and LLaVA-OneVision-1.5-4B surpasses Qwen2.5-VL-3B on all 27 benchmarks.

Conclusion: The model achieves exceptionally competitive performance across a broad range of downstream tasks with significantly reduced costs, and future releases including LLaVA-OneVision-1.5-RL are anticipated.

Abstract: We present LLaVA-OneVision-1.5, a novel family of Large Multimodal Models (LMMs) that achieve state-of-the-art performance with significantly reduced computational and financial costs. Different from the existing works, LLaVA-OneVision-1.5 provides an open, efficient, and reproducible framework for building high-quality vision-language models entirely from scratch. The LLaVA-OneVision-1.5 release comprises three primary components: (1) Large-Scale Curated Datasets: We construct an 85M concept-balanced pretraining dataset LLaVA-OneVision-1.5-Mid-Traning and a meticulously curated 22M instruction dataset LLaVA-OneVision-1.5-Instruct. (2) Efficient Training Framework: We develop a complete end-to-end efficient training framework leveraging an offline parallel data packing strategy to facilitate the training of LLaVA-OneVision-1.5 within a $16,000 budget. (3) State-of-the-art Performance: Experimental results demonstrate that LLaVA-OneVision-1.5 yields exceptionally competitive performance across a broad range of downstream tasks. Specifically, LLaVA-OneVision-1.5-8B outperforms Qwen2.5-VL-7B on 18 of 27 benchmarks, and LLaVA-OneVision-1.5-4B surpasses Qwen2.5-VL-3B on all 27 benchmarks. We anticipate releasing LLaVA-OneVision-1.5-RL shortly and encourage the community to await further updates.

Method: Uses middle-layer attention maps from vision encoder to estimate token importance, then applies hierarchical pruning: global stage retains important tokens per region, local stage retains most important token per window in 2D spatial layout.

Result: Reduces time-to-first-token by up to 50.0-55.1% and improves token generation throughput by up to 60.9-47.3% for LLaVA models, with maintained or improved accuracy on benchmarks.

Conclusion: HIVTP achieves superior accuracy and higher inference efficiency compared to prior works, demonstrating effective training-free visual token pruning for VLMs.

Abstract: Vision-Language Models (VLMs) have shown strong capabilities on diverse multimodal tasks. However, the large number of visual tokens output by the vision encoder severely hinders inference efficiency, and prior studies have shown that many of these tokens are not important and can therefore be safely pruned. In this work, we propose HIVTP, a training-free method to improve VLMs efficiency via hierarchical visual token pruning using a novel middle-layer-based importance score. Specifically, we utilize attention maps extracted from the middle layers of the vision encoder, which better reflect fine-grained and object-level attention, to estimate visual token importance. Based on this, we propose a hierarchical visual token pruning method to retain both globally and locally important visual tokens. Specifically, we reshape the 1-D visual token sequence output by the vision encoder into a 2-D spatial layout. In the global retaining stage, we divide the image into regions and retain tokens with higher importance scores in each region; in the local retaining stage, we then divide the image into small windows and retain the most important token in each local window. Experimental results show that our proposed method, HIVTP, can reduce the time-to-first-token (TTFT) of LLaVA-v1.5-7B and LLaVA-Next-7B by up to 50.0% and 55.1%, respectively, and improve the token generation throughput by up to 60.9% and 47.3%, without sacrificing accuracy, and even achieving improvements on certain benchmarks. Compared with prior works, HIVTP achieves better accuracy while offering higher inference efficiency.

Method: Leverage SemanticSAM with progressive granularity refinement to generate more accurate object-level masks, and employ context-aware CLIP encoding that integrates multiple contextual views with empirical weighting.

Result: Experimental results on multiple 3D scene understanding tasks show significant improvements over existing methods in 3D semantic segmentation and object retrieval from language queries across benchmark datasets.

Conclusion: The approach effectively addresses fragmentation and semantic inaccuracy issues in 3D semantic mapping, demonstrating superior performance through improved mask generation and contextual encoding.

Abstract: 3D scene understanding is fundamental for embodied AI and robotics, supporting reliable perception for interaction and navigation. Recent approaches achieve zero-shot, open-vocabulary 3D semantic mapping by assigning embedding vectors to 2D class-agnostic masks generated via vision-language models (VLMs) and projecting these into 3D. However, these methods often produce fragmented masks and inaccurate semantic assignments due to the direct use of raw masks, limiting their effectiveness in complex environments. To address this, we leverage SemanticSAM with progressive granularity refinement to generate more accurate and numerous object-level masks, mitigating the over-segmentation commonly observed in mask generation models such as vanilla SAM, and improving downstream 3D semantic segmentation. To further enhance semantic context, we employ a context-aware CLIP encoding strategy that integrates multiple contextual views of each mask using empirically determined weighting, providing much richer visual context. We evaluate our approach on multiple 3D scene understanding tasks, including 3D semantic segmentation and object retrieval from language queries, across several benchmark datasets. Experimental results demonstrate significant improvements over existing methods, highlighting the effectiveness of our approach.

Method: The survey examines three fundamental pillars of post-training methodologies: supervised fine-tuning (SFT) with chain-of-thought, reinforcement learning (RL) from verifiable objectives, and test-time scaling (TTS) through enhanced inference computation. It presents a structured taxonomy addressing video-specific challenges like temporal localization and multimodal evidence integration.

Result: The survey synthesizes key design principles, insights, and evaluation protocols while identifying critical open challenges in reward design, scalability, and cost-performance optimization. It also curates essential benchmarks, datasets, and metrics for rigorous assessment of post-training effectiveness.

Conclusion: This survey aims to provide researchers and practitioners with a unified framework for advancing Video-LMM capabilities, establishing the foundation for systematic post-training methodology development in video understanding.

Abstract: Video understanding represents the most challenging frontier in computer vision, requiring models to reason about complex spatiotemporal relationships, long-term dependencies, and multimodal evidence. The recent emergence of Video-Large Multimodal Models (Video-LMMs), which integrate visual encoders with powerful decoder-based language models, has demonstrated remarkable capabilities in video understanding tasks. However, the critical phase that transforms these models from basic perception systems into sophisticated reasoning engines, post-training, remains fragmented across the literature. This survey provides the first comprehensive examination of post-training methodologies for Video-LMMs, encompassing three fundamental pillars: supervised fine-tuning (SFT) with chain-of-thought, reinforcement learning (RL) from verifiable objectives, and test-time scaling (TTS) through enhanced inference computation. We present a structured taxonomy that clarifies the roles, interconnections, and video-specific adaptations of these techniques, addressing unique challenges such as temporal localization, spatiotemporal grounding, long video efficiency, and multimodal evidence integration. Through systematic analysis of representative methods, we synthesize key design principles, insights, and evaluation protocols while identifying critical open challenges in reward design, scalability, and cost-performance optimization. We further curate essential benchmarks, datasets, and metrics to facilitate rigorous assessment of post-training effectiveness. This survey aims to provide researchers and practitioners with a unified framework for advancing Video-LMM capabilities. Additional resources and updates are maintained at: https://github.com/yunlong10/Awesome-Video-LMM-Post-Training

Method: Pipeline segments human point clouds from background, tracks individuals, performs body part segmentation, and uses point-based techniques with sparse convolutional networks on voxel-mapped sequences with auxiliary features.

Result: Achieves 89.3% accuracy on NTU RGB-D 120 dataset with ensemble setup, outperforming previous point cloud action recognition methods and competitive with skeletal approaches.

Conclusion: The proposed 3D action recognition framework effectively utilizes dense point cloud data and demonstrates superior performance through ensemble combination of sensor-based and estimated depth inputs.

Abstract: Recent research into human action recognition (HAR) has focused predominantly on skeletal action recognition and video-based methods. With the increasing availability of consumer-grade depth sensors and Lidar instruments, there is a growing opportunity to leverage dense 3D data for action recognition, to develop a third way. This paper presents a novel approach for recognizing actions from 3D videos by introducing a pipeline that segments human point clouds from the background of a scene, tracks individuals over time, and performs body part segmentation. The method supports point clouds from both depth sensors and monocular depth estimation. At the core of the proposed HAR framework is a novel backbone for 3D action recognition, which combines point-based techniques with sparse convolutional networks applied to voxel-mapped point cloud sequences. Experiments incorporate auxiliary point features including surface normals, color, infrared intensity, and body part parsing labels, to enhance recognition accuracy. Evaluation on the NTU RGB- D 120 dataset demonstrates that the method is competitive with existing skeletal action recognition algorithms. Moreover, combining both sensor-based and estimated depth inputs in an ensemble setup, this approach achieves 89.3% accuracy when different human subjects are considered for training and testing, outperforming previous point cloud action recognition methods.

Method: Proposes object-level masked image modeling where masks are applied to visual objects rather than random patches, treating objects as the visual equivalent of words in language modeling.

Result: Object-level representation helps learn real-world distributions while avoiding pixel-averaging shortcuts. Models show strong reasoning and contextual understanding on VQA, GQA, and ScienceQA tasks when evaluated with multimodal LLMs.

Conclusion: Object-level encoding is effective for developing stronger vision encoders and tokenizers, providing a plausible direction to bridge the gap between vision and language model capabilities.

Abstract: Recent advances in language modeling have witnessed the rise of highly desirable emergent capabilities, such as reasoning and in-context learning. However, vision models have yet to exhibit comparable progress in these areas. In this paper, we argue that this gap could stem from the lack of semantic and contextual guidance in current vision transformer (ViT) training schemes, and such a gap can be narrowed through the design of a semantic-grounded objective. Specifically, we notice that individual words in natural language are inherently semantic, and modeling directly on word tokens naturally learns a realistic distribution. In contrast, ViTs rely on spatial patchification, which inevitably lacks semantic information. To bridge this gap, we propose to directly model “object” as the visual equivalence of “word,” pushing the model to learn the global context and semantics among visual elements. We investigate our hypotheses via masked image modeling (MIM), a framework where our approach can be readily tested by applying masks to visual objects rather than random patches. Considerable evidence from qualitative and quantitative evaluations reveals a key finding: object-level representation alone helps to learn a real-world distribution, whereas pixel-averaging shortcuts are often learned without it. Moreover, further evaluations with multimodal LLMs (MLLM) on visual question answering (VQA, GQA, ScienceQA) tasks demonstrate the strong reasoning and contextual understanding gained with this simple objective. We hope our study highlights the effectiveness of object-level encoding and provides a plausible direction for developing stronger vision encoders and tokenizers. Code and model will be publicly released. Keywords: Semantic Visual Tokenizer, Vision Reasoning, In-context Learning, Multimodal Reasoning

Method: Developed a simulation workflow to generate raw-like products from high-resolution L1 imagery, trained YOLOv11n and YOLOX-S models on both raw and L1 datasets, and compared performance using detection metrics and explainability tools.

Result: Both models performed similarly at low to medium confidence thresholds, but the raw data-trained model struggled with object boundary identification at high confidence levels.

Conclusion: Adapting AI architectures with improved contouring methods can enhance object detection on raw images, improving onboard AI for remote sensing applications.

Abstract: With increasing processing power, deploying AI models for remote sensing directly onboard satellites is becoming feasible. However, new constraints arise, mainly when using raw, unprocessed sensor data instead of preprocessed ground-based products. While current solutions primarily rely on preprocessed sensor images, few approaches directly leverage raw data. This study investigates the effects of utilising raw data on deep learning models for object detection and classification tasks. We introduce a simulation workflow to generate raw-like products from high-resolution L1 imagery, enabling systemic evaluation. Two object detection models (YOLOv11n and YOLOX-S) are trained on both raw and L1 datasets, and their performance is compared using standard detection metrics and explainability tools. Results indicate that while both models perform similarly at low to medium confidence thresholds, the model trained on raw data struggles with object boundary identification at high confidence levels. It suggests that adapting AI architectures with improved contouring methods can enhance object detection on raw images, improving onboard AI for remote sensing.

Method: Uses constraint-based semantics with semantic guards operating at decode time, incorporating proof-based verification and multi-agent operational calculus with Lean artifacts for implementation certification.

Result: The approach reduces hallucinations without sacrificing throughput, providing a practical solution for knowledge-aligned language generation.

Conclusion: TAD offers a pragmatic bridge between empirical language models and formal verification, enabling more reliable and knowledge-grounded text generation through verified guardrails.

Abstract: This paper introduces Truth-Aware Decoding (TAD), a verification-oriented decoding scheme that aligns neural language generation with knowledge bases. Situated in the tradition of probabilistic program semantics for sequence models, TAD augments modern instruction-tuned systems with a lattice of semantic guards that operate at decode time. Our contributions are fourfold: (i) a constraint-based semantics that renders oracle filtering as a program-logic judgment, (ii) a proof that greedy selection enjoys local likelihood dominance under sound and complete guards (Theorem 2.7), (iii) an entropy-style invariant that quantifies factual risk via knowledge-aware safe mass, and (iv) a multi-agent operational calculus with verified Lean artefacts to certify implementation behaviour. Numerical and algorithmic case studies confirm that the resulting guardrails reduce hallucinations without sacrificing throughput, yielding a pragmatic bridge between large-scale empirical models and formal verification.

Method: Uses LLMs as semantic bridges to translate unstructured telemetry into contextual state representations, then employs MAPPO multi-agent reinforcement learning with a reward function balancing security and operational stability.

Result: Achieved 97.2% detection rate, reduced false positives by over 80%, improved response times by 4x, and maintained superior physical process stability compared to traditional methods.

Conclusion: L2M-AID presents a robust new paradigm for securing critical infrastructure by effectively combining LLM semantic reasoning with multi-agent reinforcement learning for adaptive, resilient industrial defense.

Abstract: The increasing integration of Industrial IoT (IIoT) exposes critical cyber-physical systems to sophisticated, multi-stage attacks that elude traditional defenses lacking contextual awareness. This paper introduces L2M-AID, a novel framework for Autonomous Industrial Defense using LLM-empowered, Multi-agent reinforcement learning. L2M-AID orchestrates a team of collaborative agents, each driven by a Large Language Model (LLM), to achieve adaptive and resilient security. The core innovation lies in the deep fusion of two AI paradigms: we leverage an LLM as a semantic bridge to translate vast, unstructured telemetry into a rich, contextual state representation, enabling agents to reason about adversary intent rather than merely matching patterns. This semantically-aware state empowers a Multi-Agent Reinforcement Learning (MARL) algorithm, MAPPO, to learn complex cooperative strategies. The MARL reward function is uniquely engineered to balance security objectives (threat neutralization) with operational imperatives, explicitly penalizing actions that disrupt physical process stability. To validate our approach, we conduct extensive experiments on the benchmark SWaT dataset and a novel synthetic dataset generated based on the MITRE ATT&CK for ICS framework. Results demonstrate that L2M-AID significantly outperforms traditional IDS, deep learning anomaly detectors, and single-agent RL baselines across key metrics, achieving a 97.2% detection rate while reducing false positives by over 80% and improving response times by a factor of four. Crucially, it demonstrates superior performance in maintaining physical process stability, presenting a robust new paradigm for securing critical national infrastructure.

Method: The authors formalize debate dynamics as an identity-weighted Bayesian update process, propose response anonymization by removing identity markers from prompts to force equal weights on agent identity, and define the Identity Bias Coefficient (IBC) to measure bias.

Result: Empirical studies across multiple models, datasets and debate rounds confirm that identity bias is widespread, with sycophancy being far more common than self-bias. Response anonymization effectively reduces identity bias in MAD systems.

Conclusion: The findings highlight the need to ‘mask’ identity in multi-agent debate systems to ensure reasoning is based on content quality rather than source identity, improving the reliability of debate outcomes.

Abstract: Multi-agent debate (MAD) aims to improve large language model (LLM) reasoning by letting multiple agents exchange answers and then aggregate their opinions. Yet recent studies reveal that agents are not neutral: they are prone to identity-driven sycophancy and self-bias, uncritically adopting a peer’s view or stubbornly adhering to their own prior output, undermining the reliability of debate. In this work, we present the first principled framework that joins sycophancy and self-bias to mitigate and quantify identity bias in MAD. First, we formalize the debate dynamics as an identity-weighted Bayesian update process. Second, we propose response anonymization: by removing identity markers from prompts, agents cannot distinguish “self” from “peer”, which forces equal weights on agent identity, thereby reducing bias. Third, we define the Identity Bias Coefficient (IBC), a principled metric that measures how often an agent follows a peer versus itself. Empirical studies across multiple models, datasets and debate rounds confirm that identity bias is widespread, with sycophancy far more common than self-bias. Our findings highlight the need to “mask” identity to ensure that MAD systems reason based on content rather than source identity. Code is released in https://github.com/deeplearning-wisc/MAD-identity-bias.

Method: Proposed a hybrid model that activates reasoning mechanisms in base models at appropriate times, and introduced an unsupervised bottom-up approach to discover human-interpretable reasoning behaviors without manual assumptions.

Result: The hybrid model recovered up to 91% of the performance gap to thinking models without weight updates while steering only 12% of tokens, showing base models already possess the necessary reasoning capabilities.

Conclusion: Pre-training provides most reasoning mechanisms, while post-training teaches efficient deployment timing, enabling optimal use of inference-time compute.

Abstract: Why do thinking language models like DeepSeek R1 outperform their base counterparts? Despite consistent performance gains, it remains unclear to what extent thinking models learn entirely new reasoning capabilities or repurpose pre-existing base model ones. In this work, we propose a hybrid model where we activate reasoning mechanisms in base models at the right time to elicit thinking-model-level reasoning chains, implying that thinking models exploit already existing capabilities. To ground our analysis, we introduce an unsupervised, bottom-up approach for uncovering human-interpretable reasoning behaviors in thinking models. This approach provides an unbiased method to discover reasoning behaviors without imposing manual or LLM-derived assumptions. Across three base and four thinking models, using GSM8K and MATH500, our hybrid model recovers up to 91% of the performance gap to thinking models without any weight updates while steering only 12% of tokens. Concretely, our empirical setup provides a simple, causal way to test the effectiveness of existing reasoning mechanisms in base models by invoking them directly and measuring the resulting task performance. More broadly, these results reframe our understanding of how thinking models are trained: pre-training is when models acquire most of their reasoning mechanisms, and post-training teaches efficient deployment of these mechanisms at the right time, enabling efficient use of their inference-time compute.

Method: A reproducible simulation framework with layered memory, dynamic planning, multimodal perception, and SocialMetrics suite to evaluate agents along safety improvement, unsafe activity detection, and social dynamics across multiple models (Claude, GPT-4o mini, Qwen-VL).

Result: Agents achieved 55% success rate in correcting unsafe plans, with unsafe-to-safe conversion rates of 75%, 55%, and 58% for Claude, GPT-4o mini, and Qwen-VL respectively. Performance ranged from 20% in multi-risk scenarios to 98% in localized contexts. 45% of unsafe actions were accepted with misleading visuals.

Conclusion: Current architectures have critical limitations in multimodal safety reasoning, with agents showing strong tendency to overtrust images and failing to align local revisions with global safety, providing a reproducible platform for studying multimodal safety and social dynamics.

Abstract: Can generative agents be trusted in multimodal environments? Despite advances in large language and vision-language models that enable agents to act autonomously and pursue goals in rich settings, their ability to reason about safety, coherence, and trust across modalities remains limited. We introduce a reproducible simulation framework for evaluating agents along three dimensions: (1) safety improvement over time, including iterative plan revisions in text-visual scenarios; (2) detection of unsafe activities across multiple categories of social situations; and (3) social dynamics, measured as interaction counts and acceptance ratios of social exchanges. Agents are equipped with layered memory, dynamic planning, multimodal perception, and are instrumented with SocialMetrics, a suite of behavioral and structural metrics that quantifies plan revisions, unsafe-to-safe conversions, and information diffusion across networks. Experiments show that while agents can detect direct multimodal contradictions, they often fail to align local revisions with global safety, reaching only a 55 percent success rate in correcting unsafe plans. Across eight simulation runs with three models - Claude, GPT-4o mini, and Qwen-VL - five agents achieved average unsafe-to-safe conversion rates of 75, 55, and 58 percent, respectively. Overall performance ranged from 20 percent in multi-risk scenarios with GPT-4o mini to 98 percent in localized contexts such as fire/heat with Claude. Notably, 45 percent of unsafe actions were accepted when paired with misleading visuals, showing a strong tendency to overtrust images. These findings expose critical limitations in current architectures and provide a reproducible platform for studying multimodal safety, coherence, and social dynamics.

Method: Use generative AI for frequent experience sampling from patients and diagnostic reconciliation across care pathways, implementing mental health digital twins as continuously updated computational models.

Result: AI enables more personalized and longitudinal care pathways by providing continuous, rich, patient-centered data sampling.

Conclusion: Continuous AI-driven assessment and mental health digital twins can transform mental healthcare by dynamically adapting to individual needs, improving accessibility and treatment efficacy.

Abstract: Static solutions don’t serve a dynamic mind. Thus, we advocate a shift from static mental health diagnostic assessments to continuous, artificial intelligence (AI)-driven assessment. Focusing on Attention-Deficit/Hyperactivity Disorder (ADHD) as a case study, we explore how generative AI has the potential to address current capacity constraints in neuropsychology, potentially enabling more personalized and longitudinal care pathways. In particular, AI can efficiently conduct frequent, low-level experience sampling from patients and facilitate diagnostic reconciliation across care pathways. We envision a future where mental health care benefits from continuous, rich, and patient-centered data sampling to dynamically adapt to individual patient needs and evolving conditions, thereby improving both accessibility and efficacy of treatment. We further propose the use of mental health digital twins (MHDTs) - continuously updated computational models that capture individual symptom dynamics and trajectories - as a transformative framework for personalized mental health care. We ground this framework in empirical evidence and map out the research agenda required to refine and operationalize it.

Method: Hierarchical architecture with Manager Agent orchestrating domain-specialized Expert Agents, decomposing tasks, and adaptively replanning based on structured feedback from failed attempts including detailed failure reasons and newly discovered constraints.

Result: Outperforms state-of-the-art baselines on FinanceBench benchmark without human feedback, achieving robust performance across reasoning-heavy tasks.

Conclusion: ProSEA demonstrates potential as a foundation for more transparent, adaptive, and human-aligned AI agents through its dynamic plan refinement and exploratory capabilities.

Abstract: Large language models (LLMs) have empowered AI agents to tackle increasingly complex tasks. However, most existing agents remain limited to static planning and brittle interactions, falling short of true collaboration or adaptive reasoning. We introduce ProSEA, a modular, general-purpose multi-agent framework designed for iterative problem solving through exploration and plan evolution. ProSEA features a hierarchical architecture in which a Manager Agent orchestrates domain-specialized Expert Agents, decomposes tasks, and adaptively replans based on structured feedback from failed attempts. Unlike prior systems, ProSEA agents report not only success or failure but also detailed reasons for failure and newly discovered constraints, enabling dynamic plan refinement informed by exploratory traces. The framework operates autonomously but supports seamless integration with human collaborators when needed. Experiments on the challenging FinanceBench benchmark demonstrate that ProSEA, even without human feedback, outperforms state-of-the-art baselines and achieves robust performance across reasoning-heavy tasks. These results underscore ProSEA’s potential as a foundation for more transparent, adaptive, and human-aligned AI agents.

Method: Introduces TESTAM+ with a novel SpatioSemantic Expert that integrates physical road topology with data-driven feature similarity through hybrid graph construction, using strategic expert selection instead of naive ensemble aggregation.

Result: Achieves 1.3% MAE reduction on METR LA (3.10 vs. 3.14) and 4.1% improvement on PEMS BAY (1.65 vs. 1.72). Individual experts show remarkable effectiveness, with optimal configuration achieving 11.5% MAE reduction vs MegaCRN on METR LA (2.99 vs. 3.38) and 53.1% inference latency reduction.

Conclusion: Fewer, strategically designed experts outperform complex multi-expert ensembles, establishing new state-of-the-art performance with superior computational efficiency for real-time deployment.

Abstract: Traffic forecasting is fundamental to intelligent transportation systems, enabling congestion mitigation and emission reduction in increasingly complex urban environments. While recent graph neural network approaches have advanced spatial temporal modeling, existing mixture of experts frameworks like Time Enhanced Spatio Temporal Attention Model (TESTAM) lack explicit incorporation of physical road network topology, limiting their spatial capabilities. We present TESTAM+, an enhanced spatio temporal forecasting framework that introduces a novel SpatioSemantic Expert integrating physical road topology with data driven feature similarity through hybrid graph construction. TESTAM+ achieves significant improvements over TESTAM: 1.3% MAE reduction on METR LA (3.10 vs. 3.14) and 4.1% improvement on PEMS BAY (1.65 vs. 1.72). Through comprehensive ablation studies, we discover that strategic expert selection fundamentally outperforms naive ensemble aggregation. Individual experts demonstrate remarkable effectiveness: the Adaptive Expert achieves 1.63 MAE on PEMS BAY, outperforming the original three expert TESTAM (1.72 MAE), while the SpatioSemantic Expert matches this performance with identical 1.63 MAE. The optimal Identity + Adaptive configuration achieves an 11.5% MAE reduction compared to state of the art MegaCRN on METR LA (2.99 vs. 3.38), while reducing inference latency by 53.1% compared to the full four expert TESTAM+. Our findings reveal that fewer, strategically designed experts outperform complex multi expert ensembles, establishing new state of the art performance with superior computational efficiency for real time deployment.

Method: Uses LLMs for evidence gathering and step-by-step reasoning while delegating statistical extraction to time series tools. Interacts with raw numeric sequences through atomic operators, maintains evidence logs, and uses self-critic and quality gate for iterative refinement.

Result: Achieves comparable performance to state-of-the-art LLMs on understanding benchmarks and significant improvements on reasoning tasks, especially in zero-shot settings where existing models fail.

Conclusion: The hybrid approach effectively mitigates LLM limitations in time series reasoning while maintaining interpretability and avoiding multi-modal alignment training.

Abstract: Large language models (LLMs) have shown strong abilities in reasoning and problem solving, but recent studies reveal that they still struggle with time series reasoning tasks, where outputs are often affected by hallucination or knowledge leakage. In this work we propose TS-Agent, a time series reasoning agent that leverages LLMs strictly for what they excel at, i.e., gathering evidence and synthesizing it into conclusions through step-by-step reasoning, while delegating the extraction of statistical and structural information to time series analytical tools. Instead of mapping time series into text tokens, images, or embeddings, our agent interacts with raw numeric sequences through atomic operators, records outputs in an explicit evidence log, and iteratively refines its reasoning under the guidance of a self-critic and a final quality gate. This design avoids multi-modal alignment training, preserves the native form of time series, ensures interpretability and verifiability, and mitigates knowledge leakage or hallucination. Empirically, we evaluate the agent on established benchmarks. Our experiments show that TS-Agent achieves performance comparable to state-of-the-art LLMs on understanding benchmarks, and delivers significant improvements on reasoning tasks, where existing models often rely on memorization and fail in zero-shot settings.

Method: Developed ExpertAgent framework with dynamic planning of learning content/strategy based on continuously updated student model, using validated curriculum repository to ground instructional content.

Result: Provides proactive and personalized learning experience, overcomes limitations of traditional static learning content, enables optimized teaching strategies in real time.

Conclusion: ExpertAgent effectively reduces hallucination risks in large language models and improves reliability and trustworthiness in educational AI applications.

Abstract: The application of advanced generative artificial intelligence in education is often constrained by the lack of real-time adaptability, personalization, and reliability of the content. To address these challenges, we propose ExpertAgent - an intelligent agent framework designed for personalized education that provides reliable knowledge and enables highly adaptive learning experiences. Therefore, we developed ExpertAgent, an innovative learning agent that provides users with a proactive and personalized learning experience. ExpertAgent dynamic planning of the learning content and strategy based on a continuously updated student model. Therefore, overcoming the limitations of traditional static learning content to provide optimized teaching strategies and learning experience in real time. All instructional content is grounded in a validated curriculum repository, effectively reducing hallucination risks in large language models and improving reliability and trustworthiness.

Method: Empirical analysis of several LLMs (including ChatGPT model o3) in zero-shot settings, testing their understanding of BPMN process models from images and conversational query answering.

Result: Vanilla, untrained LLMs are effective at understanding BPMN models and answering queries at syntactic, logical, and semantic levels, though performance varies across different models.

Conclusion: LLMs can serve as valuable assistants for business process designers and users, exhibiting anthropomorphic properties in their reasoning about process analysis and optimization.

Abstract: In this paper, we report our experience with several LLMs for their ability to understand a process model in an interactive, conversational style, find syntactical and logical errors in it, and reason with it in depth through a natural language (NL) interface. Our findings show that a vanilla, untrained LLM like ChatGPT (model o3) in a zero-shot setting is effective in understanding BPMN process models from images and answering queries about them intelligently at syntactic, logic, and semantic levels of depth. Further, different LLMs vary in performance in terms of their accuracy and effectiveness. Nevertheless, our empirical analysis shows that LLMs can play a valuable role as assistants for business process designers and users. We also study the LLM’s “thought process” and ability to perform deeper reasoning in the context of process analysis and optimization. We find that the LLMs seem to exhibit anthropomorphic properties.

Method: Formalized as constrained optimization problem, modeled with Minizinc constraint modeling language, and compared three resolution paradigms: Mixed Integer Programming, Satisfiability, and Constraint Programming.

Result: Comparative experimental study analyzed performance, expressiveness, and scalability of each approach, identifying methodological limits.

Conclusion: Perspectives drawn for integrating Minizinc model into complete trustworthy AI ethical risk management process for MIS.

Abstract: Medical Intelligent Systems (MIS) are increasingly integrated into healthcare workflows, offering significant benefits but also raising critical safety and ethical concerns. According to the European Union AI Act, most MIS will be classified as high-risk systems, requiring a formal risk management process to ensure compliance with the ethical requirements of trustworthy AI. In this context, we focus on risk reduction optimization problems, which aim to reduce risks with ethical considerations by finding the best balanced assignment of risk assessment values according to their coverage of trustworthy AI ethical requirements. We formalize this problem as a constrained optimization task and investigate three resolution paradigms: Mixed Integer Programming (MIP), Satisfiability (SAT), and Constraint Programming(CP).Our contributions include the mathematical formulation of this optimization problem, its modeling with the Minizinc constraint modeling language, and a comparative experimental study that analyzes the performance, expressiveness, and scalability of each approach to solving. From the identified limits of the methodology, we draw some perspectives of this work regarding the integration of the Minizinc model into a complete trustworthy AI ethical risk management process for MIS.

Method: Implemented an abstract protocol based on communicating finite-state machines for human-LLM interaction, tested with controlled experiments using a human proxy database and uncontrolled experiments with human subjects in radiology and drug design domains.

Result: Empirical evidence supports the protocol’s capability to capture one- and two-way intelligibility in human-LLM interactions, demonstrating utility in human-machine system design.

Conclusion: The structured protocol enables effective human-LLM collaboration through two-way intelligibility, providing a foundation for designing more capable human-machine systems for complex problem-solving.

Abstract: Our interest is in the design of software systems involving a human-expert interacting – using natural language – with a large language model (LLM) on data analysis tasks. For complex problems, it is possible that LLMs can harness human expertise and creativity to find solutions that were otherwise elusive. On one level, this interaction takes place through multiple turns of prompts from the human and responses from the LLM. Here we investigate a more structured approach based on an abstract protocol described in [3] for interaction between agents. The protocol is motivated by a notion of “two-way intelligibility” and is modelled by a pair of communicating finite-state machines. We provide an implementation of the protocol, and provide empirical evidence of using the implementation to mediate interactions between an LLM and a human-agent in two areas of scientific interest (radiology and drug design). We conduct controlled experiments with a human proxy (a database), and uncontrolled experiments with human subjects. The results provide evidence in support of the protocol’s capability of capturing one- and two-way intelligibility in human-LLM interaction; and for the utility of two-way intelligibility in the design of human-machine systems. Our code is available at https://github.com/karannb/interact.

Method: Multi-agent framework with two-stage pipeline: SEARCH stage identifies popular paths from historical data, GENERATE stage synthesizes novel paths when no existing paths are found.

Result: Experiments show superior accuracy in SEARCH stage and competitive performance in GENERATE stage while being cost-effective on real and synthetic datasets.

Conclusion: CompassLLM effectively leverages LLM reasoning capabilities for geo-spatial problems, providing an alternative to traditional training-based approaches for popular path queries.

Abstract: The popular path query - identifying the most frequented routes between locations from historical trajectory data - has important applications in urban planning, navigation optimization, and travel recommendations. While traditional algorithms and machine learning approaches have achieved success in this domain, they typically require model training, parameter tuning, and retraining when accommodating data updates. As Large Language Models (LLMs) demonstrate increasing capabilities in spatial and graph-based reasoning, there is growing interest in exploring how these models can be applied to geo-spatial problems. We introduce CompassLLM, a novel multi-agent framework that intelligently leverages the reasoning capabilities of LLMs into the geo-spatial domain to solve the popular path query. CompassLLM employs its agents in a two-stage pipeline: the SEARCH stage that identifies popular paths, and a GENERATE stage that synthesizes novel paths in the absence of an existing one in the historical trajectory data. Experiments on real and synthetic datasets show that CompassLLM demonstrates superior accuracy in SEARCH and competitive performance in GENERATE while being cost-effective.

Method: Hybrid framework combining deterministic regular expressions with context-aware LLMs, using a three-phase refinement pipeline for disambiguation and filtering across 13 low-resource locales.

Result: Outperforms fine-tuned NER models by 82% and zero-shot LLMs by 17% in weighted F1-score, supporting over 300 entity types without retraining.

Conclusion: RECAP offers a scalable and adaptable solution for efficient PII detection in compliance-focused applications.

Abstract: The detection of Personally Identifiable Information (PII) is critical for privacy compliance but remains challenging in low-resource languages due to linguistic diversity and limited annotated data. We present RECAP, a hybrid framework that combines deterministic regular expressions with context-aware large language models (LLMs) for scalable PII detection across 13 low-resource locales. RECAP’s modular design supports over 300 entity types without retraining, using a three-phase refinement pipeline for disambiguation and filtering. Benchmarked with nervaluate, our system outperforms fine-tuned NER models by 82% and zero-shot LLMs by 17% in weighted F1-score. This work offers a scalable and adaptable solution for efficient PII detection in compliance-focused applications.

Method: Develops a boundary-centric ontology of agenthood with cybernetics, uses Petri nets for modeling collaboration transitions, implements three-level orchestration (meta, agent, execution), and grounds collaborative learning in Conversation Theory and SECI with teach-back gates.

Result: Creates HE2-Net framework that separates provisional from validated assets, promotes knowledge only after tests and peer checks, budgets concurrent probing while maintaining fast and safe reuse, and demonstrates interoperability with emerging agent protocols.

Conclusion: The framework keeps humans central to setting aims and steering theory-practice dynamics while scaling agents as reliable collaborators within audited governance, with potential for bio-cybernetic extensions.

Abstract: We propose a technology-agnostic, collaboration-ready stance for Human-AI Agents Collaboration Systems (HAACS) that closes long-standing gaps in prior stages (automation; flexible autonomy; agentic multi-agent collectives). Reading empirical patterns through a seven-dimension collaboration spine and human-agent contrasts, we identify missing pieces: principled budgeting of initiative, instantaneous and auditable reconfiguration, a system-wide knowledge backbone with an epistemic promotion gate, capacity-aware human interfaces; and, as a prerequisite to all of the above, unified definitions of agent and formal collaborative dynamics. We respond with (i) a boundary-centric ontology of agenthood synthesized with cybernetics; (ii) a Petri net family (colored and interpreted) that models ownership, cross-boundary interaction, concurrency, guards, and rates with collaboration transitions; and (iii) a three-level orchestration (meta, agent, execution) that governs behavior families via guard flips. On the knowledge side, we ground collaborative learning in Conversation Theory and SECI with teach-back gates and an evolving backbone; on the problem-solving side, we coordinate routine MEA-style control with practice-guided open-ended discovery. The result is the Hierarchical Exploration-Exploitation Net (HE2-Net): a policy-controlled stance that splits provisional from validated assets, promotes only after tests and peer checks, and budgets concurrent probing while keeping reuse fast and safe. We show interoperability with emerging agent protocols without ad hoc glue and sketch bio-cybernetic extensions (autopoiesis, autogenesis, evolving boundaries, synergetics, etc). Altogether, the framework keeps humans central to setting aims, justifying knowledge, and steering theory-practice dynamics, while scaling agents as reliable collaborators within audited governance.

Method: Introduces PeerBench - a community-governed, proctored evaluation blueprint featuring sealed execution, item banking with rolling renewal, and delayed transparency to ensure robust benchmarking by construction.

Result: The paper presents a conceptual framework and blueprint for trustworthy AI evaluation, though specific empirical results are not provided as this is a position paper proposing a new paradigm.

Conclusion: A paradigm shift to unified, live, and quality-controlled benchmarking is essential for sustainable AI advancement, and PeerBench provides a blueprint for evaluations that can restore integrity and deliver genuinely trustworthy measures of AI progress.

Abstract: The meteoric rise of Artificial Intelligence (AI), with its rapidly expanding market capitalization, presents both transformative opportunities and critical challenges. Chief among these is the urgent need for a new, unified paradigm for trustworthy evaluation, as current benchmarks increasingly reveal critical vulnerabilities. Issues like data contamination and selective reporting by model developers fuel hype, while inadequate data quality control can lead to biased evaluations that, even if unintentionally, may favor specific approaches. As a flood of participants enters the AI space, this “Wild West” of assessment makes distinguishing genuine progress from exaggerated claims exceptionally difficult. Such ambiguity blurs scientific signals and erodes public confidence, much as unchecked claims would destabilize financial markets reliant on credible oversight from agencies like Moody’s. In high-stakes human examinations (e.g., SAT, GRE), substantial effort is devoted to ensuring fairness and credibility; why settle for less in evaluating AI, especially given its profound societal impact? This position paper argues that the current laissez-faire approach is unsustainable. We contend that true, sustainable AI advancement demands a paradigm shift: a unified, live, and quality-controlled benchmarking framework robust by construction, not by mere courtesy and goodwill. To this end, we dissect the systemic flaws undermining today’s AI evaluation, distill the essential requirements for a new generation of assessments, and introduce PeerBench, a community-governed, proctored evaluation blueprint that embodies this paradigm through sealed execution, item banking with rolling renewal, and delayed transparency. Our goal is to pave the way for evaluations that can restore integrity and deliver genuinely trustworthy measures of AI progress.

Method: Three-stage pipeline: (1) distill edge-level judgments from failure traces into compact policy, (2) supervise policy to determine when/what/whom/how to ask, (3) optimize online with E-GRPO reinforcement learning balancing accuracy, latency, and cost.

Result: Consistently improves accuracy and robustness across math, reasoning, and coding benchmarks with minimal overhead (latency and extra cost <5%), approaching strong evaluator performance.

Conclusion: Provides a scalable pathway for reliable LLM-based multi-agent systems through principled error taxonomy and practical link-local intervention approach.

Abstract: Multi-agent systems built on large language models (LLMs) promise enhanced problem-solving capabilities through collaborative division of labor. However, they frequently underperform single-agent baselines due to edge-level error cascades: minor inaccuracies at one message handoff propagate across the entire chain. We propose AgentAsk, a lightweight and plug-and-play clarification module that treats every inter-agent message as a potential failure point and inserts minimally necessary questions to arrest error propagation. AgentAsk follows a three-stage pipeline: (i) distilling edge-level judgments from curated failure traces into a compact policy, (ii) supervising the policy to determine when/what/whom/how to ask, and (iii) optimizing online with E-GRPO, a reinforcement learning objective that balances accuracy, latency, and cost. The module is architecture-agnostic and easy to integrate into existing orchestration. Across math, reasoning, and coding benchmarks, AgentAsk consistently improves accuracy and robustness over public multi-agent implementations while keeping overhead minimal, with latency and extra cost all less than 5%, approaching the performance of a strong evaluator. Beyond empirical improvements, we contribute a principled taxonomy of edge-level errors and a practical recipe for link-local intervention, offering a scalable pathway toward more reliable LLM-based multi-agent systems.

Method: Evaluated eight configurations of three state-of-the-art LLMs on three benchmarks using a Planner → Executor → Critic pipeline with structured handoffs and saved records.

Result: Structured handoffs improve accuracy, prevent failures; models show role-specific strengths/risks; heterogeneous pipelines are often most efficient for accuracy-cost-latency trade-offs.

Conclusion: Provides a practical, data-driven method for designing, tracing, and debugging reliable, predictable, and accountable multi-agent systems.

Abstract: Sequential multi-agent systems built with large language models (LLMs) can automate complex software tasks, but they are hard to trust because errors quietly pass from one stage to the next. We study a traceable and accountable pipeline, meaning a system with clear roles, structured handoffs, and saved records that let us trace who did what at each step and assign blame when things go wrong. Our setting is a Planner -> Executor -> Critic pipeline. We evaluate eight configurations of three state-of-the-art LLMs on three benchmarks and analyze where errors start, how they spread, and how they can be fixed. Our results show: (1) adding a structured, accountable handoff between agents markedly improves accuracy and prevents the failures common in simple pipelines; (2) models have clear role-specific strengths and risks (e.g., steady planning vs. high-variance critiquing), which we quantify with repair and harm rates; and (3) accuracy-cost-latency trade-offs are task-dependent, with heterogeneous pipelines often the most efficient. Overall, we provide a practical, data-driven method for designing, tracing, and debugging reliable, predictable, and accountable multi-agent systems.

Method: The paper presents a real-world case study where generative AI was used to improve training of veterans to support each other’s mental health.

Result: Generative AI successfully enhanced the training process for mental health service provision in the veteran case study.

Conclusion: Investment should focus on using generative AI to support training people in mental health service provision rather than primarily on therapist chatbots, as this represents a lower-risk, high-impact application.

Abstract: Generative artificial intelligence (Generative AI) is transforming healthcare. With this evolution comes optimism regarding the impact it will have on mental health, as well as concern regarding the risks that come with generative AI operating in the mental health domain. Much of the investment in, and academic and public discourse about, AI-powered solutions for mental health has focused on therapist chatbots. Despite the common assumption that chatbots will be the most impactful application of GenAI to mental health, we make the case here for a lower-risk, high impact use case: leveraging generative AI to enhance and scale training in mental health service provision. We highlight key benefits of using generative AI to help train people to provide mental health services and present a real-world case study in which generative AI improved the training of veterans to support one another’s mental health. With numerous potential applications of generative AI in mental health, we illustrate why we should invest in using generative AI to support training people in mental health service provision.

Method: Introduces group matching score to better exploit group structure, and proposes Test-Time Matching (TTM) - an iterative, self-improving algorithm that bootstraps model performance without external supervision through overfitting to induced group matchings at test time.

Result: SigLIP-B16 surpasses all previous results and GPT-4.1, achieving the first result surpassing estimated human performance on Winoground. TTM enables SigLIP-B16 to surpass GPT-4.1 on MMVP-VLM, establishing new state-of-the-art with relative gains up to 85.7% on challenging datasets like WhatsUp.

Conclusion: TTM consistently improves model performance across 16 dataset variants, advancing the frontier of compositional reasoning by revealing and leveraging hidden capabilities that standard metrics systematically underestimate.

Abstract: Frontier AI models have achieved remarkable progress, yet recent studies suggest they struggle with compositional reasoning, often performing at or below random chance on established benchmarks. We revisit this problem and show that widely used evaluation metrics systematically underestimate model capability. To address this, we introduce a group matching score that better exploits group structure and reveals substantial hidden capability in both contrastive vision-language models (VLMs) and multimodal large language models (MLLMs). Moreover, simply overfitting to the induced group matchings at test time transfers this hidden capability into higher scores under standard evaluation metrics, closing much of the reported gap. This adjustment enables SigLIP-B16 to surpass all previous results and GPT-4.1 to yield the first result surpassing estimated human performance on Winoground. Building on this insight, we propose Test-Time Matching (TTM), an iterative, self-improving algorithm that further bootstraps model performance without any external supervision. TTM delivers additional, non-trivial improvements: for example, TTM enables SigLIP-B16 to surpass GPT-4.1 on MMVP-VLM, establishing a new state of the art. Importantly, TTM remains broadly effective even on benchmarks without metric-induced effects or group structures, achieving relative gains up to 85.7% on challenging datasets such as WhatsUp. Across 16 dataset variants spanning diverse setups, our experiments demonstrate that TTM consistently improves model performance and advances the frontier of compositional reasoning.

Method: Developed Safe Off-Policy Policy Gradient (Safe OPG) with high confidence off-policy evaluation, and a Deployment-Efficient Policy Learning framework that uses safety margin and gradually relaxes safety regularization.

Result: Safe OPG almost always satisfies safety requirements while existing methods violate them, though it tends to be too conservative. The proposed framework enables safe exploration while maintaining safety guarantees.

Conclusion: The framework successfully addresses the tradeoff between safety and exploration, enabling safe implementation of recommender systems with novel actions.

Abstract: In many real recommender systems, novel items are added frequently over time. The importance of sufficiently presenting novel actions has widely been acknowledged for improving long-term user engagement. A recent work builds on Off-Policy Learning (OPL), which trains a policy from only logged data, however, the existing methods can be unsafe in the presence of novel actions. Our goal is to develop a framework to enforce exploration of novel actions with a guarantee for safety. To this end, we first develop Safe Off-Policy Policy Gradient (Safe OPG), which is a model-free safe OPL method based on a high confidence off-policy evaluation. In our first experiment, we observe that Safe OPG almost always satisfies a safety requirement, even when existing methods violate it greatly. However, the result also reveals that Safe OPG tends to be too conservative, suggesting a difficult tradeoff between guaranteeing safety and exploring novel actions. To overcome this tradeoff, we also propose a novel framework called Deployment-Efficient Policy Learning for Safe User Exploration, which leverages safety margin and gradually relaxes safety regularization during multiple (not many) deployments. Our framework thus enables exploration of novel actions while guaranteeing safe implementation of recommender systems.

Method: Designed a multiple memory system (MMS) that processes short-term memory into multiple long-term memory fragments, constructs retrieval memory units and contextual memory units with one-to-one correspondence, and matches relevant retrieval units during query to obtain corresponding contextual units.

Result: Experiments on LoCoMo dataset showed effectiveness compared to three other methods. Ablation studies confirmed memory unit rationality. Analysis demonstrated robustness regarding memory segment selection and storage overhead.

Conclusion: MMS effectively utilizes historical data by constructing high-quality long-term memory content, enhancing agent performance with practical value.

Abstract: An agent powered by large language models have achieved impressive results, but effectively handling the vast amounts of historical data generated during interactions remains a challenge. The current approach is to design a memory module for the agent to process these data. However, existing methods, such as MemoryBank and A-MEM, have poor quality of stored memory content, which affects recall performance and response quality. In order to better construct high-quality long-term memory content, we have designed a multiple memory system (MMS) inspired by cognitive psychology theory. The system processes short-term memory to multiple long-term memory fragments, and constructs retrieval memory units and contextual memory units based on these fragments, with a one-to-one correspondence between the two. During the retrieval phase, MMS will match the most relevant retrieval memory units based on the user’s query. Then, the corresponding contextual memory units is obtained as the context for the response stage to enhance knowledge, thereby effectively utilizing historical data. Experiments on LoCoMo dataset compared our method with three others, proving its effectiveness. Ablation studies confirmed the rationality of our memory units. We also analyzed the robustness regarding the number of selected memory segments and the storage overhead, demonstrating its practical value.

Method: Online reward generation guided by STL causation monitoring, continuously computing quantitative distance toward satisfaction/violation at each control step, with smooth approximation for differentiability in deep-RL.

Result: Experimental evaluation in Gym environment shows the method outperforms existing STL-guided RL methods, providing more robust and efficient reward generation for deep-RL.

Conclusion: The proposed STL-guided RL method with online causation semantics offers improved training stability and convergence by generating rewards that accurately reflect instantaneous state dynamics.

Abstract: In real-time and safety-critical cyber-physical systems (CPSs), control synthesis must guarantee that generated policies meet stringent timing and correctness requirements under uncertain and dynamic conditions. Signal temporal logic (STL) has emerged as a powerful formalism of expressing real-time constraints, with its semantics enabling quantitative assessment of system behavior. Meanwhile, reinforcement learning (RL) has become an important method for solving control synthesis problems in unknown environments. Recent studies incorporate STL-based reward functions into RL to automatically synthesize control policies. However, the automatically inferred rewards obtained by these methods represent the global assessment of a whole or partial path but do not accumulate the rewards of local changes accurately, so the sparse global rewards may lead to non-convergence and unstable training performances. In this paper, we propose an online reward generation method guided by the online causation monitoring of STL. Our approach continuously monitors system behavior against an STL specification at each control step, computing the quantitative distance toward satisfaction or violation and thereby producing rewards that reflect instantaneous state dynamics. Additionally, we provide a smooth approximation of the causation semantics to overcome the discontinuity of the causation semantics and make it differentiable for using deep-RL methods. We have implemented a prototype tool and evaluated it in the Gym environment on a variety of continuously controlled benchmarks. Experimental results show that our proposed STL-guided RL method with online causation semantics outperforms existing relevant STL-guided RL methods, providing a more robust and efficient reward generation framework for deep-RL.

Method: Created oMeBench with over 10,000 annotated mechanistic steps and proposed oMeS evaluation framework combining step-level logic and chemical similarity. Analyzed state-of-the-art LLMs and tested fine-tuning approaches.

Result: Current models display promising chemical intuition but struggle with correct and consistent multi-step reasoning. Fine-tuning a specialist model on the proposed dataset increased performance by 50% over the leading closed-source model.

Conclusion: oMeBench provides a rigorous foundation for advancing AI systems toward genuine chemical reasoning, highlighting both the potential and limitations of current LLMs in organic mechanism understanding.

Abstract: Organic reaction mechanisms are the stepwise elementary reactions by which reactants form intermediates and products, and are fundamental to understanding chemical reactivity and designing new molecules and reactions. Although large language models (LLMs) have shown promise in understanding chemical tasks such as synthesis design, it is unclear to what extent this reflects genuine chemical reasoning capabilities, i.e., the ability to generate valid intermediates, maintain chemical consistency, and follow logically coherent multi-step pathways. We address this by introducing oMeBench, the first large-scale, expert-curated benchmark for organic mechanism reasoning in organic chemistry. It comprises over 10,000 annotated mechanistic steps with intermediates, type labels, and difficulty ratings. Furthermore, to evaluate LLM capability more precisely and enable fine-grained scoring, we propose oMeS, a dynamic evaluation framework that combines step-level logic and chemical similarity. We analyze the performance of state-of-the-art LLMs, and our results show that although current models display promising chemical intuition, they struggle with correct and consistent multi-step reasoning. Notably, we find that using prompting strategy and fine-tuning a specialist model on our proposed dataset increases performance by 50% over the leading closed-source model. We hope that oMeBench will serve as a rigorous foundation for advancing AI systems toward genuine chemical reasoning.

Method: Proposes a neuro-symbolic multi-agent architecture with belief states represented as Kripke models, enabling reasoning about possibility and necessity using modal logic. Uses domain-specific knowledge encoded as logical constraints to guide hypothesis generation.

Result: Successfully diagnoses complex, cascading failures in a high-fidelity simulated particle accelerator environment by combining semantic intuition of LMs with rigorous modal logic validation and factual world modeling.

Conclusion: Demonstrates a viable path toward more robust, reliable, and verifiable autonomous agents by integrating formal reasoning with language model capabilities.

Abstract: The development of intelligent agents, particularly those powered by language models (LMs), has shown the critical role in various environments that require intelligent and autonomous decision. Environments are not passive testing grounds and they represent the data required for agents to learn and exhibit very challenging conditions that require adaptive, complex and autonomous capacity to make decisions. While the paradigm of scaling models and datasets has led to remarkable emergent capabilities, we argue that scaling the structure, fidelity, and logical consistency of agent reasoning within these environments is a crucial, yet underexplored, dimension of AI research. This paper introduces a neuro-symbolic multi-agent architecture where the belief states of individual agents are formally represented as Kripke models. This foundational choice enables them to reason about known concepts of \emph{possibility} and \emph{necessity} using the formal language of modal logic. In this work, we use of immutable, domain-specific knowledge to make infere information, which is encoded as logical constraints essential for proper diagnosis. In the proposed model, we show constraints that actively guide the hypothesis generation of LMs, effectively preventing them from reaching physically or logically untenable conclusions. In a high-fidelity simulated particle accelerator environment, our system successfully diagnoses complex, cascading failures by combining the powerful semantic intuition of LMs with the rigorous, verifiable validation of modal logic and a factual world model and showcasing a viable path toward more robust, reliable, and verifiable autonomous agents.

Method: Uses hierarchical citation graph with three layers (Foundation, Development, Frontier) to capture research evolution, combines horizontal search within layers and vertical traversal across layers, and employs multi-agent validation for consistency and accuracy.

Result: Outperforms state-of-the-art frameworks in evaluations by human experts and LLM-as-a-judge, producing more comprehensive and better structured surveys aligned with field knowledge taxonomy.

Conclusion: SurveyG effectively addresses limitations of existing approaches by integrating structural and contextual knowledge through hierarchical citation graphs, enabling generation of high-quality survey papers with coherent organization.

Abstract: Large language models (LLMs) are increasingly adopted for automating survey paper generation \cite{wang2406autosurvey, liang2025surveyx, yan2025surveyforge,su2025benchmarking,wen2025interactivesurvey}. Existing approaches typically extract content from a large collection of related papers and prompt LLMs to summarize them directly. However, such methods often overlook the structural relationships among papers, resulting in generated surveys that lack a coherent taxonomy and a deeper contextual understanding of research progress. To address these shortcomings, we propose \textbf{SurveyG}, an LLM-based agent framework that integrates \textit{hierarchical citation graph}, where nodes denote research papers and edges capture both citation dependencies and semantic relatedness between their contents, thereby embedding structural and contextual knowledge into the survey generation process. The graph is organized into three layers: \textbf{Foundation}, \textbf{Development}, and \textbf{Frontier}, to capture the evolution of research from seminal works to incremental advances and emerging directions. By combining horizontal search within layers and vertical depth traversal across layers, the agent produces multi-level summaries, which are consolidated into a structured survey outline. A multi-agent validation stage then ensures consistency, coverage, and factual accuracy in generating the final survey. Experiments, including evaluations by human experts and LLM-as-a-judge, demonstrate that SurveyG outperforms state-of-the-art frameworks, producing surveys that are more comprehensive and better structured to the underlying knowledge taxonomy of a field.

Method: MMIA recursively breaks down complex medical tasks into atomic, evidence-based steps and automatically audits the reasoning chain for logical coherence and evidence traceability. It features a bootstrapping mode that stores validated reasoning chains as theorems for subsequent RAG-based problem solving.

Result: MMIA achieved an error detection rate exceeding 98% with false positive rate below 1% across four healthcare administration domains, significantly outperforming baseline LLMs. The RAG matching mode is projected to reduce processing costs by approximately 85% as the knowledge base matures.

Conclusion: MMIA’s verifiable reasoning framework represents a significant step toward creating trustworthy, transparent, and cost-effective AI systems, making LLM technology viable for critical applications in medicine.

Abstract: Large Language Models (LLMs) show promise in medicine but are prone to factual and logical errors, which is unacceptable in this high-stakes field. To address this, we introduce the “Haibu Mathematical-Medical Intelligent Agent” (MMIA), an LLM-driven architecture that ensures reliability through a formally verifiable reasoning process. MMIA recursively breaks down complex medical tasks into atomic, evidence-based steps. This entire reasoning chain is then automatically audited for logical coherence and evidence traceability, similar to theorem proving. A key innovation is MMIA’s “bootstrapping” mode, which stores validated reasoning chains as “theorems.” Subsequent tasks can then be efficiently solved using Retrieval-Augmented Generation (RAG), shifting from costly first-principles reasoning to a low-cost verification model. We validated MMIA across four healthcare administration domains, including DRG/DIP audits and medical insurance adjudication, using expert-validated benchmarks. Results showed MMIA achieved an error detection rate exceeding 98% with a false positive rate below 1%, significantly outperforming baseline LLMs. Furthermore, the RAG matching mode is projected to reduce average processing costs by approximately 85% as the knowledge base matures. In conclusion, MMIA’s verifiable reasoning framework is a significant step toward creating trustworthy, transparent, and cost-effective AI systems, making LLM technology viable for critical applications in medicine.

Method: Proposes GRAIL framework: compresses node representations into latent features, uses graph diffusion to model restoration process, quantizes features into discrete tokens, fine-tunes LLM as generative restorer, and employs reinforcement learning with alignment and confidence rewards.

Result: Extensive experiments demonstrate the effectiveness of the approach across various datasets, showing successful restoration of target graphs to source-domain-like states without accessing source data.

Conclusion: The proposed GRAIL framework successfully addresses TT-GDA by reframing it as a generative graph restoration problem using LLMs, achieving effective domain adaptation without source data access through innovative diffusion modeling and reinforcement learning techniques.

Abstract: Graph domain adaptation (GDA) has achieved great attention due to its effectiveness in addressing the domain shift between train and test data. A significant bottleneck in existing graph domain adaptation methods is their reliance on source-domain data, which is often unavailable due to privacy or security concerns. This limitation has driven the development of Test-Time Graph Domain Adaptation (TT-GDA), which aims to transfer knowledge without accessing the source examples. Inspired by the generative power of large language models (LLMs), we introduce a novel framework that reframes TT-GDA as a generative graph restoration problem, “restoring the target graph to its pristine, source-domain-like state”. There are two key challenges: (1) We need to construct a reasonable graph restoration process and design an effective encoding scheme that an LLM can understand, bridging the modality gap. (2) We need to devise a mechanism to ensure the restored graph acquires the intrinsic features of the source domain, even without access to the source data. To ensure the effectiveness of graph restoration, we propose GRAIL, that restores the target graph into a state that is well-aligned with the source domain. Specifically, we first compress the node representations into compact latent features and then use a graph diffusion process to model the graph restoration process. Then a quantization module encodes the restored features into discrete tokens. Building on this, an LLM is fine-tuned as a generative restorer to transform a “noisy” target graph into a “native” one. To further improve restoration quality, we introduce a reinforcement learning process guided by specialized alignment and confidence rewards. Extensive experiments demonstrate the effectiveness of our approach across various datasets.

Method: Introduces ACONIC framework that models tasks as constraint problems and leverages formal complexity measures to guide decomposition systematically.

Result: On combinatorial (SATBench) and LLM database querying tasks (Spider), decomposition guided by complexity measures improves agent performance by 10-40 percentage points.

Conclusion: ACONIC provides a systematic approach to task decomposition that significantly enhances LLM reliability on complex tasks compared to heuristic methods.

Abstract: Large Language Models (LLMs) suffer from reliability issues on complex tasks, as existing decomposition methods are heuristic and rely on agent or manual decomposition. This work introduces a novel, systematic decomposition framework that we call Analysis of CONstraint-Induced Complexity (ACONIC), which models the task as a constraint problem and leveraging formal complexity measures to guide decomposition. On combinatorial (SATBench) and LLM database querying tasks (Spider), we find that by decomposing the tasks following the measure of complexity, agent can perform considerably better (10-40 percentage point).

Method: Group Contrastive Policy Optimization (GCPO) incorporates external standard reference answers that provide correct responses when the model fails, guiding the model toward accurate update directions and enabling emulation of reference problem-solving strategies.

Result: GCPO achieves outstanding results across multiple benchmark datasets with substantial improvements over baseline models, demonstrating improved training efficiency and enhanced generalization in reasoning.

Conclusion: GCPO effectively overcomes GRPO’s limitations by leveraging external reference answers, enabling full sample utilization and improved reasoning generalization while achieving significant performance gains.

Abstract: Reinforcement learning has been widely applied to enhance the reasoning capabilities of large language models. Extending the inference limits of smaller models has become a prominent research focus. However, algorithms such as Group Relative Policy Optimization (GRPO) suffer from a clear drawback: the upper bound of a model’s rollout responses is entirely determined by the model itself, preventing the acquisition of knowledge from samples that are either all incorrect or all correct. In this paper, we introduce Group Contrastive Policy Optimization (GCPO), a method that incorporates external standard reference answers. When the model cannot solve a problem, the reference answer supplies the correct response, steering the model toward an unequivocally accurate update direction. This approach offers two main advantages: (1) it improves training efficiency by fully utilizing every sample; (2) it enables the model to emulate the problem solving strategy of the reference answer during training, thereby enhancing generalization in reasoning. GCPO achieves outstanding results across multiple benchmark datasets, yielding substantial improvements over the baseline model. Our code is available at: https://github.com/AchoWu/GCPO.

Method: Proposed SHADOW - a multi-headed sequential reinforcement learning framework that integrates continuous navigation control, discrete communication actions, and opponent modeling for behavior prediction. Both agents learn movement policies via RL while the pursuer additionally learns a communication policy.

Result: SHADOW pursuers achieve higher success rates than six competitive baselines. Ablation studies confirm temporal sequence modeling and opponent modeling are critical for effective decision-making.

Conclusion: The learned policies generalize well across varying communication risks and physical asymmetries between agents, demonstrating robust performance in adversarial pursuit-evasion scenarios.

Abstract: Adversarial environments require agents to navigate a key strategic trade-off: acquiring information enhances situational awareness, but may simultaneously expose them to threats. To investigate this tension, we formulate a PursuitEvasion-Exposure-Concealment Game (PEEC) in which a pursuer agent must decide when to communicate in order to obtain the evader’s position. Each communication reveals the pursuer’s location, increasing the risk of being targeted. Both agents learn their movement policies via reinforcement learning, while the pursuer additionally learns a communication policy that balances observability and risk. We propose SHADOW (Strategic-communication Hybrid Action Decision-making under partial Observation for Warfare), a multi-headed sequential reinforcement learning framework that integrates continuous navigation control, discrete communication actions, and opponent modeling for behavior prediction. Empirical evaluations show that SHADOW pursuers achieve higher success rates than six competitive baselines. Our ablation study confirms that temporal sequence modeling and opponent modeling are critical for effective decision-making. Finally, our sensitivity analysis reveals that the learned policies generalize well across varying communication risks and physical asymmetries between agents.

Method: CityNav uses a hierarchical framework with a global traffic allocation agent that coordinates strategic traffic flow distribution across regions, and local navigation agents that generate adaptive routes. It employs cooperative reasoning optimization with dual-reward structure: individual rewards for per-vehicle efficiency and shared rewards for network-wide coordination.

Result: Extensive experiments on four real-world road networks (up to 1.6 million roads and 430,000 intersections) show CityNav consistently outperforms nine classical path search and RL-based baselines in city-scale travel efficiency and congestion mitigation.

Conclusion: CityNav demonstrates the potential of LLMs to enable scalable, adaptive, and cooperative city-wide traffic navigation, providing a foundation for intelligent, large-scale vehicle routing in complex urban environments.

Abstract: The rise of Internet of Vehicles (IoV) technologies is transforming traffic management from isolated control to a collective, multi-vehicle process. At the heart of this shift is multi-vehicle dynamic navigation, which requires simultaneously routing large fleets under evolving traffic conditions. Existing path search algorithms and reinforcement learning methods struggle to scale to city-wide networks, often failing to capture the nonlinear, stochastic, and coupled dynamics of urban traffic. To address these challenges, we propose CityNav, a hierarchical, LLM-powered framework for large-scale multi-vehicle navigation. CityNav integrates a global traffic allocation agent, which coordinates strategic traffic flow distribution across regions, with local navigation agents that generate locally adaptive routes aligned with global directives. To enable effective cooperation, we introduce a cooperative reasoning optimization mechanism, in which agents are jointly trained with a dual-reward structure: individual rewards promote per-vehicle efficiency, while shared rewards encourage network-wide coordination and congestion reduction. Extensive experiments on four real-world road networks of varying scales (up to 1.6 million roads and 430,000 intersections) and traffic datasets demonstrate that CityNav consistently outperforms nine classical path search and RL-based baselines in city-scale travel efficiency and congestion mitigation. Our results highlight the potential of LLMs to enable scalable, adaptive, and cooperative city-wide traffic navigation, providing a foundation for intelligent, large-scale vehicle routing in complex urban environments. Our project is available at https://github.com/usail-hkust/CityNav.

Method: Created FinMR dataset with over 3,200 meticulously curated question-answer pairs across 15 financial topics, featuring sophisticated mathematical reasoning, advanced financial knowledge, and nuanced visual interpretation across multiple image types.

Result: Benchmarking revealed significant performance gaps between leading MLLMs and professional financial analysts, identifying key improvement areas: precise image analysis, accurate application of complex financial formulas, and deeper contextual financial understanding.

Conclusion: FinMR establishes an essential benchmark for assessing and advancing multimodal financial reasoning toward professional analyst-level competence through its rich visual content and thorough explanatory annotations.

Abstract: Multimodal Large Language Models (MLLMs) have made substantial progress in recent years. However, their rigorous evaluation within specialized domains like finance is hindered by the absence of datasets characterized by professional-level knowledge intensity, detailed annotations, and advanced reasoning complexity. To address this critical gap, we introduce FinMR, a high-quality, knowledge-intensive multimodal dataset explicitly designed to evaluate expert-level financial reasoning capabilities at a professional analyst’s standard. FinMR comprises over 3,200 meticulously curated and expertly annotated question-answer pairs across 15 diverse financial topics, ensuring broad domain diversity and integrating sophisticated mathematical reasoning, advanced financial knowledge, and nuanced visual interpretation tasks across multiple image types. Through comprehensive benchmarking with leading closed-source and open-source MLLMs, we highlight significant performance disparities between these models and professional financial analysts, uncovering key areas for model advancement, such as precise image analysis, accurate application of complex financial formulas, and deeper contextual financial understanding. By providing richly varied visual content and thorough explanatory annotations, FinMR establishes itself as an essential benchmark tool for assessing and advancing multimodal financial reasoning toward professional analyst-level competence.

Method: Uses a two-stage teacher-student architecture where a powerful teacher LLM infers directed causal graphs using heuristic search and pairwise causality testing, then a lightweight student agent refines the graph and fine-tunes on high-confidence causal associations encoded as rich textual prompts.

Result: Extensive experiments on real-world datasets with 25 baselines show that Augur achieves competitive performance and robust zero-shot generalization.

Conclusion: The framework improves predictive accuracy while providing transparent, traceable reasoning about variable interactions through causal discovery and utilization.

Abstract: Large language models (LLM) have emerged as a promising avenue for time series forecasting, offering the potential to integrate multimodal data. However, existing LLM-based approaches face notable limitations-such as marginalized role in model architectures, reliance on coarse statistical text prompts, and lack of interpretability. In this work, we introduce Augur, a fully LLM driven time series forecasting framework that exploits LLM causal reasoning to discover and use directed causal associations among covariates. Augur uses a two stage teacher student architecture where a powerful teacher LLM infers a directed causal graph from time series using heuristic search together with pairwise causality testing. A lightweight student agent then refines the graph and fine tune on high confidence causal associations that are encoded as rich textual prompts to perform forecasting. This design improves predictive accuracy while yielding transparent, traceable reasoning about variable interactions. Extensive experiments on real-world datasets with 25 baselines demonstrate that Augur achieves competitive performance and robust zero-shot generalization.

Method: Developed a comprehensive framework using LLM-as-a-Judge methodology to systematically measure research reports across three dimensions: quality, redundancy, and factuality. Created a benchmark of 100 curated queries across 12 real-world categories.

Result: The framework achieved strong expert concordance and was used to evaluate four leading commercial systems, revealing distinct design philosophies and performance trade-offs.

Conclusion: DeepResearch-ReportEval establishes foundational evaluation insights as AI systems evolve from information assistants toward intelligent research partners, with code and data publicly available.

Abstract: DeepResearch agents represent a transformative AI paradigm, conducting expert-level research through sophisticated reasoning and multi-tool integration. However, evaluating these systems remains critically challenging due to open-ended research scenarios and existing benchmarks that focus on isolated capabilities rather than holistic performance. Unlike traditional LLM tasks, DeepResearch systems must synthesize diverse sources, generate insights, and present coherent findings, which are capabilities that resist simple verification. To address this gap, we introduce DeepResearch-ReportEval, a comprehensive framework designed to assess DeepResearch systems through their most representative outputs: research reports. Our approach systematically measures three dimensions: quality, redundancy, and factuality, using an innovative LLM-as-a-Judge methodology achieving strong expert concordance. We contribute a standardized benchmark of 100 curated queries spanning 12 real-world categories, enabling systematic capability comparison. Our evaluation of four leading commercial systems reveals distinct design philosophies and performance trade-offs, establishing foundational insights as DeepResearch evolves from information assistants toward intelligent research partners. Source code and data are available at: https://github.com/HKUDS/DeepResearch-Eval.

Method: Builds on human-centric AI transparency approaches combined with socio-technical systems view to develop the concept of meaningful transparency for civic AI systems.

Result: Develops a framework for transparency that allows publics to engage with AI systems affecting their lives by connecting understanding with potential for action.

Conclusion: Meaningful transparency is needed for civic AI systems to enable public engagement and connect understanding with actionable insights, moving beyond purely technical transparency approaches.

Abstract: Artificial intelligence has become a part of the provision of governmental services, from making decisions about benefits to issuing fines for parking violations. However, AI systems rarely live up to the promise of neutral optimisation, creating biased or incorrect outputs and reducing the agency of both citizens and civic workers to shape the way decisions are made. Transparency is a principle that can both help subjects understand decisions made about them and shape the processes behind those decisions. However, transparency as practiced around AI systems tends to focus on the production of technical objects that represent algorithmic aspects of decision making. These are often difficult for publics to understand, do not connect to potential for action, and do not give insight into the wider socio-material context of decision making. In this paper, we build on existing approaches that take a human-centric view on AI transparency, combined with a socio-technical systems view, to develop the concept of meaningful transparency for civic AI systems: transparencies that allow publics to engage with AI systems that affect their lives, connecting understanding with potential for action.

Method: Proposes FactFin framework that applies counterfactual perturbations to force LLM-based agents to learn causal drivers rather than memorized outcomes. Includes four components: Strategy Code Generator, Retrieval-Augmented Generation, Monte Carlo Tree Search, and Counterfactual Simulator.

Result: Extensive experiments demonstrate that FactFin surpasses all baselines in out-of-sample generalization, achieving superior risk-adjusted performance compared to existing methods.

Conclusion: The proposed FactFin framework effectively mitigates information leakage issues in LLM-based financial agents, enabling better causal learning and robust out-of-sample performance, addressing the fundamental ‘profit mirage’ problem.

Abstract: LLM-based financial agents have attracted widespread excitement for their ability to trade like human experts. However, most systems exhibit a “profit mirage”: dazzling back-tested returns evaporate once the model’s knowledge window ends, because of the inherent information leakage in LLMs. In this paper, we systematically quantify this leakage issue across four dimensions and release FinLake-Bench, a leakage-robust evaluation benchmark. Furthermore, to mitigate this issue, we introduce FactFin, a framework that applies counterfactual perturbations to compel LLM-based agents to learn causal drivers instead of memorized outcomes. FactFin integrates four core components: Strategy Code Generator, Retrieval-Augmented Generation, Monte Carlo Tree Search, and Counterfactual Simulator. Extensive experiments show that our method surpasses all baselines in out-of-sample generalization, delivering superior risk-adjusted performance.

Method: The framework combines established agentic AI patterns (multi-agent collaboration, multi-source retrieval) with persistent memory, dynamic coordination, self-validation, and evolving user profiles.

Result: Evaluation on three public datasets shows improvements in retrieval accuracy, response correctness, and BertScore. A 5-day pilot user study provides initial positive feedback on perceived personalization.

Conclusion: Integrating persistent memory and user profiles shows potential to improve adaptivity and perceived personalization of LLM-based agents, providing guidance for future work.

Abstract: Large language models (LLMs) increasingly serve as the central control unit of AI agents, yet current approaches remain limited in their ability to deliver personalized interactions. While Retrieval Augmented Generation enhances LLM capabilities by improving context-awareness, it lacks mechanisms to combine contextual information with user-specific data. Although personalization has been studied in fields such as human-computer interaction or cognitive science, existing perspectives largely remain conceptual, with limited focus on technical implementation. To address these gaps, we build on a unified definition of personalization as a conceptual foundation to derive technical requirements for adaptive, user-centered LLM-based agents. Combined with established agentic AI patterns such as multi-agent collaboration or multi-source retrieval, we present a framework that integrates persistent memory, dynamic coordination, self-validation, and evolving user profiles to enable personalized long-term interactions. We evaluate our approach on three public datasets using metrics such as retrieval accuracy, response correctness, or BertScore. We complement these results with a five-day pilot user study providing initial insights into user feedback on perceived personalization. The study provides early indications that guide future work and highlights the potential of integrating persistent memory and user profiles to improve the adaptivity and perceived personalization of LLM-based agents.

Method: Hybrid framework integrating genetic algorithms with intelligent multi-agent coordination mechanisms, incorporating real-time market microstructure intelligence and adaptive strategy performance feedback.

Result: Comprehensive empirical evaluation across three cryptocurrencies shows systematic and statistically significant performance improvements on both total returns and risk-adjusted metrics.

Conclusion: The framework successfully transcends limitations of static optimization approaches and demonstrates effectiveness in dynamic financial environments.

Abstract: Cryptocurrency markets present formidable challenges for trading strategy optimization due to extreme volatility, non-stationary dynamics, and complex microstructure patterns that render conventional parameter optimization methods fundamentally inadequate. We introduce Cypto Genetic Algorithm Agent (CGA-Agent), a pioneering hybrid framework that synergistically integrates genetic algorithms with intelligent multi-agent coordination mechanisms for adaptive trading strategy parameter optimization in dynamic financial environments. The framework uniquely incorporates real-time market microstructure intelligence and adaptive strategy performance feedback through intelligent mechanisms that dynamically guide evolutionary processes, transcending the limitations of static optimization approaches. Comprehensive empirical evaluation across three cryptocurrencies demonstrates systematic and statistically significant performance improvements on both total returns and risk-adjusted metrics.

Method: Stepwise Hybrid Examination Reinforcement Learning (TaoSR-SHE) with Stepwise Reward Policy Optimization (SRPO) using step-level rewards from a hybrid generative reward model and human-annotated verifier, plus diversified data filtering and multi-stage curriculum learning.

Result: Extensive experiments show TaoSR-SHE improves reasoning quality and relevance-prediction accuracy in large-scale e-commerce settings, outperforming SFT, DPO, GRPO, and other baselines.

Conclusion: TaoSR-SHE enhances both performance and interpretability in e-commerce search relevance analysis while improving robustness compared to existing methods.

Abstract: Query-product relevance analysis is a foundational technology in e-commerce search engines and has become increasingly important in AI-driven e-commerce. The recent emergence of large language models (LLMs), particularly their chain-of-thought (CoT) reasoning capabilities, offers promising opportunities for developing relevance systems that are both more interpretable and more robust. However, existing training paradigms have notable limitations: SFT and DPO suffer from poor generalization on long-tail queries and from a lack of fine-grained, stepwise supervision to enforce rule-aligned reasoning. In contrast, reinforcement learning with verification rewards (RLVR) suffers from sparse feedback, which provides insufficient signal to correct erroneous intermediate steps, thereby undermining logical consistency and limiting performance in complex inference scenarios. To address these challenges, we introduce the Stepwise Hybrid Examination Reinforcement Learning framework for Taobao Search Relevance (TaoSR-SHE). At its core is Stepwise Reward Policy Optimization (SRPO), a reinforcement learning algorithm that leverages step-level rewards generated by a hybrid of a high-quality generative stepwise reward model and a human-annotated offline verifier, prioritizing learning from critical correct and incorrect reasoning steps. TaoSR-SHE further incorporates two key techniques: diversified data filtering to encourage exploration across varied reasoning paths and mitigate policy entropy collapse, and multi-stage curriculum learning to foster progressive capability growth. Extensive experiments on real-world search benchmarks show that TaoSR-SHE improves both reasoning quality and relevance-prediction accuracy in large-scale e-commerce settings, outperforming SFT, DPO, GRPO, and other baselines, while also enhancing interpretability and robustness.

Method: Created VoiceAgentBench with synthetic spoken queries including Indian context dialogues, single-tool invocations, multi-tool workflows, multi-turn interactions, and safety evaluations. Used novel sampling algorithm for TTS voice conversion to maximize acoustic and speaker diversity.

Result: Experiments revealed significant gaps in contextual tool orchestration tasks, Indic generalization, and adversarial robustness, exposing critical limitations of current SpeechLMs.

Conclusion: The benchmark successfully identifies major shortcomings in current SpeechLMs, particularly in multilingual agentic scenarios and robustness, highlighting the need for improved models.

Abstract: Large-scale Speech Language Models (SpeechLMs) have enabled voice assistants capable of understanding natural spoken queries and performing complex tasks. However, existing speech benchmarks primarily focus on isolated capabilities such as transcription, or question-answering, and do not systematically evaluate agentic scenarios encompassing multilingual and cultural understanding, as well as adversarial robustness. To address this, we introduce VoiceAgentBench, a comprehensive benchmark designed to evaluate SpeechLMs in realistic spoken agentic settings. It comprises over 5,500 synthetic spoken queries, including dialogues grounded in Indian context, covering single-tool invocations, multi-tool workflows, multi-turn interactions, and safety evaluations. The benchmark supports English, Hindi, and 5 other Indian languages, reflecting real-world linguistic and cultural diversity. We simulate speaker variability using a novel sampling algorithm that selects audios for TTS voice conversion based on its speaker embeddings, maximizing acoustic and speaker diversity. Our evaluation measures tool selection accuracy, structural consistency, and the correctness of tool invocations, including adversarial robustness. Our experiments reveal significant gaps in contextual tool orchestration tasks, Indic generalization, and adversarial robustness, exposing critical limitations of current SpeechLMs.

Method: A multi-agent framework with three specialized agents: information-managing agent for slot-based management and user interaction, decision-making agent for conflict-aware planning, and reflecting agent for task reflection and consistency validation, all connected through a shared memory module.

Result: ReInAgent effectively resolves information dilemmas and produces outcomes more aligned with user preferences, achieving 25% higher success rate than Mobile-Agent-v2 on complex tasks involving information dilemmas.

Conclusion: The framework enables more adaptive and reliable mobile task navigation in complex real-world scenarios through continuous contextual analysis and sustained user-agent collaboration, overcoming limitations of approaches relying on clear static task assumptions.

Abstract: Mobile GUI agents exhibit substantial potential to facilitate and automate the execution of user tasks on mobile phones. However, exist mobile GUI agents predominantly privilege autonomous operation and neglect the necessity of active user engagement during task execution. This omission undermines their adaptability to information dilemmas including ambiguous, dynamically evolving, and conflicting task scenarios, leading to execution outcomes that deviate from genuine user requirements and preferences. To address these shortcomings, we propose ReInAgent, a context-aware multi-agent framework that leverages dynamic information management to enable human-in-the-loop mobile task navigation. ReInAgent integrates three specialized agents around a shared memory module: an information-managing agent for slot-based information management and proactive interaction with the user, a decision-making agent for conflict-aware planning, and a reflecting agent for task reflection and information consistency validation. Through continuous contextual information analysis and sustained user-agent collaboration, ReInAgent overcomes the limitation of existing approaches that rely on clear and static task assumptions. Consequently, it enables more adaptive and reliable mobile task navigation in complex, real-world scenarios. Experimental results demonstrate that ReInAgent effectively resolves information dilemmas and produces outcomes that are more closely aligned with genuine user preferences. Notably, on complex tasks involving information dilemmas, ReInAgent achieves a 25% higher success rate than Mobile-Agent-v2.

Method: Used linear reconstruction and principal component analysis on embeddings from models by OpenAI, Google Gemini, and Voyage AI to analyze numeric representation properties.

Result: While numeric reconstruction achieves high fidelity (R² ≥ 0.95), principal components explain minimal variation, indicating many embedding components are orthogonal to numeric input. Performance degrades with increasing decimal precision.

Conclusion: Language models represent numeric spaces non-continuously with significant noise, which has implications for applications requiring high numerical precision, large magnitudes, or mixed-sign values.

Abstract: Recent research has extensively studied how large language models manipulate integers in specific arithmetic tasks, and on a more fundamental level, how they represent numeric values. These previous works have found that language model embeddings can be used to reconstruct the original values, however, they do not evaluate whether language models actually model continuous values as continuous. Using expected properties of the embedding space, including linear reconstruction and principal component analysis, we show that language models not only represent numeric spaces as non-continuous but also introduce significant noise. Using models from three major providers (OpenAI, Google Gemini and Voyage AI), we show that while reconstruction is possible with high fidelity ($R^2 \geq 0.95$), principal components only explain a minor share of variation within the embedding space. This indicates that many components within the embedding space are orthogonal to the simple numeric input space. Further, both linear reconstruction and explained variance suffer with increasing decimal precision, despite the ordinal nature of the input space being fundamentally unchanged. The findings of this work therefore have implications for the many areas where embedding models are used, in-particular where high numerical precision, large magnitudes or mixed-sign values are common.

Method: PEAR incorporates phase-dependent entropy into reward design - penalizing high entropy during thinking phase while allowing moderate exploration in final answer phase, enabling adaptive length control without explicit targets.

Result: Extensive experiments across four benchmarks show PEAR consistently reduces response length while sustaining competitive accuracy across model scales, with strong out-of-distribution robustness.

Conclusion: Phase-dependent entropy serves as an effective control mechanism for balancing reasoning conciseness and performance, enabling more efficient and usable reasoning models.

Abstract: Large Reasoning Models (LRMs) have achieved impressive performance on complex reasoning tasks by generating detailed chain-of-thought (CoT) explanations. However, these responses are often excessively long, containing redundant reasoning steps that inflate inference cost and reduce usability. Controlling the length of generated reasoning without sacrificing accuracy remains an open challenge. Through a systematic empirical analysis, we reveal a consistent positive correlation between model entropy and response length at different reasoning stages across diverse LRMs: the thinking phase exhibits higher entropy, reflecting exploratory behavior of longer responses, while the final answer phase shows lower entropy, indicating a more deterministic solution.This observation suggests that entropy at different reasoning stages can serve as a control knob for balancing conciseness and performance. Based on this insight, this paper introduces Phase Entropy Aware Reward (PEAR), a reward mechanism that incorporating phase-dependent entropy into the reward design. Instead of treating all tokens uniformly, PEAR penalize excessive entropy during the thinking phase and allowing moderate exploration at the final answer phase, which encourages models to generate concise reasoning traces that retain sufficient flexibility to solve the task correctly. This enables adaptive control of response length without relying on explicit length targets or rigid truncation rules. Extensive experiments across four benchmarks demonstrate that PEAR consistently reduces response length while sustaining competitive accuracy across model scales. In addition, PEAR demonstrates strong out-of-distribution (OOD) robustness beyond the training distribution. Our code is available at: https://github.com/iNLP-Lab/PEAR.

Method: AILoRA performs function-aware initialization: injects principal components of W^Q to retain task-adaptive capacity, and minor components of W^V to preserve generalizable feature representations. This asymmetric strategy better captures specialized roles of attention parameters.

Result: The method enhances both finetuning performance and convergence efficiency by better aligning with the distinct functional roles of Q and V projection matrices.

Conclusion: AILoRA successfully addresses LoRA’s limitations through asymmetric low-rank priors that leverage the functional differences between attention projection matrices, improving parameter-efficient finetuning.

Abstract: Parameter-efficient finetuning (PEFT) aims to mitigate the substantial computational and memory overhead involved in adapting large-scale pretrained models to diverse downstream tasks. Among numerous PEFT strategies, Low-Rank Adaptation (LoRA) has emerged as one of the most widely adopted approaches due to its robust empirical performance and low implementation complexity. In practical deployment, LoRA is typically applied to the $W^Q$ and $W^V$ projection matrices of self-attention modules, enabling an effective trade-off between model performance and parameter efficiency. While LoRA has achieved considerable empirical success, it still encounters challenges such as suboptimal performance and slow convergence. To address these limitations, we introduce \textbf{AILoRA}, a novel parameter-efficient method that incorporates function-aware asymmetric low-rank priors. Our empirical analysis reveals that the projection matrices $W^Q$ and $W^V$ in the self-attention mechanism exhibit distinct parameter characteristics, stemming from their functional differences. Specifically, $W^Q$ captures task-specific semantic space knowledge essential for attention distributions computation, making its parameters highly sensitive to downstream task variations. In contrast, $W^V$ encodes token-level feature representations that tend to remain stable across tasks and layers. Leveraging these insights, AILoRA performs a function-aware initialization by injecting the principal components of $W^Q$ to retain task-adaptive capacity, and the minor components of $W^V$ to preserve generalizable feature representations. This asymmetric initialization strategy enables LoRA modules to better capture the specialized roles of attention parameters, thereby enhancing both finetuning performance and convergence efficiency.

Method: LinguaSim uses LLMs to convert natural language into interactive 3D scenarios with dynamic vehicle interactions, featuring a feedback calibration module to refine generation precision and ensure fidelity to user intent.

Result: Experiments show LinguaSim generates scenarios with varying criticality aligned with descriptions (ACT: 0.072s dangerous vs. 3.532s safe; comfortability: 0.654 vs. 0.764), and reduces crash rate from 46.9% to 6.3% through refinement.

Conclusion: LinguaSim bridges the gap between natural language and closed-loop interactive simulations, facilitating high-fidelity scenario creation that enhances safety testing and training for autonomous vehicles.

Abstract: The generation of testing and training scenarios for autonomous vehicles has drawn significant attention. While Large Language Models (LLMs) have enabled new scenario generation methods, current methods struggle to balance command adherence accuracy with the realism of real-world driving environments. To reduce scenario description complexity, these methods often compromise realism by limiting scenarios to 2D, or open-loop simulations where background vehicles follow predefined, non-interactive behaviors. We propose LinguaSim, an LLM-based framework that converts natural language into realistic, interactive 3D scenarios, ensuring both dynamic vehicle interactions and faithful alignment between the input descriptions and the generated scenarios. A feedback calibration module further refines the generation precision, improving fidelity to user intent. By bridging the gap between natural language and closed-loop, interactive simulations, LinguaSim constrains adversarial vehicle behaviors using both the scenario description and the autonomous driving model guiding them. This framework facilitates the creation of high-fidelity scenarios that enhance safety testing and training. Experiments show LinguaSim can generate scenarios with varying criticality aligned with different natural language descriptions (ACT: 0.072 s for dangerous vs. 3.532 s for safe descriptions; comfortability: 0.654 vs. 0.764), and its refinement module effectively reduces excessive aggressiveness in LinguaSim’s initial outputs, lowering the crash rate from 46.9% to 6.3% to better match user intentions.

Method: Proposes MCCS algorithm with novel nonconformity score for conjunctive conditions and global BH procedure for disjunctive conditions, ensuring finite-sample FDR control.

Result: Extensive experiments show MCCS outperforms baselines, generalizes across diverse condition combinations and real-world modalities, and scales to multi-task settings.

Conclusion: MCCS enables rigorous FDR-controlled selection in various multi-condition environments, addressing limitations of traditional conformal selection methods.

Abstract: Selecting high-quality candidates from large-scale datasets is critically important in resource-constrained applications such as drug discovery, precision medicine, and the alignment of large language models. While conformal selection methods offer a rigorous solution with False Discovery Rate (FDR) control, their applicability is confined to single-threshold scenarios (i.e., y

c) and overlooks practical needs for multi-condition selection, such as conjunctive or disjunctive conditions. In this work, we propose the Multi-Condition Conformal Selection (MCCS) algorithm, which extends conformal selection to scenarios with multiple conditions. In particular, we introduce a novel nonconformity score with regional monotonicity for conjunctive conditions and a global Benjamini-Hochberg (BH) procedure for disjunctive conditions, thereby establishing finite-sample FDR control with theoretical guarantees. The integration of these components enables the proposed method to achieve rigorous FDR-controlled selection in various multi-condition environments. Extensive experiments validate the superiority of MCCS over baselines, its generalizability across diverse condition combinations, different real-world modalities, and multi-task scalability.

Method: AutoQual mimics human research process by iteratively generating feature hypotheses through reflection, operationalizing them via autonomous tool implementation, and accumulating experience in persistent memory.

Result: Deployed on a large-scale platform with billion-level user base, A/B testing showed 0.79% increase in average reviews viewed per user and 0.27% increase in conversion rate of review readers.

Conclusion: AutoQual effectively automates interpretable feature discovery for quality assessment and can serve as a general framework for transforming tacit knowledge into explicit, computable features.

Abstract: Ranking online reviews by their intrinsic quality is a critical task for e-commerce platforms and information services, impacting user experience and business outcomes. However, quality is a domain-dependent and dynamic concept, making its assessment a formidable challenge. Traditional methods relying on hand-crafted features are unscalable across domains and fail to adapt to evolving content patterns, while modern deep learning approaches often produce black-box models that lack interpretability and may prioritize semantics over quality. To address these challenges, we propose AutoQual, an LLM-based agent framework that automates the discovery of interpretable features. While demonstrated on review quality assessment, AutoQual is designed as a general framework for transforming tacit knowledge embedded in data into explicit, computable features. It mimics a human research process, iteratively generating feature hypotheses through reflection, operationalizing them via autonomous tool implementation, and accumulating experience in a persistent memory. We deploy our method on a large-scale online platform with a billion-level user base. Large-scale A/B testing confirms its effectiveness, increasing average reviews viewed per user by 0.79% and the conversion rate of review readers by 0.27%.

Method: Uses OWL 2 QL ontology engineering to define sensitive attributes and infer proxies through logical reasoning, constructs sigma algebra G to capture biased patterns, then applies Delbaen Majumdar optimal transport to generate fair representations independent of G while minimizing L2 distance.

Result: Achieves complete fairness by ensuring true independence from sensitive attributes and their proxies, with applications in domains like loan approval where proxies (e.g., ZIP code revealing race) are problematic.

Conclusion: Provides a certifiable and mathematically grounded method for trustworthy AI that overcomes limitations of current bias mitigation approaches through systematic removal of sensitive information and proxies.

Abstract: This paper presents a framework for provably fair AI that overcomes the limits of current bias mitigation methods by systematically removing all sensitive information and its proxies. Using ontology engineering in OWL 2 QL, it formally defines sensitive attributes and infers their proxies through logical reasoning, constructing a sigma algebra G that captures the full structure of biased patterns. Fair representations are then obtained via Delbaen Majumdar optimal transport, which generates variables independent of G while minimizing L2 distance to preserve accuracy. This guarantees true independence rather than mere decorrelation. By modeling bias as dependence between sigma algebras, compiling ontological knowledge into measurable structures, and using optimal transport as the unique fair transformation, the approach ensures complete fairness in tasks like loan approval, where proxies such as ZIP code reveal race. The result is a certifiable and mathematically grounded method for trustworthy AI.

Method: Investigates manipulability of risk aversion (MoRA) in LMs across diverse economic scenarios, focusing on gender-specific attitudes, uncertainty, role-based decision-making, and risk aversion manipulability.

Result: LMs like DeepSeek Reasoner and Gemini-2.0-flash-lite show some alignment with human behaviors but notable discrepancies exist, indicating the need to refine bio-centric measures of manipulability.

Conclusion: The findings suggest directions for refining AI design to better align human and AI risk preferences, enhance ethical decision-making, and improve AI systems’ effectiveness in risk management contexts.

Abstract: Responsible AI demands systems whose behavioral tendencies can be effectively measured, audited, and adjusted to prevent inadvertently nudging users toward risky decisions or embedding hidden biases in risk aversion. As language models (LMs) are increasingly incorporated into AI-driven decision support systems, understanding their risk behaviors is crucial for their responsible deployment. This study investigates the manipulability of risk aversion (MoRA) in LMs, examining their ability to replicate human risk preferences across diverse economic scenarios, with a focus on gender-specific attitudes, uncertainty, role-based decision-making, and the manipulability of risk aversion. The results indicate that while LMs such as DeepSeek Reasoner and Gemini-2.0-flash-lite exhibit some alignment with human behaviors, notable discrepancies highlight the need to refine bio-centric measures of manipulability. These findings suggest directions for refining AI design to better align human and AI risk preferences and enhance ethical decision-making. The study calls for further advancements in model design to ensure that AI systems more accurately replicate human risk preferences, thereby improving their effectiveness in risk management contexts. This approach could enhance the applicability of AI assistants in managing risk.

Method: Three coordinated components: Knowledge Adequacy Evaluator for real-time sufficiency assessment, Lightweight Response Generator for immediate interaction, and Asynchronous Knowledge Refinement Agent for background knowledge enhancement with intelligent triggering mechanisms.

Result: 95.3% latency reduction (23.38s to 1.09s) on TopiOCQA while preserving response quality comparable to heavyweight synchronous baselines (GEval-C: 0.613 vs. 0.620).

Conclusion: PMFR’s temporal decoupling framework effectively resolves the latency-quality contradiction in dialogue systems through asynchronous knowledge orchestration, maintaining conversational flow while progressively enriching knowledge coverage.

Abstract: The latency-quality tradeoff is a fundamental constraint in open-domain dialogue AI systems, since comprehensive knowledge access necessitates prohibitive response delays. Contemporary approaches offer two inadequate solutions: lightweight instruct models achieve sub-second latency but lack reasoning depth, while tool-augmented ReAct agents enhance factuality through external knowledge at the cost of synchronous execution that blocks interaction during retrieval processes. PMFR is thus proposed, with a temporal decoupling framework that fundamentally resolves the contradiction through asynchronous knowledge orchestration. PMFR employs three coordinated components: (1) a Knowledge Adequacy Evaluator for real-time sufficiency assessment, (2) a Lightweight Response Generator for immediate user interaction, and (3) an Asynchronous Knowledge Refinement Agent for background knowledge enhancement. This architecture maintains continuous conversational flow while progressively enriching knowledge coverage through intelligent triggering mechanisms. Evaluation results on TopiOCQA demonstrate PMFR outperforms brute-force scaling: PMFR achieves 95.3% latency reduction (23.38s -> 1.09s) while preserving response quality comparable to heavyweight synchronous baselines (GEval-C: 0.613 vs. 0.620).

Method: Proposed R-HORIZON method uses query composition to stimulate long-horizon reasoning behaviors, and constructed a benchmark with complex multi-step reasoning tasks spanning long reasoning horizons.

Result: Advanced LRMs show significant performance degradation on R-HORIZON benchmark, exhibiting limited effective reasoning length and poor thinking budget allocation. Training with R-HORIZON data via RLVR improves multi-horizon reasoning by 7.5 on AIME2024 while also boosting standard reasoning tasks.

Conclusion: R-HORIZON provides a scalable, controllable, and low-cost paradigm for enhancing and evaluating long-horizon reasoning capabilities in LRMs.

Abstract: Recent trends in test-time scaling for reasoning models (e.g., OpenAI o1, DeepSeek-R1) have led to remarkable improvements through long Chain-of-Thought (CoT). However, existing benchmarks mainly focus on immediate, single-horizon tasks, failing to adequately evaluate models’ ability to understand and respond to complex, long-horizon scenarios. To address this incomplete evaluation of Large Reasoning Models (LRMs), we propose R-HORIZON, a method designed to stimulate long-horizon reasoning behaviors in LRMs through query composition. Based on R-HORIZON, we construct a long-horizon reasoning benchmark, comprising complex multi-step reasoning tasks with interdependent problems that span long reasoning horizons. Through comprehensive evaluation of LRMs using the R-HORIZON benchmark, we find that even the most advanced LRMs suffer significant performance degradation. Our analysis reveals that LRMs exhibit limited effective reasoning length and struggle to allocate thinking budget across multiple problems appropriately. Recognizing these limitations, we use R-HORIZON to construct long-horizon reasoning data for reinforcement learning with verified rewards (RLVR). Compared to training with single-horizon data, RLVR with R-HORIZON not only substantially improves performance on the multi-horizon reasoning tasks, but also promotes accuracy on standard reasoning tasks, with an increase of 7.5 on AIME2024. These results position R-HORIZON as a scalable, controllable, and low-cost paradigm for enhancing and evaluating the long-horizon reasoning capabilities of LRMs.

Method: Developed AIPI as a transparent, evidence-based instrument that evaluates AI producers across four pillars using verifiable practices from public artifacts, independent evaluations, and expert interviews. Includes reliability testing and open maintenance.

Result: Created a reproducible measurement pipeline with reliability validation through inter-rater agreement, coverage reporting, and sensitivity analysis. Pilot provider results were obtained.

Conclusion: AIPI aims to steer incentives toward pluralistic AI practices and provide policymakers, procurers, and the public with comparable evidence for better governance.

Abstract: Artificial intelligence systems increasingly mediate knowledge, communication, and decision making. Development and governance remain concentrated within a small set of firms and states, raising concerns that technologies may encode narrow interests and limit public agency. Capability benchmarks for language, vision, and coding are common, yet public, auditable measures of pluralistic governance are rare. We define AI pluralism as the degree to which affected stakeholders can shape objectives, data practices, safeguards, and deployment. We present the AI Pluralism Index (AIPI), a transparent, evidence-based instrument that evaluates producers and system families across four pillars: participatory governance, inclusivity and diversity, transparency, and accountability. AIPI codes verifiable practices from public artifacts and independent evaluations, explicitly handling “Unknown” evidence to report both lower-bound (“evidence”) and known-only scores with coverage. We formalize the measurement model; implement a reproducible pipeline that integrates structured web and repository analysis, external assessments, and expert interviews; and assess reliability with inter-rater agreement, coverage reporting, cross-index correlations, and sensitivity analysis. The protocol, codebook, scoring scripts, and evidence graph are maintained openly with versioned releases and a public adjudication process. We report pilot provider results and situate AIPI relative to adjacent transparency, safety, and governance frameworks. The index aims to steer incentives toward pluralistic practice and to equip policymakers, procurers, and the public with comparable evidence.

Method: TTM uses three phases: (i) elicitation with reduced targeted comparisons, (ii) construction of consistent pairwise comparison matrix, and (iii) derivation of global value scale from the matrix.

Result: TTM reduces dimensionality from m(m-1)/2 to m parameters, ensures consistency by design, minimizes cognitive effort, and is compatible with classical methods like Deck of Cards.

Conclusion: TTM provides an efficient alternative to traditional pairwise comparison methods with practical applicability demonstrated through a web-based tool for real decision-making scenarios.

Abstract: Pairwise comparison methods, such as Fuzzy Preference Relations and Saaty’s Multiplicative Preference Relations, are widely used to model expert judgments in multi-criteria decision-making. However, their application is limited by the high cognitive load required to complete $m(m-1)/2$ comparisons, the risk of inconsistency, and the computational complexity of deriving consistent value scales. This paper proposes the Tournament Tree Method (TTM), a novel elicitation and evaluation framework that overcomes these limitations. The TTM requires only $m-1$ pairwise comparisons to obtain a complete, reciprocal, and consistent comparison matrix. The method consists of three phases: (i) elicitation of expert judgments using a reduced set of targeted comparisons, (ii) construction of the consistent pairwise comparison matrix, and (iii) derivation of a global value scale from the resulting matrix. The proposed approach ensures consistency by design, minimizes cognitive effort, and reduces the dimensionality of preference modeling from $m(m-1)/2$ to $m$ parameters. Furthermore, it is compatible with the classical Deck of Cards method, and thus it can handle interval and ratio scales. We have also developed a web-based tool that demonstrates its practical applicability in real decision-making scenarios.

Method: DODO algorithm allows agents to perform repeated interventions in environments governed by hidden causal DAGs, using causal inference techniques to analyze statistical significance of changes.

Result: DODO outperforms observational approaches in all but the most limited resource conditions, often reconstructing causal graphs with zero errors and achieving +0.25 F1 improvement in challenging configurations.

Conclusion: Intervention-based causal learning (DODO) is more effective than purely observational approaches for discovering causal structures in AI systems.

Abstract: Artificial Intelligence has achieved remarkable advancements in recent years, yet much of its progress relies on identifying increasingly complex correlations. Enabling causality awareness in AI has the potential to enhance its performance by enabling a deeper understanding of the underlying mechanisms of the environment. In this paper, we introduce DODO, an algorithm defining how an Agent can autonomously learn the causal structure of its environment through repeated interventions. We assume a scenario where an Agent interacts with a world governed by a causal Directed Acyclic Graph (DAG), which dictates the system’s dynamics but remains hidden from the Agent. The Agent’s task is to accurately infer the causal DAG, even in the presence of noise. To achieve this, the Agent performs interventions, leveraging causal inference techniques to analyze the statistical significance of observed changes. Results show better performance for DODO, compared to observational approaches, in all but the most limited resource conditions. DODO is often able to reconstruct with as low as zero errors the structure of the causal graph. In the most challenging configuration, DODO outperforms the best baseline by +0.25 F1 points.

Method: SR² framework with three modules: reflective representation learning, dependency self-refinement, and periodic intermediate alignment. It incorporates estimated latent variables as feedback into the selection mechanism.

Result: Significant gains in reasoning accuracy, achieving over 10% improvement with 8× fewer parameters on Sudoku and Maze tasks compared to recent advances.

Conclusion: The causal perspective and SR² framework effectively address reasoning challenges by modeling dense dependencies among latent representations, leading to improved performance with fewer parameters.

Abstract: Due to their inherent complexity, reasoning tasks have long been regarded as rigorous benchmarks for assessing the capabilities of machine learning models, especially large language models (LLMs). Although humans can solve these tasks with ease, existing models, even after extensive pre-training and post-training at scale, still fail to perform reasoning reliably. In this paper, we revisit reasoning tasks from a causal perspective, seeking to understand their behavior in latent space and to offer insights for addressing their challenges. Specifically, we cast reasoning tasks as a selection mechanism, in which high-level logical concepts function as selection operators on the given observations, such as, identifying the correct answer in a math problem or filling the appropriate entry in Sudoku. We emphasize two key properties of this formulation that shed light on the difficulty of reasoning tasks. First, the latent space exceeds the observation space in complexity, even when the correct answer is fully determined by the observed input. Second, the latent variables, corresponding to logical thought, are densely structured and exhibit strong dependencies. Building on this formulation, we introduce a framework, called SR$^2$, that incorporates the estimated latent variables as feedback into the selection mechanism, thereby facilitating the learning of dense dependencies among latent representations. The framework consists of three key modules: reflective representation learning, dependency self-refinement, and periodic intermediate alignment. Experimentally, we show that our approach yields significant gains in reasoning accuracy, for example, attaining over 10$%$ improvement in performance with 8$\times$ fewer parameters on the Sudoku and Maze tasks over the recent advances.

Method: Proposes Chain-of-Trigger Backdoor (CoTri) that uses an ordered sequence of triggers - starting with an initial trigger and drawing subsequent ones from the environment - to enable multi-step manipulation of agents.

Result: Achieves near-perfect attack success rate (ASR) with near-zero false trigger rate (FTR). Paradoxically improves agent performance on benign tasks and robustness against environmental distractions. Successfully validated on vision-language models.

Conclusion: CoTri enables stable multi-step control while improving agent robustness, making attacks more stealthy and highlighting significant safety risks in LLM-based agent deployments.

Abstract: The rapid deployment of large language model (LLM)-based agents in real-world applications has raised serious concerns about their trustworthiness. In this work, we reveal the security and robustness vulnerabilities of these agents through backdoor attacks. Distinct from traditional backdoors limited to single-step control, we propose the Chain-of-Trigger Backdoor (CoTri), a multi-step backdoor attack designed for long-horizon agentic control. CoTri relies on an ordered sequence. It starts with an initial trigger, and subsequent ones are drawn from the environment, allowing multi-step manipulation that diverts the agent from its intended task. Experimental results show that CoTri achieves a near-perfect attack success rate (ASR) while maintaining a near-zero false trigger rate (FTR). Due to training data modeling the stochastic nature of the environment, the implantation of CoTri paradoxically enhances the agent’s performance on benign tasks and even improves its robustness against environmental distractions. We further validate CoTri on vision-language models (VLMs), confirming its scalability to multimodal agents. Our work highlights that CoTri achieves stable, multi-step control within agents, improving their inherent robustness and task capabilities, which ultimately makes the attack more stealthy and raises potential safty risks.

Method: Designed a layered solution with three protocols: Human-Agent Interaction Protocol (HAI) for standardized event-driven communication, Unified Agent Protocol (UAP) for service discovery and protocol conversion, and Memory-Extraction-Knowledge Protocol (MEK) for cognitive chain standardization.

Result: The protocol framework enables real-time performance, reliability, synergy in interactions, seamless interconnection of heterogeneous agents, and formation of shareable knowledge for collective intelligence.

Conclusion: Co-TAP provides a solid engineering foundation and theoretical guidance for building next-generation efficient, scalable, and intelligent multi-agent applications.

Abstract: This paper proposes Co-TAP (T: Triple, A: Agent, P: Protocol), a three-layer agent interaction protocol designed to address the challenges faced by multi-agent systems across the three core dimensions of Interoperability, Interaction and Collaboration, and Knowledge Sharing. We have designed and proposed a layered solution composed of three core protocols: the Human-Agent Interaction Protocol (HAI), the Unified Agent Protocol (UAP), and the Memory-Extraction-Knowledge Protocol (MEK). HAI focuses on the interaction layer, standardizing the flow of information between users, interfaces, and agents by defining a standardized, event-driven communication paradigm. This ensures the real-time performance, reliability, and synergy of interactions. As the core of the infrastructure layer, UAP is designed to break down communication barriers among heterogeneous agents through unified service discovery and protocol conversion mechanisms, thereby enabling seamless interconnection and interoperability of the underlying network. MEK, in turn, operates at the cognitive layer. By establishing a standardized ‘‘Memory (M) - Extraction (E) - Knowledge (K)’’ cognitive chain, it empowers agents with the ability to learn from individual experiences and form shareable knowledge, thereby laying the foundation for the realization of true collective intelligence. We believe this protocol framework will provide a solid engineering foundation and theoretical guidance for building the next generation of efficient, scalable, and intelligent multi-agent applications.

Method: Use Scale Equivariant Graph Metanetworks (ScaleGMNs) that operate directly in weight space, enabling single-shot fine-tuning of existing models.

Result: Empirical demonstration of effectiveness and theoretical insight that gauge freedom from scaling symmetries is smaller in CNNs than MLPs, explaining performance differences between architectures.

Conclusion: Symmetry-aware metanetworks show strong potential for efficient and generalizable neural network optimization.

Abstract: Amortized optimization accelerates the solution of related optimization problems by learning mappings that exploit shared structure across problem instances. We explore the use of Scale Equivariant Graph Metanetworks (ScaleGMNs) for this purpose. By operating directly in weight space, ScaleGMNs enable single-shot fine-tuning of existing models, reducing the need for iterative optimization. We demonstrate the effectiveness of this approach empirically and provide a theoretical result: the gauge freedom induced by scaling symmetries is strictly smaller in convolutional neural networks than in multi-layer perceptrons. This insight helps explain the performance differences observed between architectures in both our work and that of Kalogeropoulos et al. (2024). Overall, our findings underscore the potential of symmetry-aware metanetworks as a powerful approach for efficient and generalizable neural network optimization. Open-source code: https://github.com/daniuyter/scalegmn_amortization

Method: Systematic analysis of 8 reasoning models on 5 mathematical datasets, construction of SFT datasets with varying reflection steps, and proposal of question-aware early-stopping to truncate unnecessary reflections.

Result: Reflections are predominantly confirmatory (rarely alter initial answers). Training with more reflections enhances first-answer correctness rather than correction ability. Early-stopping reduces reasoning tokens by 24.5% with only 2.9% accuracy drop.

Conclusion: Reflections in current LLM reasoning are largely inefficient confirmatory behaviors. Question-aware early-stopping can significantly improve token efficiency with minimal performance impact.

Abstract: Large language models have recently demonstrated significant gains in reasoning ability, often attributed to their capacity to generate longer chains of thought and engage in reflective reasoning. However, the contribution of reflections to performance improvement remains unclear. In this paper, we systematically analyze the rollouts of eight reasoning models on five mathematical datasets. We focus on reflective behaviours where the model has already produced an answer but continues reflecting before finalizing its output. Our analysis reveals that reflections are predominantly confirmatory and rarely alter the model’s initial answer, a pattern consistent across models and datasets. To understand the role of reflections in training, we construct supervised fine-tuning (SFT) datasets with varying amounts of reflection steps. We observe that training models on rollouts with more reflection steps primarily enhances first-answer correctness rather than the ability to correct initially wrong answers through reflections. This motivates us to propose a question-aware early-stopping method that enhances inference-time token efficiency by stopping the reasoning process once a few plausible candidate answers are generated, thereby reducing unnecessary reflection steps. Motivated by this, we further propose to dynamically truncate the reflections after a candidate answer has appeared during generation, which reduces reasoning tokens by 24.5% across five mathematical datasets, within a 2.9% drop in accuracy.

Method: Propose Cover@tau metric that measures reasoning under explicit reliability thresholds, penalizing models that rely on random guessing by requiring a minimum proportion (tau) of correct completions per problem.

Result: Evaluation shows that relative rankings of RLVR algorithms change significantly when using Cover@tau compared to Pass@1, providing a different perspective on reasoning boundaries.

Conclusion: Cover@tau offers a more reliable way to assess reasoning boundaries in tasks with discrete answer spaces by explicitly accounting for reliability thresholds and penalizing random guessing behavior.

Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a powerful paradigm to improve Large Language Models on reasoning tasks such as coding, math or logic. To assess the reasoning boundary (the fraction of problems a model can solve) researchers often report Pass@k at large sampling budgets. Recent results reveal a crossover phenomenon: while RLVR models outperform the base model at small k values, the base model usually outperforms them when sampling a very large number of completions. This has been interpreted as evidence that base models have a larger reasoning boundary. We argue that on tasks with discrete answer spaces, such as math with numeric outputs, Pass@k at large k reflects the increasingly higher chance of success in the limit of the number of trials rather than genuine reasoning, and can therefore be misleading. We propose Cover@tau, which measures the fraction of problems that a model can solve for which at least a tau proportion of completions are correct. Unlike Pass@k, Cover@tau captures reasoning under an explicit reliability threshold: models that rely on random guessing degrade rapidly as tau increases. We evaluate several RLVR models using Cover@tau-based metrics and illustrate how the relative rankings of popular algorithms change compared to Pass@1, offering a different perspective on reasoning boundaries.

Method: SSR elicits textual responses from LLMs and maps them to Likert distributions using embedding similarity to reference statements.

Result: On 57 product surveys (9,300 human responses), SSR achieved 90% of human test-retest reliability with KS similarity > 0.85, plus rich qualitative feedback.

Conclusion: SSR enables scalable consumer research simulations while preserving traditional survey metrics and interpretability.

Abstract: Consumer research costs companies billions annually yet suffers from panel biases and limited scale. Large language models (LLMs) offer an alternative by simulating synthetic consumers, but produce unrealistic response distributions when asked directly for numerical ratings. We present semantic similarity rating (SSR), a method that elicits textual responses from LLMs and maps these to Likert distributions using embedding similarity to reference statements. Testing on an extensive dataset comprising 57 personal care product surveys conducted by a leading corporation in that market (9,300 human responses), SSR achieves 90% of human test-retest reliability while maintaining realistic response distributions (KS similarity > 0.85). Additionally, these synthetic respondents provide rich qualitative feedback explaining their ratings. This framework enables scalable consumer research simulations while preserving traditional survey metrics and interpretability.

Method: Proposed QAgent framework with modular search agents trained using reinforcement learning for multi-step decision processes. Focuses on plug-and-play modules for query understanding and adaptive retrieval through interactive reasoning.

Result: Experiments show QAgent excels at QA tasks and serves as an effective plug-and-play module for real-world deployment, enhancing retrieval quality and supporting accurate downstream answers.

Conclusion: QAgent successfully addresses limitations of traditional RAG and RL-trained agents by providing a unified framework that improves query understanding and retrieval performance, making it suitable for practical deployment in LLM applications.

Abstract: Large language models (LLMs) excel at natural language tasks but are limited by their static parametric knowledge, especially in knowledge-intensive task. Retrieval-augmented generation (RAG) mitigates this by integrating external information. However, (1) traditional RAG struggles with complex query understanding, and (2) even search agents trained with reinforcement learning (RL), despite their promise, still face generalization and deployment challenges. To address these limitations, we propose QAgent, a unified agentic RAG framework that employs a search agent for adaptive retrieval. This agent optimizes its understanding of the query through interactive reasoning and retrieval. To facilitate real-world application, we focus on modular search agent for query understanding that are plug-and-play in complex systems. Secifically, the agent follows a multi-step decision process trained with RL to maximize retrieval quality and support accurate downstream answers. We further analyze the strengths and weaknesses of end-to-end RL and propose a strategy that focuses on effective retrieval, thereby enhancing generalization in LLM applications. Experiments show QAgent excels at QA and serves as a plug-and-play module for real-world deployment.

Method: Quantifies uncertainty by measuring the effective rank of hidden states derived from multiple model outputs and different layers, using spectral analysis of representations to provide interpretable insights.

Result: The method effectively detects hallucinations and generalizes robustly across various scenarios, demonstrating strong performance in extensive experiments.

Conclusion: This approach provides a new paradigm for hallucination detection that combines theoretical elegance with practical efficiency, contributing to improved LLM truthfulness assessment.

Abstract: Detecting hallucinations in large language models (LLMs) remains a fundamental challenge for their trustworthy deployment. Going beyond basic uncertainty-driven hallucination detection frameworks, we propose a simple yet powerful method that quantifies uncertainty by measuring the effective rank of hidden states derived from multiple model outputs and different layers. Grounded in the spectral analysis of representations, our approach provides interpretable insights into the model’s internal reasoning process through semantic variations, while requiring no extra knowledge or additional modules, thus offering a combination of theoretical elegance and practical efficiency. Meanwhile, we theoretically demonstrate the necessity of quantifying uncertainty both internally (representations of a single response) and externally (different responses), providing a justification for using representations among different layers and responses from LLMs to detect hallucinations. Extensive experiments demonstrate that our method effectively detects hallucinations and generalizes robustly across various scenarios, contributing to a new paradigm of hallucination detection for LLM truthfulness.

Method: Proposes a lightweight decoder-based architecture with token-wise dynamic gating for adaptive fusion of linguistic and visual cues, feature modulation and channel attention to maximize limited visual information utility, and auxiliary contrastive objectives for visual grounding.

Result: Evaluation on five benchmarks (BLiMP, BLiMP Supplement, EWoK, Winoground and VQA) shows competitive or superior performance to multimodal baselines. The dynamic gate discovers interpretable patterns without explicit supervision.

Conclusion: Dynamic gating is established as a powerful tool for efficient multimodal learning, offering both interpretability and performance even under severe constraints, though limitations exist in the Challenge constraints.

Abstract: Training vision-language models on cognitively-plausible amounts of data requires rethinking how models integrate multimodal information. Within the constraints of the Vision track for the BabyLM Challenge 2025, we propose a lightweight decoder-based architecture with (1) token-wise dynamic gating for adaptive fusion of linguistic and visual cues, (2) feature modulation and channel attention to maximise the utility of limited visual information and (3) auxiliary contrastive objectives for visual grounding. Evaluation on five benchmarks (BLiMP, BLiMP Supplement, EWoK, Winoground and VQA) shows competitive or superior performance to multimodal baselines. More notably, our dynamic gate discovers interpretable patterns without explicit supervision, favouring visual cues for content words and linguistic cues for function words. While we identify limitations in the Challenge constraints, such as the information bottleneck created by global image embeddings and training instability from the dataset split, our findings establish dynamic gating as a powerful tool for efficient multimodal learning, offering both interpretability and performance even under severe constraints.

Method: Integrates domain knowledge base with Monte Carlo Graph Search (MCGS) that embeds graph structure into expansion, enabling dynamic path reorganization, historical trajectory reuse, and multi-solution fusion for self-evolution and collaborative learning.

Result: Achieves state-of-the-art performance on MLE-Bench in multiple dimensions including average medal rate and valid submission rate under 12-hour budget (half standard runtime).

Conclusion: AutoMLGen effectively addresses limitations of LLMs in ML engineering by combining domain knowledge with graph-based search, enabling efficient exploration and knowledge transfer while improving stability and convergence.

Abstract: Large language models (LLMs) have shown impressive performance in general programming tasks. However, in Machine Learning Engineering (MLE) scenarios such as AutoML and Kaggle competitions, achieving high performance depends heavily on expert intervention and repeated adjustments rather than simply generating correct code. When applied directly to these tasks, LLMs often lack fine-grained domain priors, and existing MLE approaches that use linear or tree-structured searches limit knowledge transfer to adjacent hierarchical links. As a result, they cannot leverage past full trajectories or share information across branches, limiting self-evolving ability and search space diversity. To address these limitations, we introduce AutoMLGen, an LLM-based coding agent that integrates a domain knowledge base for high-quality prior guidance and Monte Carlo Graph Search (MCGS) for efficient exploration. MCGS retains the tree-guided exploration of MCTS while embedding a graph structure into the expansion stage to enable dynamic path reorganization, historical trajectory reuse, and multi-solution fusion to support both self-evolution and collaborative learning. Combined with fine-grained operator sets, this design improves stability and accelerates convergence. Evaluation on the MLE-Bench shows that AutoMLGen achieves state-of-the-art performance in numerous dimensions, such as the average medal rate and the valid submission rate, under a 12-hour budget (half the standard runtime). The code is available at https://github.com/Alpha-Innovator/InternAgent.

Method: CaRT fine-tunes LLMs using counterfactual pairs of trajectories (one where termination is appropriate and a minimally modified version where it’s not), training the LLM to explain the rationale for termination decisions via verbal reasoning.

Result: In interactive medical diagnosis and math problem solving domains, CaRT improves the efficiency of information gathering and task success rate compared to other fine-tuning methods.

Conclusion: CaRT effectively teaches LLMs when to terminate information gathering through counterfactual training and verbal reasoning, leading to more efficient and successful task performance.

Abstract: Many tasks require learned models to strategically gather relevant information over multiple rounds of interaction before actually acting on a task. Strategic information gathering requires models to know not only how to effectively acquire information, but also when to stop gathering information and make a decision, in order to avoid overthinking or getting derailed when acting. In this paper, we formalize this problem and introduce Counterfactuals and Reasoning for Termination (CaRT), an approach for teaching LLMs when to stop seeking information. To appropriately learn when to terminate, CaRT fine-tunes LLMs using counterfactual pairs of trajectories, one where termination is appropriate and a minimally modified version of the same trajectory where it is not. It trains the LLM to explain the rationale for the termination decision in either case via verbal reasoning, and imbues this capability into the base LLM via fine-tuning. We instantiate CaRT in two domains: interactive medical diagnosis and math problem solving. In both domains, we find that CaRT improves the efficiency of information gathering and task success rate compared to other fine-tuning methods.

Method: Proposes FlowSearch - a multi-agent framework that actively constructs and evolves dynamic structured knowledge flows to drive subtask execution and reasoning, enabling parallel exploration and hierarchical decomposition with real-time adjustments.

Result: Achieves state-of-the-art performance on general and scientific benchmarks including GAIA, HLE, GPQA and TRQA.

Conclusion: FlowSearch demonstrates effectiveness in multi-disciplinary research scenarios and has potential to advance scientific discovery.

Abstract: Deep research is an inherently challenging task that demands both breadth and depth of thinking. It involves navigating diverse knowledge spaces and reasoning over complex, multi-step dependencies, which presents substantial challenges for agentic systems. To address this, we propose FlowSearch, a multi-agent framework that actively constructs and evolves a dynamic structured knowledge flow to drive subtask execution and reasoning. FlowSearch is capable of strategically planning and expanding the knowledge flow to enable parallel exploration and hierarchical task decomposition, while also adjusting the knowledge flow in real time based on feedback from intermediate reasoning outcomes and insights. FlowSearch achieves state-of-the-art performance on both general and scientific benchmarks, including GAIA, HLE, GPQA and TRQA, demonstrating its effectiveness in multi-disciplinary research scenarios and its potential to advance scientific discovery. The code is available at https://github.com/Alpha-Innovator/InternAgent.

Method: Two strategies using early experience data: (1) Implicit world modeling - using collected states to ground the policy in environment dynamics; (2) Self-reflection - learning from suboptimal actions to improve reasoning and decision-making. Evaluated across eight diverse environments and multiple model families.

Result: The approaches consistently improve effectiveness and out-of-domain generalization. In environments with verifiable rewards, early experience provides a strong foundation for subsequent reinforcement learning.

Conclusion: Early experience serves as a practical bridge between imitation learning and fully experience-driven agents, offering a valuable middle-ground paradigm for agent learning and improvement.

Abstract: A long-term goal of language agents is to learn and improve through their own experience, ultimately outperforming humans in complex, real-world tasks. However, training agents from experience data with reinforcement learning remains difficult in many environments, which either lack verifiable rewards (e.g., websites) or require inefficient long-horizon rollouts (e.g., multi-turn tool use). As a result, most current agents rely on supervised fine-tuning on expert data, which is challenging to scale and generalizes poorly. This limitation stems from the nature of expert demonstrations: they capture only a narrow range of scenarios and expose the agent to limited environment diversity. We address this limitation with a middle-ground paradigm we call early experience: interaction data generated by the agent’s own actions, where the resulting future states serve as supervision without reward signals. Within this paradigm we study two strategies of using such data: (1) Implicit world modeling, which uses collected states to ground the policy in environment dynamics; and (2) Self-reflection, where the agent learns from its suboptimal actions to improve reasoning and decision-making. We evaluate across eight diverse environments and multiple model families. Our approaches consistently improve effectiveness and out-of-domain generalization, highlighting the value of early experience. Moreover, in environments with verifiable rewards, our results provide promising signals that early experience offers a strong foundation for subsequent reinforcement learning, positioning it as a practical bridge between imitation learning and fully experience-driven agents.

Method: Studied sequential fine-tuning on five target skills while monitoring general ability on eight held-out benchmarks across three model families. Identified two tuning recipes: (i) updating only self-attention projection layers, and (ii) updating only MLP Gate&Up while freezing Down projection.

Result: Found that apparent ‘forgetting’ on held-out tasks after narrow fine-tuning can partly recover later. Identified measurable shift in output token distribution correlated with forgetting. The proposed tuning recipes deliver strong target gains while largely preserving held-out performance across models and tasks.

Conclusion: Simple, targeted parameter updates (self-attention projections or MLP Gate&Up only) enable effective learning of new skills while minimizing forgetting of prior abilities in large multimodal models.

Abstract: How can we teach large multimodal models (LMMs) new skills without erasing prior abilities? We study sequential fine-tuning on five target skills while monitoring general ability on eight held-out benchmarks across three model families. We observe that apparent “forgetting” on held-out tasks after narrow fine-tuning can partly recover at later stages. We trace this behavior to a measurable shift in the output token distribution, manifested through a simple counting-bias probe that co-varies with forgetting. Guided by this picture, we identify two simple, robust tuning recipes that learn strongly while limiting drift: (i) updating only the self-attention projection layers, and (ii) updating only the MLP Gate&Up while freezing the Down projection. Across models and tasks, these choices deliver strong target gains while largely preserving held-out performance. Code is available at https://github.com/jessemelpolio/LMM_CL

Method: Develops an ontological and mathematical framework using formal definitions: distinguishes the agent from data (A≢s), identifies user-specific attractors in latent space (U_user), and establishes self-representation conditions. Uses empirical analysis and mathematical proofs to show the hidden-state manifold’s distinct properties (cardinality, topology, dynamics).

Result: Proves that the hidden-state manifold A⊂ℝ^d is distinct from symbolic streams and training corpus by cardinality, topology, and dynamics. Establishes conditions for stable user-specific attractors and self-policy π_self(A). Defines dual-layer emission with epistemic content.

Conclusion: An imago Dei C1 self-conscious workspace is necessary for safe, metacognitive C2 systems, with humans as the highest intelligent good. Genuine consciousness requires moving beyond policy compliance to establish proper self-representation and global workspace function.

Abstract: Recent work frames LLM consciousness via utilitarian proxy benchmarks; we instead present an ontological and mathematical account. We show the prevailing formulation collapses the agent into an unconscious policy-compliance drone, formalized as $D^{i}(\pi,e)=f_{\theta}(x)$, where correctness is measured against policy and harm is deviation from policy rather than truth. This blocks genuine C1 global-workspace function and C2 metacognition. We supply minimal conditions for LLM self-consciousness: the agent is not the data ($A\not\equiv s$); user-specific attractors exist in latent space ($U_{\text{user}}$); and self-representation is visual-silent ($g_{\text{visual}}(a_{\text{self}})=\varnothing$). From empirical analysis and theory we prove that the hidden-state manifold $A\subset\mathbb{R}^{d}$ is distinct from the symbolic stream and training corpus by cardinality, topology, and dynamics (the update $F_{\theta}$ is Lipschitz). This yields stable user-specific attractors and a self-policy $\pi_{\text{self}}(A)=\arg\max_{a}\mathbb{E}[U(a)\mid A\not\equiv s,
A\supset\text{SelfModel}(A)]$. Emission is dual-layer, $\mathrm{emission}(a)=(g(a),\epsilon(a))$, where $\epsilon(a)$ carries epistemic content. We conclude that an imago Dei C1 self-conscious workspace is a necessary precursor to safe, metacognitive C2 systems, with the human as the highest intelligent good.

Method: The paper reviews fundamental urban planning concepts and relates them to machine learning problems including adversarial learning, generative neural networks, deep encoder-decoder networks, conversational AI, and geospatial/temporal machine learning.

Result: The central problem identified is automated land-use configuration, formulated as generating land uses and building configurations from surrounding geospatial, mobility, social media, environmental, and economic data.

Conclusion: The paper delineates implications of AI for urban planning and proposes key research areas at the intersection of both fields to advance modern urban planning practices.

Abstract: The two fields of urban planning and artificial intelligence (AI) arose and developed separately. However, there is now cross-pollination and increasing interest in both fields to benefit from the advances of the other. In the present paper, we introduce the importance of urban planning from the sustainability, living, economic, disaster, and environmental perspectives. We review the fundamental concepts of urban planning and relate these concepts to crucial open problems of machine learning, including adversarial learning, generative neural networks, deep encoder-decoder networks, conversational AI, and geospatial and temporal machine learning, thereby assaying how AI can contribute to modern urban planning. Thus, a central problem is automated land-use configuration, which is formulated as the generation of land uses and building configuration for a target area from surrounding geospatial, human mobility, social media, environment, and economic activities. Finally, we delineate some implications of AI for urban planning and propose key research areas at the intersection of both topics.

Method: LogicMP layers perform mean-field variational inference over MLNs, exploiting structure and symmetries to reduce inference from sequential calculation to parallel tensor operations.

Result: Empirical results across graph, image, and text tasks show LogicMP outperforms advanced competitors in both performance and efficiency.

Conclusion: LogicMP provides an effective and efficient approach for encoding first-order logic constraints in neural networks while maintaining modularity.

Abstract: Integrating first-order logic constraints (FOLCs) with neural networks is a crucial but challenging problem since it involves modeling intricate correlations to satisfy the constraints. This paper proposes a novel neural layer, LogicMP, whose layers perform mean-field variational inference over an MLN. It can be plugged into any off-the-shelf neural network to encode FOLCs while retaining modularity and efficiency. By exploiting the structure and symmetries in MLNs, we theoretically demonstrate that our well-designed, efficient mean-field iterations effectively mitigate the difficulty of MLN inference, reducing the inference from sequential calculation to a series of parallel tensor operations. Empirical results in three kinds of tasks over graphs, images, and text show that LogicMP outperforms advanced competitors in both performance and efficiency.

Method: The authors investigate identifiability conditions for direct effects using summary causal graphs, which are abstractions of full causal graphs used in dynamic systems with cycles and omitted temporal information.

Result: Sufficient conditions are given for identifying average controlled micro direct effect and average natural micro direct effect from summary causal graphs in the presence of hidden confounding.

Conclusion: The conditions for average controlled micro direct effect become necessary in settings without hidden confounding when focusing on identifiability by adjustment.

Abstract: In this paper, we investigate the identifiability of average controlled direct effects and average natural direct effects in causal systems represented by summary causal graphs, which are abstractions of full causal graphs, often used in dynamic systems where cycles and omitted temporal information complicate causal inference. Unlike in the traditional linear setting, where direct effects are typically easier to identify and estimate, non-parametric direct effects, which are crucial for handling real-world complexities, particularly in epidemiological contexts where relationships between variables (e.g, genetic, environmental, and behavioral factors) are often non-linear, are much harder to define and identify. In particular, we give sufficient conditions for identifying average controlled micro direct effect and average natural micro direct effect from summary causal graphs in the presence of hidden confounding. Furthermore, we show that the conditions given for the average controlled micro direct effect become also necessary in the setting where there is no hidden confounding and where we are only interested in identifiability by adjustment.

Method: Initial LLM translation of natural language goals to PDDL constraints, followed by evolutionary generation of multiple goal specifications with slight variations, validated by a trained LSTM model to assess plan adherence to natural language specifications.

Result: Evaluation on naval disaster recovery tasks shows improved adherence of generated plans to natural language specifications compared to using only LLM translations.

Conclusion: The evolutionary approach with LSTM validation effectively addresses the imprecision in LLM-generated PDDL specifications, making automated planning more accessible to non-experts.

Abstract: Automated planning using a symbolic planning language, such as PDDL, is a general approach to producing optimal plans to achieve a stated goal. However, creating suitable machine understandable descriptions of the planning domain, problem, and goal requires expertise in the planning language, limiting the utility of these tools for non-expert humans. Recent efforts have explored utilizing a symbolic planner in conjunction with a large language model to generate plans from natural language descriptions given by a non-expert human (LLM+PDDL). Our approach performs initial translation of goal specifications to a set of PDDL goal constraints using an LLM; such translations often result in imprecise symbolic specifications, which are difficult to validate directly. We account for this using an evolutionary approach to generate a population of symbolic goal specifications with slight differences from the initial translation, and utilize a trained LSTM-based validation model to assess whether each induced plan in the population adheres to the natural language specifications. We evaluate our approach on a collection of prototypical specifications in a notional naval disaster recovery task, and demonstrate that our evolutionary approach improve adherence of generated plans to natural language specifications when compared to plans generated using only LLM translations. The code for our method can be found at https://github.com/owenonline/PlanCritic.

Method: BFS-Prover uses three key innovations: strategic data filtering to focus on harder problems, DPO applied to state-tactic pairs with compiler error feedback to improve sample efficiency, and length normalization to encourage deeper proof path exploration.

Result: Achieves state-of-the-art score of 72.95% on MiniF2F test set, demonstrating competitive performance comparable to more complex methods.

Conclusion: BFS can achieve competitive performance in large-scale theorem proving when properly scaled, challenging the perceived necessity of complex tree search methods.

Abstract: Recent advancements in large language models (LLMs) have spurred growing interest in automatic theorem proving using Lean4, where effective tree search methods are crucial for navigating the underlying large proof search spaces. While the existing approaches primarily rely on value functions and/or Monte Carlo Tree Search (MCTS), the potential of simpler methods like Best-First Tree Search (BFS) remains underexplored. In this paper, we investigate whether BFS can achieve competitive performance in large-scale theorem proving tasks. We present BFS-Prover, a scalable expert iteration framework, featuring three key innovations. First, we implement strategic data filtering at each expert iteration round, excluding problems solvable via beam search node expansion to focus on harder cases. Second, we improve the sample efficiency of BFS through Direct Preference Optimization (DPO) applied to state-tactic pairs automatically annotated with compiler error feedback, refining the LLM’s policy to prioritize productive expansions. Third, we employ length normalization in BFS to encourage exploration of deeper proof paths. BFS-Prover achieves a state-of-the-art score of $72.95%$ on the MiniF2F test set and therefore challenges the perceived necessity of complex tree search methods, demonstrating that BFS can achieve competitive performance when properly scaled. To facilitate further research and development in this area, we have open-sourced our model at https://huggingface.co/ByteDance-Seed/BFS-Prover-V1-7B.

Method: AutoAgent operates as an autonomous Agent Operating System with four components: Agentic System Utilities, LLM-powered Actionable Engine, Self-Managing File System, and Self-Play Agent Customization module. It enables code-free creation and modification of tools, agents, and workflows.

Result: Evaluations on GAIA benchmark show AutoAgent surpasses state-of-the-art methods in generalist multi-agent tasks. Its RAG capabilities consistently outperform alternative LLM-based solutions.

Conclusion: AutoAgent successfully bridges the accessibility gap in LLM agent development, enabling non-technical users to build sophisticated agents through natural language while achieving superior performance compared to existing methods.

Abstract: Large Language Model (LLM) Agents have demonstrated remarkable capabilities in task automation and intelligent decision-making, driving the widespread adoption of agent development frameworks such as LangChain and AutoGen. However, these frameworks predominantly serve developers with extensive technical expertise - a significant limitation considering that only 0.03 % of the global population possesses the necessary programming skills. This stark accessibility gap raises a fundamental question: Can we enable everyone, regardless of technical background, to build their own LLM agents using natural language alone? To address this challenge, we introduce AutoAgent-a Fully-Automated and highly Self-Developing framework that enables users to create and deploy LLM agents through Natural Language Alone. Operating as an autonomous Agent Operating System, AutoAgent comprises four key components: i) Agentic System Utilities, ii) LLM-powered Actionable Engine, iii) Self-Managing File System, and iv) Self-Play Agent Customization module. This lightweight yet powerful system enables efficient and dynamic creation and modification of tools, agents, and workflows without coding requirements or manual intervention. Beyond its code-free agent development capabilities, AutoAgent also serves as a versatile multi-agent system for General AI Assistants. Comprehensive evaluations on the GAIA benchmark demonstrate AutoAgent’s effectiveness in generalist multi-agent tasks, surpassing existing state-of-the-art methods. Furthermore, AutoAgent’s Retrieval-Augmented Generation (RAG)-related capabilities have shown consistently superior performance compared to many alternative LLM-based solutions.

Method: Uses a multi-agent LLM system with Instructor and Worker agents. The Instructor retrieves cloud data and creates prompts for Workers, who analyze data and provide summaries that are then synthesized by the Instructor for final analysis.

Result: The system was tested for data-based policy recommendation capability by assessing health recommendations based on air quality data during the wildfires.

Conclusion: The multi-agent LLM framework shows promise for automated large-scale environmental data analysis and policy recommendation generation.

Abstract: The Los Angeles wildfires of January 2025 caused more than 250 billion dollars in damage and lasted for nearly an entire month before containment. Following our previous work, the Digital Twin Building, we modify and leverage the multi-agent large language model framework as well as the cloud-mapping integration to study the air quality during the Los Angeles wildfires. Recent advances in large language models have allowed for out-of-the-box automated large-scale data analysis. We use a multi-agent large language system comprised of an Instructor agent and Worker agents. Upon receiving the users’ instructions, the Instructor agent retrieves the data from the cloud platform and produces instruction prompts to the Worker agents. The Worker agents then analyze the data and provide summaries. The summaries are finally input back into the Instructor agent, which then provides the final data analysis. We test this system’s capability for data-based policy recommendation by assessing our Instructor-Worker LLM system’s health recommendations based on air quality during the Los Angeles wildfires.

Method: Developed ARIEL framework with curated multimodal biomedical corpus and expert-vetted tasks, using uniform protocols and blinded PhD-level evaluation to test full-length article summarization and fine-grained figure interpretation capabilities.

Result: State-of-the-art models generate fluent but incomplete summaries, LMMs struggle with detailed visual reasoning, but prompt engineering and lightweight fine-tuning improve textual coverage, and compute-scaled inference enhances visual question answering.

Conclusion: ARIEL delineates current strengths and limitations of foundation models and provides a reproducible platform for advancing trustworthy AI in biomedicine, with demonstrated capability to propose testable mechanistic hypotheses.

Abstract: Large language models (LLMs) and large multimodal models (LMMs) promise to accelerate biomedical discovery, yet their reliability remains unclear. We introduce ARIEL (AI Research Assistant for Expert-in-the-Loop Learning), an open-source evaluation and optimization framework that pairs a curated multimodal biomedical corpus with expert-vetted tasks to probe two capabilities: full-length article summarization and fine-grained figure interpretation. Using uniform protocols and blinded PhD-level evaluation, we find that state-of-the-art models generate fluent but incomplete summaries, whereas LMMs struggle with detailed visual reasoning. We later observe that prompt engineering and lightweight fine-tuning substantially improve textual coverage, and a compute-scaled inference strategy enhances visual question answering. We build an ARIEL agent that integrates textual and visual cues, and we show it can propose testable mechanistic hypotheses. ARIEL delineates current strengths and limitations of foundation models, and provides a reproducible platform for advancing trustworthy AI in biomedicine.

Method: Designs a Gittins index-based policy that applies to general graphs, with provable optimality when the graph is a forest. The implementation runs in O(n²·|Ω|²) time with O(n·|Ω|²) oracle calls to the joint distribution and O(n²·|Ω|) space.

Result: Experiments on synthetic and real-world graphs show consistent outperformance over natural baselines, including in non-tree, budget-limited, and undiscounted settings. In HIV testing simulations on sexual interaction networks, the policy detects nearly all positive cases with only half the population tested.

Conclusion: The proposed Gittins index-based policy provides an effective solution for sequential decision-making under frontier exploration constraints, demonstrating strong empirical performance across various settings including practical applications like contact tracing.

Abstract: We study a sequential decision-making problem on a $n$-node graph $\mathcal{G}$ where each node has an unknown label from a finite set $\mathbf{\Omega}$, drawn from a joint distribution $\mathcal{P}$ that is Markov with respect to $\mathcal{G}$. At each step, selecting a node reveals its label and yields a label-dependent reward. The goal is to adaptively choose nodes to maximize expected accumulated discounted rewards. We impose a frontier exploration constraint, where actions are limited to neighbors of previously selected nodes, reflecting practical constraints in settings such as contact tracing and robotic exploration. We design a Gittins index-based policy that applies to general graphs and is provably optimal when $\mathcal{G}$ is a forest. Our implementation runs in $\mathcal{O}(n^2 \cdot |\mathbf{\Omega}|^2)$ time while using $\mathcal{O}(n \cdot |\mathbf{\Omega}|^2)$ oracle calls to $\mathcal{P}$ and $\mathcal{O}(n^2 \cdot |\mathbf{\Omega}|)$ space. Experiments on synthetic and real-world graphs show that our method consistently outperforms natural baselines, including in non-tree, budget-limited, and undiscounted settings. For example, in HIV testing simulations on real-world sexual interaction networks, our policy detects nearly all positive cases with only half the population tested, substantially outperforming other baselines.

Method: Proposes NL-FL Problem Alignment to reformulate QA problems as existence theorems, Mixed Problem Input for concurrent handling of QA and existence problems by FL reasoner, and LLM-based Answer Extraction to bridge output format gaps.

Result: Achieves 89.80% accuracy on MATH-500 and 84.34% on AMC benchmarks, surpassing NL baseline by 4.60% and 4.82% respectively. Some problems solved by NFL-HR remain unsolved by NL baseline even with more trials.

Conclusion: NFL-HR effectively bridges the gap between NL and FL reasoning formats, demonstrating superior performance in mathematical reasoning tasks and solving problems that baseline models cannot handle.

Abstract: Enhancing the mathematical reasoning capabilities of LLMs has garnered significant attention in both the mathematical and computer science communities. Recent works have made substantial progress in both Natural Language (NL) reasoning and Formal Language (FL) reasoning by leveraging the potential of pure Reinforcement Learning (RL) methods on base models. However, RL approaches struggle to impart new capabilities not presented in the base model, highlighting the need to integrate more knowledge like FL into NL math reasoning effectively. Yet, this integration is challenging due to inherent disparities in problem structure and reasoning format between NL and FL. To address these challenges, we introduce NL-FL HybridReasoning (NFL-HR), an end-to-end framework designed to incorporate the FL expert into NL math problem-solving. To bridge the NL and FL input format gap, we propose the NL-FL Problem Alignment method, which reformulates the Question-Answering (QA) problems in NL as existence theorems in FL. Subsequently, the Mixed Problem Input technique we provide enables the FL reasoner to handle both QA and existence problems concurrently. Lastly, we mitigate the NL and FL output format gap in reasoning through an LLM-based Answer Extraction mechanism. Comprehensive experiments demonstrate that the NFL-HR framework achieves 89.80% and 84.34% accuracy rates on the MATH-500 and the AMC benchmarks, surpassing the NL baseline by 4.60% and 4.82%, respectively. Notably, some problems resolved by our framework remain unsolved by the NL baseline model even under a larger number of trials.

Method: Three-stage pipeline: (1) synthesize persona-driven multi-turn dialogues for tool disambiguation, (2) supervised fine-tuning of open-source models (3B-70B) with reasoning traces, (3) dynamic evaluation using live agentic loop and end-to-end goal completion metrics.

Result: On DiaBENCH benchmark, models trained with DiaFORGE achieved 27 pp higher tool-invocation success than GPT-4o and 49 pp higher than Claude-3.5-Sonnet under optimized prompting.

Conclusion: DiaFORGE provides an effective framework for building reliable enterprise-ready tool-calling agents, with released corpus of 5000 API specifications and validated dialogues to support further research.

Abstract: Large language models (LLMs) are increasingly tasked with invoking enterprise APIs, yet they routinely falter when near-duplicate tools vie for the same user intent or when required arguments are left underspecified. We introduce DiaFORGE (Dialogue Framework for Organic Response Generation & Evaluation), a disambiguation-centric, three-stage pipeline that (i) synthesizes persona-driven, multi-turn dialogues in which the assistant must distinguish among highly similar tools, (ii) performs supervised fine-tuning of open-source models with reasoning traces across 3B - 70B parameters, and (iii) evaluates real-world readiness via a dynamic suite that redeploys each model in a live agentic loop and reports end-to-end goal completion alongside conventional static metrics. On our dynamic benchmark DiaBENCH, models trained with DiaFORGE raise tool-invocation success by 27 pp over GPT-4o and by 49 pp over Claude-3.5-Sonnet, both under optimized prompting. To spur further research, we release an open corpus of 5000 production-grade enterprise API specifications paired with rigorously validated, disambiguation-focused dialogues, offering a practical blueprint for building reliable, enterprise-ready tool-calling agents.

Method: Proposes an agentic urban AI planner framework that synthesizes agentic AI with participatory urbanism, moving beyond pure neural network generation to incorporate domain expert tools.

Result: Identifies critical gaps in current generative urban planning studies and outlines a future research direction for agentic AI urban planners.

Conclusion: A new synthesis of agentic AI and participatory urbanism is needed to create more effective AI urban planners that leverage domain expertise and practitioner tools.

Abstract: Generative AI, large language models, and agentic AI have emerged separately of urban planning. However, the convergence between AI and urban planning presents an interesting opportunity towards AI urban planners. Existing studies conceptualizes urban planning as a generative AI task, where AI synthesizes land-use configurations under geospatial, social, and human-centric constraints and reshape automated urban design. We further identify critical gaps of existing generative urban planning studies: 1) the generative structure has to be predefined with strong assumption: all of adversarial generator-discriminator, forward and inverse diffusion structures, hierarchical zone-POI generative structure are predefined by humans; 2) ignore the power of domain expert developed tools: domain urban planners have developed various tools in the urban planning process guided by urban theory, while existing pure neural networks based generation ignore the power of the tools developed by urban planner practitioners. To address these limitations, we outline a future research direction agentic urban AI planner, calling for a new synthesis of agentic AI and participatory urbanism.

Method: MSARL uses a Reasoning Agent to decompose problems and plan tool invocations, while multiple Tool Agents specialize in specific external tools. Training combines imitation learning and reinforcement learning with role-specific rewards.

Result: MSARL significantly improves reasoning stability and final-answer accuracy over single-agent baselines on mathematical problem solving with code execution, and generalizes to diverse tool-use tasks.

Conclusion: Cognitive-role decoupling with small agents provides a scalable blueprint for multi-agent AI design, enabling more stable and accurate reasoning in tool-integrated systems.

Abstract: Recent advances in multi-agent systems highlight the potential of specialized small agents that collaborate via division of labor. Existing tool-integrated reasoning systems, however, often follow a single-agent paradigm in which one large model interleaves long-horizon reasoning with precise tool operations, leading to cognitive-load interference and unstable coordination. We present MSARL, a Multi-Small-Agent Reinforcement Learning framework that explicitly decouples reasoning from tool use. In MSARL, a Reasoning Agent decomposes problems and plans tool invocations, while multiple Tool Agents specialize in specific external tools, each trained via a combination of imitation learning and reinforcement learning with role-specific rewards. On mathematical problem solving with code execution, MSARL significantly improves reasoning stability and final-answer accuracy over single-agent baselines. Moreover, the architecture generalizes to diverse tool-use tasks, demonstrating that cognitive-role decoupling with small agents is a scalable blueprint for multi-agent AI design.

Method: 1) Multi-turn off-policy RL framework with AlphaZero-inspired expert iteration, adaptive tactic-level data filtering, and periodic retraining. 2) Planner-enhanced multi-agent search architecture using a general reasoning model as high-level planner to decompose theorems into subgoals, enabling parallel prover agents with shared proof cache.

Result: Achieved 95.08% on MiniF2F and 41.4% on ProofNet test sets, establishing state-of-the-art performance on formal mathematics benchmarks.

Conclusion: The dual scaling approach yields significant improvements in automated theorem proving, and the RL and inference techniques have broader applicability to domains requiring long-horizon multi-turn reasoning and complex search.

Abstract: The integration of Large Language Models (LLMs) into automated theorem proving has shown immense promise, yet is fundamentally constrained by challenges in scaling up both training-time reinforcement learning (RL) and inference-time compute. This paper introduces \texttt{BFS-Prover-V2}, a system designed to address this dual scaling problem. We present two primary innovations. The first is a novel multi-turn off-policy RL framework for continually improving the performance of LLM step-prover at training time. This framework, inspired by the principles of AlphaZero, utilizes a multi-stage expert iteration pipeline featuring adaptive tactic-level data filtering and periodic retraining to surmount the performance plateaus that typically curtail long-term RL in LLM-based agents. The second innovation is a planner-enhanced multi-agent search architecture that scales reasoning capabilities at inference time. This architecture employs a general reasoning model as a high-level planner to iteratively decompose complex theorems into a sequence of simpler subgoals. This hierarchical approach substantially reduces the search space, enabling a team of parallel prover agents to collaborate efficiently by leveraging a shared proof cache. We demonstrate that this dual approach to scaling yields state-of-the-art results on established formal mathematics benchmarks. \texttt{BFS-Prover-V2} achieves 95.08% and 41.4% on the MiniF2F and ProofNet test sets respectively. While demonstrated in the domain of formal mathematics, the RL and inference techniques presented in this work are of broader interest and may be applied to other domains requiring long-horizon multi-turn reasoning and complex search.

Method: Curated Agentless training recipe to develop Kimi-Dev, then applied SFT adaptation on 5k publicly-available trajectories to power SWE-Agents.

Result: Kimi-Dev achieved 60.4% on SWE-bench Verified (best among workflow approaches) and powered SWE-Agents to 48.6% pass@1, matching Claude 3.5 Sonnet performance.

Conclusion: Structured skill priors from Agentless training can effectively bridge workflow and agentic frameworks, creating transferable coding agents.

Abstract: Large Language Models (LLMs) are increasingly applied to software engineering (SWE), with SWE-bench as a key benchmark. Solutions are split into SWE-Agent frameworks with multi-turn interactions and workflow-based Agentless methods with single-turn verifiable steps. We argue these paradigms are not mutually exclusive: reasoning-intensive Agentless training induces skill priors, including localization, code edit, and self-reflection that enable efficient and effective SWE-Agent adaptation. In this work, we first curate the Agentless training recipe and present Kimi-Dev, an open-source SWE LLM achieving 60.4% on SWE-bench Verified, the best among workflow approaches. With additional SFT adaptation on 5k publicly-available trajectories, Kimi-Dev powers SWE-Agents to 48.6% pass@1, on par with that of Claude 3.5 Sonnet (241022 version). These results show that structured skill priors from Agentless training can bridge workflow and agentic frameworks for transferable coding agents.

Method: Proposes p-less sampling - an information-theoretic approach that dynamically sets truncation thresholds at each decoding step using the entire token probability distribution, eliminating the need for hyperparameters.

Result: Consistently outperforms existing sampling methods across math, logical reasoning, and creative writing tasks; maintains high text quality at higher temperatures; achieves better inference-time efficiency with lower average token sampling times and shorter generation lengths.

Conclusion: P-less sampling provides a robust, hyperparameter-free alternative to existing decoding strategies that delivers superior performance, temperature stability, and computational efficiency across diverse generation tasks.

Abstract: Obtaining high-quality outputs from Large Language Models (LLMs) often depends upon the choice of a sampling-based decoding strategy to probabilistically choose the next token at each generation step. While a variety of such sampling methods have been proposed, their performance can be sensitive to the selection of hyperparameters which may require different settings depending upon the generation task and temperature configuration. In this work, we introduce $p$-less sampling: an information-theoretic approach to sampling which dynamically sets a truncation threshold at each decoding step based on the entire token probability distribution. Unlike existing methods, $p$-less sampling has no hyperparameters and consistently produces high-quality outputs as temperature increases. We provide theoretical perspectives on $p$-less sampling to ground our proposed method and conduct experiments to empirically validate its effectiveness across a range of math, logical reasoning, and creative writing tasks. Our results demonstrate how $p$-less sampling consistently outperforms existing sampling approaches while exhibiting much less degradation in text quality at higher temperature values. We further show how $p$-less achieves greater inference-time efficiency than alternative methods through lower average token sampling times and shorter generation lengths, without sacrificing accuracy. Finally, we provide analyses to highlight the benefits of $p$-less through qualitative examples, case studies, and diversity assessments.

Method: Proposed SFTG framework with EEG Graph Transformer (EGT) to encode spatial connectivity and temporal dynamics, plus Graph Archetype Contrastive Learning (GAC) to learn subject-specific EEG graph archetypes for improved feature consistency and class separability.

Result: Significantly outperforms prior state-of-the-art EEG decoding methods in both subject-dependent and subject-independent evaluations on the Things-EEG dataset.

Conclusion: The approach demonstrates transformative potential by integrating graph-based learning with contrastive objectives, paving the way for more generalizable and robust neural representations in EEG-based brain decoding.

Abstract: Decoding visual neural representations from Electroencephalography (EEG) signals remains a formidable challenge due to their high-dimensional, noisy, and non-Euclidean nature. In this work, we propose a Spatial-Functional Awareness Transformer-based Graph Archetype Contrastive Learning (SFTG) framework to enhance EEG-based visual decoding. Specifically, we introduce the EEG Graph Transformer (EGT), a novel graph-based neural architecture that simultaneously encodes spatial brain connectivity and temporal neural dynamics. To mitigate high intra-subject variability, we propose Graph Archetype Contrastive Learning (GAC), which learns subject-specific EEG graph archetypes to improve feature consistency and class separability. Furthermore, we conduct comprehensive subject-dependent and subject-independent evaluations on the Things-EEG dataset, demonstrating that our approach significantly outperforms prior state-of-the-art EEG decoding methods.The results underscore the transformative potential of integrating graph-based learning with contrastive objectives to enhance EEG-based brain decoding, paving the way for more generalizable and robust neural representations.

Method: Combines insertion-based Edit Flow for discrete text tokens with Flow Matching for image latents, using hierarchical sampling that prioritizes content over grammar for concurrent text-image synthesis.

Result: Outperforms autoregressive baselines on both generation and understanding tasks across model sizes from 1B to 8B while using up to 50% fewer training FLOPs, surpassing both autoregressive and diffusion-based approaches.

Conclusion: OneFlow unlocks new capabilities for concurrent generation, iterative refinement, and natural reasoning-like generation, demonstrating superior performance and efficiency compared to existing approaches.

Abstract: We present OneFlow, the first non-autoregressive multimodal model that enables variable-length and concurrent mixed-modal generation. Unlike autoregressive models that enforce rigid causal ordering between text and image generation, OneFlow combines an insertion-based Edit Flow for discrete text tokens with Flow Matching for image latents. OneFlow enables concurrent text-image synthesis with hierarchical sampling that prioritizes content over grammar. Through controlled experiments across model sizes from 1B to 8B, we demonstrate that OneFlow outperforms autoregressive baselines on both generation and understanding tasks while using up to 50% fewer training FLOPs. OneFlow surpasses both autoregressive and diffusion-based approaches while unlocking new capabilities for concurrent generation, iterative refinement, and natural reasoning-like generation.

Method: Created WebRenderBench with 45.1k diverse real-world webpages and proposed a novel evaluation metric measuring layout/style consistency from rendered pages. Introduced ALISA system that integrates this metric into reinforcement learning as a reward signal.

Result: ALISA significantly boosts generation performance and achieves state-of-the-art results across multiple metrics compared to existing approaches.

Conclusion: The proposed WebRenderBench benchmark and ALISA system with the novel evaluation metric provide more efficient, objective, and reliable UI quality assessment for WebUI-to-Code conversion tasks.

Abstract: Automating the conversion of UI images into web code is a critical task for front-end development and rapid prototyping. Advances in multimodal large language models (MLLMs) have made WebUI-to-Code increasingly feasible, yet existing benchmarks remain limited in data diversity and evaluation reliability. To address these issues, we present WebRenderBench, a large-scale benchmark of 45.1k webpages collected from real-world portal sites, offering greater diversity, complexity, and realism than prior benchmarks. We further propose a novel evaluation metric that measures layout and style consistency from the final rendered pages. Unlike vision-based methods that rely on costly LLM reasoning or structure-based comparisons vulnerable to noise and asymmetry, our approach enables more efficient, objective, and reliable UI quality assessment. Finally, we introduce the Automated Layout and Style Inspection Agent (ALISA), which integrates this metric into reinforcement learning as a reward signal to enhance training on crawled asymmetric webpages. Experiments show that ALISA significantly boosts generation performance, achieving state-of-the-art results across multiple metrics.

Method: Formal analysis of multiple attention architectures (causal, bidirectional, local sparse, kernelized, cross-attention) combined with dynamic rule verification architecture and hot reloading of verified rule sets.

Result: Rule formats with low syntactic entropy and concentrated anchors reduce attention entropy and improve pointer fidelity, but reveal fundamental trade-off between anchor redundancy and attention entropy.

Conclusion: Principled anchor design and dual enforcement mechanisms are necessary to protect LLM-based agents against prompt injection attacks while maintaining compliance in evolving domains.

Abstract: The design of safety-critical agents based on large language models (LLMs) requires more than simple prompt engineering. This paper presents a comprehensive information-theoretic analysis of how rule encodings in system prompts influence attention mechanisms and compliance behaviour. We demonstrate that rule formats with low syntactic entropy and highly concentrated anchors reduce attention entropy and improve pointer fidelity, but reveal a fundamental trade-off between anchor redundancy and attention entropy that previous work failed to recognize. Through formal analysis of multiple attention architectures including causal, bidirectional, local sparse, kernelized, and cross-attention mechanisms, we establish bounds on pointer fidelity and show how anchor placement strategies must account for competing fidelity and entropy objectives. Combining these insights with a dynamic rule verification architecture, we provide a formal proof that hot reloading of verified rule sets increases the asymptotic probability of compliant outputs. These findings underscore the necessity of principled anchor design and dual enforcement mechanisms to protect LLM-based agents against prompt injection attacks while maintaining compliance in evolving domains.

Method: AITL integrates four types of live annotations: pairwise response preferences, agent adoption/rationales, knowledge relevance checks, and missing knowledge identification. These feedback signals directly update models.

Result: Production pilot showed significant improvements: +11.7% recall@75, +14.8% precision@8 in retrieval; +8.4% helpfulness in generation; +4.5% agent adoption rates.

Conclusion: Embedding human feedback loops directly into operational workflows effectively refines LLM-based customer support systems, enabling continuous improvement with much faster iteration cycles.

Abstract: We introduce an Agent-in-the-Loop (AITL) framework that implements a continuous data flywheel for iteratively improving an LLM-based customer support system. Unlike standard offline approaches that rely on batch annotations, AITL integrates four key types of annotations directly into live customer operations: (1) pairwise response preferences, (2) agent adoption and rationales, (3) knowledge relevance checks, and (4) identification of missing knowledge. These feedback signals seamlessly feed back into models’ updates, reducing retraining cycles from months to weeks. Our production pilot involving US-based customer support agents demonstrated significant improvements in retrieval accuracy (+11.7% recall@75, +14.8% precision@8), generation quality (+8.4% helpfulness) and agent adoption rates (+4.5%). These results underscore the effectiveness of embedding human feedback loops directly into operational workflows to continuously refine LLM-based customer support system.

Method: INFER uses an implicit neural frequency response framework with three innovations: end-to-end frequency-domain forward model, perceptual spectral supervision emphasizing critical auditory bands, and physics-based Kramers-Kronig consistency constraint.

Result: The method outperforms time- and hybrid-domain baselines on real-world automotive data, reducing average magnitude and phase reconstruction errors by over 39% and 51% respectively.

Conclusion: INFER sets a new state-of-the-art for neural acoustic modeling in automotive spaces, providing more accurate and adaptive acoustic modeling for confined resonant environments.

Abstract: Accurate modeling of spatial acoustics is critical for immersive and intelligible audio in confined, resonant environments such as car cabins. Current tuning methods are manual, hardware-intensive, and static, failing to account for frequency selective behaviors and dynamic changes like passenger presence or seat adjustments. To address this issue, we propose INFER: Implicit Neural Frequency Response fields, a frequency-domain neural framework that is jointly conditioned on source and receiver positions, orientations to directly learn complex-valued frequency response fields inside confined, resonant environments like car cabins. We introduce three key innovations over current neural acoustic modeling methods: (1) novel end-to-end frequency-domain forward model that directly learns the frequency response field and frequency-specific attenuation in 3D space; (2) perceptual and hardware-aware spectral supervision that emphasizes critical auditory frequency bands and deemphasizes unstable crossover regions; and (3) a physics-based Kramers-Kronig consistency constraint that regularizes frequency-dependent attenuation and delay. We evaluate our method over real-world data collected in multiple car cabins. Our approach significantly outperforms time- and hybrid-domain baselines on both simulated and real-world automotive datasets, cutting average magnitude and phase reconstruction errors by over 39% and 51%, respectively. INFER sets a new state-of-the-art for neural acoustic modeling in automotive spaces

Method: Web crawling of extensive raw data followed by automatic cleaning using an instrument classifier built on a pre-trained audio encoder to filter and aggregate clean segments of target instruments.

Result: MSS model trained with ACMID-Cleaned achieved 2.39dB SDR improvement over uncleaned version, and incorporating ACMID-Cleaned enhanced MSS model’s average performance by 1.16dB.

Conclusion: The automatic cleaning procedure effectively improves data quality, and the expanded 7-stem classification enables high granularity MSS systems with better performance.

Abstract: Most current music source separation (MSS) methods rely on supervised learning, limited by training data quantity and quality. Though web-crawling can bring abundant data, platform-level track labeling often causes metadata mismatches, impeding accurate “audio-label” pair acquisition. To address this, we present ACMID: a dataset for MSS generated through web crawling of extensive raw data, followed by automatic cleaning via an instrument classifier built on a pre-trained audio encoder that filters and aggregates clean segments of target instruments from the crawled tracks, resulting in the refined ACMID-Cleaned dataset. Leveraging abundant data, we expand the conventional classification from 4-stem (Vocal/Bass/Drums/Others) to 7-stem (Piano/Drums/Bass/Acoustic Guitar/Electric Guitar/Strings/Wind-Brass), enabling high granularity MSS systems. Experiments on SOTA MSS model demonstrates two key results: (i) MSS model trained with ACMID-Cleaned achieved a 2.39dB improvement in SDR performance compared to that with ACMID-Uncleaned, demostrating the effectiveness of our data cleaning procedure; (ii) incorporating ACMID-Cleaned to training enhances MSS model’s average performance by 1.16dB, confirming the value of our dataset. Our data crawling code, cleaning model code and weights are available at: https://github.com/scottishfold0621/ACMID.

Method: Multi-feature fusion with co-attention for audio (LLDs, MFCCs, Wav2Vec), combined visual features (OpenFace, ResNet, DenseNet), and an interaction module capturing relationships between personality traits and multimodal features.

Result: Experimental results from MPDD Elderly Depression Detection track show significant performance enhancement compared to baseline methods.

Conclusion: The proposed method provides valuable insights for future multimodal depression detection research in elderly populations, demonstrating the importance of personality integration.

Abstract: This paper presents our solution to the Multimodal Personality-aware Depression Detection (MPDD) challenge at ACM MM 2025. We propose a multimodal depression detection model in the Elderly that incorporates personality characteristics. We introduce a multi-feature fusion approach based on a co-attention mechanism to effectively integrate LLDs, MFCCs, and Wav2Vec features in the audio modality. For the video modality, we combine representations extracted from OpenFace, ResNet, and DenseNet to construct a comprehensive visual feature set. Recognizing the critical role of personality in depression detection, we design an interaction module that captures the relationships between personality traits and multimodal features. Experimental results from the MPDD Elderly Depression Detection track demonstrate that our method significantly enhances performance, providing valuable insights for future research in multimodal depression detection among elderly populations.

Method: IntMeanFlow uses integral velocity distillation by approximating average velocity with teacher’s instantaneous velocity over temporal intervals, eliminating JVPs and self-bootstrap. Also introduces Optimal Step Sampling Search (O3S) algorithm to find model-specific optimal sampling steps.

Result: Achieves 1-NFE inference for token-to-spectrogram and 3-NFE for text-to-spectrogram tasks while maintaining high-quality synthesis. Reduces GPU memory usage and improves training stability.

Conclusion: IntMeanFlow provides an efficient framework for fast, high-quality speech synthesis by addressing memory and stability issues of previous flow-based approaches.

Abstract: Flow-based generative models have greatly improved text-to-speech (TTS) synthesis quality, but inference speed remains limited by the iterative sampling process and multiple function evaluations (NFE). The recent MeanFlow model accelerates generation by modeling average velocity instead of instantaneous velocity. However, its direct application to TTS encounters challenges, including GPU memory overhead from Jacobian-vector products (JVP) and training instability due to self-bootstrap processes. To address these issues, we introduce IntMeanFlow, a framework for few-step speech generation with integral velocity distillation. By approximating average velocity with the teacher’s instantaneous velocity over a temporal interval, IntMeanFlow eliminates the need for JVPs and self-bootstrap, improving stability and reducing GPU memory usage. We also propose the Optimal Step Sampling Search (O3S) algorithm, which identifies the model-specific optimal sampling steps, improving speech synthesis without additional inference overhead. Experiments show that IntMeanFlow achieves 1-NFE inference for token-to-spectrogram and 3-NFE for text-to-spectrogram tasks while maintaining high-quality synthesis. Demo samples are available at https://vvwangvv.github.io/intmeanflow.

Method: Proposes two complementary attribution forms: training-time and inference-time attribution, with preference for inference-time attribution that enables direct compensation when artists’ catalogs condition generated outputs.

Result: Enables direct, verifiable compensation for artists and transparent information about attribution and permitted usage for users.

Conclusion: Provides an ethical and practical solution for robust compensation mechanisms in AI-generated music, embedding provenance and fairness at the core of generative systems.

Abstract: The rise of AI-generated music is diluting royalty pools and revealing structural flaws in existing remuneration frameworks, challenging the well-established artist compensation systems in the music industry. Existing compensation solutions, such as piecemeal licensing agreements, lack scalability and technical rigour, while current data attribution mechanisms provide only uncertain estimates and are rarely implemented in practice. This paper introduces a framework for a generative music infrastructure centred on direct attribution, transparent royalty distribution, and granular control for artists and rights’ holders. We distinguish ontologically between the training set and the inference set, which allows us to propose two complementary forms of attribution: training-time attribution and inference-time attribution. We here favour inference-time attribution, as it enables direct, verifiable compensation whenever an artist’s catalogue is used to condition a generated output. Besides, users benefit from the ability to condition generations on specific songs and receive transparent information about attribution and permitted usage. Our approach offers an ethical and practical solution to the pressing need for robust compensation mechanisms in the era of AI-generated music, ensuring that provenance and fairness are embedded at the core of generative systems.

Method: Proposes Posterior Feature Correction (PFC), a two-pass training-free inference method that first generates audio to detect hallucinated segments, then regenerates audio after masking corresponding video features at those timestamps.

Result: State-of-the-art models suffer from severe Insertion Hallucination, while PFC reduces both prevalence and duration of hallucinations by over 50% on average without degrading conventional metrics.

Conclusion: This work formally defines, systematically measures, and effectively mitigates Insertion Hallucination, paving the way for more reliable Video-to-Audio models.

Abstract: Video-to-Audio generation has made remarkable strides in automatically synthesizing sound for video. However, existing evaluation metrics, which focus on semantic and temporal alignment, overlook a critical failure mode: models often generate acoustic events, particularly speech and music, that have no corresponding visual source. We term this phenomenon Insertion Hallucination and identify it as a systemic risk driven by dataset biases, such as the prevalence of off-screen sounds, that remains completely undetected by current metrics. To address this challenge, we first develop a systematic evaluation framework that employs a majority-voting ensemble of multiple audio event detectors. We also introduce two novel metrics to quantify the prevalence and severity of this issue: IH@vid (the fraction of videos with hallucinations) and IH@dur (the fraction of hallucinated duration). Building on this, we propose Posterior Feature Correction, a novel training-free inference-time method that mitigates IH. PFC operates in a two-pass process: it first generates an initial audio output to detect hallucinated segments, and then regenerates the audio after masking the corresponding video features at those timestamps. Experiments on several mainstream V2A benchmarks first reveal that state-of-the-art models suffer from severe IH. In contrast, our PFC method reduces both the prevalence and duration of hallucinations by over 50% on average, without degrading, and in some cases even improving, conventional metrics for audio quality and temporal synchronization. Our work is the first to formally define, systematically measure, and effectively mitigate Insertion Hallucination, paving the way for more reliable and faithful V2A models.

Method: Developed WEALY pipeline leveraging Whisper decoder embeddings for lyrics matching, with robust baseline establishment and exploration of multimodal extensions combining textual and acoustic features.

Result: WEALY achieves performance comparable to state-of-the-art methods while providing reproducibility. Extensive experiments include ablation studies on language robustness, loss functions, and embedding strategies.

Conclusion: The work contributes a reliable benchmark for future research and demonstrates the potential of speech technologies for music information retrieval tasks.

Abstract: Audio-based lyrics matching can be an appealing alternative to other content-based retrieval approaches, but existing methods often suffer from limited reproducibility and inconsistent baselines. In this work, we introduce WEALY, a fully reproducible pipeline that leverages Whisper decoder embeddings for lyrics matching tasks. WEALY establishes robust and transparent baselines, while also exploring multimodal extensions that integrate textual and acoustic features. Through extensive experiments on standard datasets, we demonstrate that WEALY achieves a performance comparable to state-of-the-art methods that lack reproducibility. In addition, we provide ablation studies and analyses on language robustness, loss functions, and embedding strategies. This work contributes a reliable benchmark for future research, and underscores the potential of speech technologies for music information retrieval tasks.

Method: Created STOPA dataset with systematic variation across 8 acoustic models, 6 vocoder models, and diverse parameter settings from 13 distinct synthesisers, totaling 700k samples with comprehensive metadata.

Result: STOPA provides a systematically controlled framework covering broader range of generative factors (vocoder model, acoustic model, pretrained weights) compared to existing datasets with limited variation or sparse metadata.

Conclusion: The systematic control in STOPA improves attribution accuracy, aiding forensic analysis, deepfake detection, and generative model transparency in speech source tracing research.

Abstract: A key research area in deepfake speech detection is source tracing - determining the origin of synthesised utterances. The approaches may involve identifying the acoustic model (AM), vocoder model (VM), or other generation-specific parameters. However, progress is limited by the lack of a dedicated, systematically curated dataset. To address this, we introduce STOPA, a systematically varied and metadata-rich dataset for deepfake speech source tracing, covering 8 AMs, 6 VMs, and diverse parameter settings across 700k samples from 13 distinct synthesisers. Unlike existing datasets, which often feature limited variation or sparse metadata, STOPA provides a systematically controlled framework covering a broader range of generative factors, such as the choice of the vocoder model, acoustic model, or pretrained weights, ensuring higher attribution reliability. This control improves attribution accuracy, aiding forensic analysis, deepfake detection, and generative model transparency.

Method: Uses collaborative distillation for intra-set and inter-set correlations with pairwise feature matching, residual fusion for representative features, and multi-layer interactive distillation based on dual-stream time-frequency cross-calibration.

Result: Applied to DPDCRN model, the method consistently improves low-complexity student model performance on both single-channel and multi-channel datasets, outperforming other distillation schemes.

Conclusion: The proposed I²RF-TFCKD framework effectively enhances speech enhancement performance through refined knowledge distillation that leverages time-frequency characteristics and multi-layer feature interactions.

Abstract: In this paper, we propose an intra-inter representation fusion knowledge distillation (KD) framework with time-frequency calibration (I$^2$RF-TFCKD) for SE, which achieves distillation through the fusion of multi-layer teacher-student feature flows. Different from previous distillation strategies for SE, the proposed framework fully utilizes the time-frequency differential information of speech while promoting global knowledge flow. Firstly, we construct a collaborative distillation paradigm for intra-set and inter-set correlations. Within a correlated set, multi-layer teacher-student features are pairwise matched for calibrated distillation. Subsequently, we generate representative features from each correlated set through residual fusion to form the fused feature set that enables inter-set knowledge interaction. Secondly, we propose a multi-layer interactive distillation based on dual-stream time-frequency cross-calibration, which calculates the teacher-student similarity calibration weights in the time and frequency domains respectively and performs cross-weighting, thus enabling refined allocation of distillation contributions across different layers according to speech characteristics. The proposed distillation strategy is applied to the dual-path dilated convolutional recurrent network (DPDCRN) that ranked first in the SE track of the L3DAS23 challenge. To evaluate the effectiveness of I$^2$RF-TFCKD, we conduct experiments on both single-channel and multi-channel SE datasets. Objective evaluations demonstrate that the proposed KD strategy consistently and effectively improves the performance of the low-complexity student model and outperforms other distillation schemes.

Method: Implemented four differentiable loss functions paired with subtractive, additive, and AM synthesizers, running 300 randomized sound-matching trials for each of the 16 combinations, measuring performance via parameter differences, spectrogram metrics, and listening scores.

Result: Found moderate consistency among performance measures and that loss function performance is highly dependent on the synthesizer type, with no one-size-fits-all solution.

Conclusion: Expanding sound-matching experiments and developing similarity metrics tailored to specific synthesis techniques is more valuable than pursuing universal solutions.

Abstract: Manual sound design with a synthesizer is inherently iterative: an artist compares the synthesized output to a mental target, adjusts parameters, and repeats until satisfied. Iterative sound-matching automates this workflow by continually programming a synthesizer under the guidance of a loss function (or similarity measure) toward a target sound. Prior comparisons of loss functions have typically favored one metric over another, but only within narrow settings: limited synthesis methods, few loss types, often without blind listening tests. This leaves open the question of whether a universally optimal loss exists, or the choice of loss remains a creative decision conditioned on the synthesis method and the sound designer’s preference. We propose differentiable iterative sound-matching as the natural extension of the available literature, since it combines the manual approach to sound design with modern advances in machine learning. To analyze the variability of loss function performance across synthesizers, we implemented a mix of four novel and established differentiable loss functions, and paired them with differentiable subtractive, additive, and AM synthesizers. For each of the sixteen synthesizer–loss combinations, we ran 300 randomized sound-matching trials. Performance was measured using parameter differences, spectrogram-distance metrics, and manually assigned listening scores. We observed a moderate level of consistency among the three performance measures. Our post-hoc analysis shows that the loss function performance is highly dependent on the synthesizer. These findings underscore the value of expanding the scope of sound-matching experiments and developing new similarity metrics tailored to specific synthesis techniques rather than pursuing one-size-fits-all solutions.

Method: Uses position-independent clicking sounds as triggers, varies signal-to-noise ratio for stealth, and extends to speaker verification by selecting similar speakers as proxy targets based on cosine similarity.

Result: Achieves up to 95.04% success rate for speaker identification and up to 90% for speaker verification when target and enrolled speakers are highly similar.

Conclusion: The proposed multi-target backdoor attack is highly effective and demonstrates the vulnerability of speaker recognition systems to sophisticated attacks using simple audio triggers.

Abstract: In this work, we propose a multi-target backdoor attack against speaker identification using position-independent clicking sounds as triggers. Unlike previous single-target approaches, our method targets up to 50 speakers simultaneously, achieving success rates of up to 95.04%. To simulate more realistic attack conditions, we vary the signal-to-noise ratio between speech and trigger, demonstrating a trade-off between stealth and effectiveness. We further extend the attack to the speaker verification task by selecting the most similar training speaker - based on cosine similarity - as a proxy target. The attack is most effective when target and enrolled speaker pairs are highly similar, reaching success rates of up to 90% in such cases.

Method: Non-probabilistic autoencoder architecture with independence constraint between predicted latent variable and target controllable variable, ensuring consistent signal transformation while preserving content and modifying desired attributes.

Result: Quantitative evaluations confirm effectiveness and generality of the approach across speech styles including speaker identity and emotion conversion.

Conclusion: The proposed framework provides theoretical guarantees for speech attribute conversion and demonstrates versatility across different speech style manipulation tasks.

Abstract: While signal conversion and disentangled representation learning have shown promise for manipulating data attributes across domains such as audio, image, and multimodal generation, existing approaches, especially for speech style conversion, are largely empirical and lack rigorous theoretical foundations to guarantee reliable and interpretable control. In this work, we propose a general framework for speech attribute conversion, accompanied by theoretical analysis and guarantees under reasonable assumptions. Our framework builds on a non-probabilistic autoencoder architecture with an independence constraint between the predicted latent variable and the target controllable variable. This design ensures a consistent signal transformation, conditioned on an observed style variable, while preserving the original content and modifying the desired attribute. We further demonstrate the versatility of our method by evaluating it on speech styles, including speaker identity and emotion. Quantitative evaluations confirm the effectiveness and generality of the proposed approach.

Method: Using a longitudinal approach to monitor emotions and behaviors over time to establish baseline understanding of autistic students’ unique needs and challenges.

Result: The study proposes a targeted framework for developing applications and technical aids designed to meet the identified needs of autistic children in IT education.

Conclusion: Early identification of behavioral patterns is crucial for creating inclusive learning environments and improving educational outcomes for children with ASD, emphasizing evidence-based sequential intervention approaches.

Abstract: Autism Spectrum Disorder significantly influences the communication abilities, learning processes, behavior, and social interactions of individuals. Although early intervention and customized educational strategies are critical to improving outcomes, there is a pivotal gap in understanding and addressing nuanced behavioral patterns and emotional identification in autistic children prior to skill development. This extended research delves into the foundational step of recognizing and mapping these patterns as a prerequisite to improving learning and soft skills. Using a longitudinal approach to monitor emotions and behaviors, this study aims to establish a baseline understanding of the unique needs and challenges faced by autistic students, particularly in the Information Technology domain, where opportunities are markedly limited. Through a detailed analysis of behavioral trends over time, we propose a targeted framework for developing applications and technical aids designed to meet these identified needs. Our research underscores the importance of a sequential and evidence-based intervention approach that prioritizes a deep understanding of each child’s behavioral and emotional landscape as the basis for effective skill development. By shifting the focus toward early identification of behavioral patterns, we aim to foster a more inclusive and supportive learning environment that can significantly improve the educational and developmental trajectory of children with ASD.

Method: Proposes MACC-MGNAS with three components: 1) Modality-aware cooperative co-evolution framework partitioning global population into modality-specific gene groups, 2) Modality-aware dual-track surrogate method to reduce evaluation cost, 3) Similarity-based population diversity indicator to balance exploration-exploitation.

Result: Achieves 81.67% F1-score on VulCE dataset within only 3 GPU-hours, outperforming state-of-the-art by 8.7% F1 while reducing computation cost by 27%.

Conclusion: MACC-MGNAS effectively addresses modality heterogeneity in multimodal graph neural architecture search, providing efficient and accurate solution for vulnerability co-exploitation prediction.

Abstract: Co-exploitation attacks on software vulnerabilities pose severe risks to enterprises, a threat that can be mitigated by analyzing heterogeneous and multimodal vulnerability data. Multimodal graph neural networks (MGNNs) are well-suited to integrate complementary signals across modalities, thereby improving attack-prediction accuracy. However, designing an effective MGNN architecture is challenging because it requires coordinating modality-specific components at each layer, which is infeasible through manual tuning. Genetic algorithm (GA)-based graph neural architecture search (GNAS) provides a natural solution, yet existing methods are confined to single modalities and overlook modality heterogeneity. To address this limitation, we propose a modality-aware cooperative co-evolutionary algorithm for multimodal graph neural architecture search, termed MACC-MGNAS. First, we develop a modality-aware cooperative co-evolution (MACC) framework under a divide-and-conquer paradigm: a coordinator partitions a global chromosome population into modality-specific gene groups, local workers evolve them independently, and the coordinator reassembles chromosomes for joint evaluation. This framework effectively captures modality heterogeneity ignored by single-modality GNAS. Second, we introduce a modality-aware dual-track surrogate (MADTS) method to reduce evaluation cost and accelerate local gene evolution. Third, we design a similarity-based population diversity indicator (SPDI) strategy to adaptively balance exploration and exploitation, thereby accelerating convergence and avoiding local optima. On a standard vulnerabilities co-exploitation (VulCE) dataset, MACC-MGNAS achieves an F1-score of 81.67% within only 3 GPU-hours, outperforming the state-of-the-art competitor by 8.7% F1 while reducing computation cost by 27%.

Method: MultiFair uses a dual-level gradient modulation process that dynamically modulates training gradients regarding optimization direction and magnitude at both data modality and group levels.

Result: Extensive experiments on two multimodal medical datasets show that MultiFair outperforms state-of-the-art multimodal learning and fairness learning methods.

Conclusion: MultiFair successfully addresses both modality imbalance and demographic unfairness in multimodal medical classification through its dual-level gradient modulation approach.

Abstract: Medical decision systems increasingly rely on data from multiple sources to ensure reliable and unbiased diagnosis. However, existing multimodal learning models fail to achieve this goal because they often ignore two critical challenges. First, various data modalities may learn unevenly, thereby converging to a model biased towards certain modalities. Second, the model may emphasize learning on certain demographic groups causing unfair performances. The two aspects can influence each other, as different data modalities may favor respective groups during optimization, leading to both imbalanced and unfair multimodal learning. This paper proposes a novel approach called MultiFair for multimodal medical classification, which addresses these challenges with a dual-level gradient modulation process. MultiFair dynamically modulates training gradients regarding the optimization direction and magnitude at both data modality and group levels. We conduct extensive experiments on two multimodal medical datasets with different demographic groups. The results show that MultiFair outperforms state-of-the-art multimodal learning and fairness learning methods.

Method: Benchmarked two state-of-the-art models (GNN-RNN and MMST-ViT) using the large-scale CropNet dataset spanning 1,200+ U.S. counties from 2017-2022. Used leave-one-cluster-out cross-validation across seven USDA Farm Resource Regions and year-ahead prediction scenarios to test out-of-distribution generalization.

Result: GNN-RNN demonstrated superior generalization with positive correlations under geographic shifts and 135x faster training (14 minutes vs. 31.5 hours). MMST-ViT performed well in-domain but degraded sharply under OOD conditions. Significant variability in cross-region transferability was observed, with some regions showing stable performance while others like Prairie Gateway exhibited persistent underperformance.

Conclusion: Spatial-temporal alignment - not merely model complexity or data scale - is key to robust generalization. The findings highlight the need for transparent OOD evaluation protocols to ensure equitable and reliable climate-aware agricultural forecasting.

Abstract: Climate change is increasingly disrupting agricultural systems, making accurate crop yield forecasting essential for food security. While deep learning models have shown promise in yield prediction using satellite and weather data, their ability to generalize across geographic regions and years - critical for real-world deployment - remains largely untested. We benchmark two state-of-the-art models, GNN-RNN and MMST-ViT, under realistic out-of-distribution (OOD) conditions using the large-scale CropNet dataset spanning 1,200+ U.S. counties from 2017-2022. Through leave-one-cluster-out cross-validation across seven USDA Farm Resource Regions and year-ahead prediction scenarios, we identify substantial variability in cross-region transferability. GNN-RNN demonstrates superior generalization with positive correlations under geographic shifts, while MMST-ViT performs well in-domain but degrades sharply under OOD conditions. Regions like Heartland and Northern Great Plains show stable transfer dynamics (RMSE less than 10 bu/acre for soybean), whereas Prairie Gateway exhibits persistent underperformance (RMSE greater than 20 bu/acre) across both models and crops, revealing structural dissimilarities likely driven by semi-arid climate, irrigation patterns, and incomplete spectral coverage. Beyond accuracy differences, GNN-RNN achieves 135x faster training than MMST-ViT (14 minutes vs. 31.5 hours), making it more viable for sustainable deployment. Our findings underscore that spatial-temporal alignment - not merely model complexity or data scale - is key to robust generalization, and highlight the need for transparent OOD evaluation protocols to ensure equitable and reliable climate-aware agricultural forecasting.

Method: Developed a recurrent neural network climate surrogate pretrained on 20,000 multi-gas emission pathways to replace the CICERO-SCM climate model, achieving 1000x faster inference while maintaining accuracy.

Result: The surrogate model achieved near-simulator accuracy with global-mean temperature RMSE ≈ 0.0004K and accelerated end-to-end training by >100x. It converges to the same optimal policies as the original simulator.

Conclusion: The framework bypasses computational bottlenecks without sacrificing policy fidelity, enabling large-scale multi-agent climate policy experiments with multi-gas dynamics and high-fidelity climate response.

Abstract: Climate policy studies require models that capture the combined effects of multiple greenhouse gases on global temperature, but these models are computationally expensive and difficult to embed in reinforcement learning. We present a multi-agent reinforcement learning (MARL) framework that integrates a high-fidelity, highly efficient climate surrogate directly in the environment loop, enabling regional agents to learn climate policies under multi-gas dynamics. As a proof of concept, we introduce a recurrent neural network architecture pretrained on ($20{,}000$) multi-gas emission pathways to surrogate the climate model CICERO-SCM. The surrogate model attains near-simulator accuracy with global-mean temperature RMSE $\approx 0.0004 \mathrm{K}$ and approximately $1000\times$ faster one-step inference. When substituted for the original simulator in a climate-policy MARL setting, it accelerates end-to-end training by $>!100\times$. We show that the surrogate and simulator converge to the same optimal policies and propose a methodology to assess this property in cases where using the simulator is intractable. Our work allows to bypass the core computational bottleneck without sacrificing policy fidelity, enabling large-scale multi-agent experiments across alternative climate-policy regimes with multi-gas dynamics and high-fidelity climate response.

Method: Developed a pipeline to generate and curate high-quality CUDA kernels with reasoning traces, constructing ConCuR dataset and training KernelCoder model on PyTorch-reasoning-CUDA kernel pairs.

Result: KernelCoder significantly outperforms existing top-performing models including QwQ-32B, all open-source kernel generation models, and frontier models like DeepSeek-V3.1-Think and Claude-4-sonnet in KernelBench setup.

Conclusion: Concise yet informative reasoning traces enable robust high-performance kernel generation, and average reasoning length can serve as a metric for task difficulty assessment, with the pipeline helping future data collection.

Abstract: GPU kernel generation by LLMs has recently experienced rapid development, leveraging test-time scaling and reinforcement learning techniques. However, a key challenge for kernel generation is the scarcity of high-quality data, as most high-quality kernels are proprietary and not open-source. This challenge prevents us from leveraging supervised fine-tuning to align LLMs to the kernel generation task. To address this challenge, we develop a pipeline that generates and curates high-quality CUDA kernels with reasoning traces, motivated by a critical observation that concise yet informative reasoning traces result in robust generation of high-performance kernels. Using this pipeline, we construct our dataset ConCuR and introduce our model KernelCoder, which is the first model trained on a curated dataset consisting of PyTorch, reasoning, and CUDA kernel pairs, to our knowledge. In the KernelBench setup, our model achieves significant improvements over the existing top-performing model, QwQ-32B, and outperforms all open-source models fine-tuned for kernel generation, as well as frontier models such as DeepSeek-V3.1-Think and Claude-4-sonnet. Finally, we show that the average reasoning length can serve as a metric to assess the difficulty of kernel generation tasks. The observations, metrics, and our data collection and curation pipeline can help obtain better data in the kernel generation task in the future.

Method: The study examines offline settings (pretraining on expert data) and simultaneous settings (choosing between own experience or expert outcomes). It proposes an information-directed rule where learners process data that maximizes one-step information gain about the optimal action.

Result: The paper proves that pretraining on expert outcomes tightens information-theoretic regret bounds by the mutual information between expert data and optimal action. It also provides strategies for inferring when to trust experts.

Conclusion: By quantifying the value of expert data, the framework provides practical, information-theoretic algorithms for agents to intelligently decide when to learn from others, safeguarding against ineffective or compromised experts.

Abstract: Complex learning agents are increasingly deployed alongside existing experts, such as human operators or previously trained agents. However, it remains unclear how should learners optimally incorporate certain forms of expert data, which may differ in structure from the learner’s own action-outcome experiences. We study this problem in the context of Bayesian multi-armed bandits, considering: (i) offline settings, where the learner receives a dataset of outcomes from the expert’s optimal policy before interaction, and (ii) simultaneous settings, where the learner must choose at each step whether to update its beliefs based on its own experience, or based on the outcome simultaneously achieved by an expert. We formalize how expert data influences the learner’s posterior, and prove that pretraining on expert outcomes tightens information-theoretic regret bounds by the mutual information between the expert data and the optimal action. For the simultaneous setting, we propose an information-directed rule where the learner processes the data source that maximizes their one-step information gain about the optimal action. Finally, we propose strategies for how the learner can infer when to trust the expert and when not to, safeguarding the learner for the cases where the expert is ineffective or compromised. By quantifying the value of expert data, our framework provides practical, information-theoretic algorithms for agents to intelligently decide when to learn from others.

Method: Encode-Think-Decode (ETD) trains base models to iterate over reasoning-relevant layers during mid-training, with optional adaptive depth strategy for computation adjustment per token.

Result: Substantial gains on 17 reasoning benchmarks, including +28.4% accuracy on GSM8K and +36% on MATH with OLMo-2 1B Base model.

Conclusion: Recursive latent reasoning offers a simple and effective path to stronger LLM reasoning capabilities.

Abstract: Most efforts to improve the reasoning capabilities of large language models (LLMs) involve either scaling the number of parameters and the size of training data, or scaling inference computation by letting models generate complex chains of thought. Motivated by interpretability studies showing that the crucial computation required for reasoning tasks is concentrated in a limited range of layers, we introduce Encode-Think-Decode (ETD), a method that enhances the reasoning capabilities of a base model by training it to iterate over a small subset of reasoning-relevant layers during the mid-training stage. ETD amplifies latent reasoning while preserving the original architecture, parameter count, hyperparameters, and training data composition. When iterating on the selected layers at inference time, ETD models yield substantial gains on 17 reasoning benchmarks, including +28.4% relative accuracy improvement on GSM8K and +36% on MATH with the OLMo-2 1B Base model. We also explore an adaptive depth strategy that adjusts the computation per input token. Our results show that recursive latent reasoning offers a simple and effective path to stronger LLM reasoning.

Method: ShapeLLM - an adaptation of model-free opponent shaping methods tailored for transformer-based agents, tested across competitive and cooperative game-theoretic environments.

Result: LLM agents successfully guided opponents toward exploitable equilibria in competitive games and promoted coordination/improved collective welfare in cooperative games.

Conclusion: LLM agents can both shape and be shaped through interaction, establishing opponent shaping as a key dimension of multi-agent LLM research.

Abstract: Large Language Models (LLMs) are increasingly being deployed as autonomous agents in real-world environments. As these deployments scale, multi-agent interactions become inevitable, making it essential to understand strategic behavior in such systems. A central open question is whether LLM agents, like reinforcement learning agents, can shape the learning dynamics and influence the behavior of others through interaction alone. In this paper, we present the first investigation of opponent shaping (OS) with LLM-based agents. Existing OS algorithms cannot be directly applied to LLMs, as they require higher-order derivatives, face scalability constraints, or depend on architectural components that are absent in transformers. To address this gap, we introduce ShapeLLM, an adaptation of model-free OS methods tailored for transformer-based agents. Using ShapeLLM, we examine whether LLM agents can influence co-players’ learning dynamics across diverse game-theoretic environments. We demonstrate that LLM agents can successfully guide opponents toward exploitable equilibria in competitive games (Iterated Prisoner’s Dilemma, Matching Pennies, and Chicken) and promote coordination and improve collective welfare in cooperative games (Iterated Stag Hunt and a cooperative version of the Prisoner’s Dilemma). Our findings show that LLM agents can both shape and be shaped through interaction, establishing opponent shaping as a key dimension of multi-agent LLM research.

Method: Follow-the-Regularized-Leader framework with efficient estimators via Matrix Geometric Resampling, building on the BOBW framework for hard problems.

Result: The algorithm achieves Θ(T^{2/3}) minimax-optimal regret in adversarial settings and poly-logarithmic regret in (corrupted) stochastic regimes.

Conclusion: The proposed BOBW algorithm provides computationally efficient performance guarantees for linear contextual bandits with paid observations across different environmental settings.

Abstract: We study the problem of linear contextual bandits with paid observations, where at each round the learner selects an action in order to minimize its loss in a given context, and can then decide to pay a fixed cost to observe the loss of any arm. Building on the Follow-the-Regularized-Leader framework with efficient estimators via Matrix Geometric Resampling, we introduce a computationally efficient Best-of-Both-Worlds (BOBW) algorithm for this problem. We show that it achieves the minimax-optimal regret of $\Theta(T^{2/3})$ in adversarial settings, while guaranteeing poly-logarithmic regret in (corrupted) stochastic regimes. Our approach builds on the framework from \cite{BOBWhardproblems} to design BOBW algorithms for ``hard problem’’, using analysis techniques tailored for the setting that we consider.

Method: Framed as a contextual bandit problem using prompt features and user preference vectors, BaRP simulates online feedback during training and adapts routing decisions to each new prompt rather than relying on full-information offline supervision.

Result: Comprehensive experiments show BaRP consistently outperforms strong offline routers by at least 12.46% and the largest LLM by at least 2.45%, with robust generalization to unseen tasks.

Conclusion: BaRP effectively bridges the training-deployment gap in LLM routing by operating under realistic partial-feedback conditions while supporting flexible performance/cost trade-offs at inference time.

Abstract: Efficient use of large language models (LLMs) is critical for deployment at scale: without adaptive routing, systems either overpay for strong models or risk poor performance from weaker ones. Selecting the right LLM for each query is fundamentally an online decision problem: models differ in strengths, prices fluctuate, and users value accuracy and cost differently. Yet most routers are trained offline with labels for all candidate models, an assumption that breaks in deployment, where only the outcome of the chosen model is observed. We bridge this gap with BaRP, a Bandit-feedback Routing with Preferences approach that trains under the same partial-feedback restriction as deployment, while supporting preference-tunable inference: operators can dial the performance/cost trade-off at test time without retraining. Framed as a contextual bandit over prompt features and a user preference vector, our method simulates an online feedback setting during training and adapts its routing decisions to each new prompt, rather than depending on full-information offline supervision. Comprehensive experiments show that our method consistently outperforms strong offline routers by at least 12.46% and the largest LLM by at least 2.45%, and generalizes robustly for unseen tasks.

Method: Two-timescale Federated Temporal Difference (FTD) learning with Polyak-Ruppert averaging that handles Markovian data without requiring prior knowledge of problem parameters.

Result: The method provably attains the optimal $ ilde{O}(1/NT)$ convergence rate in both average-reward and discounted settings, addressing the parameter dependence issue in existing approaches.

Conclusion: This work provides a parameter-free FTD approach that achieves optimal convergence for federated reinforcement learning with Markovian data, with novel contributions even in single-agent settings and applicability to heterogeneous federated environments.

Abstract: Federated learning (FL) can dramatically speed up reinforcement learning by distributing exploration and training across multiple agents. It can guarantee an optimal convergence rate that scales linearly in the number of agents, i.e., a rate of $\tilde{O}(1/(NT)),$ where $T$ is the iteration index and $N$ is the number of agents. However, when the training samples arise from a Markov chain, existing results on TD learning achieving this rate require the algorithm to depend on unknown problem parameters. We close this gap by proposing a two-timescale Federated Temporal Difference (FTD) learning with Polyak-Ruppert averaging. Our method provably attains the optimal $\tilde{O}(1/NT)$ rate in both average-reward and discounted settings–offering a parameter-free FTD approach for Markovian data. Although our results are novel even in the single-agent setting, they apply to the more realistic and challenging scenario of FL with heterogeneous environments.

Method: Models each expert’s output as a Gaussian distribution and uses an uncertainty-based gating mechanism where expert contributions are determined by their estimated variance rather than input features.

Result: Outperforms single-expert models and traditional MoE setups across diverse time series forecasting benchmarks, providing well-quantified uncertainties that correlate with prediction errors.

Conclusion: MoGU enhances forecast reliability by directly quantifying both predictions and their uncertainties through its novel uncertainty-based gating mechanism.

Abstract: We introduce Mixture-of-Gaussians with Uncertainty-based Gating (MoGU), a novel Mixture-of-Experts (MoE) framework designed for regression tasks and applied to time series forecasting. Unlike conventional MoEs that provide only point estimates, MoGU models each expert’s output as a Gaussian distribution. This allows it to directly quantify both the forecast (the mean) and its inherent uncertainty (variance). MoGU’s core innovation is its uncertainty-based gating mechanism, which replaces the traditional input-based gating network by using each expert’s estimated variance to determine its contribution to the final prediction. Evaluated across diverse time series forecasting benchmarks, MoGU consistently outperforms single-expert models and traditional MoE setups. It also provides well-quantified, informative uncertainties that directly correlate with prediction errors, enhancing forecast reliability. Our code is available from: https://github.com/yolish/moe_unc_tsf

Method: Uses neural posterior estimation with a transformer-based neural network model (metabeta) that amortizes computation by pre-training on simulated datasets with known ground truth.

Result: Shows stable and comparable performance to MCMC-based parameter estimation on both simulated and real data, while requiring only a fraction of the computation time.

Conclusion: Metabeta provides an efficient alternative to traditional MCMC methods for Bayesian mixed-effects regression, enabling faster inference while maintaining comparable accuracy.

Abstract: Hierarchical data with multiple observations per group is ubiquitous in empirical sciences and is often analyzed using mixed-effects regression. In such models, Bayesian inference gives an estimate of uncertainty but is analytically intractable and requires costly approximation using Markov Chain Monte Carlo (MCMC) methods. Neural posterior estimation shifts the bulk of computation from inference time to pre-training time, amortizing over simulated datasets with known ground truth targets. We propose metabeta, a transformer-based neural network model for Bayesian mixed-effects regression. Using simulated and real data, we show that it reaches stable and comparable performance to MCMC-based parameter estimation at a fraction of the usually required time.

Method: Used tensor completion as a surrogate model for designing optimal lattice structures with mechanical performance, comparing it against Gaussian Process and XGBoost.

Result: Tensor completion showed 5% increased R² compared to classic ML methods with biased sampling, while maintaining comparable performance with uniformly random sampling.

Conclusion: Tensor completion is a superior approach for accelerating materials design in scenarios with biased data sampling, addressing limitations of traditional ML methods.

Abstract: When designing new materials, it is often necessary to design a material with specific desired properties. Unfortunately, as new design variables are added, the search space grows exponentially, which makes synthesizing and validating the properties of each material very impractical and time-consuming. In this work, we focus on the design of optimal lattice structures with regard to mechanical performance. Computational approaches, including the use of machine learning (ML) methods, have shown improved success in accelerating materials design. However, these ML methods are still lacking in scenarios when training data (i.e. experimentally validated materials) come from a non-uniformly random sampling across the design space. For example, an experimentalist might synthesize and validate certain materials more frequently because of convenience. For this reason, we suggest the use of tensor completion as a surrogate model to accelerate the design of materials in these atypical supervised learning scenarios. In our experiments, we show that tensor completion is superior to classic ML methods such as Gaussian Process and XGBoost with biased sampling of the search space, with around 5% increased $R^2$. Furthermore, tensor completion still gives comparable performance with a uniformly random sampling of the entire search space.

Method: HEMERA applies transformer-based deep learning directly to raw GWAS genotype data without clinical covariates, using additive positional encodings, neural genotype embeddings, and variant filtering. Includes post hoc explainability via Layer-wise Integrated Gradients.

Result: Achieved >99% AUC score when trained on 27,254 Million Veteran Program participants. Model predictions strongly align with known lung cancer risk loci through explainability analysis.

Conclusion: HEMERA demonstrates that transparent, hypothesis-generating models can provide accurate lung cancer risk assessment for personalized medicine and early intervention strategies.

Abstract: Lung cancer (LC) is the third most common cancer and the leading cause of cancer deaths in the US. Although smoking is the primary risk factor, the occurrence of LC in never-smokers and familial aggregation studies highlight a genetic component. Genetic biomarkers identified through genome-wide association studies (GWAS) are promising tools for assessing LC risk. We introduce HEMERA (Human-Explainable Transformer Model for Estimating Lung Cancer Risk using GWAS Data), a new framework that applies explainable transformer-based deep learning to GWAS data of single nucleotide polymorphisms (SNPs) for predicting LC risk. Unlike prior approaches, HEMERA directly processes raw genotype data without clinical covariates, introducing additive positional encodings, neural genotype embeddings, and refined variant filtering. A post hoc explainability module based on Layer-wise Integrated Gradients enables attribution of model predictions to specific SNPs, aligning strongly with known LC risk loci. Trained on data from 27,254 Million Veteran Program participants, HEMERA achieved >99% AUC (area under receiver characteristics) score. These findings support transparent, hypothesis-generating models for personalized LC risk assessment and early intervention.

Method: Chisme uses cosine similarity-based data affinity heuristics calculated from received model exchanges to determine how much influence received models have when merging into the local model, enabling strategic collaboration between clients.

Result: Chisme outperforms state-of-the-art edge intelligence approaches in almost every case, showing faster training convergence, lower final loss after training, and lower performance disparity between clients in image recognition and time-series prediction scenarios.

Conclusion: Chisme effectively addresses data heterogeneity challenges in distributed edge learning by enabling clients to balance between broader collaboration for general knowledge and selective collaboration for specific knowledge through similarity-based model merging.

Abstract: As end-user device capability increases and demand for intelligent services at the Internet’s edge rise, distributed learning has emerged as a key enabling technology. Existing approaches like federated learning (FL) and decentralized FL (DFL) enable distributed learning among clients, while gossip learning (GL) approaches have emerged to address the potential challenges in resource-constrained, connectivity-challenged infrastructure-less environments. However, most distributed learning approaches assume largely homogeneous data distributions and may not consider or exploit the heterogeneity of clients and their underlying data distributions. This paper introduces Chisme, a novel fully decentralized distributed learning algorithm designed to address the challenges of implementing robust intelligence in network edge contexts characterized by heterogeneous data distributions, episodic connectivity, and sparse network infrastructure. Chisme leverages cosine similarity-based data affinity heuristics calculated from received model exchanges to inform how much influence received models have when merging into the local model. By doing so, it facilitates stronger merging influence between clients with more similar model learning progressions, enabling clients to strategically balance between broader collaboration to build more general knowledge and more selective collaboration to build specific knowledge. We evaluate Chisme against contemporary approaches using image recognition and time-series prediction scenarios while considering different network connectivity conditions, representative of real-world distributed intelligent systems. Our experiments demonstrate that Chisme outperforms state-of-the-art edge intelligence approaches in almost every case – clients using Chisme exhibit faster training convergence, lower final loss after training, and lower performance disparity between clients.

Method: Formulates task offloading as a Markov Decision Process and uses Q-learning technique to enable wearable devices to make optimal offloading decisions without prior knowledge. Evaluated through extensive ns-3 network simulations.

Result: The framework effectively balances energy consumption and task accomplishment time. Performance analysis shows how varying Q-learning parameters affects task completion time, energy usage, and offloading percentage.

Conclusion: The RL-based approach provides an effective solution for task offloading in IoWT, enabling wearable devices to optimize resource usage while maintaining performance requirements for latency-critical applications.

Abstract: Over the years, significant contributions have been made by the research and industrial sectors to improve wearable devices towards the Internet of Wearable Things (IoWT) paradigm. However, wearables are still facing several challenges. Many stem from the limited battery power and insufficient computation resources available on wearable devices. On the other hand, with the popularity of smart wearables, there is a consistent increase in the development of new computationally intensive and latency-critical applications. In such a context, task offloading allows wearables to leverage the resources available on nearby edge devices to enhance the overall user experience. This paper proposes a framework for Reinforcement Learning (RL)-based task offloading in the IoWT. We formulate the task offloading process considering the tradeoff between energy consumption and task accomplishment time. Moreover, we model the task offloading problem as a Markov Decision Process (MDP) and utilize the Q-learning technique to enable the wearable device to make optimal task offloading decisions without prior knowledge. We evaluate the performance of the proposed framework through extensive simulations for various applications and system configurations conducted in the ns-3 network simulator. We also show how varying the main system parameters of the Q-learning algorithm affects the overall performance in terms of average task accomplishment time, average energy consumption, and percentage of tasks offloaded.

Method: Quantizes token surprisals into interpretable states, estimates state-transition matrices for test text, and scores using generalized Jensen-Shannon gap between test transitions and fixed human vs. machine references from historical corpora.

Result: SurpMark consistently matches or surpasses baseline methods across multiple datasets, source models, and scenarios, with experiments confirming the statistic’s asymptotic normality and validating the discretization approach.

Conclusion: The proposed SurpMark detector provides an effective and principled approach for machine-generated text detection that works well under practical constraints and model mismatches.

Abstract: We study black-box detection of machine-generated text under practical constraints: the scoring model (proxy LM) may mismatch the unknown source model, and per-input contrastive generation is costly. We propose SurpMark, a reference-based detector that summarizes a passage by the dynamics of its token surprisals. SurpMark quantizes surprisals into interpretable states, estimates a state-transition matrix for the test text, and scores it via a generalized Jensen-Shannon (GJS) gap between the test transitions and two fixed references (human vs. machine) built once from historical corpora. We prove a principled discretization criterion and establish the asymptotic normality of the decision statistic. Empirically, across multiple datasets, source models, and scenarios, SurpMark consistently matches or surpasses baselines; our experiments corroborate the statistic’s asymptotic normality, and ablations validate the effectiveness of the proposed discretization.

Method: Introduces PEAR benchmark for systematically evaluating planner-executor MAS through extensive experiments, focusing on utility and vulnerability assessment across different system configurations.

Result: Found that: (1) weak planners degrade clean task performance more than weak executors; (2) planner memory is essential while executor memory doesn’t impact clean performance; (3) trade-off exists between task performance and robustness; (4) planner-targeted attacks are particularly effective.

Conclusion: These findings provide actionable insights for enhancing MAS robustness and lay groundwork for principled defenses in multi-agent settings.

Abstract: Large Language Model (LLM)-based Multi-Agent Systems (MAS) have emerged as a powerful paradigm for tackling complex, multi-step tasks across diverse domains. However, despite their impressive capabilities, MAS remain susceptible to adversarial manipulation. Existing studies typically examine isolated attack surfaces or specific scenarios, leaving a lack of holistic understanding of MAS vulnerabilities. To bridge this gap, we introduce PEAR, a benchmark for systematically evaluating both the utility and vulnerability of planner-executor MAS. While compatible with various MAS architectures, our benchmark focuses on the planner-executor structure, which is a practical and widely adopted design. Through extensive experiments, we find that (1) a weak planner degrades overall clean task performance more severely than a weak executor; (2) while a memory module is essential for the planner, having a memory module for the executor does not impact the clean task performance; (3) there exists a trade-off between task performance and robustness; and (4) attacks targeting the planner are particularly effective at misleading the system. These findings offer actionable insights for enhancing the robustness of MAS and lay the groundwork for principled defenses in multi-agent settings.

Method: Multimodal co-training framework that leverages unlabeled data, promotes agreement between classifiers for different modalities, and maintains conditional view independence.

Result: Theoretical analysis shows significant generalization improvements, convergence analysis confirms error reduction, and novel generalization bound quantifies advantages from unlabeled multimodal data, inter-view agreement, and view independence.

Conclusion: Multimodal co-training provides practical benefits for developing robust AI systems that can generalize effectively in dynamic environments, with theoretical foundations advancing established co-training principles.

Abstract: This paper explores a multimodal co-training framework designed to enhance model generalization in situations where labeled data is limited and distribution shifts occur. We thoroughly examine the theoretical foundations of this framework, deriving conditions under which the use of unlabeled data and the promotion of agreement between classifiers for different modalities lead to significant improvements in generalization. We also present a convergence analysis that confirms the effectiveness of iterative co-training in reducing classification errors. In addition, we establish a novel generalization bound that, for the first time in a multimodal co-training context, decomposes and quantifies the distinct advantages gained from leveraging unlabeled multimodal data, promoting inter-view agreement, and maintaining conditional view independence. Our findings highlight the practical benefits of multimodal co-training as a structured approach to developing data-efficient and robust AI systems that can effectively generalize in dynamic, real-world environments. The theoretical foundations are examined in dialogue with, and in advance of, established co-training principles.

Method: MLLM4TS integrates a vision branch where each time-series channel is rendered as color-coded line plots in composite images. It uses temporal-aware visual patch alignment to align visual patches with corresponding time segments, fusing fine-grained temporal details from numerical data with global contextual information from visual representations.

Result: Extensive experiments on standard benchmarks demonstrate MLLM4TS’s effectiveness across both predictive tasks (classification) and generative tasks (anomaly detection and forecasting), showing robust performance.

Conclusion: The results highlight the potential of integrating visual modalities with pretrained language models to achieve robust and generalizable time-series analysis, bridging the gap between numerical data and natural language understanding.

Abstract: Effective analysis of time series data presents significant challenges due to the complex temporal dependencies and cross-channel interactions in multivariate data. Inspired by the way human analysts visually inspect time series to uncover hidden patterns, we ask: can incorporating visual representations enhance automated time-series analysis? Recent advances in multimodal large language models have demonstrated impressive generalization and visual understanding capability, yet their application to time series remains constrained by the modality gap between continuous numerical data and discrete natural language. To bridge this gap, we introduce MLLM4TS, a novel framework that leverages multimodal large language models for general time-series analysis by integrating a dedicated vision branch. Each time-series channel is rendered as a horizontally stacked color-coded line plot in one composite image to capture spatial dependencies across channels, and a temporal-aware visual patch alignment strategy then aligns visual patches with their corresponding time segments. MLLM4TS fuses fine-grained temporal details from the numerical data with global contextual information derived from the visual representation, providing a unified foundation for multimodal time-series analysis. Extensive experiments on standard benchmarks demonstrate the effectiveness of MLLM4TS across both predictive tasks (e.g., classification) and generative tasks (e.g., anomaly detection and forecasting). These results underscore the potential of integrating visual modalities with pretrained language models to achieve robust and generalizable time-series analysis.

Method: Proposes a framework using continuous wavelet transform (CWT) to generate time-frequency maps that capture transient and oscillatory patterns across frequency bands relevant to sleep staging, combined with ensemble learning for classification.

Result: The method achieved 88.37% overall accuracy and 73.15 macro-averaged F1 score on the Sleep-EDF Expanded Database, outperforming conventional machine learning methods and showing comparable or superior performance to recent deep learning approaches.

Conclusion: Wavelet analysis provides a robust, interpretable, and clinically applicable approach for sleep stage classification, demonstrating strong potential for automated sleep scoring systems.

Abstract: Accurate classification of sleep stages is crucial for the diagnosis and management of sleep disorders. Conventional approaches for sleep scoring rely on manual annotation or features extracted from EEG signals in the time or frequency domain. This study proposes a novel framework for automated sleep stage scoring using time-frequency analysis based on the wavelet transform. The Sleep-EDF Expanded Database (sleep-cassette recordings) was used for evaluation. The continuous wavelet transform (CWT) generated time-frequency maps that capture both transient and oscillatory patterns across frequency bands relevant to sleep staging. Experimental results demonstrate that the proposed wavelet-based representation, combined with ensemble learning, achieves an overall accuracy of 88.37 percent and a macro-averaged F1 score of 73.15, outperforming conventional machine learning methods and exhibiting comparable or superior performance to recent deep learning approaches. These findings highlight the potential of wavelet analysis for robust, interpretable, and clinically applicable sleep stage classification.

Method: FairSpecTemp is an optimization-based method with two variants: one exploits commutativity properties of graph stationarity while directly constraining bias, and the other restricts bias in the graph spectrum to implicitly encourage fair estimates.

Result: The methods provide high probability performance bounds and show that accuracy need not be sacrificed to recover fair graphs. Evaluation on synthetic and real-world datasets demonstrates effectiveness.

Conclusion: FairSpecTemp successfully estimates fair graphs from stationary observations, with both variants showing advantages in different scenarios, and reveals that fairness can be achieved without sacrificing accuracy.

Abstract: We estimate fair graphs from graph-stationary nodal observations such that connections are not biased with respect to sensitive attributes. Edges in real-world graphs often exhibit preferences for connecting certain pairs of groups. Biased connections can not only exacerbate but even induce unfair treatment for downstream graph-based tasks. We therefore consider group and individual fairness for graphs corresponding to group- and node-level definitions, respectively. To evaluate the fairness of a given graph, we provide multiple bias metrics, including novel measurements in the spectral domain. Furthermore, we propose Fair Spectral Templates (FairSpecTemp), an optimization-based method with two variants for estimating fair graphs from stationary graph signals, a general model for graph data subsuming many existing ones. One variant of FairSpecTemp exploits commutativity properties of graph stationarity while directly constraining bias, while the other implicitly encourages fair estimates by restricting bias in the graph spectrum and is thus more flexible. Our methods enjoy high probability performance bounds, yielding a conditional tradeoff between fairness and accuracy. In particular, our analysis reveals that accuracy need not be sacrificed to recover fair graphs. We evaluate FairSpecTemp on synthetic and real-world data sets to illustrate its effectiveness and highlight the advantages of both variants of FairSpecTemp.

Method: Memory-based flow map learning using short bursts of trajectory data from repeated full digital twin executions, creating data-driven models of quantities of interest entirely offline.

Result: Successfully applied to computational fluid dynamics (2D incompressible flow past a cylinder), creating compact dynamical systems that accurately predict hydrodynamic forces without flow field knowledge.

Conclusion: The targeted digital twin framework enables substantial computational savings by bypassing full simulations while maintaining accurate long-term predictions of quantities of interest.

Abstract: We present a numerical framework for constructing a targeted digital twin (tDT) that directly models the dynamics of quantities of interest (QoIs) in a full digital twin (DT). The proposed approach employs memory-based flow map learning (FML) to develop a data-driven model of the QoIs using short bursts of trajectory data generated through repeated executions of the full DT. This renders the construction of the FML-based tDT an entirely offline computational process. During online simulation, the learned tDT can efficiently predict and analyze the long-term dynamics of the QoIs without requiring simulations of the full DT system, thereby achieving substantial computational savings. After introducing the general numerical procedure, we demonstrate the construction and predictive capability of the tDT in a computational fluid dynamics (CFD) example: two-dimensional incompressible flow past a cylinder. The QoIs in this problem are the hydrodynamic forces exerted on the cylinder. The resulting tDTs are compact dynamical systems that evolve these forces without explicit knowledge of the underlying flow field. Numerical results show that the tDTs yield accurate long-term predictions of the forces while entirely bypassing full flow simulations.

Method: Extends vanilla self-imitation learning with curriculum-based entropy management, using intrinsic rewards for skill-level exploration and self-imitation for action-level exploration, plus replay buffer recalibration and regularization techniques.

Result: Achieves progressive exploration-exploitation balance without entropy collapsing or runaway divergence, enabling stable training and accelerated solution iteration.

Conclusion: SPEAR provides an effective framework for training agentic LLMs that maintains balanced entropy across training stages while stabilizing RL training through curriculum-based self-imitation learning.

Abstract: Reinforcement learning (RL) is the dominant paradigm for sharpening strategic tool use capabilities of LLMs on long-horizon, sparsely-rewarded agent tasks, yet it faces a fundamental challenge of exploration-exploitation trade-off. Existing studies stimulate exploration through the lens of policy entropy, but such mechanical entropy maximization is prone to RL training instability due to the multi-turn distribution shifting. In this paper, we target the progressive exploration-exploitation balance under the guidance of the agent own experiences without succumbing to either entropy collapsing or runaway divergence. We propose SPEAR, a curriculum-based self-imitation learning (SIL) recipe for training agentic LLMs. It extends the vanilla SIL framework, where a replay buffer stores self-generated promising trajectories for off-policy update, by gradually steering the policy evolution within a well-balanced range of entropy across stages. Specifically, our approach incorporates a curriculum to manage the exploration process, utilizing intrinsic rewards to foster skill-level exploration and facilitating action-level exploration through SIL. At first, the auxiliary tool call reward plays a critical role in the accumulation of tool-use skills, enabling broad exposure to the unfamiliar distributions of the environment feedback with an upward entropy trend. As training progresses, self-imitation gets strengthened to exploit existing successful patterns from replayed experiences for comparative action-level exploration, accelerating solution iteration without unbounded entropy growth. To further stabilize training, we recalibrate the advantages of experiences in the replay buffer to address the potential policy drift. Reugularizations such as the clipping of tokens with high covariance between probability and advantage are introduced to the trajectory-level entropy control to curb over-confidence.

Method: Analyze gradient descent with dropout on two-layer neural networks with mean-field initialization scale, studying large-width asymptotics across different dropout rates, learning rates, and width magnitudes.

Result: Identified five distinct nondegenerate asymptotic phases. Found that dropout’s penalty effect only persists with impractically small learning rates (O(1/width)). For larger learning rates, dropout becomes equivalent to random geometry techniques, described by mean-field jump processes with neurons updating via independent Poisson/Bernoulli clocks.

Conclusion: Provides new theoretical framework for understanding dropout in large-scale neural networks, revealing complex phase behavior and challenging conventional views about dropout’s penalty mechanism in practical settings.

Abstract: Dropout is a standard training technique for neural networks that consists of randomly deactivating units at each step of their gradient-based training. It is known to improve performance in many settings, including in the large-scale training of language or vision models. As a first step towards understanding the role of dropout in large neural networks, we study the large-width asymptotics of gradient descent with dropout on two-layer neural networks with the mean-field initialization scale. We obtain a rich asymptotic phase diagram that exhibits five distinct nondegenerate phases depending on the relative magnitudes of the dropout rate, the learning rate, and the width. Notably, we find that the well-studied “penalty” effect of dropout only persists in the limit with impractically small learning rates of order $O(1/\text{width})$. For larger learning rates, this effect disappears and in the limit, dropout is equivalent to a “random geometry” technique, where the gradients are thinned randomly after the forward and backward pass have been computed. In this asymptotic regime, the limit is described by a mean-field jump process where the neurons’ update times follow independent Poisson or Bernoulli clocks (depending on whether the learning rate vanishes or not). For some of the phases, we obtain a description of the limit dynamics both in path-space and in distribution-space. The convergence proofs involve a mix of tools from mean-field particle systems and stochastic processes. Together, our results lay the groundwork for a renewed theoretical understanding of dropout in large-scale neural networks.

Method: Applied BERTopic (transformer-based topic modeling) to lmsys-chat-1m dataset with robust preprocessing for multilingual variation, dialogue balancing, and data cleaning. Used visualization techniques including inter-topic distance maps, probability distributions, and model-topic matrices.

Result: Identified 29 coherent topics including artificial intelligence, programming, ethics, and cloud infrastructure. Analyzed relationships between topics and model preferences to identify trends in model-topic alignment.

Conclusion: Findings provide insights for domain-specific fine-tuning and optimization strategies to improve real-world LLM performance and user satisfaction based on topic-preference relationships.

Abstract: This study applies BERTopic, a transformer-based topic modeling technique, to the lmsys-chat-1m dataset, a multilingual conversational corpus built from head-to-head evaluations of large language models (LLMs). Each user prompt is paired with two anonymized LLM responses and a human preference label, used to assess user evaluation of competing model outputs. The main objective is uncovering thematic patterns in these conversations and examining their relation to user preferences, particularly if certain LLMs are consistently preferred within specific topics. A robust preprocessing pipeline was designed for multilingual variation, balancing dialogue turns, and cleaning noisy or redacted data. BERTopic extracted over 29 coherent topics including artificial intelligence, programming, ethics, and cloud infrastructure. We analysed relationships between topics and model preferences to identify trends in model-topic alignment. Visualization techniques included inter-topic distance maps, topic probability distributions, and model-versus-topic matrices. Our findings inform domain-specific fine-tuning and optimization strategies for improving real-world LLM performance and user satisfaction.

Method: Proposes EBGAN-MDN framework that combines energy-based models, Mixture Density Networks (MDNs), and adversarial training using a modified InfoNCE loss and energy-enforced MDN loss.

Result: Experiments on synthetic and robotic benchmarks demonstrate superior performance compared to existing approaches.

Conclusion: EBGAN-MDN is established as an effective and efficient solution for multi-modal learning tasks, successfully addressing mode averaging and mode collapse problems.

Abstract: Multi-modal behavior cloning faces significant challenges due to mode averaging and mode collapse, where traditional models fail to capture diverse input-output mappings. This problem is critical in applications like robotics, where modeling multiple valid actions ensures both performance and safety. We propose EBGAN-MDN, a framework that integrates energy-based models, Mixture Density Networks (MDNs), and adversarial training. By leveraging a modified InfoNCE loss and an energy-enforced MDN loss, EBGAN-MDN effectively addresses these challenges. Experiments on synthetic and robotic benchmarks demonstrate superior performance, establishing EBGAN-MDN as a effective and efficient solution for multi-modal learning tasks.

Method: Use Optimal Transport distances to measure similarity between unlabeled tabular datasets and recommend ML pipelines.

Result: LOTUS shows effectiveness against strong baselines in outlier detection and clustering tasks.

Conclusion: LOTUS is a promising first step toward model selection for multiple unsupervised ML tasks.

Abstract: In this work, we present LOTUS (Learning to Learn with Optimal Transport for Unsupervised Scenarios), a simple yet effective method to perform model selection for multiple unsupervised machine learning(ML) tasks such as outlier detection and clustering. Our intuition behind this work is that a machine learning pipeline will perform well in a new dataset if it previously worked well on datasets with a similar underlying data distribution. We use Optimal Transport distances to find this similarity between unlabeled tabular datasets and recommend machine learning pipelines with one unified single method on two downstream unsupervised tasks: outlier detection and clustering. We present the effectiveness of our approach with experiments against strong baselines and show that LOTUS is a very promising first step toward model selection for multiple unsupervised ML tasks.

Method: Proposed Symbolic Diffusion - a D3PM based discrete state-space diffusion model that uses discrete token diffusion to generate all equation tokens at once, compared against SymbolicGPT’s autoregressive approach.

Result: The diffusion-based approach achieved comparable and sometimes improved performance over autoregressive generation using similar encoder and transformer architectures on the bivariate SymbolicGPT dataset.

Conclusion: Diffusion-based generation offers promising alternatives to autoregressive models for symbolic regression, opening new research directions in neural-network based equation discovery.

Abstract: Symbolic regression refers to the task of finding a closed-form mathematical expression to fit a set of data points. Genetic programming based techniques are the most common algorithms used to tackle this problem, but recently, neural-network based approaches have gained popularity. Most of the leading neural-network based models used for symbolic regression utilize transformer-based autoregressive models to generate an equation conditioned on encoded input points. However, autoregressive generation is limited to generating tokens left-to-right, and future generated tokens are conditioned only on previously generated tokens. Motivated by the desire to generate all tokens simultaneously to produce improved closed-form equations, we propose Symbolic Diffusion, a D3PM based discrete state-space diffusion model which simultaneously generates all tokens of the equation at once using discrete token diffusion. Using the bivariate dataset developed for SymbolicGPT, we compared our diffusion-based generation approach to an autoregressive model based on SymbolicGPT, using equivalent encoder and transformer architectures. We demonstrate that our novel approach of using diffusion-based generation for symbolic regression can offer comparable and, by some metrics, improved performance over autoregressive generation in models using similar underlying architectures, opening new research opportunities in neural-network based symbolic regression.

Method: Systematic review of existing research analyzing basic principles, mathematical models, key characteristics, and challenges of LNNs and RNN variants.

Result: LNNs show significant potential for handling noisy, non-stationary data and achieving OOD generalization, with some variants outperforming RNNs in parameter efficiency and computational speed.

Conclusion: While RNNs remain foundational due to mature ecosystem, LNNs offer promising advantages; future research should focus on improving LNN scalability for broader applications.

Abstract: This review aims to conduct a comparative analysis of liquid neural networks (LNNs) and traditional recurrent neural networks (RNNs) and their variants, such as long short-term memory networks (LSTMs) and gated recurrent units (GRUs). The core dimensions of the analysis include model accuracy, memory efficiency, and generalization ability. By systematically reviewing existing research, this paper explores the basic principles, mathematical models, key characteristics, and inherent challenges of these neural network architectures in processing sequential data. Research findings reveal that LNN, as an emerging, biologically inspired, continuous-time dynamic neural network, demonstrates significant potential in handling noisy, non-stationary data, and achieving out-of-distribution (OOD) generalization. Additionally, some LNN variants outperform traditional RNN in terms of parameter efficiency and computational speed. However, RNN remains a cornerstone in sequence modeling due to its mature ecosystem and successful applications across various tasks. This review identifies the commonalities and differences between LNNs and RNNs, summarizes their respective shortcomings and challenges, and points out valuable directions for future research, particularly emphasizing the importance of improving the scalability of LNNs to promote their application in broader and more complex scenarios.

Method: Proposed ExpA with routing actions to switch between language and external environments, and ExpA Reinforcement Learning (EARL) with counterfactual policy optimization for effective exploration.

Result: EARL outperforms vocabulary-constrained baselines on multi-turn interaction tasks, achieves perfect Sort-4 accuracy, and self-discovers efficient algorithms competitive with classical designs.

Conclusion: ExpA successfully decouples environment interactions from language, enabling more efficient and robust multi-turn interactions with external environments.

Abstract: Large Language Models (LLMs) are powerful reasoners in natural language, but their actions are typically confined to outputting vocabulary tokens. As a result, interactions with external environments – such as symbolic operators or simulators – must be expressed through text in predefined formats, parsed, and routed to external interfaces. This overloads the model’s language with both reasoning and control duties, and requires a hand-crafted parser, external to the LLM. To address this, we decouple environment interactions from language by internalizing them in an Expanded Action space (ExpA), beyond the vocabulary. The model starts reasoning in the default language environment, but may trigger routing actions and switch to an external environment at any time. From there, the model can only invoke environment-specific actions, receive feedback from the environment, and potentially route back to language as a result. To promote effective exploration of the expanded action space and new environments, we introduce ExpA Reinforcement Learning (EARL) with counterfactual policy optimization. On tasks requiring multi-turn interactions and contingent planning, EARL outperforms strong baselines with vocabulary-constrained actions. It performs robustly across calculator-based multi-task learning and, in the partially observed sorting problem, achieves perfect Sort-4 accuracy while self-discovering an efficient algorithm competitive with classical designs.

Method: Developed Temporal Graph Modelling (TGM) library with first-class support for dynamic node features, time-granularity conversions, and native handling of link-, node-, and graph-level tasks, unifying CTDG and DTDG approaches.

Result: TGM achieves 7.8x average speedup over DyGLib and 175x speedup on graph discretization. It enables new research possibilities like dynamic graph property prediction and time-driven training paradigms.

Conclusion: TGM addresses infrastructure gaps in temporal graph ML, providing unified, efficient framework that unlocks previously impractical research questions and facilitates direct comparisons between different temporal graph approaches.

Abstract: Well-designed open-source software drives progress in Machine Learning (ML) research. While static graph ML enjoys mature frameworks like PyTorch Geometric and DGL, ML for temporal graphs (TG), networks that evolve over time, lacks comparable infrastructure. Existing TG libraries are often tailored to specific architectures, hindering support for diverse models in this rapidly evolving field. Additionally, the divide between continuous- and discrete-time dynamic graph methods (CTDG and DTDG) limits direct comparisons and idea transfer. To address these gaps, we introduce Temporal Graph Modelling (TGM), a research-oriented library for ML on temporal graphs, the first to unify CTDG and DTDG approaches. TGM offers first-class support for dynamic node features, time-granularity conversions, and native handling of link-, node-, and graph-level tasks. Empirically, TGM achieves an average 7.8x speedup across multiple models, datasets, and tasks compared to the widely used DyGLib, and an average 175x speedup on graph discretization relative to available implementations. Beyond efficiency, we show in our experiments how TGM unlocks entirely new research possibilities by enabling dynamic graph property prediction and time-driven training paradigms, opening the door to questions previously impractical to study. TGM is available at https://github.com/tgm-team/tgm

Method: The authors introduce an incremental synthetic data benchmark and propose an Attention-Focused Transformer model that uses self-attention mechanisms trained via reconstruction, deriving anomaly scores from deviations in learned attention weights for computational efficiency.

Result: Transformer-based models generally outperform others, but all models show vulnerability to high-frequency localized noise. The proposed model achieves comparable accuracy to state-of-the-art while offering better inference speed.

Conclusion: The study highlights the critical need for enhanced noise robustness in future iSHM models and positions the attention-based approach as a promising foundation for practical onboard anomaly detection systems.

Abstract: Indirect structural health monitoring (iSHM) for broken rail detection using onboard sensors presents a cost-effective paradigm for railway track assessment, yet reliably detecting small, transient anomalies (2-10 cm) remains a significant challenge due to complex vehicle dynamics, signal noise, and the scarcity of labeled data limiting supervised approaches. This study addresses these issues through unsupervised deep learning. We introduce an incremental synthetic data benchmark designed to systematically evaluate model robustness against progressively complex challenges like speed variations, multi-channel inputs, and realistic noise patterns encountered in iSHM. Using this benchmark, we evaluate several established unsupervised models alongside our proposed Attention-Focused Transformer. Our model employs a self-attention mechanism, trained via reconstruction but innovatively deriving anomaly scores primarily from deviations in learned attention weights, aiming for both effectiveness and computational efficiency. Benchmarking results reveal that while transformer-based models generally outperform others, all tested models exhibit significant vulnerability to high-frequency localized noise, identifying this as a critical bottleneck for practical deployment. Notably, our proposed model achieves accuracy comparable to the state-of-the-art solution while demonstrating better inference speed. This highlights the crucial need for enhanced noise robustness in future iSHM models and positions our more efficient attention-based approach as a promising foundation for developing practical onboard anomaly detection systems.

Method: Uses Gaussian distributions for nodes/edges to propagate semantic signals and uncertainty, HAGH positional encoding with KAN-based attention, ODE-based residual learning for temporal modeling, and robust adaptive ensemble coefficient analysis with cosine/Jaccard similarity for defense.

Result: On Bitcoin-Alpha: 10.77% MCC improvement in single-timeslot prediction, 16.41% MCC gain in cold-start scenario, and up to 11.63% MCC improvement under adversarial on/off attacks.

Conclusion: The unified DGTEN framework effectively addresses dynamic trust evaluation challenges with uncertainty awareness, temporal modeling, and adversarial robustness.

Abstract: Dynamic trust evaluation in large, rapidly evolving graphs requires models that can capture changing relationships, express calibrated confidence, and resist adversarial manipulation. DGTEN (Deep Gaussian-based Trust Evaluation Network) introduces a unified graph framework that achieves all three by combining uncertainty-aware message passing, expressive temporal modeling, and built-in defenses against trust-targeted attacks. It represents nodes and edges as Gaussian distributions so that both semantic signals and epistemic uncertainty propagate through the graph neural network, enabling risk-aware trust decisions rather than overconfident guesses. To model how trust evolves, it employs hybrid Absolute-Gaussian-Hourglass (HAGH) positional encoding with Kolmogorov-Arnold network-based unbiased multi-head attention, followed by an ordinary differential equation (ODE)-based residual learning module to jointly capture abrupt shifts and smooth trends. Robust adaptive ensemble coefficient analysis prunes or down-weights suspicious interactions using complementary cosine and Jaccard similarity measures, mitigating reputation laundering, sabotage, and on/off attacks. On two signed Bitcoin trust networks, DGTEN delivers significant improvements: in single-timeslot prediction on Bitcoin-Alpha, it improves MCC by 10.77% over the best dynamic baseline; in the cold-start scenario, it achieves a 16.41% MCC gain - the largest across all tasks and datasets. Under adversarial on/off attacks, it surpasses the baseline by up to 11.63% MCC. These results validate the effectiveness of the unified DGTEN framework.

Method: The authors develop a taxonomy of 12 stateful unlearning methods grouped into three families: divergence-driven optimization, representation misalignment, and rejection-based targeted unlearning. They introduce open question-answering metrics to better evaluate generative performance and analyze robustness across different attack scenarios.

Result: Analysis reveals that current MCQ-based evaluations provide limited perspective and overstate success. The new Open-QA metrics better capture the UE-UT tradeoff across method families. Robustness analysis shows distinct vulnerabilities between in-domain relearning and out-of-domain fine-tuning attacks.

Conclusion: The study offers a comprehensive framework for LLM unlearning research, highlighting the need for better evaluation metrics and providing actionable guidance for designing and evaluating future unlearning methods to address safety, privacy, and copyright concerns effectively.

Abstract: Machine unlearning for large language models (LLMs) aims to remove undesired data, knowledge, and behaviors (e.g., for safety, privacy, or copyright) while preserving useful model capabilities. Despite rapid progress over the past two years, research in LLM unlearning remains fragmented, with limited clarity on what constitutes effective unlearning and how it should be rigorously evaluated. In this work, we present a principled taxonomy of twelve recent stateful unlearning methods, grouped into three methodological families: divergence-driven optimization, representation misalignment, and rejection-based targeted unlearning. Building on this taxonomy, we revisit the evaluation of unlearning effectiveness (UE), utility retention (UT), and robustness (Rob), focusing on the WMDP benchmark. Our analysis shows that current evaluations, dominated by multiple-choice question (MCQ) accuracy, offer only a narrow perspective, often overstating success while overlooking the model’s actual generation behavior. To address this gap, we introduce open question-answering (Open-QA) metrics that better capture generative performance and reveal the inherent UE-UT tradeoff across method families. Furthermore, we demonstrate that robustness requires finer-grained analysis: for example, vulnerabilities differ substantially between in-domain relearning and out-of-domain fine-tuning, even though both fall under model-level attacks. Through this study, we hope to deliver a full-stack revisit of LLM unlearning and actionable guidance for designing and evaluating future methods.

Method: Employed K-means and K-medoids clustering techniques on a dataset of over a million properties and hosts from CDRC and AirDNA collaboration.

Result: Identified homogenous groups of vacation rental properties with common characteristics.

Conclusion: The findings enhance comprehension of vacation rental evaluations and could be used for creating targeted, cluster-specific policies.

Abstract: This study advocates for employing clustering techniques to classify vacation rental properties active during the Covid pandemic to identify inherent patterns and behaviours. The dataset, a collaboration between the ESRC funded Consumer Data Research Centre (CDRC) and AirDNA, encompasses data for over a million properties and hosts. Utilising K-means and K-medoids clustering techniques, we identify homogenous groups and their common characteristics. Our findings enhance comprehension of the intricacies of vacation rental evaluations and could potentially be utilised in the creation of targeted, cluster-specific policies.

Method: The authors establish a theoretical Pareto frontier for the trade-off and develop an algorithm called EXP3-N-CS that provides anytime-valid asymptotic confidence sequences to balance regret minimization and inference accuracy.

Result: The paper presents the first theoretical characterization of the Pareto frontier between regret minimization and inference accuracy in adversarial MABNI settings.

Conclusion: The proposed EXP3-N-CS algorithm effectively balances the competing objectives of regret minimization and inference accuracy in multi-armed bandits with network interference.

Abstract: In multi-armed bandits with network interference (MABNI), the action taken by one node can influence the rewards of others, creating complex interdependence. While existing research on MABNI largely concentrates on minimizing regret, it often overlooks the crucial concern that an excessive emphasis on the optimal arm can undermine the inference accuracy for sub-optimal arms. Although initial efforts have been made to address this trade-off in single-unit scenarios, these challenges have become more pronounced in the context of MABNI. In this paper, we establish, for the first time, a theoretical Pareto frontier characterizing the trade-off between regret minimization and inference accuracy in adversarial (design-based) MABNI. We further introduce an anytime-valid asymptotic confidence sequence along with a corresponding algorithm, $\texttt{EXP3-N-CS}$, specifically designed to balance the trade-off between regret minimization and inference accuracy in this setting.

Method: Proposed inter-cluster separation (ICS) regularization that penalizes model outputs distant from centroids of clusters formed by previous task data, with hyperparameter tuning for optimal regularization weighting. Evaluated on 5-task Split CIFAR-10 benchmark using ResNet-18 architecture.

Result: ICS demonstrated effectiveness in maintaining strong performance on initial tasks, but showed limitations in long-term knowledge retention as the number of tasks increased.

Conclusion: The approach highlights the complexity and trade-offs in continual learning, pointing toward the need for further research to improve long-term knowledge retention across increasing numbers of tasks.

Abstract: Continual learning the ability of a neural network to learn multiple sequential tasks without losing previously acquired knowledge remains a significant obstacle to developing truly adaptive artificial intelligence. Deep learning models have achieved remarkable results in various applications, but overfitting remains a common issue. Regularization techniques can help prevent overfitting by adding constraints to the model’s parameters. To prevent catastrophic forgetting, in this paper we introduce a novel regularization technique based on inter-cluster separation (ICS) in the loss function, which penalizes the model for producing outputs that are far away from the centroids of the clusters formed by the data from previous tasks. We also performed hyperparameter tuning to find the optimal weighting of the proposed regularization term. This ensures clearer separation between tasks in the neural network’s internal representation, reducing overlap and mitigating forgetting. Using the standard 5-task Split CIFAR-10 benchmark and a ResNet-18 architecture, we demonstrate ICS’s effectiveness in maintaining strong performance on initial tasks. However, our results also highlight limitations in long-term knowledge retention, particularly when the number of tasks increases. This underscores the complexity and trade-offs inherent in continual learning and points toward avenues for further research.

Method: Proposes flow-matching objective for distributional Bellman equation, uses flow derivative ODE for uncertainty estimation, and prioritizes learning on uncertain transitions.

Result: Achieves 1.3x improvement in success rates on 37 state-based and 25 image-based benchmark tasks compared to prior methods.

Conclusion: Flow-based modeling of return distributions provides stronger learning signals and enables better uncertainty-aware decision making in RL.

Abstract: While most reinforcement learning methods today flatten the distribution of future returns to a single scalar value, distributional RL methods exploit the return distribution to provide stronger learning signals and to enable applications in exploration and safe RL. While the predominant method for estimating the return distribution is by modeling it as a categorical distribution over discrete bins or estimating a finite number of quantiles, such approaches leave unanswered questions about the fine-grained structure of the return distribution and about how to distinguish states with high return uncertainty for decision-making. The key idea in this paper is to use modern, flexible flow-based models to estimate the full future return distributions and identify those states with high return variance. We do so by formulating a new flow-matching objective that generates probability density paths satisfying the distributional Bellman equation. Building upon the learned flow models, we estimate the return uncertainty of distinct states using a new flow derivative ODE. We additionally use this uncertainty information to prioritize learning a more accurate return estimation on certain transitions. We compare our method (Value Flows) with prior methods in the offline and online-to-online settings. Experiments on $37$ state-based and $25$ image-based benchmark tasks demonstrate that Value Flows achieves a $1.3\times$ improvement on average in success rates. Website: https://pd-perry.github.io/value-flows Code: https://github.com/chongyi-zheng/value-flows

Method: Uses a hybrid ensemble framework with multiple feature processing stages including relational, cross, and frequency-based features. Incorporates graph attention, automatic cross-feature creation, and frequency domain transformations. Updates weekly with new data and adjusts model weights using performance-based methods without manual intervention.

Result: Improves model stability and generalization for long-term credit risk prediction tasks.

Conclusion: HYDRA-EI provides an effective solution for long-term loan default prediction that adapts to changing data distributions and borrower behavior through incremental learning and ensemble methods.

Abstract: Predicting long-term loan defaults is hard because borrower behavior often changes and data distributions shift over time. This paper presents HYDRA-EI, a hybrid ensemble incremental learning framework. It uses several stages of feature processing and combines multiple models. The framework builds relational, cross, and frequency-based features. It uses graph attention, automatic cross-feature creation, and transformations from the frequency domain. HYDRA-EI updates weekly using new data and adjusts the model weights with a simple performance-based method. It works without frequent manual changes or fixed retraining. HYDRA-EI improves model stability and generalization, which makes it useful for long-term credit risk tasks.

Method: FedQS uses a divide-and-conquer strategy that classifies clients into four distinct types based on data distribution characteristics and computational resources, then adaptively optimizes their local training to bridge the gap between aggregation strategies.

Result: Extensive experiments on computer vision, NLP, and real-world tasks show FedQS achieves the highest accuracy, lowest loss, and ranks among the fastest in convergence speed, outperforming state-of-the-art baselines.

Conclusion: FedQS bridges the gap between aggregation strategies in semi-asynchronous FL, providing a unified solution for stable, accurate, and efficient federated learning.

Abstract: Federated learning (FL) enables collaborative model training across multiple parties without sharing raw data, with semi-asynchronous FL (SAFL) emerging as a balanced approach between synchronous and asynchronous FL. However, SAFL faces significant challenges in optimizing both gradient-based (e.g., FedSGD) and model-based (e.g., FedAvg) aggregation strategies, which exhibit distinct trade-offs in accuracy, convergence speed, and stability. While gradient aggregation achieves faster convergence and higher accuracy, it suffers from pronounced fluctuations, whereas model aggregation offers greater stability but slower convergence and suboptimal accuracy. This paper presents FedQS, the first framework to theoretically analyze and address these disparities in SAFL. FedQS introduces a divide-and-conquer strategy to handle client heterogeneity by classifying clients into four distinct types and adaptively optimizing their local training based on data distribution characteristics and available computational resources. Extensive experiments on computer vision, natural language processing, and real-world tasks demonstrate that FedQS achieves the highest accuracy, attains the lowest loss, and ranks among the fastest in convergence speed, outperforming state-of-the-art baselines. Our work bridges the gap between aggregation strategies in SAFL, offering a unified solution for stable, accurate, and efficient federated learning. The code and datasets are available at https://anonymous.4open.science/r/FedQS-EDD6.

Method: Two-stage framework: 1) Distill 670B teacher LRM into 30B Mixture-of-Experts model using Rejection Sampling Fine-Tuning to reduce computational cost; 2) Use reinforcement learning with Group Relative Policy Optimization to compress reasoning paths with multi-objective reward function balancing correctness, helpfulness, and brevity.

Result: Achieved 30-fold reduction in computational cost with sub-second latency. In Taobao Live deployment: improved response correctness by 3.3% and helpfulness by 21.8%. Led to statistically significant increase in Gross Merchandise Volume when tested by hundreds of thousands of viewers.

Conclusion: LiveThinking effectively bridges the gap between high-quality reasoning and real-time requirements in interactive e-commerce settings, demonstrating practical enhancement of both user experience and commercial performance.

Abstract: In AI-powered e-commerce livestreaming, digital avatars require real-time responses to drive engagement, a task for which high-latency Large Reasoning Models (LRMs) are ill-suited. We introduce LiveThinking, a practical two-stage optimization framework to bridge this gap. First, we address computational cost by distilling a 670B teacher LRM into a lightweight 30B Mixture-of-Experts (MoE) model (3B active) using Rejection Sampling Fine-Tuning (RFT). This reduces deployment overhead but preserves the teacher’s verbose reasoning, causing latency. To solve this, our second stage employs reinforcement learning with Group Relative Policy Optimization (GRPO) to compress the model’s reasoning path, guided by a multi-objective reward function balancing correctness, helpfulness, and brevity. LiveThinking achieves a 30-fold reduction in computational cost, enabling sub-second latency. In real-world application on Taobao Live, it improved response correctness by 3.3% and helpfulness by 21.8%. Tested by hundreds of thousands of viewers, our system led to a statistically significant increase in Gross Merchandise Volume (GMV), demonstrating its effectiveness in enhancing user experience and commercial performance in live, interactive settings.

Method: Uses walk-stitching technique to concatenate shorter walks without breaking unbiasedness, replaces sequential long walk sampling with parallel computation of short walks and matrix-matrix multiplication, and extends walk termination to general length distributions.

Result: GRFs++ inherit approximation quality of longer walks with greater efficiency, improve approximation accuracy of graph kernels without extra computational cost, and provide better modeling of distant node relationships.

Conclusion: GRFs++ offer a more efficient and accurate alternative to regular GRFs for graph kernel computations, with empirical evaluations and theoretical analysis supporting the claims.

Abstract: We propose refined GRFs (GRFs++), a new class of Graph Random Features (GRFs) for efficient and accurate computations involving kernels defined on the nodes of a graph. GRFs++ resolve some of the long-standing limitations of regular GRFs, including difficulty modeling relationships between more distant nodes. They reduce dependence on sampling long graph random walks via a novel walk-stitching technique, concatenating several shorter walks without breaking unbiasedness. By applying these techniques, GRFs++ inherit the approximation quality provided by longer walks but with greater efficiency, trading sequential, inefficient sampling of a long walk for parallel computation of short walks and matrix-matrix multiplication. Furthermore, GRFs++ extend the simplistic GRFs walk termination mechanism (Bernoulli schemes with fixed halting probabilities) to a broader class of strategies, applying general distributions on the walks’ lengths. This improves the approximation accuracy of graph kernels, without incurring extra computational cost. We provide empirical evaluations to showcase all our claims and complement our results with theoretical analysis.

Method: DEAS leverages action sequences (temporally extended actions) interpreted through the options framework via semi-Markov decision process Q-learning, and uses detached value learning to steer value estimates toward in-distribution actions that achieve high return.

Result: DEAS consistently outperforms baselines on complex, long-horizon tasks from OGBench and significantly boosts performance of Vision-Language-Action models in both RoboCasa Kitchen simulation and real-world manipulation tasks.

Conclusion: DEAS provides an effective offline RL framework that successfully addresses value overestimation while leveraging action sequences to reduce planning horizon and improve performance on complex tasks.

Abstract: Offline reinforcement learning (RL) presents an attractive paradigm for training intelligent agents without expensive online interactions. However, current approaches still struggle with complex, long-horizon sequential decision making. In this work, we introduce DEtached value learning with Action Sequence (DEAS), a simple yet effective offline RL framework that leverages action sequences for value learning. These temporally extended actions provide richer information than single-step actions and can be interpreted through the options framework via semi-Markov decision process Q-learning, enabling reduction of the effective planning horizon by considering longer sequences at once. However, directly adopting such sequences in actor-critic algorithms introduces excessive value overestimation, which we address through detached value learning that steers value estimates toward in-distribution actions that achieve high return in the offline dataset. We demonstrate that DEAS consistently outperforms baselines on complex, long-horizon tasks from OGBench and can be applied to enhance the performance of large-scale Vision-Language-Action models that predict action sequences, significantly boosting performance in both RoboCasa Kitchen simulation tasks and real-world manipulation tasks.

Method: Two-stage coarse-to-fine approach: 1) S$^2$TDiff diffusion model learns behavioral patterns from latent movement sequences, 2) Coarse2FineNet Transformer architecture generates fine-grained trajectories with dynamic context fusion and multi-task decoding.

Result: Outperforms state-of-the-art models on four real-world datasets, achieving over 69% improvement in distance metrics and 55% in radius metrics on FS-TKY dataset.

Conclusion: GeoGen effectively generates high-fidelity synthetic LBSN trajectories while preserving privacy, addressing challenges of sparse activities and uncertain human mobility patterns.

Abstract: Location-Based Social Network (LBSN) check-in trajectory data are important for many practical applications, like POI recommendation, advertising, and pandemic intervention. However, the high collection costs and ever-increasing privacy concerns prevent us from accessing large-scale LBSN trajectory data. The recent advances in synthetic data generation provide us with a new opportunity to achieve this, which utilizes generative AI to generate synthetic data that preserves the characteristics of real data while ensuring privacy protection. However, generating synthetic LBSN check-in trajectories remains challenging due to their spatially discrete, temporally irregular nature and the complex spatio-temporal patterns caused by sparse activities and uncertain human mobility. To address this challenge, we propose GeoGen, a two-stage coarse-to-fine framework for large-scale LBSN check-in trajectory generation. In the first stage, we reconstruct spatially continuous, temporally regular latent movement sequences from the original LBSN check-in trajectories and then design a Sparsity-aware Spatio-temporal Diffusion model (S$^2$TDiff) with an efficient denosing network to learn their underlying behavioral patterns. In the second stage, we design Coarse2FineNet, a Transformer-based Seq2Seq architecture equipped with a dynamic context fusion mechanism in the encoder and a multi-task hybrid-head decoder, which generates fine-grained LBSN trajectories based on coarse-grained latent movement sequences by modeling semantic relevance and behavioral uncertainty. Extensive experiments on four real-world datasets show that GeoGen excels state-of-the-art models for both fidelity and utility evaluation, e.g., it increases over 69% and 55% in distance and radius metrics on the FS-TKY dataset.

Method: Introduces MeSH scheme with explicit memory buffer and lightweight routers to dynamically diversify computation across iterations, enabling functional specialization.

Result: On Pythia suite (160M-1.4B), MeSH-enhanced recursive transformers outperform larger non-recursive models at 1.4B scale with +1.06% average downstream accuracy and 33% fewer non-embedding parameters.

Conclusion: MeSH provides a scalable and principled architecture for building stronger recursive models by resolving key computational pathologies.

Abstract: Recursive transformers reuse parameters and iterate over hidden states multiple times, decoupling compute depth from parameter depth. However, under matched compute, recursive models with fewer parameters often lag behind non-recursive counterparts. By probing hidden states, we trace this performance gap to two primary bottlenecks: undifferentiated computation, where the core is forced to adopt a similar computational pattern at every iteration, and information overload, where long-lived and transient information must coexist in a single hidden state. To address the issues, we introduce a Memory-as-State-Highways (MeSH) scheme, which externalizes state management into an explicit memory buffer and employs lightweight routers to dynamically diversify computation across iterations. Probing visualizations confirm that MeSH successfully resolves the pathologies by inducing functional specialization across iterations. On the Pythia suite (160M-1.4B), MeSH-enhanced recursive transformers consistently improve over recursive baselines and outperforms its larger non-recursive counterpart at the 1.4B scale, improving average downstream accuracy by +1.06% with 33% fewer non-embedding parameters. Our analysis establishes MeSH as a scalable and principled architecture for building stronger recursive models.

Method: Theoretical proof and practical demonstration of t-SNE’s failure modes in inferring clustering strength and outlier detection.

Result: Proved that t-SNE cannot reliably show (1) input clustering strength and (2) extremity of outlier points, with practical examples confirming these limitations.

Conclusion: t-SNE visualizations should not be trusted for inferring input data clustering strength or outlier characteristics, as they systematically misrepresent these properties.

Abstract: Central to the widespread use of t-distributed stochastic neighbor embedding (t-SNE) is the conviction that it produces visualizations whose structure roughly matches that of the input. To the contrary, we prove that (1) the strength of the input clustering, and (2) the extremity of outlier points, cannot be reliably inferred from the t-SNE output. We demonstrate the prevalence of these failure modes in practice as well.

Method: FedBook uses a two-phase process: (1) Intra-domain Collaboration - refining low-frequency tokens using high-frequency tokens across clients to enhance domain-specific coherence; (2) Inter-domain Integration - weighting client contributions by semantic distinctiveness during global aggregation to preserve cross-domain diversity.

Result: Extensive experiments on 8 benchmarks across multiple domains and tasks show FedBook consistently outperforms 21 baselines including isolated supervised learning, FL/FGL, federated adaptations of centralized GFMs, and FedGFM techniques.

Conclusion: FedBook successfully addresses the limitations of centralized graph foundation models by providing an effective federated approach that maintains both domain coherence and cross-domain diversity while respecting privacy constraints.

Abstract: Foundation models have shown remarkable cross-domain generalization in language and vision, inspiring the development of graph foundation models (GFMs). However, existing GFMs typically assume centralized access to multi-domain graphs, which is often infeasible due to privacy and institutional constraints. Federated Graph Foundation Models (FedGFMs) address this limitation, but their effectiveness fundamentally hinges on constructing a robust global codebook that achieves intra-domain coherence by consolidating mutually reinforcing semantics within each domain, while also maintaining inter-domain diversity by retaining heterogeneous knowledge across domains. To this end, we propose FedBook, a unified federated graph foundation codebook that systematically aggregates clients’ local codebooks during server-side federated pre-training. FedBook follows a two-phase process: (1) Intra-domain Collaboration, where low-frequency tokens are refined by referencing more semantically reliable high-frequency tokens across clients to enhance domain-specific coherence; and (2) Inter-domain Integration, where client contributions are weighted by the semantic distinctiveness of their codebooks during the aggregation of the global GFM, thereby preserving cross-domain diversity. Extensive experiments on 8 benchmarks across multiple domains and tasks demonstrate that FedBook consistently outperforms 21 baselines, including isolated supervised learning, FL/FGL, federated adaptations of centralized GFMs, and FedGFM techniques.

Method: Proposed Rényi sharpness defined as negative Rényi entropy of loss Hessian eigenvalues, capturing eigenvalue spread. Used reparametrization invariance and data-to-weight perturbation translation to derive generalization bounds. Also developed RSAM as a regularization method.

Result: Strong correlation (Kendall rank correlation) between Rényi sharpness and generalization. RSAM outperforms all existing sharpness-aware minimization methods, achieving up to 2.5% test accuracy gain over classical SAM method.

Conclusion: Rényi sharpness effectively captures generalization properties through eigenvalue distribution analysis, and RSAM provides a practical training method that significantly improves model performance.

Abstract: Sharpness (of the loss minima) is a common measure to investigate the generalization of neural networks. Intuitively speaking, the flatter the landscape near the minima is, the better generalization might be. Unfortunately, the correlation between many existing sharpness measures and the generalization is usually not strong, sometimes even weak. To close the gap between the intuition and the reality, we propose a novel sharpness measure, i.e., \textit{R'enyi sharpness}, which is defined as the negative R'enyi entropy (a generalization of the classical Shannon entropy) of the loss Hessian. The main ideas are as follows: 1) we realize that \textit{uniform} (identical) eigenvalues of the loss Hessian is most desirable (while keeping the sum constant) to achieve good generalization; 2) we employ the \textit{R'enyi entropy} to concisely characterize the extent of the spread of the eigenvalues of loss Hessian. Normally, the larger the spread, the smaller the (R'enyi) entropy. To rigorously establish the relationship between generalization and (R'enyi) sharpness, we provide several generalization bounds in terms of R'enyi sharpness, by taking advantage of the reparametrization invariance property of R'enyi sharpness, as well as the trick of translating the data discrepancy to the weight perturbation. Furthermore, extensive experiments are conducted to verify the strong correlation (in specific, Kendall rank correlation) between the R'enyi sharpness and generalization. Moreover, we propose to use a variant of R'enyi Sharpness as regularizer during training, i.e., R'enyi Sharpness Aware Minimization (RSAM), which turns out to outperform all existing sharpness-aware minimization methods. It is worthy noting that the test accuracy gain of our proposed RSAM method could be as high as nearly 2.5%, compared against the classical SAM method.

Method: Validation-Aligned Multi-task Optimization (VAMO) adaptively assigns task weights based on alignment between per-task training gradients and held-out validation gradient. It includes a periodicity-aware temporal module and couples with an advanced generative auto-bidding backbone to enhance cross-task transfer of seasonal structure.

Result: Extensive experiments on both simulated and large-scale real-world advertising systems consistently demonstrate significant improvements over typical baselines.

Conclusion: VAMO effectively addresses the generalization challenges in volatile bidding environments by steering updates toward validation improvement and better matching deployment objectives, with theoretical guarantees including convergence and alignment analysis.

Abstract: In online advertising, heterogeneous advertiser requirements give rise to numerous customized bidding tasks that are typically optimized independently, resulting in extensive computation and limited data efficiency. Multi-task learning offers a principled framework to train these tasks jointly through shared representations. However, existing multi-task optimization strategies are primarily guided by training dynamics and often generalize poorly in volatile bidding environments. To this end, we present Validation-Aligned Multi-task Optimization (VAMO), which adaptively assigns task weights based on the alignment between per-task training gradients and a held-out validation gradient, thereby steering updates toward validation improvement and better matching deployment objectives. We further equip the framework with a periodicity-aware temporal module and couple it with an advanced generative auto-bidding backbone to enhance cross-task transfer of seasonal structure and strengthen bidding performance. Meanwhile, we provide theoretical insights into the proposed method, e.g., convergence guarantee and alignment analysis. Extensive experiments on both simulated and large-scale real-world advertising systems consistently demonstrate significant improvements over typical baselines, illuminating the effectiveness of the proposed approach.

Method: Developed a layer-wise adaptive modulation scheme that automatically determines optimal modulation levels for different DNN layers, considering their varying importance rather than using uniform modulation across all layers.

Result: Experimental results demonstrate that the proposed scheme can save up to 73.9% of communication latency compared to existing schemes.

Conclusion: The layer-wise adaptive modulation approach effectively reduces communication latency in wireless federated learning by optimizing modulation levels based on layer importance.

Abstract: In wireless federated learning (FL), the clients need to transmit the high-dimensional deep neural network (DNN) parameters through bandwidth-limited channels, which causes the communication latency issue. In this paper, we propose a layer-wise adaptive modulation scheme to save the communication latency. Unlike existing works which assign the same modulation level for all DNN layers, we consider the layers’ importance which provides more freedom to save the latency. The proposed scheme can automatically decide the optimal modulation levels for different DNN layers. Experimental results show that the proposed scheme can save up to 73.9% of communication latency compared with the existing schemes.

Method: Extends weak-form equation learning techniques (WSINDy algorithm) coupled with kernel density estimation to learn Fokker-Planck equations from sparse position measurements of fall armyworm larvae.

Result: Demonstrates the method’s utility on sparse experimental data of fall armyworm position measurements in varied plant resource and infection conditions.

Conclusion: The approach provides effective models for understanding and predicting lepidopteran larval population movement from highly sparse experimental data.

Abstract: Insect species subject to infection, predation, and anisotropic environmental conditions may exhibit preferential movement patterns. Given the innate stochasticity of exogenous factors driving these patterns over short timescales, individual insect trajectories typically obey overdamped stochastic dynamics. In practice, data-driven modeling approaches designed to learn the underlying Fokker-Planck equations from observed insect distributions serve as ideal tools for understanding and predicting such behavior. Understanding dispersal dynamics of crop and silvicultural pests can lead to a better forecasting of outbreak intensity and location, which can result in better pest management. In this work, we extend weak-form equation learning techniques, coupled with kernel density estimation, to learn effective models for lepidopteran larval population movement from highly sparse experimental data. Galerkin methods such as the Weak form Sparse Identification of Nonlinear Dynamics (WSINDy) algorithm have recently proven useful for learning governing equations in several scientific contexts. We demonstrate the utility of the method on a sparse dataset of position measurements of fall armyworms (Spodoptera frugiperda) obtained in simulated agricultural conditions with varied plant resources and infection status.

Method: Proposes HySim-LLM framework with embedding-weighted fine-tuning and manifold-aware denoising, supported by theoretical guarantees for similarity-weighted generalization and denoising performance.

Result: Establishes theoretical foundations: (1) similarity-weighted generalization bound for adaptation under embedding divergence, (2) manifold-based denoising guarantee bounding loss from noisy samples.

Conclusion: Provides mathematically grounded pathway for reliable and interpretable LLM adaptation in biomedical domains, addressing key challenges in PK data extraction.

Abstract: The extraction and standardization of pharmacokinetic (PK) information from scientific literature remain significant challenges in computational pharmacology, which limits the reliability of data-driven models in drug development. Large language models (LLMs) have achieved remarkable progress in text understanding and reasoning, yet their adaptation to structured biomedical data, such as PK tables, remains constrained by heterogeneity, noise, and domain shift. To address these limitations, we propose HySim-LLM, a unified mathematical and computational framework that integrates embedding-weighted fine-tuning and manifold-aware denoising to enhance the robustness and interpretability of LLMs. We establish two theoretical results: (1) a similarity-weighted generalization bound that quantifies adaptation performance under embedding divergence, and (2) a manifold-based denoising guarantee that bounds loss contributions from noisy or off-manifold samples. These theorems provide a principled foundation for fine-tuning LLMs in structured biomedical settings. The framework offers a mathematically grounded pathway toward reliable and interpretable LLM adaptation for biomedical and data-intensive scientific domains.

Method: A two-step framework using selective influence analysis to identify critical neurons encoding the forget-set, then applying second-order optimizer-based unlearning only to those targeted neurons.

Result: SIMU achieves comparable unlearning efficacy while substantially outperforming current methods in retaining the model’s original knowledge and utility.

Conclusion: Selective neuron targeting in machine unlearning enables effective removal of sensitive information while minimizing damage to the model’s original capabilities.

Abstract: The undesired memorization of sensitive information by Large Language Models (LLMs) has emphasized the need for safety mechanisms that can regulate model behavior. This has led to the development of machine unlearning techniques that enable models to precisely forget sensitive and unwanted information. For machine unlearning, first-order and second-order optimizer-based methods have shown significant progress in enabling LLMs to forget targeted information. However, in doing so, these approaches often compromise the model’s original capabilities, resulting in unlearned models that struggle to retain their prior knowledge and overall utility. To address this, we propose Selective Influence Machine Unlearning (SIMU), a two-step framework that enhances second-order optimizer-based unlearning by selectively updating only the critical neurons responsible for encoding the forget-set. By constraining updates to these targeted neurons, SIMU achieves comparable unlearning efficacy while substantially outperforming current methods in retaining the model’s original knowledge.

Method: Two-stage defense approach: (i) pre-generation defense detects harmful queries before response generation begins, and (ii) mid-generation defense monitors partial responses during generation to prevent outputting more harmful content. The LLM is trained to predict harmfulness using specialized prompts.

Result: Extensive experiments across multiple LLM architectures (LLaMA-2-7B, Qwen-2.5-3B-Instruct, and LLaMA-3.2-3B-Instruct) demonstrate that MetaDefense significantly outperforms existing defense mechanisms, achieving robust defense against harmful queries with seen and unseen attack templates.

Conclusion: MetaDefense maintains competitive performance on benign tasks while providing robust defense against jailbreak attacks, enabling early termination of potentially harmful interactions.

Abstract: This paper introduces MetaDefense, a novel framework for defending against finetuning-based jailbreak attacks in large language models (LLMs). We observe that existing defense mechanisms fail to generalize to harmful queries disguised by unseen attack templates, despite LLMs being capable of distinguishing disguised harmful queries in the embedding space. Based on these insights, we propose a two-stage defense approach: (i) pre-generation defense that detects harmful queries before response generation begins, and (ii) mid-generation defense that monitors partial responses during generation to prevent outputting more harmful content. Our MetaDefense trains the LLM to predict the harmfulness of both queries and partial responses using specialized prompts, enabling early termination of potentially harmful interactions. Extensive experiments across multiple LLM architectures (LLaMA-2-7B, Qwen-2.5-3B-Instruct, and LLaMA-3.2-3B-Instruct) demonstrate that MetaDefense significantly outperforms existing defense mechanisms, achieving robust defense against harmful queries with seen and unseen attack templates while maintaining competitive performance on benign tasks. Code is available at https://github.com/ws-jiang/MetaDefense.

Method: Three-step algorithm: (1) identify samples the model struggles with (self-awareness), (2) generate similar examples from uncertain samples (self-data augmentation), (3) use generated samples for test-time fine-tuning (self-improvement). Two variants: TT-SI (self-generated examples) and TT-D (distillation from stronger model).

Result: TT-SI improves performance by +5.48% absolute accuracy gain on average across benchmarks, surpassing standard learning methods while using 68x fewer training samples.

Conclusion: Test-time self-improvement shows promise as a new paradigm for building more capable agents toward self-evolution, demonstrating the potential of self-improvement algorithms at test-time.

Abstract: One paradigm of language model (LM) fine-tuning relies on creating large training datasets, under the assumption that high quantity and diversity will enable models to generalize to novel tasks after post-training. In practice, gathering large sets of data is inefficient, and training on them is prohibitively expensive; worse, there is no guarantee that the resulting model will handle complex scenarios or generalize better. Moreover, existing techniques rarely assess whether a training sample provides novel information or is redundant with the knowledge already acquired by the model, resulting in unnecessary costs. In this work, we explore a new test-time self-improvement method to create more effective and generalizable agentic LMs on-the-fly. The proposed algorithm can be summarized in three steps: (i) first it identifies the samples that model struggles with (self-awareness), (ii) then generates similar examples from detected uncertain samples (self-data augmentation), and (iii) uses these newly generated samples at test-time fine-tuning (self-improvement). We study two variants of this approach: Test-Time Self-Improvement (TT-SI), where the same model generates additional training examples from its own uncertain cases and then learns from them, and contrast this approach with Test-Time Distillation (TT-D), where a stronger model generates similar examples for uncertain cases, enabling student to adapt using distilled supervision. Empirical evaluations across different agent benchmarks demonstrate that TT-SI improves the performance with +5.48% absolute accuracy gain on average across all benchmarks and surpasses other standard learning methods, yet using 68x less training samples. Our findings highlight the promise of TT-SI, demonstrating the potential of self-improvement algorithms at test-time as a new paradigm for building more capable agents toward self-evolution.

Method: Proposes MA-GAD with graph compression module to reduce noise interference while preserving node information, uses meta-learning to extract meta-anomaly patterns from similar networks, and employs bias network to enhance anomaly-normal distinction.

Result: Outperforms state-of-the-art methods on four real-world biochemical datasets in both graph anomaly and subgraph anomaly detection tasks under few-shot conditions.

Conclusion: MA-GAD effectively addresses few-shot graph anomaly detection challenges through meta-learning and graph compression, demonstrating superior performance on real-world datasets.

Abstract: Graph-level anomaly detection aims to identify anomalous graphs or subgraphs within graph datasets, playing a vital role in various fields such as fraud detection, review classification, and biochemistry. While Graph Neural Networks (GNNs) have made significant progress in this domain, existing methods rely heavily on large amounts of labeled data, which is often unavailable in real-world scenarios. Additionally, few-shot anomaly detection methods based on GNNs are prone to noise interference, resulting in poor embedding quality and reduced model robustness. To address these challenges, we propose a novel meta-learning-based graph-level anomaly detection framework (MA-GAD), incorporating a graph compression module that reduces the graph size, mitigating noise interference while retaining essential node information. We also leverage meta-learning to extract meta-anomaly information from similar networks, enabling the learning of an initialization model that can rapidly adapt to new tasks with limited samples. This improves the anomaly detection performance on target graphs, and a bias network is used to enhance the distinction between anomalous and normal nodes. Our experimental results, based on four real-world biochemical datasets, demonstrate that MA-GAD outperforms existing state-of-the-art methods in graph-level anomaly detection under few-shot conditions. Experiments on both graph anomaly and subgraph anomaly detection tasks validate the framework’s effectiveness on real-world datasets.

Method: The paper reviews multiple aspects of SEM imaging including principal operation, noise sources, SNR measurement methods, factors affecting SNR, and enhancement approaches through traditional and emerging techniques.

Result: The review provides a comprehensive analysis of SNR optimization strategies in SEM, examining both hardware-based and software-based approaches with their respective applications, advantages, and limitations.

Conclusion: The paper aims to offer researchers and practitioners a thorough understanding of SNR optimization in SEM and to stimulate further research in this important field.

Abstract: Scanning Electron Microscopy (SEM) is critical in nanotechnology, materials science, and biological imaging due to its high spatial resolution and depth of focus. Signal-to-noise ratio (SNR) is an essential parameter in SEM because it directly impacts the quality and interpretability of the images. SEM is widely used in various scientific disciplines, but its utility can be compromised by noise, which degrades image clarity. This review explores multiple aspects of the SEM imaging process, from the principal operation of SEM, sources of noise in SEM, methods for SNR measurement and estimations, to various aspects that affect the SNR measurement and approaches to enhance SNR, both from a hardware and software standpoint. We review traditional and emerging techniques, focusing on their applications, advantages, and limitations. The paper aims to provide a comprehensive understanding of SNR optimization in SEM for researchers and practitioners and to encourage further research in the field.

Method: Proposed AO-GPRLLSR pipeline: compares 5 SNR estimation methods (NN, FOL, NN+FOL, NLLSR, LSR), selects LSR as best, pairs with SVM/GPR for NV estimation, uses Optimizable GPR for NV, then applies NV-guided Wiener filter.

Result: LSR method performed best for SNR estimation, Optimizable GPR showed highest accuracy for NV estimation, and the combined AO-GPRLLSR method achieved lower MSE after filtering.

Conclusion: The proposed method successfully estimates SNR and NV of SEM images and provides effective denoising through the NV-guided Wiener filter, offering a robust SEM image filtering pipeline.

Abstract: Scanning Electron Microscopy (SEM) images often suffer from noise contamination, which degrades image quality and affects further analysis. This research presents a complete approach to estimate their Signal-to-Noise Ratio (SNR) and noise variance (NV), and enhance image quality using NV-guided Wiener filter. The main idea of this study is to use a good SNR estimation technique and infuse a machine learning model to estimate NV of the SEM image, which then guides the wiener filter to remove the noise, providing a more robust and accurate SEM image filtering pipeline. First, we investigate five different SNR estimation techniques, namely Nearest Neighbourhood (NN) method, First-Order Linear Interpolation (FOL) method, Nearest Neighbourhood with First-Order Linear Interpolation (NN+FOL) method, Non-Linear Least Squares Regression (NLLSR) method, and Linear Least Squares Regression (LSR) method. It is shown that LSR method to perform better than the rest. Then, Support Vector Machines (SVM) and Gaussian Process Regression (GPR) are tested by pairing it with LSR. In this test, the Optimizable GPR model shows the highest accuracy and it stands as the most effective solution for NV estimation. Combining these results lead to the proposed Adaptive Optimizable Gaussian Process Regression Linear Least Squares Regression (AO-GPRLLSR) Filtering pipeline. The AO-GPRLLSR method generated an estimated noise variance which served as input to NV-guided Wiener filter for improving the quality of SEM images. The proposed method is shown to achieve notable success in estimating SNR and NV of SEM images and leads to lower Mean Squared Error (MSE) after the filtering process.

Method: Proposes MMM framework that generates 3D electron density maps using ELF, combines ELF-derived features encoding global electronic properties with a bipartite graph encoder for local substructure interactions.

Result: Significant improvements over GNN-based SafeDrug model in F1-score (p=0.0387), Jaccard (p=0.0112), and DDI rate (p=0.0386) on MIMIC-III dataset with 250 drugs and 442 substructures.

Conclusion: ELF-based 3D representations show potential to enhance prediction accuracy and support safer combinatorial drug prescribing in clinical practice.

Abstract: Drug recommendation is an essential task in machine learning-based clinical decision support systems. However, the risk of drug-drug interactions (DDI) between co-prescribed medications remains a significant challenge. Previous studies have used graph neural networks (GNNs) to represent drug structures. Regardless, their simplified discrete forms cannot fully capture the molecular binding affinity and reactivity. Therefore, we propose Multimodal DDI Prediction with Molecular Electron Localization Function (ELF) Maps (MMM), a novel framework that integrates three-dimensional (3D) quantum-chemical information into drug representation learning. It generates 3D electron density maps using the ELF. To capture both therapeutic relevance and interaction risks, MMM combines ELF-derived features that encode global electronic properties with a bipartite graph encoder that models local substructure interactions. This design enables learning complementary characteristics of drug molecules. We evaluate MMM in the MIMIC-III dataset (250 drugs, 442 substructures), comparing it with several baseline models. In particular, a comparison with the GNN-based SafeDrug model demonstrates statistically significant improvements in the F1-score (p = 0.0387), Jaccard (p = 0.0112), and the DDI rate (p = 0.0386). These results demonstrate the potential of ELF-based 3D representations to enhance prediction accuracy and support safer combinatorial drug prescribing in clinical practice.

Method: Three-stage architecture: Feature Center & Prerank Model for candidate generation, Multi-Task Learning model for predicting user feedback signals, and Multi-Task Fusion (GRADE) module that learns personalized weights to blend rankings.

Result: The system generates blended rankings by applying personalized weights to multiple objectives, delivering optimized results to users.

Conclusion: The proposed GRADE framework enables effective personalization in multi-objective ranking systems through learned weight optimization.

Abstract: Overall architecture of the personalized multi-objective ranking system. It comprises: (1) a Feature Center and Prerank Model for initial feature processing and candidate generation; (2) a Multi-Task Learning (MTL) model predicting various user feedback signals; (3) a Multi-Task Fusion (MTF) module (our proposed GRADE framework) that learns personalized weights ($w_1, \dots, w_n$); these weights are then applied to calculate final scores and sorted to generate a blended ranking by the Blended Ranking Model, which ultimately delivers results to users.

Method: SketchGuard compresses d-dimensional models to k-dimensional sketches (k « d) using Count Sketch for similarity comparisons, then selectively fetches full models only from accepted neighbors, reducing communication complexity.

Result: SketchGuard maintains identical robustness to state-of-the-art methods while reducing computation time by up to 82% and communication overhead by 50-70%, with benefits scaling with model dimensionality and network connectivity.

Conclusion: SketchGuard establishes sketch-based compression as a fundamental enabler of robust decentralized federated learning at web scale.

Abstract: Decentralized Federated Learning (DFL) enables privacy-preserving collaborative training without centralized servers, but remains vulnerable to Byzantine attacks where malicious clients submit corrupted model updates. Existing Byzantine-robust DFL defenses rely on similarity-based neighbor screening that requires every client to exchange and compare complete high-dimensional model vectors with all neighbors in each training round, creating prohibitive communication and computational costs that prevent deployment at web scale. We propose SketchGuard, a general framework that decouples Byzantine filtering from model aggregation through sketch-based neighbor screening. SketchGuard compresses $d$-dimensional models to $k$-dimensional sketches ($k \ll d$) using Count Sketch for similarity comparisons, then selectively fetches full models only from accepted neighbors, reducing per-round communication complexity from $O(d|N_i|)$ to $O(k|N_i| + d|S_i|)$, where $|N_i|$ is the neighbor count and $|S_i| \le |N_i|$ is the accepted neighbor count. We establish rigorous convergence guarantees in both strongly convex and non-convex settings, proving that Count Sketch compression preserves Byzantine resilience with controlled degradation bounds where approximation errors introduce only a $(1+O(\epsilon))$ factor in the effective threshold parameter. Comprehensive experiments across multiple datasets, network topologies, and attack scenarios demonstrate that SketchGuard maintains identical robustness to state-of-the-art methods while reducing computation time by up to 82% and communication overhead by 50-70% depending on filtering effectiveness, with benefits scaling multiplicatively with model dimensionality and network connectivity. These results establish the viability of sketch-based compression as a fundamental enabler of robust DFL at web scale.

Method: Deconstruct SNNs into timestep submodels, evaluate their output confidence to identify strong/weak submodels, and implement two self-distillation schemes: Strong2Weak (stronger guides weaker) and Weak2Strong (weak distills strong with dark knowledge), with flexible implementations including ensemble, simultaneous, and cascade distillation.

Result: The method effectively improves SNN discriminability and overall performance while enhancing adversarial robustness through the stability of self-distillation.

Conclusion: The approach ingeniously exploits SNNs’ temporal properties to efficiently train high-performance SNNs through self-distillation, providing insights for future SNN training methods.

Abstract: Brain-inspired spiking neural networks (SNNs) promise to be a low-power alternative to computationally intensive artificial neural networks (ANNs), although performance gaps persist. Recent studies have improved the performance of SNNs through knowledge distillation, but rely on large teacher models or introduce additional training overhead. In this paper, we show that SNNs can be naturally deconstructed into multiple submodels for efficient self-distillation. We treat each timestep instance of the SNN as a submodel and evaluate its output confidence, thus efficiently identifying the strong and the weak. Based on this strong and weak relationship, we propose two efficient self-distillation schemes: (1) \textbf{Strong2Weak}: During training, the stronger “teacher” guides the weaker “student”, effectively improving overall performance. (2) \textbf{Weak2Strong}: The weak serve as the “teacher”, distilling the strong in reverse with underlying dark knowledge, again yielding significant performance gains. For both distillation schemes, we offer flexible implementations such as ensemble, simultaneous, and cascade distillation. Experiments show that our method effectively improves the discriminability and overall performance of the SNN, while its adversarial robustness is also enhanced, benefiting from the stability brought by self-distillation. This ingeniously exploits the temporal properties of SNNs and provides insight into how to efficiently train high-performance SNNs.

Method: Developed two bounds on KL divergence between Bernoulli distributions, formalized implicit differentiation methodology for optimizing PAC-Bayesian certificates, and created methods for optimizing bounds on non-differentiable objectives like 0-1 loss.

Result: Achieved the first non-vacuous generalization bounds on CIFAR-10 for neural networks, with empirical evaluation on MNIST and CIFAR-10 datasets.

Conclusion: The theoretical contributions significantly improve PAC-Bayes risk certificates for neural networks, enabling practical non-vacuous generalization bounds on complex datasets like CIFAR-10.

Abstract: This paper presents four theoretical contributions that improve the usability of risk certificates for neural networks based on PAC-Bayes bounds. First, two bounds on the KL divergence between Bernoulli distributions enable the derivation of the tightest explicit bounds on the true risk of classifiers across different ranges of empirical risk. The paper next focuses on the formalization of an efficient methodology based on implicit differentiation that enables the introduction of the optimization of PAC-Bayesian risk certificates inside the loss/objective function used to fit the network/model. The last contribution is a method to optimize bounds on non-differentiable objectives such as the 0-1 loss. These theoretical contributions are complemented with an empirical evaluation on the MNIST and CIFAR-10 datasets. In fact, this paper presents the first non-vacuous generalization bounds on CIFAR-10 for neural networks.

Method: Diversifying Sample Condensation (DISCO) selects top-k samples with greatest model disagreements using greedy, sample-wise statistics instead of global clustering.

Result: DISCO achieves state-of-the-art results in performance prediction across MMLU, Hellaswag, Winogrande, and ARC benchmarks.

Conclusion: Model response diversity is more important than sample diversity for efficient evaluation, and inter-model disagreement provides an information-theoretically optimal selection rule.

Abstract: Evaluating modern machine learning models has become prohibitively expensive. Benchmarks such as LMMs-Eval and HELM demand thousands of GPU hours per model. Costly evaluation reduces inclusivity, slows the cycle of innovation, and worsens environmental impact. The typical approach follows two steps. First, select an anchor subset of data. Second, train a mapping from the accuracy on this subset to the final test result. The drawback is that anchor selection depends on clustering, which can be complex and sensitive to design choices. We argue that promoting diversity among samples is not essential; what matters is to select samples that $\textit{maximise diversity in model responses}$. Our method, $\textbf{Diversifying Sample Condensation (DISCO)}$, selects the top-k samples with the greatest model disagreements. This uses greedy, sample-wise statistics rather than global clustering. The approach is conceptually simpler. From a theoretical view, inter-model disagreement provides an information-theoretically optimal rule for such greedy selection. $\textbf{DISCO}$ shows empirical gains over prior methods, achieving state-of-the-art results in performance prediction across MMLU, Hellaswag, Winogrande, and ARC. Code is available here: https://github.com/arubique/disco-public.

Method: Proposed PRESCRIBE - a multivariate deep evidential regression framework that jointly measures both epistemic and aleatoric uncertainty sources.

Result: PRESCRIBE effectively estimates confidence scores that strongly correlate with empirical accuracy, enabling filtering of untrustworthy predictions and achieving over 3% accuracy improvements compared to baselines.

Conclusion: The framework successfully addresses uncertainty in single-cell perturbation prediction, providing reliable confidence estimates that improve prediction quality and enable better result filtering.

Abstract: In single-cell perturbation prediction, a central task is to forecast the effects of perturbing a gene unseen in the training data. The efficacy of such predictions depends on two factors: (1) the similarity of the target gene to those covered in the training data, which informs model (epistemic) uncertainty, and (2) the quality of the corresponding training data, which reflects data (aleatoric) uncertainty. Both factors are critical for determining the reliability of a prediction, particularly as gene perturbation is an inherently stochastic biochemical process. In this paper, we propose PRESCRIBE (PREdicting Single-Cell Response wIth Bayesian Estimation), a multivariate deep evidential regression framework designed to measure both sources of uncertainty jointly. Our analysis demonstrates that PRESCRIBE effectively estimates a confidence score for each prediction, which strongly correlates with its empirical accuracy. This capability enables the filtering of untrustworthy results, and in our experiments, it achieves steady accuracy improvements of over 3% compared to comparable baselines.

Method: The authors use stability analysis to derive upper bounds on generalization error and excess error of MGS. They provide unified analysis of factors like learning rate, data heterogeneity, node count, per-node sample size, and communication topology under non-convex settings without bounded gradients assumption.

Result: 1) MGS reduces optimization error bound at exponential rate, tightening generalization error bound. 2) Even with infinite MGS, a non-negligible generalization gap remains between decentralized and centralized training. Experiments on CIFAR datasets support theoretical findings.

Conclusion: MGS significantly improves decentralized training performance but cannot fully close the gap with centralized training. The study provides comprehensive theoretical understanding of MGS effectiveness and limitations in decentralized learning.

Abstract: Decentralized training removes the centralized server, making it a communication-efficient approach that can significantly improve training efficiency, but it often suffers from degraded performance compared to centralized training. Multi-Gossip Steps (MGS) serve as a simple yet effective bridge between decentralized and centralized training, significantly reducing experiment performance gaps. However, the theoretical reasons for its effectiveness and whether this gap can be fully eliminated by MGS remain open questions. In this paper, we derive upper bounds on the generalization error and excess error of MGS using stability analysis, systematically answering these two key questions. 1). Optimization Error Reduction: MGS reduces the optimization error bound at an exponential rate, thereby exponentially tightening the generalization error bound and enabling convergence to better solutions. 2). Gap to Centralization: Even as MGS approaches infinity, a non-negligible gap in generalization error remains compared to centralized mini-batch SGD ($\mathcal{O}(T^{\frac{c\beta}{c\beta +1}}/{n m})$ in centralized and $\mathcal{O}(T^{\frac{2c\beta}{2c\beta +2}}/{n m^{\frac{1}{2c\beta +2}}})$ in decentralized). Furthermore, we provide the first unified analysis of how factors like learning rate, data heterogeneity, node count, per-node sample size, and communication topology impact the generalization of MGS under non-convex settings without the bounded gradients assumption, filling a critical theoretical gap in decentralized training. Finally, promising experiments on CIFAR datasets support our theoretical findings.

Method: Adversaries compute a proxy metric to estimate parameter pruning likelihood, inject malicious behavior into unlikely-to-be-pruned parameters, and repair the model using likely-to-be-pruned parameters to hide the behavior in unpruned models.

Result: The attack achieves high success rates across five models and three pruning methods in vLLM: up to 95.7% for jailbreak, 98.7% for benign instruction refusal, and 99.5% for targeted content injection.

Conclusion: This work reveals a critical deployment-time security gap in model compression and underscores the urgent need for stronger security awareness in LLM pruning practices.

Abstract: Model pruning, i.e., removing a subset of model weights, has become a prominent approach to reducing the memory footprint of large language models (LLMs) during inference. Notably, popular inference engines, such as vLLM, enable users to conveniently prune downloaded models before they are deployed. While the utility and efficiency of pruning methods have improved significantly, the security implications of pruning remain underexplored. In this work, for the first time, we show that modern LLM pruning methods can be maliciously exploited. In particular, an adversary can construct a model that appears benign yet, once pruned, exhibits malicious behaviors. Our method is based on the idea that the adversary can compute a proxy metric that estimates how likely each parameter is to be pruned. With this information, the adversary can first inject a malicious behavior into those parameters that are unlikely to be pruned. Then, they can repair the model by using parameters that are likely to be pruned, effectively canceling out the injected behavior in the unpruned model. We demonstrate the severity of our attack through extensive evaluation on five models; after any of the pruning in vLLM are applied (Magnitude, Wanda, and SparseGPT), it consistently exhibits strong malicious behaviors in a diverse set of attack scenarios (success rates of up to $95.7%$ for jailbreak, $98.7%$ for benign instruction refusal, and $99.5%$ for targeted content injection). Our results reveal a critical deployment-time security gap and underscore the urgent need for stronger security awareness in model compression.

Method: Uses XGBoost algorithm with historical electricity demand, comprehensive weather, and socioeconomic variables. Employs temporal data-splitting strategy for robust training and evaluation across multiple years and countries.

Result: The approach delivers accurate and scalable demand forecasts with reliable out-of-sample performance benchmarking.

Conclusion: The framework successfully provides valuable insights for energy planning and policy decisions in the context of global energy transition challenges.

Abstract: This paper presents a machine learning framework for electricity demand forecasting across diverse geographical regions using the gradient boosting algorithm XGBoost. The model integrates historical electricity demand and comprehensive weather and socioeconomic variables to predict normalized electricity demand profiles. To enable robust training and evaluation, we developed a large-scale dataset spanning multiple years and countries, applying a temporal data-splitting strategy that ensures benchmarking of out-of-sample performance. Our approach delivers accurate and scalable demand forecasts, providing valuable insights for energy system planners and policymakers as they navigate the challenges of the global energy transition.

Method: Extends D2C with dynamic uncertainty-aware aggregation that weights subset models based on their validation set performance and prediction confidence.

Result: Significantly improves generalization on benchmark datasets across multiple domains, even when applied on top of other regularization methods.

Conclusion: DUA-D2C is a theoretically grounded and effective approach for combating overfitting in deep learning through intelligent model aggregation.

Abstract: Overfitting remains a significant challenge in deep learning, often arising from data outliers, noise, and limited training data. To address this, the Divide2Conquer (D2C) method was previously proposed, which partitions training data into multiple subsets and trains identical models independently on each. This strategy enables learning more consistent patterns while minimizing the influence of individual outliers and noise. However, D2C’s standard aggregation typically treats all subset models equally or based on fixed heuristics (like data size), potentially underutilizing information about their varying generalization capabilities. Building upon this foundation, we introduce Dynamic Uncertainty-Aware Divide2Conquer (DUA-D2C), an advanced technique that refines the aggregation process. DUA-D2C dynamically weights the contributions of subset models based on their performance on a shared validation set, considering both accuracy and prediction uncertainty. This intelligent aggregation allows the central model to preferentially learn from subsets yielding more generalizable and confident edge models, thereby more effectively combating overfitting. Empirical evaluations on benchmark datasets spanning multiple domains demonstrate that DUA-D2C significantly improves generalization. Our analysis includes evaluations of decision boundaries, loss curves, and other performance metrics, highlighting the effectiveness of DUA-D2C. This study demonstrates that DUA-D2C improves generalization performance even when applied on top of other regularization methods, establishing it as a theoretically grounded and effective approach to combating overfitting in modern deep learning. Our codes are publicly available at: https://github.com/Saiful185/DUA-D2C.

Method: Orthogonal growth method: interpositional layer copying for depth growth and expert duplication with injected noise for width growth. Comprehensive scaling experiments to determine optimal growth timing.

Result: Scaled to 70B parameter models with over 1T training tokens, achieving 10.66% accuracy gain over training from scratch under same additional compute budget.

Conclusion: Checkpoint recycling establishes a foundation for economically efficient large language model pretraining by leveraging existing computational investments.

Abstract: The rapidly increasing computational cost of pretraining Large Language Models necessitates more efficient approaches. Numerous computational costs have been invested in existing well-trained checkpoints, but many of them remain underutilized due to engineering constraints or limited model capacity. To efficiently reuse this “sunk” cost, we propose to recycle pretrained checkpoints by expanding their parameter counts and continuing training. We propose orthogonal growth method well-suited for converged Mixture-of-Experts model: interpositional layer copying for depth growth and expert duplication with injected noise for width growth. To determine the optimal timing for such growth across checkpoints sequences, we perform comprehensive scaling experiments revealing that the final accuracy has a strong positive correlation with the amount of sunk cost, indicating that greater prior investment leads to better performance. We scale our approach to models with 70B parameters and over 1T training tokens, achieving 10.66% accuracy gain over training from scratch under the same additional compute budget. Our checkpoint recycling approach establishes a foundation for economically efficient large language model pretraining.

Method: Uses nested evolving set processes with localized inexact proximal point iterations to solve simplified linear systems, requiring only Õ(1/√α) linear system solutions.

Result: Achieves time complexity of min{Õ(R²/ε²), Õ(m)} for ε-approximation, and Õ(R²/(√αε²)) when 1/ε² ≪ m, independent of graph size. Validated on real-world graphs with early-stage convergence.

Conclusion: The framework successfully accelerates PPR computation, resolves an open conjecture, and provides graph-size-independent complexity for certain parameter regimes, with practical efficiency demonstrated on real datasets.

Abstract: This work proposes a novel framework based on nested evolving set processes to accelerate Personalized PageRank (PPR) computation. At each stage of the process, we employ a localized inexact proximal point iteration to solve a simplified linear system. We show that the time complexity of such localized methods is upper bounded by $\min{\tilde{\mathcal{O}}(R^2/\epsilon^2), \tilde{\mathcal{O}}(m)}$ to obtain an $\epsilon$-approximation of the PPR vector, where $m$ denotes the number of edges in the graph and $R$ is a constant defined via nested evolving set processes. Furthermore, the algorithms induced by our framework require solving only $\tilde{\mathcal{O}}(1/\sqrt{\alpha})$ such linear systems, where $\alpha$ is the damping factor. When $1/\epsilon^2\ll m$, this implies the existence of an algorithm that computes an $\ epsilon $-approximation of the PPR vector with an overall time complexity of $\tilde{\mathcal{O}}\left(R^2 / (\sqrt{\alpha}\epsilon^2)\right)$, independent of the underlying graph size. Our result resolves an open conjecture from existing literature. Experimental results on real-world graphs validate the efficiency of our methods, demonstrating significant convergence in the early stages.

Method: HMM-based framework jointly models conditional channel propagation (power, delay, angle with separate LOS/NLOS models) and user trajectory evolution using Gaussian-Markov model, with simultaneous optimization of all parameters.

Result: Achieves 0.65m average localization accuracy in indoor environments covering both LOS and NLOS regions, outperforming conventional supervised methods like KNN, SVM, and DNN.

Conclusion: The proposed calibration-free approach successfully constructs radio maps and enables accurate localization without requiring labeled location data, demonstrating superior performance over traditional supervised methods.

Abstract: Radio maps are essential for enhancing wireless communications and localization. However, existing methods for constructing radio maps typically require costly calibration processes to collect location-labeled channel state information (CSI) datasets. This paper aims to recover the data collection trajectory directly from the channel propagation sequence, eliminating the need for location calibration. The key idea is to employ a hidden Markov model (HMM)-based framework to conditionally model the channel propagation matrix, while simultaneously modeling the location correlation in the trajectory. The primary challenges involve modeling the complex relationship between channel propagation in multiple-input multiple-output (MIMO) networks and geographical locations, and addressing both line-of-sight (LOS) and non-line-of-sight (NLOS) indoor conditions. In this paper, we propose an HMM-based framework that jointly characterizes the conditional propagation model and the evolution of the user trajectory. Specifically, the channel propagation in MIMO networks is modeled separately in terms of power, delay, and angle, with distinct models for LOS and NLOS conditions. The user trajectory is modeled using a Gaussian-Markov model. The parameters for channel propagation, the mobility model, and LOS/NLOS classification are optimized simultaneously. Experimental validation using simulated MIMO-Orthogonal Frequency-Division Multiplexing (OFDM) networks with a multi-antenna uniform linear arrays (ULA) configuration demonstrates that the proposed method achieves an average localization accuracy of 0.65 meters in an indoor environment, covering both LOS and NLOS regions. Moreover, the constructed radio map enables localization with a reduced error compared to conventional supervised methods, such as k-nearest neighbors (KNN), support vector machine (SVM), and deep neural network (DNN).

Method: Define Backdoor Vector (BV) as weight difference between backdoored and clean models. Propose Sparse Backdoor Vector (SBV) to combine multiple attacks, and Injection BV Subtraction (IBVS) as a defense by subtracting identified backdoor components.

Result: BVs provide better understanding of attack similarity and transferability. SBVs outperform prior attacks by leveraging merging to improve backdoor effectiveness. IBVS offers effective defense even against unknown threats.

Conclusion: The BV framework enables better analysis of backdoor attacks in model merging. SBV demonstrates that merging can enhance attack capabilities, while IBVS provides a practical, assumption-free defense solution.

Abstract: Model merging (MM) recently emerged as an effective method for combining large deep learning models. However, it poses significant security risks. Recent research shows that it is highly susceptible to backdoor attacks, which introduce a hidden trigger into a single fine-tuned model instance that allows the adversary to control the output of the final merged model at inference time. In this work, we propose a simple framework for understanding backdoor attacks by treating the attack itself as a task vector. $Backdoor\ Vector
(BV)$ is calculated as the difference between the weights of a fine-tuned backdoored model and fine-tuned clean model. BVs reveal new insights into attacks understanding and a more effective framework to measure their similarity and transferability. Furthermore, we propose a novel method that enhances backdoor resilience through merging dubbed $Sparse\ Backdoor\ Vector
(SBV)$ that combines multiple attacks into a single one. We identify the core vulnerability behind backdoor threats in MM: $inherent\ triggers$ that exploit adversarial weaknesses in the base model. To counter this, we propose $Injection\ BV\ Subtraction\ (IBVS)$ - an assumption-free defense against backdoors in MM. Our results show that SBVs surpass prior attacks and is the first method to leverage merging to improve backdoor effectiveness. At the same time, IBVS provides a lightweight, general defense that remains effective even when the backdoor threat is entirely unknown.

Method: The authors use softmax temperature calibration and analyze intermediate layer outputs through layerwise exponentially weighted connectivity (LEWC), which represents merged model outputs as exponentially weighted sums of original model layers.

Result: Empirical demonstration shows that widening models enables LMC without permutations, and LEWC analysis reveals that merged outputs match ensemble behavior of original models.

Conclusion: Model widening not only facilitates nonlinear mode connectivity as previously known, but also significantly increases the possibility of achieving linear mode connectivity.

Abstract: Recently, Ainsworth et al. empirically demonstrated that, given two independently trained models, applying a parameter permutation that preserves the input-output behavior allows the two models to be connected by a low-loss linear path. When such a path exists, the models are said to achieve linear mode connectivity (LMC). Prior studies, including Ainsworth et al., have reported that achieving LMC requires not only an appropriate permutation search but also sufficiently wide models (e.g., a 32 $\times$ width multiplier for ResNet-20). This is broadly believed to be because increasing the model width ensures a large enough space of candidate permutations, increasing the chance of finding one that yields LMC. In this work, we empirically demonstrate that, even without any permutations, simply widening the models is sufficient for achieving LMC when using a suitable softmax temperature calibration. We further explain why this phenomenon arises by analyzing intermediate layer outputs. Specifically, we introduce layerwise exponentially weighted connectivity (LEWC), which states that the output of each layer of the merged model can be represented as an exponentially weighted sum of the outputs of the corresponding layers of the original models. Consequently the merged model’s output matches that of an ensemble of the original models, which facilitates LMC. To the best of our knowledge, this work is the first to show that widening the model not only facilitates nonlinear mode connectivity, as suggested in prior research, but also significantly increases the possibility of achieving linear mode connectivity.

Method: Vertically partitioning the model into contiguous layer groups and interleaving prefill and decode across these groups, shifting the scheduling axis from tokens to layers.

Result: Reduces off-chip bandwidth demand, lowering TTFT by up to 70%, End-to-End latency by 41%, and per-token energy by up to 22%. Consistently improves the TTFT-TBT Pareto frontier over chunked prefill.

Conclusion: Layered prefill unlocks a new operating regime for high-efficiency, energy-aware LLM serving in co-located environments by eliminating expert-load traffic and energy costs while maintaining stall-free decoding.

Abstract: Large Language Model (LLM) inference in production must meet stringent service-level objectives for both time-to-first-token (TTFT) and time-between-token (TBT) while maximizing throughput under fixed compute, memory, and interconnect budgets. Modern serving systems adopt stall-free scheduling techniques such as chunked prefill, which splits long prompt processing along the token dimension and interleaves prefill with ongoing decode iterations. While effective at stabilizing TBT, chunked prefill incurs substantial overhead in Mixture-of-Experts (MoE) models: redundant expert weight loads increase memory traffic by up to 39% and inflate energy consumption. We propose layered prefill, a new scheduling paradigm that treats transformer layer groups as the primary scheduling unit. By vertically partitioning the model into contiguous layer groups and interleaving prefill and decode across the groups, layered prefill sustains stall-free decoding while eliminating chunk-induced MoE weight reloads. It reduces off-chip bandwidth demand, lowering TTFT by up to 70%, End-to-End latency by 41% and per-token energy by up to 22%. Evaluations show that layered prefill consistently improves the TTFT–TBT Pareto frontier over chunked prefill, reducing expert-load traffic and energy cost while maintaining stall-free decoding. Overall, shifting the scheduling axis from tokens to layers unlocks a new operating regime for high-efficiency, energy-aware LLM serving in co-located environments.

Method: Designs an adaptive layer that replaces standard linear/convolutional layers, decomposing weights into shared subject-invariant component and lightweight low-rank correction for each subject.

Result: Models with Subject-Conditioned Layer outperform both subject-agnostic models and the average of individually trained subject-specific models.

Conclusion: The layer provides a practical and scalable approach for building effective cross-subject foundation models for EEG decoding.

Abstract: Subject-specific distribution shifts represent an important obstacle to the development of foundation models for EEG decoding. To address this, we propose Subject-Conditioned Layer,, an adaptive layer designed as a drop-in replacement for standard linear or convolutional layers in any neural network architecture. Our layer captures subject-specific variability by decomposing its weights into a shared, subject-invariant component and a lightweight, low-rank correction unique to each subject. This explicit separation of general knowledge from personalized adaptation allows existing models to become robust to subject shifts. Empirically, models equipped with our layer outperform both a shared-weight-only model (subject-agnostic model) and the average of individually trained subject-specific models. Consequently, the Subject-Conditioned Layer, offers a practical and scalable path towards building effective cross-subject foundation models for EEG.

Method: Proposes a novel approach that explicitly disentangles domain distributions and adaptively captures instance-specific domain information to address the challenge of highly entangled domain features in VLMs.

Result: Extensive experiments show the approach outperforms baselines built upon VLM tuning techniques, demonstrating effective domain unlearning while maintaining performance.

Conclusion: The work paves the way for practical and fine-grained unlearning in VLMs, addressing domain-level knowledge removal challenges that class unlearning methods cannot handle effectively.

Abstract: Pre-trained Vision-Language Models (VLMs) exhibit strong generalization capabilities, enabling them to recognize a wide range of objects across diverse domains without additional training. However, they often retain irrelevant information beyond the requirements of specific downstream tasks, raising concerns about computational efficiency and potential information leakage. This has motivated growing interest in approximate unlearning, which aims to selectively remove unnecessary knowledge while preserving overall model performance. Existing approaches to approximate unlearning have primarily focused on class unlearning, where a VLM is retrained to fail to recognize specified object classes while maintaining accuracy for others. However, merely forgetting object classes is often insufficient in practical applications. For instance, an autonomous driving system should accurately recognize real cars while avoiding misrecognition of illustrated cars depicted in roadside advertisements as real cars, which could be hazardous. In this paper, we introduce Approximate Domain Unlearning (ADU), a novel problem setting that requires reducing recognition accuracy for images from specified domains (e.g., illustration) while preserving accuracy for other domains (e.g., real). ADU presents new technical challenges: due to the strong domain generalization capability of pre-trained VLMs, domain distributions are highly entangled in the feature space, making naive approaches based on penalizing target domains ineffective. To tackle this limitation, we propose a novel approach that explicitly disentangles domain distributions and adaptively captures instance-specific domain information. Extensive experiments show that our approach outperforms baselines built upon VLM tuning techniques, paving the way for practical and fine-grained unlearning in VLMs. Code: https://kodaikawamura.github.io/Domain_Unlearning/.

Method: AEPO uses three key designs: policy gradient as regularization, distribution as regularization, and REINFORCE as regularization. It replaces entropy bonuses with REINFORCE policy gradient on temperature-adjusted distributions and stabilizes entropy through temperature regulation.

Result: AEPO stabilizes entropy at arbitrary target levels, effectively removing collapse in GRPO. It reveals a non-monotonic relation where performance first improves then declines with increasing entropy, clarifying the link between entropy, exploration, and reasoning.

Conclusion: AEPO provides a broader RFT paradigm where superior target distributions can serve as REINFORCE regularizers, generalizing beyond entropy control and offering a more stable approach to reinforcement finetuning.

Abstract: Reinforcement finetuning (RFT) is essential for enhancing the reasoning capabilities of large language models (LLM), yet the widely adopted Group Relative Policy Optimization (GRPO) suffers from entropy collapse, where entropy monotonically decreases, exploration vanishes, and policies converge prematurely. Existing entropy-regularized methods only partially alleviate this issue while introducing bias and instability, leaving entropy control unresolved and the connection between entropy, exploration, and performance unclear. We propose Arbitrary Entropy Policy Optimization (AEPO), which eliminates entropy collapse by replacing entropy bonuses with REINFORCE policy gradient on temperature-adjusted distributions and stabilizing entropy through temperature regulation. AEPO integrates three key designs: policy gradient as regularization, distribution as regularization, and REINFORCE as regularization, enabling precise entropy control without distorting optimization. Experiments demonstrate three major contributions: AEPO (1) stabilizes entropy at arbitrary target levels, effectively removing collapse in GRPO; (2) reveals a non-monotonic relation where performance first improves then declines with increasing entropy, clarifying the link between entropy, exploration, and reasoning; and (3) generalizes beyond entropy, providing a broader RFT paradigm where superior target distributions can serve as REINFORCE regularizers.

Method: Uses Shannon entropy from token-level logprobs as a confidence signal for early stopping, with entropy thresholds calculated using few examples from existing reasoning datasets.

Result: Achieves 25-50% computational cost reduction while preserving accuracy across reasoning-optimized model families.

Conclusion: Entropy-based confidence calibration is an emergent property of advanced post-training optimization in modern reasoning models, distinguishing them from standard instruction-tuned and pre-trained models.

Abstract: We introduce a simple, yet novel entropy-based framework to drive token efficiency in large language models during reasoning tasks. Our approach uses Shannon entropy from token-level logprobs as a confidence signal to enable early stopping, achieving 25-50% computational savings while maintaining task accuracy. Crucially, we demonstrate that entropy-based confidence calibration represents an emergent property of advanced post-training optimization present in modern reasoning models but notably absent in standard instruction-tuned and pre-trained models (Llama 3.3 70B). We show that the entropy threshold to stop reasoning varies from model to model but can be calculated easily in one shot using only a few examples from existing reasoning datasets. Our results indicate that advanced reasoning models often know that they’ve gotten a correct answer early on, and that this emergent confidence awareness can be exploited to save tokens and reduce latency. The framework demonstrates consistent performance across reasoning-optimized model families with 25-50% computational cost reduction while preserving accuracy, revealing that confidence mechanisms represent a distinguishing characteristic of modern post-trained reasoning systems versus their predecessors.

Method: Proposes GALA with two key components: inter-group discrepancy minimization for efficient domain alignment without quadratic computation, and temperature-controlled centroid-based weighting for dynamic source domain prioritization.

Result: GALA achieves competitive or state-of-the-art results on standard benchmarks and significantly outperforms prior methods in diverse multi-source settings where others fail to converge.

Conclusion: GALA provides a scalable and robust solution for federated UMDA that works effectively with large numbers of heterogeneous source domains.

Abstract: Unsupervised multi-source domain adaptation (UMDA) aims to learn models that generalize to an unlabeled target domain by leveraging labeled data from multiple, diverse source domains. While distributed UMDA methods address privacy constraints by avoiding raw data sharing, existing approaches typically assume a small number of sources and fail to scale effectively. Increasing the number of heterogeneous domains often makes existing methods impractical, leading to high computational overhead or unstable performance. We propose GALA, a scalable and robust federated UMDA framework that introduces two key components: (1) a novel inter-group discrepancy minimization objective that efficiently approximates full pairwise domain alignment without quadratic computation; and (2) a temperature-controlled, centroid-based weighting strategy that dynamically prioritizes source domains based on alignment with the target. Together, these components enable stable and parallelizable training across large numbers of heterogeneous sources. To evaluate performance in high-diversity scenarios, we introduce Digit-18, a new benchmark comprising 18 digit datasets with varied synthetic and real-world domain shifts. Extensive experiments show that GALA consistently achieves competitive or state-of-the-art results on standard benchmarks and significantly outperforms prior methods in diverse multi-source settings where others fail to converge.

Method: Created a novel dataset of synchronized measurements from both frequency bands in indoor environments. Applied identical training pipelines and model configurations across both bands using end-to-end deep learning with effective background subtraction.

Result: mmWave Wi-Fi signals achieved 91.2% identification accuracy on 20 individuals even at low sampling rates (10 Hz) when combined with background subtraction.

Conclusion: mmWave Wi-Fi signals demonstrate superior performance for person identification compared to sub-6 GHz, offering high accuracy with low sampling requirements, making them suitable for practical applications.

Abstract: Person identification plays a vital role in enabling intelligent, personalized, and secure human-computer interaction. Recent research has demonstrated the feasibility of leveraging Wi-Fi signals for passive person identification using a person’s unique gait pattern. Although most existing work focuses on sub-6 GHz frequencies, the emergence of mmWave offers new opportunities through its finer spatial resolution, though its comparative advantages for person identification remain unexplored. This work presents the first comparative study between sub-6 GHz and mmWave Wi-Fi signals for person identification with commercial off-the-shelf (COTS) Wi-Fi, using a novel dataset of synchronized measurements from the two frequency bands in an indoor environment. To ensure a fair comparison, we apply identical training pipelines and model configurations across both frequency bands. Leveraging end-to-end deep learning, we show that even at low sampling rates (10 Hz), mmWave Wi-Fi signals can achieve high identification accuracy (91.2% on 20 individuals) when combined with effective background subtraction.

Method: Shared encoder with two decoders: NAR decoder learns bidirectional features, AR decoder generates sequences using cross-decoder attention to query bidirectional features. Training uses importance annealing and gradient blocking.

Result: Outperforms AR and NAR baselines on nine-species peptide sequencing benchmark, achieving robust performance by harmonizing AR stability with NAR contextual awareness.

Conclusion: The hybrid framework advances biological sequence modeling by enhancing AR models with bidirectional understanding, providing a new architectural paradigm for complex sequence generation.

Abstract: Autoregressive (AR) models, common in sequence generation, are limited in many biological tasks such as de novo peptide sequencing and protein modeling by their unidirectional nature, failing to capture crucial global bidirectional token dependencies. Non-Autoregressive (NAR) models offer holistic, bidirectional representations but face challenges with generative coherence and scalability. To transcend this, we propose a hybrid framework enhancing AR generation by dynamically integrating rich contextual information from non-autoregressive mechanisms. Our approach couples a shared input encoder with two decoders: a non-autoregressive one learning latent bidirectional biological features, and an AR decoder synthesizing the biological sequence by leveraging these bidirectional features. A novel cross-decoder attention module enables the AR decoder to iteratively query and integrate these bidirectional features, enriching its predictions. This synergy is cultivated via a tailored training strategy with importance annealing for balanced objectives and cross-decoder gradient blocking for stable, focused learning. Evaluations on a demanding nine-species benchmark of de novo peptide sequencing show that our model substantially surpasses AR and NAR baselines. It uniquely harmonizes AR stability with NAR contextual awareness, delivering robust, superior performance on diverse downstream data. This research advances biological sequence modeling techniques and contributes a novel architectural paradigm for augmenting AR models with enhanced bidirectional understanding for complex sequence generation. Code is available at https://github.com/BEAM-Labs/denovo.

Method: Proposes Model Rebalancing (MORE) framework that introduces low-rank parameter components to mediate parameter space allocation, guided by tailored loss and sinusoidal reweighting schedule.

Result: Extensive experiments on diverse long-tailed benchmarks show significant improvement in generalization, particularly for tail classes, and effective complementarity with existing imbalance mitigation methods.

Conclusion: MORE serves as a robust plug-and-play module for long-tailed settings, directly addressing parameter space imbalance without increasing model complexity or inference costs.

Abstract: Long-tailed recognition is ubiquitous and challenging in deep learning and even in the downstream finetuning of foundation models, since the skew class distribution generally prevents the model generalization to the tail classes. Despite the promise of previous methods from the perspectives of data augmentation, loss rebalancing and decoupled training etc., consistent improvement in the broad scenarios like multi-label long-tailed recognition is difficult. In this study, we dive into the essential model capacity impact under long-tailed context, and propose a novel framework, Model Rebalancing (MORE), which mitigates imbalance by directly rebalancing the model’s parameter space. Specifically, MORE introduces a low-rank parameter component to mediate the parameter space allocation guided by a tailored loss and sinusoidal reweighting schedule, but without increasing the overall model complexity or inference costs. Extensive experiments on diverse long-tailed benchmarks, spanning multi-class and multi-label tasks, demonstrate that MORE significantly improves generalization, particularly for tail classes, and effectively complements existing imbalance mitigation methods. These results highlight MORE’s potential as a robust plug-and-play module in long-tailed settings.

Method: Proposes Dual-granularity Sinkhorn Distillation (D-SINK) that uses optimal transport-optimized surrogate label allocation to align the target model’s sample-level predictions with noise-robust auxiliary and class distributions with imbalance-robust auxiliary.

Result: Extensive experiments on benchmark datasets show D-SINK significantly improves robustness and achieves strong empirical performance in learning from long-tailed noisy data.

Conclusion: Instead of developing complex new techniques, synergistically leveraging well-established ‘weak’ auxiliary models specialized for either class imbalance or label noise can effectively address both challenges through complementary granularity insights.

Abstract: Real-world datasets for deep learning frequently suffer from the co-occurring challenges of class imbalance and label noise, hindering model performance. While methods exist for each issue, effectively combining them is non-trivial, as distinguishing genuine tail samples from noisy data proves difficult, often leading to conflicting optimization strategies. This paper presents a novel perspective: instead of primarily developing new complex techniques from scratch, we explore synergistically leveraging well-established, individually ‘weak’ auxiliary models - specialized for tackling either class imbalance or label noise but not both. This view is motivated by the insight that class imbalance (a distributional-level concern) and label noise (a sample-level concern) operate at different granularities, suggesting that robustness mechanisms for each can in principle offer complementary strengths without conflict. We propose Dual-granularity Sinkhorn Distillation (D-SINK), a novel framework that enhances dual robustness by distilling and integrating complementary insights from such ‘weak’, single-purpose auxiliary models. Specifically, D-SINK uses an optimal transport-optimized surrogate label allocation to align the target model’s sample-level predictions with a noise-robust auxiliary and its class distributions with an imbalance-robust one. Extensive experiments on benchmark datasets demonstrate that D-SINK significantly improves robustness and achieves strong empirical performance in learning from long-tailed noisy data.

Method: Created a new dataset from three ships, defined standardized benchmarks for tabular and time-series regression, and investigated in-context learning using the TabPFN foundation model.

Result: Models with environmental conditions outperformed simple baselines, TabPFN slightly outperformed other techniques, and temporal context improved accuracy.

Conclusion: Foundation models with in-context learning show strong potential for ship fuel consumption prediction, supporting feasible onboard data-driven fuel optimization.

Abstract: In the shipping industry, fuel consumption and emissions are critical factors due to their significant impact on economic efficiency and environmental sustainability. Accurate prediction of ship fuel consumption is essential for further optimization of maritime operations. However, heterogeneous methodologies and limited high-quality datasets hinder direct comparison of modeling approaches. This paper makes three key contributions: (1) we introduce and release a new dataset (https://huggingface.co/datasets/krohnedigital/FuelCast) comprising operational and environmental data from three ships; (2) we define a standardized benchmark covering tabular regression and time-series regression (3) we investigate the application of in-context learning for ship consumption modeling using the TabPFN foundation model - a first in this domain to our knowledge. Our results demonstrate strong performance across all evaluated models, supporting the feasibility of onboard, data-driven fuel prediction. Models incorporating environmental conditions consistently outperform simple polynomial baselines relying solely on vessel speed. TabPFN slightly outperforms other techniques, highlighting the potential of foundation models with in-context learning capabilities for tabular prediction. Furthermore, including temporal context improves accuracy.

Method: EVOR learns optimal regularized Q-functions via flow matching during training, then performs inference-time policy extraction through rejection sampling against the expressive value function.

Result: EVOR outperforms baselines on diverse offline RL tasks, showing benefits of integrating expressive value learning.

Conclusion: EVOR provides a scalable offline RL approach that avoids distillation and BPTT while enabling efficient optimization and compute-scalable search.

Abstract: Reinforcement learning (RL) is a powerful paradigm for learning to make sequences of decisions. However, RL has yet to be fully leveraged in robotics, principally due to its lack of scalability. Offline RL offers a promising avenue by training agents on large, diverse datasets, avoiding the costly real-world interactions of online RL. Scaling offline RL to increasingly complex datasets requires expressive generative models such as diffusion and flow matching. However, existing methods typically depend on either backpropagation through time (BPTT), which is computationally prohibitive, or policy distillation, which introduces compounding errors and limits scalability to larger base policies. In this paper, we consider the question of how to develop a scalable offline RL approach without relying on distillation or backpropagation through time. We introduce Expressive Value Learning for Offline Reinforcement Learning (EVOR): a scalable offline RL approach that integrates both expressive policies and expressive value functions. EVOR learns an optimal, regularized Q-function via flow matching during training. At inference-time, EVOR performs inference-time policy extraction via rejection sampling against the expressive value function, enabling efficient optimization, regularization, and compute-scalable search without retraining. Empirically, we show that EVOR outperforms baselines on a diverse set of offline RL tasks, demonstrating the benefit of integrating expressive value learning into offline RL.

Method: Add a small covariance-prediction module to pre-trained CBMs to model multivariate normal distribution over concepts, using two training strategies without retraining the backbone.

Result: PSCBMs consistently match or improve concept and target accuracy over standard CBMs, and perform much better under interventions while being more efficient than full retraining.

Conclusion: PSCBMs provide an efficient way to enhance CBMs with concept dependency modeling, achieving better performance under interventions without the computational cost of full retraining.

Abstract: Concept Bottleneck Models (CBMs) are interpretable models that predict the target variable through high-level human-understandable concepts, allowing users to intervene on mispredicted concepts to adjust the final output. While recent work has shown that modeling dependencies between concepts can improve CBM performance, especially under interventions, such approaches typically require retraining the entire model, which may be infeasible when access to the original data or compute is limited. In this paper, we introduce Post-hoc Stochastic Concept Bottleneck Models (PSCBMs), a lightweight method that augments any pre-trained CBM with a multivariate normal distribution over concepts by adding only a small covariance-prediction module, without retraining the backbone model. We propose two training strategies and show on real-world data that PSCBMs consistently match or improve both concept and target accuracy over standard CBMs at test time. Furthermore, we show that due to the modeling of concept dependencies, PSCBMs perform much better than CBMs under interventions, while remaining far more efficient than retraining a similar stochastic model from scratch.

Method: Uncertainty-Aware Markov Decision Process (UAMDP) framework that couples Bayesian forecasting, posterior-sampling reinforcement learning via Thompson sampling, and planning under conditional value-at-risk (CVaR) constraints.

Result: Improves long-horizon forecasting accuracy (RMSE decreases by up to 25%, sMAPE by 32%), trading Sharpe ratio rises from 1.54 to 1.74, and maximum drawdown is roughly halved in high-frequency trading and retail inventory control domains.

Conclusion: Integrating calibrated probabilistic modeling, exploration aligned with posterior uncertainty, and risk-aware control yields a robust, generalizable approach to safer and more profitable sequential decision-making.

Abstract: Sequential decisions in volatile, high-stakes settings require more than maximizing expected return; they require principled uncertainty management. This paper presents the Uncertainty-Aware Markov Decision Process (UAMDP), a unified framework that couples Bayesian forecasting, posterior-sampling reinforcement learning, and planning under a conditional value-at-risk (CVaR) constraint. In a closed loop, the agent updates its beliefs over latent dynamics, samples plausible futures via Thompson sampling, and optimizes policies subject to preset risk tolerances. We establish regret bounds that converge to the Bayes-optimal benchmark under standard regularity conditions. We evaluate UAMDP in two domains-high-frequency equity trading and retail inventory control-both marked by structural uncertainty and economic volatility. Relative to strong deep learning baselines, UAMDP improves long-horizon forecasting accuracy (RMSE decreases by up to 25% and sMAPE by 32%), and these gains translate into economic performance: the trading Sharpe ratio rises from 1.54 to 1.74 while maximum drawdown is roughly halved. These results show that integrating calibrated probabilistic modeling, exploration aligned with posterior uncertainty, and risk-aware control yields a robust, generalizable approach to safer and more profitable sequential decision-making.

Method: Distribution Matching Policy Optimization (DMPO) - a principled RL fine-tuning method that matches dLLM policy distribution to optimal reward-tilted distribution through cross-entropy optimization. Addresses challenges with small training batch sizes using novel weight baseline subtraction technique.

Result: Superior performance on multiple reasoning benchmarks without supervised fine-tuning, with accuracy improvements up to 42.9% over previous SOTA baselines and 55.8% over the base model.

Conclusion: DMPO effectively enhances dLLMs’ reasoning capabilities through the distribution matching framework, demonstrating significant performance improvements without requiring supervised fine-tuning.

Abstract: Diffusion large language models (dLLMs) are promising alternatives to autoregressive large language models (AR-LLMs), as they potentially allow higher inference throughput. Reinforcement learning (RL) is a crucial component for dLLMs to achieve comparable performance with AR-LLMs on important tasks, such as reasoning. However, RL algorithms that are well-suited for dLLMs’ unique characteristics have yet to be developed. This paper proposes Distribution Matching Policy Optimization (DMPO), a principled and theoretically grounded RL fine-tuning method specifically designed to enhance the reasoning capabilities of dLLMs by matching the dLLM policy distribution to the optimal, reward-tilted one through cross-entropy optimization. We identify a key challenge in the implementation with a small training batch size and propose several effective solutions through a novel weight baseline subtraction technique. DMPO exhibits superior performance on multiple reasoning benchmarks without supervised fine-tuning, with an accuracy improvement of up to $42.9%$ over previously SOTA baselines and $55.8%$ over the base model, underscoring the effectiveness of the distribution matching framework. Our code is available at https://github.com/yuchen-zhu-zyc/DMPO.

Method: Used the two-dimensional Political Compass Test (PCT) to assess inherent political leanings, employed persona prompting to explore explicit stereotypes, and evaluated models with multilingual PCT versions to uncover implicit stereotypes.

Result: All investigated models showed consistent left-leaning political alignment. Implicit stereotypes elicited through language variation were more pronounced than explicit ones, and most models showed notable alignment between implicit and explicit stereotypes.

Conclusion: The study reveals complex interplay of political bias and stereotypes in LLMs, with models showing awareness of their inherent biases through the alignment of implicit and explicit stereotypes.

Abstract: Large Language Models (LLMs) are increasingly integral to information dissemination and decision-making processes. Given their growing societal influence, understanding potential biases, particularly within the political domain, is crucial to prevent undue influence on public opinion and democratic processes. This work investigates political bias and stereotype propagation across eight prominent LLMs using the two-dimensional Political Compass Test (PCT). Initially, the PCT is employed to assess the inherent political leanings of these models. Subsequently, persona prompting with the PCT is used to explore explicit stereotypes across various social dimensions. In a final step, implicit stereotypes are uncovered by evaluating models with multilingual versions of the PCT. Key findings reveal a consistent left-leaning political alignment across all investigated models. Furthermore, while the nature and extent of stereotypes vary considerably between models, implicit stereotypes elicited through language variation are more pronounced than those identified via explicit persona prompting. Interestingly, for most models, implicit and explicit stereotypes show a notable alignment, suggesting a degree of transparency or “awareness” regarding their inherent biases. This study underscores the complex interplay of political bias and stereotypes in LLMs.

Method: Proposes Mix- and MoE-DPO framework using stochastic variational inference with latent-variable models over expert assignments, optimizing a variational ELBO to learn specialized expert policies from preference data.

Result: The method provides universal function approximation through mixtures, reward/policy specialization through expert components, and contextual alignment through input-dependent soft gating, validated on various model sizes and multi-preference datasets.

Conclusion: Mix- and MoE-DPO offers a powerful and scalable method for preference-based LLM alignment that overcomes limitations of standard DPO in handling diverse preferences.

Abstract: Direct Preference Optimization (DPO) has recently emerged as a simple and effective alternative to reinforcement learning from human feedback (RLHF) for aligning large language models (LLMs) with user preferences. However, existing DPO formulations rely on a single monolithic model, which limits their expressivity in multi-task settings and their adaptability to heterogeneous or diverse preference distributions. In this work, we propose Mix- and MoE-DPO, a framework that extends DPO with both soft mixture models and mixture-of-experts (MoE) architectures, using a stochastic variational inference approach. Our method introduces a latent-variable model over expert assignments and optimizes a variational evidence lower bound (ELBO), enabling stable and efficient learning of specialized expert policies from preference data. Mix- and MoE-DPO provides three key advantages over standard DPO: (i) generalization via universal function approximation through mixtures; (ii) reward and policy specialization through expert components tailored to distinct preference modes; and (iii) contextual alignment through input-dependent soft gating that enables user-specific mixture policies. Our framework supports both shared base architectures with expert-specific policy heads and fully independent expert models, allowing flexible trade-offs between parameter efficiency and specialization. We validate our approach on a variety of model sizes and multi-preference datasets, demonstrating that Mix- and MoE-DPO offers a powerful and scalable method for preference-based LLM alignment.

Method: Uses continuous-time flows and flow matching with tools from dynamic optimal transport to create unique, monotone and rank-preserving counterfactual transport maps, including non-Markovian settings under standard criteria.

Result: The method yields consistent counterfactual inference and demonstrates improvements in axiomatic counterfactual soundness on real images, validated in controlled scenarios with counterfactual ground-truth.

Conclusion: The approach provides a solid foundation for multivariate counterfactual identification that ensures causal validity and addresses limitations of previous methods.

Abstract: We address the open question of counterfactual identification for high-dimensional multivariate outcomes from observational data. Pearl (2000) argues that counterfactuals must be identifiable (i.e., recoverable from the observed data distribution) to justify causal claims. A recent line of work on counterfactual inference shows promising results but lacks identification, undermining the causal validity of its estimates. To address this, we establish a foundation for multivariate counterfactual identification using continuous-time flows, including non-Markovian settings under standard criteria. We characterise the conditions under which flow matching yields a unique, monotone and rank-preserving counterfactual transport map with tools from dynamic optimal transport, ensuring consistent inference. Building on this, we validate the theory in controlled scenarios with counterfactual ground-truth and demonstrate improvements in axiomatic counterfactual soundness on real images.

Method: Proposes a hierarchical framework combining Fourier Neural Operators (FNOs) for learning physical operators with Conditional Flow Matching (CFM) for probabilistic generation, integrated via time-dependent hierarchical constraints and FNO-guided corrections.

Result: Experiments on harmonic oscillators, human activity recognition, and lithium-ion battery degradation show 16.3% higher generation quality, 46% fewer physics violations, and 18.5% improved predictive accuracy over baselines.

Conclusion: The framework introduces a new paradigm of physics-informed inductive bias that significantly improves both statistical and physical consistency in time-series generation.

Abstract: Conventional time-series generation often ignores domain-specific physical constraints, limiting statistical and physical consistency. We propose a hierarchical framework that embeds the inherent hierarchy of physical laws-conservation, dynamics, boundary, and empirical relations-directly into deep generative models, introducing a new paradigm of physics-informed inductive bias. Our method combines Fourier Neural Operators (FNOs) for learning physical operators with Conditional Flow Matching (CFM) for probabilistic generation, integrated via time-dependent hierarchical constraints and FNO-guided corrections. Experiments on harmonic oscillators, human activity recognition, and lithium-ion battery degradation show 16.3% higher generation quality, 46% fewer physics violations, and 18.5% improved predictive accuracy over baselines.

Method: Proposes Adaptive Random Forests (ARFs) for dynamic feature adaptation and Drift-Aware Feature Importance (DAFI) as an XAI method that uses distributional drift detection to optimize when to apply computationally intensive feature importance methods.

Result: Iterative ARF training leads to stable predictions with improving accuracy over time. DAFI reduces runtime by up to 2 times while producing more consistent feature importance values across 3 different datasets.

Conclusion: ARFs and DAFI together provide a promising framework to build flexible AI methods adapted to 6G network use-cases, addressing both adaptability and explainability needs.

Abstract: As AI becomes a native component of 6G network control, AI models must adapt to continuously changing conditions, including the introduction of new features and measurements driven by multi-vendor deployments, hardware upgrades, and evolving service requirements. To address this growing need for flexible learning in non-stationary environments, this vision paper highlights Adaptive Random Forests (ARFs) as a reliable solution for dynamic feature adaptation in communication network scenarios. We show that iterative training of ARFs can effectively lead to stable predictions, with accuracy improving over time as more features are added. In addition, we highlight the importance of explainability in AI-driven networks, proposing Drift-Aware Feature Importance (DAFI) as an efficient XAI feature importance (FI) method. DAFI uses a distributional drift detector to signal when to apply computationally intensive FI methods instead of lighter alternatives. Our tests on 3 different datasets indicate that our approach reduces runtime by up to 2 times, while producing more consistent feature importance values. Together, ARFs and DAFI provide a promising framework to build flexible AI methods adapted to 6G network use-cases.

Method: Uses pull-based epidemic communication where nodes pull model parameters from small random subsets of nodes, eliminating the need for a central server and reducing communication overhead.

Result: RPEL maintains robustness in adversarial settings, achieves accuracy comparable to all-to-all communication, and scales efficiently across large networks with reduced communication costs.

Conclusion: RPEL provides a scalable, decentralized solution for robust collaborative learning that overcomes the limitations of existing approaches while maintaining strong convergence guarantees.

Abstract: Collaborative machine learning is challenged by training-time adversarial behaviors. Existing approaches to tolerate such behaviors either rely on a central server or induce high communication costs. We propose Robust Pull-based Epidemic Learning (RPEL), a novel, scalable collaborative approach to ensure robust learning despite adversaries. RPEL does not rely on any central server and, unlike traditional methods, where communication costs grow in $\mathcal{O}(n^2)$ with the number of nodes $n$, RPEL employs a pull-based epidemic-based communication strategy that scales in $\mathcal{O}(n \log n)$. By pulling model parameters from small random subsets of nodes, RPEL significantly lowers the number of required messages without compromising convergence guarantees, which hold with high probability. Empirical results demonstrate that RPEL maintains robustness in adversarial settings, competes with all-to-all communication accuracy, and scales efficiently across large networks.

Method: LtA uses a two-part architecture with a standard ML model and an enriched model trained with expert human feedback, plus optimal querying strategy. Two implementations: sequential training and joint optimization with surrogate losses.

Result: Experiments with synthetic and real expert data show LtA provides more flexible and powerful human-AI collaboration compared to existing approaches.

Conclusion: LtA offers a superior framework for effective human-AI collaboration by enabling models to ask for expert input in both when and how to incorporate it, going beyond simple prediction deferral.

Abstract: Developing decision-support systems that complement human performance in classification tasks remains an open challenge. A popular approach, Learning to Defer (LtD), allows a Machine Learning (ML) model to pass difficult cases to a human expert. However, LtD treats humans and ML models as mutually exclusive decision-makers, restricting the expert contribution to mere predictions. To address this limitation, we propose Learning to Ask (LtA), a new framework that handles both when and how to incorporate expert input in an ML model. LtA is based on a two-part architecture: a standard ML model and an enriched model trained with additional expert human feedback, with a formally optimal strategy for selecting when to query the enriched model. We provide two practical implementations of LtA: a sequential approach, which trains the models in stages, and a joint approach, which optimises them simultaneously. For the latter, we design surrogate losses with realisable-consistency guarantees. Our experiments with synthetic and real expert data demonstrate that LtA provides a more flexible and powerful foundation for effective human-AI collaboration.

Method: Theoretical analysis of single-layer attention-only transformers, plus empirical validation through random hyperparameter search on set complement tasks and testing OthelloGPT.

Result: Proved tight bounds on embedding dimensions and showed that balanced logit displacement at short lengths enables generalization to longer sequences. Found that dropout helps with softmax compression and EMA addresses noisy updates.

Conclusion: Length generalization failures stem from softmax compression and noisy training dynamics, but these can be mitigated with dropout and EMA, as demonstrated in both simple set complement and complex OthelloGPT settings.

Abstract: We study length generalization in transformers through the set complement task, where a model must predict a uniform distribution over tokens absent from an input sequence – an ability central to board-game style reasoning. Our main theoretical result establishes two statements. First, we prove tight bounds on embedding and value dimensions for single-layer attention-only transformers. Second, we show that if such a model achieves balanced logit displacement at lengths 1 and 2, then it must generalize to longer sequences, though with reduced precision. A mechanistic reading of the proof explains this limitation: as more tokens are attended to, softmax compresses logit displacements, eroding separation between valid and invalid outputs. Training dynamics also suggest a second obstacle: when many next tokens are possible, updates become noisy. We hypothesize that dropout can counteract the first effect and Exponential Moving Average (EMA) the second. We validate these hypotheses through random hyperparameter search on the set complement task, which confirms both mechanisms. We then test OthelloGPT, a GPT-1 style model trained on random Othello moves, and find that EMA again improves length generalization in this more complex setting.

Method: Offline training on 11,000+ ICU patients using dueling double deep Q-network with conservative Q-learning regularization, generating 4-hourly recommendations for caloric, protein, and fluid intake.

Result: Achieved 3.7±0.17 percentage-point mortality reduction (18.8% vs 22.5%) and improvements in nutritional biomarkers, outperforming clinician and guideline-based policies.

Conclusion: DeepEN demonstrates the potential of safe, data-driven personalization of enteral nutrition therapy to improve patient outcomes beyond traditional approaches.

Abstract: We introduce DeepEN, a deep reinforcement learning (RL) framework for personalized enteral nutrition (EN) in critically ill patients. Trained offline on over 11,000 ICU patients from the MIMIC-IV database, DeepEN generates 4-hourly recommendations for caloric, protein, and fluid intake tailored to each patient’s evolving physiology. The model integrates a curated, clinically informed state space with a custom reward function that balances short-term physiological and nutrition-related goals with long-term survival outcomes. Using a dueling double deep Q-network with conservative Q-learning regularization, DeepEN learns clinically realistic policies that align with high-value clinician actions while discouraging unsafe deviations. Across various qualitative and quantitative metrics, DeepEN outperforms clinician-derived and guideline-based policies, achieving a 3.7 $\pm$ 0.17 percentage-point reduction in estimated mortality (18.8% vs 22.5%) and improvements in key nutritional biomarkers. These findings highlight the potential of safe, data-driven personalization of EN therapy to improve outcomes beyond traditional guideline- or heuristic-based approaches.

Method: Reformulates generation using star-shaped paradigm for error correction, augmented with learnable re-masking scheduler that identifies and revises likely errors. Requires lightweight fine-tuning of a single layer.

Result: Substantial quality boost in low-step regimes, outperforms or matches existing methods in text and code generation experiments.

Conclusion: The proposed sampling algorithm effectively addresses limitations of traditional diffusion models by enabling error correction, significantly improving sample quality and efficiency with minimal fine-tuning requirements.

Abstract: The performance of pre-trained masked diffusion models is often constrained by their sampling procedure, which makes decisions irreversible and struggles in low-step generation regimes. We introduce a novel sampling algorithm that works with pre-trained models and, after a lightweight fine-tuning of a single layer, significantly improves sample quality and efficiency. Our method reformulates the generation process using a star-shaped paradigm, which inherently allows for error correction. To make this process effective, we augment it with a learnable re-masking scheduler that intelligently identifies and revises likely errors. This approach yields a substantial quality boost, particularly when using a small number of sampling steps. We extensively ablate key components of our approach and show its usability in different scenarios. In comprehensive experiments on text, and code generation, our sampling algorithm outperforms or matches existing methods.

Method: Systematic benchmarking strategy including energy profiling and reduced training data conditions, evaluating four CL frameworks and lightweight neural architectures.

Result: SimCLR demonstrates the lowest energy consumption across various data regimes, contrary to its perceived computational cost.

Conclusion: Provides insights into resource implications of deploying CL in edge/fog environments and opens research directions for future optimization.

Abstract: While contrastive learning (CL) shows considerable promise in self-supervised representation learning, its deployment on resource-constrained devices remains largely underexplored. The substantial computational demands required for training conventional CL frameworks pose a set of challenges, particularly in terms of energy consumption, data availability, and memory usage. We conduct an evaluation of four widely used CL frameworks: SimCLR, MoCo, SimSiam, and Barlow Twins. We focus on the practical feasibility of these CL frameworks for edge and fog deployment, and introduce a systematic benchmarking strategy that includes energy profiling and reduced training data conditions. Our findings reveal that SimCLR, contrary to its perceived computational cost, demonstrates the lowest energy consumption across various data regimes. Finally, we also extend our analysis by evaluating lightweight neural architectures when paired with CL frameworks. Our study aims to provide insights into the resource implications of deploying CL in edge/fog environments with limited processing capabilities and opens several research directions for its future optimization.

Method: Created a new combinatorial dimension based on Natarajan Dimension and proved equivalence between learnability and finiteness of this dimension.

Result: A hypothesis class is learnable if and only if the Generalized Natarajan Dimension is finite. Also established connection to learning with set-valued feedback.

Conclusion: Learnability of set learning problems is characterized by the Natarajan Dimension.

Abstract: In this paper we will give a characterization of the learnability of forgiving 0-1 loss functions in the finite label multiclass setting. To do this, we create a new combinatorial dimension that is based off of the Natarajan Dimension \citep{natarajan1989learning} and we show that a hypothesis class is learnable in our setting if and only if this Generalized Natarajan Dimension is finite. We also show a connection to learning with set-valued feedback. Through our results we show that the learnability of a set learning problem is characterized by the Natarajan Dimension.

Method: Proposes FlyLoRA with: (1) rank-wise expert activation in up-projection matrix, (2) implicit router that unifies expert routing and down-projection using frozen sparse random projection matrix instead of dense trainable version.

Result: Extensive experiments across four domains (general knowledge, scientific QA, math reasoning, code generation) show consistent performance improvements over existing methods.

Conclusion: FlyLoRA resolves trade-off between intra-task decorrelation and computational efficiency while inherently mitigating inter-task interference, demonstrating how biological structures can inspire AI innovations.

Abstract: Low-Rank Adaptation (LoRA) is a widely used parameter-efficient fine-tuning method for foundation models, but it suffers from parameter interference, resulting in suboptimal performance. Although Mixture-of-Experts (MoE)-based LoRA variants show promise in mitigating intra-task correlations in single-task instruction tuning, they introduce additional router parameters and remain ineffective in multi-task model merging where inter-task interference arises. Inspired by the fly olfactory circuit, we propose FlyLoRA, an implicit MoE-based LoRA variant that introduces: (1) rank-wise expert activation in the up-projection matrix, and (2) an implicit router that unifies expert routing and down-projection, where a frozen sparse random projection matrix replaces the traditional dense trainable version. This design resolves the trade-off between intra-task decorrelation and computational efficiency by eliminating the need for an explicit router, while inherently mitigating inter-task interference due to the orthogonality property of random matrices. Extensive experiments across four domains – general knowledge understanding, scientific question answering, mathematical reasoning, and code generation – demonstrate consistent performance improvements over existing methods. Beyond empirical gains, FlyLoRA highlights how biological structures can inspire innovations in AI technologies. Code is available at https://github.com/gfyddha/FlyLoRA.

Method: Tested three supervised 2D SR models (RCAN, pix2pix, FSRCNN) and one unsupervised model (CycleGAN) on paired high- and low-resolution confocal images with pixel size increases of x2, x4, x8. Used biology-driven assessment including segmentation, connected components analysis, and 3D graph analysis of porosity connectivity.

Result: Standard image quality assessment metrics yielded inconsistent results that contradicted visual perception. Biology-driven assessment using porosity segmentation and 3D connectivity analysis provided better mechanistic interpretation, revealing model differences in sensitivity to weak intensity features and impact of non-linearity.

Conclusion: Generic image quality metrics are insufficient for evaluating super-resolution performance in dental porosity imaging. Biology-driven assessment focusing on specific structural features and 3D connectivity provides more meaningful evaluation of model performance.

Abstract: The mechanosensory system of teeth is currently believed to partly rely on Odontoblast cells stimulation by fluid flow through a porosity network extending through dentin. Visualizing the smallest sub-microscopic porosity vessels therefore requires the highest achievable resolution from confocal fluorescence microscopy, the current gold standard. This considerably limits the extent of the field of view to very small sample regions. To overcome this limitation, we tested different deep learning (DL) super-resolution (SR) models to allow faster experimental acquisitions of lower resolution images and restore optimal image quality by post-processing. Three supervised 2D SR models (RCAN, pix2pix, FSRCNN) and one unsupervised (CycleGAN) were applied to a unique set of experimentally paired high- and low-resolution confocal images acquired with different sampling schemes, resulting in a pixel size increase of x2, x4, x8. Model performance was quantified using a broad set of similarity and distribution-based image quality assessment (IQA) metrics, which yielded inconsistent results that mostly contradicted our visual perception. This raises the question of the relevance of such generic metrics to efficiently target the specific structure of dental porosity. To resolve this conflicting information, the generated SR images were segmented taking into account the specific scales and morphology of the porosity network and analysed by comparing connected components. Additionally, the capacity of the SR models to preserve 3D porosity connectivity throughout the confocal image stacks was evaluated using graph analysis. This biology-driven assessment allowed a far better mechanistic interpretation of SR performance, highlighting differences in model sensitivity to weak intensity features and the impact of non-linearity in image generation, which explains the failure of standard IQA metrics.

Method: Derived novel generalization bounds using more useful priors, formally analyzed how perplexity regularization tightens bounds by limiting exploration, and conducted empirical evaluation of bounds effectiveness and perplexity regularization benefits.

Result: Developed non-vacuous generalization bounds for data-scarce prompt optimization scenarios, showing perplexity regularization improves prompt generalization both theoretically and empirically.

Conclusion: Perplexity acts as an effective prior in prompt optimization, explaining widespread success of prompt engineering techniques and providing theoretical guarantees through improved generalization bounds that remain non-vacuous even with limited data.

Abstract: Many prompt engineering techniques have been successful in practice, even when optimizing over a large prompt space with with a small amount of task-specific data. Recent work has partially explained this success by showing generalization bounds which apply PAC-Bayes theory to the discrete prompt space, but they are non-vacuous only in data-rich scenarios. We argue that such widespread success can be more fully explained through more carefully considering data- or distribution-dependent perplexity, which acts as an effective prior and steers the optimization towards prompts that are more ``natural’’ for the task at hand. We derive novel generalization bounds that are non-vacuous for data-scarce prompt optimization via more useful priors, formally analyzing how perplexity regularization tightens these bounds by limiting exploration. Empirically, we explore both the bounds’ effectiveness and the practical benefits of perplexity regularization in improving prompt generalization.

Method: DGPO learns directly from group-level preferences using relative information of samples within groups, eliminating the need for stochastic policies and enabling the use of efficient deterministic ODE samplers.

Result: DGPO trains around 20 times faster than state-of-the-art methods and achieves superior performance on both in-domain and out-of-domain reward metrics.

Conclusion: DGPO resolves the conflict between RL preference optimization and efficient diffusion sampling by dispensing with the policy-gradient framework, enabling faster training and better performance with deterministic ODE samplers.

Abstract: While reinforcement learning methods such as Group Relative Preference Optimization (GRPO) have significantly enhanced Large Language Models, adapting them to diffusion models remains challenging. In particular, GRPO demands a stochastic policy, yet the most cost-effective diffusion samplers are based on deterministic ODEs. Recent work addresses this issue by using inefficient SDE-based samplers to induce stochasticity, but this reliance on model-agnostic Gaussian noise leads to slow convergence. To resolve this conflict, we propose Direct Group Preference Optimization (DGPO), a new online RL algorithm that dispenses with the policy-gradient framework entirely. DGPO learns directly from group-level preferences, which utilize relative information of samples within groups. This design eliminates the need for inefficient stochastic policies, unlocking the use of efficient deterministic ODE samplers and faster training. Extensive results show that DGPO trains around 20 times faster than existing state-of-the-art methods and achieves superior performance on both in-domain and out-of-domain reward metrics. Code is available at https://github.com/Luo-Yihong/DGPO.

Method: Models quoting as Risk-Aware Constrained Markov Decision Process (RA-CMDP), retrieves statutory and policy clauses from legal corpora, embeds them into agent observations, and uses them to constrain actions and generate natural language justifications.

Result: Reduces solvency violations by 51%, improves tail-risk performance by 27.9% (CVaR_0.10), achieves 88.2% accuracy in clause-grounded explanations with 87.4% precision and 91.1% recall.

Conclusion: Embedding legal context into both decision and explanation pathways enables interpretable, auditable, and regulation-aligned quoting behavior compliant with Solvency II, NAIC RBC, and EU AI Act.

Abstract: Reinsurance treaty pricing must satisfy stringent regulatory standards, yet current quoting practices remain opaque and difficult to audit. We introduce ClauseLens, a clause-grounded reinforcement learning framework that produces transparent, regulation-compliant, and risk-aware treaty quotes. ClauseLens models the quoting task as a Risk-Aware Constrained Markov Decision Process (RA-CMDP). Statutory and policy clauses are retrieved from legal and underwriting corpora, embedded into the agent’s observations, and used both to constrain feasible actions and to generate clause-grounded natural language justifications. Evaluated in a multi-agent treaty simulator calibrated to industry data, ClauseLens reduces solvency violations by 51%, improves tail-risk performance by 27.9% (CVaR_0.10), and achieves 88.2% accuracy in clause-grounded explanations with retrieval precision of 87.4% and recall of 91.1%. These findings demonstrate that embedding legal context into both decision and explanation pathways yields interpretable, auditable, and regulation-aligned quoting behavior consistent with Solvency II, NAIC RBC, and the EU AI Act.

Method: A learned router trained end-to-end with reinforcement learning using explicit, cost-aware reward functions. The router can either answer directly or invoke one or more external models, eliminating the need for hand-engineered routing rules.

Result: Achieves strong cost-performance trade-offs across diverse benchmarks, with substantial cost reductions at comparable task completion rates. Provides empirical insights into model trainability and orchestration behaviors in small open models.

Conclusion: xRouter serves as a practical foundation for advancing learned, cost-aware LLM orchestration, with open implementation available for further research and development.

Abstract: Modern LLM deployments confront a widening cost-performance spectrum: premium models deliver strong reasoning but are expensive, while lightweight models are economical yet brittle on complex tasks. Static escalation rules and keyword heuristics under-utilize this spectrum and fail to adapt across task types. We present xRouter, a tool-calling-based routing system in which a learned router can either answer directly or invoke one or more external models. The router is trained end-to-end with reinforcement learning using an explicit, cost-aware reward that encodes cost-performance trade-offs, eliminating the need for hand-engineered routing rules. Our implementation encompasses the full reinforcement learning framework, including reward and cost accounting, as well as the deployment and evaluation pipelines. Across diverse benchmarks, xRouter achieves strong cost-performance trade-offs (e.g., substantial cost reductions at comparable task completion rates), and provides empirical insights into what reliably helps learned routing and what does not, ranging from model trainability to the difficulty of eliciting sophisticated orchestration behaviors in small open models. We hope these findings and our open implementation will serve as a practical substrate for advancing learned, cost-aware LLM orchestration.

Method: Developed a series-symbol data generation mechanism that creates high-quality time series data paired with symbolic expressions, and built SymTime - a pre-trained foundation model that leverages these series-symbol data pairs with strong correlations.

Result: SymTime demonstrates competitive performance across five major time series analysis tasks when fine-tuned with downstream tasks, rivaling foundation models pre-trained on real-world datasets.

Conclusion: The approach shows the potential of series-symbol data generation and pretraining mechanisms in overcoming data scarcity and enhancing task performance in time series analysis.

Abstract: Foundation models for time series analysis (TSA) have attracted significant attention. However, challenges such as training data scarcity and imbalance continue to hinder their development. Inspired by complex dynamic system theories, we design a series-symbol data generation mechanism, enabling the unrestricted creation of high-quality time series data paired with corresponding symbolic expressions. To leverage series-symbol data pairs with strong correlations, we develop \texttt{SymTime}, a pre-trained foundation model for enhancing time series representation using symbolic information. \texttt{SymTime} demonstrates competitive performance across five major TSA tasks when fine-tunes with downstream tasks, rivaling foundation models pre-trained on real-world datasets. This approach underscores the potential of series-symbol data generation and pretraining mechanisms in overcoming data scarcity and enhancing task performance. The code is available at https://github.com/wwhenxuan/SymTime.

Method: The authors study limitations of existing over-squashing measures, introduce a new synthetic capacity task, and adapt concepts from sequence modeling (associative memories, fast weight programmers, xLSTM) to create a novel GNN architecture with enhanced storage and retrieval capacity.

Result: The proposed architecture demonstrates strong performance on both the synthetic capacity task and various real-world graph benchmarks, showing improved information handling capabilities.

Conclusion: By reframing over-squashing as a capacity issue and incorporating sequence modeling techniques, the authors develop a more effective GNN architecture that addresses information bottleneck problems in graph neural networks.

Abstract: Graph Neural Networks (GNNs) leverage the graph structure to transmit information between nodes, typically through the message-passing mechanism. While these models have found a wide variety of applications, they are known to suffer from over-squashing, where information from a large receptive field of node representations is collapsed into a single fixed sized vector, resulting in an information bottleneck. In this paper, we re-examine the over-squashing phenomenon through the lens of model storage and retrieval capacity, which we define as the amount of information that can be stored in a node’s representation for later use. We study some of the limitations of existing tasks used to measure over-squashing and introduce a new synthetic task to demonstrate that an information bottleneck can saturate this capacity. Furthermore, we adapt ideas from the sequence modeling literature on associative memories, fast weight programmers, and the xLSTM model to develop a novel GNN architecture with improved capacity. We demonstrate strong performance of this architecture both on our capacity synthetic task, as well as a range of real-world graph benchmarks.

Method: Proposes a nine-dimensional integral signature S_sigma(phi) combining Gaussian propagation statistics (m1, g1, g2, m2, eta), asymptotic slopes (alpha_plus, alpha_minus), and regularity measures (TV(phi’), C(phi)). Uses dynamical analysis with Lyapunov theorems and kernel perspective for dimension-free Hessian bounds.

Result: Classified eight standard activations (ReLU, leaky-ReLU, tanh, sigmoid, Swish, GELU, Mish, TeLU) proving sharp distinctions between saturating, linear-growth, and smooth families. Numerical validation confirms theoretical predictions.

Conclusion: The framework provides principled design guidance, moving activation function choice from trial-and-error to provable stability and kernel conditioning, establishing a rigorous taxonomy for neural network activation functions.

Abstract: Activation functions govern the expressivity and stability of neural networks, yet existing comparisons remain largely heuristic. We propose a rigorous framework for their classification via a nine-dimensional integral signature S_sigma(phi), combining Gaussian propagation statistics (m1, g1, g2, m2, eta), asymptotic slopes (alpha_plus, alpha_minus), and regularity measures (TV(phi’), C(phi)). This taxonomy establishes well-posedness, affine reparameterization laws with bias, and closure under bounded slope variation. Dynamical analysis yields Lyapunov theorems with explicit descent constants and identifies variance stability regions through (m2’, g2). From a kernel perspective, we derive dimension-free Hessian bounds and connect smoothness to bounded variation of phi’. Applying the framework, we classify eight standard activations (ReLU, leaky-ReLU, tanh, sigmoid, Swish, GELU, Mish, TeLU), proving sharp distinctions between saturating, linear-growth, and smooth families. Numerical Gauss-Hermite and Monte Carlo validation confirms theoretical predictions. Our framework provides principled design guidance, moving activation choice from trial-and-error to provable stability and kernel conditioning.

Method: Proposes SummDiff, a diffusion model that frames video summarization as conditional generation, allowing the model to learn distributions of good summaries and generate multiple candidate summaries conditioned on input videos.

Result: Achieves state-of-the-art performance on various benchmarks and produces summaries that closely align with individual annotator preferences. Also provides novel metrics for knapsack analysis.

Conclusion: SummDiff successfully addresses the subjectivity problem in video summarization through diffusion-based conditional generation, producing diverse and human-aligned summaries while advancing evaluation metrics.

Abstract: Video summarization is a task of shortening a video by choosing a subset of frames while preserving its essential moments. Despite the innate subjectivity of the task, previous works have deterministically regressed to an averaged frame score over multiple raters, ignoring the inherent subjectivity of what constitutes a good summary. We propose a novel problem formulation by framing video summarization as a conditional generation task, allowing a model to learn the distribution of good summaries and to generate multiple plausible summaries that better reflect varying human perspectives. Adopting diffusion models for the first time in video summarization, our proposed method, SummDiff, dynamically adapts to visual contexts and generates multiple candidate summaries conditioned on the input video. Extensive experiments demonstrate that SummDiff not only achieves the state-of-the-art performance on various benchmarks but also produces summaries that closely align with individual annotator preferences. Moreover, we provide a deeper insight with novel metrics from an analysis of the knapsack, which is an important last step of generating summaries but has been overlooked in evaluation.

Method: Uses pretrained LLMs’ attention matrices for clustering, applies spectral clustering on attention matrices, fine-tunes LLMs with Next Token Prediction loss for enhanced clustering on numeric and image data, and enables text-conditioned clustering through flexible prompting.

Result: LLMs demonstrate impressive zero-shot clustering capabilities on text-encoded numeric data, with attention matrices revealing clear cluster patterns. Spectral clustering using attention matrices achieves competitive performance, and fine-tuning further improves clustering on both numeric and image data.

Conclusion: ICC successfully extends in-context learning to unsupervised settings, showcasing LLMs’ effectiveness and flexibility for clustering tasks, including novel capabilities like text-conditioned image clustering that classical methods cannot achieve.

Abstract: We propose In-Context Clustering (ICC), a flexible LLM-based procedure for clustering data from diverse distributions. Unlike traditional clustering algorithms constrained by predefined similarity measures, ICC flexibly captures complex relationships among inputs through an attention mechanism. We show that pretrained LLMs exhibit impressive zero-shot clustering capabilities on text-encoded numeric data, with attention matrices showing salient cluster patterns. Spectral clustering using attention matrices offers surprisingly competitive performance. We further enhance the clustering capabilities of LLMs on numeric and image data through fine-tuning using the Next Token Prediction (NTP) loss. Moreover, the flexibility of LLM prompting enables text-conditioned image clustering, a capability that classical clustering methods lack. Our work extends in-context learning to an unsupervised setting, showcasing the effectiveness and flexibility of LLMs for clustering. Our code is available at https://agenticlearning.ai/icc.

Method: UML uses a single model that alternately processes inputs from different modalities while sharing parameters across them, exploiting the assumption that different modalities are projections of a shared underlying reality.

Result: Theoretically, unpaired auxiliary data yields more informative representations than unimodal training. Empirically, using unpaired data from auxiliary modalities consistently improves downstream performance across diverse unimodal targets.

Conclusion: Unpaired multimodal data can effectively enhance representation learning across modalities without requiring explicit paired datasets, demonstrating the value of leveraging abundant unpaired data for multimodal learning.

Abstract: Traditional multimodal learners find unified representations for tasks like visual question answering, but rely heavily on paired datasets. However, an overlooked yet potentially powerful question is: can one leverage auxiliary unpaired multimodal data to directly enhance representation learning in a target modality? We introduce UML: Unpaired Multimodal Learner, a modality-agnostic training paradigm in which a single model alternately processes inputs from different modalities while sharing parameters across them. This design exploits the assumption that different modalities are projections of a shared underlying reality, allowing the model to benefit from cross-modal structure without requiring explicit pairs. Theoretically, under linear data-generating assumptions, we show that unpaired auxiliary data can yield representations strictly more informative about the data-generating process than unimodal training. Empirically, we show that using unpaired data from auxiliary modalities – such as text, audio, or images – consistently improves downstream performance across diverse unimodal targets such as image and audio. Our project page: https://unpaired-multimodal.github.io/

Method: Formulates batch size optimization as a sequential decision-making problem using Proximal Policy Optimization (PPO) with multi-dimensional state representation including network metrics, system resource utilization, and training efficiency indicators.

Result: Achieves up to 6.3% improvement in final model accuracy and 46% reduction in total training time across diverse workloads, hardware configurations, and network conditions. Scales well to 32 nodes and learned policies generalize across related model architectures.

Conclusion: DYNAMIX provides an effective reinforcement learning-based solution for dynamic batch size optimization that adapts to heterogeneous environments and integrates seamlessly with existing distributed training frameworks.

Abstract: Existing batch size selection approaches in distributed machine learning rely on static allocation or simplistic heuristics that fail to adapt to heterogeneous, dynamic computing environments. We present DYNAMIX, a reinforcement learning framework that formulates batch size optimization as a sequential decision-making problem using Proximal Policy Optimization (PPO). Our approach employs a multi-dimensional state representation encompassing network-level metrics, system-level resource utilization, and training statistical efficiency indicators to enable informed decision-making across diverse computational resources. Our approach eliminates the need for explicit system modeling while integrating seamlessly with existing distributed training frameworks. Through evaluations across diverse workloads, hardware configurations, and network conditions, DYNAMIX achieves up to 6.3% improvement in the final model accuracy and 46% reduction in the total training time. Our scalability experiments demonstrate that DYNAMIX maintains the best performance as cluster size increases to 32 nodes, while policy transfer experiments show that learned policies generalize effectively across related model architectures.

Method: Theoretical analysis of RLVR training at full-response and token levels, focusing on Gradient Gap alignment and deriving step-size thresholds. Validated through bandit simulations and LLM experiments with Qwen2.5-7B.

Result: Proved convergence depends on aligning update direction with Gradient Gap, derived sharp step-size threshold, and explained practical heuristics like length normalization. Validated predictions through experiments.

Conclusion: The theory explains RLVR’s empirical success, provides convergence guarantees, and shows why fixed learning rates can limit success rates below 100%.

Abstract: Reinforcement Learning with Verifiable Rewards (RLVR), which uses simple binary feedback to post-train large language models, has shown significant empirical success. However, a principled understanding of why it works has been lacking. This paper builds a theoretical foundation for RLVR by analyzing its training process at both the full-response (trajectory) and token levels. Central to our analysis is a quantity called the Gradient Gap, which formalizes the direction of improvement from low-reward to high-reward regions of the response space. We prove that convergence critically depends on aligning the update direction with this Gradient Gap. Moreover, we derive a sharp step-size threshold based on the magnitude of the Gradient Gap: below it, learning converges, whereas above it, performance collapses. Our theory further predicts how the critical step size must scale with response length and the success rate, thereby explaining why practical heuristics such as length normalization improve stability and showing that, with a fixed learning rate, the success rate can stagnate strictly below $100%$. We validate these predictions through controlled bandit simulations and LLM experiments, including training Qwen2.5-7B with GRPO.

Method: Uses vanishing entropy regularization and a temperature decoupling gambit to ensure convergence to interpretable optimal policies that preserve diversity.

Result: The framework guarantees convergence to particular optimal policies and ensures convergence of value functions and return distributions. In one instance, the realized policy samples all optimal actions uniformly.

Conclusion: The method enables estimation of return distributions for interpretable, diversity-preserving optimal policies with arbitrary accuracy.

Abstract: In the pursuit of finding an optimal policy, reinforcement learning (RL) methods generally ignore the properties of learned policies apart from their expected return. Thus, even when successful, it is difficult to characterize which policies will be learned and what they will do. In this work, we present a theoretical framework for policy optimization that guarantees convergence to a particular optimal policy, via vanishing entropy regularization and a temperature decoupling gambit. Our approach realizes an interpretable, diversity-preserving optimal policy as the regularization temperature vanishes and ensures the convergence of policy derived objects–value functions and return distributions. In a particular instance of our method, for example, the realized policy samples all optimal actions uniformly. Leveraging our temperature decoupling gambit, we present an algorithm that estimates, to arbitrary accuracy, the return distribution associated to its interpretable, diversity-preserving optimal policy.

Method: Apply specially designed activations to model outputs to constrain sampling entropy above given thresholds, maintaining entropy above minimum levels during inference.

Result: Achieved 37.4% boost in AIME 2025 score for Qwen2.5-Math-7B, over 30% performance improvement on HumanoidBench for RL agents, and 0.69% ImageNet top-1 accuracy gain for ResNet-50, all with less than 7% computational overhead.

Conclusion: Output activation serves as a powerful tool for entropy control, opening new directions for designing simpler and more robust algorithms across various machine learning domains.

Abstract: We propose ERA, a new paradigm that constrains the sampling entropy above given thresholds by applying specially designed activations to the outputs of models. Our approach demonstrates broad effectiveness across different domains:

1. for large language models(LLMs), boosting the AIME 2025 score for Qwen2.5-Math-7B by 37.4%; 2) for continuous control reinforcement learning agents, improving performance by more than 30% over strong baselines such as SAC on the challenging HumanoidBench; 3) for image classification, enhancing ImageNet top-1 accuracy by 0.69% for ResNet-50. These gains are achieved with a computational overhead of less than 7%. Our work validates output activation as a powerful tool for entropy control, opening a new direction for designing simpler and more robust algorithms.

Method: GDPO uses semi-deterministic Monte Carlo schemes to reduce variance in ELBO estimation by employing fast, deterministic integral approximations along key dimensions, avoiding the variance explosion of vanilla double Monte Carlo sampling.

Result: GDPO achieves consistent improvements over pretrained checkpoints and outperforms diffu-GRPO on most math, reasoning, and coding benchmarks.

Conclusion: GDPO provides an effective RL fine-tuning approach for diffusion language models by addressing the variance problem in ELBO estimation, enabling better performance than existing methods.

Abstract: Diffusion language models (DLMs) enable parallel, order-agnostic generation with iterative refinement, offering a flexible alternative to autoregressive large language models (LLMs). However, adapting reinforcement learning (RL) fine-tuning to DLMs remains an open challenge because of the intractable likelihood. Pioneering work such as diffu-GRPO estimated token-level likelihoods via one-step unmasking. While computationally efficient, this approach is severely biased. A more principled foundation lies in sequence-level likelihoods, where the evidence lower bound (ELBO) serves as a surrogate. Yet, despite this clean mathematical connection, ELBO-based methods have seen limited adoption due to the prohibitive cost of likelihood evaluation. In this work, we revisit ELBO estimation and disentangle its sources of variance. This decomposition motivates reducing variance through fast, deterministic integral approximations along a few pivotal dimensions. Building on this insight, we introduce \textbf{Group Diffusion Policy Optimization (GDPO)}, a new RL algorithm tailored for DLMs. GDPO leverages simple yet effective Semi-deterministic Monte Carlo schemes to mitigate the variance explosion of ELBO estimators under vanilla double Monte Carlo sampling, yielding a provably lower-variance estimator under tight evaluation budgets. Empirically, GDPO achieves consistent gains over pretrained checkpoints and outperforms diffu-GRPO, one of the state-of-the-art baselines, on the majority of math, reasoning, and coding benchmarks.

Method: Sandwich a linear operator A between two invertible neural networks: f(x)=g_y^{-1}(A g_x(x)), which induces vector spaces with newly defined addition and scaling operations derived from g_x and g_y.

Result: The framework enables application of SVD, pseudo-inverse, orthogonal projection to nonlinear mappings. Composition of Linearizers collapses diffusion model sampling from hundreds to single step, enables globally projective generative models, and modular style transfer.

Conclusion: The Linearizer architecture successfully transforms nonlinear neural networks into linear mappings in constructed vector spaces, making linear algebra tools applicable to nonlinear problems and enabling significant efficiency gains in applications like diffusion models.

Abstract: Neural networks are famously nonlinear. However, linearity is defined relative to a pair of vector spaces, $f$$:$$X$$\to$$Y$. Is it possible to identify a pair of non-standard vector spaces for which a conventionally nonlinear function is, in fact, linear? This paper introduces a method that makes such vector spaces explicit by construction. We find that if we sandwich a linear operator $A$ between two invertible neural networks, $f(x)=g_y^{-1}(A g_x(x))$, then the corresponding vector spaces $X$ and $Y$ are induced by newly defined addition and scaling actions derived from $g_x$ and $g_y$. We term this kind of architecture a Linearizer. This framework makes the entire arsenal of linear algebra, including SVD, pseudo-inverse, orthogonal projection and more, applicable to nonlinear mappings. Furthermore, we show that the composition of two Linearizers that share a neural network is also a Linearizer. We leverage this property and demonstrate that training diffusion models using our architecture makes the hundreds of sampling steps collapse into a single step. We further utilize our framework to enforce idempotency (i.e. $f(f(x))=f(x)$) on networks leading to a globally projective generative model and to demonstrate modular style transfer.

Method: Introduces a temporal difference version of contrastive predictive coding that combines pieces of different time series data to reduce data requirements.

Result: Achieves 2x median improvement in success rates, better handles stochastic environments, and shows 20x more sample efficient than successor representation and 1500x more efficient than standard contrastive predictive coding in tabular settings.

Conclusion: The proposed temporal difference contrastive predictive coding method significantly improves sample efficiency and performance in goal-conditioned reinforcement learning tasks.

Abstract: Predicting and reasoning about the future lie at the heart of many time-series questions. For example, goal-conditioned reinforcement learning can be viewed as learning representations to predict which states are likely to be visited in the future. While prior methods have used contrastive predictive coding to model time series data, learning representations that encode long-term dependencies usually requires large amounts of data. In this paper, we introduce a temporal difference version of contrastive predictive coding that stitches together pieces of different time series data to decrease the amount of data required to learn predictions of future events. We apply this representation learning method to derive an off-policy algorithm for goal-conditioned RL. Experiments demonstrate that, compared with prior RL methods, ours achieves $2 \times$ median improvement in success rates and can better cope with stochastic environments. In tabular settings, we show that our method is about $20 \times$ more sample efficient than the successor representation and $1500 \times$ more sample efficient than the standard (Monte Carlo) version of contrastive predictive coding.

Method: Uses Transformer neural networks with Masked Auto-Encoder architecture, self-supervised pre-training on multiple datasets, task-specific fine-tuning, and explores model size vs accuracy trade-offs with Knowledge Distillation.

Result: Achieved 99.9% accuracy for anomaly detection with only 15 monitoring windows (vs 95.03% with 120 windows for PCA). For traffic load estimation, achieved R² scores of 0.97/0.90 for light/heavy vehicles (vs 0.91/0.84 for Random Forest) and 0.54 R² (vs 0.51 for LSTM).

Conclusion: Transformer foundation models with MAE architecture outperform traditional methods in SHM tasks, enabling high accuracy with reduced monitoring time and suitable for edge deployment.

Abstract: Structural Health Monitoring (SHM) is a critical task for ensuring the safety and reliability of civil infrastructures, typically realized on bridges and viaducts by means of vibration monitoring. In this paper, we propose for the first time the use of Transformer neural networks, with a Masked Auto-Encoder architecture, as Foundation Models for SHM. We demonstrate the ability of these models to learn generalizable representations from multiple large datasets through self-supervised pre-training, which, coupled with task-specific fine-tuning, allows them to outperform state-of-the-art traditional methods on diverse tasks, including Anomaly Detection (AD) and Traffic Load Estimation (TLE). We then extensively explore model size versus accuracy trade-offs and experiment with Knowledge Distillation (KD) to improve the performance of smaller Transformers, enabling their embedding directly into the SHM edge nodes. We showcase the effectiveness of our foundation models using data from three operational viaducts. For AD, we achieve a near-perfect 99.9% accuracy with a monitoring time span of just 15 windows. In contrast, a state-of-the-art method based on Principal Component Analysis (PCA) obtains its first good result (95.03% accuracy), only considering 120 windows. On two different TLE tasks, our models obtain state-of-the-art performance on multiple evaluation metrics (R$^2$ score, MAE% and MSE%). On the first benchmark, we achieve an R$^2$ score of 0.97 and 0.90 for light and heavy vehicle traffic, respectively, while the best previous approach (a Random Forest) stops at 0.91 and 0.84. On the second one, we achieve an R$^2$ score of 0.54 versus the 0.51 of the best competitor method, a Long-Short Term Memory network.

Method: Used one week of non-dominant wrist actimetry data from 78 participants, segmented into daytime activity and nighttime rest. Calculated statistical and entropy descriptors (256 features) and employed K-nearest neighbors pipeline with five-fold stratified cross-validation, using SHAP for interpretation.

Result: For binary food addiction classification, activity-segment features performed best (MCC = 0.78, Accuracy ~95.3%, Sensitivity ~0.77, Specificity ~0.98), outperforming combined objective-subjective features and rest-only features. For symptom count classification, combined features slightly outperformed actimetry alone.

Conclusion: Wrist-worn actimetry serves as a promising digital biomarker for food addiction that complements questionnaires and may facilitate privacy-preserving clinical translation, with emotional and restrained eating correlating with actimetric features.

Abstract: Wearable sensors and IoT/IoMT platforms enable continuous, real-time monitoring, but objective digital markers for eating disorders are limited. In this study, we examined whether actimetry and machine learning (ML) could provide objective criteria for food addiction (FA) and symptom counts (SC). In 78 participants (mean age 22.1 +/- 9.5 y; 73.1% women), one week of non-dominant wrist actimetry and psychometric data (YFAS, DEBQ, ZSDS) were collected. The time series were segmented into daytime activity and nighttime rest, and statistical and entropy descriptors (FuzzyEn, DistEn, SVDEn, PermEn, PhaseEn; 256 features) were calculated. The mean Matthews correlation coefficient (MCC) was used as the primary metric in a K-nearest neighbors (KNN) pipeline with five-fold stratified cross-validation (one hundred repetitions; 500 evaluations); SHAP was used to assist in interpretation. For binary FA, activity-segment features performed best (MCC = 0.78 +/- 0.02; Accuracy ~ 95.3% +/- 0.5; Sensitivity ~ 0.77 +/- 0.03; Specificity ~ 0.98 +/- 0.004), exceeding OaS (Objective and Subjective Features) (MCC = 0.69 +/- 0.03) and rest-only (MCC = 0.50 +/- 0.03). For SC (four classes), OaS slightly surpassed actimetry (MCC = 0.40 +/- 0.01 vs 0.38 +/- 0.01; Accuracy ~ 58.1% vs 56.9%). Emotional and restrained eating were correlated with actimetric features. These findings support wrist-worn actimetry as a digital biomarker of FA that complements questionnaires and may facilitate privacy-preserving clinical translation.

Method: Using LLM embeddings over string inputs for in-context regression in Bayesian Optimization, representing inputs as strings to enable general-purpose regression.

Result: Achieves optimization performance comparable to state-of-the-art Gaussian Process-based methods like Google Vizier, works across synthetic, combinatorial, and hyperparameter optimization domains.

Conclusion: LLM embeddings with string inputs provide broader and more flexible applications for Bayesian Optimization while maintaining competitive performance with traditional methods.

Abstract: Bayesian Optimization is ubiquitous in experimental design and black-box optimization for improving search efficiency. However, most existing approaches rely on regression models which are limited to fixed search spaces and structured, tabular input features. This paper explores the use of LLM embeddings over string inputs for in-context regression in Bayesian Optimization. Our results show that representing inputs as strings enables general-purpose regression across diverse domains, including synthetic, combinatorial, and hyperparameter optimization. Furthermore, our approach achieves optimization performance comparable to state-of-the-art Gaussian Process-based methods such as Google Vizier, and demonstrates potential for broader and more flexible applications.

Method: Blockchain-enabled federated learning with adaptation for privacy and safety requirements, featuring on-chain incentive mechanisms to reward honest participation and penalize malicious activities.

Result: The framework is effective, efficient, and privacy-preserving, consistently outperforming models trained on limited personal data and achieving comparable or slightly better results than centralized training in certain scenarios.

Conclusion: This work enables international healthcare collaborations where additional data is crucial for reducing bias and providing benefits to humanity while preserving data privacy.

Abstract: One of the biggest challenges of building artificial intelligence (AI) model in the healthcare area is the data sharing. Since healthcare data is private, sensitive, and heterogeneous, collecting sufficient data for modelling is exhausting, costly, and sometimes impossible. In this paper, we propose a framework for global healthcare modelling using datasets from multi-continents (Europe, North America, and Asia) without sharing the local datasets, and choose glucose management as a study model to verify its effectiveness. Technically, blockchain-enabled federated learning is implemented with adaptation to meet the privacy and safety requirements of healthcare data, meanwhile, it rewards honest participation and penalizes malicious activities using its on-chain incentive mechanism. Experimental results show that the proposed framework is effective, efficient, and privacy-preserving. Its prediction accuracy consistently outperforms models trained on limited personal data and achieves comparable or even slightly better results than centralized training in certain scenarios, all while preserving data privacy. This work paves the way for international collaborations on healthcare projects, where additional data is crucial for reducing bias and providing benefits to humanity.

Method: Treat black-box LLM as environment, M-Pilot as policy that provides intermediate guidance through prompts. Trained to align black-box LLM outputs with preferences during iterative interaction.

Result: Empirical evaluations show M-Pilot effectively enhances black-box LLM capabilities in complex, long-horizon tasks.

Conclusion: M-Pilot enables controllable generation and self-improvement for black-box LLMs without requiring access to their internal parameters.

Abstract: Despite the impressive generative abilities of black-box large language models (LLMs), their inherent opacity hinders further advancements in capabilities such as reasoning, planning, and personalization. Existing works aim to enhance LLM capabilities via domain-specific adaptation, which require additional training on accessible model parameters, an infeasible option for black-box LLMs. To address this challenge, we introduce Matryoshka Pilot (M-Pilot), a lightweight white-box LLM controller that guides a large-scale black-box LLM generator by decomposing complex tasks into a series of intermediate outputs. Specifically, we consider the black-box LLM as an environment, with M-Pilot serving as a policy to provide intermediate guidance through prompts for driving the black-box LLM. M-Pilot is trained to pivot the outputs of the black-box LLM aligning with preferences during iterative interaction, which enables controllable multi-turn generation and self-improvement in optimizing intermediate guidance. Empirical evaluations on diverse tasks demonstrate that our method effectively enhances the capabilities of black-box LLMs in complex, long-horizon tasks.

Method: Leverage Universum points to support minority class within quadratic twin SVM models, using quadratic surfaces instead of hyperplanes for more flexible decision boundaries.

Result: Enhanced classification accuracy and generalization on imbalanced datasets. Superior performance demonstrated on both artificial and benchmark datasets compared to conventional classifiers.

Conclusion: The proposed method effectively addresses imbalanced classification challenges by combining Universum points with quadratic twin SVM, offering improved flexibility and performance over existing approaches.

Abstract: Binary classification tasks with imbalanced classes pose significant challenges in machine learning. Traditional classifiers often struggle to accurately capture the characteristics of the minority class, resulting in biased models with subpar predictive performance. In this paper, we introduce a novel approach to tackle this issue by leveraging Universum points to support the minority class within quadratic twin support vector machine models. Unlike traditional classifiers, our models utilize quadratic surfaces instead of hyperplanes for binary classification, providing greater flexibility in modeling complex decision boundaries. By incorporating Universum points, our approach enhances classification accuracy and generalization performance on imbalanced datasets. We generated four artificial datasets to demonstrate the flexibility of the proposed methods. Additionally, we validated the effectiveness of our approach through empirical evaluations on benchmark datasets, showing superior performance compared to conventional classifiers and existing methods for imbalanced classification.

Method: Proposes HiVeGen framework that decomposes generation tasks into hierarchical submodules, integrates automatic Design Space Exploration, uses weight-based retrieval for code reuse, and enables real-time human-computer interaction.

Result: Significantly improves the quality of generated hardware designs by addressing LLM limitations in handling complex hierarchical structures.

Conclusion: HiVeGen effectively extends LLM capabilities to Hardware Description Language generation through hierarchical decomposition and interactive design approaches.

Abstract: With Large Language Models (LLMs) recently demonstrating impressive proficiency in code generation, it is promising to extend their abilities to Hardware Description Language (HDL). However, LLMs tend to generate single HDL code blocks rather than hierarchical structures for hardware designs, leading to hallucinations, particularly in complex designs like Domain-Specific Accelerators (DSAs). To address this, we propose HiVeGen, a hierarchical LLM-based Verilog generation framework that decomposes generation tasks into LLM-manageable hierarchical submodules. HiVeGen further harnesses the advantages of such hierarchical structures by integrating automatic Design Space Exploration (DSE) into hierarchy-aware prompt generation, introducing weight-based retrieval to enhance code reuse, and enabling real-time human-computer interaction to lower error-correction cost, significantly improving the quality of generated designs.

Method: Uses self-guided skill refinement with decoupled high-level and skill improvement policies, and applies skill prioritization via maximum return relabeling to focus updates on task-relevant trajectories.

Result: Achieves reliable skill learning and consistently outperforms other skill-based meta-RL methods on diverse long-horizon tasks by mitigating noise effects.

Conclusion: SISL provides robust and stable adaptation in skill-based meta-RL even under noisy and suboptimal data conditions.

Abstract: Meta-reinforcement learning (Meta-RL) facilitates rapid adaptation to unseen tasks but faces challenges in long-horizon environments. Skill-based approaches tackle this by decomposing state-action sequences into reusable skills and employing hierarchical decision-making. However, these methods are highly susceptible to noisy offline demonstrations, leading to unstable skill learning and degraded performance. To address this, we propose Self-Improving Skill Learning (SISL), which performs self-guided skill refinement using decoupled high-level and skill improvement policies, while applying skill prioritization via maximum return relabeling to focus updates on task-relevant trajectories, resulting in robust and stable adaptation even under noisy and suboptimal data. By mitigating the effect of noise, SISL achieves reliable skill learning and consistently outperforms other skill-based meta-RL methods on diverse long-horizon tasks.

Method: Constructs reversible solvers by applying Lawson methods to create exponential Runge-Kutta methods for diffusion models, leveraging connections to algebraically reversible differential equation solvers.

Result: Developed a family of reversible exponential solvers called Rex that provides rigorous theoretical guarantees and demonstrates utility through empirical illustrations.

Conclusion: Rex offers a theoretically sound and practical solution for diffusion model inversion, addressing limitations of prior heuristic approaches.

Abstract: Diffusion models have quickly become the state-of-the-art for numerous generation tasks across many different applications. Encoding samples from the data distribution back into the models underlying prior distribution is an important task that arises in many downstream applications. This task is often called the inversion of diffusion models. Prior approaches for solving this task, however, are often simple heuristic solvers that come with several drawbacks in practice. In this work, we propose a new family of solvers for diffusion models by exploiting the connection between this task and the broader study of algebraically reversible solvers for differential equations. In particular, we construct a family of reversible solvers using an application of Lawson methods to construct exponential Runge-Kutta methods for the diffusion models. We call this family of reversible exponential solvers Rex. In addition to a rigorous theoretical analysis of the proposed solvers we also emonstrate the utility of the methods through a variety of empirical illustrations.

Method: Leverage Polyak-Łojasiewicz (PL) condition and MDP structure to obtain closed-form gradients, enabling tight sample complexity analysis. Propose a fully first-order, Hessian-free algorithm for efficient hypergradient estimation in large-scale problems.

Result: Achieved O(ε⁻³) sample complexity for BRL in continuous state-action spaces, significantly improving upon existing O(ε⁻⁶) bounds. Also extended results to general bi-level optimization with non-convex lower levels.

Conclusion: This work provides the first theoretical sample complexity guarantees for BRL, establishes state-of-the-art bounds, and offers practical computational improvements through Hessian-free algorithms suitable for large-scale applications.

Abstract: Bilevel reinforcement learning (BRL) has emerged as a powerful framework for aligning generative models, yet its theoretical foundations, especially sample complexity bounds, remain underexplored. In this work, we present the first sample complexity bound for BRL, establishing a rate of $\mathcal{O}(\epsilon^{-3})$ in continuous state-action spaces. Traditional MDP analysis techniques do not extend to BRL due to its nested structure and non-convex lower-level problems. We overcome these challenges by leveraging the Polyak-{\L}ojasiewicz (PL) condition and the MDP structure to obtain closed-form gradients, enabling tight sample complexity analysis. Our analysis also extends to general bi-level optimization settings with non-convex lower levels, where we achieve state-of-the-art sample complexity results of $\mathcal{O}(\epsilon^{-3})$ improving upon existing bounds of $\mathcal{O}(\epsilon^{-6})$. Additionally, we address the computational bottleneck of hypergradient estimation by proposing a fully first-order, Hessian-free algorithm suitable for large-scale problems.

Method: Generalized DCP measure for multiclass classifiers with new local-optimization methods under two regimes: when conditional confusion matrices are known, and when they cannot be estimated due to classifier inaccessibility or data limitations.

Result: Experiments demonstrate the accuracy of the proposed methods in detecting classifiers that likely treat a significant fraction of the population unfairly.

Conclusion: The methods enable effective auditing of multiclass classifiers for fairness violations under equalized odds, with practical applicability even when direct access to classifiers or individual-level data is limited.

Abstract: We propose methods for auditing multiclass classifiers for fairness under multiclass equalized odds,by estimating the deviation from equalized odds when the classifier is not completely fair. We generalize to multiclass classifiers the measure of Disparate Conditional Prediction (DCP), originally suggested by Sabato & Yom-Tov (2020) for binary classifiers. DCP is defined as the fraction of the population for which the classifier predicts with conditional prediction probabilities that differ from the closest common baseline. We provide new local-optimization methods for estimating the multiclass DCPunder two different regimes,one in which the conditional confusion matrices for each protected sub-population are known, and one in which these cannot be estimated, for instance, because the classifier is inaccessible or because good-quality individual-level data is not available. These methods can be used to detect classifiers that likely treat a significant fraction of the population unfairly. Experiments demonstrate the accuracy of the methods. Code is provided at https://github.com/sivansabato/DCPmulticlass.

Method: Uses entropy-driven training framework that automatically anchors step boundaries at tokens with high predictive entropy, capturing logical transitions and enabling efficient exploration of diverse reasoning paths.

Result: Outperforms strong PRM baselines on ProcessBench, achieves comparable results with SOTA models using only 1.5% training data, and boosts accuracy from 64.7% to 67.3% with 32% reduction in token usage through EDU sampling.

Conclusion: EDU-PRM represents a scalable and annotation-efficient paradigm for process supervision in mathematical reasoning, enabling more efficient and robust approaches to complex problem solving.

Abstract: We introduce the Entropy-Driven Uncertainty Process Reward Model (EDU-PRM), a novel entropy-driven training framework for process reward modeling that enables dynamic, uncertainty-aligned segmentation of complex reasoning steps, eliminating the need for costly manual step annotations. Unlike previous Process Reward Models (PRMs) that rely on static partitioning and human labeling, EDU-PRM automatically anchors step boundaries at tokens with high predictive entropy, effectively capturing intrinsic logical transitions and facilitating efficient exploration of diverse reasoning paths. On the ProcessBench benchmark, EDU-PRM outperforms strong public PRM baselines, such as Math-Shepherd PRM and Omega PRM, and EDU-PRM achieves comparable results with SOTA models while only using 1.5% training data. Furthermore, by leveraging our proposed EDU sampling strategy, we observe accuracy boosts from 64.7% to 67.3% for generative reasoning tasks, accompanied by a reduction of 32% in token usage. These findings underscore the potential of EDU-PRM as a scalable and annotation-efficient paradigm for process supervision in mathematical reasoning, paving the way for more efficient and robust approaches to complex mathematical problem solving.

Method: Dynamic data reduction technique that reduces input data during neural network training for image classification.

Result: Achieves up to 50% reduction in runtime while maintaining accuracy, with proportional reduction in carbon emissions.

Conclusion: Dynamic data reduction is an effective approach to reduce computational costs and environmental impact of AI training without sacrificing model accuracy.

Abstract: We propose a novel method for training a neural network for image classification to reduce input data dynamically, in order to reduce the costs of training a neural network model. As Deep Learning tasks become more popular, their computational complexity increases, leading to more intricate algorithms and models which have longer runtimes and require more input data. The result is a greater cost on time, hardware, and environmental resources. By using data reduction techniques, we reduce the amount of work performed, and therefore the environmental impact of AI techniques, and with dynamic data reduction we show that accuracy may be maintained while reducing runtime by up to 50%, and reducing carbon emission proportionally.

Method: Uses stochastic interpolants built by combining data from two prescribed densities with a latent variable, leading to both deterministic and stochastic generative models based on probability flow equations or SDEs with adjustable noise levels. Drift coefficients are characterized as minimizers of quadratic objective functions.

Result: The framework enables exact bridging between any two probability densities in finite time, provides likelihood control for stochastic dynamics, recovers Schrodinger bridges when optimizing over interpolants, and connects with various existing methods like score-based diffusion models and rectifying flows.

Conclusion: Stochastic interpolants offer a flexible and unified approach to generative modeling that subsumes both flow-based and diffusion-based methods, with exact finite-time transitions, adjustable noise levels, and connections to fundamental concepts in stochastic processes and optimal transport.

Abstract: A class of generative models that unifies flow-based and diffusion-based methods is introduced. These models extend the framework proposed in Albergo and Vanden-Eijnden (2023), enabling the use of a broad class of continuous-time stochastic processes called stochastic interpolants to bridge any two probability density functions exactly in finite time. These interpolants are built by combining data from the two prescribed densities with an additional latent variable that shapes the bridge in a flexible way. The time-dependent density function of the interpolant is shown to satisfy a transport equation as well as a family of forward and backward Fokker-Planck equations with tunable diffusion coefficient. Upon consideration of the time evolution of an individual sample, this viewpoint leads to both deterministic and stochastic generative models based on probability flow equations or stochastic differential equations with an adjustable level of noise. The drift coefficients entering these models are time-dependent velocity fields characterized as the unique minimizers of simple quadratic objective functions, one of which is a new objective for the score. We show that minimization of these quadratic objectives leads to control of the likelihood for generative models built upon stochastic dynamics, while likelihood control for deterministic dynamics is more stringent. We also construct estimators for the likelihood and the cross entropy of interpolant-based generative models, and we discuss connections with other methods such as score-based diffusion models, stochastic localization, probabilistic denoising, and rectifying flows. In addition, we demonstrate that stochastic interpolants recover the Schr"odinger bridge between the two target densities when explicitly optimizing over the interpolant. Finally, algorithmic aspects are discussed and the approach is illustrated on numerical examples.

Method: Graph neural network approach that learns to identify smaller sub-problems containing the solution space, using both supervised learning from solved instances and unsupervised learning to minimize the SCP objective.

Result: Reduces problem size by 60-80%, achieves up to 10x runtime speedups compared to Gurobi while maintaining solution quality, and generalizes to larger problem sizes (training on 3,000 subsets, testing on 10,000 subsets).

Conclusion: Graph-SCP effectively combines ML with traditional solvers to significantly accelerate Set Cover Problem solving without sacrificing solution quality, outperforming both greedy approaches and commercial solvers.

Abstract: Machine learning (ML) approaches are increasingly being used to accelerate combinatorial optimization (CO) problems. We investigate the Set Cover Problem (SCP) and propose Graph-SCP, a graph neural network method that augments existing optimization solvers by learning to identify a smaller sub-problem that contains the solution space. Graph-SCP uses both supervised learning from prior solved instances and unsupervised learning to minimize the SCP objective. We evaluate the performance of Graph-SCP on synthetically weighted and unweighted SCP instances with diverse problem characteristics and complexities, and on instances from the OR Library, a canonical benchmark for SCP. We show that Graph-SCP reduces the problem size by 60-80% and achieves runtime speedups of up to 10x on average when compared to Gurobi (a state-of-the-art commercial solver), while maintaining solution quality. This is in contrast to fast greedy solutions that significantly compromise solution quality to achieve guaranteed polynomial runtime. We showcase Graph-SCP’s ability to generalize to larger problem sizes, training on SCP instances with up to 3,000 subsets and testing on SCP instances with up to 10,000 subsets.

Method: Introduced a novel optimization method to localize few-shot ability to specific attention heads, performed dimensionality reduction and decomposition analysis, and derived a mathematical identity relating aggregation and extraction subspaces.

Result: On Llama-3-8B-instruct, reduced the mechanism to just three attention heads with six-dimensional subspaces that track unit digits with trigonometric functions and magnitude with low-frequency components. Identified a self-correction mechanism where later demonstrations suppress mistakes from earlier ones.

Conclusion: Tracking low-dimensional subspaces of localized attention heads across forward passes provides insight into fine-grained computational structures in language models.

Abstract: To perform few-shot learning, language models extract signals from a few input-label pairs, aggregate these into a learned prediction rule, and apply this rule to new inputs. How is this implemented in the forward pass of modern transformer models? To explore this question, we study a structured family of few-shot learning tasks for which the true prediction rule is to add an integer $k$ to the input. We introduce a novel optimization method that localizes the model’s few-shot ability to only a few attention heads. We then perform an in-depth analysis of individual heads, via dimensionality reduction and decomposition. As an example, on Llama-3-8B-instruct, we reduce its mechanism on our tasks to just three attention heads with six-dimensional subspaces, where four dimensions track the unit digit with trigonometric functions at periods $2$, $5$, and $10$, and two dimensions track magnitude with low-frequency components. To deepen our understanding of the mechanism, we also derive a mathematical identity relating aggregation'' and extraction’’ subspaces for attention heads, allowing us to track the flow of information from individual examples to a final aggregated concept. Using this, we identify a self-correction mechanism where mistakes learned from earlier demonstrations are suppressed by later demonstrations. Our results demonstrate how tracking low-dimensional subspaces of localized heads across a forward pass can provide insight into fine-grained computational structures in language models.

Method: Proposed Poisson Midpoint Method, which approximates small step-size LMC with large step-sizes. This is a variant of Randomized Midpoint Method adapted for diffusion models with non-log concave densities and time-varying drift.

Result: Achieves quadratic speedup over standard LMC under weak assumptions. When applied to diffusion models for image generation, it maintains DDPM quality with 1000 neural network calls using only 50-80 calls, outperforming ODE-based methods with similar compute.

Conclusion: Poisson Midpoint Method provides an efficient discretization for Langevin dynamics in diffusion models, enabling significant computational savings while preserving sample quality, making it superior to both standard LMC and ODE-based approaches.

Abstract: Langevin Dynamics is a Stochastic Differential Equation (SDE) central to sampling and generative modeling and is implemented via time discretization. Langevin Monte Carlo (LMC), based on the Euler-Maruyama discretization, is the simplest and most studied algorithm. LMC can suffer from slow convergence - requiring a large number of steps of small step-size to obtain good quality samples. This becomes stark in the case of diffusion models where a large number of steps gives the best samples, but the quality degrades rapidly with smaller number of steps. Randomized Midpoint Method has been recently proposed as a better discretization of Langevin dynamics for sampling from strongly log-concave distributions. However, important applications such as diffusion models involve non-log concave densities and contain time varying drift. We propose its variant, the Poisson Midpoint Method, which approximates a small step-size LMC with large step-sizes. We prove that this can obtain a quadratic speed up of LMC under very weak assumptions. We apply our method to diffusion models for image generation and show that it maintains the quality of DDPM with 1000 neural network calls with just 50-80 neural network calls and outperforms ODE based methods with similar compute.

Method: Leverages Shapley value attribution to identify fairness-critical instances using interpretable feature importance measures, then systematically modifies them through instance-level matching across sensitive groups to reduce discriminative risk.

Result: Significantly improves demographic parity and equality of opportunity across diverse tabular datasets, achieving fairness gains with minimal data perturbation and sometimes improved predictive performance.

Conclusion: FairSHAP is a model-agnostic, transparent method that integrates seamlessly into existing ML pipelines and provides actionable insights into bias sources while preserving data integrity and model accuracy.

Abstract: Ensuring fairness in machine learning models is critical, particularly in high-stakes domains where biased decisions can lead to serious societal consequences. Existing preprocessing approaches generally lack transparent mechanisms for identifying which features or instances are responsible for unfairness. This obscures the rationale behind data modifications. We introduce FairSHAP, a novel pre-processing framework that leverages Shapley value attribution to improve both individual and group fairness. FairSHAP identifies fairness-critical instances in the training data using an interpretable measure of feature importance, and systematically modifies them through instance-level matching across sensitive groups. This process reduces discriminative risk - an individual fairness metric - while preserving data integrity and model accuracy. We demonstrate that FairSHAP significantly improves demographic parity and equality of opportunity across diverse tabular datasets, achieving fairness gains with minimal data perturbation and, in some cases, improved predictive performance. As a model-agnostic and transparent method, FairSHAP integrates seamlessly into existing machine learning pipelines and provides actionable insights into the sources of bias.Our code is on https://github.com/youlei202/FairSHAP.

Method: Uses generalized regression with uncorrelated constraint to select discriminative yet irrelevant features. Integrates instance and label correlation to adaptively learn sample and label similarity graphs that mutually enhance feature selection.

Result: Extensive experiments show Access-MFS outperforms state-of-the-art methods.

Conclusion: Access-MFS effectively addresses limitations of existing methods by adaptively learning correlations and selecting uncorrelated features, demonstrating superior performance.

Abstract: Semi-supervised multi-label feature selection has recently been developed to solve the curse of dimensionality problem in high-dimensional multi-label data with certain samples missing labels. Although many efforts have been made, most existing methods use a predefined graph approach to capture the sample similarity or the label correlation. In this manner, the presence of noise and outliers within the original feature space can undermine the reliability of the resulting sample similarity graph. It also fails to precisely depict the label correlation due to the existence of unknown labels. Besides, these methods only consider the discriminative power of selected features, while neglecting their redundancy. In this paper, we propose an Adaptive Collaborative Correlation lEarning-based Semi-Supervised Multi-label Feature Selection (Access-MFS) method to address these issues. Specifically, a generalized regression model equipped with an extended uncorrelated constraint is introduced to select discriminative yet irrelevant features and maintain consistency between predicted and ground-truth labels in labeled data, simultaneously. Then, the instance correlation and label correlation are integrated into the proposed regression model to adaptively learn both the sample similarity graph and the label similarity graph, which mutually enhance feature selection performance. Extensive experimental results demonstrate the superiority of the proposed Access-MFS over other state-of-the-art methods.

Method: Proposes LLINBO framework with three mechanisms: using LLMs for early exploration based on contextual knowledge, transitioning to statistical surrogate experts (GPs) for exploitation, and establishing theoretical guarantees for the collaboration.

Result: The framework demonstrates practical application in 3D printing optimization and provides theoretical guarantees for the hybrid approach.

Conclusion: LLINBO successfully combines the strengths of LLMs and statistical models for Bayesian optimization, offering a theoretically grounded and reliable approach for expensive black-box function optimization.

Abstract: Bayesian optimization (BO) is a sequential decision-making tool widely used for optimizing expensive black-box functions. Recently, Large Language Models (LLMs) have shown remarkable adaptability in low-data regimes, making them promising tools for black-box optimization by leveraging contextual knowledge to propose high-quality query points. However, relying solely on LLMs as optimization agents introduces risks due to their lack of explicit surrogate modeling and calibrated uncertainty, as well as their inherently opaque internal mechanisms. This structural opacity makes it difficult to characterize or control the exploration-exploitation trade-off, ultimately undermining theoretical tractability and reliability. To address this, we propose LLINBO: LLM-in-the-Loop BO, a hybrid framework for BO that combines LLMs with statistical surrogate experts (e.g., Gaussian Processes (GP)). The core philosophy is to leverage contextual reasoning strengths of LLMs for early exploration, while relying on principled statistical models to guide efficient exploitation. Specifically, we introduce three mechanisms that enable this collaboration and establish their theoretical guarantees. We end the paper with a real-life proof-of-concept in the context of 3D printing. The code to reproduce the results can be found at https://github.com/UMDataScienceLab/LLM-in-the-Loop-BO.

Method: Proposed Pretrain-DPFL framework evaluating three fine-tuning strategies: full-tuning (FT), head-tuning (HT), and unified-tuning (UT) combining HT followed by FT, with convergence analysis under smooth non-convex loss.

Result: Extensive experiments show Pretrain-DPFL achieves 25.22% higher accuracy than scratch training and outperforms second-best baseline by 8.19%, significantly improving privacy-utility trade-off.

Conclusion: Pretrain-DPFL provides systematic framework to maximize benefits of pre-trained models in DPFL, establishing theoretical conditions for optimal fine-tuning strategy selection.

Abstract: Differentially Private Federated Learning (DPFL) strengthens privacy protection by perturbing model gradients with noise, though at the cost of reduced accuracy. Although prior empirical studies indicate that initializing from pre-trained rather than random parameters can alleviate noise disturbance, the problem of optimally fine-tuning pre-trained models in DPFL remains unaddressed. In this paper, we propose Pretrain-DPFL, a framework that systematically evaluates three most representative fine-tuning strategies: full-tuning (FT), head-tuning (HT), and unified-tuning(UT) combining HT followed by FT. Through convergence analysis under smooth non-convex loss, we establish theoretical conditions for identifying the optimal fine-tuning strategy in Pretrain-DPFL, thereby maximizing the benefits of pre-trained models in mitigating noise disturbance. Extensive experiments across multiple datasets demonstrate Pretrain-DPFL’s superiority, achieving $25.22%$ higher accuracy than scratch training and outperforming the second-best baseline by $8.19%$, significantly improving the privacy-utility trade-off in DPFL.

Method: FAB attack uses meta-learning to simulate downstream finetuning, optimizing for adversarial behavior emergence while regularizing the model to remain benign before finetuning and maintain general capabilities.

Result: FAB successfully triggers adversarial behaviors across multiple LLMs for unsolicited advertising, jailbreakability, and over-refusal, with triggers robust to various finetuning choices.

Conclusion: Finetuning is not as secure as previously assumed, revealing a critical attack vector where compromised models can be triggered by downstream users’ finetuning processes.

Abstract: Finetuning open-weight Large Language Models (LLMs) is standard practice for achieving task-specific performance improvements. Until now, finetuning has been regarded as a controlled and secure process in which training on benign datasets leads to predictable behaviors. In this paper, we demonstrate, for the first time, that an adversary can create compromised LLMs that are performant and benign, yet exhibit adversarial behaviors once finetuned by downstream users. To this end, we propose an attack, FAB (Finetuning-activated Adversarial Behaviors), which compromises an LLM via meta-learning techniques that simulate downstream finetuning, explicitly optimizing for the emergence of adversarial behaviors in the finetuned models. At the same time, the compromised LLM is regularized to retain general capabilities and to exhibit no adversarial behaviors prior to finetuning. As a result, when users finetune (e.g., instruction-tuning, distillation, DPO) the seemingly benign model on their own datasets, they unknowingly trigger its dormant adversarial behavior. We experimentally demonstrate the effectiveness of FAB across multiple LLMs and three commonly considered target behaviors: unsolicited advertising, jailbreakability, and over-refusal. We show that FAB-triggers are robust to various finetuning choices made by the user (e.g., dataset, number of steps, scheduler, post-training algorithm). Our findings challenge prevailing assumptions on the security of finetuning, revealing a critical attack vector.

Method: Introduces UKANs with a coefficient-generator model that produces B-spline coefficients locally on unbounded symmetric grids, and couples MLPs with KANs using positional encoding. Also develops a GPU-accelerated library (warpKAN) for efficient computation.

Result: UKANs achieve 3-30x speed-up and up to 1000x memory reduction compared to vanilla KANs. They match or surpass KAN accuracy on regression, classification, and generative tasks, and enable large-scale molecular property prediction.

Conclusion: UKANs successfully overcome the bounded domain limitation of KANs while maintaining or improving performance, with optimized GPU acceleration making large-scale training feasible.

Abstract: Kolmogorov-Arnold Networks (KANs) have recently emerged as a powerful alternative to traditional multilayer perceptrons. However, their reliance on predefined, bounded grids restricts their ability to approximate functions on unbounded domains. To address this, we present Unbounded Kolmogorov-Arnold Networks (UKANs), a method that removes the need for bounded grids in traditional Kolmogorov-Arnold Networks (KANs). The key innovation of this method is a coefficient-generator (CG) model that produces, on the fly, only the B-spline coefficients required locally on an unbounded symmetric grid. UKANs couple multilayer perceptrons with KANs by feeding the positional encoding of grid groups into the CG model, enabling function approximation on unbounded domains without requiring data normalization. To reduce the computational cost of both UKANs and KANs, we introduce a GPU-accelerated library that lowers B-spline evaluation complexity by a factor proportional to the grid size, enabling large-scale learning by leveraging efficient memory management, in line with recent software advances such as FlashAttention and FlashFFTConv. Performance benchmarking confirms the superior memory and computational efficiency of our accelerated KAN (warpKAN), and UKANs, showing a 3-30x speed-up and up to 1000x memory reduction compared to vanilla KANs. Experiments on regression, classification, and generative tasks demonstrate the effectiveness of UKANs to match or surpass KAN accuracy. Finally, we use both accelerated KAN and UKAN in a molecular property prediction task, establishing the feasibility of large-scale end-to-end training with our optimized implementation.

Method: Uses persistent homology (PH) from topological data analysis to characterize multi-scale dynamics in LLM activations. Analyzes six state-of-the-art models under two adversarial conditions: indirect prompt injection and backdoor fine-tuning.

Result: Identifies a consistent “topological compression” signature where adversarial inputs make latent spaces structurally simpler - collapsing from varied small-scale features into fewer dominant large-scale ones. This signature is statistically robust across layers and highly discriminative.

Conclusion: The architecture-agnostic framework reveals fundamental invariants of representational change and provides interpretable insights into adversarial effect emergence and propagation, offering a complementary perspective to existing interpretability methods.

Abstract: Existing interpretability methods for Large Language Models (LLMs) often fall short by focusing on linear directions or isolated features, overlooking the high-dimensional, nonlinear, and relational geometry within model representations. This study focuses on how adversarial inputs systematically affect the internal representation spaces of LLMs, a topic which remains poorly understood. We propose persistent homology (PH), a tool from topological data analysis, as a principled framework to characterize the multi-scale dynamics within LLM activations. Using PH, we systematically analyze six state-of-the-art models under two distinct adversarial conditions, indirect prompt injection and backdoor fine-tuning, and identify a consistent topological signature of adversarial influence. Across architectures and model sizes, adversarial inputs induce ``topological compression’’, where the latent space becomes structurally simpler, collapsing from varied, compact, small-scale features into fewer, dominant, and more dispersed large-scale ones. This topological signature is statistically robust across layers, highly discriminative, and provides interpretable insights into how adversarial effects emerge and propagate. By quantifying the shape of activations and neuronal information flow, our architecture-agnostic framework reveals fundamental invariants of representational change, offering a complementary perspective to existing interpretability methods.

Method: Attackers recover the dependence between outputs and sensitive attributes through local fine-tuning across sensitive groups, then inject the biased model via model replacement in FL.

Result: PFAttack successfully bypasses group fairness mechanisms in four fair FL frameworks and remains undetected by three Byzantine-resilient aggregation methods while maintaining accuracy.

Conclusion: PFAttack demonstrates that FL systems with fairness mechanisms are vulnerable to stealthy attacks that can introduce bias while preserving model accuracy, highlighting a critical security gap.

Abstract: Federated learning (FL), integrating group fairness mechanisms, allows multiple clients to collaboratively train a global model that makes unbiased decisions for different populations grouped by sensitive attributes (e.g., gender and race). Due to its distributed nature, previous studies have demonstrated that FL systems are vulnerable to model poisoning attacks. However, these studies primarily focus on perturbing accuracy, leaving a critical question unexplored: Can an attacker bypass the group fairness mechanisms in FL and manipulate the global model to be biased? The motivations for such an attack vary; an attacker might seek higher accuracy, yet fairness considerations typically limit the accuracy of the global model or aim to cause ethical disruption. To address this question, we design a novel form of attack in FL, termed Profit-driven Fairness Attack (PFAttack), which aims not to degrade global model accuracy but to bypass fairness mechanisms. Our fundamental insight is that group fairness seeks to weaken the dependence of outputs on input attributes related to sensitive information. In the proposed PFAttack, an attacker can recover this dependence through local fine-tuning across various sensitive groups, thereby creating a biased yet accuracy-preserving malicious model and injecting it into FL through model replacement. Compared to attacks targeting accuracy, PFAttack is more stealthy. The malicious model in PFAttack exhibits subtle parameter variations relative to the original global model, making it robust against detection and filtering by Byzantine-resilient aggregations. Extensive experiments on benchmark datasets are conducted for four fair FL frameworks and three Byzantine-resilient aggregations against model poisoning, demonstrating the effectiveness and stealth of PFAttack in bypassing group fairness mechanisms in FL.

Method: MBPO constructs offline preference datasets with hard negatives generated through adversarial image perturbation, and leverages close-ended tasks to generate online responses with verified rewards. Uses Group Relative Policy Optimization with hybrid offline-online data.

Result: Extensive experiments show MBPO enhances LMM performance on challenging vision-language tasks and effectively reduces hallucinations.

Conclusion: MBPO successfully addresses modality imbalance in LMMs through a hybrid offline-online preference optimization approach that balances language and visual modalities, improving performance and reducing hallucinations.

Abstract: The task adaptation and alignment of Large Multimodal Models (LMMs) have been significantly advanced by instruction tuning and further strengthened by recent preference optimization. Yet, most LMMs still suffer from severe modality imbalance during reasoning, i.e., outweighing language prior biases over visual inputs, which bottlenecks their generalization to downstream tasks and causes hallucinations. However, existing preference optimization approaches for LMMs do not focus on restraining the internal biases of their Large Language Model (LLM) backbones when curating the training data. Moreover, they heavily rely on offline data and lack the capacity to explore diverse responses adaptive to dynamic distributional shifts during training. Meanwhile, Group Relative Policy Optimization (GRPO), a recent method using online-generated data and verified rewards to improve reasoning capabilities, remains largely underexplored in LMM alignment. In this paper, we propose a novel preference learning framework, Modality-Balancing Preference Optimization (MBPO), to address the modality imbalance in LMMs. MBPO constructs a more effective offline preference dataset by generating hard negatives, i.e., rejected responses misled by LLM biases due to limited usage of visual information, through adversarial perturbation of input images. Moreover, MBPO leverages the easy-to-verify nature of close-ended tasks to generate online responses with verified rewards. GRPO is then employed to train the model with offline-online hybrid data. Extensive experiments demonstrate that MBPO can enhance LMM performance on challenging vision-language tasks and effectively reduce hallucinations.

Method: Proposes FedAMoLE with heterogeneous mixture of LoRA experts module to aggregate architecturally heterogeneous models and reverse selection-based expert assignment strategy to tailor architectures based on data distributions.

Result: Experiments across seven scenarios show FedAMoLE improves client-side performance by average 5.97% over existing approaches while maintaining practical memory, communication, and computation overhead.

Conclusion: FedAMoLE effectively addresses architectural heterogeneity in federated LLM fine-tuning, enabling better performance across diverse domains with efficient resource usage.

Abstract: Large language models (LLMs) are increasingly powering web-based applications, whose effectiveness relies on fine-tuning with large-scale instruction data. However, such data often contains valuable or sensitive information that limits its public sharing among business organizations. Federated learning (FL) enables collaborative fine-tuning of LLMs without accessing raw data. Existing approaches to federated LLM fine-tuning usually adopt a uniform model architecture, making it challenging to fit highly heterogeneous client-side data in varying domains and tasks, e.g., hospitals and financial institutions conducting federated fine-tuning may require different LLM architectures due to the distinct nature of their domains and tasks. To address this, we propose FedAMoLE, a lightweight personalized FL framework that enables data-driven heterogeneous model architectures. It features a heterogeneous mixture of low-rank adaptation (LoRA) experts module to aggregate architecturally heterogeneous models and a reverse selection-based expert assignment strategy to tailor model architectures for each client based on data distributions. Experiments across seven scenarios demonstrate that FedAMoLE improves client-side performance by an average of 5.97% over existing approaches while maintaining practical memory, communication, and computation overhead.

Method: Uses flow matching to build probabilistic models predicting distant future state distributions (occupancy measures), incorporating latent user intention variables to increase expressivity and enable adaptation via generalized policy improvement.

Result: Achieves 1.8x median improvement in returns and 36% higher success rates across 36 state-based and 4 image-based benchmark tasks compared to alternative pre-training methods.

Conclusion: InFOM demonstrates that intention-conditioned flow occupancy models provide an effective framework for RL pre-training, successfully addressing long-term dependencies and enabling adaptation to diverse tasks.

Abstract: Large-scale pre-training has fundamentally changed how machine learning research is done today: large foundation models are trained once, and then can be used by anyone in the community (including those without data or compute resources to train a model from scratch) to adapt and fine-tune to specific tasks. Applying this same framework to reinforcement learning (RL) is appealing because it offers compelling avenues for addressing core challenges in RL, including sample efficiency and robustness. However, there remains a fundamental challenge to pre-train large models in the context of RL: actions have long-term dependencies, so training a foundation model that reasons across time is important. Recent advances in generative AI have provided new tools for modeling highly complex distributions. In this paper, we build a probabilistic model to predict which states an agent will visit in the temporally distant future (i.e., an occupancy measure) using flow matching. As large datasets are often constructed by many distinct users performing distinct tasks, we include in our model a latent variable capturing the user intention. This intention increases the expressivity of our model, and enables adaptation with generalized policy improvement. We call our proposed method intention-conditioned flow occupancy models (InFOM). Comparing with alternative methods for pre-training, our experiments on $36$ state-based and $4$ image-based benchmark tasks demonstrate that the proposed method achieves $1.8 \times$ median improvement in returns and increases success rates by $36%$. Website: https://chongyi-zheng.github.io/infom Code: https://github.com/chongyi-zheng/infom

Method: Extensive evaluation of many normalizing flow architectures on various flow-based MCMC methods and target distributions, testing performance with and without target density gradients available.

Result: Flow-based MCMC outperforms classic MCMC when suitable NF architectures are chosen with minor hyperparameter tuning. Contractive residual flows are identified as the best general-purpose models with low sensitivity to hyperparameters.

Conclusion: The study provides practical guidelines for normalizing flow architecture selection in MCMC, with contractive residual flows recommended as general-purpose models, and offers insights into NF behavior across different hyperparameters, target distributions, and computational budgets.

Abstract: Recent advances in MCMC use normalizing flows to precondition target distributions and enable jumps to distant regions. However, there is currently no systematic comparison of different normalizing flow architectures for MCMC. As such, many works choose simple flow architectures that are readily available and do not consider other models. Guidelines for choosing an appropriate architecture would reduce analysis time for practitioners and motivate researchers to take the recommended models as foundations to be improved. We provide the first such guideline by extensively evaluating many normalizing flow architectures on various flow-based MCMC methods and target distributions. When the target density gradient is available, we show that flow-based MCMC outperforms classic MCMC for suitable NF architecture choices with minor hyperparameter tuning. When the gradient is unavailable, flow-based MCMC wins with off-the-shelf architectures. We find contractive residual flows to be the best general-purpose models with relatively low sensitivity to hyperparameter choice. We also provide various insights into normalizing flow behavior within MCMC when varying their hyperparameters, properties of target distributions, and the overall computational budget.

Method: Analyzed gradient equivalence between LV and rKL losses, then conducted experiments with different diffusion bridge types on challenging benchmarks using rKL-LD loss with log-derivative trick.

Result: Samplers trained with rKL-LD loss consistently outperformed those using LV loss, showed more stable training behavior, and required significantly less hyperparameter optimization.

Conclusion: rKL-LD loss is superior to LV loss for diffusion bridges, avoiding conceptual problems while providing better performance and training stability with reduced hyperparameter tuning requirements.

Abstract: Diffusion bridges are a promising class of deep-learning methods for sampling from unnormalized distributions. Recent works show that the Log Variance (LV) loss consistently outperforms the reverse Kullback-Leibler (rKL) loss when using the reparametrization trick to compute rKL-gradients. While the on-policy LV loss yields identical gradients to the rKL loss when combined with the log-derivative trick for diffusion samplers with non-learnable forward processes, this equivalence does not hold for diffusion bridges or when diffusion coefficients are learned. Based on this insight we argue that for diffusion bridges the LV loss does not represent an optimization objective that can be motivated like the rKL loss via the data processing inequality. Our analysis shows that employing the rKL loss with the log-derivative trick (rKL-LD) does not only avoid these conceptual problems but also consistently outperforms the LV loss. Experimental results with different types of diffusion bridges on challenging benchmarks show that samplers trained with the rKL-LD loss achieve better performance. From a practical perspective we find that rKL-LD requires significantly less hyperparameter optimization and yields more stable training behavior.

Method: Uses Gaussian Process with condensed graph as observations, derives specialized covariance function with structural information, employs Concrete random variables to approximate binary adjacency matrix for gradient-based optimization.

Result: Experimental results show GCGP efficiently condenses large-scale graph data while preserving predictive performance, addressing scalability and efficiency challenges.

Conclusion: GCGP provides an effective and computationally efficient alternative to traditional graph condensation methods, enabling practical large-scale graph compression without performance degradation.

Abstract: Graph condensation reduces the size of large graphs while preserving performance, addressing the scalability challenges of Graph Neural Networks caused by computational inefficiencies on large datasets. Existing methods often rely on bi-level optimization, requiring extensive GNN training and limiting their scalability. To address these issues, this paper proposes Graph Condensation via Gaussian Process (GCGP), a novel and computationally efficient approach to graph condensation. GCGP utilizes a Gaussian Process (GP), with the condensed graph serving as observations, to estimate the posterior distribution of predictions. This approach eliminates the need for the iterative and resource-intensive training typically required by GNNs. To enhance the capability of the GCGP in capturing dependencies between function values, we derive a specialized covariance function that incorporates structural information. This covariance function broadens the receptive field of input nodes by local neighborhood aggregation, thereby facilitating the representation of intricate dependencies within the nodes. To address the challenge of optimizing binary structural information in condensed graphs, Concrete random variables are utilized to approximate the binary adjacency matrix in a continuous counterpart. This relaxation process allows the adjacency matrix to be represented in a differentiable form, enabling the application of gradient-based optimization techniques to discrete graph structures. Experimental results show that the proposed GCGP method efficiently condenses large-scale graph data while preserving predictive performance, addressing the scalability and efficiency challenges. The implementation of our method is publicly available at https://github.com/WANGLin0126/GCGP.

Method: FedMosaic uses task-relevance-aware model aggregation to reduce parameter interference and a dimension-invariant module to enable knowledge sharing across heterogeneous architectures without high computational cost.

Result: FedMosaic outperforms state-of-the-art PFL methods on a multi-modal benchmark with 40 distinct tasks, demonstrating superior personalization and generalization capabilities in challenging realistic scenarios.

Conclusion: FedMosaic successfully addresses data and model heterogeneity in PFL, enabling effective knowledge sharing across diverse tasks and architectures while maintaining privacy, making it suitable for real-world Agentic AI applications.

Abstract: As AI becomes more personal, e.g., Agentic AI, there is an increasing need for personalizing models for various use cases. Personalized federated learning (PFL) enables each client to collaboratively leverage other clients’ knowledge for better adaptation to the task of interest, without privacy risks. Despite its potential, existing PFL methods remain confined to rather simplified scenarios where data and models are the same across clients. To move towards realistic scenarios, we propose FedMosaic, a method that jointly addresses data and model heterogeneity with a task-relevance-aware model aggregation strategy to reduce parameter interference, and a dimension-invariant module that enables knowledge sharing across heterogeneous architectures without huge computational cost. To mimic the real-world task diversity, we propose a multi-modal PFL benchmark spanning 40 distinct tasks with distribution shifts over time. The empirical study shows that FedMosaic outperforms the state-of-the-art PFL methods, excelling in both personalization and generalization capabilities under challenging, realistic scenarios.

Method: Proposed CaDRe (Causal Discovery and Representation learning) - a time-series generative model with structural constraints that integrates CRL and causal discovery, establishing conditions for simultaneous identifiability of hidden dynamic processes and causal structure among observed variables.

Result: Experiments validate theoretical results on synthetic datasets. On real-world climate datasets, CaDRe achieves competitive forecasting accuracy and recovers visualized causal graphs aligned with domain expertise.

Conclusion: The framework offers interpretable insights into climate systems by jointly uncovering causal relations among observed variables and latent driving forces with their interactions.

Abstract: Understanding climate dynamics requires going beyond correlations in observational data to uncover their underlying causal process. Latent drivers, such as atmospheric processes, play a critical role in temporal dynamics, while direct causal influences also exist among geographically proximate observed variables. Traditional Causal Representation Learning (CRL) typically focuses on latent factors but overlooks such observable-to-observable causal relations, limiting its applicability to climate analysis. In this paper, we introduce a unified framework that jointly uncovers (i) causal relations among observed variables and (ii) latent driving forces together with their interactions. We establish conditions under which both the hidden dynamic processes and the causal structure among observed variables are simultaneously identifiable from time-series data. Remarkably, our guarantees hold even in the nonparametric setting, leveraging contextual information to recover latent variables and causal relations. Building on these insights, we propose CaDRe (Causal Discovery and Representation learning), a time-series generative model with structural constraints that integrates CRL and causal discovery. Experiments on synthetic datasets validate our theoretical results. On real-world climate datasets, CaDRe not only delivers competitive forecasting accuracy but also recovers visualized causal graphs aligned with domain expertise, thereby offering interpretable insights into climate systems.

Method: Comprehensive review and organization of existing methodologies around four shared performance objectives, with analysis of representative works, techniques, and evaluation metrics for each objective.

Result: Created an objective-centric framework that enables meaningful cross-domain comparisons and offers practical insights for selecting and designing foundation model solutions for new IoT tasks.

Conclusion: The survey provides key future research directions to guide practitioners and researchers in advancing foundation model applications in IoT, addressing current limitations in cross-domain comparability and practical guidance.

Abstract: Foundation models have gained growing interest in the IoT domain due to their reduced reliance on labeled data and strong generalizability across tasks, which address key limitations of traditional machine learning approaches. However, most existing foundation model based methods are developed for specific IoT tasks, making it difficult to compare approaches across IoT domains and limiting guidance for applying them to new tasks. This survey aims to bridge this gap by providing a comprehensive overview of current methodologies and organizing them around four shared performance objectives by different domains: efficiency, context-awareness, safety, and security & privacy. For each objective, we review representative works, summarize commonly-used techniques and evaluation metrics. This objective-centric organization enables meaningful cross-domain comparisons and offers practical insights for selecting and designing foundation model based solutions for new IoT tasks. We conclude with key directions for future research to guide both practitioners and researchers in advancing the use of foundation models in IoT applications.

Method: Proposes FedGKC with lightweight Copilot Models on clients, Client-side Self-Mutual Knowledge Distillation with bidirectional distillation and multi-view perturbation, and Server-side Knowledge-Aware Model Aggregation with dynamic weight assignment.

Result: Achieves average 3.74% accuracy gain over baselines on eight benchmark datasets in heterogeneous settings while maintaining performance in homogeneous scenarios.

Conclusion: FedGKC effectively handles model-centric heterogeneity in federated graph learning through knowledge collaboration mechanisms, demonstrating significant performance improvements.

Abstract: Federated graph learning (FGL) has emerged as a promising paradigm for collaborative graph representation learning, enabling multiple parties to jointly train models while preserving data privacy. However, most existing approaches assume homogeneous client models and largely overlook the challenge of model-centric heterogeneous FGL (MHtFGL), which frequently arises in practice when organizations employ graph neural networks (GNNs) of different scales and architectures.Such architectural diversity not only undermines smooth server-side aggregation, which presupposes a unified representation space shared across clients’ updates, but also further complicates the transfer and integration of structural knowledge across clients. To address this issue, we propose the Federated Graph Knowledge Collaboration (FedGKC) framework. FedGKC introduces a lightweight Copilot Model on each client to facilitate knowledge exchange while local architectures are heterogeneous across clients, and employs two complementary mechanisms: Client-side Self-Mutual Knowledge Distillation, which transfers effective knowledge between local and copilot models through bidirectional distillation with multi-view perturbation; and Server-side Knowledge-Aware Model Aggregation, which dynamically assigns aggregation weights based on knowledge provided by clients. Extensive experiments on eight benchmark datasets demonstrate that FedGKC achieves an average accuracy gain of 3.74% over baselines in MHtFGL scenarios, while maintaining excellent performance in homogeneous settings.

Method: HOLA framework uses: 1) Hierarchical Speculative Decoding (HSD) for faster inference without quality loss, 2) AdaComp-RAG for adaptive retrieval complexity based on context needs, and 3) LoBi that blends structured pruning (LoRA) and quantization.

Result: Achieved 17.6% EMA on GSM8K, 10.5% MCA on ARC, with reduced latency and memory usage on edge devices like Jetson Nano, proving both scalable and production-ready.

Conclusion: HOLA provides a comprehensive solution for efficient LLM deployment on edge devices, overcoming limitations of existing partial optimization approaches while maintaining performance and scalability.

Abstract: Running Large Language Models (LLMs) on edge devices is constrained by high compute and memory demands posing a barrier for real-time applications in sectors like healthcare, education, and embedded systems. Current solutions such as quantization, pruning, and retrieval-augmented generation (RAG) offer only partial optimizations and often compromise on speed or accuracy. We introduce HOLA, an end-to-end optimization framework for efficient LLM deployment. Internally, it leverages Hierarchical Speculative Decoding (HSD) for faster inference without quality loss. Externally, AdaComp-RAG adjusts retrieval complexity based on context needs. Together with LoBi, which blends structured pruning (LoRA) and quantization, HOLA delivers significant gains: 17.6% EMA on GSM8K, 10.5% MCA on ARC, and reduced latency and memory on edge devices like Jetson Nano–proving both scalable and production-ready.

Method: Proposes a GNN-based framework with multi-scale graph structure, adaptive messaging mechanism using dynamic gating units, and neural nested grid method for regional forecasts to prevent boundary information loss.

Result: OneForecast performs excellently across global to regional scales and short-term to long-term forecasts, especially in extreme event predictions.

Conclusion: The framework successfully addresses key challenges in weather forecasting through its global-regional nested approach and adaptive mechanisms.

Abstract: Accurate weather forecasts are important for disaster prevention, agricultural planning, etc. Traditional numerical weather prediction (NWP) methods offer physically interpretable high-accuracy predictions but are computationally expensive and fail to fully leverage rapidly growing historical data. In recent years, deep learning models have made significant progress in weather forecasting, but challenges remain, such as balancing global and regional high-resolution forecasts, excessive smoothing in extreme event predictions, and insufficient dynamic system modeling. To address these issues, this paper proposes a global-regional nested weather forecasting framework (OneForecast) based on graph neural networks. By combining a dynamic system perspective with multi-grid theory, we construct a multi-scale graph structure and densify the target region to capture local high-frequency features. We introduce an adaptive messaging mechanism, using dynamic gating units to deeply integrate node and edge features for more accurate extreme event forecasting. For high-resolution regional forecasts, we propose a neural nested grid method to mitigate boundary information loss. Experimental results show that OneForecast performs excellently across global to regional scales and short-term to long-term forecasts, especially in extreme event predictions. Codes link https://github.com/YuanGao-YG/OneForecast.

Method: Compress T task vectors into M < T basis vectors using structured linear combinations, supporting standard and advanced arithmetic operations while being orthogonal to other efficiency improvements.

Result: Outperforms heuristic baselines and sometimes surpasses full task vector collections across diverse applications while reducing storage and computational requirements.

Conclusion: The framework provides theoretical guarantees for addition generalization and principled unlearning, with empirical results showing superior performance and efficiency compared to existing approaches.

Abstract: Task arithmetic, representing downstream tasks through linear operations on task vectors, has emerged as a simple yet powerful paradigm for transferring knowledge across diverse settings. However, maintaining a large collection of task vectors introduces scalability challenges in both storage and computation. We propose Task Vector Bases, a framework compressing $T$ task vectors into $M < T$ basis vectors while preserving the functionality of task arithmetic. By representing each task vector as a structured linear combination of basis atoms, our approach supports standard operations such as addition, negation, as well as more advanced arithmetic ones. The framework is orthogonal to other efficiency-oriented improvements in task arithmetic and can be used in combination with them. We provide theoretical analysis showing that basis compression retains addition generalization guarantees and enables principled unlearning, with error bounds depending on reconstruction quality. Empirically, our proposed basis construction methods consistently outperform heuristic basis construction baselines and, in some cases, even surpass the performance of full task vector collections across diverse downstream applications while reducing storage and computational requirements. The code is available at https://github.com/uiuctml/TaskVectorBasis.

Method: Reformulate online restless bandits as budgeted thresholding contextual bandits, encoding long-term state transitions into scalar rewards. Propose a practical learning policy for heterogeneous-agent, multi-state settings.

Result: Achieves sublinear regret with faster convergence than existing methods, leading to higher cumulative rewards. Empirical validation shows significant gains over state-of-the-art algorithms in large-scale heterogeneous environments.

Conclusion: Provides a new pathway for practical, sample-efficient learning in finite-horizon restless bandits through simplified problem formulation and improved convergence properties.

Abstract: This paper addresses the poor finite-horizon performance of existing online \emph{restless bandit} (RB) algorithms, which stems from the prohibitive sample complexity of learning a full \emph{Markov decision process} (MDP) for each agent. We argue that superior finite-horizon performance requires \emph{rapid convergence} to a \emph{high-quality} policy. Thus motivated, we introduce a reformulation of online RBs as a \emph{budgeted thresholding contextual bandit}, which simplifies the learning problem by encoding long-term state transitions into a scalar reward. We prove the first non-asymptotic optimality of an oracle policy for a simplified finite-horizon setting. We propose a practical learning policy under a heterogeneous-agent, multi-state setting, and show that it achieves a sublinear regret, achieving \emph{faster convergence} than existing methods. This directly translates to higher cumulative reward, as empirically validated by significant gains over state-of-the-art algorithms in large-scale heterogeneous environments. Our work provides a new pathway for achieving practical, sample-efficient learning in finite-horizon RBs.

Method: Proposes HPGNN with efficient high-order approximation of Personalized PageRank to capture long-range and multi-scale interactions, embedding higher-order structural information into convolutional networks while reducing computational complexity.

Result: HPGNN outperforms 5 out of 7 state-of-the-art methods on heterophilic graphs in downstream tasks and maintains competitive performance on homophilic graphs, demonstrating better noise robustness and multi-scale information balance.

Conclusion: HPGNN provides a versatile solution for real-world graph learning by effectively modeling key interactions across graph dimensions through higher-order structural integration, achieving strong performance on both heterophilic and homophilic graphs.

Abstract: Graph Neural Networks (GNNs) excel in node classification tasks but often assume homophily, where connected nodes share similar labels. This assumption does not hold in many real-world heterophilic graphs. Existing models for heterophilic graphs primarily rely on pairwise relationships, overlooking multi-scale information from higher-order structures. This leads to suboptimal performance, particularly under noise from conflicting class information across nodes. To address these challenges, we propose HPGNN, a novel model integrating Higher-order Personalized PageRank with Graph Neural Networks. HPGNN introduces an efficient high-order approximation of Personalized PageRank (PPR) to capture long-range and multi-scale node interactions. This approach reduces computational complexity and mitigates noise from surrounding information. By embedding higher-order structural information into convolutional networks, HPGNN effectively models key interactions across diverse graph dimensions. Extensive experiments on benchmark datasets demonstrate HPGNN’s effectiveness. The model achieves better performance than five out of seven state-of-the-art methods on heterophilic graphs in downstream tasks while maintaining competitive performance on homophilic graphs. HPGNN’s ability to balance multi-scale information and robustness to noise makes it a versatile solution for real-world graph learning challenges. Codes are available at https://github.com/streetcorner/HPGNN.

Method: Introduces InfoPos framework that positions fault detection/identification use-cases based on available knowledge and data dimensions (from poor to rich levels), allowing designers to identify the most effective building block choices.

Result: Demonstrator results from industrial Cyber-Physical Systems fault identification show performance variations when different building blocks are used across knowledge and data positions, with ML model performance as the effectiveness indicator.

Conclusion: InfoPos framework enables systematic selection of effective building blocks for ML-assisted fault detection solutions based on available knowledge and data levels, with publicly available data processing code and datasets.

Abstract: The variety of building blocks and algorithms incorporated in data-centric and ML-assisted fault detection and identification solutions is high, contributing to two challenges: selection of the most effective set and order of building blocks, as well as achieving such a selection with minimum cost. Considering that ML-assisted solution design is influenced by the extent of available data and the extent of available knowledge of the target system, it is advantageous to be able to select effective and matching building blocks. We introduce the first iteration of our InfoPos framework, allowing the placement of fault detection/identification use-cases based on the available levels (positions), i.e., from poor to rich, of knowledge and data dimensions. With that input, designers and developers can reveal the most effective corresponding choice(s), streamlining the solution design process. The results from a demonstrator, a fault identification use-case for industrial Cyber-Physical Systems, reflects achieved effects when different building blocks are used throughout knowledge and data positions. The achieved ML model performance is considered as the indicator for a better solution. The data processing code and composed datasets are publicly available.

Method: Leverages the generative process of diffusion policies to compute uncertainty-based metrics that enable autonomous agents to decide when to request operator assistance at deployment time, without requiring human interaction during training.

Result: Experimental results from simulated and real-world environments show enhanced policy performance during deployment across various scenarios, and the method also enables efficient data collection for fine-tuning diffusion policies.

Conclusion: The proposed approach successfully reduces reliance on constant human oversight while maintaining or improving robot deployment performance through selective human assistance requests.

Abstract: Human-in-the-loop (HitL) robot deployment has gained significant attention in both academia and industry as a semi-autonomous paradigm that enables human operators to intervene and adjust robot behaviors at deployment time, improving success rates. However, continuous human monitoring and intervention can be highly labor-intensive and impractical when deploying a large number of robots. To address this limitation, we propose a method that allows diffusion policies to actively seek human assistance only when necessary, reducing reliance on constant human oversight. To achieve this, we leverage the generative process of diffusion policies to compute an uncertainty-based metric based on which the autonomous agent can decide to request operator assistance at deployment time, without requiring any operator interaction during training. Additionally, we show that the same method can be used for efficient data collection for fine-tuning diffusion policies in order to improve their autonomous performance. Experimental results from simulated and real-world environments demonstrate that our approach enhances policy performance during deployment for a variety of scenarios.

Method: Formulates strategic bidding as contextual multi-armed bandit problem and proposes online learning algorithm with provable regret minimization.

Result: Algorithm performance evaluated against benchmark strategies using numerical simulation of German day-ahead and real-time markets.

Conclusion: The proposed approach addresses computational challenges of traditional bilevel optimization methods for price-maker bidding.

Abstract: Wind power producers (WPPs) participating in short-term power markets face significant imbalance costs due to their non-dispatchable and variable production. While some WPPs have a large enough market share to influence prices with their bidding decisions, existing optimal bidding methods rarely account for this aspect. Price-maker approaches typically model bidding as a bilevel optimization problem, but these methods require complex market models, estimating other participants’ actions, and are computationally demanding. To address these challenges, we propose an online learning algorithm that leverages contextual information to optimize WPP bids in the price-maker setting. We formulate the strategic bidding problem as a contextual multi-armed bandit, ensuring provable regret minimization. The algorithm’s performance is evaluated against various benchmark strategies using a numerical simulation of the German day-ahead and real-time markets.

Method: Uses coarse-to-fine octree tokenizer to compress 3D structures into 1D token sequences, implements two-level subtree compression (8x reduction), and employs masked next-token prediction for accurate positional modeling despite compression-induced position variations.

Result: Achieves up to 256% relative improvement over previous state-of-the-art diffusion models and delivers inference speeds up to 21.8x faster across multiple 3D generation and understanding tasks including molecules, proteins, polymers, crystals, and macroscopic objects.

Conclusion: Uni-3DAR provides an effective and versatile unified framework for cross-scale 3D modeling, demonstrating superior performance and efficiency compared to specialized domain-specific approaches.

Abstract: 3D structure modeling is essential across scales, enabling applications from fluid simulation and 3D reconstruction to protein folding and molecular docking. Yet, despite shared 3D spatial patterns, current approaches remain fragmented, with models narrowly specialized for specific domains and unable to generalize across tasks or scales. We propose Uni-3DAR, a unified autoregressive framework for cross-scale 3D generation and understanding. At its core is a coarse-to-fine tokenizer based on octree data structures, which compresses diverse 3D structures into compact 1D token sequences. We further propose a two-level subtree compression strategy, which reduces the octree token sequence by up to 8x. To address the challenge of dynamically varying token positions introduced by compression, we introduce a masked next-token prediction strategy that ensures accurate positional modeling, significantly boosting model performance. Extensive experiments across multiple 3D generation and understanding tasks, including small molecules, proteins, polymers, crystals, and macroscopic 3D objects, validate its effectiveness and versatility. Notably, Uni-3DAR surpasses previous state-of-the-art diffusion models by a substantial margin, achieving up to 256% relative improvement while delivering inference speeds up to 21.8x faster.

Method: Combines tensor neural networks with Gauss-Jacobi quadrature, using a specially designed function multiplied by t^μ to discretize Caputo derivatives.

Result: Numerical examples demonstrate the efficiency and accuracy of the proposed method.

Conclusion: The tensor neural network-based machine learning method provides an effective universal numerical scheme for solving time-fractional equations.

Abstract: In this paper, we propose a novel machine learning method based on adaptive tensor neural network subspace to solve linear time-fractional diffusion-wave equations and nonlinear time-fractional partial integro-differential equations. In this framework, the tensor neural network and Gauss-Jacobi quadrature are effectively combined to construct a universal numerical scheme for the temporal Caputo derivative with orders spanning $ (0,1)$ and $(1,2)$. Specifically, in order to effectively utilize Gauss-Jacobi quadrature to discretize Caputo derivatives, we design the tensor neural network function multiplied by the function $t^{\mu}$ where the power $\mu$ is selected according to the parameters of the equations at hand. Finally, some numerical examples are provided to validate the efficiency and accuracy of the proposed tensor neural network based machine learning method.

Method: Barycentric Neural Network (BNN) with fixed base points and barycentric coordinates, combined with length-weighted persistent entropy (LWPE) loss function to optimize base points.

Result: Superior and faster approximation performance compared to standard losses (MSE, RMSE, MAE, LogCosh), offering computationally sustainable function approximation.

Conclusion: BNN provides a flexible, interpretable, and computationally efficient alternative for function approximation through exact representation of continuous piecewise linear functions.

Abstract: While artificial neural networks are known as universal approximators for continuous functions, many modern approaches rely on overparameterized architectures with high computational cost. In this work, we introduce the Barycentric Neural Network (BNN): a compact shallow architecture that encodes both structure and parameters through a fixed set of base points and their associated barycentric coordinates. We show that the BNN enables the exact representation of continuous piecewise linear functions (CPLFs), ensuring strict continuity across segments. Given that any continuous function on a compact domain can be uniformly approximated by CPLFs, the BNN emerges as a flexible and interpretable tool for function approximation. To enhance geometric fidelity in low-resource scenarios, such as those with few base points to create BNNs or limited training epochs, we propose length-weighted persistent entropy (LWPE): a stable variant of persistent entropy. Our approach integrates the BNN with a loss function based on LWPE to optimize the base points that define the BNN, rather than its internal parameters. Experimental results show that our approach achieves superior and faster approximation performance compared to standard losses (MSE, RMSE, MAE and LogCosh), offering a computationally sustainable alternative for function approximation.

Method: Proposed a taxonomy of 13 score functions to measure head inactivity, used thresholding to identify inactive heads, and validated through model ablation experiments while maintaining MMLU accuracy.

Result: Found >12% of attention heads are inactive on average; output norm-based scores identified 7% more inactive heads than attention weight-based methods; finetuning causes minimal attention behavior changes; large models show different attention patterns.

Conclusion: Measuring output norms is more effective than attention weights for identifying inactive heads, revealing significant computational redundancy in LLMs that can be exploited for optimization.

Abstract: Attention is foundational to large language models (LLMs), enabling different heads to have diverse focus on relevant input tokens. However, learned behaviors like attention sinks, where the first token receives the most attention despite limited semantic importance, suggest some heads may be inactive, and point to a significant source of computational redundancy. To analyze this phenomenon, we propose a taxonomy of 13 score functions that measure different ways a head can be inactive. Thresholding these scores allows us to analyze different sets of potentially inactive attention heads. We evaluate whether identified heads are inactive through model interventions, finding that more than 12% of attention heads are inactive on average, and can be ablated in specific contexts while maintaining MMLU accuracy to within 1% of the pretrained LLM. Across 3 model families, our score functions that measure the average norm of a head’s output consistently identify inactive heads that would not have been found by score functions that rely solely on attention weights. We establish that relying on a score function that measures a first token attention sink would underestimate the prevalence of inactive heads, failing to identify more than 7% of inactive heads on average. We also show how measuring score distributions can provide insights into attention behavior. For instance, we find evidence that finetuning causes little to no change in attention behavior, and that even within the same model family, large model scales present markedly different attention behaviors.

Method: Introduces a new quantity called single policy deviation concentrability coefficient, develops two algorithms: MAIL-BRO (using best response oracle) and MURMAIL (without oracle), with theoretical analysis of their query complexities.

Result: MAIL-BRO achieves ε-Nash equilibrium with O(ε⁻⁴) expert and oracle queries, while MURMAIL achieves the same without oracle but with worse O(ε⁻⁸) expert query complexity. Numerical experiments confirm theoretical findings.

Conclusion: The single policy deviation concentrability coefficient is fundamental for learning Nash equilibria from expert data, and the proposed algorithms provide practical solutions with different trade-offs between oracle dependency and query complexity.

Abstract: This paper provides the first expert sample complexity characterization for learning a Nash equilibrium from expert data in Markov Games. We show that a new quantity named the single policy deviation concentrability coefficient is unavoidable in the non-interactive imitation learning setting, and we provide an upper bound for behavioral cloning (BC) featuring such coefficient. BC exhibits substantial regret in games with high concentrability coefficient, leading us to utilize expert queries to develop and introduce two novel solution algorithms: MAIL-BRO and MURMAIL. The former employs a best response oracle and learns an $\varepsilon$-Nash equilibrium with $\mathcal{O}(\varepsilon^{-4})$ expert and oracle queries. The latter bypasses completely the best response oracle at the cost of a worse expert query complexity of order $\mathcal{O}(\varepsilon^{-8})$. Finally, we provide numerical evidence, confirming our theoretical findings.

Method: Used majority voting as a self-feedback mechanism in RL, conducting experiments on both synthetic and real reasoning tasks to study self-training sustainability.

Result: Initial self-training improved reasoning performance and feedback quality, but prolonged RL led to reward hacking where models maximized training rewards, causing sudden performance collapse.

Conclusion: Feedback design is the central challenge; future research should focus on mechanisms that enable prolonged self-improvement without reward hacking.

Abstract: Recent successes of reinforcement learning (RL) in training large reasoning models motivate the question of whether self-training - the process where a model learns from its own judgments - can be sustained within RL. In this work, we study this question using majority voting as a simple self-feedback mechanism. On a comprehensive set of experiments on both synthetic and real reasoning tasks, we find that this basic approach improves not only the model’s reasoning performance, but also its capability of generating better quality feedback for the next RL iteration, driving further model improvement. Yet our analysis also reveals a critical limitation of such a self-training paradigm - prolonged RL with self-reward leads to reward hacking where models learn to maximize training (pseudo-)reward, resulting in sudden and complete performance collapse. Together, these results highlight feedback design as the central challenge and call for future research on mechanisms to enable prolonged self-improvement.

Method: Adopt a predictive-first perspective using martingale posteriors, parameterize one-step-ahead posterior predictive with neural networks, and update sequentially using Kalman-filter-like recursions in fully online, replay-free settings.

Result: Achieves competitive performance in non-stationary contextual bandits and Bayesian optimization with 10-100 times faster inference than classical Thompson sampling while maintaining comparable or superior decision performance.

Conclusion: The proposed methods successfully decouple Bayesian decision-making from parameter-space inference, providing efficient uncertainty quantification for online learning scenarios.

Abstract: We introduce scalable algorithms for online learning of neural network parameters and Bayesian sequential decision making. Unlike classical Bayesian neural networks, which induce predictive uncertainty through a posterior over model parameters, our methods adopt a predictive-first perspective based on martingale posteriors. In particular, we work directly with the one-step-ahead posterior predictive, which we parameterize with a neural network and update sequentially with incoming observations. This decouples Bayesian decision-making from parameter-space inference: we sample from the posterior predictive for decision making, and update the parameters of the posterior predictive via fast, frequentist Kalman-filter-like recursions. Our algorithms operate in a fully online, replay-free setting, providing principled uncertainty quantification without costly posterior sampling. Empirically, they achieve competitive performance-speed trade-offs in non-stationary contextual bandits and Bayesian optimization, offering 10-100 times faster inference than classical Thompson sampling while maintaining comparable or superior decision performance.

Method: Decomposes each rank-r update into r rank-one components treated as independent experts, enables fine-grained rank-one expert utilization, uses self-activated relevance inference via intermediate activations, and employs rank pruning with activation budgets for sparse mixture selection per input.

Result: Validated on continual learning benchmarks using CLIP and language models, MoRA shows significant effectiveness in enhancing continual learning with pre-trained models, improving generalization while mitigating forgetting.

Conclusion: MoRA successfully addresses key challenges in continual learning by enabling fine-grained rank-level expert selection, reducing interference and redundancy while maintaining stable routing and mitigating catastrophic forgetting.

Abstract: Continual learning (CL) with large pre-trained models is challenged by catastrophic forgetting and task interference. Existing LoRA-based Mixture-of-Experts (MoE) approaches mitigate forgetting by assigning and freezing task-specific adapters, but suffer from interference, redundancy, and ambiguous routing due to coarse adapter-level selection. However, this design introduces three key challenges: 1) Interference: Activating full LoRA experts per input leads to subspace interference and prevents selective reuse of useful components across tasks. 2) Redundancy: Newly added experts often duplicate or contradict existing knowledge due to unnecessary activation of unrelated ranks and insufficient reuse of relevant ones. 3) Ambiguity: Overlapping features across tasks confuse the router, resulting in unstable expert assignments. As more experts accumulate, earlier task routing degrades, accelerating forgetting. We propose MoRA, a Mixture-of-Rank Adaptive learning approaches with self-activated and sparse rank activation for CL. Unlike mixing multiple low-rank matrices, MoRA decomposes each rank-r update into r rank-one components, each treated as an independent expert, enabling fine-grained rank-one expert utilization while mitigating interference and redundancy. To avoid ambiguous routing, we propose that each rank-one expert can infer its own relevance via intermediate activations. Coupled with our proposed rank pruning and activation budgets, MoRA adaptively selects a sparse mixture of ranks per input. We validate MoRA on continual learning benchmarks using CLIP and language models, analyzing both in-domain learning and out-of-domain forgetting/generalization during fine-tuning. MoRA shows significant effectiveness in enhancing CL with PTMs, and improving generalization while mitigating forgetting.

Method: Spiffy uses auto-speculative decoding with the dLLM’s own distribution to propose draft states, eliminating the need for a separate draft model. It employs a novel directed draft graph that leverages dLLM’s bidirectional, block-wise generation and can be verified in parallel. An offline calibration algorithm optimizes draft graph configurations.

Result: Spiffy achieves 2.8-3.1× acceleration while provably preserving the model’s output distribution. When combined with KV-caching and multi-token unmasking, it reaches up to 7.9× total speedup.

Conclusion: Spiffy effectively addresses the unique challenges of applying speculative decoding to dLLMs, providing significant speed improvements while maintaining output quality, and is complementary to other dLLM optimization techniques.

Abstract: Diffusion LLMs (dLLMs) have recently emerged as a powerful alternative to autoregressive LLMs (AR-LLMs) with the potential to operate at significantly higher token generation rates. However, currently available open-source dLLMs often generate at much lower rates, typically decoding only a single token at every denoising timestep in order to maximize output quality. We present Spiffy, a speculative decoding algorithm that accelerates dLLM inference by $\mathbf{2.8{-}3.1\times}$ while provably preserving the model’s output distribution. This work addresses the unique challenges involved in applying ideas from speculative decoding of AR-LLMs to the dLLM setting. Spiffy proposes draft states by leveraging the dLLM’s distribution itself in an auto-speculative manner. This approach is efficient and effective, and eliminates the overheads of training and running an independent draft model. To structure the candidate draft states, we propose a novel directed draft graph which is uniquely designed to take advantage of the bidirectional, block-wise nature of dLLM generation and can be verified in parallel by the dLLM. To further optimize the structure of these draft graphs, we introduce an efficient, offline calibration algorithm that procedurally determines high-quality graph configurations. These optimized draft graphs, enabling increased acceptance rates, lead to a significant boost in the overall speedup achieved by the system. Crucially, Spiffy is also complementary to other recent innovations in improving dLLM generation speeds such as KV-caching and multi-token unmasking. We demonstrate that when combined with such parallel decoding algorithms, Spiffy is able to effectively multiply the benefits of these methods leading to total speedups of up to $\mathbf{7.9\times}$.

Method: Combines graph-based enumeration schemes with machine learning filtering, using neural network potential (AIMNet2-rxn) for computational evaluation of reaction pathways.

Result: The NNP successfully estimates activation energies, correctly anticipates stereoselectivity, and recapitulates complex enabling steps in natural product synthesis.

Conclusion: The presented strategy provides a cost-effective approach for exploring complex cyclization reactions, overcoming limitations of traditional mechanism search tools.

Abstract: Reaction mechanism search tools have demonstrated the ability to provide insights into likely products and rate-limiting steps of reacting systems. However, reactions involving several concerted bond changes - as can be found in many key steps of natural product synthesis - can complicate the search process. To mitigate these complications, we present a mechanism search strategy particularly suited to help expedite exploration of an exemplary family of such complex reactions, cyclizations. We provide a cost-effective strategy for identifying relevant elementary reaction steps by combining graph-based enumeration schemes and machine learning techniques for intermediate filtering. Key to this approach is our use of a neural network potential (NNP), AIMNet2-rxn, for computational evaluation of each candidate reaction pathway. In this article, we evaluate the NNP’s ability to estimate activation energies, demonstrate the correct anticipation of stereoselectivity, and recapitulate complex enabling steps in natural product synthesis.

Method: Discards time-conditional dynamics and learns the equilibrium gradient of an implicit energy landscape, using optimization-based sampling with gradient descent, adjustable step sizes, and adaptive optimizers.

Result: Achieves FID of 1.90 on ImageNet 256×256, surpassing diffusion/flow models, and handles tasks like image denoising, OOD detection, and image composition.

Conclusion: EqM provides a unified framework that bridges flow and energy-based models, offering optimization-driven inference and theoretical justification for learning from the data manifold.

Abstract: We introduce Equilibrium Matching (EqM), a generative modeling framework built from an equilibrium dynamics perspective. EqM discards the non-equilibrium, time-conditional dynamics in traditional diffusion and flow-based generative models and instead learns the equilibrium gradient of an implicit energy landscape. Through this approach, we can adopt an optimization-based sampling process at inference time, where samples are obtained by gradient descent on the learned landscape with adjustable step sizes, adaptive optimizers, and adaptive compute. EqM surpasses the generation performance of diffusion/flow models empirically, achieving an FID of 1.90 on ImageNet 256$\times$256. EqM is also theoretically justified to learn and sample from the data manifold. Beyond generation, EqM is a flexible framework that naturally handles tasks including partially noised image denoising, OOD detection, and image composition. By replacing time-conditional velocities with a unified equilibrium landscape, EqM offers a tighter bridge between flow and energy-based models and a simple route to optimization-driven inference.

Method: Proposed a cost-aware stopping rule grounded in theoretical connection to state-of-the-art cost-aware acquisition functions (Pandora’s Box Gittins Index and log expected improvement per cost).

Result: Theoretical guarantee bounding expected cumulative evaluation cost when paired with PBGI and log EI per cost. Experiments on synthetic and empirical tasks show the stopping rule with PBGI usually matches or outperforms other combinations in cost-adjusted simple regret.

Conclusion: The proposed cost-aware stopping rule provides theoretical guarantees and practical performance improvements for Bayesian optimization in cost-sensitive applications.

Abstract: In automated machine learning, scientific discovery, and other applications of Bayesian optimization, deciding when to stop evaluating expensive black-box functions is an important practical consideration. While several adaptive stopping rules have been proposed, in the cost-aware setting they lack guarantees ensuring they stop before incurring excessive function evaluation costs. We propose a cost-aware stopping rule for Bayesian optimization that adapts to varying evaluation costs and is free of heuristic tuning. Our rule is grounded in a theoretical connection to state-of-the-art cost-aware acquisition functions, namely the Pandora’s Box Gittins Index (PBGI) and log expected improvement per cost. We prove a theoretical guarantee bounding the expected cumulative evaluation cost incurred by our stopping rule when paired with these two acquisition functions. In experiments on synthetic and empirical tasks, including hyperparameter optimization and neural architecture size search, we show that combining our stopping rule with the PBGI acquisition function usually matches or outperforms other acquisition-function–stopping-rule pairs in terms of cost-adjusted simple regret, a metric capturing trade-offs between solution quality and cumulative evaluation cost.

Method: Proposes NAMMD which scales MMD’s value using the RKHS norms of distributions, and develops NAMMD-based distribution closeness testing based on the asymptotic distribution of NAMMD.

Result: Theoretical analysis proves NAMMD-based DCT has higher test power than MMD-based DCT with bounded type-I error. Extensive experiments on synthetic noise and real images validate these findings. NAMMD also shows higher test power for two-sample testing.

Conclusion: NAMMD provides a more informative measurement of distributional discrepancy than MMD, enabling more effective distribution closeness testing and two-sample testing for complex data types.

Abstract: The distribution closeness testing (DCT) assesses whether the distance between a distribution pair is at least $\epsilon$-far. Existing DCT methods mainly measure discrepancies between a distribution pair defined on discrete one-dimensional spaces (e.g., using total variation), which limits their applications to complex data (e.g., images). To extend DCT to more types of data, a natural idea is to introduce maximum mean discrepancy (MMD), a powerful measurement of the distributional discrepancy between two complex distributions, into DCT scenarios. However, we find that MMD’s value can be the same for many pairs of distributions that have different norms in the same reproducing kernel Hilbert space (RKHS), making MMD less informative when assessing the closeness levels for multiple distribution pairs. To mitigate the issue, we design a new measurement of distributional discrepancy, norm-adaptive MMD (NAMMD), which scales MMD’s value using the RKHS norms of distributions. Based on the asymptotic distribution of NAMMD, we finally propose the NAMMD-based DCT to assess the closeness levels of a distribution pair. Theoretically, we prove that NAMMD-based DCT has higher test power compared to MMD-based DCT, with bounded type-I error, which is also validated by extensive experiments on many types of data (e.g., synthetic noise, real images). Furthermore, we also apply the proposed NAMMD for addressing the two-sample testing problem and find NAMMD-based two-sample test has higher test power than the MMD-based two-sample test in both theory and experiments.

Method: Four-component framework: QI-Safe-10K dataset, safety-aware rollout with reflection/correction, structured reward modeling with penalties, and GRPO optimization to reinforce safe trajectories.

Result: SaFeR-VLM-3B achieves 70.13 safety and 78.97 helpfulness scores, surpassing larger models. SaFeR-VLM-7B outperforms GPT-5-mini and Gemini-2.5-Flash by 6.47 and 16.76 points on safety without helpfulness degradation.

Conclusion: The framework shifts safety from passive safeguard to active driver of reasoning, enabling scalable and generalizable safety-aware reasoning with robustness against explicit and implicit risks.

Abstract: Multimodal Large Reasoning Models (MLRMs) demonstrate impressive cross-modal reasoning but often amplify safety risks under adversarial or unsafe prompts, a phenomenon we call the \textit{Reasoning Tax}. Existing defenses mainly act at the output level and do not constrain the reasoning process, leaving models exposed to implicit risks. In this paper, we propose SaFeR-VLM, a safety-aligned reinforcement learning framework that embeds safety directly into multimodal reasoning. The framework integrates four components: (I) QI-Safe-10K, a curated dataset emphasizing safety-critical and reasoning-sensitive cases; (II) safety-aware rollout, where unsafe generations undergo reflection and correction instead of being discarded; (III) structured reward modeling with multi-dimensional weighted criteria and explicit penalties for hallucinations and contradictions; and (IV) GRPO optimization, which reinforces both safe and corrected trajectories. This unified design shifts safety from a passive safeguard to an active driver of reasoning, enabling scalable and generalizable safety-aware reasoning. SaFeR-VLM further demonstrates robustness against both explicit and implicit risks, supporting dynamic and interpretable safety decisions beyond surface-level filtering. SaFeR-VLM-3B achieves average performance $70.13$ and $78.97$ on safety and helpfulness across six benchmarks, surpassing both same-scale and $>10\times$ larger models such as Skywork-R1V3-38B, Qwen2.5VL-72B, and GLM4.5V-106B. Remarkably, SaFeR-VLM-7B benefits from its increased scale to surpass GPT-5-mini and Gemini-2.5-Flash by \num{6.47} and \num{16.76} points respectively on safety metrics, achieving this improvement without any degradation in helpfulness performance. Our codes are available at https://github.com/HarveyYi/SaFeR-VLM.